hrm...

jv

grr.

jbk see impl_setup_ddi() in i86pc/os/ddi_impl.c

for props..

heh grr is my reaction after looknig at pci_boot.c and trying to add in segment support :)

i'm probably going to reboot a VM a few times w/ kmdb just to figure out what's actually happening when in relation to other stuff (we're still early enough no other cpus or threads are going yet)

because i'm getting a headache trying to reason about all of that (and it wouldn't surprise me if a lot of it is to deal with circa mid-1990s systems)

but i'm hoping (praying) that any root complexes on a > 0 segment should by enumerated by ACPI (so we'll have the segment #) and should happen before any children on those complexes are enumerated (so we'll have the dip of the >0 segment root complex already available)

because if not, i'm not sure how it can work without fairly major surgery in there that'd i'd prefer to avoid if at all possible

(I also have no idea how we should name these things. right now we do (roughly) 'pci@<order found in enumeration>,0' which is roughly the bus# (I'm still teasing apart the code, but I think pci-pci bridges might throw off the numbering)

jbk: I'd think something like "don't change names of stuff on segment #0; prefix stuff on segments > 0 with some identifier of the segment" ?

that's probably what I'm leaning towards...

heh.. this is getting tricky because of all the legacy bits..

hrm... is there a way for memory allocated with BOP_ALLOC() to persist after release_bootstrap() (it looks like all that memory gets released if not there, by the point that's run)

at least from waht I can tell..

(ultimately the problem is we're doing things early in boot... likely for legacy reasons where it's unclear if it's still necessary or if changing things would break things... that really is less than ideal

err doing things in an order less than ideal

bop alloc takes from MISC_VA_* and lives long enoug for you

there's also boot_alloc which fills a gap between bop and full kmem

I don't think you should be that early in boot for anything pci though

I wish that was the case

the problem is since the # of segments can be arbitrary, unless we want to parse the MCFG table every time we do a cfgspace access, we need to stash the base address and min, max bus for each segment, which means either declaring a > 512KB array (UINT16_MAX * (sizeof (uint64_t) + 2 * sizeof (uint8_t)) or allocating some memory

though it might be better to just parse it each time (it's not too bad) since it's not clear if segment values are guaranteed to be consecutive

I was going for the mcfg_base_addr[segment] where it's allocated based on the max segmentr #, but that might end up not being great either..

it seems like it'd be cleaner (and certainly would be easier for this) if the pci/pcie drivers actually drove (ahem) drove the enumeration of their children instead of a separate module, but there's probably a reason for it (unfortunately I've not found anything so far to explain what it'd be)

and would be a pretty big change

I told you I'd forget to do that

(thanks hadfl)

jbk: I reorganized aarch64 so the PCI drivers drove their own enumeration. It doesn't work like that on x86 because you don't have a device node to kick it off

you could discover only the roots, and have them enumerate themselves, but it turns out you'll use the same code you're using anyway andi t's not worth it

(on x86)

the magical reason things happen that way is there was no other way to discover PCI in the x86 world

being told about it in detail by firmware is novel up until acpi and really pci-e

jbk: I guess relevant to that also is that we drive config space access through the root dip

so the drivers know how to access their own config space and nobody else does

(because some PCIe on ARM boards isn't strictly PCIe per the spec)

richlowe: yeah, the problem is on x86, we don't use the root dip while enumerating or re-enumerating (renumerating? :P)

or adding/removing workaround while doing all of that

I know, that's why I had to change all on the arm side

am I blind, or do we not have a easy CLI way to see if LSO is enabled on an interface.

I'm not sure of the specifics, but I'm pretty sure michael has run arm64-gate+acpi on an unfortunately segmented ampere system

so I'm maybe 90% sure what I did is enough for the "has segments for more bdf space, but doesn't really care after that" class of system.

sommerfeld: I feel like you might be right, and that it should really appear in "ipadm show-ifprop" presumably

yeah, I looked in both ipadm and dladm and didn't see it in either place.

found it via mdb -k (and discovered that our e1000g driver doesn't enable LSO on the 82574 but does enable it on the 82753)

i suspect it might (at best) be a private property of the NIC

yeah, it really belongs in ipadm (showing what was actually negotiated).