argh, more internal server error failures from code.illumos.org ..

it's another round of AI hammering I think

automated irritation.

i guess i can also put up what I have for the pci segment bits... but I do worry there's maybe something I'm missing...

specifically surrounding pci_boot.c

I think a handful the oxide people know at least a decent amount about that, and that v. the hotplug code that does the same thing.

I know a small amount about porting it, but not enough to rely on

and full disclosure, I tried for an extended period of time to replace it instead, and failed

hrm yeah.. i suspect we can't talk to devices in the second segment... even if the device tree knows about them...

wiki.osdev.org/PCI_Express shows tthe MCFG table, and we appear to only look at the first structure in it

(and store the values in 'ecfg' in the root nexus)

The bigger problem right now is the pci_regspec_t.

since it doesn't contain the segment# ?

How we handle the mcfg table is definitely also a problem, but that's going to end up being thornier.

Yes, and because the iommu isn't usefully in the tree in a way that you can usually id things beneath it without it.

But I've been punting on it for a while since our AMD platforms aren't multi-segment.

apparently these new(er) HPE servers are

(I have access through Friday, so if there is anything useful to gather, I can btw)

How are you intended to discover the segment? Is it an ACPI method on the root port?

Or something else.

all I have right now (which I strongly suspected was incomplete at best) was evaluating the ACPI _SEG method on root complexes (in acpidev_pci_update_status()) and setting a property with the value if it returned one and then using that in immu_devi_set() to match a device to the immu_t for it (basically go up the tree for the dev's dip to find the segment on the root complex, or 0 if none)

since I don't have physical access to these systems, I'm also not sure what exactly all is in them...

it does appear though that the 2nd CPU socket goes with (for lack of a better term) the 2nd PCI riser and I suspect is where the 2nd PCI segment is

OK. We may be able to do something like that and inherit it.

But I think that'll make a mess of a bunch of other stuff so we'll need to think through that. I think trying to avoid that and having overlapping b/d/f values will lead to more chaos.

at least the DRAT data suggests that the bus #s in each segment are duplicated...

I would expect them to be.

Otherwise there's not much point in it!

Segments are a fiction that don't really go on the wire, so it's all internal baseball.

yeah.. from what little i've gathered so far suggests having multiple configuration spaces is maybe the thornier issue, since it looks like we tend to assume there's only one in a lot of places

but I was just trying to do as much as I could while I had access to the machine

(I might be able to get access to it again in the future, not sure)

I don't think that bit is too bad, we just have to figure out how we extend things into the segment and how that works for regmap stuff.

It just can't really be in the pci_regspec_t which means probaly figuring out a new property and propagating it.

One you have that, the actual config space mapping is easy.

if you want to take a look at what I have, I can throw it up in a branch -- I didn't want to put it out for review yet because I figured there were probably more things that need to happen

basically it was enough to make the iommu stuff at least happy enough to boot...

Specifically just handling the ecfg stuff?

i didn't mess with that.. (since I just found some info online -- don't have a copy of the actual specs)

and from that started looking at how we use MCFG and found that...

it's been a journey :)

I'm not really in a place where I think I can provide useful review or next steps, since I'm a bit swamped, but let me know what's helpful.

since i've never really waded into this part of the code base.. so it was 'system hands when it boots' (turns out interrupts aren't getting delivered) 'oh, it's using apic ids > 255, need iommu', turn iommu on, then all sorts of hijinks ensue (multiple iommus tripping bugs, multiple pci segments)