Hey all, so I'm having an issue with the zig test suite. They have this thing where they compile various code for different targets and try to run it. If the executable can't exec the test runner will handle that fine. But in this case it's building a static executable with no interpreter specified and that seems to cause us to immediately SIGKILL the process and I think perhaps we are the first platform that does this. So the

runner gets confused.

I could go code spelunking but if anyone could shortcut my knowledge here that would help. As in, the SIGKILL is expected?

Can you share the elf header for reference?

That is, the runner can handle exec(2) returning ENOEXEC just fine, but having a SIGKILL delivered is not something it expects

sure

And is it really static?

if it is really static, no interpreter is fine

are you sure it doesn't get killed for executing a random system call?

Aside from the fact that it probably won't work on us.

rmustacc: see the comment on that giest

And it doesn't have a dynamic section I take it?

nbjoerg: I should rephrase, the test runner is compiling this executable for a linux target and macos target, it then tries to run them, if it gets ENOEXEC then it just ignores it, but we are also delivering a SIGKILL so it reports an error.

src.illumos.org/source/xref/illumos…mmon/exec/elf/elf.c?r=6233c8a8#1046 maybe?

rmustacc: What would I search for?

I see no Interpreter section

It would be elfdump -d, but if it's for another platform that's probably neither here nor there.

(and lines 726-741 in the same file)

yea eldfump -d shows nothing

We shouldn't be in that because there's no PT_INTERP..

The 726-741 bit.

rzezeski: Can you put an fbt on exec_args:return for this?

The SIGKILL is probably because we got far enough that the child is in an undefined state so it had to kill it and couldn't just drop an ENOEXEC at that point.

For example, if you were part way through replacing the process image or stack or whatever.

rzezeski: Actually, also add mapelfexec to that.

rmustacc: 0 for both

looks like I'll be writing another blog post here soon, haha

OK, I have other questions likely, but they'd require the elf object.

rzezeski: If I was to guess it was that we have a pgrogram header but no PT_INTERP.

rmustacc: okay, yea I DM'd you a path to the file

I figure I might have to do some spelunking on this one

Yeah, it was from looking at that I'm making that guess.

k

thank you for looking

But I don't know for certain.

Specifically src.illumos.org/source/xref/illumos…ommon/exec/elf/elf.c?r=6233c8a8#634.

I was already wrong with other gueses. But most of the rest of that is on having an interpeter.

what is the uphdr?

oh

is that the program header?

Yes, from mapelfexec.

cool, well this at least gives me something to look into, so that's great

Unfortunately my stomach is yelling at me so I need to make dinner now. Thanks for all the help everyone. I'll let you know what I find later.

So anyways, given how much is going on, I expect that the child process getting killed is unfortunately somewhat reasonable.

Because it's far enough along and we can't get you back to an ENOEXEC cleanly.

Doesn't mean we can't maybe figure out sooner or do something different. Just all I know so far.

We appear to be the first platform acting this way, according to the creator of Zig. FWIW.

But doesn't mean we are wrong.

So yea, first step is I need to make sure I understand why it's happening.

rzezeski: Are they expecting to get an ENOEXEC and for it not to be run or?

rmustacc: correct

Based on what property of the elf object?

I...dunno

I did confirm you're in this case with DTrace.

I don't know enough about the expected semantics. If I were to guess the phdr is supposed to have semantics for an interpreter.

andyf: I didn't know about the strings command, thanks!

I've noticed I have build errors because I'm running out of space in /tmp

How big should I make /tmp to be safe for building?

(And out of curiosity what's the specs of the machines you guys are using to build illumos?)

It really is only a superficial check for something like this because it just looks for sequences of printable characters over a certain length. However it clearly shows the original problem and a quick before and after proves things look better now.

A more robust approach would be to look at the string table, but I don't think I'd bother unless an advocate comes back and asks for it.

How could I look at the string table if it was needed? Inspecting the binary?

I usually use a VM with at least 16 GiB of RAM, and I limit the number of build threads.. although the illumos.sh is supposed to do that for you based on the amount of installed memory.

I'm using an OI VM with 8 vcpus and 8GiB RAM running on a bhyve zone in an omnios system which also hosts in which im also hosting a windows vm for other people. The host has 32GiB ram but the windows machine is using 16 and I'm running into hangs if i allocate more memory for my vm (i.e 12GB). I want to investigate that once im more comfortable with the os

pilonsi - something like this (this is omnios but OI should also have GNU readelf somewhere) /usr/gnu/bin/readelf -p .rodata.str1.8 /kernel/misc/amd64/kbtrans

(and someone may be a long later to tell me how to do it with native illumos tools :) )

*along

You can see the embedded tabs in the existing message:

[ a0] kbtrans_queueevent: Can't allocate ^I^I^I^I^Iblock for event.

Thanks! I'll look into that as well later as an exercise

Do you know why nvme disk device names are in the form of c3t00E04C1C0C2F6F08d0 instead of the simpler names sata disks have?

if an nvme device supports a unique id, that is used for the target number

similar concept as a SAS WWN, really

WWN's are great

and i suspect the reason SATA drives don't is just because we never really exposed it for SATA drives

at least until relatively recently

can we get wwn's for sata?

sure, but can we fix zfs rpool import code to handle device path changes in all possible cases? :)

sjorge: yes (if it has one)..

previously we weren't translating page 83

so most sata ssd would have one then

yeah, probably

even a lot of HDDs will as well (I'm guessing most likely really old SATA HDDs would be the ones that don't)

Woodstock: main advantage is that the WWN-based names stay the same if you accidentally swap cables around.

jbk: all the SATA spinning rust I've plugged into SAS controllers would appear to have WWNs

yeah, i'm thinking like _really_ old drives

hm. pata -> sata bridges perhaps? :)

[illumos-gate] 15910 Favor judgment over judgement -- Dan Cross <cross⊙oc>

[illumos-gate] 11616 Support getaddrinfo() with socktype 0/AI_NUMERICSERV -- Andy Fiddaman <illumos⊙fn>

sommerfeld: so I'm in the libc locale code, for reasons, and I remember you had done stuff there. Did that ever land?

sommerfeld: and if not, do you remember thinking "Huh, this seems buggy" about anything around the xpg7 bits, especially?

Is there an mdb command that can give me the value at some offset of a pointer? I know you can do the <addr>,40::dump to repeat and dump the data. But I am looking at a problem where the offset looks differnt in my corefile than what CTF seems to think the real offset is

you can just do math on <addr>

`<addr>+5/J`

or whatever

okay, I was just curious if there was anything built in that would do the math

ahh that makes sense

thanks!

remember the default radix is hex

0t<blah> for decimal

yup, thanks

richlowe: no, I need to get back to that.

I think what I am seeing is a different versions of libssl being pulled into my binary (version 1.x and v 3.x). So mdb shows me valid data if I do <add>::print SSL session

but the ::dis shows an offset different from ::offsetof SSL session

multi-library messes. ugh.

pldd should show if that's happening

Well, multiple version sis going to be very sad.

if you're unsure about that bit

jbk: pldd shows me:

/lib/amd64/libssl.so.3

/opt/local/lib/libssl.so.1.1

which is what tipped me off

richlowe: my messing with the locale stuff was excruciatingly localized to optimizing how you find the current locale

so the pkg mediator version vs the pkgin version

do you know what version it's supposed to use?

papertigers: what is the executable showing this? you probably need to link it differently or otherwise be more selective about what libraries get pulled in.

sommerfeld: this was a result of me running `cargo install cargo-outdated`

aka a rust binary

and yeah, I just built the binary on a different illumos box that doesn't have pkgin setup and everything works as expected

I believe the rust crate uses pkgconfig

which probably means it finds the pkgsrc libssl, where something else finds the system one.

ldd -v can help you there

richlowe: yup that makes it pretty clear :) -- the rust binary cargo-outdated requires version OPENSSL_3.0.0 while libcurl that's getting pulled in is depending on OPENSSL_1_1_1

hmm maybe I will turn this into a fun short blog post

yeah, build a newer libcurl against openssl 3

the real solution is probably to clean up my dev zone so that all of these deps come from pkg and not a mix of pkg/pkgin

or link against a pkgsrc libcurl