richlowe: I'm seeing stacks similar to illumos#14722 on a platform which should include that fix, do you want me to comment on the original bug or open a new one?

jperkin: I'd probably suggest opening a new bug.

ok

Mostly because if there is a problem, we'll want a new bug to describe it anyways.

rmustacc: 16 512-bit ZMM registers

heh, actually, 32

I am a bit worried there are still some bugs in the kernel FPU path, though. The parity calculations in ZFS have been utilizing this for a while, haven't they?

KungFuJesus: are you referring to my comment on 15254?

And when you say the kernel fpu path, what do you actually mean?

yes, and I mean the kernel corrupting userspace's register state for ymm*

and zmm* I guess

KungFuJesus: Because the kernel is using the FPU or something else?

Do you have anything more for me to go on?

The signal handling issue is a serious one that I'm trying to get turned around and solved.

jperkin: do you have the objects etc?

jperkin: bugger-ld or whatever would help

jperkin: you could also check `strings libld.so.4 | grep 'Link Editor'` to be sure you got both bits at the right version.

jperkin: Can you also try running elftest and tell me if the tests I added fail?

ah yeh, lemme get a bugger-ld run

rmustacc: sorry, went and grabbed lunch. I'm only surmising what that bug report already claims

jperkin: super interested in the test run too

your analysis does indicate that it's an issue relegated to signal handling, so perhaps that's not the biggest bug surface

the bits from that bug I have here work.

KungFuJesus: you're making a limited amount of sense, I think you missed what is affected and how.

the problem is that when a user process takes a signal, its avx state isn't saved so if the signal handler uses avx that state is corrupt when we return from the signal.

nothing in the kernel, except that that's where signals are implemented.

Perhaps? My understanding is that the kernel is corrupting the state of the FPU when the kernel context switches out of a signal interrupt

meaning, the FPU state is not properly restored after the kernel clobbers those vector registers

Now from what I understand, this only happens in the scenario that the kernel makes use of those registers, which is limited to the very limited use case where the kernel does hardware floating point math (almost never), or needs to use the vector registers (e.g. SIMD vectorized parity calculations, AES-NI for crypto, etc)

no, the kernel isn't clobbering them because it's using them

it's clobbering them because the _user signal handler_ is using them.

they're not being actively corrupted, they're being passively not saved when they should be

ooof, I see

this is similar to say, addressing an xmm register is a ymm register, where the upper half is going to be undefined

I've been bit by that one, before

as a*

Yes, none of this is due to kernel use. It requires a rather exacting combination of things.

well that's at least a good limited surface area for the bug, anyway

After a fashion, I guess.

It's true it can always be worse, but I don't feel good about what appears to a user program as arbitrary data corruption and that's why I'm trying to write up how it gets fixed, the nuances, and the challenges.

but yeah, so the hazard is something like user program context switches, kernel runs proper xsave to dump register state to stack, user program sends signal, signal handling logic calls older fxsave or equivalent behavior, leaving half of the vectore register state on the stack as undefined

No.

It is not that.

richlowe: bugger-ld tarball posted to the issue, stupid elftests question, can I just run them in an illumos-gate checkout on a non-build machine?

you don't need a build machine, you don't need an illumos checkout, you need the package and the test runner. It defaults to testing /bin/ld

oh, and a compiler/assembler.

FWIW, all passing for me right now.

me too

where do I find them then?

In smartos land they're in the proto area, but not in the ramdisk.

I don't actually know smartos-ishly, danmcd do you make this convenient?

jperkin: /opt/elf-tests and /opt/test-runner, conventionally

I already have an illumos-gate checkout on the host, just wondering if I can run them from there

and in the proto

jperkin: Maybe if you lofs mount it into /opt

I have been known to link the proto area copy into /opt

I don't know if it'll run outside of there, because of test-runner stuff

I'll pull your tarball down when this build/test cycle for another thing is finished

I'm sorry I seem to have screwed it up somehow.

don't take this as any indication that things were fine before and now they're breaking, it's just that I only now actually have time to start filing bugs for stuff that previously I'd have just rm /cores/* ;)

no, but it means the fix was incomplete or bad.

KungFuJesus: The kernel is only saving an fxsave state and retreiving it across the signal handler. It uses that to update the xsave state. So it the upper register state is well defined to be whatever was in the FPU. So if the signal handler doesn't use it, it'll not be touched.

jperkin: this might be a different bug that crashes in a similar place

and/or a bug I introduced fixing the other

since this is rust, has anyone else seen this?

oxide people?

jperkin: so I have suspicions, but I don't know why an entire company isn't shouting at me about this if I'm right :\

jclulow: Do you know how to influence the flags rustc passes to ld?

so looking at the other cores the others are from rust programs too (devel/git-cliff, www/zola), still looking to see if there are others..

jperkin: right, I don't see why this isn't making oxide _real_ angry

which is interesting in its own right

jperkin: I want you to try something for me, but I don't know what that something is

but there are also hundreds of other rust programs that build fine..

jperkin: see dm

richlowe: A lot of it is pretty hard-coded I believe in the target type

jclulow: well I tihnk I know what's happening here, but I don't know why it's not happening to you

jclulow: can I ask you questions about how y'all build stuff, or are you busy?

I'm always pretty busy haha but fire away

(richlowe: SOrry I've been away... if you need something built on SmartOS I'm here-ish now, modulo me upgrading Kebecloud to the Triton update we cut last night.)

I believe, rich, you need to be in a zone that can build smartos-live to do what you wish.

(/bin/ld is there, e.g.)

it feels like the only real path to a fix in finite time is a bit shit

at least I have a fix for pkgsrc, with the added benefit of ensuring rust stuff has CTF now

What do you mean there?

Are you generating a subste of rust types as CTF or something where stuff with things partially built from rust has problems.

removing -Wl,--strip-all avoids the ld bug and ensures I can ctfconvert stuff before pkgsrc does its own strip handling

Ah, gotcha.

jperkin: I can give you a fix to try in a little bit too

sure, though will definitely be tomorrow now, nearly zzz

same, I have to stop typing soon

_a_ fix is easy, convincing someone to approve it might not be

rmustacc: if you're playing along at home this is #15260 (fenix?)

BUG 15260: ld segfault similar to 14722 (New)

↳ illumos.org/issues/15260

Oh. That's going to be a thing.

I will try to carve out some thoughts after I get through this fpu write up.

luckily, it appears to only happen if you explicit configure a rust profile with strip=true, or otherwise use ld -s

tsoome: replied again about cw, you might want to try to attract some attention that isn't me, too :)

I'm much more open to "in a followup" than other people might be

richlowe, tsoome: Is there something you'd like me to look at?

rmustacc absolutely. code.illumos.org/c/illumos-gate/+/2541

→ CODE REVIEW 2541: 15230 cw: add support for clang (NEW) | illumos.org/issues/15230

OK. Can you add me to it? It might take me a bit to get there as this fpu issues is... gnarly.

I'll add some background to issue too -- this feature would just be there to make it possible for people to start poking clang -- and I still need to push some tools patches too to make it even remotely possible:D

richlowe that followup sounds really reasonable, however.

be it real followup, or just update to this patch:)

jperkin: sent you the branch and binaries to try out

if anyone doesn't like the awful conservative approach, I look forward to them trying to untangle what is less conservative but also provably safe

because I sure can't.

commenting on mastodon (subtooting?) is an acceptable way to let you know your proc(1) man page is missing some proc tools, right? fosstodon.org/@alanc/109520277885645860

[illumos-gate] 15207 libfmd_agent: memory leak in fmd_agent_nvl_ioctl -- Yuri Pankov <ypankov⊙tc>