-
pjusticeThis is making me crazy and I can't seem to find any chinks in its armor, so I'ma ask here. GZ running platform 20230504T000449Z. Native guest based on base-64-trunk 20220708. Unbound 1.18. Unbound is encountering a lot of EINVAL trying to sendmsg(). This started after a machine reboot yesterday. A second more or less identical guest on another machine, same versions of the above, is working
-
pjusticefine. I've ...
-
pjustice... rebooted the troubled guest, rebooted the whole machine, destroyed and re-created the guest, no change in behavior. ipfstat says no packets blocked, there are no ipf rules for outbounds on guest or gz. These are resolvers used only by our own servers. A truss session indicates that the attempted sends aren't stoopid long or anything. In 30ish seconds, there were 11 attempted sendmsg() and
-
pjusticesendto() calls; ...
-
pjustice... the sendto() calls worked, the sendmsg() all failed. I do occasionally get answers back; it's not every query that fails.
-
jperkinpjustice: so if you have a read of the sendmsg manual page you'll see the failure modes that cause EINVAL - as there are a few, you could dtrace it to see exactly which condition is causing them, and then go from there
-
jperkinif you aren't familiar with that, it'll be along the lines of adding a probe on pid$target::sendmsg:entry which does print(*args[1]) and specify the unbound pid to dtrace with -p, various brendan/rm blogs are helpful for explaining the pid provider and userland ctf for more info
-
jperkinneed to put kids to bed shortly but happy to help knock something up if you get stuck
-
jperkinas everything in pkgsrc that can be is built with ctf then you should be able to directly reference struct members etc to drill down further
-
pjusticeThat's a good place to start. Thanks!
-
pjusticejperkin: the pid provider with the sendmsg routine name never seems to actually see an event.
-
pjusticeusing send instead of sendmsg sees occasional calls.
-
pjusticethe syscall provider sees sendmsg
-
pjusticeThe address of the msg structure passed to sendmsg is very high, e.g. 0xfffffbffeb7ff840
-
pjusticeFor sendto and send, it is more like 0x58c2010 or 0x9451010