another data point on the whole _maybe_unused thing: I can no longer build the gate on omnios r42

I'll go file a couple tickets for omnios to get it backported to r42 (nominally supported through may) and r38 (LTS)

I've been bitten by this before, with some other token, and was advised at the time not to use such things in the native tools/ code until there has been ample time for distributions to pick it up. I'm trying to find that thread.

OmniOS can backport it, but I would bet the fallout is wider.

Without releases in gate though, I don't know how we set a policy around this.

as I wrote in mail, only option is to create build farm.

IMO

this is exacly why they have the same solution in OpenZFS - noone can track all the zoo

We could just avoid using such things in the native tools until they have been in gate for X time.

well, if im not mistaken, this particular one is on gate since april. how many years I need to wait till I can push in next change?

Yes, that's the tension here, and I don't have the solution.

with build farm we can detect such issues early and get a chance to address it before making it huge issue.

I can't find the thread from last time this happened

you can observe the example of automated checks there: openzfs/zfs #14520

Also, the list of "registered" OS instances in automated test facility would give us documented baseline of systems we need to consider to be supported.

is it me, or is src.illumos.org/source/xref/illumos…rs.c?r=3ee59242&mo=13124&fi=536#536 buggy as hell?

hm, seems like me...

anyhow, found core in my test system, from nwamd. it was crashed with 'free(8133008): double-free or invalid buffer'. and free was called from libsocket.so.1'getallifaddrs+0x7b6

andyf: i think and better solution - update build host before next build with publishers

tsoome: Keep in mind you asked at 0615 US/Eastern, 0315 US/Pacific. I sure wasn't up then, and unless the west-coast folks are super-late night owls neither were they.

I would definitely classify this code as "confusing and hard to read". Not 100% sure about buggy, but it could be.

If your nwam is reproducible, I wonder if you can/should run the service with `UMEM_DEBUG=default` ? I see that nwamd is already linking to libumem, so it's not like you'd need to LD_PRELOAD it.

I just did confuse myself about caddr_t (which is char *, not char)

but I do not know about repeatability, will try to see. what was actually confusing is that I was having some hard time matching disaasm with C code -- the number of calls to free( ) was there, but follow up calls to close() did not match:D

anyhow, yes I also was thinking to add umem_debug there.

I remember people here saying in the past that Intel ethernet NICs were always solid (which ignored the multiple 82574L fiascos). It seems that is no longer the case with their newer ethernet NICs: tomshardware.com/news/intel-patches…s-but-its-just-a-workaround-for-now

The I225 has always been a mess.

The I226 appears to have inherited that mess.

Whatever cards are using the Linux "ice" driver have been giving us trouble at $work.

That is the 800 series.

rmustacc beat me to the punch. AIUI 800-series are just more of the same terror that the 700-series had, but faster.

s/faster/more bandwidth/g

It's actually rather different.

of course if they're problematic even on linux, probably not a good sign for us (IIRC we don't have a driver yet, but could port from fbsd)

I was stressing about both 800-series and I22[56] but the more I hear the less I stress.

It's fine. In another few years Oxide will hire some the right people and make good NICs. :-P

s/some //

obviously the answer is to just use infiniband and RDMA

I started an ice driver a long time ago.

I'm so happy just not caring about modern ethernet

except being sad whenever cscope leads me to the drivers.

Lucky SOB... :)

with words like that, you know you're going to end up neck deep in the scsi stack :)

illumos friends! how are you? asking here for generic Kernel engineering advice :)) I have a FreeBSD Jail that's not releasing its resources. All I know is that I need to panic the system, but no idea what to do after that :)) can anyone guide me to kernel programming/debugging... (and I can't believe I'm saying this) tutorial! :D

BTW, DTrace has been hella handy to figure out the issue :D

you're not asking in quite the right place for q's like that. (I'm not sure what the state of the art in FreeBSD kernel debugging is and whether it's better to use a postmortem analysis of a crash dump or poke at the live system with a kernel debugger and what the best options for either of those is...)

kernel is just a big multithreaded C program that doesn't have the OS to fall back on. illumos has tools (like ::findleaks in mdb) which might be relevant but sometimes problems like what you describe can be due to refcounting issues, etc., and it's a matter of hunting down the foo_rele() call for some object/type you've never heard of before.

The most useful equivalent is to figure out how to walk the kernel data structures in the equivalen tof say mdb -k or a crash dump.

Sometimes it feels that illumos is the most advanced OS on the planet :)

actually, both advices were handy! just learned about kdb :D

jbk: I'm with you on IB+RDMA

RDMA access in zones mapped across an infrastructure seemed like the grand next step

converged cloud and HPC