hello, is it possible to use oxide’s crucible storage from a local zone? eg: use crucible from OS based VM in smartos?

tozhu: It would be in principle, but you'd need to write some more software; in particular, you would need an illumos block device driver that was able to speak to Crucible

The way we use it at Oxide, the hypervisor, Propolis, speaks directly to Crucible as the backend storage of the emulated (NVMe, vioblk, etc) disk

fancy

It probably wouldn't even be that complicated, to be honest; you could get started with a usermode daemon that did the Crucible bits, and just a relatively simple "bent pipe" in the kernel. Like FUSE but for block devices. The block device surface area is not that complex, especially with blkdev

jclulow: Thanks for the response,

You are most welcome!

so, all of the crucible is running in user space?

is it possible to implement the connection base on RoCE/RDMA between crucible server and vm/crucible client?

tozhu: Yes, crucible is all user mode software so far.

And, if RoCE/RDMA supports TCP/IP on top, then sure!

We use ethernet though, obviously.

TCP/IP had some issue on the stream control, it only use about 60% of the bandwidt

guys, I still fail to find my nvidia driver issues (NVRM: GPU 0000:01:00.0: Failed to copy vbios to system memory.), the question is would I get wiser if I turn on kmem flags and use kmdb voodoo? or will it be useless because without driver source we can't seen anything useful?

Does illumos have anything like linux futex?

In the sense of the direct system call?

Yes, as system call, but could be in a different way (I don' t even know if futex can be implemented without support of a system call)

My experience is more with BSDs, Darwin and Linux, and most of them have a somewhat recent system call that is futex-like and used to create fast user space synchronization primitives

[illumos-gate] 16615 ena assertion failure in ena_tx_intr_work() -- Andy Fiddaman <illumos⊙fn>

I don't think of us as having something quite like that today. libc and the kernel are working together though.

That is, the system call layer isn't where we promise stability.

So, for now, there is no system call that is futex-like and libc also don't have/expose something similar at this point in time, correct?

Sorry to ask for confirmation, English is not my native tongue and also I'm neurodivergent, so I often do that to make sure I understood correctly

That's correct. There is nothing futex-like exposed today.

Is there something in particular that you're working on where it would be useful?

I'm creating my own libc, to learn more about architecting something that is so essential to C, as well to learn more about the internals of OSes that I'm exited about.

Gotcha, okay. Well, if you have other questions about the mutex subsystem, feel free to reach out.

I'm doing that in Rust, because I like the lang and it can expose C ABI function and types, but internally I often create more Rustic abstractions for the implementation details

Thanks rmustacc! If you can point out where I can learn more about illumos mutex subsystem, I would love dive right in

I don't have specific docs for it, unfortunately.

Though with a caveat that I'm likely going to have to change some of the system call ABIs to deal with some upcoming posix work to let locking be based on other clocks and use non-absolute times.

if you want to look at the current implementation (if you haven't found it already), src.illumos.org/source/xref/illumos…ibc/port/threads/synch.c?r=7f9dc0ee

Thanks rmustacc and jbk, I was close! LOL I was looking for it!

If any question come up in my head, I'll come back here!

and obviously if you have a clone of the git repo, you can use that to figure out the path to the file from the url

I am working on a CI system thing and I am trying to figure out: how would I ensure, with 100% certainty, that no more processes exist for a particular uid?

I feel like it is actually surprisingly difficult

you can use proc to stop a particular process, and ensure it can't fork anything more, but what if that process had already forked five children who are forking their own children and so on

oh, wait, sigsend(2) maybe

Yeah, sigsendset() holds pidlock.

So it guarantees a consistent view of all procs.

Nice

Well that solves that problem!

And I belive it guarantees that it will hold it across that time.

sigsendset() gives you a little more flexibility if you want it.

Yeah, I see that!

This is a very "us" interface

I like it

It's used by the tty code for doing pid/pgid.

I guess I'll "sigsend(P_UID, $uid, SIGKILL)" my way to ESRCH/success!