00:13:18 neirac: I'd be curious if you compiled and ran https://gist.github.com/jasonbking/6f1475e4fe1684042b9fb17c617d3151 on there to see what it returns 00:14:21 (might be nothing) 00:45:22 jbk I thought of that, but omnios bloody did not have that change, I'll try to test it 00:46:54 well this will tell if it'd help or not 01:59:26 jbk: rmustacc: rzezeski, some NIC support RoCEv2, to use SR-IOV for a NIC can pass through the configured VFs to Bhyve VMs to run RDMA over RoCE; not sure if vnic would run RDMA or support RoCEv2 ? I have not find document say vnic support RoCEv2, a big thanks 02:20:55 tozhu: No, VNICs are not setup for that. What you are saying is technically possible. But we currently don't support setting up VFs and handing them off to VMs. In our deployments we have preferred to have the hypervisor (illumos) control the classification/switching with hardware acceleration from the NIC by mapping MAC addresses and VLANs to specific queues, and use the viona/VIRTIO device to deliver traffic. 02:27:10 rzezeski: Thank you, so there is no way to do RDMA/RoCE on Bhyve? my usage case is rdbms,the SDS (Software Defined Storage) targets supports RDMA over RoCEv2, I’m looking for the solution for RDMA over RoCEv2 for the Bhyve VM for the initinator, 02:30:53 I read this link say virtio-net added RoCE support on Linux ,https://lore.kernel.org/all/20220511095900.343-1-xieyongji⊙bc/T/ , not sure if illumos had it now. best wishes 03:43:39 there really hasn't been much interest in rdma in illumos to date 03:44:16 (doesn't mean taht can't change, just how it's been historically) 04:14:46 hrm.. 04:14:56 i think i'll save this crash for tomorrow 04:15:04 in mac_util.c 04:24:40 ...hrm.. and i feel like i've seen this before 04:25:12 with emulated e1000g panic on DEBUG in mac_sw_lso 04:54:49 I think the RDMA based application is very importand, esp for NVMe over Fabric, with high performance database application, works with Bhyve on illumos, need very low latency, else the DB size is limited by the phy host configured disk, or have to work with very low performance mode due to very high latency with network 15:28:24 anyone have access to the opengrok instance? seems like it might need to be kicked 19:15:55 jbk I did not read your message I thought it was a vioscsi commit, because I saw a commit sometime ago for vioscsi, I cannot compile the program omnios boots I cannot install software 19:19:30 jbk: keep that panic (in mac_sw_lso) around, that's precisely the sort of function that can be tested well with ktest. We should be able to extract your input mblk from the dump and make a regression test along with the fix 19:20:05 I'm happy to grab a copy of it and give it a look 19:40:20 rzezeski: looking forward to playing with ktest.. 19:46:51 sommerfeld: Good to know. I kind of let it stall out because I thought maybe there wasn't too much interest (also dealt with some life stuff that kept me out of the game for a bit). But I was recently assured that it would be useful. So I'm happy to do the work to get it in an acceptable state. 20:17:09 [illumos-gate] 15973 Win11 SMB set atime=-2 gets internal error -- Toomas Soome 20:55:45 [illumos-gate] 15974 SMB logoff door up-call can suffer a thundering herd -- Gordon Ross 21:02:38 rzezeski: ok... it looks like somesort of mismatch w/ checksum offload and lso when i looked at it briefly 21:03:05 it could be the emulated e1000 is somewhat brain dead 21:03:19 but i'll keep that panic around 21:19:52 tozhu What OS are you running your DBS on inside of Bhyve? 21:22:14 Are you wanting to pass in a PCIe device so that you can run an RDMA stack inside of it? Or are you thinking that the paravirtualized NVMe in Bhyve can be pass through to the host through NVMoF? Sorry, what you said brings up a lot of questions to me 21:24:22 Although there isn't SR-IOV support to pass through a virtual PCI function with illumos as a host in the drivers, NVMe/TCP is pretty close performance wise, with less CPU overhead and additional latency than one would think 21:26:33 Actually, with NVMe/TCP, the only applications that seem worth it to benefit from RDMA is HPC stuff with MPI. With the availability of CPU cores these days I'd rather have something IP/Ethernet based 21:45:12 neirac: i can give you a compiled binary if that'd work 22:00:41 rzezeski: also, I'll hopefully try to look again at the ktest stuff soon...