17:15:15 Had a bhyve VM displaying OOM Killer in the console logs I ran `vmadm shut -F`. In another terminal I had a bhyve console connect and I saw the `zone rebooting` message pop-up. Then never go any further. I finally `ctl-c`ed my `vmadm stop` command. I was left with a Zombie process that cannot be reaped with `preap` 17:15:52 john: What platform version are you on? 17:16:26 20221225T144337Z 17:16:44 This is barfield btw, not sure why my nick is wrong I did set it on login. 17:17:32 We had a major bhyve upgrade starting with 20230223 17:17:56 I remember seeing Dan's notifications about upstream merges 17:18:05 Ultimately, what you're describing is a bug in bhyve, and the update might have fixed that already. 17:18:36 Aha, that explains it. Is there a work around to kill the VM in the future? like bhyvectl --destroy or something rather than vmadm stop -F? 17:19:00 Unfortunately things like that aren't just easily reproducible to test if it's fixed, but if you're using bhyve you almost certainly want to be on 20230223 or later. 17:19:22 I will start a platform upgrade then ASAP. All of these systems are bhyve 17:19:58 With that being said, does Triton and Manta need to be updated for any reason as well? I've been considering going through that as well. 17:20:26 Maybe? But also remember that bhyve (as well as qemu) runs inside a zone so take that into account. 17:21:17 It's best to review the release announcements for highlights. I don't know what release you're on so I can't say what you're missing out on. 17:21:22 Yeah, I knew that, because the actual zombie is is coming from zone init 17:21:37 I'm only a couple of months in arrears on triton 17:23:13 The big things from the most recent release is that pkgsrc-tools can be set up during headnode install, rather than having (and potentially forgetting) to do it later, and the shared/support shell (triton-scry) utility. 17:24:23 Cool. I saw the new shell, that is pretty awesome. We could have used that back when I worked at Computer Associates lol 17:24:59 Yeah, it's super handy. 17:28:46 20230406T001127Z 17:28:52 looks like that is the latest available 17:29:53 Theres no way to hotpatch bhyve without bouncing every node that you're aware of is there? 17:39:37 bahamat: is the linuxCN project still under development? 18:19:23 john: vmm and viona are separate modules. You _might_ be able to do some funky stuff to "hotpatch" bhyve. It's been awhile since I tried doing that. And iirc you needed mdb -kw to actually unload vmm etc. pmooney might know if it's even worth a try these days 18:20:30 and of course that means you would also need the right bhyve executable that works with those kernel modules 18:20:35 I would not recommend loading a newer bhyve or viona module on an older kernel 18:20:45 there is so much which could go wrong 18:21:14 and yes, you might need an updated libvmmapi or bhyve binary in userspace as well 18:21:48 that isn't to say that doing such a thing is necessarily impossible 18:21:57 but it's fraught, and would require careful research 18:22:17 and is likely to put you in an even more tenuous position when it comes to correct behavior 18:32:53 papertigers: Sounds like more work than I am willing to even attempt! Hows your new(ish) gig anyway? Been a long time! 18:33:27 pmooney: Thank you for the insight definitely sounds risky. I'll just deal with the reboots. But I appreciate the architectural breakdown at the very least. Good information. 18:33:28 john: going well! still doing all the illumos things in my spare time haha. How are you? 18:34:29 I will DM you sometime and give you all the updates :) ... I'm still doing it too...illumos4life 18:35:54 I always have to remember how to even do a DM in irssi/irc lol 19:06:48 john: about linuxcn: Depends on your definition of "still". It's still a project that we want to pursue, but there hasn't been much active development on it lately (largely due to the multiple truckloads of work moving all of Triton from one owner to another). But it's also being rekindled and hopefully we'll have a new "blessed" release soon-ish. 19:46:40 bahamat: thank you, I am looking forward to playing around with it. Though I will always be a smartOS user first.