04:20:20 <thetooth> rebooted a machine after a PI upgrade, all of the zfs pools went missing, zones NVME offlined itself due to missing mirror, no networking other than the admin interface. who can guess what went wrong? :)
14:41:40 <nahamu> do you already know? did rolling back fix things?
14:42:10 <nahamu> sjorge and I have been seeing oddities with some NVMe drives (though I think sjorge might be on OmniOS these days)
14:42:30 <sjorge> i am on omnios
14:42:34 <sjorge> but bloody
14:42:40 <sjorge> so very new bits
14:43:15 <sjorge> had nvme's just drop with nothing in dmesg/serial console during a scrub
14:43:18 <nahamu> thetooth: if you know something I am very interested. If not, you're not completely alone in seeing weird stuff with NVMe.
14:43:23 <sjorge> i did swap nvme recently so
14:43:37 <sjorge> but it's odd that is very similar to nahamu 's issue
14:45:28 <nahamu> thetooth: which versions did you upgrade from and to?
14:48:29 <sjorge> I do vagually recall some nvme changes in the last few months
15:39:01 <thetooth> nahamu: the bios battery died, in a 11month on machine
15:40:00 <thetooth> so it reset to defaults on power cycle because I guess that part of the board doesn't need power during the brief moment it goes from off to on again
15:43:13 <thetooth> took quite a while to figure out, i eventually saw that the OCU links had been set to pcie instead of sata, fixed that settings and had the drives show up in the bios (hotswap port configuration is a thing i guess) only to have it reset when rebooting smartos