10:55:48 <tomww> out of interest, has been anyone able to run Solaris 11.4 (CBE) in bhyve? I get the boot stuck in module loading.
10:56:26 <tomww> With disabling ohci / ehci / uhci I could get it a bit further but then it gets stuck around iwscn
10:57:32 <tomww> Only if I could send the debugger a break to see more... (tried on SmartOS latest, what does OmniOS bhyve?)
10:58:34 <andyf> `zadm nmi <zone>` ?
10:58:55 <andyf> I run Solaris CBE under bhyve
10:59:22 <tomww> oh that is ... I WANT THAT TOO :)
10:59:29 <andyf> Have you tried booting without any network devices, just as a test?
10:59:36 <tomww> but my hypervisor is smartos only 
10:59:50 <andyf> There was definitely a problem with e1000g, and maybe vioif too
11:00:00 <tomww> not yet, I can deconfigure net in the smartos zone config.
11:05:43 <tomww> do you happen to have an example (anonymized) bhyve command line for me? maybe I can see some diff to what smartos does.
11:06:48 <andyf> https://paste.ec/paste/uc-rzKaY#kiQCugfbRVFF0fJ5MkOWGW3MTpTDHShXDohH6IsT72r
11:07:28 <andyf> for networking I ended up building the illumos vioif driver and I use that in the solaris guest (until I find time to investigate what is going on with the native solaris vioif driver)
11:12:29 <tomww> thank you very much! I'll dig into it during the day.
12:11:11 <tomww> I have these bhyve command line on SmartOS. DO you think there is something off on there that might be worth trying to changeß https://paste.ec/paste/zDOyQoQg#Tnw-plP4ZSIMybC8J8R5m5V3mNiweOwMDg3DacEjIdz
12:12:39 <tomww> s/off/odd/
12:31:08 <andyf> You should be able to use a virtio disk with CBE, in case that helps
12:31:25 <andyf> as far as I remember, the latest version has those drivers
12:31:46 <andyf> but otherwise I don't see anything in those options
13:40:11 <tomww> Maybe I should throw an ssd into the box and install OmniOS and see if I can replicate your S11.4 bhyve setup. 
13:41:06 <andyf> on smartos you should be able to inject an NMI with 'bhyvectl' to see where it's stuck
13:41:16 <andyf> that will give some pointers, anyway
13:41:30 <andyf> I don't know if that's wired back to the smartos VM tools
13:41:33 <tomww> oh good idea, I'll do that first.
13:41:55 <tomww> at least in vmadm console I wasn't able to break into the kernel debugger.
13:57:44 <tomww> hm. at the moment no luck with breaking into the kernel debugger (VM powers off too fast while in previous runs it just stayed hung)
14:03:22 <andyf> I don't know much about the smartos framework unfortunately. It's still using command line parameters and I think it has an older bootrom too.
14:14:53 <wiedi> tomww: try vmadm sysrq <uuid> nmi
14:15:38 <wiedi> ah that might only be hooked up for kvm
14:20:31 <andyf> tomww - I can set up a fresh VM on omnios and see what happens. I'll try it later on this afternoon.
14:21:48 <tomww> oh that would be wonderful. I can work with S11.4 in KVM for the current ansible projekte, but in th elong term I would 
14:22:57 <tomww> prefer a bhyve VM for s11.4 (SFE project). If your attempt in a clean env works, I might try the different bootrom eventually?
19:05:02 <tomww> oh the right people entering the channel :)
22:01:28 <tomww> well well well, one can't have a vnic in a hbyve zone on smartos. why is this. 
22:02:00 <andyf> You should be able to
22:02:45 <andyf> It won't actually be able to transmit/receive unless you add its MAC address to the secondary-macs property on the bhyve zone's VNIC in the global zone, but nothing should prevent you creating it
22:03:07 <tomww> hm well
22:03:30 <tomww> the guest is omnios:
22:03:31 <tomww> root@omnios:~# dladm create-vnic -l vioif0 huhu0
22:03:32 <tomww> dladm: vnic creation over vioif0 failed: operation not supported
22:03:52 <tomww> is my command wrong?
22:04:23 <andyf> Well then, it looks like they are missing a commit from gate that allowed that
22:04:51 <tomww> I did another try, on the smartos global zone:
22:04:59 <tomww> [root@KVM03 /]# dladm create-vnic -l e1000g0 huhu0
22:04:59 <tomww> dladm: VLAN over 'e1000g0' may not use default_tag ID (see dladm(8))
22:04:59 <tomww> [root@KVM03 /]# dladm create-vnic -l igb0 huhu0
22:05:00 <tomww> [root@KVM03 /]#
22:05:01 <andyf> illumos 13695 (fenix)
22:05:02 <fenix> BUG 13695: Can't create VNICs over vioif after 13637 (Closed)
22:05:02 <fenix> ↳ https://www.illumos.org/issues/13695 | https://code.illumos.org/c/illumos-gate/+/1402
22:05:15 <andyf> Perhaps SmartOS does not have this fix?
22:06:16 <tomww> could be. I'll verify if my bhyve VM uses the hardware e1000 or igb0 ... maybe this changes someting?
22:07:42 <tomww> okay. I totally forgot I have an aggregation running too. so this works:  dladm create-vnic -l aggr1 huhu1 (global smartos zone)
22:08:18 <tomww> I'll try to find out if BUG 1369 is merged into smartos
22:08:40 <tomww> 13695
22:09:39 <tomww> for the moment I don't need it. I was playing around with ansible to create a zone in omnios (guest on smartos) and it failed to create a virtual interface for the zone.
22:16:37 <richlowe> it's in smartos
22:16:40 <richlowe> according to git branch --contains
22:16:53 <tomww> I have joyent_20221117T002317Z
22:17:00 <richlowe> sinc release-20210408
22:18:23 <tomww> should it be possible to run this bhyve VM with e1000 nic? I have model e1000 but still shows in the guest as virtio nic
22:19:32 <richlowe> probably best to ask stuff like that in #illumos