Hi to all. Still working on test hardware emulation in OmniOS: unfortunately, behyve VMs make my host crash and reboot (I guess it's because of the old hardware I'm testing on: AMD Sempron 145)...

... so I tried to run FreeBSD in a KVM guest, but I'm still unable to solve this problem: ERROR:root:Error b'open /dev/kvm: No such device or address\nfailed to initialize KVM: Not owner\n'

did you reboot your host in between your tries with bhyve vs. kvm?

Yes.

obviously, /dev/kvm is owned by root. I didn't find anything online that helped me to understand what I'm missing

are you sure your sempron 145 is even supported by either bhyve or kvm?

and even if it is, are the virtualization features enabled in the bios?

Obviously, yes: I tested with bhhwcompat. Here is its output: "CPU vendor string: AuthenticAMD\n ... CPU supports SVM\n ... SVM support is enabled in BIOS\n bhyve is supported on this system."

But I do not know how to check KVM compatibility (first time user here, sorry)

well, if that's the case, then both bhyve and kvm should work, but obviously not at the same time.

maybe the cause is the same: old hardware behaving abnormally?

well. it's not just old hardware, it's old hardware that was low-cost consumer hardware even when it was new

if your experiments are crashing the host, you should try to get a core dump or at least a stack trace

and if you suspect your hardware being the problem, it may make sense to run memtest, or try different (newer) hardware

Thanks, actually I'm giving OnmiOS a try on a recycled and inadequate hardware and I can surely blame that for my problem.

I do not want to waste your time, but I'd like to investigate a bit further, in order to be sure it's an hardware-related problem and not a configuration fault of mine... can you point me to the docs where I can learn how to get and observe a core dump when bhyve makes the host reboot?

that should happen automatically if your OmniOS host crashes, and on the next reboot the core dump should have been saved somewhere in /var/crash/

ok, in fact there is something here (METRICS.csv, bounds, vmdump.{0,1}). How can I get some info from that?

well, the first step would be to run savecore to extract the vmdump files, giving you the uncompressed vmcore and unix files. then you can run mdb on the core dumps and start digging in

the examples in the above book should get you far enough to be able to file a bug report :)

wonderful, I'll study the docs and try to analyze the core dumps. Thank you so much for your support!

I'm a long time BSD user, but Illumos and particularly OmniOS is actracting my interest in the last months... I think there are many SMEs out there that could make a good use of it in a lot of use cases.

I've just opened the core dump and this is the message I found after typing "::status": panic message: assertion failed: vie->status & VIES_INIT, file: ../../intel/io/vmm/vmm_instruction_emul.c, line: 2767

I guess it means the hardware I'm running on is unable to serve the request from bhyve. So it's not worth to open a bug report for that.

warden : not sure, but have you tried booting smartos on that hardware?

well, I hadn't thought of that... I'll give it a try as soon as I'll be able to put another disk in the same PC (in order not to break anything for OmniOS), thanks!

warden that looks like illumos.org/issues/16726

→ BUG 16726: in/out VM_EXIT should not assert on CPUs without DecodeAssist (In Progress) | code.illumos.org/c/illumos-gate/+/3645

I can build you a hotfix with that patch to try if you're willing.

Just let me know what version of OmniOS you're running so I can match it.

it would be wonderful, thank you so much. Here is the output of "uname -a": SunOS alpha 5.11 omnios-r151050-49db1c0a0fe i86pc i386 i86pc

It will stop the kernel panic, but bhyve isn't going to work unfortunately.

To fix that someone would have to implement userspace emulation for the missing instructions.

ah, ok... so I cannot ask you to make the patch for such an unimportant problem: I'm only testing OmniOS, and that's not a production machine. But thanks for your kindness

Another question (and sorry for my ignorance): that's likely the same cause which makes KVM hang on this host?

It would still be very useful to confirm that it stops the panic if you can. It adds to the testing notes for getting that patch integrated.

It will take an hour to build the patch, but I just set it off and let it run

Regarding kvm, I haven't looked but it wouldn't surprise me if it also depended on decodeassist. As far as I remember, the hardware requirements for kvm are a superset of those that bhyve wants.

it's ok for me, so I'll keep waiting for the patch. Thanks!

warden if you do `pkg apply-hot-fix hf.omnios.org/r50/bhyve-decodeassist.p5p` that will create and activate a new boot environment. After a reboot, hopefully bhyve will just exit rather than panicking the machine.

ok @andyf, I just applied the fix and rebooted. bhyve VM still does not works, but the host does not crash anymore. Here is the content of /zones/obsd01/root/tmp/init.log:

fbuf frame buffer base: fffffc7fe8200000 [sz 16777216]

rfb: no support for fast crc32

Failed to emulate instruction sequence [f3, 6c, eb, 0a, e3, 08, ec, 88, 07, 48, ff, c7, e2, f8, 4c] @ rip = 1fb42820

Perfact, thank you. Would you mind adding that information to illumos.org/issues/16726?

→ BUG 16726: in/out VM_EXIT should not assert on CPUs without DecodeAssist (In Progress) | code.illumos.org/c/illumos-gate/+/3645

Or if you don't have an account, I can copy and paste it in there

I do not have an account, please do it for me, thanks!

Done, thanks for testing.

thanks very much for your support. I'll go on studying OmniOS on a more decent hardware ASAP! ;)

It looks like we need to update bhhwcompat too. You could probably add the missing support to bhyve, but I'm not sure how many people will be running hardware without this feature.

well, I do not know, but the hardware I'm testing on is really old and undepowered... I guess that there are not so many systems used for production which have this problem out there.

Would you be willing to run the updated bhhwcompat binary from downloads.omnios.org/misc/bhhwcompat ?

ok, now it tells me that the system is not supported: "CPU vendor string: AuthenticAMD\n ... CPU supports SVM\n ... SVM support is enabled in BIOS

... SVM supports nested paging\n ... SVM supports NRIP save\n ... SVM does not support decode assists (essential)\n bhyve is NOT supported on this system."

Thanks for checking.

warden: not sure of your budget but I'm really happy with an AMD EPYC 8004 (zen4c) system I built over the summer.

Oooh.

And @warden the Ryzen family is also quite good too; they even support ECC memory if your mobo does. (I've a 4-year-old Ryzen 3700X build.)

Love my

Ryzen build

Not running illumos though, no ppt on bhyve 🥺

Although it did all work aside from that