Hi, Im looking for some idea/help regarding OmniOS. I tried to boot it on a machine which I like to use as VM host yesterday but it halted after CPU/USB initialization.

I tried an older image I had somewhere, this one worked immediadly, the one which doesn't work was "omniosce-r151036.iso" - the older image is "omniosce-r151036.iso"

I googled and found an older thread regarding a hanging boot on an Intel machine from 9 years ago. But the flags being proposed to add to boot command didnt really help.

err, correction the image which worked to me was "omnios-r151052b.iso"

151052b is the latest release so if that's the one that worked then that shouldnt be a problem

I see, yes you're right indeed. Maybe I was confused with the 'ce' because the community edition was mentioned explicitly named in the docs. But nowadays its just "OmniOS" or are there two seperate releases?

if you're trying to boot on physical hardware I wonder if it's also worth trying to use the usb-dd image rather than the iso

ah ok there are two seperate images as well? I usually use something called 'balenaEtcher' and simply use those images to flash (it works very well usually)

and another thing I would be interested in is bhyve, how on par is it with the current FreeBSD implementation? Because from what I understood kvm is deprecated and will never be updated again right?

i think the ce was there historically to distinguish from the older omniti images

I've not made use of bhyve so don't know much about it and what it supports

ls -l

oops

ok then I'll just have to test if bhyve on OmniOS works like on FreeBSD

megaTherion - bhyve is pretty much on par with current FreeBSD. It's synced periodically.

andyf: cool, thanks for the confirmation

as I understood there was no way to get KVM more current due the ongoing changes which are very heavily relying on Linux (kernel) ecosystem

Our kernel implementation is a bit different these days, but then we're a different OS so it's not entirely surprising. The userland pieces are fairly similar still.

Right, there are no plans to update KVM now.

ok now Im trying to install OmniOS and I've two NVMe's I want to use, creating a pool fails. I guess its disturbed that there is still a gpt disklabel and all on those

you can try to clear them with the "zpool labelclear" command

no dice

fails how?

"failed to clear label for /dev/rdsk/....."

but I get warnings that the label is corrupt, they had GPT and some btrfs on them I think

and zpool create is failing how?

its too fast gone... cant read it - what's OmniOS tool for GPT labeling disks?

you can use format command for that

format -e, then select the disk and then enter verify to see what is there

ok, cleared the label with parted on Linux... now Im back on OmniOS shell, format -e / verify gives me "Cant read label"

installer still fails with the same issue

I tried labeling with format -e / label (select EFI label) this worked. Now I still need an ESP I guess

if format agrees you have correct label, you can use zpool create -B to create whole disk pool and ESP for you.

or use installer:)

the installer fails, I'd really like to use it :D

and since Im (yet) unfamiliar with OmniOS commands (my OpenSolaris knowledge is like 10+ years ago)

does it fail even with fixed gpt label?

yes

I want to try from shell, but I need to figure out which device/partition it is

ou, ok, thats interesting.

well, for whole disk setup you only need disk name without /dev/rdsk, and use zpool create rpool -B c...d0 -- thats d0 at the end of the name.

so the disks get detected as c3t001B448B4AF180B4d0 and c6t6479A789800000CDd0

format is displaying them and also the correct names (one is a WD 550 blue something and the other is a Crucial nvme)

that should create GPT with 256MB ESP on s0 and then most of the space in s1 for rpool and 8MB slice at the end.

ah ok so zpool will also create the ESP, didnt know that

thats -B

without -B, you will just get s0 and s8 (8MB)

oh now I get an error, "Boot pool can not have more than one vdev"

yes, if you want to present both disk to it, you probably want to use mirror

(because I wanted to have those in a striped configuration)

nah mirror I didnt want (I've backups elsewhere)

so its zpool create -B rpool mirror disk0 disk1

yea, boot pool is special there, it can not have multiple vdevs

it can be single disk, mirror or raidz

there was some work to fix this, but afaik it is still not completed.

ok I cheated, maybe it works - I just added the other disk afterwards to the pool

it will not. you can install it, you can boot from it, but kernel will panic.

botloader can read stripe, but kernel rootfs mount will fail because it has no idea where that second disk is.

thats sad :(

but I've a bigger problem now, it hasn't detected my NIC :D

eh...

well RTL8125 2.5gbps

indeed not an Intel chip, so it sucks

ok I've some other Intel I225-V 2.5gbps card around, I'll just drop this one in it

tsoome_: btw. booting the striped rpool works, would be suprised if not - if it works in a sane way then the kernel shouldnt bother where to find that damn vdevs

works on FreeBSD too

well, your blocks will be spread on both vdevs, yes loader can walk through the disk list (which also has limits), but kernel needs to get boot pool "imported" and root dataset mounted, and for that it needs to find out and build in memory pool config. from disk label it can collect vdev config, but to find next vdev, it needs to browse the disks (as loader also does).

however..... I just did realize we could pass device names from loader to kernel, so the kernel can have a bit easier work, but even that does not work for 100%, because if device names are changed, the kernel still needs to scan disks to find out the new reality.

rpool ONLINE 0 0 0

c1t001B448B4AF180B4d0 ONLINE 0 0 0

c4t6479A789800000CDd0 ONLINE 0 0 0

well it seems to work, maybe I get crashes later who knows :D

ou,

thats ok boot?

yes it boots

let me check if we have any notes around...

thats why I couldn't progress with the installer, because I chose both disks and striped configuration (which was a valid choice) but I got the zpool error

um, jclulow was some corner case left with disk device scan or did we just forgot to lift the multi vdev restriction on boot pool?

tsoome_: It would be more accurate to describe the thing that now works as a corner case

i.e., single vdev pools

see also the notes in illumos.org/issues/7119

→ BUG 7119: boot should handle change in physical path to ZFS root devices (Closed)

ok, so we can drop this restriction.

Which restriction exactly

multi-vdev boot pool is not allowed atm

oh, booting from a _stripe_?

yes.

I just did pfexec pkg update & reboot, still works

I don't think I would ever recommend doing it, but I can't immediately think of a reason it should be banned per se

/dev/rdsk/c4t6479A789800000CDd0: NVMe devices are not supported in this version of smartmontools :(

cd ..

oops:D

code.illumos.org/c/illumos-gate/+/3874

→ CODE REVIEW 3874: 16983 zpool: we actually do support multi-vdev boot pool (NEW) | illumos.org/issues/16983

btw. are there pkg mirrors in europe?

I think the primary pkg server is, itself, in Europe

It's why we need a us-west mirror haha

ah I see

well now I'll tinker with zones a bit and have to do a lot of reading

I created a lipkg branded zone with zadm and added allowed-address / defrouter to the config... it installed but I've no route. So the defrouter config is somehow not effective