I think it should be okay to keep builtins around and take other snapshots as things change and on an as needed basis.

At least, that would be my general disposition / expectation. But I dunno.

hello all, a simple question regarding to ZFS: if there is only SSD for a machine, eg: 12 * 7.6T ssd, do you advice slog? and what’s the raid is recommand? mirror / raidz1 or raidz2 ? best wishes

No just use slog if you can at least mirror it, however ArcL2 is ok. openzfs.github.io/openzfs-docs/Basic%20Concepts/RAIDZ.html

No recomdation depends on your usage pattern/availability etc

generally, having a separate slog device is useful when it noticably faster than the rest of the storage in the pool

e.g. SSD slog, HDD disks in the pool

if it's already SSD, having a separate slog device is likely not to be as useful

since it's the log, so (broadly speaking) I/Os get written there first

so if it's the same speed as the rest of the disks in your pool, you're artifically bottlenecking yourself

(I've seen this happen where someone created the log device on another HDD and the complained their pool was slow :P)

heh.. i think i'm going to have to knock some heads while i'm at the office next week

jbk: although I believe having a slog that's at least as fast as the pool can still be useful for cutting down on fragmentation iirc

but it kills your performance unless the pool is basically a single disk

because you end up only being able to write as fast as that slog device

again, that's usage dependent.

Though I can't think of a usage where having a single-drive write speed is optimal.

Hello! I was trying to passthrough the USB ports in my server to a bhyve VM as a dirty hack to install Windows Server 2022, I mapped the PCI address of the USB controller in /etc/ppt_aliases and /etc/ppt_matches and after rebooting the active BE fails to boot complaining that it cannot read the pool label and therefore it cannot mount root.

I'm able to boot with an older BE and mount the failing root zfs filesystem and access it. So far I've deleted the /etc/ppt_* files, deleted /etc/zfs/zpool.cache, I have booted from an install ISO and imported and exported the pool, disabled and enabled the BE, created a new BE based on it and trying to boot it... But I still get the same error.

It's not critical since I could go back to the old BE, reinstall some packages and copy the config files from the broken BE, but I really would like to know what I messed up and how to fix it.

The full error message in the console is NOTICE: Performing full ZFS device scan!

NOTICE: Cannot read the pool label from '/pci@0,0/pci103c,870c@17/disk@2,0:b'

NOTICE: spa_import_rootpool: error 5

Cannot mount root on /pci@0,0/pci103c,870c@17/disk@2,0:b fstype zfs

hrm.. 5 is EIO which isn't particularly helpful

that's EIO trying to read the label from that device

I'm running OmniOS r151046

it might mean the device in the pool config is not correct, and we can't find where it now is

But I'm able to boot from the old BE which is just a different FS on the same pool

have you scrubbed the pool?

No

Should I?

my only idea is maybe it really has got issues reading the new BE, if everything else semes to work, but you might want to wait for someone more expert to agree with me (or not)

Could be.. I hadn't thought about it... Because in the 1st reboot it must have been accessing the filesystem, then read the /etc/ppt_* files, mapped the pci thing which apparently also had the root disk hanging off it to the ppt device and immediately lost access to it

I'll try the scrub, thank you!

I'm not sure if ppt info is stored anywhere else that may have not got nuked, pmooney would be the one who knew that

and jbk probably your best bet of people currently awake for zfs

or sjorge, honestly

I've not had a big hand in the ppt stuff, besides continuing maintenance

yeah, i've not done anything really with ppt

and i think Woodstock is on vacation (ISTR he did a lot of the PPT work)

yeah, he did a bunch of the porting work

you could maybe try booting w/ kmdb and poking around with ::prtconf to see if that device is there

though I have no idea if that was getting passed-thru what it would look like in that

or what to look for

The scrub worked!

It reported 0 errors, but I rebooted just in case and it booted the main BE fine

It's behaving a little weird, I have no IP address now but that probably is a separate issue

I'll get to fix that and keep an eye on it, thank you very much for your help!

I have been using illumos for a few weeks only but it really feels very solid and reliable, I'm liking it a lot

that is... odd

Maybe the scrub made it refresh some internal device identifiers or something?.. I don't know the inner workings

yeah..

the only think I can think of (and this is a stretch) is that maybe the disk name (e.g. c0t0d0) changed because of the pass-thru so maybe the cache file was wrong?

The ip thing was me disconnecting the ethernet cable while I was fiddling with cables to connect a monitor and keyboard to it to debug why it was not booting... :)

though i think it should still be able to figure things out

pilonsi: it's always nice when it's a simple fix :)

Yeah, and the other be could boot fine off that disk as well.. its strange

Anyways I should start getting used to creating backup bes before I do weird thing

well the cache is per-BE

though also... that might end up in the ramdisk...

not sure offhand

(just thinking out loud)

I did delete the /etc/zfs/zpool.cache file just in case, but it made no effect

Unless it has some other cache, pool information, etc.. files stored under / ?

pilonsi: you deleted from mounted dataset of failed BE or booted dataset?

from mounted dataset of failed be

pilonsi: Can you check whether 'prtconf -v' find that device path '/pci@0,0/pci103c,870c@17/disk@2,0:b' ?

pilonsi: something like this: prtconf -v |grep -A 2 'dev_path=/pci@0,0/pci103c,870c@17/disk@2,0:b'

dev_path=/pci@0,0/pci103c,870c@17/disk@2,0:b

spectype=blk type=minor

dev_link=/dev/dsk/c1t2d0s1

dev_path=/pci@0,0/pci103c,870c@17/disk@2,0:b,raw

spectype=chr type=minor

dev_link=/dev/rdsk/c1t2d0s1

pilonsi: BTW, could you check that /etc/path_to_inst in both datasets have that path '/pci@0,0/pci103c,870c@17/disk@2,0' ?

pilonsi: Also I would suggest to recreate '/platform/i86pc/amd64/boot_archive' in the failing dataset

pilonsi: but save a copy before.

In both /etc/path_to_inst I get the same:

"/pci@0,0/pci103c,870c@17/disk@2,0" 1 "sd"

Why do you think I should recreate the the boot archive in the dataset that was failing?

Even if it's already booting ok?

(And how could I do it?)

`bootadm update-archive` creates/updates the archive, but if it's outdated on boot we recreate it and reboot again (if we get far enough)

I didn't know about the bootadm tool.. What is the boot archive? Is it a Solaris/illumos specific thing?

it's (just about) what linux calls an initrd

Thanks, I figured.. There is still some illumos terminology I have to get used to

jbk: Thank you

?

what did i do? :)

thank for the answer to setup a slog for all SSD device :)

ahh ok