is there a way to like... reboot into a read-only mode?

if I've set all the datasets in the zfs pool to `readonly` is it theoretically safe to dd if=/dev/dsk/<disk> of=wherever-really and expect the resulting data in wherever-really to be reasonably consistent?

MelMalik: what's your goal? backup? replicating a running system?

and do you want to do this to the root/boot pool or to a data pool?

this is about the root pool...

this is probably a case of "you shouldn't want to do that" is it

there are other things you could do that would be less disruptive and more likely to produce the desired result.

(if we knew what your actual desired result is, that is..)

the desired result is to have a copy of the disk somewhere else, in a state that is reasonably consistent; the resulting disruption from having everything be readonly is, if uneasily, acceptable.

zfs snapshot -r rpool@snapshot_N ; zfs send -R rpool@snapshot_N | ssh elsewhere zfs receive someotherpool

and you can do that online where the only disruption to the running workload is added I/O contention ...

that does not take the partition table with me.

i guess I'm not really asking for alternatives, but more of a "will what I plan on doing have the intended effect or will the result on the other end be unusable"

well, after the dd, you can import the resulting pool and run zpool scrub and know if it will be usable by the end of the scrub.

right

gotcha. think i'll do that then

another option would be to add the other disk as a mirror and then use "zpool split"

(assuming you're starting with a single-disk or mirrored rpool)

if that were practical

anyway, so, "and we're off" - I'm going to make a coffee, or probably go to bed, and see how things end up going

update: that was a disaster, but what, I suppose, did I really expect

I guess I'll have to reboot the machine into something not running off the disk, then

How going the making of r151054? It supposed to get released in a couple of days...

MelMalik : idk if the zfs retention feature is available on Illumos, but if yes it can be interesting for you: c0t0d0s0.org/blog/preventaccidentaldestruction.html

nothing was destroyed on the source machine, it was just unreadable on the target for whatever reason

copying as described with ssh but by using a live cd running freebsd worked in the end

szilard - r151054 is on track for release on the 5th of May (first Monday in May)

Great!

This will be my first revision-update, I will see, how hard it is!

is realtek rtl8125 2.5GbE NIC supported?

it's not listed on illumos.org/hcl so I guess not.

It is not currently, sorry.

Let me buy another NIC. Thanks.

If you *need* 2.5Gbit, get an Intel I225 or I226 that supports 2.5Gbit, or if you have deeper pockets, get an Intel 10GBaseT that also supports 5 and 2.5 like the X550 or X710. (If you see an X722 DO NOT GET IT for illumos#13230 reasons.)

danmcd: for 13230 the patch you posted looked good though, so theres hope, right? :D

It needs SO MUCH TESTING but folks who've used it have noticed no more dups. My BIG concern is that it might break on other i40e parts, but NGL, 13230 is what the Linux i40e appears to use. The test matrix to confirm 13230 is quite large, unfortunately

I have wondered if 13230 should only be applied (using device PCI ID and media information) to X722 using BaseT only.

Hi ^

danmcd: do you have adequate testing for your draft fix for illumos.org/issues/13230 on the X710? ("pciex8086,15ff"); I've got a system with a couple of them..

I ran it a while ago on X710-Base-T-equipped `curly` in Kebecloud, but it was a while ago.

I didn't see any problem, but I didn't exercise the crap out of it with, say, multicast. `curly` has both BHYVE and multiple-native zone types.

The draft fix is tiny: kebe.com/~danmcd/webrevs/13230-newtry

You can see how (relatively) much if ifdef'ed out in that.

I also worry about if, say, gz uses `i40e0` but there are vnics on top of it too, what happens if GZ does something, will it affect the VNICs? ANd vice-versa (that is what 13230 is all about at its beginning).

MelMalik: Thought of one more thing you could try if you still want to do things the hard way - a "zpool checkpoint" prior to the copy might make the copy more recoverable.

what I ended up doing, with the dd method but using a different OS in a livecd, ended up working

yeah, that would avoid updates to the disk while the copy is in flight giving you a consistent snapshot

Sorry, I need some help to understand this: FreeBSD writes at almost 500 MBytes/s on a ZFS pool built with six Seagate Ironwolf 8TB SATA2. On the same machine, OmniOS writes at 160 KBytes/s... how can I identify where the problem lies?

wow, I just discovered that the cause lies in the way I test write speed: I'm running the command 'dd if=/dev/random of=/export/test.im bs=1204 count=1024'... it looks like that reading from /dev/random is around 120 KByte/s on this system with OmniOS!! :S

warden: the other thing to check is how the data is being written (or not) you may also find that data is being buffered in ram so initially the speeds look good.

thanks, anyone can tell me why /dev/random is so slow in generating random data compared to FreeBSD on the same hardware?

from memory when i did some testing like that many years ago that tested write speeds I had a script that would create some example type 'files' in ram and then write the to disk with random dirpaths/names for a sustained period (so that you're trying to write several times the ram capacity to disk) and watch the write speed.

check how random the sources are and what they do if there's not enough entropy. I don't know FreeBSD or Illumous for that but...

... on Linux I know you have /dev/random and /dev/urandom. I forget which way around but one i truely random and will block if there's not enough entropy

the will give pseudo random data but if there's not enough entropy it'll keep going but the quality of the randomness may drop.

m1ari: thanks! In the mean time I did a test similar to yours: I saved a big file generated by FreeBSD's /dev/random in an OmniOS's ramdisk, then I run 'time cp ...', writing to the same ZFS pool. The result is comparable with FreeBSD performance: around 480 MByte/s! :)

a freshly booted system may not have as much entropy as something that's beem running a while, there may also be daemons that can help provide entropy (again something I've looked at more on Linux than other systems)

how are your drives configured in the ZFS pool ?

warden: the original distinction between /dev/random and /dev/urandom was that the former was stronger "real" randomness and the latter was fast "pseudo" randomness. the distinction turns out to be harder to nail down than you'd like from a mathematical basis

but solaris and derivatives continue to make the distinction

You should use /dev/urandom for most things and save /dev/random for those times when you're generating a public key you want to last for the next decade...

I created the pool with this command: 'zpool create -O atime=off -O compression=on export raidz c3t0d0 c3t1d0 c3t2d0 c3t3d0 c3t4d0 c3t5d0'

s/#illumos/#omnios/g

I wonder if you've got some ram caching going on as well, 500MBytes/s from spinning rust in raidz sounds quite fast (It looks like those drives also have a lot of cache)

for the records, I'm using this SATA controller: ASM1166 (asmedia.com.tw/product/45aYq54sP8Qh7WH8/58dYQ8bxZ4UR9wG5), which I discovered not behaving properly at hotplug... you can only hot-swapping disks connected since boot time, and plugged only on channel lower than 5 (tested also with FreeBSD)! :S

m1ari: yes, the disks I'm using have 256 MB of cache

mlari: sequential write to raidz should perform quite well (should approach N-1 disks worth of sequential performance)

I stand corrected, it's been a while since I had to do much drive performance stuff, and havn't got to play with zfs at that scale as much (too many people still seem to prefer Linux)

warden: depening on your data you might also want to consider raidz2 rather than raidz. Especially if the drives are all about the same age. You may find that if one fails another may go shortly after and you wouldn't want that during resilvering.

mlari: replicating your experiments on my system: /dev/urandom rate limits me to ~64MB/s. file to file goes more like 500-1000MB/s as reported by dd for short durations; watch the disks with iostat and you'll see each one going at ~50-150MB/s for a bit.

did run into this oddity from dd: 268435456 bytes (256 MiB) transferred in 0.250209 secs (??iB/sec)

aha, NN_NUMBUF_SZ is one character too small for that value.

268435456 in 0.250209s is ~1023MB/s

m1ari: I’ve a bit of experience with ZFS (max. 30 TB pools until now), and I definitely prefer FreeBSD to Linux speaking about ZFS. Getting my first steps on illumos because I feel that OpenZFS is getting too “Linuxed”! ;)

warden: bad news: illumos.org links to openzfs :p