richlowe: got the *scanf checking enabled as well (much less noise from that).

Errm: openzfs/zfs #14756/files#diff-539674efbce9f51b3f6b2b9c18ecaace3de7999a5c4d40e28d0a1db69bc6f3f2R237

This is normal practice on Illumos installs, is it not?

Is this deadlock mentioned a thing on Illumos?

The FAQ itself seems to indicate that it might be a more Linux specific issue

people have been doing swap on zfs on illumos (and solaris prior to that) for decades

i've not heard of any problems

doesn't mean there may not be any, but if there are, they seem to be rare

I feel like there is an intrinsically greater change of deadlock than using a raw device.

In the case that pages are being ejected to swap because the system is short of memory..

writing a page to a swap device backed by ZFS is going to require more memory to do

It's why I often see people setting ZFS options like `logbias=throughput` on swap zvols

We have definitely seen deadlocks where the system is so short of memory that it cannot find memory to eject pages to disk

and it's why jclulow added a memory reserve just for this

Or, more precisely, made ZFS use the existing `pageout_reserve` pool. (sorry, misremembered)

illumos 13092 (fenix)

BUG 13092: ZFS I/O pipeline should use the pageout_reserve pool (Closed)

↳ illumos.org/issues/13092 | code.illumos.org/c/illumos-gate/+/891

though generally if you're paging to swap, you've probably already lost because drives are so much slower these days

I mean yes it makes sense, going through the DMU layer is going to have some memory costs to it, but there are some pretty clear advantages, too

Is that a reference to that strange DISKRPM constant? I often see systems ejecting pages to swap when they need to reclaim memory - usually when I accidentally build too many copies of clang in parallel though

it's definitely an uh oh situation but it's at least a recoverable one sometimes

(although maaaaybe with some of the newer NVMe devices it might actually make more sense again to use it again)

at the very least if it's on a pool, besides the obvious integrity benefits, you get the striped bandwidth from it

You can easily encrypt it too, without having to involve lofi

(fairly easily)

however helpful that bandwidth really is for a half dozen 4k pages that are somewhat disjoint, I dunno

A downside is that you can share a raw swap device with the system dump device, but not a zvol.

Can we trust ZFS encryption yet? The issues page on OpenZFS for encryption related problems make me very nervous about it

not from a cryptographic standpoint but from a "swallowed my data and can't recover" standpoint

I am not aware of any issues, MNX probably have most experience using it.

There's a problematic incompatibility with openzfs though, which I don't think is resolved yet.

yeah i need some time to finish fixing that

I have the core bit of the fix done, it's just wiring up all the pieces to control the behavior

that i haven't finished yet

A lot of the issues revolve around send and receive but there are others as well that make me nervous

14166, 13521, 13491, 12614, 11679 to name a few

yeah, send/recv problems were what delayed encryption support in illumos for quite a while

unfortunately, it was 'well it's not happening to me, so i don't care about it'

until i finally made it happen on ubuntu

[illumos-gate] 15600 SMB2_read Length=0 assertion panic -- Gordon Ross <gordon.ross⊙tc>

another thing about zfs swap is that illumos uses a clock-type allocator -- all pages in the swap device get written, so the device ceases to be sparse after one lap of the clock.

wouldn't be a problem *if* the VM system knew how to trim/overwrite freed pages with zeros.

for swap devices, I wonder if sync=disabled would make any difference (since nothing on the swap device is useful after a crash..). Maybe the primarycache/secondarycache should be set to "metadata".

tsoome_: is 15608 suppose to include a change with actually disables the referenced warnings/gags, or is that coming later?

(if later, it would be good to note that in the ticket)

sommerfeld - github.com/omniosorg/kayak/blob/master/lib/disk_help.sh#L158-L166 is what omnios does fwiw

hunh. sync=always ?

Yes, interesting

pmooney yes, once we have last one (rtld - because it is one really fragile bit) done, we can drop sgs "global" gags. I'll add note in issue, thanks.

sounds good

tsoome_: If you haven't already, it might make sense to file that "central" ticket (where you intend to actually drop the global gags) and associate these related tickets

That would leave a nice trail for anyone looking at it (now, or in the future)

hm, yes, thats good idea

pmooney: I've heard that sort of thing called an "umbrella" bug/ticket/...

sommerfeld: maybe as a first approximation i don't think it'd be too difficult to try to issue a TRIM for the entire LBA range of a swap device when added

won't help as much if it's getting used a lot

but at least it would start out as empty

in a similar vein, I believe at least on FreeBSD, when adding a disk to a zpool, it also tries to trim the entire range of the device

(of course there are some that at least on enterprise disks, TRIM should never be necessary)

err some that believe

[illumos-gate] 15608 link_audit: clean up warning gags -- Toomas Soome <tsoome⊙mc>

I've always seen the overprovisioning amounts based more around meeting an expected lifetime vs. guaranteeing a certain I/O rate

so *shrug8

Yeah. ssd is more likely to make better wear-levelling choices when it doesn't think it has to preserve data we don't care about.

sommerfeld: thanks for noticing make -w! life changing.

both for compilation-mode, and also for sneaky nightly.log grepping.

??

make -w is definitely awesome.

Don't make us read the source, what does it do?

Lists the directory that you're entering.

`man make` is all you would need to do. Anyway, it prints the current working directory before and after processing, which means if you need to rerun a command for debugging purposes, you no longer need to jump through hoops to figure out what the current working directory is.

I did not see -w in the man page

That needs to be fixed.

that's what sommerfeld is fixing, in fact.

Yes. I also have some patches to nightly and bldenv that make them play nicer with make -w (bldenv: let you set MAKEFLAGS in your env file. nightly: don't interpret the entering/leaving messages as errors and filter them out from stuff that lands in the mail_msg file)

man page fix out for review: code.illumos.org/c/illumos-gate/+/2850

→ CODE REVIEW 2850: 15654 Document make -w (NEW) | illumos.org/issues/15654

ahh nice