00:54:42 ok 05:11:37 is there currently a recommended way of setting zfs ashift parameter for SSDs or is it done automagically? 05:13:02 for advanced format HDDs I've set ashift=12 just to make sure 05:14:28 looking at a few SSD I probably didn't set it, "zpool get ashift mypool" shows that ashift=0 and zdb shows =9 05:35:17 holin: It depends on what the SSD firmware tells us about the block size of the device 05:35:48 Some SSDs correctly say they support a physical block size (e.g., 4K) that is distinct from the logical block size (e.g., 512B) 05:35:53 And some just say the logical size 05:36:25 does illumos have any tools to get that? smartctl shows 512/512 for all (SATA) SSDs that I have 05:39:10 You know, I actually don't remember how to fish it out 05:41:06 Ultimately if you absolutely want a 4k (=12) pool, you should probably specify it 05:41:15 But we will under many conditions do the right thing 05:43:25 Frankly, I've never cared about this on SSDs, thinking that trying to too clever on the user's side will likely just go against the SSD firmware trying to also be clever, but now I got some new disks and thought about trying to be clever nonetheless :-) 05:43:53 If they are NVMe disks, you may be able to reconfigure/"reformat" them to be 4k native 05:44:02 Which might allow them to perform better, depending on how smart the firmware is 05:44:45 See the "format" subcommand in nvmeadm(8) 05:45:07 haha.. i managed to make reboot(1M) segfault somehow 05:47:17 actually.. ficlVmInnerLoop is crashing (called by be_get_boot_args) 05:54:52 one other thing regarding SSDs: does setting autotrim make sense? On linux I have usually a cron job doing manual TRIM once a week.. 05:56:20 I think trim has generally been found to improve performance on consumer grade SSDs, even on some enterprise SSDs that supposedly don't need it 05:56:38 It's pretty deep firmware magic though, so it could also make things worse 05:57:02 If you care about the performance you'll probably need to do experiments with your particular workload 05:57:09 and your particular drives and firmware revisions 05:58:25 it also wouldn't be surprising if it also depended on how full your SSD is 05:58:55 for the drives at hand, longevity is more important than performance. These are consumer grade SSDs fwiw. 06:01:54 btw. does illumos have the NCQ TRIM exceptions for Samsung EVO 850/860 drives that Linux has? 06:02:30 have some of those around, too, and might end up using them for illumos deployments 11:56:23 jbk core? 13:50:21 what could possibly make a disk not seen by illumos? Two SSDs behind mpt2sas controller and linux sees them both, while illumos (oi) only sees one according to "format" and also /dev/rdsk 13:50:36 identical, unformatted (unpartitioned) SSDs 13:51:01 previously the same bays/slots had two other disks which showed up fine 13:52:14 i'm thinking that not having partition table has something to do with this.. 13:59:54 holin - do they show up in the output of `diskinfo`? 14:00:39 no, same thing as format 14:01:06 i could try partitioning in linux and see how that goes 14:02:51 diskinfo shouldn't care about the partition table - any difference if you run it a second time? 14:02:58 Another command that shows disks is `iostat -En` 14:05:15 iostat shows the same thing also 14:06:08 one'd think that if it was the disks (incompatible etc.), neither would show up; if it was the controller, the previous two disks wouldn't have worked either 14:06:12 That's strange. It definitely sounds like it's worth doing what you suggested 14:12:48 d'oh. gave them gpt disklabels with parted and still no dice 14:14:11 my second guess: the disks differ only by the last three digits in their serial# so maybe whatever in illumos generated the disk id's uses too few digits and gets confused(?) 14:14:39 (i.e. considers the disks the same) 14:15:13 the previous, also identical, disks had a bigger difference in serial# 14:17:18 the ports on the motherboard get names like c3t1d0 c3t2d0 c3t3d0 and so forth 14:18:15 these disk behind the mpt2sas controller all start with c4t3 juxtaposed with something generated from the disk identification info 14:19:41 so, it would at least appear possible to have collisions in device ids 14:30:05 Here's an interesting discovery: With MTU=9000 on all of: a vnic on an aggr over two mlxcx physical interfaces, something above mac_ring_tx is dividing outbound messages into 1460 byte segments. That means mlxcx always copies here: https://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/io/mlxcx/mlxcx_ring.c?r=7fdea60d#2268 20:57:20 [illumos-gate] 15016 find_elf should cope with O_DIRECTORY absence -- Yuri Pankov 21:32:57 gwr: a question for dan, or dtrace, or both? 22:16:49 gwr: Is it a local LAN, or is it possible there has been some path MTU detection keeping it down