02:32:09 tsoome_: oh, so we and lld do what it expects, but binutils doesn't now, and won't? 02:32:41 despite both of us having implemented it based on what gnu used to do 04:20:29 [illumos-gate] 16317 SHA2Update() is wrong for 512 MiB or bigger blocks -- Bill Sommerfeld 11:56:53 It was fixed by illumos 13970 (fenix), so 'startstop.patch' is no longer required for building illumos. 11:56:54 BUG 13970: loader: BIOS loader ld script needs to use KEEP statement with linker sets (Closed) 11:56:54 ↳ https://www.illumos.org/issues/13970 12:05:03 I think it's been long enough now that we can drop that patch in omnios - I don't think we're building any gate versions from 2021 any more. 15:20:14 hi all, i have panic at nvme with latest illumos updates 15:20:15 https://paste.dilos.org/?c88aa53247bfefa0#Te7I6JHzc2RWi41zhnYkVasMoKODH/CuHhs0CZtOPQk= 15:20:22 1 panic[cpu1]/thread=fffffe000aaebc20: programming error: invalid NS/format in cmd fffffe065806ee00 15:20:50 it is panic under vmware 6.7 with nvme controller 15:28:23 With VMware's virtualized controller or passthrough? 15:31:52 So, uh, that's an identify namespace command with nsid 1 set. So at first glance seems pretty legal? What's the state of the namespace? 16:01:07 rmustacc: added NVMe virtual controller to vm 16:01:48 no drives attached to this controller yet, one drive attached to SATA virtual hba 16:10:38 @andyf: Thanks for the info I removed the patch from my commit again. 16:10:58 rmustacc: no panic if i have one nvme attached to controller. 16:11:25 i have panic if i have just NVMe virtual controller without attached drives 16:49:27 I guess we'll need to look at the virtual controller state then to understand why asking for this is crashing. Is it actually advertizing a namespace being present and active? 17:16:41 it wouldn't surprise me if VMware is perhaps playing loose with the NVMe standards 17:17:45 IIRC, it advertises trim support (whatever the NVMe term is.. every protocol uses a different name), but then returns an error if you actually issue the request 17:18:31 ... though I wonder now if maybe instead on VMware we should do write zeros 17:18:37 that's what you have to do with virtual scsi disls 17:18:40 err disks 17:19:38 (which reminds me, I need to upstream that here at some point) 18:00:10 basically on vmware virtual disks, it uses WRITE SAME to write out 0s instead of UNMAP 18:00:25 which will actually shrink a thin provisioned image 18:01:09 because vmware doesn't support unmap for virtual disks 18:03:06 while zfs can compress an all-zero block to a null blockpointer.. 18:03:29 this is in lieu of issuing a SCSI UNMAP command 18:03:32 so below zfs 18:03:39 (for VMware disks) 18:04:06 so you can zpool trim a pool on vmware and shrink the vmdk size 18:04:45 Beats having to shut down the VM and use the hole punch thing from the ESX CLI 18:04:58 Or maybe both are necessary? 18:05:49 i mean, internally, when i issued the ioctl to a virtual disk, i could see the size of the vmdk shrink without any other intervention 18:05:54 so I don't think so 18:06:25 it doesn't help there's actually two different ways to unmap in the scsi standard 18:06:35 (because of course why not) 18:06:40 there's the actual UNMAP op 18:06:52 but you can also set a bit in a WRITE SAME op to UNMAP 18:08:01 Things may have improved since I last used VMWare in earnest, which was mostly 5.5 18:08:58 i just need to pull out a few bits -- mostly because I don't think they'd be wanted upstream (for disks that support absolutely nothing of unmap/etc., it resorts to WRITE SAME w/ a block of zeros to basically erase those areas of a disk... which turned out to be useful for other stuff for us, but probably not something that'd be useful more generally 18:09:09 andyf: they haven't 18:09:35 their SCSI support is still pretty shit IMO 18:09:50 and for some reason, i suspect that's unlikely to change any time soon :) 18:09:56 just call it a hunch 18:10:42 :) 18:16:10 jbk: vaguely related in terms of thin provisioning -- I've wondered for some how hard it would be to get swap to use UNMAP or equivalent. 18:34:19 ISTR some OSes (can't remember if its Linux or FreeBSD) attempt to trim the swap device when it's added 18:35:02 though I guess we'd probably want to make it an additional flag to swap -- since people could still be using the swap device as a dump device as well 18:55:22 in terms of while the system is running, that I don't know -- i mean it just needs to call ldi_ioctl(), but haven't looked at teh swap code too closely to know how problematic that might be in practice 18:58:58 rmustacc: i have no ideas and it was working before latest refactoring of nvme driver 19:20:23 [illumos-gate] 16020 Borrowed abds ignore metadata flag during allocation -- Jason King 19:37:10 good evening. asuming you are a nvidia graphics driver for solaris, which kernel function would you use to reserve memory to copy your vbios into? I dtraced on :cmn_err:entry and printed the stack(), however the deepest call I see is from :nvidia:nv_printf:entry - and this is pretty sure not the call that produces the error... 19:42:01 If I were such adriver, I might call one of the functions in https://github.com/illumos/illumos-gate/blob/master/usr/src/uts/common/sys/gfx_private.h since they were specifically created for me and me alone to use 19:42:17 but I don't know what the actual driver does 19:42:53 alanc: thanks for the pointer, I have so no clue about driver programming I just poke around and hope to touch a nerve :) 19:43:45 and I was just reminded by the comment in the file that while it was originally created for the nvidia driver it was later extended to use by the drm graphics drivers as well 19:44:28 ah! I'll browse through that in a minute 19:49:09 jbk: one of the things I've noticed is that the swap block allocator is a clock algorithm, which means that, over time, your originally-sparsely-provisioned swap vdev gets fully populated. 19:50:13 trim on swapoff/shutdown would also be good for transient vm's (no need to retain swap contents while the thing is off). 20:01:00 sommerfeld, that used to be classically well known that Solaris swap was the only *nix whose swap worked well 20:03:51 sommerfeld, you should check out Solaris Internals 2nd edition, it really hammers home how much it is a highly engineered BoeingAirbus, compared to Linux (which I still love Linux) 20:07:07 Every class of hypervisor still has to implement all the core kernel components to share the machine. As good as Xen is now, it never made sense to me to discard all the hard earned knowledge 20:07:37 alanc: after reading throught that I would suspect gfxp_alloc_kernel_space() being called, would you agree? 20:08:42 copec: I'm not sure what your point is. TRIM basically didn't exist in the timeframe you're talking about. Illumos should evolve to use new features of both physical and virtual hardware. 20:09:10 sorry, I tangent-ed off of the solar swap 20:09:56 I agree, although modern SSDs internal allocation seem to manage just fine with or without TRIM 20:10:01 but it takes a size_t as an argument, and I should be no problem if you ask for x +size_t, you get it. maybe something in that function 20:11:44 I have many SATA SSDs in SAS enclosures that don't use TRIM and I compare their performance to the same disks in enclosures with SATA, and the performance difference after some time is indistinguishable 20:12:51 There are all "enterprise" level disks though 20:14:26 what's more, illumos by default swaps on a zvol rather than a dedicated slice of physical disk; a "trim" (via the ldi_ioctl mentioned above) would free blocks to the pool for reuse even if there was no TRIM going to the physical hardware. 20:21:48 Yeah, TRIM in that regard makes total sense. The swap zvol shouldn't be sparse though, and at least for me that zvol is a drop in every zpool I have, so it doesn't really make the difference in terms of a zpool getting towards full and slowing allocations 20:23:35 ok, it's not fbt:gfx_private:gfxp_alloc_kernel_space 20:30:55 sommerfeld: it looks like maybe if we added an ldi handle to struct swapent (and allocated a flag to incidate if trim should be done), we could maybe issue the ioctl in swap_phys_free() 20:34:21 ok, different approach. https://paste.ec/paste/eIj27fAE#4cWppNTJaVyAcqLy6W0wB3MkGJSc6dnyCsyL4WuuVjl shows a call to nvidia`_nv001009rm, however dtrace -l doesn't find that one 20:34:37 jbk: yeah, but can you do I/O from where swap_phys_free is called without creating bigger problems? 20:34:44 I tought fbt are generic probes? 20:35:41 IIRC the fbt provider will decline to create a probe for a function if the prologue or epilogue are not what it's expecting. 20:36:57 sommerfeld: what does that mean? 21:12:27 Agnar: the function prologue is the set of instructions that first execute on entry to a function that do things like set up the stack frame, save registers, etc.; epilogue undoes that on the way out. 21:13:00 fbt has some assumptions about the structure of the prologue and if it's not shaped the way it expects it doesn't create fbt probes for that function. 21:13:17 ah,understood 21:13:43 so for "private" functions in closed source code, this could happen and we have no clue of course. 21:14:48 yes, there are compiler options that affect how the prologue is generated and if you're going for performance above all other considerations you probably will step outside the line of what dtrace can cope with. 21:15:55 which makes sense for a graphics driver 21:17:20 heck, I need to sleep...thanks so far for your help, alanc and sommerfeld 21:25:12 if you disassemble that function, we should be able to tell whether dtrace would like it 21:25:22 if I remember, the most important part is that it begins by pushing a frame 21:34:53 if I remember correctly, the nvidia drivers are cross-compiled using gcc on Linux, and thus likely without the stack-frame patches in the illlumos gcc 21:35:08 sommerfeld: good question, unfortunately comments don't really answer that.. an alternative would be more complicated 21:37:47 e.g. you'd probably need to track both 'needs trim' and 'free/ready for reuse' so you're not holding the swapinfo_lock while trimming (since that'd block allocation/freeing) 21:39:31 (side note, it looks like the swap code is using it's own homegrown bitmap implementation to track free/in use... wonder if it'd be nicer to convert it to use sys/bitmap.h 22:26:41 copec: It's actually fine for the swap zvol to be sparse, FWIW. 22:27:18 You can make a reservation for it, obviously, to make it thick 22:27:43 But it is not required. And pre-zeroing it does not improve performance; if anything it makes it worse because of COW. 22:28:27 I think that's what sommerfeld mentioned, they can be sparse but algorithmically stop being sparse as fast as possible 22:30:46 jclulow, I mean you'll also have other problems at that point, but what happens if the zpool fills while the sparse zvol can't allocate? 22:31:10 You can't page more things out 22:31:23 It is not ideal, but it is not necessarily immediately fatal 22:31:31 you'd probably want a reservation, at least. 22:31:57 I mean I think you want _some_ reservation. But the tension created by never-overcommit + fork() leads to some hard bargains 22:32:46 I can't believe it's the future, and fork v. vfork still matters 22:32:54 memory big, software bigger 22:33:15 I mean really we should posix_spawn(3C) wherever we can -- and probably look at moving it into the kernel 22:33:46 Is anyone here using Tribblix? would love some pointers with setting up wifi (wificonfig) 22:34:09 yeah, if no one else gets to it first, it's something I'd like to look at (though seems like there's a lot of pieces to get right).. IIUC Solaris made it a separate syscall at some point 22:34:46 pfr: I don't see ptribble here right now. Is "wificonfig" a Tribblix tool? 22:34:56 Oh, no, that's part of the base OS 22:35:02 I can't imagine it is Tribblix specific 22:35:18 jclulow: not sure tbh. 22:35:44 I have read the man page but I cannot figure out how or where to add my password 22:36:24 'wificonfig connect' doesn't have a password flag 22:37:12 pfr: I fear you may be in for some disappointment 22:37:16 wificonfig is the base OS, but I think old 22:37:49 pfr: I would actually look at the dladm page 22:37:58 I think you want dladm *-wifi 22:38:00 yeah 22:38:11 I didn't find wpa_supplicant in any of the tribblix overlays 22:38:14 and we need to work out why wificonfig is (still?) there? 22:38:21 richlowe: Yes I am unclear. 22:38:43 pfr: The details are managed for you by "dladm *-wifi" I believe 22:39:17 richlowe: yes, I also looked at dladm, but was also unclear how to connect. The tool looks pretty comprehensive for complex networks. I just need to connect to a private home network 22:39:33 You would use "dladm connect-wifi" 22:39:52 and I'll presumably be prompted for the SSID and password? 22:40:12 Sorry, I'm at work procrastinating so not on my illumos machine rn 22:40:26 I think to stick a password in there you want to create a secobj, so "dladm create-secobj" 22:40:47 e.g., see "Example 3 Creating a WiFi Key" and then "Example 4 Connecting to a Specified Encrypted WiFi Link" I guess 22:40:51 in dladm(8) 22:40:54 thanks. I'll look into that 22:41:46 igork: Yeah, I get it. Just the more you can provide me, the more I can try to help. I don't understand how that's currently an illegal request without more context about the state of the controller. 22:41:50 It does look like "dladm create-secobj -c wpa somenameforkey" would prompt for the password yeah 22:42:35 [illumos-gate] 16324 Want Micron 7300, 7400, 7450 log support -- Robert Mustacchi 22:42:36 [illumos-gate] 16325 nvmeadm: want ability to write log page to raw file -- Robert Mustacchi 22:42:36 [illumos-gate] 16326 Update NVMe error status codes for 2.x -- Robert Mustacchi 22:42:36 [illumos-gate] 16327 wdc nvme assertion clearing support -- Robert Mustacchi