00:22:35 rmustacc: thank you, that was helpful. I need to get more familiar with the existing data structures, that will probably make it easier to navigate and poke around. 01:09:00 hrm.. this is maybe a dumb question, but are there any sort of PCI bus stats 01:10:38 e.g. if you wanted to see utilization or saturation 01:41:15 Was buying a server with 12tb drives, they ran out of 4kn and want to sell me 512e 01:41:26 Anything I need to lookout for when creating my zpool with 512e? 04:37:22 512e ? 04:37:45 doesn't the 'e' mean emulated? (i.e. it's likely 4kn under the hood?) 04:37:58 do you know what brand 04:38:24 some manufacturers actually have ways to switch between 4k/512e (destructive, so obviousy do before putting any data on it) 05:35:49 HGST/WD 3.5" 12TB SAS 12Gb/s 7.2K 256M 0F29560 4kn ISE P3 and HGST/WD 3.5" 12TB SAS 12Gb/s 7.2K 256M 0F29530 512e ISE P3 05:43:56 I think we should use physical sector size by default, but you may want to check this to be sure 06:07:14 [illumos-gate] 15908 upgrade nfs/server properties server_versmax and server_versmin -- Toomas Soome 06:34:36 I'm looking into adding `sendfile` support for Zig, and I just want to get my history straight. 06:35:03 AFAIK sendfile/sendfilev was introduced in Solaris 8, based on the system call manpages I found. 06:35:51 Whereas Wikipedia puts the Linux implementation in "Linux 2.2", but I don't know if it was there at the start or it was introduced later on. 06:37:24 If it was, that would mean that Linux's implementation predates Solaris' by at least 3 years (1999 vs 2002). 06:42:29 What I'm getting at is: if Solaris came later, why does the implementation differ so much? The biggest problem I have is that Solaris will send EINVAL and return -1 for incomplete writes, but Linux/BSD will at least return what was written. 06:46:45 It also means that I can't easily emulate the BSD behaviour of "send everything if len == 0". 07:00:37 from sendfile(3C): The sendfile() function was added to Oracle Solaris in the Solaris 9 release. 07:01:07 it is good thing they did add HISTORY section;) 08:25:16 Yay the server_vermin/max change landed! 08:25:24 Thanks for working on that tsoome 10:02:25 toasterking Interestingly enough, sendfilev(3C) says that it was added to Solaris 8 Update 5 in July 2001. 10:16:57 sjorge now it is time to add knobs to 15405, then we can put it back 14:00:45 it might have been 'backported' 14:02:51 Smithx10: i think zfs is fine with using a smaller block size in a pool created with a larger disk 14:03:24 it's the opposite that's usually the problem 14:54:19 jbk: Presuming you mean PCIe and not PCI, I don't know of the processor really exposing that kind of information in any architectural way. 18:14:17 robertdfrench: I sometimes find it useful to use the -t option to print in mdb instead of flipping back and forth to code. Example ::print -t kthread_t 18:14:42 obviously you wont see the code comments etc that are always useful but to quickly dump out what a struct looks like, it's pretty useful 19:17:50 [illumos-gate] 15938 amdzen_df_rome_ids check needs to be >= -- Robert Mustacchi 23:32:27 rmustacc: once an mblk_t has been handed downstack (i.e. TX) to mac, can mac assume it has exclusive control of that mblk_t (at least until it passes it downstack)? 23:35:51 What does exclusive mean to you? 23:36:18 Are you just wanting to manipulate the mblk_t fields, the dblk, something else? 23:37:33 well ideally the dblk.. though i guess thinking about it, that could be shared, couldn't it? 23:37:41 Yes, the dblk has a db_ref on it. 23:37:44 What do you want to modify? 23:38:00 I wouldn't expect any need to modify the dblk in tx. 23:38:13 (Or even really the mblk_t fields). 23:39:10 But pretty much every tcp dblk is going to have a ref of 2 at least because the stack wants to hold onto the dblk until it gets an ack. 23:41:08 as i'm digging into some network stuff, I keep encountering these arguably pathological TSO mblk_ts -- a single packet, rife with long strings of tiny, large, tiny, large, ... and in a lot of the cases the previous mblk_t has more than enough space to append the data 23:42:11 I would look at that at TCP then and not in mac. 23:43:04 The sizing is based on the combo of what are the actual socket write systemcalls and then window sizing. 23:43:44 The bcopy there may simplify things, but honestly I would expect most cards to be pretty good at the sl. 23:43:47 *sgl 23:44:07 Are you running out of descriptors or just doing a pull up too much there or? 23:44:58 it's often leading to pullups in the driver because of segment limits 23:45:31 since i think just about every driver will consolidate runs of small into one descriptor (and copy) 23:45:55 i40e I'm assuming? 23:46:01 It has a rather low segment limit. 23:46:19 But I do think that this is something you want to look at above mac rather than in the driver. 23:46:25 (Ed McMahon impression) you are correct! 23:46:30 That is why is TCP carving up the mblks that way. 23:46:39 Yeah, i40e's I/O engine with only 8 SGL entries is pretty pessimal. 23:47:37 I think the question I'd basically ask is when we're getting a chain built up on a socket and sending, at what point do we consolidate? 23:47:40 i thought ice was similar (started looking at the datasheet for that since there's talk of maybe finishing your driver so we can use it) 23:47:43 Dunno. Just thinking outloud. 23:47:54 Yes, ice has the exact same I/O constraints. 23:47:58 That is the one part that is the same. 23:48:18 For better and worse. 23:48:21 though i'm wondering if other nics might be a better fit 23:48:37 Depends on what the surrounding situation and what your options are. 23:48:46 at least if we want to look to take advtange of any offloading beyond TSO + checksum offloading 23:48:48 That is, is it a situation where you pick the NIC or where someone else picks it. 23:49:01 In terms of other stateless offloads, that'd depend on what traffic you're doing. 23:49:02 (where we have a choice) 23:49:22 We will be looking at things around encapsulation offloads, but otherwise there's not a ton IIRC. 23:50:36 this would be SMB, NFS, HTTP[S].. so all the virtualization / hosting VM bits (e.g. effectively having a whole hw switch in the NIC) aren't very useful 23:51:40 i mean they don't hurt, but they don't really help either 23:53:52 I mean there are things we could do potentially around fast-pathing a ring/group filtered to traffic for some of those, but otherwise I don't think there's a lot there.