rmustacc: thank you, that was helpful. I need to get more familiar with the existing data structures, that will probably make it easier to navigate and poke around.

hrm.. this is maybe a dumb question, but are there any sort of PCI bus stats

e.g. if you wanted to see utilization or saturation

Was buying a server with 12tb drives, they ran out of 4kn and want to sell me 512e

Anything I need to lookout for when creating my zpool with 512e?

512e ?

doesn't the 'e' mean emulated? (i.e. it's likely 4kn under the hood?)

do you know what brand

some manufacturers actually have ways to switch between 4k/512e (destructive, so obviousy do before putting any data on it)

HGST/WD 3.5" 12TB SAS 12Gb/s 7.2K 256M 0F29560 4kn ISE P3 and HGST/WD 3.5" 12TB SAS 12Gb/s 7.2K 256M 0F29530 512e ISE P3

I think we should use physical sector size by default, but you may want to check this to be sure

[illumos-gate] 15908 upgrade nfs/server properties server_versmax and server_versmin -- Toomas Soome <tsoome⊙mc>

I'm looking into adding `sendfile` support for Zig, and I just want to get my history straight.

AFAIK sendfile/sendfilev was introduced in Solaris 8, based on the system call manpages I found.

Whereas Wikipedia puts the Linux implementation in "Linux 2.2", but I don't know if it was there at the start or it was introduced later on.

If it was, that would mean that Linux's implementation predates Solaris' by at least 3 years (1999 vs 2002).

What I'm getting at is: if Solaris came later, why does the implementation differ so much? The biggest problem I have is that Solaris will send EINVAL and return -1 for incomplete writes, but Linux/BSD will at least return what was written.

It also means that I can't easily emulate the BSD behaviour of "send everything if len == 0".

from sendfile(3C): The sendfile() function was added to Oracle Solaris in the Solaris 9 release.

it is good thing they did add HISTORY section;)

Yay the server_vermin/max change landed!

Thanks for working on that tsoome

toasterking Interestingly enough, sendfilev(3C) says that it was added to Solaris 8 Update 5 in July 2001.

sjorge now it is time to add knobs to 15405, then we can put it back

it might have been 'backported'

Smithx10: i think zfs is fine with using a smaller block size in a pool created with a larger disk

it's the opposite that's usually the problem

jbk: Presuming you mean PCIe and not PCI, I don't know of the processor really exposing that kind of information in any architectural way.

robertdfrench: I sometimes find it useful to use the -t option to print in mdb instead of flipping back and forth to code. Example ::print -t kthread_t

obviously you wont see the code comments etc that are always useful but to quickly dump out what a struct looks like, it's pretty useful

[illumos-gate] 15938 amdzen_df_rome_ids check needs to be >= -- Robert Mustacchi <rm⊙fo>

rmustacc: once an mblk_t has been handed downstack (i.e. TX) to mac, can mac assume it has exclusive control of that mblk_t (at least until it passes it downstack)?

What does exclusive mean to you?

Are you just wanting to manipulate the mblk_t fields, the dblk, something else?

well ideally the dblk.. though i guess thinking about it, that could be shared, couldn't it?

Yes, the dblk has a db_ref on it.

What do you want to modify?

I wouldn't expect any need to modify the dblk in tx.

(Or even really the mblk_t fields).

But pretty much every tcp dblk is going to have a ref of 2 at least because the stack wants to hold onto the dblk until it gets an ack.

as i'm digging into some network stuff, I keep encountering these arguably pathological TSO mblk_ts -- a single packet, rife with long strings of tiny, large, tiny, large, ... and in a lot of the cases the previous mblk_t has more than enough space to append the data

I would look at that at TCP then and not in mac.

The sizing is based on the combo of what are the actual socket write systemcalls and then window sizing.

The bcopy there may simplify things, but honestly I would expect most cards to be pretty good at the sl.

*sgl

Are you running out of descriptors or just doing a pull up too much there or?

it's often leading to pullups in the driver because of segment limits

since i think just about every driver will consolidate runs of small into one descriptor (and copy)

i40e I'm assuming?

It has a rather low segment limit.

But I do think that this is something you want to look at above mac rather than in the driver.

(Ed McMahon impression) you are correct!

That is why is TCP carving up the mblks that way.

Yeah, i40e's I/O engine with only 8 SGL entries is pretty pessimal.

I think the question I'd basically ask is when we're getting a chain built up on a socket and sending, at what point do we consolidate?

i thought ice was similar (started looking at the datasheet for that since there's talk of maybe finishing your driver so we can use it)

Dunno. Just thinking outloud.

Yes, ice has the exact same I/O constraints.

That is the one part that is the same.

For better and worse.

though i'm wondering if other nics might be a better fit

Depends on what the surrounding situation and what your options are.

at least if we want to look to take advtange of any offloading beyond TSO + checksum offloading

That is, is it a situation where you pick the NIC or where someone else picks it.

In terms of other stateless offloads, that'd depend on what traffic you're doing.

(where we have a choice)

We will be looking at things around encapsulation offloads, but otherwise there's not a ton IIRC.

this would be SMB, NFS, HTTP[S].. so all the virtualization / hosting VM bits (e.g. effectively having a whole hw switch in the NIC) aren't very useful

i mean they don't hurt, but they don't really help either

I mean there are things we could do potentially around fast-pathing a ring/group filtered to traffic for some of those, but otherwise I don't think there's a lot there.