first step of course is to configure all the relevant ethernet switches to do jumboframes. (one of these days I'll try that at home..)

don't try this at home xD

Thankfully in this case I only have one relevant switch. :)

but ... but... that's where I tried it!

was part of a successful but very constrained jumbo(ish)frames deployment at a past employer. we ended up picking 4K+headers as the MTU rather than anything larger due to some odd hardware constraints on the rx side..

(let's just say it was an exception to jbk's "cards support this just fine" assertion above)

i meant i40e specifically

(and that it can handle a packet being split across multiple DMA descriptors for both TX and RX)

Does illumos have any system calls like FreeBSD posix_fallocate that do _not_ set  the carry flag when there is an error?

paulf: I don't know that we make any promises one way or the other about that?

I just fixed a bug in Valgrind on FreeBSD where I was using SET_STATUS_Failure() in the posix_fallocate and posix_fadvise syscall wrappers if the fd is invalid.

 That was wrong as it will cause the carry bit to be set. The libc wrapper will then return -1 and set errno. These 2 syscalls don't use the carty flag so the retrun value is either 0 for OK or an error code, errno not set.

I'd like to check if there are any similar syscalls on illumos.

It seems like our posix_fallocate() also returns an errno rather than setting errno

Which makes sense I guess as it's a POSIX thing

Probably but I don't think that posix_fallocate is a syscall on illumos, and the return code is handled by libc

Yes

We don't really make any promises about actual system calls

Sure, but in Valgrind we need to work at the syscall level a lot

I guess I would look at usr/src/uts/common/os/sysent.c and usr/src/uts/i86pc/ml/syscall_asm_amd64.S in the kernel and usr/src/lib/libc/amd64/sys/syscall.S for the usermode bit

There is also usr/src/lib/libc/amd64/inc/SYS.h which mentions the carry bit around SYSCERROR

It does seem like there are some SYSCALL_NOERROR* macros for a handful of system calls

e.g., lib/libc/common/sys/getuid.S etc

paulf: Hopefully that helps! I have to get to bed

thanks for the pointers, I'll have a poke around the source

hrm.. it would be helpful to have versions of pci_{put,get}{b,w,l}_func that use MMIO (the pci_cfgacc_acc does, but not those on x86)...

(I'm wondering if it'd make sense to use the ones from oxide -- seems like they might work better than just dereferencing a pointer

maybe with a tunable escape hatch to fallback to io space

since we do use mmio for the pci_cfgacc_acc calls (unless io space is specifially requested), so I'm thinking it's probably low risk since we're already doing it at least some of the time (when supported)

jbk: I wouldn't add a new version of them. Just let the impl fallback to it.

That's what we did on the Oxide arch, fwiw.

I don't think we really need a tunable here.

My intent was at some point to basically just change things to prefer it like we do and then when it's not there, use the existing mechanism.

github.com/oxidecomputer/illumos-ga…sr/src/uts/oxide/os/pcie_cfgspace.c I'm not seeing where they fallback -- am I looking at the wrong bits?

We don't fallback on Oxide becuse we don't support anything pre-Zen.

oh you meant, if mcfg is there, just mmio and don't fallback?

I just meant that i86pc I imagined we would want to add a new mechanism and not have new top-level functions.

But it's been a few years since I wrote that code, so I don't have as clear a picture any more.

yeah, i meant new implemetations to plug into those function pointers