First try the usual suspects, MTU and what-not.

Also a usual suspect, what PI are you running?

Depending on the guest OS, also look at the CC algorithm. We use (new?) reno, linux use cubic.

I forgot what freebsd uses, but none of it was matching up at my HDC, swapped it all to cubic seems to have improved things.

Hello all, question about network performance, I have machine run SmartOS with 10G NIC, and run a bhyve vm with vNIC on the 10G phy NIC, however, it work only 1G, may I know if there anything is incorrect? here is the details pastebin.com/btiEeVYz

danmcd: mtu 9000, @sjorge bhyve guest is cubic

hello! I'm running a VM on SmartOS which has a 600GB volume, but it is occupying around 1.4TB in ZFS. I already checked, and it isn't creating any kind of snapshot so I don't know exactly what is happening and if I can reclaim the disk space. does anyone here have an idea what this might be?

Guest32 (LIBERA-IRC): Is this on a raidz pool?

yeah on a raidz3 pool

Then you're probably experiencing something like: openzfs/zfs #548#issuecomment-3791251

TL;DR the volblocksize is probably 8K which leads to many padding blocks

The last comment in that issue has a summary of various comments on the topic. One possible solution could be creating a new volume with volblocksize=32K for that raidz3 pool

alright, thank you for the information. I will take a look

tozhu: Don't see the MTU anywhere in your gist.

mtu is default, I think it’s 1500

let me check

@Smithx10 of course you're running the `mlxcx` branch of illumos-joyent build.

danmcd: the mtu is 1500 on both machine

I don’t think mtu is root cause that the 10G NIC work on 1G

maybe some limit with smartos or bhyve

So wait are you same-machine NFS-ing?

(Re-read this.)

(I re-read your gist, I mean.)

danmcd: us-east-storage.solutions.iqvia.com/bruce_dev/public/iperf3.jpeg.

just running iperf3 from native zone getting the same perf as in GZ at 15gbps but bhyve is about 1/3 the speed at 5gbps. All iperf3 tests are going from Illumos CN -> Switch -> Linux Storage Server. All with the same hardware.

danmcd: yes, the Bhyve VM is created on the host which is run SmartOS and nfs server by zfs create … -o sharenfs xxx ; Bhyve VM is running CentOS 7 with nfs client;

So tozhu you might be getting bitten by illumos.org/issues/15464 and illumos.org/issues/13463

Your packets are never going out on the wire

is there any workaround to slove it?

or have to wait for the fix?

@tozhu read the bug reports.

There's an etc/system (or even `mdb -kw`) variable IIRC.

sjorge can give you more details.

hi

thanks for smartos!

I need advice with ipnat.conf ...

I am on a single IP and want to nat an FTP server from a zone ...

the man page has something like

"port" port range port

I've tried lots of things ...

rt" port range port

what exactly does that mean

rdr e1000g0 from any to 144.76.69.252 port 65001-65100 -> 10.0.5.81

rdr e1000g0 from any to 144.76.69.252 port 65001:65100 -> 10.0.581

rdr e1000g0 from any to 144.76.69.252 port 65001 range 65100 -> 10.0.5.81

rdr e1000g0 from any to 144.76.69.252 port 65001 ... 65100 -> 10.0.5.81

I'm simply not getting it

what does the range between the port and port mean ?

Port ranges are colon-separated low:high ranges of ports. E.g. `6001:65535`.

A generic outbound NAT rule, e.g.:

map igb1 10.19.84.0/23 -> X.Y.Z.NN/32 portmap tcp/udp 6001:65535

(That's a made-up example entry for a line in /etc/ipf/ipnat.conf)

FTP gets tricky because of it's wideranging port usage.

(Same with NFS prior to NFSv4.)

This was nice blog and page. Maybe could be bring ip again if asked.. web.archive.org/web/20220703031731/…//timboudreau.com/blog/smartos/read

thanks, I'll try that

danmcd: having sender and receive aligned CC algorithm wise just gave me more consistent performance

i just double checked and ended with cubic and ecn active everywhere (host, zone, freebsd bhyve firewall/router, and linux vms)

danmcd: Thank you

danmcd: I’m going to use a new image to resolve this issue, Thank you