Hello everyone,

I am looking for a way to connect a server to the internet via PPPoE.

Is it still possible on OmniOSce/OpenIndiana?

I haven't found how to do it so far.

The only workaround I could find is to use netcat and ppp to simulate a PPPoE connection... it's not really fantastic, I think :-D

majekla: maybe that's helpful gist.github.com/jperkin/7717d3e84e93885ab14da3bce3039f4b?

Wow.. thank you very much ! I'm gonna try this immediately

It seems that it will not work because the pppoec lib he is using isn't in any packages available

$ pkg search /usr/lib/inet/pppoec

INDEX ACTION VALUE PACKAGE

path file usr/lib/inet/pppoec pkg:/system/network/ppp/tunnel⊙0

Great ! Now it will be easier. Just to know, when I pkg search /usr/lib/inet/pppoec, I don't get this result, I get nothing... I'm on the last r151050 version

Thank you very much

it works for me when i try it on the latest r151050, maybe a `pkg refresh --full` does help on your end?

yes it works with a full refresh now. Thank you. It's strange, I've just installed omnios on a vm, shouldn't this be automatic after the first pkg update ? Anyway, thank you very much for your help. I'm gonna try to connect to my ISP now ?

Wow I used to use the PPPoE stuff maybe 15+ years ago haha

Thank the Bell-Heads for PPPoE....

jclulow, I'm pretty sure my ISP is still using it. I just don't feel like asking my firewall right now. It would just be depressing.

speaking of depressing, I've been forced to face the fact that my fileservers have much slower network connectivitiy than they should.

What sort of speeds are you getting/expecting?

getting

[ 5] 0.00-10.00 sec 3.35 GBytes 2.88 Gbits/sec sender

[ 5] 0.00-10.00 sec 3.35 GBytes 2.88 Gbits/sec receiver

expecting

[ 5] 0.00-10.00 sec 10.8 GBytes 9.28 Gbits/sec 8 sender

[ 5] 0.00-10.04 sec 10.8 GBytes 9.25 Gbits/sec receiver

the first number is a test between two OmniOS boxes with dual 10G (LACP)

the second number is a test between two Linux hosts, one a virt, with single 10G connections.

all connections go via the same single Juniper switch.

forgot the header:

[ ID] Interval Transfer Bandwidth

those are all iperf3 using defaults.

nomad, what NICs?

ik5pvx, we've got a mix. i40e, ixgbe for the most part.

nomad: single TCP connection should only use one path (if it's being load-spread across multiple paths, that could result in packet reordering with hurts performance.

so LACP *could* be (part of) the problem?

could be. at the very least it's an apples vs oranges difference.

: || lvd@hvfs1 ~ [505] ; dladm

LINK CLASS MTU STATE BRIDGE OVER

i40e0 phys 1500 up -- --

i40e1 phys 1500 up -- --

aggr0 aggr 1500 up -- i40e0,i40e1

ixgbe0 phys 1500 up -- --

ixgbe1 phys 1500 up -- --

aggr_nas0 aggr 1500 up -- ixgbe0,ixgbe1

what does dladm show-aggr show?

LINK POLICY ADDRPOLICY LACPACTIVITY LACPTIMER FLAGS

aggr0 L4 auto active short -----

aggr_nas0 L4 auto active short -----

what load-balancing is configured on the switch end?

IIRC, none.

It's a juniper. Let me go see if I can find the config for it.

(I think it's more likely to be some need for tuning in our TCP stack than switch being wrong, but never hurts to check)

should be some sort of L4 load-balancing policy.

some switches can be configured to do round-robin which is dumb

I am so very much not a network admin. I had Juniper help me with the initial config of this switch so hopefully we got it right.

set interfaces ae2.0 family ethernet-switching vlan members default 

delete interfaces xe-0/0/44 

delete interfaces xe-0/0/45 

delete protocols rstp interface xe-0/0/44 

delete protocols rstp interface xe-0/0/45 

set interfaces xe-0/0/44 ether-options 802.3ad ae2 

set interfaces xe-0/0/45 ether-options 802.3ad ae2 

set interfaces ae2 apply-groups LACP 

set interfaces ae2 apply-groups JUMBO_FRAMES 

I'm a bit concerned right now as I can't seem to ssh into the switch to get the actual config. Those are my notes about how we configured the ports.

whew, I was able to get in.

yep, config matches my notes.

unclear if this is relevant to you but google found: juniper.net/documentation/us/en/sof…tches-interface-load-balancing.html

This can't possibly be right:

set groups LACP interfaces <ae*> aggregated-ether-options lacp active

set groups LACP interfaces <ae*> aggregated-ether-options lacp periodic fast

aren't active and periodic contradictory?

no that's fine

lacp can go slow or fast in its detection of faulty links

fast is good unless you're dealing with an arista switch

ah, ok.

as for the load balancing, there could be a knob under forwarding-options, but I'm rusty on juniper

sommerfeld, is there something I can check about the tuning while I'm poking at this?

default should be some l2-l3 hashing

set forwarding-options storm-control-profiles default all

that's the only forwarding-options hit I get in the config file.

mine only has this related loadbal: set load-balance-label-capability

I don't think you need that

We have expensive vendor support. I suspect it's time for me to use that.

hah

yes

I have a cheap ex3300-48p off ebay

nomad: setting tcp connection_control to cubic and boosting the tcp send_buf/max_buf/recv_buf will likely help somewhat.

where are these things done in illumos?

(all this via ipadm {set,show}-prop)

nice, thanks

argh, I of course mean "congestion_control"

sommerfeld, these hosts are fileservers. Their only reason to exist is to serve NFS or SMB packets.

Will these changes negatively impact those services?

nfsv4 and smb both run over TCP. if the changes improve TCP performance it will likely help them be better fileservers.

that's my theory but I am required to ask these questions :)

and are likely to be neutral for nfs-over-UDP

looks like cubic is already set in ipadm

tcp congestion_control rw sunreno -- sunreno sunreno,newreno,cubic

no, it's at the default. last column is "POSSIBLE"

ah

well, that host is a dev/test host so I can make changes there.

change one, watch it closely for a while, revert on suspicion of badness, spread to others if it's better.

do you ever get close to maxing out the lan?

I'm changing that one prop on my two test/dev boxes and will re-run the iperf3 tests between them now.

ik5pvx, not with these fileservers, no. I do with my Linux hosts.

well, setting them both to cubic did not change the numbers.

on linux there's also that other policy.... bbn? can't remember. Supposedly it's the most efficient, but can be a little aggressive with other traffic

[ ID] Interval Transfer Bandwidth

[ 5] 0.00-10.00 sec 3.06 GBytes 2.63 Gbits/sec sender

[ 5] 0.00-10.00 sec 3.06 GBytes 2.63 Gbits/sec receiver

my linux hosts don't have LACP enabled, just the file servers.

so cubic didn't seem to have helped directly.

bbr is the name

: || lvd@fs2 ~ [526] ; ipadm show-prop | grep tcp | grep buf

tcp max_buf rw 1048576 -- 1048576 8192-1073741824

tcp recv_buf rw 128000 -- 128000 2048-1048576

tcp send_buf rw 49152 -- 49152 4096-1048576

that's what it is now. What would you suggest I bump it to?

BBR is good, but we don't have it illumos (yet?)

I doubled the current numbers and tested again...no change in the results.

set-prop diddn't ask for an interface but reset-prop does.

no, that's just me being stupid. Never mind me.

I've reset congestion_control, max_buf, recv_buf, and send_buf to default on both hosts.

I guess tomorrow I'll get to spend quality time with Juniper TAC.

good luck