i'm trying to learn how to use freebsd as a wireguard server to torrent through. i can connect and pass traffic but something feels like not all traffic is going through. i'm using pf.conf to nat traffic out from wg client and rdr traffic in to wg client. paste.debian.net/hidden/551bbd71 anything i can do better pls?

demido: do you want all traffic on that system to go through the vpn, or only torrent traffic?

well originally i just wanted the torrent traffic but now i'm trying anything, so i was basically passing ALL traffic through to wg client except the wg connection itself, and ssh

kinda desperate to figure it out hehe

demido: okay, well rather than using pf nat, i'd suggest using separate fibs (routing tables). for all traffic, set the fib 0 default route to the wg interface and configure the tunnels in fib 1, or for only torrent traffic, set the fib 1 default route to the wg interface and start the torrent client in fib 1 (but you need to be careful of leaking DNS traffic here)

ok can i ask eli5 why do "that"?

it's just neater - rather than messing around with NAT to make it work, you're just telling the kernel to do what you actually want, i.e. route traffic X via the vpn and traffic Y via the local router

also probably more reliable and easier to understand once you have it working :-)

ah ok so my pf is kinda a hack and maybe abusing pf

how do we work with fibs/routing tables?

so first off set net.fibs=1 and net.add_addr_allfibs=1 in /etc/sysctl.conf, then run "/etc/rc.d/sysctl start". now you have two routing tables, which you can view with "netstat -rn -F0" and "netstat -rn -F1"

er, net.fibs=2, sorry!

should net.add_addr_allfibs=2 too? or 1

no, 1 is fine for that

should i disable pf before doing this change?

if it says net.fibs is read only you need to set it in /boot/loader.conf instead, but i think it's been a dynamic tunable for ages

i would turn off your pf nat rules, you don't need to disable pf entirely if you have filtering

no filtering atm. want to get a solid connection first, then i'll experiment with filtering

ok i'll make those changes sec

I'm using FreeBSD since version 4.something, and I never heard of fibs (probably not knowing that I needed them :-) )

demido: actually, after you set that i suggest rebooting to make sure add_addr_allfibs takes effect (i think this is only applied when interfaces come up initially)

ivy i have net.inet.ip.forwarding=1 in sysctl.conf, should i remove that? i read it was needed for the wg natting

that's not necessary for this setup but it won't hurt either

i'll remove. k gonna reboot box sec

also, do you have IPv6 here or only IPv4? and is the internet connection doing anything weird like DHCP or is it just a static config?

i didn't configure ipv6. don't know if it's automatic somehow?

just static conf

when reboot finishes i'll run netstat -rn -F0 and netstat -rn -F1 to look

ok i looked at routing table and routing table (fib: 1)

they look pretty similar at this point

yeah, they should be mostly identical except you won't have a default route in fib 1

so i guess to start with it's easier to just send all the traffic through wireguard

ok yes

all except the ssh into the wg server, and the wg client connection to the wg server which would create a loop, right?

first off you need to duplicate your current default route in fib 1, so for example "route add -inet default 1.2.3.4 -fib 1" (replace 1.2.3.4 with your default router), then make sure you can "setfib 1 ping freebsd.org"

yeah, the idea is to put wireguard itself in fib 1, which is similar to how linux does this with network namespaces

ahhhh

(and probably sshd too)

i ssh to wg server separately from the wg connection from wg client fyi

oh, do you have a serial / vnc console to access the server in case something goes wrong? this is not the type of config i like to do with only ssh access :-)

ok so that route command will add default route to fib1, the wg connection fib, so that the wg connection has a way out to the inet using the vpn's static ipv4 ip?

ya i do hehe

right, so to make it use that you need to put the wireguard interface in fib 1: "ifconfig wg0 tunnelfib 1"

is it possible to put "route add -inet default 1.2.3.4 -fib 1" into a file like rc.conf so it's permanent? i'm afraid to just type it, then forget later what all i did

(there are two ifconfig options here, 'fib' would put the encapsulated traffic into fib 1, 'tunnelfib' puts the wireguard traffic itself into fib 1, which is what you want)

demido: yes, we'll set this up in /etc/rc.conf in a sec, if it works :-)

ah ok great! sec i'll type

so the rc.conf option for the route is: static_routes="vpn"; route_vpn="default 1.2.3.4 -fib 1" (except use a newline instead of ';'), then you need to add tunnelfib to the ifconfig line, so something like ifconfig_wg0="<other stuff> tunnelfib 1"

i ran sudo route add -inet default 1.2.3.4 -fib 1, then sudo ifconfig wg0 tunnelfib 1

i don't have an ifconfig_wg0. i have ifconfig_vtnet0..., then i have wireguard_enable="YES" and wireguard_interfaces="wg0"

is that a new line to add?

ah, you're using wireguard from ports, hmm

yep

it has config file in /usr/local/etc/wireguard/wg0.conf

there's a way to run a command in wg.conf for that, i think? in which case you'd want it to run 'ifconfig wg0 tunnelfib 1'

looks like PostUp=...

sec

brb

i still add static_routes="vpn"; route_vpn="default 1.2.3.4 -fib 1" to rc.conf right?

(on 2 lines)

ok i put those 2 lines into rc.conf, and the PostUp line into wg0.conf under [Interface]

yeah, that's right

they match the sudo commands

do i reboot now?

yes, reboot and make sure everything looks right

ok ssh back in gonna run netstat -rn -F0 and netstat -rn -F1 now

ok fib: 1 has default destination now

which is the static ip of the server and its physical if

well, "physical" (vtnet0)

is that all? connect from wg client and it should be good?

the route should be via the ip address of your default router, not the server's ip address... whatever's currently in defaultrouter="" in /etc/rc.conf

oh ok i'll fix that then reboot again sorry

same for the vpn route in rc.conf, that should be the same address as the default route

that's the only place i had to fix the ip

rebooting

ok now both fibs have default route, and gateway is the same for both, the router ip

now should wg client be connected?

yeah, wg should work now - after it comes up, make sure 'ifconfig wg0' shows 'tunnelfib 1'

on the wg server right?

wg0 has tunnelfib: 1!

is the machine you're working on now the client or the server? (i assumed this is the machine you're running the BT client on, or no?)

while wg client reboots.. how does wg server know that traffic coming to it should get redirected to the wg client EXCEPT wg connections from the client, and ssh connections from 'any'?

no i have 3

this 1, the vpn server is a vps in a shitty host, then the 3rd box is on my lan for torrenting

it's the wg client, the 3rd box

ah ok

i ssh from this computer into both

wg server and client

how does wg server know that traffic coming to it should get redirected to the wg client EXCEPT wg connections from the client, and ssh connections from 'any'?

hm, this may actually be an unnecessarily complicated configuration for what you're trying to do :-)

did we mess up?

i was thinking you wanted traffic from this machine to go over wireguard, but you want this machine to just be a router for another wg client, right?

ya

i want wg server to take connection from wg client, and basically pass ALL traffic through to/from wg client, as if wg client was the wg server

except ssh connections to wg server, and wg connections to wg server from client

it's so my isp doesn't see me running torrent traffic in/out

lots of udp and other p2p stuff

hmm, in that case you don't need the fib stuff (sorry!) but you might still need the static route to route traffic over the wireguard tunnel... or are you doing that in wireguard config right now? iirc, wg-quick can add a route for AllowedIPs automatically, but in this case you'd want AllowedIPS to be 0.0.0.0/0

btw wg client can ping wg server's private ip, but can't ping www.freebsd.org. wg client's wg0.conf has allowedips 0.0.0.0/0 for [peer]

not doing any route stuff in wg on server or client

only setting address, dns, private key, allowed ips, endpoint, publickey, and keepalive on client

wg server wg0.conf has address, listen port, postup we added, privatekey, allowedips is wg private ip, publickey

what should we do?

ok, so try this: remove the static route and fib config from rc.conf and the PostUp from wg config and reboot then, then in the server's wg.conf, you want something like "PostUp=/sbin/route add 192.16.0.10/32 wg0" ... i *think* AllowedIPs set to the client IP (192.168.0.10) there should be fine

although wg-quick may do this automatically, i'm not sure - if you start the wg tunnel without PostUp, do you see a route for 192.168.0.10 in netstat -rn?

lemme reset. on wg server commented out postup. now going to comment out 2 fib lines from sysctl, then comment out static_routes and route_vpn from rc.conf. then reboot. good?

yep

don't put the PostUp with /sbin/route in yet, see what the routing table looks like first

i know wg-quick has some magic here, i just don't remember what it is exactly :-)

yep, rebooting then will answer your q

yea lol

ok rebooted, on wg server, netstat -rn: 192.168.0.10 is in Internet: section, link#4 under gateway, UHS flags, wg0 netif

ok, make sure ip.forwarding is 1 still, put back your pf rule for 'nat on vtnet0', but not the rdr, just to test, then run 'tcpdump -ni wg0' on the server and try to connect to the internet from the client... you should at least see incoming traffic from the client

(i'm assuming the client is not freebsd)

k sec. no client is debian

ok forwarding is back in sysctl.conf. nat on vtnet0 from $wgclientip to any -> vtnet0, is back in /etc/pf.conf. reboot now?

(then tcpdump)

you can probably just /etc/rc.d/sysctl start; /etc/rc.d/pf reload

ok sec

on the client, just ping an IP address or something to avoid any other issues like DNS

ok on wg client i ping www.freebsd.org ip, and i see in wg server tcpdump it go through

oh not ntp went through too hehe

wg client got reply too fwiw

s/not/now

you're making a mistake here: wireguard doesn't know about server/client, only peers. And traffic goes from peer to peer, not 'through peers'. If you want a peer to route traffic for you to somewhere else than the peer, you need to NAT the traffic out of there

Afterglow: we're back to their original pf setup which has an outgoing nat for the wg peer

that was my pf.conf

(and yes, 'client' and 'server' is technically not correct terminology for pf, but... people do tend to use it :-)

s/for pf/for wg/

demido: so is the client internet working now (via wg)?

so can i make my pf.conf better?

lemme test

ya i can curl an iso file

and do dns queries etc

that's odd, since i think we're back to your original config that didn't work... does it still work if you put the rdr rules back as well?

i didn't add the rdr nordr stuff back btw. only the nat

do i even need them?

i mean i guess so

i do want the wg client to be 'as if' it is the wg server, in the datacenter and not in my home behind my isp

depends, if you want to receive incoming traffic then yes, although my preference would be to just rdr the specific ports you need

well the wg client worked before as far as i could tell, it just didn't seem to get full bandwidth like it did from the server itself

(actually, my preference would be to use an entirely separate public IP address for incoming traffic, but i know that can be hard to do at cheap vps providers)

so i felt like i must not be forwarding everything bidirectional

ya

maybe the problem was i wasn't redirecting everything, but only >1024?

does anything <1024 make udp bandwidth better?

demido: are you downloading and uploading at the same time? because it's all going over vtnet0, your incoming traffic is duplicated as outgoing traffic and vice versa

yea

sorry i meant port 1024

(like say you have 100Mbps on the server, and the client is downloading something at 50Mbps, the server is sending 50Mbps of data out already, so the client would only be able to upload at 50Mbps...)

yea

what seems to happen is wg client doesn't establish as many p2p connections as wg server itself gets. for the same files

so it just felt like the proxying had to be flawe

d

but you guys can't see a prob in my pf.conf so it must be ok?

Afterglow you agree with ivy that paste.debian.net/hidden/551bbd71 can't be improved?

ivy - true for half duplex, but I think full duplex through a switch can get you close to full bandwidth. On a hub, probably not.

demido, I'm not very specialized with pf, but I'll take a look

tyvm

it's mostly/completely udp traffic going through fwiw

but you see i redirected even tcp

is there anything in that pf.conf that doesn't keep established connections alive? i guess that doesn't even make sense in context of udp

find anything?

nope

ok so it looks good then?

Hi, Is it possible to read the handbook for older versions? Like imagine I want to use the handbook for FreeBSD 5

I found it anyway docs-archive.freebsd.org/doc/5.5-RE…/doc/en_US.ISO8859-1/books/handbook

that was posted in the wrong forum

does anyone here running FreeBSD on macbook air 2012 and have get touchpad to work? and can share configuration for xorg.conf.d/touchpad.conf ?

got it working now, forgot to add moused_port="wsp0" and moused_enable="YES" :)

how can I find the service listening on a certain port (tcp4 882) when sockstat gives me question marks on USER, COMMAND, PID and FD?

k

are you sure it is listening?

does it appear in sockstat -sl

tcp4 0 0 192.168.13.16.882 *.* LISTEN

This is what netstat gives me

and what does sockstat give you?

? ? ? ? tcp4 192.168.13.16:882 *:*

if I telnet into it, it sets up the connection

with -ls?

? ? ? ? tcp4 192.168.13.16:882 *:* LISTEN

cool, I have no idea

I'm now tcpdumping it to see if anyone connects. This machine exposes nothing to the internet, btw

Afterglow: do you have lsof installed?

yup

Afterglow: sockstat -l and check suspicous PID

look at my sockstat output above: PID shows a questionmark

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS PATH STATE CONN STATE

? ? ? ? tcp4 192.168.13.16:882 *:* LISTEN

ridcully, trying `lsof -Pi4TCP | grep 882` doesn't show the process

ridcully, correction, doesn't show anything about port 882

To be complete: fstat -s | grep "tcp.*:882" isn't showing anything either

machine was rebooted yesterday, after upgrading to 14.2

Afterglow: the ? are specifically because there's no socket associated with it anymore

Found the culprit: after restarting lockd (nfs) the listening port vanished, and it opened another one

yeah that'd do it, too

Afterglow I had exactly the same finding today

have a nice run kevans

.oO unleash the dogs:)

kevans: when you're done with your run please fix bug 284857!

i can unleash dogs but one is a little dachsund/chihuahua thing and the other only has three legs

i can not run fast but i could probably outrun them

:)

oh, a a ferret dog? we have one of those here

it can't go on a walk with the other dogs because it can't walk fast enough

ivy: I've still no idea how to fix this, I guess melifaro might have some idea

maybe i should try to fix it, it feels like it *should* be easy

last time i tried to fix a wg(4) everything went very badly though

i wonder where your EAFNOSUPPORT is coming from, precisely

i believe it must be in wg_output():

uskerine_, hope you did find the perp too

if (parsed_af != af) {

xmit_err(ifp, m, NULL, AF_UNSPEC);

return (EAFNOSUPPORT);

}

i could add some debugging logs if that would help, though

ivy: tunnelfib: 1 ≠ route fib, what if tunnelfib = 0 ?

mzar: huh?

in your PR

which PR? 284857?

yes

i don't understand what you mean, "what if tunnelfib = 0" - but it's not, it's 1

OK, so set route with the same fib and use ping with the same fib and reproduce to make it consistent

huh? why? the route doesn't need to have the same fib

OK, if you think so

i know so, this is how tunnelfib works

cool

this is not a misconfiguration on my side, the problem is a missing feature in wg(4)

you have to submit a patch then

oh fuck off

i do not need to submit a patch just to report a bug

the bug exists, i reported it

obviously if i could fix it i would do, maybe i will have a go at that one day...

nice, but if you want to fix it, please submit patch

what is your point even

we always encourage to submit patches;)

:=D

next question, why can't i get Cy's patch in bug 284709 to work

for some reason the files that are meant to be in krb5-server keep ending up in krb5-libs

and the files that are meant to be in krb5-libs end up nowhere

okay, i think Cy's patch is wrong, this seems to fix it: llfw/freebsd-ports 2dc14ff

ivy: yeah, if you can track it down (or ship me a VM or jail config that can reproduce it in isolation? that seems like it'd be feasible) I'd be happy to look more into it

kevans: i can give you two commands right now that would reproduce it?

alright let me grab my notepad

3!hemlock ~# ifconfig wg1 create inet6 -auto_linklocal -ifdisabled fe80::1/64 up

4!hemlock ~# route add -inet 192.168.1.1 -inet6 fe80::2%wg1

add host 192.168.1.1: gateway fe80::2%wg1 fib 0

5!hemlock ~# ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1): 56 data bytes

ping: sendto: Address family not supported by protocol family

ping: sendto: Address family not supported by protocol family

you don't need a working wg peer since the bug is inside local packet processing

sorry, that was 3 commands, i lied :-)

ok fine

:-p

your guess about where it was erroring is infact correct

hmm, AFAICT that can likely just be removed and wg_xmit() will probably do the right thing; presumably your wg configuration would have IPv4 peers (we can have both IPv4 and IPv6 peers on one wg device, right?)

ivy: you want to test that with a valid setup? termbin.com/l8t6

kevans: i have no ipv4 peers at all, but i'm happy to test this

patch applied, building now...

hmm, that probably won't really function, then

Anyone else happen to be seeing strange issues with ROOT DNS servers right now?

ek: there are not thirteen but hundreds of root DNS server, so there's nothing to worry about

ivy: when I say 'ipv4 peers', I really mean one with some ipv4 allowed-ips which you'd need for the wg_xmit() lookup to work

kevans: does if (!(parsed_af == AF_INET || parsed_af == AF_INET6)) make any sense here ?

mzar: Yeah. The problem I'm seeing is that I'm no longer able to look anything up.

I have just built and tested it, works for me

mzar: no, we'll get that same effect in wg_xmit() anyways

OK, so kill it

kevans: oh right -- all my peers have allowedips 0.0.0.0/0, ::/0. but they run over ipv6 transport

haven't tested yet as build is still running

ivy: ok, so yeah- I think just removing that check entirely makes sense and will fix it, but I'll wait for confirmation before throwing it into a review

I have looked at tests, but we are not tesing ip6 for wireguard

kevans: in my case it worked, but I have changed to (!(parsed_af == AF_INET || parsed_af == AF_INET6)) what probably be always false

determine_af_and_pullup() will notably not return anything but one of the two or an error

so maybe a better fit as an assertion, but I'd be inclined to just let wg_xmit() complain

we can MFC it to 14 too

I have not spotted such a check for other tunneling interfaces

FreeBSD is working out for me

could have been lot better if we could have postgreSQL client in ports

Bushmaster: Not sure what you mean, but I see plenty of postgres clients available in ports.

ek, can you point me one

pgAdmin demand to work only with postgres older version and it removes my postgres17.4

hence i cant work with pgAdmin

do you have DBeaver in ports ek

Bushmaster: /usr/ports/databases/postgresql17-client ? Not what you're looking for?

Maybe our interpretations of "client" are different.

yes that is what I am looking for

you mean your interpretation

Bushmaster: there are some tricks you can try, likg locking the package, adding it by hand from file not repo, etc, you don't have to reubuild port in this case

s/liking/locking

mzar, have you tested it, i am kind of new to BSD

if you can help me to get pgAdmin install without it forcing me to remove postgre17 that would be BIG HELP

right now, if I do pkg install pgadmin it will just ask me to remove postgre17

that is not I am gonna do

if you can walk me from here, that would be big help

Bushmaster: read about locking packages and adding pkg by hand, read pkg(8), search for lock and add

not much help

dont't hesitate to do some experimentationg

experiment is for syadmin

i am not a sysadmin , I am Java Developer

you are probably one of devops ?

I'd keep quiet about that

ivy: thank you for fighting with bugs preventing IPv6-only deployment !

mzar, no, not devops, java developer

Bushmaster: please don't hesitate to play a bit with packages, and you will be fine, there is no need to rebuild port in this case

okay cool

lot to catch up on when it comes to sysadmin, but yeah, I will look into it

Bushmaster: pgadmin is an graphical user interface client.. client is just the console client

i really need a better build system than this Ryzen 2700X

hernan604, i do not see any other SQL client in port collections, DBeaver is good one and like to see in port

i mean you even have RKWard for R programming

Bushmaster: stop saying client. you want a GUI

why cant you integrate DBeaver

these are SQL client, GUI client yes

Bushmaster: there are plenty of clients for postgres ot mysql in ports/pkg

hernan604, can you find me one?

Bushmaster: maybe this docs.vultr.com/install-pgadmin-4-for-postgresql-on-freebsd-12-2

let me check

I am in FreeBSD 14.2

not in 12.2

you think it will work?

should be the same

or similar

but you can always try a VM... or install pgadmin inside a linux vm

check vm-bhyve to manage VMs

this is web client, i may go for it

its pgadmin4

ivy: I went to write a test for it, but I'm realizing my route(8) knowledge is nonexistent beyond the absolute most basic usage

ivy: going for the reverse situation because it's easier to just adapt an existing test: route add -inet6 2001:db8::/64 -inet 169.254.0.2 ->

add net 2001:db8::/64: gateway 169.254.0.2 fib 0: Invalid argument

that doesn't seem very invalid damnit

kevans: i have never tried adding an ipv6 route with an ipv4 nexthop, but are you sure 169.254.0.2 is directly reachable?

e.g. did you add 169.254.0.0/24 over a wg tunnel already

yeah, it's the other side of this tunnel

epair on the other side is 192.168.2.2, wg in the other jail has 169.254.0.2

it might be this simply doesn't work because no one would ever test it :-)

what does "route -n get 169.254.0.2" say?

hernan604, that will not work, cos I have updated python in FreeBSD14.2

seems happy and describes a route to 169.254.0.0 with mask 255.255.255.0 via wg0

and virtual machine option is absurd, why on earth I need to install Linux inside FreeBSD and then try to run pgAdmin from there

kevans: you should probably submit this as a bug :-)

you have pgAfmin3 in ports ... all you have to do is find out a way to disentangle it from older version of portsgres

so that it can work for any postgres version

or add DBeaver in ports

kevans: le-fay.org/tmp/7d/route.txt -- this is definitely a bug

probably a bug in netlink, that broke a lot of things

many of which have seen since fixed, but i doubt anyone is using ipv6 route with ipv4 nexthop

s/seen/since

noone but Juliusz Chroboczek

that's new. though

babel users gonna babel use

Bushmaster: then try building pgadmin from ports whith whatever version you want it to

hernan604, as I said I am not sysadmin

you said you have many SQL client, and then carried on talking 'crap' ... stay quiet my dude

lol

Bushmaster: you are lost. good luck

Bushmaster: i will be telling my friends about this java developer... that never used freebsd, and he said freebsd has no clients for postgres LOL... and he is looking for a postgre client... and then when i suggested him to manually build a pgamin port, he said "he is not a sysadmin" hahahhahahaha

welcome to the console my friend

to unix

you talking crap ... stay quiet

you are a troll

bye

Bushmaster: pgAdmin works with any posgres version, the problem is that it has to be build for a concrete version, so it is build against the 'default' version, so either switch stuff to the default versions in the official pkg repos, build your own pkgs with changed default versions or search for some other solution

nimaje, i understand , as I said I am not a sysadmin but yeah build your own pkgs with changed default versions is something I sure will able to do over time, I do not see you or anyone else can help me to get it down, if I wanna get it done, I sure can, just will take me time

it should not be build against the default version, it should be independent cos it is simply a postgrSQL client, which should work for any postgreSQL versions. Same as DBeaver, it can work with any SQL servers, so do SSMS which work with any Microsoft SQL Server versions

ivy: weirdly enough I reconfigured it to use your original scenario and I get a routing error instead of EAFNOSUPPORT, but `route get` clearly indicates it trying to route traffic to this IP via wg0

??

kevans: on 15.0?

yeah

though I'm a few months old

it's possible this actually worked a few months ago, i'm sure i used it before, i only noticed it was broken recently

ah, build finished, let's test your patch

kevans: bad news, it still doesn't work, same error

[2!] uk-myb-1 /# route add -inet 192.168.1.1/32 -inet6 fe80::2c8f:916:ed09:eae1%wg.uk-aai-1

add net 192.168.1.1: gateway fe80::2c8f:916:ed09:eae1%wg.uk-aai-1 fib 0

[3!] uk-myb-1 /# ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1): 56 data bytes

ping: sendto: Address family not supported by protocol family

wait

this is the wrong kernel

lmao i forgot to push

ha...

it worked for me, I applied on both ends

i cannot be held responsible for your shenanigans :-)

ivy: I am going to reply to your PR

ivy: I will submit a patch, which will probably be rejected ;-)

;-P

kevans: in my defense i'm working on like 5 PRs right now across src and ports

mzar: if it's just kevans' patch i assume he'll accept it? :-)

ha... that's different one

buildworld done (52 seconds), buildkernel done (25 seconds), now waiting for update-packages which is usually like 500 seconds

i wish we could find a way to make this faster

i just want this test case to work

i'm |-| here at posting a phab review, but I want to see a test case go vroom vroom firs.

first

i can't test this properly until i work out how to make OSPFv3 advertise IPv4 routes with IPv6 nexthop, which i'm honestly not sure is even possible with BIRD

but i can at least show you packets going over the tunnel

this routing table seems sane, right? termbin.com/lx9z -- in this one, 2001:db8:1::1 is the local wg1, 2001:db8:1::1 is the remote wg2, 192.168.3.1 is some address on the other side

... ::2 is the remote wg2

ping: sendto: No route to host

that looks wrong, 192.168.3.0/24 is via 2001:db8:1::1, but 2001:db8:1::1 is via lo0

perhaps you meant to add 192.168.3.0/24 via 2001:db8:1::2

ok, packages built, rebooting now

[3!] uk-myb-1 /# route add -inet 192.168.1.1/32 -inet6 fe80::2c8f:916:ed09:eae1%wg.uk-aai-1

add net 192.168.1.1: gateway fe80::2c8f:916:ed09:eae1%wg.uk-aai-1 fib 0

[4!] uk-myb-1 /# ping 192.168.1.1

PING 192.168.1.1 (192.168.1.1): 56 data bytes

[1!] uk-myb-1 /# tcpdump -ni wg.uk-aai-1 host 192.168.1.1

tcpdump: verbose output suppressed, use -v[v]... for full protocol decode

listening on wg.uk-aai-1, link-type NULL (BSD loopback), snapshot length 262144 bytes

17:39:51.238732 IP 81.187.47.201 > 192.168.1.1: ICMP echo request, id 58121, seq 0, length 64

that seems to be working

there, got the test to work

don't ask me how, but it does what I want it to and demonstrates the error but works after

that's probably another bug: link-local address not working over wg(4) interface

err - wrong window

but wg interfaces aren't really local, are they

yep

so ND resolution is rather not supposed to work here

anyway, the PR 284857 describes the problem with IPv4 routes with IPv6 next hops over wg intefrace, and proposed solution fixes this issue

mzar: link local addresses work fine over wg(4) though?

i have like 20 bgp/ospf peers over link local addresses on wg(4)

bird> show ospf neigh lf6

lf6:

Router ID Pri State DTime Interface Router IP

10.1.2.2 1 Full/PtP 35.866 wg.uk-myb-2 fe80::340c:19c6:1b3b:730b

81.187.47.194 1 Full/PtP 34.618 wg.uk-aai-1 fe80::2c8f:916:ed09:eae1

nice, so it's not broken

there's no NS/ND since wg(4) is a PtP interface that pretends not to be in a way i find unhelpful

but that's how wireguard works, i don't think freebsd can fix that

ivy: I have responed to your PR, since I have spent some time testing this issue, there is no any race with kevans's approach, just to confirm that it was broken

mzar: i saw your patch, i have no opinion on whether that or kevans' is preferable

huh what? pkg-static: krb5-1.21.3 conflicts with krb5-libs-1.21.3 (installs files into the same place). Problematic file: /usr/local/bin/compile_et

oh never mind, i know what's going on

well, sort of -- i'm not sure why jellyfin depends on krb5... i don't remember it supporting Kerberos authentication

I have one lingering problem with pf: the rules won't load on boot. bsd.network/web/@dvl/114082898655646261

dvl: what's the point of a bogons table?

ivy: Blocking networks which have addresses which are not assigned to anyone.

dvl: sure, but why?

ivy: ... because I can...?

Or perhaps I can.

but you can't :-P

if pf cooperates

i mean your question is valid, i'm just not sure this is a useful thing to do...

Not at boot, yet. This must be a solved problem.

FWIW, it is commonly done on appliances

less nowadays, i feel

it's not like the 90s when everyone took a bogons feed from team cymru

Ironically, that's the one I'm using.

yeah, i looked at doing that too but i couldn't see any benefit

*maybe* for a transit ISP, to avoid wasted bandwidth if nothing else

but for an end system? i don't think you gain anything

why is security@ discussing Bible passages?

dvl: what does "pfctl -sm" show ? have you tried to increase the limit ? how many addresses do you have ?

mzar: dpaste.org/VgJOM - I have not tried increasing the limit, mainly because it works after boot.

ivy: looks like a troll or maybe misdirected mail? seems like it's in reply to something but the thread it's in doesn't seem to match where they replied

this is new i.imgur.com/gmTDD7q.png

no, I found it. They seem to have been set off by something in someone's signature.

ah, looks like bugs.freebsd.org/bugzilla/show_bug.cgi?id=282605

dvl: it could be problem with: 1. table too small 2. pf cannot parse so large file (too many entries to be added at once)

if the table is too small, try to increase its size

mzar: I wonder why it fails at boot time. Let's see what I have now:

if there is a problem with processing large file, please split it

at boot time ?

PF i loaded later

s/i/is

mzar: The problem occurs when booting the host.

mzar: After it boots, I ssh in, run `pfctl -f /etc/pf.conf`, done.

ha..

mzar: I say boot, I mean during system startup.

anyway, you are close to this value, maybe it's worth increasing this limit ?

mzar: Agreed. I'll paste what I did.

mzar: dpaste.org/gzzRM

Next goal, reboot.

mr gman999

gman999: I thought about you one cold morning... it reminded me of you saying it was pretty safe in the neigborhood when it was real cold, not so much when it was real hot.

:)

unlikely i'l be at bsdcan this year

unfortunately

now there's a product: bsd in a can

located in a small cold city with good kebab

gman999: I will probably be there.

good.

patrick will be my proxy... as always

sends a flurry of 'hellos' when at a con

all we need to make that happen is FreeBSD running on a 2013 Mac Pro

gman999: You'll remember the year I won a NetGate appliance, 10 years ago... that unit will be up for auction this year. I am retiring it.

ha

yes... long time ago

oh BSDcan is in the USA

jbo: You are mistaken.

ha..

dvl, I think trump disagrees

i'll give ottawa the kebab point.

well, and other things

another nycbsdcon will be difficult in the near future...

we have to see at some point.

i dont have the time...

and we struggle to get speakers for regular meetings at this point.

we see though

gw reboot time /cc mzar

mzar: Still does not load on boot.

ha.. problem with parsing so large file, most likely

maybe you can split it into two and add some bogons later, as a workaround

wait you're parsing bogons for something dvl?

issue loading a bogon table or something?

gman999: yes, also tried moving the syctl into /boot/loader.conf, but nope. bsd.network/web/@dvl/114082898655646261

mzar: on boot, 1 - empty the file, 2 - start pf, 3 - load up the file

I'd rather do it in diffent way

pfctl -t bogons -T add -f complete_set

and at boot try smaller set

adding this to rc.local will work, but after reloading PF someone has to fix it, either the sysadmin or cron

anyway, duct tape fix

mzar: a cron job for @reboot

it's worth filling a PR though

mzar: I don't want anything requiring manual steps

mzar: OK, I shall.

in the past I tried to fix similiar problem using anchors and loading these anchor from files, but it didn't help much and only made things more complex

mzar: bugs.freebsd.org/bugzilla/show_bug.cgi?id=285081

dvl: have you tried setting the pf table sizes via loader.conf? sysctl.conf stuff is loaded much much later

zi: How do I set pf tables sizes in loader.conf?

vi /boot/loader.conf

shove sysctl.conf lines into file

^C:wq

shutdown -r now PRAISE ZI

something to that effect

First, I need to find the sysctl items. I have tried net.pf.request_maxcount=350000 in there, no fix.

net.pf.states_hashsize

Size of hash tables that store states. Should be power of 2.

Default value is 131072.

net.pf.source_nodes_hashsize

Size of hash table that store source nodes. Should be power of

2. Default value is 32768.

here we go again!

may the odds be ever in your favor

zi: added the entries, rebooted, values are not changed.

zi: this is how we do it... dpaste.org/HyXLS

what do you mean that they were not changed?

are you loading pf @ boot as well?

[19:59 gw01 dvl ~] % sysctl net.pf.states_hashsize net.pf.source_nodes_hashsize

net.pf.states_hashsize: 131072

net.pf.source_nodes_hashsize: 32768

if the module isnt there, you cant set settings from within the module

oooooo

Never done that before~

pf_load="YES"

and again

hope you are getting your card punched each time

you'll be in the frequent rebooters hall of fame in no time

indeed

zi: see anything wrong in my config? dpaste.org/zjJ5s

can you share the output of the 2 sysctls you are changing (sysctl xx)

and then grep -r net.pf /etc/sysc*

and grep set /etc/pf.conf*

[20:10 gw01 dvl ~] % sysctl net.pf.states_hashsize net.pf.source_nodes_hashsize

net.pf.states_hashsize: 131072

net.pf.source_nodes_hashsize: 32768

[20:22 gw01 dvl ~] % grep -r net.pf /etc/sysc*

/etc/sysctl.conf.local:net.pf.request_maxcount=550000

[20:23 gw01 dvl ~] % grep set /etc/pf.conf

set limit { states 200000, src-nodes 100000, frags 200000, table-entries 350000 }

set block-policy return

set loginterface $ext_if

set skip on lo0

and no dpaste for me.

% grep pf loader.conf

net.pf.source_nodes_hashsize="1048576"

% sysctl net.pf.source_nodes_hashsize

net.pf.source_nodes_hashsize: 1048576

it looks like if you have /usr/obj on tmpfs, WITH_META_MODE=yes is not respected. Of course, if you try consecutive rebuilds ( without reboot, because /usr/obj gets cleared ). Is this expected behavior?

quotes

you must quote it

quote it good

can't reboot right now... $WORK

hah

keep your punchcard handy

dvl: maybe adding PF to kernel will help to fix this problem

mzar: and go away from RELEASE ...?! o.O ;)

I am sorry, wrong advice, let me apologize, I am not much using RELEASEs

mzar: No apology necessary, it is a valid suggestion, and my reply was joking.

bye

zi: no workie. I find edge cases.

must be doing something funky here

those settings def work, at least on this 13.4-R box

I do find edge cases.

i love edge cases

Interesting, if I empty the large file, reboot, still does not load the rules. I suspect something else is blocking this. I'm going to visit the console in the basement. SOmetime....

maybe fire up a VM to test the loader bits in isolation

zi: You'll remember the old console server.... I scrolled back, found the pf errors stopping the load. Well, I got the rules to load by fixing the hostnames, vs IP addresses, which crept into my pf macros. I also emptied the table file. I'm going to load that back up and try again. Still default values for net.pf.states_hashsize & net.pf.source_nodes_hashsize

maybe restore pf.conf to defaults/no tables and just get the loader knobs applying correctly, then start reintroducing stuff

and/or move this into a vm for testing so you arent bonking (what im assuming is) your nat gateway

Yeah, it's my nat gateway.

zi: I have solved the major problem of the rules not loading on reboot.

I am not able to get net.pf.states_hashsize set away from default. Yet that is not stopping the table from loading.

rgr

I've closed my PR and apologized.

zi: Thanks for helping. Sorry for the noise.

mzar: you as well, thanks. :) Fixed.

nw

thank you for giving the opportunity to help with this troubleshooting dvl !

no worries. always happy to share the grief.

lol

are we sharing grief? I'm here.

jbo: I kind of think the answer is that the -newlib port needs to go away

kevans so we are indeed sharing grief then :<

I think life would be better if the -gcc port instead fetched newlib and built it, then libstdc++, and distributed both (maybe later in subpackages)

I don't think it realistically adds much to the build time of a gcc

kevans, is -gcc fetching and building newlib even an option? that sounds like a lot of hassle.

I wouldn't be worried about build time either. I'm more worried about the complexity.

well it's not technically hard to do in ports

kevans, are you going to do it then? <3

it would save me so much hassle every day...

my main concern would be how it current builds, and whether we'd need to swap in our own do-build to drive it all manually

kevans, just out of interest - why do you think that this is the solution vs. just having a -newlib port? Is the problem that the -newlib port would need to be built by the particular -gcc port?

it's mainly that, afaict, a -libstdc++ port would largely be duplicating a -gcc port

because it's distributed with gcc, same build system, need same options, etc

I see

how complex do you think this whole operation would be?

probably not too bad

thank god it's weekend now? :')

well, there's a complication there

it's going to be above 0*C this weekend and relatively nice outsside <_<

hah

well, I hope you can enjoy that :)