has anyone tried the new PF NAT64 support in 5.0 (pass in ... nat-to)? it seems to work but i can't get ICMP errors to be delivered, so e.g. traceroute doesn't work

s/5.0/15.0

ivy: if you look at the commit which introduced nat64 support, it seems you should be getting those

gt: i think it's because i'm missing this one: cgit.freebsd.org/src/commit/?id=25d…a4fc6e152a05e091180b2e031ab495ba337 - need to wait until later to test though

interesting

ivy: can you confirm that it works trying a pure icmp traceroute?

ICMP doesn't work either, but i'm hoping this actually fixes both despite the commit message not mentioning it, if not i'll open a bug

so you can't even ping any node atm?

oh, normal ping works, i mean ICMP traceroute doesn't - the problem is specifically how the ttl exceeded packet is translated

for ICMP traceroute it seems like the error is dropped entirely though, so... we'll see what happens

traceroute -I $host, or traceroute --icmp $host should do a pure icmp traceroute with no udp shennanigans

that's what i'm talking about - neither ICMP nor UDP traceroute works, but even though the commit message only mentions UDP, i'm hoping it will also fix ICMP traceroute as the fix doesn't look protocol-specific

got it thank you, i was just curious and have no way of testing it myself atm (i actually implemented a pure icmp ping/traceroute library back ~2015 in golang)

ivy: i'm not sure they'd appear in pflog if there's an underlying bug (by explicitly adding a log rule for icmp types), but you can try tcpdump to see if they're even hitting the interface

they should be

at least the incoming ones, if you don't even see the outgoing icmp packets, then that'd be 150% pf's fault

incoming packets on the interface will be captured by tcpdup before they're processed by pf, but outgoing ones could only have gone through pf first

The latest cpu-microcode-intel 20250211 is causing my workstation to crash on boot. Anyone else have success or otherwise?

This is on a 7th gen Intel i7, but I notice the updates don't pertain to this particular generation

something seems to have broken IPv4 routes with IPv6 nexthop over wg(4), even though it works fine on Ethernet

forwarded traffic just vanishes, and local traffic gets 'no route to host'

filed bugs.freebsd.org/bugzilla/show_bug.cgi?id=284857 ... i'm sure this used to work fairly recently

ivy: Connection over IPv6 or v4?

ek: the Wireguard tunnel is using IPv6 transport

ivy: So, connecting to server via v6?

And routing v6? Or routing both v4 and v6?

do you mean the tunnel or the encapsulated traffic? the tunnel is configred with IPv6 endpoints, the traffic being passed is IPv4 but with an IPv6 nexthop (over the tunnel)

or the traffic *not* being passed, i should say... IPv6 traffic is working fine over the tunnel

ivy I guess both? I was wondering if the WG server was listening on both v4 and v6, what the WG tunnel was offering to accept as a forward (end-point or otherwise) and what you were expecting on the client-side (full v4/v6 from end-point or otherwise.)

But, I get it.

ek: wireguard only listens on IPv6

and allowed ips is set to 0.0.0.0/0, ::/0 on both sides

I'll see if I can test this and reproduce.

ivy: It's assigning both v4 and v6 then?

... to the client?

no, the only addresses on the wg interfaces are fe80::1 on one side and fe80::2 on the other side, so all the routes are using the link local gateway

I shouldn't say "assigning." But, accepting.

ivy: I would think that should be fine as long as the link-local can be used as a gateway. And, you say v6 is working fine, but v4 isn't?

right

and it's not ipv6 nexthops being broken in general (which happened a while ago) since the same thing works fine over an igc(4) interface when wireguard isn't involved

the error is weird too:

PING 46.235.229.111 (46.235.229.111) from 81.187.47.196: 56 data bytes

ping: sendto: Address family not supported by protocol family

Hrm. That almost looks like an issue with the way the routing is happening. Is the 81.187.* address your WG client?

If so, shouldn't that have an RFC1918 address the v4 side?

they're both servers (or peers) as far as wireguard is concerned, neither is really a client. 81.187.47.196/32 is assigned to lo0 on that side

Ah. I gotcha'. Peer-to-peer setup. Not a client<->server setup.

(ping -S is needed because freebsd source address selection is broken for ipv4 when the source address is on lo0, which is also annoying but it's a separate issue)

I haven't done a WF P2P setup yet. So, I'm afraid I won't be of all that much help here. However, does this help at all?: forums.freebsd.org/threads/ping6-ad…-supported-by-protocol-family.51467

that looks like it was a bug in the poster's own application

Yeah. It does. I also didn't notice how old it was. I don't think that's related. Sorry.

i suspect something is going wrong inside wg(4) when it looks up the nexthop

i don't see any recent commits to that code though

i wonder if i can get BIRD to add these as interface routes instead...

I would think that's very possible.

it is not urgent so i might just wait to see if someone fixes it :-)

ivy: With any luck, the bug reported should turn up something.

You're not doing any v6<->v4 negotiation between the two hosts, right? Nothing out of the ordinary?

Like, no conversions?

BGP is running over the tunnel (which is what adds the route), but nothing particularly unusual. other than ipv6 nexthops for ipv4 i guess, which doesn't seem to be all that well tested

Hrm. I'm not sure why any of that wouldn't work. Seems (fairly) straight forward. Perhaps it is a bug with nexthops. Not sure. I'm going to follow that bug, though.

hey guys. I'm trying to determine the current global v6 address from python code, and got all v6 addresses printed out very quickly using pyutils.net_if_addrs. my problem is now: how do i deal with deprecated prefixes? i need to know which ip address would be used to establish new connections

*psutils, not pyutils

i did some quick google searches on how to deal with that, but the complexity seems to be rather high compared to the 4 lines needed to just get the addresses

Grabunhold: a much easier way to do that is to connect() a UDP socket to the target, then use getsockname() to find the local address. that won't generate any network traffic

ivy: that sounds good!

trying to reliably reproduce freebsd's source selection algorithm would be unreasonably difficult, especially for ipv6 as you have to deal with the RFC address selection policy which the administrator can customise

so if i understand correctly, by opening a new socket, i let the OS do it's thing regarding v6 address selection and then simply look at the address it has chosen

and that should, without customization by the admin, be the most recent prefix?

not necessarily "the most recent" -- e.g. if you connect to a link-local address it will choose the appropriate link-local address as source, even if a ULA/GUA prefix is assigned

it will also prefer ULA source when connecting to a ULA, and vice versa for GUAs

ivy: my usecase is determining which address to use to update an AAAA record of the machine. my ISP might push a new v6 prefix, and I want to notice and update that record

so if i point the UDP socket to a GUA (globally unique address, i guess?), it should pick the address generated using the recently pushed v6 prefix over the deprecated one?

hi, I trying to play around with pf and altq.. pastebin.com/04F2NNa9 when I running pfctl -nf /etc/pf.conf I got syntax error on line 11 that is queue highprio.. and I cant really see whats wrong and i wondering if someone can point me in right direction?

Grabunhold: assuming nothing unusual is going on, probably, yes

if this is DHCPv6-PD though, it might make more sense to do this in a script that runs when a new binding is received

(if it's just SLAAC, i don't think FreeBSD has a way to hook a script for that though)

nwe: `max 10Mb` is the problem

ridcully: so it means it will by default only allow max 6Mb ?

is it still the case that you should disable LRO and TSO on router interfaces, or is that out of date?

ivy: it's just SLAAC

ivy: will try my luck with the UDP socket under the assumption that my network is not too unusual. we'll see how it goes. I can report back if you're interested (but it might take me a few days to get around to actually implement that script)

thanks for your input and time, it's very much appreciated

I using qemu with network-driver intel e1000 (found in FreeBSD as em0) and in man altq I can read that em driver should support ALTQ, but when I trying to load my pf.conf it complain about 'altq not defined on em0' should I take that s em0 doesnt support ALTQ ?

nwe: side note: you might want to switch to virtio interfaces when using FreeBSD as a guest on qemu

Grabunhold: I will change to start using virtio instead, do you know if vtnet0 support ALTQ ?

i do not, sorry

but virtio should universally be better if you care about network performance

emulation real hardware is usually a fallback for situations where guest-side paravitualized drivers are not an option

*emulating

em(4) does support altq according to altq(4), while vtnet(4) is not listed there

Grabunhold: I dont running FreeBSD on my firewall now, and I testing it in a virtual machine to maybe later switch and convert existing firewall-rules to pf (Freebsd)

ivy: maybe the driver supports it, but the hardware as emulated by qemu does not?

Grabunhold: looks like it is as you say.. the hardware emluated doesnt support it..

becuase both vtnet0 and em0 doesnt work both complain about vtnet0 and em0 doesnt support ALTQ

nwe: did you compile a kernel with altq support? i don't think it's in GENERIC

ivy: nope Hasn't compile anything just installed FreeBSD on this vm

nwe: you will need to build a custom kernel with the options described in altq(4), see docs.freebsd.org/en/books/handbook/kernelconfig

okey will look into it :) thanks for pointing this out =)

also i would be surprised if vtnet didn't actually support altq, the documentation may just be out of date

ivy: I will start to read and compile custom kernel

and see if it´s working after that

don't expect too much performance if you have to use emulated e1000 h/w though

emulating real hardware via qemu comes at a considerable price

Grabunhold: this is not for performance just more for learning ALTQ (FREEBSD) and also test convert existing firewall to FreeBSD PF =)

so I later on can just drop replace my hardware :P

Grabunhold: ivy: after I recompile the kernel with ALTQ it´s working :) but another stupid question.. if I want to limit one queue to say only 1 Mbit should that queue look like this "queue sco bandwidth 1Mb priority XX , I trying to scp (because I have set that pass in on $ext_if proto tcp from any to any port 22 queue scp . but when I scp a big file it still going over 1Mbit.

Where do I find a complete list of etherner vendors that is downloadable, machine-readable and free?

the people that assign internet numbers seems like a good starting point

Indeed. Thanks.vse/get-db?t=25-01-26"; proccmd="gawk --csv '{print $1 " " $2}' @tmpfile@ > @oidfile@"

oidurl="se/get-db?t=25-01-26"; proccmd="gawk --csv '{print $1 " " $2}' @tmpfile@ > @oidfile@"

bah. cat.

I'vebeen using se/get-db?t=25-01-26"; proccmd="gawk --csv '{print $1 " " $2}' @tmpfile@ > @oidfile@"

'me gives up

sign. This. wireshark.org/download/automated/data/manuf.gz

what are you trying to do?

there is a web oui look up with the wireshark data here: wireshark.org/tools/oui-lookup.html

No, I need it downloaddabke so I can read it in whatever fom it ar arrives in and ad it to be datbase.

gzcat manuf.gz | grep FreeBSD

58:9C:FC FreeBSDFound FreeBSD Foundation

ghoti: I made a Rust crate for that last week, if you look in the tools folder here there is a script to download them all: github.com/erk-/macnuf

I have a VPS on DO. Loopback jails are fine. I now created a VNET jail. First I created `bridge0` with address 172.16.0.1/24. The VNET jail (172.16.0.2) (its epair) is the only member of the bridge. In `pf.conf`, i have `int_if="bridge0"` and a rule `nat on $ext_if from $int_if:network to any -> ($ext_if:0)`, plus at the bottom `pass in on $int_if`. It nearly works. `host`/`ping`/`nc` cooperate. `fetch`/`pkg` no. Any ideas?

so I am wiping some disks and I run smart extended test on one, and then I go to wipe it clean for use and I get constant "Operation not permitted", and when I try to gpart it "Device busy"

this disk is not mounted, it is not in use, yet I cant write to it

securelevel is at -1

any ideas?

I have tried rebooting, same issue

all done as root*

erk: Haven't done any rush before. Tnis is tne consumer: github.com/chvostek/shelltools/blob/master/narp

s/sh/st/

hmmm weird, so I can delete one of the parts on it, but the second is not permitted, and its a ms-basic-data hmmm

I cant dd over the disk, I cant touch it

scoobybejesus: I'd make try it in your home lab, where you can see traffic and get pcaps from multiple perspectives. Not sure what might be missing. Might be worth revisiting docs.freebsd.org/en/books/handbook/firewalls

mason: Unfortunately, i got it working in my homelab first (fresh install on a tiny atomic pi), and i don't have this SSL issue there. But I do have another homelab box i could try it on

And, yeah, i've been reading through that chapter, and I've been trying to experiment with pf.conf. I'll experiment more. It does seem like the NAT'ing has something to do with it. I was wondering whether I needed to disable TSO on the public interface, or something like that, but it's a bit outside my realm of experience

I appreciate the reply

homelab ifconfig interface options, `options=8209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGIC,LINKSTATE>` versus digitalocean `options=4c07bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,LINKSTATE,TXCSUM_IPV6>`.

what do you see in tcpdump on bridge0 or your ext if when you try to use fetch or pkg? even something like fetch google.com doesn't work?

(might as well disable TSO and LRO, though)

polarian: weird, i remember that has happened to me when a VM has opened the device, but that's not the case for you right?

i wonder if something at startup is opening the device

same error for `fetch google.com`

what error do you have? Operation timed out? but nc works for a tcp connection?

here's tcpdump during that fetch: paste.debian.net/plain/1350399

looks normal

the paste from above shows that it's an SSL error. along the lines of `0020E1224A070000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:/usr/src/crypto/openssl/ssl/record/rec_layer_s3.c:1605:SSL alert number 80`

ah sorry i never saw your messages about any ssl error

i see now

well, then it sounds like an SSL problem. i don't know what to do about that, but "SSL alert number 80" sounds relevant. the only thing that comes to mind, is the ca_root_nss package

hmm, looking up that error it sounds like it's something else

hm.. you gave me an idea... checking...

the host and working jails were returning the time in UTC. this vnet jails was returning in EST. i ran tzsetup to choose UTC. no luck

a stackoverflow post points to part of the TLS RFC that indicates that "alert number 80" is "internal_error" i.e " internal_error: An internal error unrelated to the peer or the

correctness of the protocol (such as a memory allocation failure)

makes it impossible to continue.

so it doesn't sound like it has anything to do with time, and even then, a problem would be time being wrong, not just a different time zone

and the error is in "ssl3_read_bytes"... so maybe somehow it cannot allocate memory for the response data

From top in the jail `Mem: 512M Active, 2452M Inact, 820M Wired, 39K Buf, 133M Free`

maybe try `openssl s_client` with some verbosity and manually perform an HTTP request

i forget the syntax

maybe openssl s_client -security_debug_verbose google.com:443

paste.debian.net/plain/1350400 is just -debug

and security_debug_verbose paste.debian.net/1350401

nothing in those are interesting to me. same error. i disabled tso and lro and no change

not sure what to make of it. clearly something happens pretty near the start when it's trying to read data from the peer. when i do it, it reads quite a bit but on yours it stops after only 7 bytes read

it seems like a number of people on the web have encountered this error though, so that or people in #openssl or similar might know more

jmnbtslsQE: nope... its a base freebsd install just for wiping disks, it wiped the other disk just fine...

nothing should be using it

unless... the first disk I wiped I had plugged into a different sata port

let me try swapping them around again

this will just swap the adax identifiers

but hey ho its worth a shot

s/identifiers/block devices/

sorry I am half asleep

thanks for digging a bit on this jmnbtslsQE. it would be nice to have a vnet jail or two on my VPS, but this is a roadblock

scoobybejesus: it seems like it's actually being sent by the peer: rfc-editor.org/rfc/rfc8446#page-85

so, pretty strange, since you're seeing it anywhere you try

that almost makes me wonder if it's some kind of defective MITM attack. but i don't know much about TLS

polarian: OK

scoobybejesus: maybe your openssl version is too old. i see people on the web talking about that

though, i would think that version issues would be handled in the protocol by some other means

OpenSSL 3.0.15 3 Sep 2024 (Library: OpenSSL 3.0.15 3 Sep 2024)

heh, OK

strange that in your tcpdmp though, your client sends data first, then only reads 8 bytes. not consistent with what's in the openssl dump. so, not sure what to make of that, maybe i'm missing something

oh, nevermind it is consistent. nevermind

because in openssl s_client it says it writes first then reads.

well, then it sounds like some kind of openssl-related issue somehow. your client sends some data, and the server responds with a fatal "alert" error, surely #openssl will have something to say about it

and i guess you're not getting the same issue from the host, only from the jail.

well, i'm definitely curious to hear of whatever the resolution is

from a VNET jail at home with the same networking config (its epair is the only member of the bridge, and the bridge's traffic is NAT'd out the external interface) paste.debian.net/plain/1350404

yeah that's similar to what i see

yeah, and it's similar on the VPS host. only this VNET jail, NAT'd out, is having this issue (again, all on 14.2-RELEASEp1)

OK

when i can spend some more time on this, i will be asking #openssl. i appreciate your help. if there's a resolution (please please), i'll let you know

OK great

that is so weird...

jmnbtslsQE: works

if the disk is ada0 it doesn't allow me to write, but if it is ada1 it does

weird edge case I found maybe?

maybe freebsd is going based on the install, when I installed the system the disk I installed to was ada0, so when the identifier is moved to ada1 freebsd gets confused?

that's nice, apparently postgres GSSAPI finally got fixed to use MIT Kerberos so you can build it again

polarian: what device are you booting off of?

jmnbtslsQE: ada0 which then became ada1

HDD

because device identifiers are based off the sata slot

that happens

you should use labels and don't write into any ada*

device numbers aren't fixed!

good that it didn't allow you to write to boot disk

but if it's not mounted, you could very easily do it

if all other ids are off, you could still use smart serials and so on to verify what device it is, that's if you can't differenciate it from size maybe

ketas: no... its not the problem

the identifier changed, no biggie

the bootloader simply looks for the boot partition and boots the loader no problem

there are smartctl -x, diskinfo -v, gpart show, glabel status at your disposal

the loader knows where to boot

oh

ada0 was the identifier when I installed with a single HDD

when I installed a second, the identifier switched

ada1 was now the boot, ada0 was the disk I wanted to erase

I couldn't write to ada0

it wouldn't let me

it was the right disk, it wouldn't let me...

why?

dunno

I switched the sata cables over, and as ada1 I coulud write to the HDD

because it had zpool imported?

eh

nope

empty disk

looked like old windows install

you can write to there always

dont really care its a second hand disk I am prepping for install

geli + /dev/zero write over the disk and a extended smart test

I am just curious now *why* I couldn't

i use custom procedures, with ideas from reddit.com/r/DataHoarder/comments/qtbte3/comment/hkjxx8a

I wonder if freebsd assumed ada0 was the boot disk or something

it was not mounted, in use by any process etc

and it was "Busy"

and trying to dd to it you get "Operation not permitted"

then it was wrong disk?

nope

it was the right disk

there are no other safeguards there

thats the thing!

I double checked 52 times

it was the right disk

geom disk list + smartctl -a

it was the right disk

polarian: i missed the start of this conversation, what were you trying to do?

ivy: dd the disk :)

did you delete the partitions and label first?

(gpart destroy)

you can dd into partitioned disk too

if it's not mounted in any way

if it is, there is sysctl kern.geom.debugflags=0x10 for you

that allows it all

also ketas that reddit thread is a lot more indepth, I haven't the patience, if I did I would do a few random overwrites, I am just doing a basic check that the platters are still working (smart extended test will detect if there is something broken) and wiping any existing data, I don't need to but I feel its a good thing to do for the previous owner who left their windows install on the damn

disk unencrypted

ivy: couldn't

"Device busy"

now you could ruin boot disks too

I am still curious why freebsd wouldn't let me write to the correct disk if it was ada0

i'm not aware of any such bug

except if it was a wrongbdevice

s/b/ /

polarian: not sure. maybe swap from your fstab was specified with the identifier, and it happens to be that partition number?

/etc/fstab doesn't exist :)

OK

I did check for that one

ketas: I checked many MANY times

the boot drive is 500GB the wipe one is 1TB

I made 100% sure

unsure then

just brain is more likely cause than cpu

would be fun if it's bug

this wiki seems a bit outdated wiki.freebsd.org/arm/Ampere

who can I reach out to for updates?

*to update*

afaik you can just create an account and update it yourself

otherwise, reach out to Mark Linimon

ketas: fine I will try it again with another disk, see what happens :P

jbo: can't make account there

i bet you can have one tho

oh, there it was

still manual register

i guess there are too many idiots nowadays

this is not the first page there that's outdated, too

had idea to update but somehow didn't request access eh

contributors are always very welcomed :)

yes

i filed my 10+y old patches as pr's, but now i'm kind of stuck

eh

>10 years old PRs should just be closed tbh

jbo: nonono, i had patches sitting around locally on disk(s) for over a decade, i looked and found noone implemented those in time between, found good feedback for all of them, but then it felt like too much wor

k

even for those small things

one was suggested to add as a new boot flag which seems waaay top much to chew

aye

s/top/too/

hell knows what else sits around like this

new boot flag to what?