does FreeBSD UNIX 14.2 .ISO also work on USB flash drive (at least on its own, but maybe also in a boot manager like MultiBootUSB or YUMI) or must be on DVD?

darwin, not sure if this fits the background of your question (you likely are already aware of that) but there is a separarte image file for memory sticks which can be used to flash a bootable installation USB stick

i have that, but it's incomplete/smaller so I'd rather use a hybrid .ISO, which should work for USB flash drive (but not always in a boot manager)

if you required packages present then it's indeed incomplete

i did bring up the maxi memstick idea somewhere

there wouldn't need to be an entire other .IMG as long as the .ISO is hybrid (works for both CD & USB flash drive)

does it work?

any reason you're not trying it?

download restrictions or so

already have all the FreeBSDs I want installed but just want to update boot USB flash drive

updated 3/4 the installations within those and will do the last one within it

unless I have emergency (unlikely but why I'll update the USB flash drive)

you updated from installer eh?

i guess it works

if you have more sticks you could just have more installers eh

mzar: great idea - done

mfisher: what was the idea ?

that I could donate to the FreeBSD Foundation instead of just saying thanks for 14.2-RELEASE on IRC

sure, you can do both

done and done

cool

Say I want to initialize jails on first "boot". Like, cloning a ZFS snapshot with a barebones FreeBSD fs (just the extracted base.txz and little more), updating `/etc/jail.conf` on host, putting some file (script or archive) somewhere within the jail, and then when I run `service jail start $JAIL` I want it to install packages, update /etc/rc.conf, update some config files, etc.

Any recommendations on the least painful way to achieve that?

The goal is to have an initial setup as vanilla as possible, and from there leave the setup to some automation. I'm guessing Ansible would be the way to go? I was reading about `configinit`, but there doesn't seem to be any port so that's probably very specific to AWS EC2.

But I want to do this on my own PC (i.e. not EC2).

The problem I'm trying to solve: When I upgrade the host from (for example) 14.1 to 14.2, I want to be able to recreate the jails from scratch (like you would a Docker container). I know I can just (1) keep running the old version, or (2) do `freebsd-update -j $JAILNAME fetch install`, but I'm trying to learn a bit more around this topic.

shiroyasha: maybe you wan thin jails instead of thick ones

you could use ansible, yes

but it's not a trivial problem to solve

because things like /etc/ssh/ssh_host_*.key's and things

Happy new year!

woo hooo!

like, 96% of the time, just copying in a new 'world' and new 'kernel' would be sufficient, i would expect

if anyone is running the matrix server, conduit, in a jail, in particular behind caddy, did you have to do anything special? i'm not getting a web interface on normal 443, and it seems port 8448 has connection refused. logging the packets in the host in pf, they arrive, so they should be redirected to the caddy jail, and caddy should be handling those packets properly. caddy logs show mgs: NOP over and over. i'm stuck

scoobybejesus: netstat -an | grep 8448 to see if it is actually listening on the correct port inside the jail.

i'm logging packets to port 8448 in pf, and they're hitting the rdr rule sending it to caddy. i can curl from the host and it returns Hello from Conduit!. But doing curl from home whether to 443 or 8448 returns either nothing or failed to connect (for 8448). Adding -v doesn't add much except that it's reaching caddy. I've been pulling my hair out

and you've ran that from within your jaili?

jail

i did sockstat -4 in the jail. the jail is listening on 6167 like it says it should be. caddy is supposed to reverse proxy to that jail IP:6167

is conduit in a seperate jail?

nothing is actually "listening" on 8448. though in the caddy config, i have my matrix url:8448 as on of the hosts

yeah, caddy in one jail, and conduit in another

you can't use localhost/127.0.0.1 in the caddyfile is what I'm reading.

caddy inside the jail sees localhost as itself. is that possible the issue?

i'm using the jail's address in the caddyfile. i'm sort of matching what is working for the other 10 jails. the config looks fine. i don't get it

you must use the jails IP address of the conduit jail

ok

tried tcdump within caddy jail?

scoobybejesus: I have a similar set-up but with haproxy and dendrite

it's not a vnet jail. i wonder if i can allow it with a devfs rule. but no, not yet. "tcpdump: vtnet0: Packet capture is not supported on that device"

you need to have caddy actively listening on 8448 and forward that port from your NAT'd iface to it

and dendrite cooperates just fine? that one is pretty lightweight, right?

yes I have no issues

lemme double check my set-up

paste.debian.net/plain/1341961 is my caddy config. that sits inside a wildcard block

scoobybejesus: yeah, dendrite is listening on 8008 in my case, and haproxy has a front-end on 8448 that does TLS termination and throws everything at matrix

sounds like caddy is trying to forward traffic to a port where there is no service running

well, i certainly am getting absolutely nothing in the conduit logs, even when i bumped it up to TRACE

scoobybejesus: do you have 8448 listed in caddy global https ports?

what does: jls say?

sockstat in the conduit jail paste.debian.net/plain/1341962

scoobybejesus: conduit can only listen on one port, that is fine

conduit isn't listening

it is

i do not have 8448 listed in global https ports. hm. looking into that

scoobybejesus: sockstat -4l in your caddy

does *caddy* listen on 8448?

shows it's not listening on 8448

bebop: it should not

looks like caddy is not listening on 8448, despite the config telling it to.. hmm

one port is enough for a matrix homeserver, the load balancer/proxy should handle TLS

shows it's listening on 6167

in the matrix jail, conduit is listening on 6167. in the caddy jail, i am reverse proxying to the matrix jail port 6167

scoobybejesus: your conduit is perfectly fine, you need to set up caddy to listen for TLS on 8448

and reverse proxy for matrixhost:8448 to caddy's 6167

what does you toml bind addr and port say?

in the matrix jail, toml says to bind to 10.0.0.121 and port 6167

should be port 8448 ?

no

bebop, do you run a matrix homeserver?

scoobybejesus: paste.ee/p/ZlzsQ this is an example from my set-up

note that HAProxy listens on 8448 for TLS and forwards to dendrite

Remilia: no, I haven't

scoobybejesus: aside from 8448 you already forward some of the 443 for well-known purposes, which is correct

8448 is the federation port. yeah, my caddyfile listens for the appropriate URL both on 443 and on 8448, though it appears it's not actually listening on 8448. looking at the docs on setting a global https_port to 8448 doesn't seem correct though. hmm.. i'm digging

caddy docs do not give any hints

scoobybejesus: I found this xiu.io/posts/14-caddy-reverse-proxy-dendrite and seems like it works for them so I am very confused

scoobybejesus: all I can think of is configuration issues

oh

scoobybejesus: yeah, look at the Caddyfile samples, you have issues in your configuration

it does *not* tell Caddy to listen on anything, so it just listens on the default ports

you need `chat.my.tld:443 chat.my.tld:8448 { .... }`, *that* will already match HTTP host

oh and you can skip :443 for the first one

right.

hmm. I will try again

xiu.io/posts/14-caddy-reverse-proxy-dendrite follow this example

you only need the 2nd part if you are not doing different-domain

I added 443 just in case it would start working

oh wait, no, you also need to move the first part's well-known probably

not sure, might work without with default schema

when I ran conduit a couple years ago I didn't need the well known part. I have never added that in. I was thinking to add it, but that isn't in the conduit docs. weird

it's needed if you use a different domain AND if you are matching on _matrix

in your case, if you run nothing else on chat.my.tld, you can just handle /*

I'm 99% sure that your issue is in using the host matcher there for your global wildcard host

the only thing I can think is that matrix requires an A record, so maybe a wildcard cert is not good enough

no, it should be fine

maybe I should have a separate section in the caddyfile. hmm. but you don't think so. hmm

look

Caddyfile effectively has matchers and virtual hosts, and you do not have a virtual host defined

your line, @chat host chat.my.tld:443, chat.my.tld:8448, means 'define a matcher'

and 'handle ....' applies to *any hostname*, then filters by host

it's like if you set up nginx as just one host, with matches on the Host header

scoobybejesus: specifically look at caddyserver.com/docs/caddyfile-tutorial#multiple-sites

scoobybejesus: paste.ee/p/1ecw5 this is *all* you need

I am going to print the json version of the caddyfile config to see if the matches appears to work the same when there are two hosts listed.

take a look at that paste

this tells caddy to match Host to chat.my.tld, listen on default 443 and on 8448, and forward everything to conduit

also maybe no , needed, not sure, I do not run caddy haha

that will pull a fresh cert specifically for chat.my.tld, going in its own block, rather than using the wildcard cert i already have. that's why i'm using the matchers. but i may do that anyway

scoobybejesus: then with your setup you need to add *.my.tld:8448 where you have *.my.tld

hmm... now you're onto something...

that *.my.tld section is what defines a listener

sockstat shows it's listening on 8448... phew... super helpful! thank you!

sorry, I had no idea how it handles wildcard stuff ahaha

this is too 'magical' for me, I manage certs the old way

now i can curl the server on port 8448 and not get the connection refused, but i'm still not getting any response. caddy logs show all these NOP (no-op) log lines

so annoying

and so much backlog was all caddy and not even freebsd. sorry for that, folks

maybe i will switch to dendrite...

haha

scoobybejesus: in `reverse_proxy /_matrix/* 10.0.0.121:6167`, remove /_matrix/* altogether

you are matching on Host and want everything to go to conduit

i was considering trying that. now seems like a great time to try it.

1) if you are cURLing 8448 without specifying /_matrix/... you will get no match, which is what NOP probably is, 2) you want webfinger to work

(/.well-known/matrix/ stuff)

nada

run tcpdump with ASCII output for the conduit port in your conduit jail, see if caddy tries to connect

caddy is not sending packets. i just tested the same command (running from host) using another jail's IP and going to the website hosted by the other jail, and tcpdump spewed out a bunch. with `tcpdump -i lo0 dst 10.0.2.121`, there's nothing

caddy log entries show "bytes_read":0 for all these chat.my.tld.log entries

scoobybejesus: ok but you are contradicting yourself right now

you posted your caddy config as reverse_proxy /_matrix/* 10.0.0.121:6167

now you are posting `tcpdump -i lo0 dst 10.0.2.121`

sorry about that. mistype! tcpdump is listening on the correct address 10.0.0.121

then it's something with your caddy setup

scoobybejesus: btw use /_matrix/federation/v1/version in curl to check what is going on on 8448

curl 10.0.0.121:6167/_matrix/federation/v1/version in both the host and caddy jail returns {"server":{"name":"Conduit","version":"0.9.0"}}. from home, curl returns nothing... i need a break...

8448 is for federation. i would think at least 443 would cooperate though and show me a web page

fascinating. when i get rid of the chat.my.tld in caddy, i am able to pull up chat.my.tld/_matrix/federation/v1/version in a website. so yeah, something with caddy? it still won't return anything at chat.my.tld, but it's a clue

anyway, i feel bad.. this is now officially caddy and not freebsd

feel free to bounce ideas in -social or somewhere. i really appreciate the help so far

I have a silly problem: my freebsd machine takes an absolute age to be available over ssh. I've recently changed its network settings so it's now in routing mode. I suspect network issues.

it becomes pingable pretty quickly over both ipv4 and ipv6, but ssh doesn't seem to come up until I can ping the jails

I suspect I'll find out very quickly once I can get over to it and plug in a display

... it's definitely the jails, hrmn

disable jails and it comes up immmmedddddiately

Hm. I disable ipv6 in the jail and it comes up nearly as quickly as with jails off, but still with a 2-3 second extra delay. So something in the process of firing up the VNET for the jail is slowing down boot

I suppose I could solve this, but I've also discovered that if I want to route traffic internally to the global ipv6 addresses on the jails I need to distribute routes to every damn computer on the network somehow, plus it's not DMZ'd, so I think my next play is to either stick the entire jail subnet into a VLAN or an l2tp tunnel and hand it off to my router, which can then do both of those things

zip: then its an issue in your jail config?

maybe try with a single jail and then check the settings of this

yup, it was a single jail

I'm wondering if it's radvd taking its time to hand over an IP

rtadvd sorry

still, I'm a little surprised that sshd depends on it

is your ssh also jailed?

you should be able to ssh in the OS immediadly, regardless of what the jails are doing

or its an network conflict or something like that

perhaps it's that

this is my first time adding routing within a home network

zip: sshd is supposed to come up before jails

(the host system's)

yeah, that's what I expected as well

yet, I get a connection refused

I suspect I'm going to have to haul a monitor over to the machine and poke at it

well, first I think I'm going to tunnel traffic back to the router so that the router just hands over IPs via slaac and dhcp so that the machine itself doesn't have to be doing any routing, so that I can DMZ my services

well, I suppose I could dmz them with pf, but I figure it's easier this way

present status: agonising about whether to backhaul over vlan, vxlan, gre, l2tp...

zip: just to make sure, are you using the standard jail.conf stuff?

you can do `rcorder /etc/rc.d/* /usr/local/etc/rc.d/*` and check if sshd comes before jail

sure

looks like I have both sshd and openssh running before jail, although they also report that they're the same PID..

oh for... it's coming up without an ipv4 router, that can't be helping

this is a large part of why I'd rather not be faffing with routing on this machine, a lot of things have to go right

I do have the gateway set in `rc.conf`

Has anyone successfully set up an IPv6 road warrior scenario with swanctl/openiked-portable using Android clients?

I cannot make it work. For IPv6 support, I tried StrongSwan 6 and openiked-portable on FreeBSD 14-Stable. It works with Linux clients, but not with Android. It seems like ESP packets are fine, but not UDP packets on port 4500

i tried a couple of times and never got it either. wireguard was a lot easier :|

spmzt: pure IKE/ESP Android tunnels are very very wonky

you really want to encapsulate it in L2TP but that means mpd