10:38:10 does FreeBSD UNIX 14.2 .ISO also work on USB flash drive (at least on its own, but maybe also in a boot manager like MultiBootUSB or YUMI) or must be on DVD? 10:48:19 darwin, not sure if this fits the background of your question (you likely are already aware of that) but there is a separarte image file for memory sticks which can be used to flash a bootable installation USB stick 10:49:46 i have that, but it's incomplete/smaller so I'd rather use a hybrid .ISO, which should work for USB flash drive (but not always in a boot manager) 10:53:15 if you required packages present then it's indeed incomplete 10:53:56 i did bring up the maxi memstick idea somewhere 10:55:15 there wouldn't need to be an entire other .IMG as long as the .ISO is hybrid (works for both CD & USB flash drive) 10:56:45 does it work? 10:57:16 any reason you're not trying it? 10:57:42 download restrictions or so 10:57:49 already have all the FreeBSDs I want installed but just want to update boot USB flash drive 10:58:21 updated 3/4 the installations within those and will do the last one within it 10:58:36 unless I have emergency (unlikely but why I'll update the USB flash drive) 10:59:04 you updated from installer eh? 10:59:13 i guess it works 11:00:01 if you have more sticks you could just have more installers eh 13:15:26 mzar: great idea - done 13:21:47 mfisher: what was the idea ? 13:22:24 that I could donate to the FreeBSD Foundation instead of just saying thanks for 14.2-RELEASE on IRC 13:22:48 sure, you can do both 13:23:19 done and done 13:24:15 cool 13:29:32 Say I want to initialize jails on first "boot". Like, cloning a ZFS snapshot with a barebones FreeBSD fs (just the extracted base.txz and little more), updating `/etc/jail.conf` on host, putting some file (script or archive) somewhere within the jail, and then when I run `service jail start $JAIL` I want it to install packages, update /etc/rc.conf, update some config files, etc. 13:29:48 Any recommendations on the least painful way to achieve that? 13:31:03 The goal is to have an initial setup as vanilla as possible, and from there leave the setup to some automation. I'm guessing Ansible would be the way to go? I was reading about `configinit`, but there doesn't seem to be any port so that's probably very specific to AWS EC2. 13:32:21 But I want to do this on my own PC (i.e. not EC2). 13:39:40 The problem I'm trying to solve: When I upgrade the host from (for example) 14.1 to 14.2, I want to be able to recreate the jails from scratch (like you would a Docker container). I know I can just (1) keep running the old version, or (2) do `freebsd-update -j $JAILNAME fetch install`, but I'm trying to learn a bit more around this topic. 13:54:25 shiroyasha: maybe you wan thin jails instead of thick ones 15:00:52 you could use ansible, yes 15:01:22 but it's not a trivial problem to solve 15:01:48 because things like /etc/ssh/ssh_host_*.key's and things 15:04:25 Happy new year! 15:06:08 woo hooo! 15:08:09 * shiroyasha takes notes. 15:11:46 like, 96% of the time, just copying in a new 'world' and new 'kernel' would be sufficient, i would expect 17:34:00 if anyone is running the matrix server, conduit, in a jail, in particular behind caddy, did you have to do anything special? i'm not getting a web interface on normal 443, and it seems port 8448 has connection refused. logging the packets in the host in pf, they arrive, so they should be redirected to the caddy jail, and caddy should be handling those packets properly. caddy logs show mgs: NOP over and over. i'm stuck 17:44:31 scoobybejesus: netstat -an | grep 8448 to see if it is actually listening on the correct port inside the jail. 17:45:22 i'm logging packets to port 8448 in pf, and they're hitting the rdr rule sending it to caddy. i can curl from the host and it returns Hello from Conduit!. But doing curl from home whether to 443 or 8448 returns either nothing or failed to connect (for 8448). Adding -v doesn't add much except that it's reaching caddy. I've been pulling my hair out 17:45:48 and you've ran that from within your jaili? 17:45:50 jail 17:47:00 i did sockstat -4 in the jail. the jail is listening on 6167 like it says it should be. caddy is supposed to reverse proxy to that jail IP:6167 17:47:21 is conduit in a seperate jail? 17:47:42 nothing is actually "listening" on 8448. though in the caddy config, i have my matrix url:8448 as on of the hosts 17:47:52 yeah, caddy in one jail, and conduit in another 17:48:17 you can't use localhost/127.0.0.1 in the caddyfile is what I'm reading. 17:48:45 caddy inside the jail sees localhost as itself. is that possible the issue? 17:48:53 i'm using the jail's address in the caddyfile. i'm sort of matching what is working for the other 10 jails. the config looks fine. i don't get it 17:48:59 you must use the jails IP address of the conduit jail 17:49:01 ok 17:51:16 tried tcdump within caddy jail? 17:52:15 scoobybejesus: I have a similar set-up but with haproxy and dendrite 17:52:23 it's not a vnet jail. i wonder if i can allow it with a devfs rule. but no, not yet. "tcpdump: vtnet0: Packet capture is not supported on that device" 17:52:39 you need to have caddy actively listening on 8448 and forward that port from your NAT'd iface to it 17:52:42 and dendrite cooperates just fine? that one is pretty lightweight, right? 17:53:05 yes I have no issues 17:53:27 lemme double check my set-up 17:54:28 https://paste.debian.net/plain/1341961 is my caddy config. that sits inside a wildcard block 17:54:29 scoobybejesus: yeah, dendrite is listening on 8008 in my case, and haproxy has a front-end on 8448 that does TLS termination and throws everything at matrix 17:54:31 sounds like caddy is trying to forward traffic to a port where there is no service running 17:55:18 well, i certainly am getting absolutely nothing in the conduit logs, even when i bumped it up to TRACE 17:55:32 scoobybejesus: do you have 8448 listed in caddy global https ports? 17:55:34 what does: jls say? 17:56:22 sockstat in the conduit jail https://paste.debian.net/plain/1341962 17:57:13 scoobybejesus: conduit can only listen on one port, that is fine 17:57:20 conduit isn't listening 17:57:24 it is 17:57:26 i do not have 8448 listed in global https ports. hm. looking into that 17:57:50 scoobybejesus: sockstat -4l in your caddy 17:57:58 does *caddy* listen on 8448? 17:58:12 shows it's not listening on 8448 17:58:30 bebop: it should not 17:58:56 looks like caddy is not listening on 8448, despite the config telling it to.. hmm 17:58:58 one port is enough for a matrix homeserver, the load balancer/proxy should handle TLS 17:59:05 shows it's listening on 6167 17:59:39 in the matrix jail, conduit is listening on 6167. in the caddy jail, i am reverse proxying to the matrix jail port 6167 17:59:43 scoobybejesus: your conduit is perfectly fine, you need to set up caddy to listen for TLS on 8448 18:00:07 and reverse proxy for matrixhost:8448 to caddy's 6167 18:00:09 what does you toml bind addr and port say? 18:01:38 in the matrix jail, toml says to bind to 10.0.0.121 and port 6167 18:01:52 should be port 8448 ? 18:01:55 no 18:02:07 bebop, do you run a matrix homeserver? 18:02:18 scoobybejesus: https://paste.ee/p/ZlzsQ this is an example from my set-up 18:02:48 note that HAProxy listens on 8448 for TLS and forwards to dendrite 18:02:55 Remilia: no, I haven't 18:04:33 scoobybejesus: aside from 8448 you already forward some of the 443 for well-known purposes, which is correct 18:04:56 8448 is the federation port. yeah, my caddyfile listens for the appropriate URL both on 443 and on 8448, though it appears it's not actually listening on 8448. looking at the docs on setting a global https_port to 8448 doesn't seem correct though. hmm.. i'm digging 18:06:02 caddy docs do not give any hints 18:09:32 scoobybejesus: I found this https://xiu.io/posts/14-caddy-reverse-proxy-dendrite/ and seems like it works for them so I am very confused 18:12:32 scoobybejesus: all I can think of is configuration issues 18:12:56 oh 18:13:21 scoobybejesus: yeah, look at the Caddyfile samples, you have issues in your configuration 18:13:45 it does *not* tell Caddy to listen on anything, so it just listens on the default ports 18:14:28 you need `chat.my.tld:443 chat.my.tld:8448 { .... }`, *that* will already match HTTP host 18:14:51 oh and you can skip :443 for the first one 18:15:07 right. 18:15:14 hmm. I will try again 18:15:22 https://xiu.io/posts/14-caddy-reverse-proxy-dendrite/ follow this example 18:15:39 you only need the 2nd part if you are not doing different-domain 18:15:42 I added 443 just in case it would start working 18:16:10 oh wait, no, you also need to move the first part's well-known probably 18:16:32 not sure, might work without with default schema 18:17:07 when I ran conduit a couple years ago I didn't need the well known part. I have never added that in. I was thinking to add it, but that isn't in the conduit docs. weird 18:17:30 it's needed if you use a different domain AND if you are matching on _matrix 18:17:56 in your case, if you run nothing else on chat.my.tld, you can just handle /* 18:18:43 I'm 99% sure that your issue is in using the host matcher there for your global wildcard host 18:18:44 the only thing I can think is that matrix requires an A record, so maybe a wildcard cert is not good enough 18:18:59 no, it should be fine 18:19:20 maybe I should have a separate section in the caddyfile. hmm. but you don't think so. hmm 18:19:26 look 18:19:54 Caddyfile effectively has matchers and virtual hosts, and you do not have a virtual host defined 18:20:15 your line, @chat host chat.my.tld:443, chat.my.tld:8448, means 'define a matcher' 18:20:56 and 'handle ....' applies to *any hostname*, then filters by host 18:21:20 it's like if you set up nginx as just one host, with matches on the Host header 18:22:30 scoobybejesus: specifically look at https://caddyserver.com/docs/caddyfile-tutorial#multiple-sites 18:24:15 scoobybejesus: https://paste.ee/p/1ecw5 this is *all* you need 18:24:29 I am going to print the json version of the caddyfile config to see if the matches appears to work the same when there are two hosts listed. 18:24:41 take a look at that paste 18:25:36 this tells caddy to match Host to chat.my.tld, listen on default 443 and on 8448, and forward everything to conduit 18:26:34 also maybe no , needed, not sure, I do not run caddy haha 18:26:57 that will pull a fresh cert specifically for chat.my.tld, going in its own block, rather than using the wildcard cert i already have. that's why i'm using the matchers. but i may do that anyway 18:28:14 scoobybejesus: then with your setup you need to add *.my.tld:8448 where you have *.my.tld 18:28:48 hmm... now you're onto something... 18:29:24 that *.my.tld section is what defines a listener 18:31:33 sockstat shows it's listening on 8448... phew... super helpful! thank you! 18:32:03 sorry, I had no idea how it handles wildcard stuff ahaha 18:32:29 this is too 'magical' for me, I manage certs the old way 18:34:37 now i can curl the server on port 8448 and not get the connection refused, but i'm still not getting any response. caddy logs show all these NOP (no-op) log lines 18:36:09 so annoying 18:36:41 and so much backlog was all caddy and not even freebsd. sorry for that, folks 18:36:58 maybe i will switch to dendrite... 18:38:57 haha 18:40:17 scoobybejesus: in `reverse_proxy /_matrix/* 10.0.0.121:6167`, remove /_matrix/* altogether 18:40:35 you are matching on Host and want everything to go to conduit 18:40:58 i was considering trying that. now seems like a great time to try it. 18:41:42 1) if you are cURLing 8448 without specifying /_matrix/... you will get no match, which is what NOP probably is, 2) you want webfinger to work 18:42:13 (/.well-known/matrix/ stuff) 18:42:31 nada 18:43:30 run tcpdump with ASCII output for the conduit port in your conduit jail, see if caddy tries to connect 18:47:34 caddy is not sending packets. i just tested the same command (running from host) using another jail's IP and going to the website hosted by the other jail, and tcpdump spewed out a bunch. with `tcpdump -i lo0 dst 10.0.2.121`, there's nothing 18:51:24 caddy log entries show "bytes_read":0 for all these chat.my.tld.log entries 18:53:47 scoobybejesus: ok but you are contradicting yourself right now 18:54:15 you posted your caddy config as reverse_proxy /_matrix/* 10.0.0.121:6167 18:54:29 now you are posting `tcpdump -i lo0 dst 10.0.2.121` 18:54:58 sorry about that. mistype! tcpdump is listening on the correct address 10.0.0.121 18:56:22 then it's something with your caddy setup 19:03:36 scoobybejesus: btw use /_matrix/federation/v1/version in curl to check what is going on on 8448 19:10:16 curl 10.0.0.121:6167/_matrix/federation/v1/version in both the host and caddy jail returns {"server":{"name":"Conduit","version":"0.9.0"}}. from home, curl returns nothing... i need a break... 19:14:00 8448 is for federation. i would think at least 443 would cooperate though and show me a web page 19:15:59 fascinating. when i get rid of the chat.my.tld in caddy, i am able to pull up chat.my.tld/_matrix/federation/v1/version in a website. so yeah, something with caddy? it still won't return anything at chat.my.tld, but it's a clue 19:16:12 anyway, i feel bad.. this is now officially caddy and not freebsd 19:16:33 feel free to bounce ideas in -social or somewhere. i really appreciate the help so far 21:12:30 I have a silly problem: my freebsd machine takes an absolute age to be available over ssh. I've recently changed its network settings so it's now in routing mode. I suspect network issues. 21:12:52 it becomes pingable pretty quickly over both ipv4 and ipv6, but ssh doesn't seem to come up until I can ping the jails 21:15:31 I suspect I'll find out very quickly once I can get over to it and plug in a display 21:15:45 ... it's definitely the jails, hrmn 21:15:54 disable jails and it comes up immmmedddddiately 21:19:51 Hm. I disable ipv6 in the jail and it comes up nearly as quickly as with jails off, but still with a 2-3 second extra delay. So something in the process of firing up the VNET for the jail is slowing down boot 21:21:35 I suppose I could solve this, but I've also discovered that if I want to route traffic internally to the global ipv6 addresses on the jails I need to distribute routes to every damn computer on the network somehow, plus it's not DMZ'd, so I think my next play is to either stick the entire jail subnet into a VLAN or an l2tp tunnel and hand it off to my router, which can then do both of those things 21:57:40 zip: then its an issue in your jail config? 21:57:55 maybe try with a single jail and then check the settings of this 22:00:03 yup, it was a single jail 22:00:12 I'm wondering if it's radvd taking its time to hand over an IP 22:00:16 rtadvd sorry 22:00:28 still, I'm a little surprised that sshd depends on it 22:00:40 is your ssh also jailed? 22:01:07 you should be able to ssh in the OS immediadly, regardless of what the jails are doing 22:01:13 or its an network conflict or something like that 22:24:13 perhaps it's that 22:24:26 this is my first time adding routing within a home network 22:43:05 zip: sshd is supposed to come up before jails 22:43:58 (the host system's) 22:47:45 yeah, that's what I expected as well 22:47:56 yet, I get a connection refused 22:48:15 I suspect I'm going to have to haul a monitor over to the machine and poke at it 22:49:03 well, first I think I'm going to tunnel traffic back to the router so that the router just hands over IPs via slaac and dhcp so that the machine itself doesn't have to be doing any routing, so that I can DMZ my services 22:49:32 well, I suppose I could dmz them with pf, but I figure it's easier this way 22:49:45 present status: agonising about whether to backhaul over vlan, vxlan, gre, l2tp... 22:51:55 zip: just to make sure, are you using the standard jail.conf stuff? 22:54:12 you can do `rcorder /etc/rc.d/* /usr/local/etc/rc.d/*` and check if sshd comes before jail 22:54:53 sure 22:57:06 looks like I have both sshd and openssh running before jail, although they also report that they're the same PID.. 23:00:39 oh for... it's coming up without an ipv4 router, that can't be helping 23:01:19 this is a large part of why I'd rather not be faffing with routing on this machine, a lot of things have to go right 23:02:17 I do have the gateway set in `rc.conf` 23:26:06 Has anyone successfully set up an IPv6 road warrior scenario with swanctl/openiked-portable using Android clients? 23:28:41 I cannot make it work. For IPv6 support, I tried StrongSwan 6 and openiked-portable on FreeBSD 14-Stable. It works with Linux clients, but not with Android. It seems like ESP packets are fine, but not UDP packets on port 4500 23:45:29 i tried a couple of times and never got it either. wireguard was a lot easier :| 23:59:05 spmzt: pure IKE/ESP Android tunnels are very very wonky 23:59:28 you really want to encapsulate it in L2TP but that means mpd