01:16:14 Hi I'm trying to use the new service jails but they don't seem to start at all 01:16:37 Is it only specific services that work with it? 01:27:08 Ohh I see, you have to actually enable the service as usual before too. I just assumed if you had _svcj="YES" it would enable the service to run it in the jail 05:42:59 Anyone have experience lumping mixed sized disks together into a single large ZFS zpool (raidz2 of similar) ? 05:56:10 SponiX: Yeah. You just build the ZFS into partitions sized for the smallest disk. 05:56:31 It wastes a little space but it works well. 06:03:28 Is there a better way, like grouping a handful to a dozen of the same sized disks as one like vdev, and then lumping the vdevs together from there? 06:03:52 My smallest disks are 3TB, and my largest are like 16-20TB 06:03:56 you could probably do that with ccd(4), but you wouldn't be able to boot from it, and i dislike that sort of complexity in my storage, personally 06:04:54 Yeah, I have a couple of 2TB NVME drives to do the OS/boot itself. These array(s) would be specifically for data/Media only 06:10:45 ivy: I didn't know this ccd existed. It might be helpful 06:11:17 I would rather do it all within ZFS, but mixing it with ccd could actually give me better solutions 07:07:32 afaik, diffrent sizes for root vdevs don't matter, but you don't have redundancy between those root vdevs, only inside a vdev (and only if it isn't just a single disk) 22:30:59 SponiX: riffing off of what nimaje noted, you might string together things that are at least mirrors, and that might use more of your available space while still protecting your data. 22:39:49 heya. i can't for the life of me figure out pf. i'm trying to set up my rules so that i block all traffic except explicitly a few ports, and then i block all traffic for one user, and reroute its traffic to a wireguard vpn. however... the line `block user ` seems to screw up the rest of the config, stopping me from doing basic stuff like ping (permission denied), etc. what's up? 22:39:55 the pf.conf in question: https://paste.c-net.org/InducedPiercing 22:49:22 hmjsp: Hm, I didn't know you could tie rules to users. That's kind of interesting. 22:50:11 Neat. 22:50:56 it's not working tho ToT 22:51:02 hmjsp: Is it just that user who's impacted for the rest of the rules? Or everyone? 22:51:19 everyone is impacted for some reason, that's what's puzzling me 22:51:57 if i uncomment the last two lines, ping works. if i uncomment "block user transmission"... then even as root i can no longer ping 22:52:51 which doesn't make sense, as man page example is similar: block out proto { tcp, udp } all 22:52:52 pass out proto { tcp, udp } all user { < 1000, dhartmei } 22:53:11 yeah, just looking at that now 22:53:31 Is it your syntax? You don't match the man page there. 22:53:57 I'm just getting into pf, although it's a lot like the ipf I used to use. 22:54:34 im not sure, dont think so 22:54:38 block all user { transmission } 22:54:44 doesn't change anything 22:55:13 it's weird, i copied a simple ruleset (not this one) from an openbsd box and it did not work the same 22:55:16 not sure why 22:56:15 OpenBSD is a different, sometimes newer PF. But that syntax is still not quite the same. Try exactly what they show maybe. At a guess, block out proto { tcp, udp } all user { transmission } 22:56:30 Although maybe {} is just a list syntax and you don't need it for a single entry? Dunno. 22:56:48 that's what i thought too 22:56:52 that's how it usually is 22:56:59 grep -r user . shows nothing un /usr/share/examples/pf on 14, looking on 15 22:57:36 oh! 22:57:38 it works? 22:57:44 with proto... 22:57:48 Does it? 22:58:02 yeah... mysterious O_o 22:58:06 hold on lemme double check 22:58:09 Good idea. 22:58:25 But yeah, when in doubt, slavishly adhere to the examples and only deviate from there once you have something that works. 22:59:11 Also, I wonder if what you really want is a block if what you really want is to redirect/twist/whatever. 22:59:31 ok so transmission is blocked and my user can ping fine... now i need to check vpn config 22:59:44 i want to block all trafic that isnt redirected through the vpn interface 22:59:45 That was anglolinguicide, but I wonder if maybe what you want is a modification if you catch packets from the user, not a block and then later something else. 23:00:07 hmjsp: tcpdump might help here, seeing what packets make it where 23:00:36 oops my client pooped itself 23:00:37 (I forget if it's last-rule-wins and you have to specify "quick" of if that's some other syntax.) 23:01:22 it's last rule wins 23:01:25 so block by default 23:01:32 and then pass through vpn 23:01:45 quick can be used to override last rule wins 23:01:47 ah, so this *is* a system where you want "quick" if you need to apply immediately? 23:01:50 right 23:01:52 good 23:01:58 yeah 23:02:13 I'm playing around with some simple PF this weekend. I've been mucking about with ipfw but it doesn't really make me happy. 23:02:41 if the idea is to force all your torrent traffic through a vpn, you may want to consider running transmission in a different fib, rather than trying to do it with a firewall 23:02:57 ivy++ even better idea 23:03:14 a different what now 23:03:20 FWIW, I found this the other day and was blown away by now minimal it was: https://blog.frankleonhardt.com/2020/networking-freebsd-jails/ 23:03:22 im new to freebsd :') 23:03:26 hmjsp: routing table == fib 23:03:35 fib = routing table... let me find an example 23:03:53 https://man.freebsd.org/cgi/man.cgi?query=setfib is useful 23:03:57 i mean it's sunken cost atp i just want this to work lol 23:04:15 hmjsp: I think ivy is right - you want to impose a routing table for this. 23:04:48 probably the best approach but... i may as well learn why my current pf approach is not working while im at it 23:06:16 hmjsp: FWIW, at a fairly recent job, I was able to do something with setfib in seven lines that was way more complex with other options. I was compelled to do it on Ubuntu despite it being way more effort for no additional functionality, despite my showing a lovely demo using FreeBSD. 23:06:49 hmjsp: https://people.freebsd.org/~ivy/app-in-fib.txt 23:07:55 ivy: can i do _fib=? 23:07:59 is that how this works? 23:08:19 it needs to be a service that exists in rc.d, so anything that you do foo_enable=YES, you can set foo_fib 23:08:35 waow, neat 23:08:36 so simple 23:08:41 see rc.conf(5), there's a bunch of service-specific options 23:08:59 suppose it's not a service tho (not my scenario atm, but im curious), what would i do? 23:09:10 setfib 1 23:09:15 waow 23:09:25 okay, thanks! 23:09:36 i'll try what you just showed me and see if it works 23:09:47 hmjsp: Here's some random thrashing I was doing during the process mentioned. Basically, I needed different default routes for different things listening on the same /24. Look how easy it was with FreeBSD: https://termbin.com/i94r 23:10:30 note, if you're using wg-quick, you'll need to do it a different way... probably using 'ifconfig fib 1' as a post-up command. i prefer not to use wg-quick as i find it simpler to configure that in rc.conf 23:10:51 mason: the sheer reduction xD 23:11:15 ivy: yeah i avoided wg-quick because it routes the whole network to the vpn 23:11:20 which is not what i want at all 23:11:29 hmjsp: I made it work in Ubuntu, but it wasn't the right call. 23:22:02 wait ivy im still confused. i got rerouting to wg0 correctly now with fib, but im a bit lost on where to put the peer's ip now...? since it's not in the wireguard config itself... 23:24:19 hmjsp: the wireguard config doesn't change, you're just putting the local ip address in rc.conf. (that's the address you'd put in Address= if you were using wg-quick) 23:26:07 in place of this? "ifconfig_wg_vpn="inet 1.2.3.4/32 fib 1""? 23:27:09 yes, replace 1.2.3.4/32 with the local ip address for inside the tunnel (not your public address) 23:29:51 ivy: still doesn't work. tcpdump on wg0 shows sending, but no receiving 23:31:08 https://paste.c-net.org/GripsSensor 23:31:42 is the tunnel actually working? 'wg show wg0' should show traffic in both directions 23:32:08 yeah it does. in the kbs, but it does 23:32:15 transfer: 2.61 KiB received, 110.67 KiB sent 23:34:29 if you run tcpdump on the internet interface, do you see traffic for the vpn endpoint in both directions or only outgoing? 23:35:44 when i run ping, only outputting. tho previously there was SOME input, presumably the server checking if im alive? 23:35:59 but with ping it's 100% packet loss 23:36:48 hmjsp: ICMP can be funny - also try netcat/telnet to various ports 23:37:03 in both directions 23:39:17 is it possible pf is blocking traffic? you may want to add a pass rule for the wg interface and remove any user-specific rules just for testing 23:39:26 perhaps 23:39:32 lemme disable pf and see 23:39:50 ok it's definitely pf xD 23:40:14 hmjsp: Did you leave in the user block rule? 23:40:19 nop 23:40:22 not sure what it is 23:41:34 pass on $vpn_if proto tcp from any to any port $tcp_services 23:41:41 i added this at the end, and it still doesn't work 23:41:49 im not sure what's blocking it in the first place 23:41:59 well, 'proto tcp' won't allow ping 23:42:50 hmjsp: Are you testing with more than just ping? Try telnet too, to various ports. From both directions. 23:42:56 pass on $vpn_if proto { tcp, udp } from any to any port $tcp_services doesn't work either 23:43:14 mason: im trying curl and ssh also 23:43:26 non vpn traffic works 23:43:51 hmjsp: Always simplify. Those aren't bad ideas, but plain telnet is simplest. It'll send a SYN wherever. 23:44:00 okok 23:44:08 just not too familiar 23:44:16 telnet foo 1234 23:44:59 or nc foo 1234 I guess 23:45:08 this probably isn't related to the current issue, but you can't really do port-specific outbound filtering for BT because peers can use any port they want... so that pass rule probably wants to be a pass in rule, with a blanket pass out 23:45:45 ah ok thank you 23:47:41 oh! 23:47:53 doing a blanket pass out made everything work 23:47:58 \o/ 23:48:07 tcpdump got a... dump of inbound traffic 23:48:08 xD 23:49:23 okay this is good. thanks a lot yall :) 23:49:26 fib is neat 23:49:32 yes indeed it is 23:49:34 is it only freebsd with it? 23:49:38 or other bsds? 23:50:00 https://man.netbsd.org/routed.8 23:50:10 same on OpenBSD 23:50:33 Linux has a similar feature but it works in a strange, harder-to-use way for no apparent reason 23:50:49 mason: neato 23:50:51 Hm, trying to figure out how to specify different tables with that NetBSD thing. 23:50:57 linux is very silly 23:51:20 oh well im way overdue on sleeping 23:51:23 mason: i'm not sure routed(8) is the right manpage? that's the RIP daemon 23:51:26 hmjsp: You can do similar with Linux using namespaces. 23:51:31 but i can rest easy with this solved 23:51:37 gn :) 23:51:45 ivy: Yeah, I saw "tables" and it seemed like it might be the right thing, but I'm thinking it's not. 23:51:50 hmjsp: enjoy 23:51:55 o/ 23:52:17 we have routed too, although if i have my way, it'll be gone before 16.0 :-) 23:53:00 I'm not immediately seeing how to go about it in NetBSd. 23:55:26 MultiProtocol Label Switching seems interesting, but isn't quite right. 23:55:49 mpls is for the other sort of vpn :-) (virtual wires) 23:55:55 Well. I have and like setfib, so I won't worry about it too much I guess. :) 23:56:17 incidentally, one of this year's GSoC projects is an MPLS implementation for freebsd 23:56:28 Oh, neat. 23:56:46 Port of NetBSD, or ground up? 23:58:06 i'm not sure, the project idea mentions the openbsd and netbsd implementations but doesn't directly suggest porting those. although it would seem silly not to take at least some code 23:58:20 mm