00:37:12 rwp: so AFAIK this isn't specific to QEMU - it only uses mechanisms bhyve also supports 00:37:35 and bhyve does provide virtio-9p like in https://www.sisyphus.de/post/2024-01-06-9p-for-freebsd-bhyve/ 01:01:08 Cool! TIL! 01:15:30 yeah, I just learned it recently too 06:15:15 tmux users FYI and warning, today's quarterly pkg upgrade breaks client server compatibility. Must kill the old server and restart with the new one. 14:27:05 Hi all, I'm trying to use jail.conf(5) which supposedly supports "Wildcards". Are there examples? because the following just fails with: jail: /etc/jail.conf.d/99-external.conf line 1: *: syntax error -- https://x0.at/iKy_.txt 14:39:22 I think you are missing the hierarchical part in your setup "If hierarchical jails are defined, a partial-matching wildcard definition may be specified." and "By setting a jail's children.max parameter, processes within a jail may be able to create jails of their own. These child jails are kept in a hierarchy" 14:41:30 russian-doll-jails 😱 14:42:30 Ok, corrollary: can I get some instruction shared across multiple jails (external/internal dichotomy) without repeating every line in 99-external.conf? 14:54:33 i have to admit... freebsd feels more like a fight than linux so far heh 14:55:08 i'm getting pretty close to proxmox parity using jails but it's been a struggle for certain things. notably getting the gpu accel working and mounting cifs 15:03:13 some gpu-workloads might be easier to run by passing it to linux bhyve vm 15:04:03 does bhyve allow for multiple passthrough? 15:04:14 i usually pass a gpu to multiple containers/jails 15:04:31 CIFS is not supported, switch to NFS please 15:04:40 with regard to mounting smb/cifs i had to resort to shoehorning rclone into everything 15:05:14 mzar: yeah i noticed :) but smb is better for me. only one infrastructure to maintain with samba_server 15:05:45 OK, so why are you pushing migration from Proxmox ? 15:05:54 licensing issues ? 15:25:19 no. just seeing if it can be done 15:26:02 speaking of which. i need to take a look at gomuks again heh 17:18:55 it can't be done 17:19:35 ? 17:19:56 oh 17:29:04 What? why? 17:35:30 rtprio, stuff about CIFS 17:36:18 hm 17:36:54 does ffmpeg need the libc6 shim even if you were to use say something like an intel arc? 17:37:04 or is this an nvidia specific thing? 17:37:38 now that i think about it. is an arc310 even an option for fbsd? 17:39:30 oh. look at the forums it seems like all of this will be shoehorned into the drm stuff .. that is pretty much using linux stuff in fbsd isn't it? 19:44:05 well.. this is unique.. after an update it seems like during boot the system is getting hung on the nvidia-drm 19:48:02 https://his.macer.life/@macer/115923488169183679 20:03:03 hm. seems like it's not rebooting even after disabling the nvidia module.... not sure what on earth happened here. 20:19:35 Macer: I would boot into single user mode and perform fsck -f -y 20:19:51 it's zfs 20:20:13 but yeah that seemed to be the issue maybe. took quite a while for it to boot after i had to hard reset it. not sure if zfs does anything there to check things out but it finally boot 20:22:17 Macer: then you are nearly fscked. try `zpool scrub'. 20:22:35 what does zpool status say? 20:47:18 that it's fine 20:47:27 it just took a while to mount it 20:47:49 but right now i'm trying to figure out why a host with a 10.0.0.x and a jail with 10.0.0.x can't ping each other 20:48:47 Macer: What are you doing with gomuks? 20:49:32 skered: well. i used to use it to connect to bluebubbles in a terminal 20:49:37 but now.. nothing :) 20:50:12 Because of the last update? 20:50:30 Macer: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=292010 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=292371 20:51:27 However, if you need a quick fix building from source just works too. Just use the master branch. main currrently tracks the web server. 20:51:30 yeah i noticed that 20:51:35 it doesn't have index.html included 20:51:59 i think i tried that at one point but the ver of go in the pkg repo was too old 20:52:23 i don't get this.. the host cna ping 10.0.0.10 the jail can ping it also. but they can't ping each other. 20:52:33 i guess maybe because they're bridged off the same interface? 20:52:37 I can say it's fine now. That's what I used while trying to figure out how to get 25.12 to build with ports. 20:53:07 ah 20:53:14 i may try it once i sort out this jail issue i'm having 20:54:57 More or less the old version (from master branch) works. main (including 25.12) is a rewritten version with a a web interface but does provide a terminal version. The terminal version isn't complete and is buggy. 20:57:54 yah. i knew that part which is why i was happy the pkg repo had the older version heh 20:58:16 i don't even think it has a maintainer. that was some sort of gitbot that built it? 21:08:41 how is networking configured on that system? 21:10:33 Macer: I dunno how that update landed. The commit message was more than just dropping maintainership. 21:12:48 nimaje: i have adedicated 10gb sfp+ card for the jails. i'm usinv vnet for the jails as well. the nic on the host side has a 10.0.0.31 ip ... in the jail i use rc.conf to configure an admin network and created an alias for the other one using 10.0.0.30 21:13:00 both sides can ping an external ip on another server. but neither can ping each other 21:16:14 ok and 10.0.0.10 does routing for that 10.0.0.10/24 subnet? 21:16:31 there is no routing. i'm just using it as a storage backend 21:17:50 or is there some switch connecting the nic for the host and the jail? 21:18:56 same nic connected to the same port on the switch.. but i'm not really too keen on how fbsd networking would handle that with a vnet 21:19:14 i'm using bastille and that created a bridge. the bridge is using the host interface which has the 10.0.0.31 ip 21:19:26 (on the host locally) 21:19:39 i'm not sure if there is maybe something special i need to do to the bridge for this 21:20:25 it's not just pinging. no traffic is traveling between the host and the jail 21:21:13 i'd love to just nullfs mount this but nullfs doesn't do permissions correctly when mounted into a jail .. you can't force uid/gid 21:21:57 can you share the output of ifconfig and ifconfig -j as well as netstat -rWn and netstat -j -rWn ? 21:31:33 https://pastebin.com/1UBA1G1w 21:31:38 that's the jail 21:31:54 ifconfig is huge on this server. let me shut down the other jails 21:35:55 https://pastebin.com/gfiN9ypC 21:35:58 that's the host 21:38:41 https://pastebin.com/ThtmCadD 21:38:43 netstat on host 21:41:48 https://pastebin.com/VkQRF6rt 21:41:53 and netstat in the jail 21:42:05 hm, ix0 shouldn't have an ip set when it is part of a bridge, view a bridge as a virtual switch and the member interfaces as the ports of that switch, the individual ports wouldn't have ip addresses, maybe the switch itself if it is managed, like that the ip should be on the bridge and not on one of its members 21:43:37 yeah, the host sends packages for 10.0.0.10/24 outside via ix0 21:48:55 This'll sound lik a stupid question, I promise it is... is WITNESS a KASSERT? 21:50:42 no 21:50:50 WITNESS observes locking stuff 21:50:59 Macer, can't you just create a user on the host with the desired UID/GID for whoever will access the dir in the jail, and then nullfs mount it? I mean, this isn't ideal (who would want to create duplicate users on the host just so they map to a jail user?), but it should work just fine 21:51:19 scoobybejesus: it's using AD 21:51:29 or shoudl i say.. they're on different ADs 21:51:36 Ah, interesting... 21:52:28 nimaje: hm. ok. i guess that makes sense 21:52:48 now the question is.. how do i add the host as a member of that bridge 21:54:20 If the bridge host needs an IP address, set it on the bridge interface, not on the member interfaces. The address can be set statically or via DHCP. This example sets a static IP address: 21:54:24 ohhhhhhh... ok 21:57:17 now i'm starting to wonder if that's even possible with bastille 21:59:49 hm, how did you create that bridge? 22:17:25 bastille created it.. but i think i know where it came off the rails 22:17:28 https://forums.freebsd.org/threads/manual-network-setup-required-for-bastille-jails.100494/ 22:17:33 i think that's what i'm looking for 22:20:00 * Macer facepalms 22:20:19 ok let me set that up and start converting jails. lol 22:23:52 nimaje: thanks for the help. that pointed me in the right direction 22:26:24 reads like bastille produces broken network setups 22:32:08 well... 22:32:22 it does some weirdness where it will automatically make a bridge on boot 22:32:44 in other words.. i should have used -B instead of -V 22:33:00 i don't understand how i missed that 23:14:47 OK... got the whole network config right in rc.conf (i think) and the bridge has the 2 ips i need so lets see if the forum edit on the jails will make them start properly heh 23:29:45 huh, just realized bhyve has a "grub" loader built in, in addition to the "uefi" one that is what I used for NixOS 23:31:32 though I suppose for my "have Linux's root FS on 9p" idea, it may be easier to still just have a small disk image for /boot & let the distro inside the VM manage that 23:32:04 even if I guess I *could* get away with kernel & initrd hosted elsewhere if I used grub2-bhyve