rwp: so AFAIK this isn't specific to QEMU - it only uses mechanisms bhyve also supports

and bhyve does provide virtio-9p like in sisyphus.de/post/2024-01-06-9p-for-freebsd-bhyve

Cool! TIL!

yeah, I just learned it recently too

tmux users FYI and warning, today's quarterly pkg upgrade breaks client server compatibility. Must kill the old server and restart with the new one.

Hi all, I'm trying to use jail.conf(5) which supposedly supports "Wildcards". Are there examples? because the following just fails with: jail: /etc/jail.conf.d/99-external.conf line 1: *: syntax error -- x0.at/iKy_.txt

I think you are missing the hierarchical part in your setup "If hierarchical jails are defined, a partial-matching wildcard definition may be specified." and "By setting a jail's children.max parameter, processes within a jail may be able to create jails of their own. These child jails are kept in a hierarchy"

russian-doll-jails 😱

Ok, corrollary: can I get some instruction shared across multiple jails (external/internal dichotomy) without repeating every line in 99-external.conf?

i have to admit... freebsd feels more like a fight than linux so far heh

i'm getting pretty close to proxmox parity using jails but it's been a struggle for certain things. notably getting the gpu accel working and mounting cifs

some gpu-workloads might be easier to run by passing it to linux bhyve vm

does bhyve allow for multiple passthrough?

i usually pass a gpu to multiple containers/jails

CIFS is not supported, switch to NFS please

with regard to mounting smb/cifs i had to resort to shoehorning rclone into everything

mzar: yeah i noticed :) but smb is better for me. only one infrastructure to maintain with samba_server

OK, so why are you pushing migration from Proxmox ?

licensing issues ?

no. just seeing if it can be done

speaking of which. i need to take a look at gomuks again heh

it can't be done

?

oh

What? why?

rtprio, stuff about CIFS

hm

does ffmpeg need the libc6 shim even if you were to use say something like an intel arc?

or is this an nvidia specific thing?

now that i think about it. is an arc310 even an option for fbsd?

oh. look at the forums it seems like all of this will be shoehorned into the drm stuff .. that is pretty much using linux stuff in fbsd isn't it?

well.. this is unique.. after an update it seems like during boot the system is getting hung on the nvidia-drm

hm. seems like it's not rebooting even after disabling the nvidia module.... not sure what on earth happened here.

Macer: I would boot into single user mode and perform fsck -f -y

it's zfs

but yeah that seemed to be the issue maybe. took quite a while for it to boot after i had to hard reset it. not sure if zfs does anything there to check things out but it finally boot

Macer: then you are nearly fscked. try `zpool scrub'.

what does zpool status say?

that it's fine

it just took a while to mount it

but right now i'm trying to figure out why a host with a 10.0.0.x and a jail with 10.0.0.x can't ping each other

Macer: What are you doing with gomuks?

skered: well. i used to use it to connect to bluebubbles in a terminal

but now.. nothing :)

Because of the last update?

Macer: bugs.freebsd.org/bugzilla/show_bug.cgi?id=292010 bugs.freebsd.org/bugzilla/show_bug.cgi?id=292371

However, if you need a quick fix building from source just works too. Just use the master branch. main currrently tracks the web server.

yeah i noticed that

it doesn't have index.html included

i think i tried that at one point but the ver of go in the pkg repo was too old

i don't get this.. the host cna ping 10.0.0.10 the jail can ping it also. but they can't ping each other.

i guess maybe because they're bridged off the same interface?

I can say it's fine now. That's what I used while trying to figure out how to get 25.12 to build with ports.

ah

i may try it once i sort out this jail issue i'm having

More or less the old version (from master branch) works. main (including 25.12) is a rewritten version with a a web interface but does provide a terminal version. The terminal version isn't complete and is buggy.

yah. i knew that part which is why i was happy the pkg repo had the older version heh

i don't even think it has a maintainer. that was some sort of gitbot that built it?

how is networking configured on that system?

Macer: I dunno how that update landed. The commit message was more than just dropping maintainership.

nimaje: i have adedicated 10gb sfp+ card for the jails. i'm usinv vnet for the jails as well. the nic on the host side has a 10.0.0.31 ip ... in the jail i use rc.conf to configure an admin network and created an alias for the other one using 10.0.0.30

both sides can ping an external ip on another server. but neither can ping each other

ok and 10.0.0.10 does routing for that 10.0.0.10/24 subnet?

there is no routing. i'm just using it as a storage backend

or is there some switch connecting the nic for the host and the jail?

same nic connected to the same port on the switch.. but i'm not really too keen on how fbsd networking would handle that with a vnet

i'm using bastille and that created a bridge. the bridge is using the host interface which has the 10.0.0.31 ip

(on the host locally)

i'm not sure if there is maybe something special i need to do to the bridge for this

it's not just pinging. no traffic is traveling between the host and the jail

i'd love to just nullfs mount this but nullfs doesn't do permissions correctly when mounted into a jail .. you can't force uid/gid

can you share the output of ifconfig and ifconfig -j <jail name> as well as netstat -rWn and netstat -j <jail name> -rWn ?

that's the jail

ifconfig is huge on this server. let me shut down the other jails

that's the host

netstat on host

and netstat in the jail

hm, ix0 shouldn't have an ip set when it is part of a bridge, view a bridge as a virtual switch and the member interfaces as the ports of that switch, the individual ports wouldn't have ip addresses, maybe the switch itself if it is managed, like that the ip should be on the bridge and not on one of its members

yeah, the host sends packages for 10.0.0.10/24 outside via ix0

This'll sound lik a stupid question, I promise it is... is WITNESS a KASSERT?

no

WITNESS observes locking stuff

Macer, can't you just create a user on the host with the desired UID/GID for whoever will access the dir in the jail, and then nullfs mount it? I mean, this isn't ideal (who would want to create duplicate users on the host just so they map to a jail user?), but it should work just fine

scoobybejesus: it's using AD

or shoudl i say.. they're on different ADs

Ah, interesting...

nimaje: hm. ok. i guess that makes sense

now the question is.. how do i add the host as a member of that bridge

If the bridge host needs an IP address, set it on the bridge interface, not on the member interfaces. The address can be set statically or via DHCP. This example sets a static IP address:

ohhhhhhh... ok

now i'm starting to wonder if that's even possible with bastille

hm, how did you create that bridge?

bastille created it.. but i think i know where it came off the rails

i think that's what i'm looking for

ok let me set that up and start converting jails. lol

nimaje: thanks for the help. that pointed me in the right direction

reads like bastille produces broken network setups

well...

it does some weirdness where it will automatically make a bridge on boot

in other words.. i should have used -B instead of -V

i don't understand how i missed that

OK... got the whole network config right in rc.conf (i think) and the bridge has the 2 ips i need so lets see if the forum edit on the jails will make them start properly heh

huh, just realized bhyve has a "grub" loader built in, in addition to the "uefi" one that is what I used for NixOS

though I suppose for my "have Linux's root FS on 9p" idea, it may be easier to still just have a small disk image for /boot & let the distro inside the VM manage that

even if I guess I *could* get away with kernel & initrd hosted elsewhere if I used grub2-bhyve