Is there a good reason find's user/group column is so long? Other than it's been that way forever.

find's -ls.

probably the max length of a username or groupname

guessing

also: never used -ls

Hey all, I'm looking for opinions: is there any reason or include or exclude boot environments from my zroot backups (handled by zfs-autobackup)?

Two disadvantages I found so far: the snapshots for each BE are huge on my external backup disk, and second have a zfs hold on them so I can't easily run 'bectl destroy ...' to clean them up.

tehpeh: If the system boots and runs fine with no reason to to use any of the backups, you're completely free to delete them.

ek: thanks, that's what I was thinking. Also with a years worth of auto-snapshots I could rollback anyway

...if I ever needed to

tehpeh: Depending on what the auto-snapshots are of, sure. If you're strictly talking about automated snapshots that happen before, say, a "freebsd-update" or something, those are safe to destroy if you're certain the currently active system is working. Old boot environment snapshots (especially really old) almost never need to be kept.

I usually keep just the previous BE snapshot. Active BE and one snapshot before "just in case." Otherwise, I wipe out anything older since it's basically useless (to me.)

the auto-snapshots are created by zfs-autobackup via a daily cron job, so in theory they should cover the same peroid of any freebsd-update BE created during that time

ooh, I'm keeping the last year of BEs... that does seem like overkill

I don't auto-remove them or anything. I just check periodically to make sure I'm not housing massive amounts of unneeded snapshots. Especially for the BE's.

I only clean them up maybe once a quarter or something. After a lengthy run of a stable system, of course.

ek:sounds like a good plan, thanks

tehpeh: Sure thing. Hopefully, it all works out! (I don't see why it wouldn't)

For those following, if you want to exclude BEs from snapshots made by zfs-autobackup, you can do that with, for example:

zfs set autobackup:local=parent zroot/ROOT

zfs set autobackup:local=true zroot/ROOT/default

Good to know, but I wouldn't recommend not backing up BE's. Of course, with autobackup, they may not be needed if the BE isn't being changed.

I still use my own custom snapshot script to backup everything I need to via cron. Hasn't failed me yet. Of course, there's been many projects providing the same thing over the years. I just haven't had a reason to migrate (yet.)

is there an offtopic channel

yes

freebsd-social

ty

Any CBSD users here? I'm trying to sort out my networking, and running into a problem.

tl;dr I'd like to have both jails and bhyve on 10.0.0.0/16 ... *that* part is working, I can even access the internet, ping everything on it, etc.

But local_unbound, listening on 10.0.0.1 (and with whitelisted access from 10.0.0.0/16) seems to be unreachable from VMs (but not from jails), with "connection refused".

/3

is anyone running redis server in a FreeBSD Jail (Yes, I know about "valkey").. what magic do you need to do with the "bind" and "tcp-backlog" to get it to run?

I have to use 'bind <JAIL IP>' ? Seems 'bind *' makes it fail immediately

I'm trying to use freebsd-update to update a 13.2-RELEASE box to 13.4, and one of the update files is not present. freebsd-update -v debug ... says:

update2.freebsd.org/13.2-RELEASE/am…6dc1021ab7b43ef1b7291e6f59cc1cc.gz: 404 Error (ignored)

ideas on how I can fis this?

does freebsd-update -r 13.3-RELEASE fetch install give same result?

i meant freebsd-update -r 13.3-RELEASE upgrade

I'll try now...

for non-VNET jails.. is '127.0.0.1' in the Jail treated as the Jail's IP itself to everything else?

Yes, I get the same error trying to upgrade from 13.2 to 13.3

pertho: within the jail do a ifconfig

voy4g3r2: bastille0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384 options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet 10.100.0.4 netmask 0xffffff00 groups:lo

I guess the jail treats' 10.100.0.4' as loopback/127.0.0.1 ?

and when you do bind * redis goes.. piss off?

I wonder if it'd be more secure to just export a socket from one jail to allow another jail to access rather than using the network stack?

voy4g3r2: it was saying couldn't look up *::

personally, i setup epairs for each jail and give it a unique ip address and use that ip address

assuming this jail, you are setting up, is a VNET one?\

yeah I'm not using VNET in this case

setting redis to use: "bind 127.0.0.1 -::1" appears to make it listen on the bastille0 IP (10.100.0.4)

pertho: from the host you can mount the FS with the sockest from one jail to another using nullfs

or that

pertho: correct, which will get "lost" because it is like what does 127.0.0.1 mean

yeah .. socket has less overhead than a network connection.. (less to build up/tear down) yeah?

that is an evaluation answer.. network stack, is easier for me.. because managing ip addresses in general are easier to me

the "overhead" does not become a concern until it is :)

it's also commont practice to have additional lo interface in the jail and run services on that address, it can be 127.0.x.1, where x≠0

voy4g3r2: I'm not surprised that upgrading to 13.3 also failed, as the file it's looking for is in the 13.2/ subdir.

mzar: there is lo0 but no addresses bound to it

so you have to assign one

whole 127.0.0.0/8 is for loopbacks

but it has to be something else than lo0

pertho: it looks like voy4g3r2 is also loopback

so you have one

this is how i setup vnet jails

yeah I don't have a specific need for a VNET jail in this case

yes, usually we don't need vnet

when you have loopback, all the services inside the jail will bind on the address of that loopback instead of 127.0.0.1

sometimes VNET is required though, for example if you need to setup poudriere in the jail

if only if it were that complex :D Nah, I'm just running separate nginx, php, redis and mysql jails

gitmir.geo.freebsd.org has IPv6 address 2a02:80:0:3ffd::24ca:1 and responds "fatal: Could not read from remote repository." for git pull. Other IPs in gitmir.geo.freebsd.org work.

git pull is via ssh://anongit⊙gfo/ports.git

pertho: I don't recall having any issues running Redis in a jail.

But it's been a while.

pertho: You use bastille?

i wonder who uses 127/8 anything other than just localhost

s/anything/for anything/

systemd-resolvd uses 127.0.0.53

besides that?

it also doesn't help that things assume

127/8 = localhost

not just 127.0.0.1

DarkUranium: yeah, using bastille

the templates are very cool

what's the best place to nullfs a socket from one jail into another? does it go in the jail's fstab or..?

pertho: Yeah, kind of deciding between {Bastille,AppJail}+vm_bhyve, or CBSD.

They all have templates of one sort or another.

I'd only use bhyve if the apps I'm trying to run don't run in FreeBSD

Same. And unfortunately, I have a few of those.

(or you actually need another OS)

(Linuxisms ...)

yeah.. Linuxisms.. I hate 'em

seeing now as more script kiddies are making malware for Linux. FreeBSD might be a much safer bet for security, plus.. Jails..

I've used FreeBSD for *aeons*.

But I do have a handful of Linux servers, mostly for stuff where pretty much the only distribution channel is Docker .....

Well, and gameservers. Those, I can *kind of* excuse, I guess?

(it's better than just Windows, at least)

I'm making a proof-of-concept FreeBSD server with jails for $WORK.. one of the other sysadmins at work constantly gives me grief for using BSD, even though OpenBSD happily passed a ton of packets seamlessly and without issue for the past 12 years.. Want to make this rock solid and fast. Next step is getting MySQL and ZFS happy with each other

Nice!

sadly, our dual OpenBSD failover (carp(4)) firewalls are being retired when we move to the new office next week. New company is using FortiGate and Juniper (*vomit*).. but won't be my responsibility any more :D

I'm moving my own stuff, which is currently a mishmash, to reproducible things. I figured I'll keep a strict directory repr (+ loop mounts to "adapt" where needed), so that I can backup trivially.

Basically separating `system` (non-persistent, possibly read-only), `cache` (persistent, but *not* backed up), `config` (persistent, possibly read-only [depending on the application, some need to write this]), and `data` (similar to config, but always read-write, and generally much bigger than config *and* updated more often)

but I wanted to make this PoC to see if it would blow our Ubuntu Linux servers out of the water.. and ZFS.. can take snapshots every hour.. yeah Linux has ZFS support but I bet it's baked into FreeBSD better than it is on Linux

Yeah, it is ^^

zfs just saved my ass when my webhost went downhill (straight off a cliff in a manner of days, really)

nice!

Basically, I did a zfs send to a backup server, and I do incremental sends periodically.

I heard it's better security if the backup server can pull

(the server's still on that host until I rent a new one ... still evaluating them, but I'd *first* like to decide on above jail+bhyve options)

That makes sense, because pushing onto backup server means that the other one needs direct access. Write, in a sense.

If the backup server pulls, you can even have it behind NAT!

but yeah the hope is FreeBSD with all the web nonsense and moving bits in their own jails.. for security

That's *exactly* what I've been doing for years now :)

The only thing running on the "host" is currently a nginx reverse proxy (and next time, I'm thinking I'll jail that too).

(and I'll probably use HAproxy instead)

(not really unhappy with nginx, as much as I've been meaning to try HAproxy --- I have another similar setup that's based on it)

right now I got (for the jails) 1) nginx reverse proxy (does letsencrypt, SSL termination) 2) php/web jail (runs nginx and php_fpm) 3) redis jail (just runs redis cache) and will have 4) mysql jail.. you reckon it's best to keep them separated like that?

I have like ... 15 or something. I lost count.

we've been using nginx for 14 years.. we're used to it. Been meaning to try out 'caddy' though.. but we lock down where PHP can run (for obvious reasons)

pertho: caddy does kind of feel weird but it 2 lines per vhost sure beats the 20 or so lines of nginx boilerplate

welp

# iocage list -h | wc -l

40

pertho: I guess 15 is ... wrong.

(and yes, iocage .... the server is *OLD*)

\o/

(not happy with iocage though)

rtprio: yeah nginx.. so much boilerplate. What's the performance of caddy, in your experience?

rtprio: I make use of nginx's includes to do that part.

can it handle high amount of connections/traffic?

What I've actually been meaning to do is make use of Dhall, Cue, or Jsonnet, and auto-(re)generate the full nginx config on changes.

(plus DNS and so on, I'd like to be able to access the jails by name via DNS)

pertho: it handles my homelab, including streaming jellyfin, so fine for my use

My own never even shows up in `top` (and a bunch of those jails *also* run nginx, mind --- mostly to adapt things for applications that rely on a reverse proxy)

(also PHP, etc)

DarkUranium: i hide the boilerplate in puppet, but it's still boilerplate

Fair.

I don't mind the boilerplate. Nginx is a good balance between Apache's .... whatever-you-might-call-it and Caddy's implicit assumptions.

pertho: my only complaint with caddy is it seems to take a long time with a soft restart

Quite happy with HAproxy too, I only have one nitpick with it, but it doesn't affect production.

rtprio: how long is a long time?

more than a minute

rtprio: I've never used Puppet, how does it operate? Is it declaration-based like Terraform (i.e. you describe the full system state and it kind of does diffs), or how does it work?

DarkUranium: we have much, much nginx boilerplate in Ansible (used to be Puppet 4 but we ditched Puppet)

Ditto for Ansible, I guess :D

service caddy restart is fast; caddy reload feels like it times out

DarkUranium: yeah, puppet changes your system to reflect its concept of state. i like the syntax better than terraform

Nice.

TBH, I've been kind of tempted to integrate this entire thing w/ Nomad, but ... extending Nomad with custom executors (to be able to adds jails + bhyve) is a PITA.

And I'd *still* need some custom work anyway, for the reverse proxy.

I recently installed freebsd on my desktop, and blindly assumed I could use binary packages for everything. But it /sounds/ like ports are maybe preferred? docs.freebsd.org/en/books/handbook/ports/#ports-using mentions you shouldn't mix ports and binary packages. If so, is there a way to automatically reinstall the binary packages to port equivalents?

I don't like Puppet now that I've switched to Ansible. The problem with Puppet is, if you're ever dependent on an external Source of Truth (tm) - like over https.. and that source of truth goes down.. Puppet will merrifly bork your server config with bad stuff :D

(when the server checks in with Puppet)

td123: I pretty much exclusively use binary packages, TBH. I only use ports if there isn't a package for a version I need (yet) or somesuch.

pertho: damn

td123: you can mix everything as far as you are able to main it

In my case, it'd be on the same server, so that's less of a problem.

s/main/maintain

pertho: yeah, the source of truth is the puppet server, not some external http server

but coding everything in yaml, jfc there has to be a better way

But yeah, I've been half-tempted to make my own thing, ha.

just having data (puppet's hiera) in yaml is bad enough

without inline yaml ansible scripts

DarkUranium: mzar: ok thanks, I'll keep using binary then

rtprio: You might actually enjoy one of those I've mentioned (Cue, Dhall, JSONNet). They can all export to YAML.

And/or JSON.

rtprio: not if you have hiera pulling in external things from another https server like we had :D

json is for machines, not for typing by hand

jsonnet is tolerable, but seesh

Yeah, of those 3, that's the one I liked the least.

Cue is the most limiting of these (I *really* miss functions ...), but I enjoyed toying around with it.

Haven't used it in anger yet, though.

anyway.. I don't suppose Bastille has a way to create zfs data sets? I only see zfs get/set in the template language

I guess I'd have to: 1. bastille create ... 2. Do a zfs create on the dataset in the path in /usr/local/bastille/jails/.../root/var/lib/mysql and all that, then install MySQL in the jail and start it up?

lmao

One of the servers I just setup backups for, I was wondering why a logfile took so long to back up ...

-rw-rw-r-- 1 1006 1006 9.5G Jul 2 2024 FactoryGame.log

I'd help you, but I've never, ever used Bastille, so.

DarkUranium: is that file on zfs dataset? What does zfs get compressratio give you? :D

pertho: alas, Linux, because UnrealEngine game.

(Satisfactory, in case you're curious ... and I didn't even run it for that long ...)

Actually, it's on a zfs dataset now (the backup is)

going to be nice to have web logs and cruft on lz4 compressed ZFS disks.. they'll take up such little space because most of the logs are repetitive

I actually did zstd because backups.

is there much overhead with zstd?

Compute, sort of? There is a fast mode, but it targets better compression rather than CPU time.

(though it *is* fast nevertheless)

Mind, the backup is on spinning platters, so compression is probably going to read faster than uncompressed :P

pertho: I just checked. I don't know what the compression ratio on the log file alone is (I guess I'll zstd it manually just to see --- I'm deleting it anyway), but the entire thing is 3.31x.

Mind, that includes (already-compressed) game assets, binaries, etc.

3.31x is pretty good!

pffft

FactoryGame.log : 2.68% (10207489230 => 273515071 bytes, FactoryGame.log.zst)

261MB vs 9.5GB

Taking the reciprocal, that's 37.31x :D

I understand that in order to get something financed by the FreeBSD foundation, you basically have to team up with a committer. freebsdfoundation.org/get-involved/project-proposal-overview

pertho: 1.74x after deleting this one file xD

Still not half bad for (after deleting the log) 22GB of data.

better than a kick in the teeth :P

lol

But anyway, yeah, what I was thinking was of using Cue, Dhall, or whatnot to --- essentially --- make my own mini-Ansible/Puppet.

cuelang.org ?

Especially since I want to do some high-level configs.

Yeah.

the micromanagement is strong with this one

Hush! :P

It's more, there's so much stuff I need to configure at a high level. And I haven't found a decent alternative to Terraform yet, though I might try puppet.

Is that one any good with jails & bhyve? FreeBSD in general, I guess.

use the best tool for the job.. might mean testing all kinds of things :)

tf and puppet solve different problems, really

so i find it extremely unlikely puppet would work as an alternative

puppet's ABI moves fast and breaks more things than Ansible's does

just from my experience

pertho: well, it's going to be closed source, so they solved that problem for you

puppet's abi changes with plenty of notice on major releases, sometimes

pertho: yeah, hence me going through Cue, Dhall, and JSONNet. I actually started investigating that for some game config, initially, ha.

Well, a game's data (think properties of individual objects, etc)

rtprio: oh wow.. Puppet is going closed source? o_O

Well, that rules that out :|

vox will be forking it

I didn't know P4 owned Puppet. I have trauma from their source control -_-

DarkUranium: so do i

Apparently since May 2022.

We used librarian-puppet extensively and it drove me mad

I much preferred to write my own code not use other peoples' bad buggy code

The more I hear about this mess, the more I'm thinking of just rolling my own >_<

DarkUranium: you know, that's probably for the best!

There's also the TrueNAS situation, I wonder what will become of that. Talks about forks, but nothing exists yet.

maybe you're better off rolling your own NAS

Yeah, exactly. I mean, it's not super-hard to do.

The builtin alerts in the web UI were handy, but.

pertho: to be honest, something like sftpgo and/or minio plus {rsync,rclone,zfs send|recv,rdiff-backup} would handle pretty much everything I need ATM.

wait, what happened to truenas

(i've always rolled my own nas)

rtprio: "TrueNAS Core", which was the FreeBSD-based thing, is being discontinued ... in favour of "TrueNAS Scale" which is Linux, and not even that focused on being a NAS anymore (they talk a lot about containers, etc)

The last official release was 13.0, and the last semi-official was 13.3.

FalseNAS

if it's not freebsd then why even bother with it

pertho: on the topic of rolling my own, I do have an experimental server setup, but the one thing I still haven't figured out is why my bhyve jails can't access local_unbound ...

rtprio: exactly.

but then what was the point of a (i'm assuming) web ui to run `adduser` and `zfs create`

I did recently learn of: xigmanas.com/xnaswp

But yeah.

DarkUranium: maybe use a bridge for bhyve? I'm not that familiar with bhyve, personally.

rtprio: It did do a bunch of other stuff, like NFS, smb, and (most importantly for me) a nice dashboard for various disk & resource-related alerts.

Nothing I can't live without, though --- and/or setup myself.

DarkUranium: Prometheus and Grafana perhaps? :)

Yeah, I didn't know of it at the time (if it even existed, that server is *also* old :P)

pertho: On the topic of a bridge, I do have one. The server's listening on 10.0.0.1, which is setup as the resolver in bhyve.

But for some reason, the connection gets rejected.

I suspect firewall issues, but not sure.

Incidentally, it has no problems accessing the DHCP server, which is sitting on the same IP.

pflog(4) is your friend

Yeah, need to figure it out properly.

looks like I need to create the jail with bastille, then: zfs create -o recordsize=16k -o mountpoint=/usr/local/bastille/jails/alcatraz/root/var/lib/mysql jails/bastille/jails/alcatraz/mysql_data

HMM, SUS

If I turn off local_unbound, I get "REFUSED" only after a delay, as opposed to immediately. I think it might be local_unbound refusing it for whatever reason.

IIRC, you need allow statements in unbound to allow clients to query

ah ha it's /var/db/mysql LOL

pertho: they do exist, I suspect unbound's seeing a different IP incoming than the bhyve's external.

DarkUranium: not NAT'ing are you?

I ..... what? `refused query from 10.0.1.1 port 43995 because of 0.0.0.0/32 refuse

`

Sorry about the newline.

pertho: I am, yeah. Or trying to, anyhow.

OMFG, I'M AN IDIOT

Excuse me while I go jump out a window or something.

pertho: So uhm ... you know how the bhyve VM has an IP of 10.0.1.1?

Well, I wanted to allow any 10.*.*.* ...

So, in my brilliant genius, I wrote 10.0.0.0/24.

Because brain fart.

It was refusing because it was told to do so, since 10.0.1.* is not in 10.0.0.0/24.

10.0.0.0/23 then? :D

lol

Well, I actually wanted /8, so :D

overly broad, but if nothing else in your network uses 10/8, it should be OK :D

It's just for the DNS server, so.

The actual jails et. al. are on /16, so I figured I'd expand it a bit for any "external" thingies.

I noticed I say "so" a lot, so.

is it best practice to create a zfs dataset outside of a jail, then use 'zfs jail' to attach it to said jail?

Probably, considering you need to explicitly give a jail permission to manipulate zfs datasets.

This way, it has fewer rights.

that makes sense

On the topic of configuration languages, just learned of this one: nickel-lang.org/getting-started

any special magic getting mysql working in a jail? When I start up mysql: /usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql

can someone explain which one of these information is the correct one? The website lists a version as both production/legacy 0x0.st/8itF.png

and then on the download page 0x0.st/8itC.png

it lists two different versions as production

pertho: I don't recall ever having problems, so. Hm.

Might be a permissions issue due to the mount?

xx: all of these are in support, that's why they're listed

And the "legacy" one is the previous major version

You probably want 14.2.

ah.. I think it was innodb_use_atomic_writes.. was removed.. but yeah.. not sure how to handle the mysql data.. I got two pools 'zroot' (the main OS with a small-ish disk) and 'jails' a big disk I allocated just for the jails. I don't think 'zfs jail' is the right way to do this is it? What's the best practice for mounting zfs datasets into jails?

DarkUranium: ok

Not sure, I tended to use higher-level tools, so ...

who is Colin Percival <cperciva⊙Fo> and why are they pgp signing the checksums now instead of Glen Barber <gjb⊙Fo>?

they are not listed on the /security page

not here either docs.freebsd.org/en/articles/pgpkeys

not here either docs.freebsd.org/en/books/handbook/pgpkeys

afaik Colin is the release manager

has there been any announcement about the change of who signs the stuff now?

maybe this, but I don't know how legitimate the site is freebsdfoundation.org/blog/guiding-…e-new-release-engineering-team-lead

he's been the one for years

like four or more years

can someone please paste the full fingerprint of his pgp key in here? just so that I have additional verification

Howdy, folks!

I almost forgot, this is another config lang: github.com/bazelbuild/starlark

It's basically a subset of Python, kinda.

xx, rtprio, I feel like "Release Engineering Team Lead" is kinda of a dumbing down in title. . . for a long time he was the Security Officer Emeritus.

it would be appropriate to list him on the mentioned links

'Course. . . I guess once you're "Emeritus," you always are.

xx: i've seen the name in videos for release/cloud related manage and profiling iirc.

armin: reviews.freebsd.org/D48347 ls: --group-directories=first/last

hi there. i'm trying to figure out whether my freebsd system is up to date. i run freebsd-version -u or -k to get the patchlevel, but i can't seem to find any resources online that tell me what the current patchlevel is for each released. i would have thought it would appear on the release errata page, but i'm not seeing it. what am i missing?

if i click on the individual security advisories, it says what patchlevel the fix appears in, but (as far as i can tell) it doesn't say whether userland or kernel, and it would be more transparent if the main errata page clearly listed the current patchlevel. assuming i'm not missing it.

On a 13.2-RELEASE box I have, freebsd-update -v debug -r 13.4-RELEASE says: ...

update2.freebsd.org/13.2-RELEASE/am…6dc1021ab7b43ef1b7291e6f59cc1cc.gz: 404 Error (ignored)

update2.freebsd.org/13.2-RELEASE/am…6dc1021ab7b43ef1b7291e6f59cc1cc.gz: 404 Error (ignored)

(sorry for the 2ble pste). I'm not seeing that file on the 'net; ideas on how I can fix this?

13.2-RELEASE is EOL, so. . . definitely no.

That info is literally always near the top of the freebsd.org main page.

thorongil: release patch levels can differ for kernel and userland

also, you can check if your system is up to date by running `freebsd-update fetch`

thorongil: oh and you can subscribe (RSS) to freebsd.org/security/feed.xml

Remilia: exactly. the fact that userland and kernel versions can differ makes the patchlevel list on the errata pages much less useful.

and rss is nice for getting periodic updates, but less convenient if i want to check what the current patchlevel is

CrtxReavr: That file is necessary to update to subsequent releases. There are 3 files that get downloaded to update to a more recent version, and that one file is missing.