rwp: BastilleBSD/bastille #1573

hernan604: Sad at the state of PowerPC support in FreeBSD?

I would love to upgrade this machine to 128GB or better yet 256GB of memory; VMs tend to me limited by your system's memory.

I have 43 jails running currently on 64GB ram

hernan604: just out of curiosity, you've mentioned that you thought the hang is io-bound. did you try running "iostat -x -z 1" and see if around the hangs there are excessive reads/writes?

karolyi: termbin.com/oljh

karolyi: do you use screen or tmux ?

maybe you could try yourself ?

start or stop a bastille jail and press ctrl+a to switch to another window (then it hangs for some seconds)

hernan604: I don't use bastille but what you've pasted is nothing, no IO load at all

must be something else then

karolyi: right

phabricator 429'ing me :(

i'll ....

just send a patch I guess?

mason: you've mentioned you're going the "other way" with VNET jails, as in going back to non-VNET ones like I have. just out of curiosity, what's the complexity you don't like with the VNET jails? I'm thinking they must be not hard to set up, although I don't have a single one. I also thought FreeBSD is deprecating non-VNET jails so I'd have to switch over, over time

no-VNET jails work just fine

there is less overhead

less overhead => better network performance, more PPS

karolyi: what are you using 43 jails for?

mzar: lots of small websites, services: prosody, matrix, yourls, unbound, named(bind), coturn, jitsi, go-away/sarracenia, fail2ban, nginx, teamspeak, a separate mailserver jail with dovecot/postfix/SA/etc, roundcube, buildbot... the list goes on

lots of jails with php-fpm and uwsgi

very nice

is the bug report about PF not being supported from within the jail yours ?

yep

OK, so fail2ban has problems now ?

if it runs within a non-vnet jail managing pf, then yes. it's what I've hit, but already got it working with a little script- and prompt-fu in python

perhaps it should be allowed, for all firewalls, like routing updates

I mean there's an 'allow.routing' for a similar purpose

so one could argue for an 'allow.pf'

for the meantime, git.ksol.io/karolyi/pf-fail2ban-file-action

karolyi: good point, let's raise this on the net@ mailing list

it was already in the opening post if one reads carefully :)

mzar: I'm just realizing you're Marek

o/

I am also 100% addicted to non-VNET jails, and have only a few VNET-enabled ones - sometimes VNET demand is indispensable

yes, you can chcke it with /whois

I read somewhere around when VNET was introduced that non-VNET will be phased out over time, which kinda made me anxious, but good to know that so many people are still relying on it that probably it's not gonna happen

FWIW, sysutils/ezjail got small update recently, so people will no longer claim that's not maintained

I use my own ansible playbooks for building my jails

comes super handy when making updates

neat

karolyi: AFAIK there are no plans do deprecate non-VNET jails

git.ksol.io/karolyi/ansible-freebsd-jailhost-tools btw

forgejo is also one of my 43 jails :)

doesn't handle vnet though so the only thing it can create is non-vnet which is what I'm using

what doesn't handle vnet ?

the playbook. it only creates non-vnet jails

you can extend it

I was thinking on it, yes

I am using ezjail for decates, and it supports vnet jails, though it's not docummented

s/decates/decades

I also took a quick peek into that bastille thing yesterday, crazy someone did it in shellscript entirely

I have to go, have a nice jailing ! BBL

mzar: my playbook relies on a template jail that gets cloned and then I did quita a bit of wizardry in ansible (at least it counted as such back when I've had to "exploit" things in it) to achieve deploying the jails properly

the vars['somevar'] wasn't really public at first

also the package revalidator evolved quite a bit and still has to with introducing the new package sources

but that's only necessary if you use a local pkgmirror (nginx acting as a cache)

oh, bbye

that's great stuff, Laszlo

nice work

43 jails, wow

st_iron: I didn't tell it to flex but thx anyways :)

karolyi: yeah, it's fine, i just dropped my jaw with my 8 production jails :D

well, not counting my bhyve vms

I have already bought some cheap (scary) replacement SSDs so I'll have a combo of 2 off-brand SSDs and one Samsung 870, but I'm wondering if anyone more familiar with LSI controllers has any thoughts on these messages that started popping up the other day...

What's odd is prior to this there was nothing of note logged in the past week. I saw one SSD was showing high wearout, so I replaced it. That drive, brand new Samsung 870 ($200 for 512GB!) is da4 which is the first error logged. Then it seems like all hell breaks loose and all the drives go crazy.

This is an OLD Dell R720 with a Dell-branded LSI controller flashed to IT firmware.

SMART info pulled from scrollback - da0 is just gone, da1 and da2 clearly have issues, da4 is new and looks fine, which makes me wonder why it's showing up with issues right at boot... paste.debian.net/hidden/0aa32aa7

I've had weird stuff long ago with LSI controllers and SATA drives so now I always get a little suspicious about whether I'm seeing a true drive issue or a controller issue.

and googling some of the controller messages brings up ancient bugs in bugzilla, some starting way back in the 11.x days and still getting new comments in 2025.

hey, if I have a freebsd 15.1 install where I selected pkgbase during the install, is it expected that FreeBSD-base is enabled=no in /etc/pkg/FreeBSD.conf?

It's expected and wrong and should be fixed

anything else I should configure before updating to make sure my local config file modifications don't get clobbered? also, do I need to enable anything further to get updates for base and kernel and etc.?

the current update list looks pretty comprehensive but i've only recently converted from truenas to freebsd so still have a lot to learn

I should do that too with my truenas... it's abandonware

it's worth it. was getting pretty leery of the lack of updates for 13.x and am pretty happy with the current setup

karolyi: I'm fine with VNET jails but they're unnecessary complexity internally. More moving parts. And if that's not needed, why have it? Here is my guide for using VNET with jails. Compare a non-VNET equivalent and it's dramatically less. wiki.freebsd.org/MasonLoringBliss/JailsEpair

So, if I need a jail to be a DHCP client, I'll use VNET. Otherwise, I'm leaning hard towards simplicity.

mason: hear hear, I'd also like to keep my non-vnet jails :)

187 Uncorrectable_Error_Cnt -O--CK 096 096 000 - 39748

should an ssd with numbers like this not be marked as "bad" by SMART?

I also kind of feel like this should qualify as "bad"?

# 1 Extended offline Completed: read failure 90% 33542 1548176

I hate SMART stuff, totally inscrutable no matter how much I poke at it.

¯\_💩_/¯ SMART overall-health self-assessment test result: PASSED

This kind of wording also makes me bonkers: "When the command that caused the error occurred, the device was active or idle."

Makes me feel like there's a third state besides active or idle.

SMART is really terrible. It never predicts a failure for me. But I do use it to confirm problems when I am seeing failures.

Almost everything about it is vendor defined. And no vendor wants to have their devices fail their own definition. So they mostly never say the device is failed. Even if it is failing a lot of things.

Meanwhile... To avoid the problem being controller or cables or other things, I always test the device in another completely independent system.

I have a "bench" system, a zimaboard system, with two SATA cables right there easy to plug in a SATA device. (Not for SAS, obviously.) And then I will run tests on it there. If it fails both places then it is the device. If it passes there then maybe I have a bad cable or power cord instead.

rwp: don't we all... i.imgur.com/cC5oDx3.jpeg

Is there a legit reason ftp-archive.freebsd.org is throttled to about 1MB/s?

It's so painful, and it looks like they really push unsupported releases off the main server quickly (trying to get a 13.x image so I can fix my bootblocks so I can update to something supported).

Oh, I see it's also 96.47.72.116/ftp0 is slow, but going in parallel to both it's 1MB/s each.

Weird TE, you'd think they'd want people in and out of there as quickly as possible.

spork_css, Yup! Though I think my area might have a little less dust on it. :-)

I doubt ftp is being bandwidth limited. More likely it is getting pummeled by botnet scrapers and you are just a single thread competing with hundreds of other threads and so only getting the slip of what is left.

I'll eat my shoe if it's not a per-connection limit. you can stack up 10 1MB/s downloads...

I think the main ftp site has the geo-guessing DNS on it, and they probably drive traffic to other sites, but ftp-archive isn't officially mirrored anywhere so it suffers the same fate as the main site.