jmnbtslsQE: making lots of progress getting a jail bin to bind to a low port without using fw forwarding

debdrvb gave link bugs.freebsd.org/bugzilla/show_bug.cgi?id=259149 that put me on the trail

Title: 259149 – mac_portacl not in affect when running VNET jail

i don't need to load the mac_portacl kernel mod, and i don't even need to set security.mac.portacl.port_high to 0, because portacl isn't used at all. all i had to set was net.inet.ip.portrange.reservedhigh=442

the jail host doesn't even need net.inet.ip.portrange.reservedhigh. only the jail needed it and it's working

i think why it wasn't working before is mac_portacl was enabled but the rules sysctl was set wrong so net.inet.ip.portrange.reservedhigh wasn't allowed to work

vnet stays winning

And the answer is, "ansible-config".

hello, has chromium (the browser) been removed from 14.0-RELEASE on amd64? I can't install the pkg, even though I see a port listed under www (and by jove I don't wanna have to compile that one)

is there ungoogled-chromium or smth?

not removed, no

nope, I also tried that one

probably just failed to build, or one of its dependencies

so what gives? pkg doesn't show it when I search for it etc

ah. So maybe a matter of wait and see

yeah

aight I see. Thanks everyone.

Hm, my how-to is maybe not a good one to follow.

kloinmpa: you can sift through here: pkg-status.freebsd.org if you have ipv6 or an ipv6 proxy and check out the ongoing build

(ipv4 front page, builders only accessible via ipv6)

ipv6 only, nice

and ppl say freebsd devs don't dogfood

thanks kevans, I'll have a look. Didn't know about that page.

iirc the cluster is all ipv6 primarily, the vast majority are dual ipv4+6 but some less critical things like builders lost their IPv4 to get allocated elsewhere

how hard is it to get ipv4 addresses now?

We haven't run out because of the ubiquity of NAT.

uh

is it normal for bins running in a jail to show up in the host's top?

yes

do admins like that? isn't it better to hide anything running in a jail from the host?

you can use -J to scope it down if you'd like

better in what way?

hiding them just gives you an incomplete view of what's consuming CPU on the system with little benefit to trade for it

ya i guess -J 0 works

(s/CPU/resources/)

alepzi: Hey, something occurred to me. You want to make sure - absolutely sure - that you've got good offline back-ups. You're talking about security knobs, and the biggest one of all is "I can come back from being nuked from orbit".

Especially since you're using ZFS, back-ups are very pleasant to do.

Also, you're doing a ton of good set-up work you'll want to have preserved regardless.

ya ty that's right. that's why i use scripted bsdinstall so all my configs are saved local

i never do manual setups

Good good.

i don't have zfs backups set up yet though but i will tyvm

I'm actually diving into Ansible here, with an eye towards cutting back on manual configs.

alepzi: I'd bump back-ups in priority given how critical they are.

ya and i run zfs everywhere so like you said zfs backups seem like a cinch

They become truly glorious when you consider things like backing up a few bytes of a giant database file thanks to snapshots capturing only what's changed. Even rsync can't compete with that.

ya that's nuts

and stored offsite all encrypted too

I mean, you still want to have explicitly quiesced systems for coherent back-ups, but it's still impressive and powerful.

so you can use dumb storage from 3rd parties

not need to trust

Take care if you're using native encryption. I'm still on GELI everywhere here.

is GELI native encryption?

GELI is FreeBSD's block storage encryption, as opposed to ZFS's new built-in encryption.

think zfs native encryption is still immature?

ZFS's built-in encryption is what would let you store remotely on an untrusted host. But it's rife with issues so far.

that's so sad. zfs has a good rep, why ruin it?

It's the primary source of data loss bugs in ZFS these last few years, as I understand it.

alepzi: Incomplete design, insufficient follow-through.

Not everyone gets bitten, but your back-ups are *not* where you want to be bitten.

that's a serious problem no?

Yes.

is it like, the original zfs geniuses left so now new stuff is added in a hacky shitty way?

The downside is that you have to be able to trust where you're sending your back-ups if you're sending to something that encrypts at rest using GELI or LUKS or whatever.

alepzi: If you ask PMT for some backstory, he's the best source of data nowadays.

ok ty

alepzi: This might be old now so it's best to ask, but gist.github.com/rincebrain/622ee499…1732774037ff44c6768085ab#encryption

Title: Random ZFS notes WIP.md · GitHub

Ah, you're asking him. Better still.

alepzi: Anyway, best that you know now as you're still defining architecture.

tyvm

got 2 jails configured now!

nice

really liking this tech

alepzi: I'd still give some thought to golden masters and using ZFS to duplicate them. Might be faster than individual, unique installs. Copy the golden master, customize rc.conf so it has a unique identity, etc.

ya i'm gonna figure all that stuff out this week. last week i just barely got stuff working now i gotta clean it up

Random thing I whipped up to save time here: bpa.st/JQ3A

Title: View paste JQ3A

If I were slicker I'd automate running freebsd-update against the base before taking the snapshot.

oh nice ty

But you can do things like this.

Aftering updating my laptop now reboots inself instantly after X tries to start

Any way to fix?

I'm still able to boot into single user mode

sfox: Reboots, or X restarts and flaps?

sfox: Might be worth disabling X and snagging a copy of /var/log/Xorg.0.log to see what it reports, assuming it gets a chance to write before exploding.

sfox: Good place to ask: efnet, #freebsd-xorg

mason: the computer goes back to edk2. no trace of crash in /var/log/messages or anything

the last thing in the log before it crashes is

Apr 14 22:50:29 lappy kernel: VT: Replacing driver "efifb" with new "fb".

only way I can get back in is with singleuser mode or using an older bootenv

sfox: Nothing interesting in /var/log/Xorg.0.log ?

looking

unfortunately no. it's been replaced with the current version

sfox: I'd definitely ask in that efnet channel. They will probably home right in on why it's blowing up with the efifb -> fb swap, and they'll have better questions.

ok

can you invite me?

nvm

sfox: It's on the efnet network.

different network

And is open to the world.

yeah

what's the url for efnet?

sfox: There are a few: efnet.org

Title: EFnet - The Original IRC Network

They seem to have a webchat too if you don't want to add anything to your client.

i managed to connect it was just extremely slow

A regular client might be faster and less painful then. I stay connected to the network. It's useful.

There's a fair amount of FreeBSD stuff there.

sfox: wiki.freebsd.org/IRC/Channels

Title: IRC/Channels - FreeBSD Wiki

sfox: I think folks there might tend towards European hours, so have patience and good luck.

Hitting the sack here.

i'm not sure why but it's working again now

i removed drm-kmod

freebsd does a lot of weird stuff to my computer

for example resume doesn't work all of he time

it only suspends my laptop when I open the lid not close it

and some of the time when it turns back on the power light will still be flashing like it's suspended despite it being on

why wouldn't the pflog service start in a jail? pf is enabled. i try service pflog start and it says Starting pflog. but then service pflog status says it's not running. nothing goes in the jail's console log either

alepzi: is /dev/bpf unhidden in the jail? is it a vnet jail?

(anything in syslog?)

jail's syslog different than jail's console log?

btw /dev/bpf doesn't show up in the jail

is there an allow. for that?

this is where you end up having to dive through the land of devfs rules

i'm using devfs 5. i guess that doesn't enable /dev/bpf?

see jail(8)'s devfs_ruleset, devfs.conf(5)

you have to explicitly unhide

ah ok

should i modify how ruleset 5 is configured or make a new one like 6?

preferably make a new one

5 is a good starting point for vnet jails, you might not want them all to be able to do bpf

ok i'll copy all of 5 in /etc/defaults into a new one then add the bpf line to it, ty!!

you don't necessarily need to copy, fwiw

you could just "add include $devfsrules_jail_vnet" then unhide bpf

is there a way to call /bin/sh and pass the command to execute on the command line or perhaps use echo "commands" | /bin/sh

crb__: isnt it /bin/sh -c ?

the echo seems to be working, but I'll go look at -c now

my sh on my linux box seems to have no options at all :-/

comrad that's actually where I really need it as I want to excuse /compat/linux/bin/sh and give it a string of commands to run echo ""| /compat/linux/bin/sh seem to work ok

yup that works here as well: echo "df" | /usr/bin/sh

lw: you could reply to lists.freebsd.org/archives/freebsd-hackers/2024-April/003155.html and mention that not only have you used /rescue recently, you have a couple changes to it, and link them? ;)

Title: Re: Question regarding crunchgen(1) binaries

debdrup: i saw that but i found the OP a bit odd and didn't think the thread would go anywhere useful -- i'm sure no one is going to remove rescue or crunchgen any time soon

fair enough

(also i just nagged imp to commit a bunch of PRs, should probably wait a bit until i do that again :-)

create a new branch, make a minor package change, wait an hour for a complete buildworld before i can test it... there has to be a better way to do this

maybe i should make one branch, commit all changes there and submit them as a single PR

jexec -l testjail /bin/sh -c "echo -n '$6$7asdf...' | pw useradd -n user -G wheel -s bash -m -H 0" <--- why does strip $ from the pass hash?

because you used "" for the outer quotes, so your interactive shell interprets the $

# /bin/sh -c "echo $PWD"

/root

do i escape the $ with \ or is there a better way? like nested single quotes?

nested single quotes are very awkward in /bin/sh, but you could just do echo -n '$6$...' | jexec testjail pw useradd ... (if i understand right what you're trying to do)

no i need jexec to run it

my command has jexec in it, i just moved it after the pipe

ah

ya sorry missed that

echo command doesn't need to run in jail

works!

lw can we also do: env ASSUME_ALWAYS_YES=YES pkg install foo < /dev/null | jexec -l testjail cat ??

i'm not sure what you're trying to do there, did you mean sh instead of cat?

env ASSUME_ALWAYS_YES=YES pkg install foo < /dev/null | cat is the original command. i guess the cat clears trailing output or smth?

ah that's probably to prevent it from doing something it does when stdout is a tty (coloured output maybe? dunno). but no, you'd need to use sh -c '...' for that one since you need the entire pipeline to run in the jail. although i suppose you could do jexec pkg ... | cat, so cat runs on the host

tyvm

Good afternoon #freebsd .. guys I need to source an rsync binary for FreeBSD 11 ... does anyone know where I could get one please?

"pkg install rsync"?

hi lw ... sadly its to run on a Dell PowerScale ...

can't install via that.. just need the binary. They removed rsync from PowerScale apparently quite recently

plus it has no internet access..

[diablo] I don't know why rsync was removed from Dell Powerscale, but if it was removed, it was for a reason. It seems that rsync was deprecated: dell.com/community/en/conversations…nc-removed/647f9e0df4ccf8a8de297074. rsync uses basesystem shared libraries such as libc and libcrypto. So possibly the latest versions of rsync can no longer be installed on FreeBSD 11 due to incompatibility of base system libraries. And to

prevent rsync from being deprecated, they chose to remove it.

hi devnull

so, I've just grabbed a 11.0 ISO, installed it in Proxmox, built rsync from ports...

have the binary.... now toying with the idea of bringing over to the powerscale.. Problem is we've 35TB+ of small. files to get off a RH GFS2 cluster, and quick

copied it over, works

what is the 'size' argument to uma_zcreate()?

uma(9) is rather vague about this

the size of object to be allocated from the zone

ah, this is a fixed-size allocator, ok

yeah

and you can just uma_zdestroy() the entire zone when you're done, no need to free/cleanup anything?

you still have to free anything that was allocated from it prior to destroying it

ah

ok, along with counter(9) this is actually quite a bit nicer than netbsd's percpu stuff

#fsf

Oops.

what is going to happen with removal of 32-bit platform and ARMv7? i understand part of the justification for removing 32-bit is that it simplifies a lot of primitive operations if you can assume (e.g.) 64-bit atomics, but plenty of new ARMv7 devices are still shipping today

New at what rate?

i don't know specific production rates but if you look at Marvell for example, they have plenty of networking-related SoCs you can still buy... not sure about Qualcomm or others

i assume this is why ARMv7 was specifically exempted from the 32-bit removal, but if you still have one 32-bit platform left, you're not really getting any of the benefits of removing 32-bit?

iirc the long-term goal is still to remove armv7, too

it's just that it was granted a temporary reprieve due to complaints

It seems there ARMv8+ that's still 32-bit

we don't support armv8+ as 32-bit, though

ah

only COMPAT32

right, but COMPAT32 on an aarch64 kernel still isn't armv8+

strictly

how do I get sudo crontab -e to use my sepecified $EDITOR?

my normal user and root both have EDITOR and VISUAL set correctly, but sudo (for security reasons?) doesn't use them?

adonis: use 'sudo -E', but i believe this can be blocked by sudoers(5) config

(not 100% sure about that)

lw: thanks!, yup worked

adonis, Instead of using a user-specific crontab for root, I always use separate /etc/cron.d/taskfile for system level cronjobs. One file per task. It's nice to keep things that way because then to remove the task specific crontab the entire file can simply be removed. And the organization of it is nice to see what is happening easily.

rwp: ok, yea that does sound nice

Note that the /etc/cron* crontabs have a field for the user to run the task in the field before the command line. The user-specific crontabs are run as the user and so do not have that field. It's a very minor difference but one that sometimes snags people.

in freebsd 16.0 you'll be able to used rc.d timers for that! (wait... is it still April 1 or am i too late)

You are a day late and a dollar short!

hi guys, is anyone using microphone in chromium?

nevermind: camera/mic works on firefox/google meet on 14.0R

adonis: if you don't want to type 'sudo -E' every time, also look at sudoers(5) and in particular the env_check/env_keep options -- however be aware that allowing EDITOR/VISUAL can open security holes if you're using very fine-grained sudoers rules

e.g. 'env EDITOR=/bin/sh sudo -u someuser crontab -e'...

poll: does anyone here use RIP/RIPng?

might not be a bad idea to cross-post that one to -net@

bit late now :D no one replied to my last mail to net@ anyway, it seems pretty quiet

bah

i do wonder if freebsd could do with fewer lists tbh

like -stable / -current / -hackers could easily be a single list

it's not like 2002 when people actually used freebsd (ooh harsh)

what was different in 2002?

sfox: Linux was awful back then and FreeBSD did everything better

now Linux ate the entire Unix market, including BSD

i mean in absolute terms there's probably more freebsd users now than there were then, but market share is definitely smaller

it seems ti be returning that way

I converted from Linux

already switched my servers over

right now just putting up with the suffering from doing it on my laptop

I'm sure this issues i'm having are fixable, they just take longer

but the help i do get when i get it is higher quality

at age 28, growing up with both, linux is plug and play, but freebsd is actually elegant.

i quit trying to understand both when the clear-linux devs told me that they depreciated manpages. takes all types of brains, but you gotta get in where you fit in

\

if i could just get my wacom drawing tablet to work and suspend/resume working i'd be happy enough

suspend resume required acpi_ibm_load="YES" in loader.conf for me.

my wacom device strangely works with no configuration in vt(4)

only in vt4?

yes, I never got it working in x, but I haven't really made it a serious study

the other thing that just magically works outside my understanding in vt(4) is ytfzf

i didn't know tablets work outside of x

i have acpi_ibm loaded

it seem doesn't work

the laptop only resumes when the lid opens not closes which is backwards

and when it resumes it starts up like a coldboot

while the power led still flashes despite screens and fans on

it's really weird

okay, well, I never got it that far. my suspend works with "zzz", one of the cutest bsd traditions

when I shut the laptop it just dims the screen and keeps running, which is what i want.

it doesn't care about opening or closing, it resumes from suspend when I hit the power button

is zzz different?

see zzz(8)

doesn seem so

kevans: i caved and asked net@ as well, you're very persuasive

tbh I'm not sure that 'too many' lists is necessarily as much of a problem as the kind of people likely to audit these lists

how do you mean

this proposal will almost certainly draw out people

ah yeah

the trolls who live in the bike shed

you're more likely to get any kind of feedback with something remotely incendiary

that's ok though, i only need to convince imp to commit it :D

he is always like "ask arch@ this" "ask arch@ that" well now i asked arch@, he never said i needed to listen to the replies :-d

(i was pleased to learn about git-filter though)

silence is also a good thingg

then you can slap a "No objection from: -arch@" on it

i've been seeing this message a lot from pkg recently, not really sure what to make of it: Cannot solve problem using SAT solver, trying another plan

Hi!. In a Linux Jail recently created (debian), when I ping 1.1.1.1 or an internal address, like 192.168.100.111 it doesn't do anything, just return without error. In jail.con I allow raw_sockets and sysvipc, also configured an ip4.addr (also apt update works, also curl, so network is working).

Um. . .. "just return without error?"

CrtxReavr: yes, just returns

autoboot_delay doesn't affect jail startup time right? :>

correct

autoboot_delay is purely a loader mechanism

in a properly running system, the ping command should return the ICMP response or unknown host, but in this case it doesn't do anything.

so a jail doesn't have loader, bootup, kernel start. does a jail startup ONLY run rc and then... it's running?

correct, and it doesn't even need to run rc necessarily

it's like a literal virtualized OS right in the kernel API

the bare minimum you need for a jail is to will it into existence, you can run whatever you want inside it

technically it doesn't need to run *anything* just to exist

you could start an empty jail with persist and then just jexec into it when you want to use it

like a virtual OS instance context handle

(in fact i think this is more or less what poudriere does)

so like all the subsystems of an OS that can be virtualized, do get virtualized by jail. very cool

does anyone run puppet inside a jail? or would that be a dumb idea?

freebsd_host.init(jailname);

mmm, from root ping works as expected, but not from a user...

rtprio: i've never tried it but i used to run puppet inside Solaris Zones all the time, i don't see why it wouldn't work fine in a jail

freebsd is the best OS to ever exist

rtprio: just brace yourself for the cpu/memory usage of like 10 jails running the puppet client, i guess...

they're already in VMs. and splay is on

you know i was complaining about the size of FreeBSD-utilities earlier but...

[3/5] Fetching FreeBSD-kernel-lf-15.snap20240415210320.pkg: 100% 42 MiB 43.8MB/s 00:01

[4/5] Fetching FreeBSD-kernel-generic-15.snap20240415210320.pkg: 100% 49 MiB 51.1MB/s 00:01

[5/5] Fetching FreeBSD-utilities-15.snap20240415210320.pkg: 100% 10 MiB 10.0MB/s 00:01

i'm pretty sure it does get a bit bigger with all the libraries it depends on though

and gems

yep

oh that's not puppet related

just base packages

seems like a base jail for me (sshd running, pf/pflog..) takes 50mb ram, and add a basic web server running 100mb ram

that's pretty good

50MB sounds like a lot, how did you calculate that?

unbound, sshd etc

just looked at the RES of sudo top

for me sshd has about 10MB RSS, syslogd is 800kB, cron is 700kB

unbound is pretty big though, 49MB

alepzi: this is funny - I'm where you were yesterday, puzzling over the fact that security.mac.portacl.rules can't exist for each jail independently, despite vtnet.

850MB for a 13.3 jail in disk

lol

i can help

alepzi: I read the discussion of port twisting inside the jail, and I'll probably go that way.

I don't like the idea of changing the global defaults on the host for something only one jail needs, or related, allowing one UID that exists in one jail to do something and having that UID now privileged across the whole system.

mason: VNET is awful technical debt, not sure more things want to be added :-(

although i do agree portacl should work in jail somehow

lw: I'll accept that. Was just surprised. As I do more in jails and less in LXC the differences are starting to pop out a bit more.

yeah, linux namespacing is a lot more developed than freebsd

i find freebsd equally useful in practice but i wish it was... better

mason: no it's easier. just get rid of portacl

i'd really like to go back and virtualize time

kevans: ntpd in every jail?

kevans: We could shard ourselves. Very productive.

mason: don't even need to set anything in the host. on the jail set net.inet.ip.portrange.reservedhigh to 0 or the lowest you need

that's it

lw: not ntpd, no; I just want to be able to operate jails on some offset from the host on occasion

alepzi: Oh, I thought the idea was that there weren't separate sysctls in jails.

i've had to look at the actual implementation of vnet recently and it's really not pretty... seems like the idea was to require the least amount of changes to src to implement it

for this oid there is, it's vnet specific

alepzi: Nifty. Thanks. You just saved me some pain.

but mac portacl is host only, no jail, and it's global

ya np

kevans: i'm really curious what your use-case is?

kevans: you could always create a custom timezone :-)

lw: originally it was freebsd-update's build server; it does two builds a number of years apart to suss out changes strictly due to time changes

huh that's weird

I really didn't want it screwing with my system clock

kevans: this will be easier once 9p lands and you can create a vmm from a jail

if that ever happens... the review seems a bit stalled

even if i set a jail's root as locked, pw lock root, from the host i sudo jexec testjail to get shell

as root

pw usermod -n root -w no, same, jexec gave me shell

jexec does not care about PAM or passwd unless you use 'jexec login -f root' or something

it just becomes the uid then enters the jail, jail cannot stop that happening

so superuser on host gives superuser on all jails

correct, this is unavoidable as host root has full control over the entire system

you could perhaps partially mitigate this with securelevel if you really care, but...

(i am pretty dubious about securelevel as a security migitation in general, it's not 1992 anymore)

Would securelevel interact with this somehow?

well you could set the entire jail schg

i'm not actually suggesting this would be useful :-P

lw: Immutability wouldn't stop execution though, unless I'm missing something.

mason: no, but it would limit what you could do - i did say 'partially mitigate'...

kk

of course you could stop the jail and start a new one, so

Alright, yeah, schg and then the securelevel doesn't allow for that to be modified.

bsdcat(1) is weird

you'd think it's like 'the BSD version of cat' but actually it's an archive decompression utility (like gzip -dc)

you mean zcat on linux?

yeah, like zcat

weird

did you know vim can operate directly on compressed text and gpg encrypted text?

damn that's slick

it's really nice especially when searching logfiles

lw consider replacing your uses of gzip with lzip

imagine using vim when vi is in base. (actually i use vim a lot but i'm sure i don't use half its functionality... i only know it can browse directories because i keep doing that by accident)

Title: Lzip - LZMA lossless data compressor

sfox: i don't use gzip, but yes, i already looking at switching from xz to lzip for my archival storage

it's a much better archive format

mostly because of that article the lzip author wrote about xz

well i tested it and the file sizes are only slightly bigger

but compression and decompression is much faster

tooling feels a little less kludgly too and tarlz is nice

so if a jail only needs to bind to port 80 no lower should net.inet.ip.portrange.reservedhigh be 0 or 79?

alepzi: if you're not running anything on lower port (22, 53, ...) it doesn't really matter

ya im just wondering what other admins do

omg why does usr.sbin/bsdconfig have so many Makefiles

it's okay to put more than one file in a directory guys

does a jail need /etc/ttys at all?

no, but jails also don't run init, so /etc/ttys is completely ignored

but /etc/login.conf is used right? like if a user ssh into jail

yes

Sigh. The FreeBSD-packaged gitea web installation is yelling about a missing pg_hba entry, despite my saying "here, use this database over on this other system"

isn't there a gitea fork?

Oh, I wonder if there are client packages missing.

alepzi: Forgejo

ya maybe try that?

with a crap name like that it HAS to be good

Well. I could. But maybe this is missing client bits.

hah

watching a jail's console log from host while starting it is cool. is that how freebsd devs iterate faster?

i've never had a need to reboot the system to test something that wasn't a kernel (which can't be tested in a jail)... maybe useful if you're testing rc.d changes

bhyve is basically equally as boot to boot if you need that though

ah ya

I've really been digging bhyve for testing as of late

kevans: do you have a quick way to create/boot a bhyve vm from a src tree? i'd find that pretty handy

how do you see it working lw?

make buildworld; make buildkernel; boot_bhyve_vm; login: root

how does jail_parallel_start="YES" conflict with jail_list and jail_reverse_stop="YES"?

like you can't have parallel and reverse set to YES at the same time right?

or parallel means with other OS subsystems and not between jails