#zfs ppl think it's good to leave the mountpoint being created how it is so i think i'll stick with the working symlink method kevans

there's something weird about commit 58f0484fa251c266ede97b591b499fe3dd4f578e

why is it touching amd64 souces when amd64 definitely did not exist in 1994? is this the result of a repocopy?

can resolvconf -i <interface> use an interface group for interface? like epair or bridge or smth

is there any problem with resolvconf -a asdfasdf? like if it accepts invalid interfaces, why bother giving any interface at all?

alepzi: you wouldn't change the mountpoint, you would just temporarily mount it elsewhere

but yeah, whatever works

getting a syntax prob i can't figure out. sudo jexec -l testjail "ASSUME_ALWAYS_YES=YES pkg bootstrap -f | cat" says jexec: execvp: ...cat: no such file or dir

Remember what I said earlier? jexec won't set up pipes. (That said, I'm not sure why you need to pipe into cat.)

Also, why do you need ASSUME_ALWAYS_YES=YES instead of the -y option?

oh ya! /bin/sh -c

can't remember

i used to use -y can't remember why i changed

Hi, in a non-vnet jail I started a service that listen in port 2002. Now, when I do telnet 127.0.0.1 2002 it hangs, the same happens when I do telnet 192.168.100.207 2002 (192.168.100.207 is the IP I assigned to this jail in /etc/jail.conf).

to which IP should I point my telnet to check if its running?

solved. I had to add a rule to my host's /etc/pf.conf

wtf happened to mixer(8)

I fail hard at using it suddenly

question : why does this fail " poudriere ports -c -U ssh://git.FreeBSD.org/ports -D -m git+ssh -p 2024Q2" however if I remove the -U option then it all just works?

this makes no sense to me --> Cloning the ports tree...root⊙gfo: Permission denied (publickey).

that makes sense.

i'd suggest to go with git.freebsd.org/ports.git

Title: ports - FreeBSD ports tree

Hey there! I'm trying to set up a headless machine with FreeBSD, and one thing I've noticed is that I don't seem to be getting notified of system mail arriving to my users. When I log in, I don't get any "You have mail." or "You have no mail." messages that I get on Linux. I tried looking it up, and I see that the MAIL variable needs to be correctly configured for the shell to check for mail,

but I do have that

And mail is working, DMA is up and running and I'm able to send messages between users

which shell?

/bin/sh

I decided that at this point, it'll probably be easiest to pull the source tree and see what it's doing / supposed to do, downloading it right now

by default, both csh and sh sets up MAIL correctly.

if you don't see that variable, that might be because of your local modifications

I do see it

MAIL is set up correctly

I'm just not getting any notification messages when logging in

I should note that I log in via ssh and via the "su -" command

well, mail -e && echo "You have mail." in global cshrc and/or profile might do what you need

tcsh can check for mail, sh not

Oh, that's the missing piece of information, sh doesn't have this functionality...

Alrighty then, I'll probably put it into a conditional block in .shrc (only printing it if in interactive mode)

there is no mention of mail checking in sh manpage, while there is a mention in tcsh manpage

Yeah, though there is mention of the MAIL variable in the sh manpage

So I assumed sh used it

hm, you are probably right (i don't use sh)

There definitely is mail-checking functionality in sh, I'm looking at the code and I see the chkmail() function there, I'll try to figure out what's going on

maybe also atime/noatime mount?

(i'm boiling from water as i don't use /bin/sh interactively)

Nope, i just checked, and I don't use noatime here

I remember having this same problem on a laptop that I installed FreeBSD on for evaluation purposes, and I did end up going with the route of an explicit check in .shrc, but that's a workaround

how much disk space do you need to build freebsd

Soni: If you don't have enough, you should be able to mount external storage on /usr/obj

Dooshki: for that, we need to know how much we need

at least we managed to get past llvm compile, hopefully that was the worst of it

(it took up 15GB of storage)

7.2G /src/obj/src/freebsd/src/lf

+ 876MB for the src tree (full git checkout)

huh

how come your numbers are so low

checked another system and it's quite a bit larger: 17G /src/obj/src/freebsd/lf/main/amd64.amd64

no idea why the difference

arm64 is smaller though 11G /src/obj/src/freebsd/lf/main/arm64.aarch64

filesystem block size?

but if you're short on space, CVS checkouts take less space

usually at least 10-15G for a full objdir set

my entire build tree on that system is 60GB, that's including pkgbase repo and everything though

oh good buildworld completed, for a total of 12G

but df -h reports 16G

well anyway

otis: After poking at the code for a bit, I see that upon starting, the shell checks what the current size of the mailbox is and stores that for later reference. If it changes at any time, the user will be notified at the start of a command loop iteration (when a command finishes). And looking at tcsh's manpage, I see that it actually behaves similarly, and that is to prevent the message from

being doubled. It expects the login program to tell you if you have mail

So now, I have to figure out how to configure "the login program" to tell me if I have mail or not

Dooshki: according to login.conf(5) it should do that by default if you didn't specify nocheckmail

although i wonder if you log in via sshd, and it doesn't use login(1), perhaps that doesn't happen

I guess I'll just put `mail -e && echo "You have mail."` into my .shrc (with a check for the shell being interactive), it seems that that will take the role of what "the login program" is supposed to do, and since I'll probably only really log in via a real TTY probably once or twice per year, I don't think I'll mind the message being doubled

Thanks a lot for your time!

It is quite cool how you can go directly poke at the base system utilities by just browsing to /usr/src, with everything already being set up for you :D

(if you have the src component installed, that is)

that's weird, net/tayga doesn't come with an rc.d script

Also, anyone else experiencing weird delays in SSH when you aren't actively using it and then return to it? I had this happen on the laptop test run as well, that was connected via WiFi so I thought it was wireless powersaving, but now I'm experiencing it on a box that's hooked up via ethernet

Basically, SSH works perfectly fine, but if I leave the session and don't touch it in any way, and then return back in let's say 5-10 minutes, it's stuck at first, and it takes about a minute or so for it to get unstuck

*if I leave the session open

yep, that's normal

minute seems kinda long, 20 seconds is or so is what i've seen

Yeah, pehraps it's not a full minute

So I guess it's a feature?

that's not normal, i've never seen that here... it sounds like you might have some kind of firewall that's timing out the connection?

you might want to experimental with setting ClientAliveInterval on the server and see if that helps

(in sshd_config)

df -h says -394M avail?

anyway we don't think we'll succeed in compiling the kernel

it finished with -517M

cutting it a bit close

lw: i've espically seen it when the laptop comes out of sleep, but manages not to lose the tcp connection

rtprio: ah maybe then, i don't use sleep/hibernation on any of my systems

but in normal use i can leave ssh connections idle for days and they respond instantly when i go back to them (or maybe in 2-3 seconds if my shell got swapped out or something)

that's only on ones that are a prompt, if tmux is open, of course they get reset

In this case, the hardware in question is a RockPro64, and I'm not doing any explicit power management on it, other than running fand to control the case fan of the NAS case

I do have a pf-based firewall on it, but I've had this behavior before I've set up pf as well. So it might be my network router doing something funky

(though I should note that none of my Linux machines behave like this)

there any practical problems having a "-" in a jail name?

never seen any, i have lots of jails with - in the name

ty!!

wish jexec -l <jailname> had completion

anybody running fbsd on their fw13 AMD? it seems like there was some movement on this a few years ago but can't find much now

fw13 = framework 13, sorry

if i have 2 dir trees, is there a unix util i can use to overlay 1 dir over the other, so that the overlay dir's files will be added to (overwriting when needed) first dir

weirdly I can find more info about openbsd on it than freebsd

alepzi: i think you're looking for unionfs(5), but beware the comment in the manpage, which basically says it's broken, don't use it

how does scripted bsdinstall work then? it expands additional distribution.txz we give it over the base file structure. i want something like that

doesn't bsdinstall just mount the system to be installed at /mnt (or something) and extract all the dist files there?

dunno but what does it do to extract the extra distribution over the base file structure? just untars it?

is untar additive?

tar doesn't delete existing files unless the archive you're extracting containts the same files

bsdinstall does something like: tar zxvf /usr/freebsd-dist/base.tzx -C /mnt; tar zxvf /usr/freebsd-dist/kernel.txz -C /mnt; ...

since base.txz and kernel.txz don't contain the files, all the files end up in /mnt

ah so i guess that's how. well if you already have the files in a separate dir, then i gotta tar it up then untar it to get tar's behavior? seems shitty

there's also rsync

you can do that with tar, rsync or pax, or perhaps even cp

rsync might be the ticket

rsync -va src_dir/ end_dir/

"cp -a /my/new/files/ /mnt" might be what you're looking for

is src_dir the dir you overlay to end_dir?

(although rsync also works fine if it's installed)

i'll read man cp and try cp -a first maybe

alepzi: It's the directory with the files that you want to move over into the second directory, overwriting existing ones if they exist

ty

this isn't an "overlay" though, it's just copying files from one place to another place

alepzi: it's not up to a binary to provide completes, it's up to the shell to make them easy to construct - that's one of the reasons i stick with tcsh

rsync is mostly used for synchronizing directories across the network, it works seamlessly with ssh, but it can be used locally as well. Though in that case, "cp" could be more efficient

debdrup: how would you add jail name completion to jexec -l <jailname>?

i normally use rsync even locally, but if this is on the context of bsdinstall, cp might be more available

in the shell i mean

true (re: cp being more available)

I normally use rsync because... swiss army chainsaw that never turned away a possible option. But cp is in base and available when ports are not.

if i got a /usr/local/jails/testjail/ can i just copy files from the host right into the jail's file structure?

And now, I have to figure out what kind of networking model to use for my server... I wonder how VNET jails interact with the main system's firewall, if they respect it or bypass it since they have their own network stack

alepzi, Yes.

yes, although there are security concerns with doing that, mainly that a malicious root user in the jail might create bad symlinks or something

yay!!

hmm

And I guess when I do create a jail with its own IP address, it will be visible on the LAN under that address? Or will it only be visible to the FreeBSD system? Do I have to set up NAT for it?

ya good point lw. better option?

Dooshki: jail has its own IP stack including its own instance of firewall. on the host, it just appears as traffic on the epair interface

Dooshki: if you want VNET jail to have external network connection, either use bridge(4), or configure the host to route packets to/from the jail

alepzi: 'tar cf - -C /some/dir . | jlogin myjail tar xf - -C /other/dir' although to be honest i usually don't bother with this

alepzi, lw's point is a good one. During provisioning of a jail there are no processes in the jail, not yet, so no malicious agents. But if you are routinely updating files in the jail after it has been containing a malicious agent then one needs to worry about all of the normal social engineering attacks trying to trick root into unsafe and exploitable behavior.

er, jexec, not jlogin (that's a local script i have)

lw: Thanks! You have no idea just how much time you've saved me :) (I know what direction to look into now when setting it up)

lw ah so you still use host root to shit files directly into jail file structure, but then you expand them into place from within jail. really smart!

Dooshki: i'm setting up jails for first time now too

i just got over the networking and fw stuff thx to rwp and lw and kevans

I just copy directly into the jail path without using jexec here. But the advantage of jexec will use the configured jail path, in case that is need to be transportable between different systems.

Dooshki: one thing to watch out for, if your host firewall is set to drop packets by default, jail will also drop packets by default, so you need at least 'allow ip from any to any' firewall in the jail in that case

alepzi: from my .tcshrc: complete jexec 'p@1@`jls -N | awk \{\ print\ \$1\ \} | grep -v JID`@'

(ipfw drops by default, pf accepts by default unless you explicitly configure it to drop)

debdrup: wow that's amazing

(I've configured pf to drop by default; I found its configuration files a bit easier to work with than ipfw)

i wonder how hard the completion would look for bash

im using pf too but i wanna try ipfw some day since it's the native freebsd firewall

don't bother, ipfw is awful

that's sad

what does negative available space mean?

lol

Soni: It's space that's been reserved for the root user

lw: all firewalls should have a drop as the last rule; the main difference between pf and ipfw is that the latter will first-match-exit by default, whereas pf needs the 'quick' keyword to behave like this

Soni: is that a UFS filesystem? UFS reservers some % of disk space for root user, if you use that space, it'll show as negative available space

ah, and how much is reserved?

ipfw is a perfectly fine firewall

debdrup: i'm not talking about rules but about the default - absent any configuration, ipfw drops by default, pf accepts by default

but isn't that good?

debdrup: ipfw can't do IPv6 fragment reassembly, in 2024, that's a joke

ahh

ufs reserves 8% of the filesystem for the root user

lw: cgit.freebsd.org/doc/commit/?id=eb2e411915187

One thing that's kinda nice about ipfw is that it's more user-friendly in certain circumstances, it has direct integration into /etc/rc.conf where you can specify the type of your setup and parameters for the firewall... But if you need anything custom, you'll have to either write your own script to call ipfw or modify the existing one and have to potentially deal with updates overriding

it(citation needed)

errr, wrong link

cgit.freebsd.org/src/commit/?id=eb2e411915187 there's the right one

Title: src - FreeBSD source tree

ah alright, thanks

debdrup: that only does IPv4

Soni, You can print the minfree with "tunefs -p /"

debdrup: there's been an open PR on bugzilla for like 10 years to add IPv6 reassembly

Title: 170604 – [ipfw] ipv6 reass broken

12 years

lw: what about the code makes it ipv4-only?

debdrup: they're separate code paths, the ipv6 code needs to call the kernel's ipv6-specific defragger... as to why no one has implemented it, i don't know

pf has done it for ages so clearly it can be done

puffy is just more passionate about networking, it seems

Title: ⚙ D10533 Add IPv6 support to ipfw reass action

iirc there was some disagreement about whether IPv6 routers "should" reassemble fragments, because unlike in IPv4, they're not allowed to fragment packets

but a firewall clearly *must* be able to defrag packets to be useful

ya you don't wanna rely on well-formed stuff even from a router you control

it's still an external device

lw: if a router fragments IPv6 packets, its not behaving according to the RFC

debdrup: i agree, that's what i just said :-)

so by all rights, throwing out those packets is more correct behaviour

but the protocol only says they can't fragment them, not that they can't defragment them, as far as i know

a firewall that drops all fragmented packets is not very useful

works fine

there are (sadly) still legitimate uses for fragments even in ipv6

like UDP

i've been using ipfw and ipv6 for quite a while now

well, ipfw for 20 years, ipv6 not quite that long :(

well if it's out of rfc i can see dropping it by default

it's not outside of RFC, the RFC says IPv6 routers can't *fragment* packets, it doesn't say they can't *defragment* them

IPv4 routers fragmenting packets was terrible and it's good that's not allowed anymore

lw: if it says that routers can't fragment them, any router that does is not behaving like it should, so those packets should be dropped.

lemme ask this, is it within rfc for a machine to ever receive a fragmented ipv6 packet?

alepzi: for ipv4 yes, for ipv6 it's debatable

debdrup: this isn't about routers -- imagine you have two end hosts (nodes) communicating with each other using fragmented packets -- ipfw cannot handle that

debdrup: debate is?

alepzi: yes, because if it wasn't, fragmented IPv6 packets wouldn't existing to begin with

lw is it within rfc for anything ever to fragment ipv6 packets?

from a security standpoint, it's easier to drop all ipv6 fragments because it means something is either not behaving properly or trying to attack you

fragmenting is perfectly fine in IPv4, the only change in IPv6 is that *routers* cannot take a single packet and fragment it into multiple packets

s/fine in IPv4/fine in IPv6/

robustness principle for networking no longer applies in a world of constant threats

the source endpoint can fragment ipv6

jgh: yes exactly

routers cannot fragmant packets, nodes can

anyway, imma go back to 54 hours of trivia

ok so it's possible in the spec for a machine to receive fragmented ipv6, so ipfw should support reassembling ipv6 no?

alepzi: yes, and yes

debdrup, Regarding your awk above, NR is the awk variable for number of records aka line number. You can avoid the grep -v by telling awk to skip the first line the header line. jls -N | awk 'NR!=1{print$1}'

imma just link reviews.freebsd.org/D10533 again, though, because i feel like it was lost in the conversation

Title: ⚙ D10533 Add IPv6 support to ipfw reass action

if you want ipv6 reassembly, go test

i was just looking at that but it's 7 years old :-/

so what?

maybe i'll merge it into my local branch and see how it works

debdrup: so it's that old and no one merged it? that doesn't bode well for ipfw being maintained :-)

nice

ya help test and review the patch so it can get pushed in

lw: or maybe it means nobody uses ipv6 reassembly

rwp: yeah, you're right

write a vnet testcase that on 1 side sends fragmented ipv6 then on other side receives and reassembles and shit

whether i bother with this depends on whether i can get tayga+pf working for NAT64

I don't use tcsh so that completion was a foreign language to me. But I saw the awk...

why use nat in ipv6?

nat64 is the only thing that ipfw does that freebsd pf doesn't do (although openbsd pf does do it which annoys me)

alepzi: NAT64 is a transition mechanism, it allows IPv6-only hosts to access IPv4 hosts

lw: why not both?

oh like onboarding a 4to6 bridge

that's cool

it's part of 464XLAT / SIIT but it's also useful on its own, it lets you gradually phase IPv4 out of your network

debdrup: both what?

lw: pf and ipfw both doing what they should do

debdrup: i tried that and it doesn't work, pf drops all the NAT64 return traffic because it isn't aware of the connection state

i was trying to get ipfw NAT64 to run before pf does its filtering but i couldn't see a way to do that

how can we make /usr/src (3.3G) smaller?

Soni: did you clone it from a git checkout?

yes

Soni: delete it and rerun the clone with --depth=1

that will avoid pulling in the history, so you only get current files

--single-branch too

yeah except we want history

hmm can we run git gc --aggressive on this...

--single-branch -b releng/13.3

soni you get ipv4 fully disabled yet?

i'll bet debdrup knows if it's possible

please don't hilight me at random

maybe i should put NAT64 in a vnet jail that only runs ipfw, i guess that would work

sorry

alepzi: "not even close" (did manage to update to 14.0-RELEASE tho)

then i have to maintain an entire jail just to run a single firewall rule though...

there's a shitload of stuff i don't know; if i know the answer to something and i see it, i'll answer

lw it's more than a fw rule, it's a whole 4to6 bridge which is huge

i know there's at least one freebsd developer who uses ipv6-only

and good to have isolated away from pure ipv6 sweetness

NICE

but we do know how to turn off ipv4, we just need to deal with... this, for now

soni how?

alepzi: by rebuilding the kernel

is there just 1 easy bool to flip off?

net_ipv4_enable=false or smth?

alepzi: it is just a single firewall rule. it's the exactly same kernel code just with a different vnet identifier

think big picture my man

bit silly to install an entire copy of userland just for that. but it might be the least worst option

evaluate reass in ipv6 patch then. we need you!

the time for ipv6 is now

when messing with libc, do you need to rebuild everything?

that patch is both large and old which means it's probably not going to apply cleanly, and then i still have to deal with ipfw weirdness like not being able to add ::/128 to a table

i think my preferred solution would be to port nat64 from openbsd pf to freebsd

although in the mean time i think i'll try to work out why tayga isn't working, because it seems like it should do, the packets are just getting lost somewhere

i'd use pf nat64 if you ported it in

the problem is openbsd has diverged significantly from freebsd pf since it was imported, the syntax is completely different now

we're screwed

oh, i remember my other complaint about ipfw: it doesn't do state tracking for related icmp packets. both pf and linux iptables do that

lw: Are there any plans to sync FreeBSD's pf with OpenBSD's pf?

Mechami: as far as i'm aware this will never happen

Why is that?

the code in freebsd has diverged too much to make it practical, it's basically a hard fork now

we might do better to look at integrating NetBSD's npf, which looks quite interesting, although it doesn't do nat64

why do you want nat64 in ipfw?

if i answer that question are you going to give me another hour-long debate about "platd"?

actually, we would give you that either way

but we mean, you mentioned tayga earlier

so specifically we'd like to ask why ipfw over tayga

hey where'd security.bsd.allow_destructive_dtrace go?

in /boot/loader.conf or in /etc/sysctl.conf

no the sysctl is gone

sysctl security.bsd

Well then

ya wtf

Did you load the dtrace module?

V_PauAmma_V: it would show up in kldstat right?

It should, if loaded.

k so i don't think it's loaded. i guess that's why sysctl doesn't show?

That would be my guess.

ty

what's the normal way to do local dns resolution in a jail? use the host's unbound instance, run its own, or?

that really depends on what your needs aree

what do you mean the MIB space security.bsd is gone?

alepzi: each jail gets its own unbound instance, you run a caching recursive resolver at the edge of your network

that box can also run a reverse http proxy, so that you can cache freebsd-update files

debdrup: so you run unbound in each jail, why not just point /etc/resolv.conf at a dns server ip? or maybe i'm not understanding

what happens if the gateway goes down, do you remember all your IPs on your network?

i don't see my jail running unbound. top only says top, cron, syslogd. and i can resolve domains because i put nameserver <ip> in /etc/resolv.conf

ah i guess that makes sense

so that's like 15M for each jail in ram just for dns

counterpoint: memory is cheap

what about making jails use the host's unbound by pointing resolv.conf at its ip? thinking is if host is down, jail is down too?

how much memory does your system have, and how much can it have if you max it out?

i'm guessing there's going to be places where you can optimize things a lot more

ya im just wondering

jails can't have their own kernel modules loaded right?

right.

can jails have their own sysctl settings?

no. the can have their own network stack and firewall rules.

and rc

and jails don't have a bootup process right? so anything in /boot that base expands there is just wasted space right?

For the counterpoint I point /etc/resolv.conf in my jails to the host and run one caching nameserver on the host for all of the jails.

I normally run a caching nameserver on every bare metal host.

alepzi, I think (not verified but I think) that is correct that nothing from /boot is needed in a jail but it is only 5.1MB on my system so not much savings and more work to avoid it.

Jails are typically configured to run through the /etc/rc and /etc/rc.shutdown process and that is after the /boot part of the process (on bare metal) but I would claim /etc/rc is still part of the "bootup" process. Just debating word semantics of "bootup" here.

alepzi: certain sysctl settings, such as certain network-related ones, can be set per-jail

how can we know which ones are per jail and which aren't jmnbtslsQE?

ty rwp

not sure

two notable ones are: net.inet.ip.forwarding, net.inet.ip.fw.one_pass

what about security.mac.portacl.rules?

i haven't tried it

let's find out!

does everyone love bastille or use something else for jails?

i'm just learning raw jails but after that i'll probably look for a manager

i run my own scripts using either nullfs basejail or cloning depending on the jail

i use /etc/jail.conf.d/

oh, and zfs clone from a snapshot

debdrup: are those a pain to update?

rtprio: not really

what's in your snapshot that you clone from?

buildworld DESTDIR=/jails/template ?

zfs+send image using poudriere-image(8)

no, sorry - zsnapshot

poudriere-image; i'm gonna have to look that one up

poudriere is the tool that the project builds packages, and it can also be used to build release images, and other things

wow

like yuck. x11 forarding disabled by default in 14.

like yuck? what are we, two?

Like yes.

yep, too bad. cannot possibly change sshd config so you can never forward x again

what the hell, man

why's that bad?

Because it was default on forever.

And following upstream sounds like it just changed... no 20 years ago?

what's it do?

alepzi: Allows you to run/tunnel X11 over the ssh conenction to your local X11 server.

is that to take away the feature in anticipation of moving ppl over to wayland that doesn't have the feature iirc?

I dunno what wayland has that's the same I think it's waypipe but it's all half baked last I read

I meaning use X11 forwarding for clipboard semi-syncing.

alepzi: i doubt it; probably just to secure something that most people don't use

like removing telnet?

telnetd it's not removed, but turned off by default, yes

make nc telnet-compatible with a flag and install it as a script that runs nc using that flag

bah

k so a jail can't set the security.mac.portacl.rules sysctl wtf? how am i supposed to let the caddy user listen on a port under the default port high limit?

lw

alepzi: if you can't get it to work, i'd recommend using NAT inside the jail

so that the desired port is available from outside the jail but inside the jail it's a high port

i want jails to be able to be more independent from the host system and admin work

is there no way to let a jail's bin bind to a low port?

...without specific host configuration for each and every case

not sure. but i'm talking about NAT inside the jail (jail configuration)

if i can't find anything else i'll check that out ty. hope i can find a better way tho

seems like a shortcoming of jails that even vtnet jail can't have a bin bind to a low port by default

am i wrong?

i don't think so: as far as i know, at least in theory, root and non-root are separated within a jail just as they are in a non-jail

eh i think you might be too deep in the rabbit hole. think about it as a user, selling freebsd to linux ppl, "ya best container tech ever. also: bins can't bind to low ports lmao"