Trying to make FreeBSD 14.1 always boot from this new BE. After `bectl activate from-nagios02` and a restart, `bectl list` indicates the next boot will be from `default`. What's up with that? Is UEFI different in this regard? First time booting from that.

Hi, I was wondering if anyone could shed some insight on what I'm seeing. I have a VNET jail with v4 and v6 IPs. After starting the jail, v6 is not reachable. From the jail, I can also not ping6 out.

Once I disable pf in the jail, I can ping6 out.

Once that is done, and I enable pf again... I can ping6 out. And I'm reachable over v6.

Did I screw up my pf to interfere with ARP resolution or something?

Hm, doesn't appear that pf can even do that.

Okay, it's actually different. The ICMP is irrelevant.

It seems I was wrong. `bectl activate from-nagios02` does not change the next BE. It's still set to `default`. However, on reboot, we're in the `from-nagios02` BE. Interesting.

according to the man page that should only be the case if you use the -t flag

... aha, I guess it's because of how the ipv6 subnet is being routed. Bugger.

v6 has prefixes, not subnets.

dvl, have a look at the -t flag. it controls whether it is only for the next reboot or permanently

nimaje: I agree

ah, that was already mentioned, sorry.

jbo: I've not been using -t, so it should be permanent. I suspect this is related to my boot drive being UEFI: dpaste.org/89ggV

why would that matter? the BE is on the same drive/volume/zvol, just a different dataset that the bootloader picks up, no?

dvl zpool list and zpool get bootfs

tsoome: Good idea: dpaste.org/iXKot

jbo: There are issues with setting `activate -t` on a UEFI boot drive, from what I read earlier. I'll see if I can find that.

interesting

I would imagine that +95% of everybody is having an UEFI boot drive these days.

well, activate -t by itself is temporary setting and should get reset back [to bootfs value] on boot.

if the UEFI boot is affected or not really depends on what you have for boot loader on your ESP

that is, if it is freebsd loader.efi, or something else.

now, if you got bootfs set to new value, you should reboot and see if it is used; that is, esc to loader ok prompt, and check with commands: show currdev and lsdev -v

and then enter boot to boot your system and verify the rootfs.

verify, what is your rootfs:)

and yes, thats assuming you are using freebsd loader.efi

I've given up searching for that post.

if there is problem with -t, please file the bug and you can drop me note;)

tsoome: This new snippet has the output of those commands.

the dpaste.org/iXKot ?

tsoome: Sorry, bad paste, here it is: dpaste.org/2T4a5

tsoome: This is a Microsoft Azure VM created with the FreeBSD Foundation 14.1 x64 gen 2

*image

ok, so, boot loader did pick up the bootfs setup (the value in currdev and value from lsdev -v match)

now, if you start the OS, does it use the correct BE?

tsoome: Yes, it does.

tsoome: It still lists `default` as N

So it seems to be doing the right thing, but not reporting what it will do on next boot

what does df -h / report ?

Or, put another way, it's reporting that it will boot `default` next, but it boots `from-nagios02` (which is my goal).

tsoome: 30% capacity (is that what you wanted?)

no, the device name

zroot/ROOT/default 21G 6.3G 15G 30% /

so, something did reset it to use zroot/ROOT/default

tsoome: remember that after `bectl activate from-nagios02`, `bectl list` still reports `default` as N

yes, because activate only does set bootfs, it does not remount your rootfs

bootfs property, that is.

and we know that bootfs is set to zroot/ROOT/from-nagios02

so, you have rootfs from default (the N means active now), and as your bootfs is changed, it has 'R' in bectl

do you have default in /etc/fstab by chance?

tsoome: There's my error, I was thinking N was next.

There is one difference about this filesystem: zroot/ROOT/from-nagios02 did not originate on this host (via bectl create) - it was copied over from another host.

[dvl@nagios03:~] $ cat /etc/fstab

# Custom /etc/fstab for FreeBSD VM images

/dev/gpt/efiesp /boot/efi msdosfs rw 2 2

the origin does not really matter. the contents does (in sense that it should have usable OS image in it)

good.

um, you can mount your from-nagios02 with bectl mount and check the etc/vfstab from it.

because, if loader did pass the currdev to kernel, then starting system is reading etc/fstab from it and if fstab redirects / to zroot/ROOT/default, it will mount it...

tsoome: dpaste.org/vLPWj

Note, I showed etc/fstab figuring vfstab was a typo and there is no such file anyway

ok, so apparently fstab is not to blame.

can you see full console history - the kernel should tell about the fact that it is mounting rootfs and what name is used there?

of course, it may be possible that your boot setup does pass zroot/ROOT/default to kernel -- can you do something like grep -r default /mnt/from-nagios/boot ?

basically, since active boot loader is using correct BE, you can end up in wrong BE because bootloader itself is passing zroot/ROOT/default to kernel, or kernel does start with correct BE, but is instructed to use zroot/ROOT/default at some point during OS startup.

checking the console screen. I'd love to copy/paste, but that does not seem possible.

I see : Trying to mount root from zfs:zroot/ROOT/default []...

hm, and no mentioning of zfs before that line?

no other mention, and I found I can search the console, just not copy from it.

ok, does grep in that boot directory find anything?

which fbsd version is it?

Oh, there is ZFS filesystem version: 5 and ZFS ... above t

This is FreeBSD 14.1 - the grep; dpaste.org/of0HT

ZFS storage pool version: features support (5000) <-- found out how I can paste.

the console scrollback: dpaste.org/j5UUs

you have 2 disks there, da0 and da1

yes, da1 has no boot partitions.

yea, and it has ufs

It has a most interesting file: /mnt/resource/DATALOSS_WARNING_README.tx

and after boot, zpool get bootfs zroot does list nagios?

the weird fact is that your 'default' dataset was created 1970-01-01 00:00, but that is not affecting boot;)

[dvl@nagios03:~] $ zpool get bootfs zroot

NAME PROPERTY VALUE SOURCE

zroot bootfs zroot/ROOT/from-nagios02 local

tsoome: Idea: I can create another VM, from the same template and see if it has the same problem with BE.

zfs list would also be nice

^ dpaste.org/nr4ue

ok does your kenv | grep vfs report anything?

[dvl@nagios03:~] $ kenv | grep vfs

vfs.root.mountfrom="zfs:zroot/ROOT/default"

vfs.zfs.arc_max="128M"

vfs.zfs.vdev.cache.size="5M"

well, thats why your rootfs is zfs:zroot/ROOT/default

I agree, but I don't know how that is being set.

/mnt/from-nagios/boot/loader.conf.local:vfs.root.mountfrom="zfs:zroot/ROOT/default"

loader.conf.local file. sorry, missed it while looking grep output:)

hello

you can just remove that line.

That seems to be an image-supplied file

ok, commented out, let's reboot.

of course, I may get wild results now...

yea, whoever put it there, did not consider BE's ;)

if the OS startup fails, you can just set currdev=zfs:zroot/ROOT/default: and then enter boot.

Well, it rebooted, but I can't ssh in. ;)

Now we have: from-nagios02 NR / 6.44G 2024-07-04 22:34

Greetings. Can I adjust the volume level of the running apps? On OpenBSD it is doable by sndioctl(1), but seems I cannot do it with mixer(1).

So, success, but not running the BE I want. seems I set everything up in default.

Any help?

now it is working as intended;)

tsoome: Agreed.

I was never testing with the right BE

However, I configured the wrong BE with the right stuff.

pr-asadi: Sorry, I don't use sound. forums.freebsd.org is where I'd look next.

tsoome: Next goal, file a bug against that template.

dvl, do you store texts on dpaste via some script of yours?

ant-x: No, I copy/paste.

ant-x: Did you think, wow, he's so fast, must be a script? ;)

dvl: Thank you.

dvl, I thought wow, he is tech-savvy, must have everything automated :-)

ant-x: ;)

tsoome: Thank you. Very patient and helpful of you.

2 services of mine were off, "killed", is there some log I can check to see what happened?

maybe OOM etc?

yourfate, Anything listed in /var/log/messages about it?

dvl yw:)

rwp: ye, that's it, found it

`Jul 5 12:50:58 cabal kernel: pid 85109 (Thread Pool Wor), jid 0, uid 1003, was killed: failed to reclaim memory`

after `Jul 5 12:50:53 cabal kernel: swap_pager: out of swap space`

Let me go on record as saying that I do not like memory overcommit when it comes to production servers. It's a problem there.

I need to learn more about the FreeBSD memory model. I know it very well on the Linux kernel. But not for FreeBSD.

One problem is that programs no longer know they are out of memory and can't deal with it themselves.

I think something is leaking memory, I never had problems on that machine before

In the old days Apache might be driven into an out of memory condition and it would log this and it would behave accordingly.

but I'm running a shoddily made gameserver in linux compat

that might be the culprit

it got killed, and took weechat with it

But with memory overcommit and the OOM Killer any program is fair game for being killed and it can't log this itself. It just stops running. And it can't clean up any temporary files so it leaves lint behind.

the machine has 16gb of ram, 2gb of swap

To deal with this I am a big user of "monit" which can monitor things like web servers and such and if they need it then with rules it can automatically restart it.

16GB of RAM and 2GB of swap feels very typical for a normal machine anyone might set up right now. (Though I am old-school and I would set up more swap because I can.)

But as to whether it is enough for any particular computing task that depends upon the computing task. It all depends.

I might too, IDK how this swap got set up

I also don't know where it is

That feels like the bsdinstaller default amount.

Try: swapinfo

oh its partition

that's unfortunate

kinda messy to resize

/dev/da0p2

oh I can just install lsblk

Because zfs uses dynamic memory itself there is a deadlock potential for using swap from a zfs dataset.

Try: geom -t

hm right

ye, it sbetween boot and the ZFS pool, damn

So swap really needs to be on a different provider than a zfs dataset. That's just one of the limitations of the design of things.

resizing a zfs pool that has the roof fs in it seems daunting

let's be idiots about this: swap on SSHFS

In the old days we always wanted swap on the "inside" of the spinning disk which is the faster part of the disk. To make actually swapping blocks faster. But these days I just want swap available but not used so I want to put it at the "outside" of the spinning disk which is the slower part of the disk and keep the faster part to the file system.

ooooh interesting

tho I guess surely these days nobody has the roof FS of anything on spinning disks

I have resized zfs configurations and it's more work than anyone really wants to go through. I would just live with your current configuration. It's not a bad one.

yes, I will

doing nothing was my plan all along

As to whether it is spinning or solid state that depends upon the size you are dealing with. If small like a 1TB size then certainly go SSD. But if it is larger like 8x 8TB then spinning is the only cost effective way!

sure, but why would the root fs be that big?

I can see home being on those

But even though NAND Flash has advantages it also has disadvantages too. Stripe sizes are at least 4K and probably 8K and so block sizes should be bigger and such. Bad tuning can cause read/write-amplification problems. It's not without issue.

I don't actually know what kind of store that vps uses

afaik it is flash

uhm... with zfs all of the datasets are pooled. So there it is not allocated all to root or all to home. It is pooled and shared among all of it. That's a huge advantage of zfs.

wait, across drives?

Yes. Across drives.

wouldn't you put one pool onto the flash drive for the root fs

and another pool onto the spinning rust

and put home onto that pool?

But hold-up because I am nto sure we rae talking about the smae thing here. We might be out of sync.

You might organize data that way. Sure.

kk

But I would hazard to guess that most people have installed on just either one device or onto two devices mirrored and therefore have everything all in one zroot pool.

ye maybe

But sure a large shop like Netflix running FreeBSD is probably booting the root from a NAND Flash root and then having large data arrays on dual-port storage trays.

That's maybe more than I can afford for my house. :-)

well, my freeBSD setup is running on a hetzner VPS

with 160gb of flash storage included

What's the fs type? ZFS or UFS?

it does some network stuff for me, like IRC

zfs

and runs the occasional gameserver

So... All good! :-)

for storage I have a synology NAS. I'd prefer zfs, but I like the low/no maintenance life

and their apps are good nowadays

for personal cloud stuff

Synology is always ranked well in reviews. For me I am running FreeBSD with a raidz2 storage array of spinning disks. Times two so that I also have a backup.

And my desktop is a 2x 3TB disk mirror too.

On a single disk system even with backup it's just very annoying when a disk dies off. I raid everything that I possibly can these days. Then when a storage device dies nothing breaks as such and it is just a time to replace the failed disk/ssd.

the synology is mirrored

and backed up

to some cloud storage

I might buy a 2nd one soon-ish

rn I have it at my parents place, and here I have an rpi with an external drive, running freeBSD, as a local media nas

nice!

but I want a 2nd one so they can back up to each other etc

I was operating mine without a backup and last year had a power connector problem that took me a while to figure out. Screwed up my disks! But zfs pulled me through it in the end. Fixed the power connector problem and recovered 100%. But that motivated me to set up a 2nd one so I would have backup.

the local rpi has no data on it that I care about, its just tv shows and movies to watch on the various devices

the nas is backed up to a hetzner storage box

It must be one of the newer pi's if it has the cpu power to do that for video.

right now using the synology proprietary backup solution, but I might switch it over to restic backup, which I use for everything else

+1 for restic

rwp: it only serves it as SMB shares

the media players are on devices that play from that

oh, okay, data moving only. gotcha

the players are mostly an apple TV and an ipad

they can play from SMB shares just fine using infuse player

it is an rpi4b with 4gb of ram

I can also access it remotely using wireguard vpn, then play the media from abroad

if the internet where I am is fast enough

Regarding your out-of-memory situation on your hetzner vps, 16GB RAM is a pretty large size VPS. I guess you just need to monitor the size of the gameserver and make decisions about it as you go.

cya!

cracauer trying hard :D

jbo: never surrender :D

Question: On a booted and running FreeBSD system how can I determine if the system booted using UEFI versus Legacy BIOS?

On a Linux kernel system /sys/firmware/efi/ will exist containing UEFI stuff. Therefore I am guessing on FreeBSD there will be a sysctl that would indicate this.

rwp, how about sysctl -a | grep bootmethod

cracauer, :D

scoobybejesus, Aha! That looks very promising. It says "machdep.bootmethod: BIOS" correctly on a BIOS booting system. Thank you! I'll use it to inventory.

I am reading the handbook about connecting to a wireless network: <jgm/pandoc #9250#issuecomment-2195254393>. Whence comes the `wlan0' interface name? It is not listed in `ifconfig -a', nor mentioned in `man ifconfig'...

ant-x: when you do ifconfig -a what do you see?

voy4g3r2, What looks like two interfaces, both Ethernet. My wireless driver (visible via pciconf -lv) is not shown.

I think ifconfig -a shows one IPv4 and one IPv6 ehternet interfaces.

The Handbook introduces `wlan0' out of thin air.

My ifconfig -a shows re0 and lo0 .

Whoops and hooops! In `ifconfig wlan0 create...', wlan0 is probably the (arbitrary) name of the /new/ interface being crated.

Wrong: it is the name of the "cloner". A full list of cloners is available from `ifconfig -C'.

I had assumed that if the driver for a device was not available that the device would not be available either. No?

What wifi device is your device?

rwp, I have figured it out. The driver /is/ available and installed, according to pciconf. In order to connect an interface to that driver, I invoke `ifconfig wlan create wlandev <mydriver>'. I have succeeded now, all is well. I am now trying to congigure wpa_supplicant.

rwp, My original question was, what is 'wlan0' -- the first argument to ifconfig. It is the name of the interface, and must start with one of the available cloner names. A list of cloner names is printed by ifconfig -C .

My /etc/rc.d already contains a script called wpa_supplicant, which accepts the wireless interface name as its second argument ($2). How does the rc system determine and pass that specific argument to that script?

I need to get back to working on FreeBSD on a laptop. I have something installed from 13 days. But I never got it into a useful state.

But I do not remember needing to clone up a wlan0 device. I recall it appearing immediately after boot time.

No, you clone wlan, and name the result wlan0. I have just done it.

I believe you.

yeah regular config is often enough

for all that

But I do not remember needing to do it.

Now I am confused about wpa_supplicant. It is meant to be used as a daemon, and has an "nostart" entry in /etc/rc.d . What do I do with daemon marked nostart?

I look at "man rc.conf" and search for wpa and find interesting information there.

Probably I had set ifconfig_wlan0="DHCP WPA" for my laptop and that allowed the network script to create the device automatically.

rwp, I have done so, it is about supplying information for that service, but not about causing it to start at boot...

it's not supposed to be started as boot

rather when wlan0 as sta comes up

rwp, No, that is a futher configuration for an already created device. You create deviice via the wlans_<driver> variable in rc.conf .

The other setting I know I had set was wlans_iwn0="wlan0" for my Centrino-N 6200 in my Thinkpad x201.

Yes.

so what's not working eh?

ant-x:

ketas, so I have to UP my wlan before expecing the wpa_supplicant to be running, right?

can't remember if it needed actual up

ketas, For now, wpa_cli can see no supplicant. I will UP my interface and test again.

but does it run?

it also logs stuff

ketas, ps -a will not show it due a sysclt security setting (haven't removed it). I can successfully scan the nearby wireless networks, so I guess my wireless interface /is/ up.

Yet, wpa_cli says: Could not connect to wpa_supplicant: (nil) - re-typing

...and there is nothing new in dmesg.

what about ps x

ketas, that shows a lot more, but `| grep suppl' does not find anything.

ps wwaux is my favorite

so run it?

wlan0 is there right?

Could wpa_supplicant fail to start because I did not fill its config yet?

ketas, Yes, wlan0 is there and woking via ifconfig.

well, if you didn't

then

logs show that too :)

my machine has all.log configured so

ketas, I have not studied centralised logging yet.

new to fbsd?

Very new.

service wpa_supplicant start wlan0

Oh! That looks more like it. Waity-minty

maybe it went to messages too

tail -F /var/log/messages

or tail -40F /var/log/messages

my fab

fav

That makes sense, not it complains about a missing config.

(in the stderr)

:)

It started successfully with an empty config. Thank you.

manpage has minimal config

Shall I configure it staring upon some ACPI or another kind of event of activation of WLAN?

as well as handbook

it will start automatically if configured

ketas, Yes, I know. Perhaps I need something in that config, because wpa_cli still fails to connect to supplicant, althogh ps -x shows it running.

did that?

unsure what wpa_cli needed

ketas, almost. I have not written down the parameters of my networks, because I thought wpa_cli or wpa_gui would take care of scanning them and prompting the user. The suppliant's example config has nothing but network-specific information.

ctrl_* or so

in conf

cntrl_* -- what is that about?

ketas, To explain -- I am preparing this laptop for a colleague, and have no idea what his local WiFi networks are...

those

ctrl_interface=/var/run/wpa_supplicant

ctrl_interface_group=wheel

in cfg

rc.conf?

no, wpa_supplicant.conf

iirc it's by default off

Is that for debugging?

Or for interaction with wpa_cli?

no it's for controlling from wpa_cli

Anyway, I already have this directory, and a file wlan0.pid inside it. No i/o file, though. Should there be one?

needs restart for that