has anyone done a ssh intermediate install for a FDE encrypted freebsd server

aka, boot freebsd, ssh into it, decrypt the actual OS, then boot that?

if someone has I would be interested in the docs for it

(I am still working on decryption over serial, I just have an idea of testing ssh decryption for serial-less devices)

i'd love to figure out how we make it easy to package up an initrd-like thing for use-cases like that

something stupid simple like I toss dropbear into an initrd to decrypt and kick-off the reroot? that /feels/ right

ANOTHER TALK IDEA!!!

kevans: you could embed a small ssh impl into the second stage loader..

loader-sshd-lua!

we have a hard no on more things for bios loader, and for uefi loader it doesn't even need to be particularly small

minimal dropbear is apparently within 1M and probably not incredibly painful to make build in the stand environment

well with UEFI just enable ssh at boot... although the network config would be complicated wouldn't it...

we allocate ~200M for our ESPs these days and use a tiny fraction of that, even a few MB is a drop in the bucket

tsoome: illumos have a need for anything funky like that? ^

oops crashed my terminal

I wonder how you would even embed dropbear

you would probably want to strip it down as much as possible, but dropbear is already pretty basic...

issue is dropbear adds attack surface, if you add dropbear, and you use openssh for your server, you have to worry about the security of both...

you're 100% not getting openssh into your loader environment any time soon, though

that brings us back to the notion of an initrd :-)

why the initrd?

wouldn't the s2 loader be better?

output the s2 loader, aka the boot menu, over ssh?

then you can also configure the server over ssh as well!

say you do an update, and fuck it, then you can select the snapshot of ROOT/default

well I keep saying stage 2, but in theory, you could output the entire loader, you could have a script which parses your rc.conf, and then copies this over to a config within the ESP which provides the network config for the sshd on boot?

hmm I just realised why this wouldn't work, theres no network knowledge in the loader, I forgot the loader is stupid

bummer :/

kevans: freebsd doesnt have an initrd does it?

I thought the third stage of the loader pretty muich does what Linux would use initrd for?

right, but you can see already where the idea starts to look attractive as an option

most people would just stay with loader, but more interesting use-cases can be served with full kernel support

hmmm

its still not as good as serial access though, moment a remote update fails to boot, and you are away at EuroBSDCon (for example) you would need remote hands

i mean, you'd still get serial into the initrd

I wonder how Linux users do this

how does Linux distros failover the backup initrd?

I know they make a backup initrd, but I never questioned how they failover

i think with a setup like that, you'd just inject the initrd as the next stage after loader and expect it to kexec or reroot depending on how much you care about packaging your production kernel into the initrd

kevans: so let me get your theory correct, you would basically embed a mini freebsd install like a linux initrd which the loader will chainload on boot, this will then have an sshd embedded into it, and be a *mostly* functional freebsd kernel (and basic userspace for recovery?) and then once you enter the passphrase over ssh, it kexec (boots the FDE kernel) and then how does rerooting work?

wouldn't you need to unmount root in order to mount the other root, but then how does the other root mount? a syscall?

if you kexec you don't reroot, and you may not need to kexec if you're OK just having a copy of the kernel unencrypted

linux initrd-style*

sorry

not actually Linux, I meant the style

reroot is a specific concept that already exists, see reboot(8) -r

kevans: is there a downside to kexec?

ooo inteeresting...

hold a minute

in theory this could be done with a RPI pretty easily

install a freebsd install onto the microsd card

kexec brings its own risks, you're relying on the current kernel to leave things in a reasonable state for the next kernel to init

have that run an sshd, then decrypt the partition, mount it somewhere, and then kexec the kernel from it?

hmmm

so kexec not perfect then :/

having your kernel unencrypted is a pretty major attack vector

it's probably fine for most drivers

you could sign the kernel, and check the signature from a signature stored encrypted on the FD

FDE

but by then you could have been pwned (already typed in the passphrase)

well, maybe a pretty major attack vector, right? that's making an assumption about your threat model

however it does allow for detection

kevans: if you are FDE your server, your threat mdoel is someone broke into your home/office and wants to exfiltrate the data on your server

if you care enough then you start with secureboot and use veriexec for next stage verification

that same secure boot which was compromised on many devices, that one? :p

I feel like the loader is a smaller attack vector anyways

"if you are FDE your server" -> you're making bold assumptions about my motivation for doing that

true true

FDE mainly just helps with disposing of the drives after :)

perhaps I don't use ZFS, or I don't like ZFS's native encryption, so I just encrypt the whole thing but the relevant bits are only a fraction of what's encrypted

this is how i do my laptop, for instance

I have long been interested in the idea of embedding the freebsd loader as a coreboot payload :p

likely chainloaded from seabios

you get a few MBs of room on most NOR flash iirc

anyways then you can move the loader to write protected NOR (although the laptop will now only ever boot FreeBSD xD)

read and write*

issue is you can get around this with a SOIC-8 clip, because its currently not possible for any open source hacker to replace bootguard :/

"that same secure boot which was compromised on many devices, that one?" -> from my perspective this isn't necessarily a complete takedown of secureboot; it's still a pretty reasonable idea and working with it still provides reasonable protections against less sophisticated actors

but hey if your attacker has the time to reprogram your hardware, you got bigger issues

yeah, there's also that :-)

although you could swap a SOIC-8 with a WSON-8, solder it to the board, just as an extra fuck you to an attacker

but then updating your loader (say if you rotated your signature key for your boot chain) would be a pain... desoldering the chip, slapping it into a socket, reprogramming it, and resoldering it

kevans: its almost 3am, my brain is just trying to nerd out over anything right now

went offtopic, so kexec/no-kexec doesn't matter too much in the grand scheme of things

which makes me wonder, in theory couldn't you reroot already?

yes, but right now your kernel is on the other side of the wall

in theory, say if you had two partitions, one unencrypted, one geli, you get the loader to boot the unencrypted one, (not sure how, but assume you do), then when this boots, it starts a sshd, you ssh in, you decrypt, and mount the FDE partition, and then reroot?

obviously not as perfect as the initrd idea of yours, but one which can be done without any osdev...

right?

yeah

i mean, initrd has its own flaws

the way I could think of doing it, is running through bsdinstall, then once that reboots you into the unencrypted install, make a partition with the free space, geli it, init zfs, make installworld (no kernel would be needed because you are rerooting) and then you can reroot into the FDE install

and doing it probably wouldn't take that much osdev, just working out the right incantations and then figuring out how you want to construct the image

oh boy, updating such an abomination would be a pain in the arse

imo you just treat it as a disposible artifact

wdym?

ideally you just have a mkinitrd type thing, specify the kernel and some binaries you want to wrap into it and it works out the dependencies somehow

now we are getting linuxy :p

so then whenever you apply an update of any kind, you just blow away the initrd and rebuild it from the running system

~just like Linux!~

is that how they do it?

kevans: yeah they regenerate the initrd every kernel update

their package managers usually have a hook to trigger a inird regen

i don't know that i'd go for an auto-update, personally, but i'm also not advocating it as part of average user's boot chain

anyways when I was talking about the abomination I was talking about my idea... I guess you could have a ufs partition for /boot, and then a ufs partition for / (both unencrypted) then do zfs just for the encrypted volume, and then mount the ufs /boot into /boot on the rerooted install, and then you can make installkernel/make installworld?

or... zroot and zboot :p

and just mount the zfs volume for /boot from the zboot pool into the reroot :p

kevans: sounds like fun!

it used to be the case that one standard setup you could get out of the installer was actually two zpools

one called whatever you name it, one called 'bootpool'

bootpool was just the contents of /boot, and your root pool had a symlink into the bootpool

yeah but your bootpool still needs a root, to boot first, so that you can ssh into the bootpool install, and then reroot?

right, you'd have to expand it a bit for your scheme to work

I should finish my serial debugging first before starting new projects :p

in theory you could probably get away with specifying via kenv an rc replacement that just runs sshd and waits

reaping the benefits of bapt's work

although I should be quick, I might get arrested for freebsd warcrimes for this one :p

kevans: you can hotplug the rc?

you're damn right you can ;)

imagine the blogpost potential for this shit...

cgit.freebsd.org/src/commit/?id=9f2ad7c09709e bapt wanted it to allow testing of his rcd

so in theory, the bootpool would need an sshd, and geli, and any libs geli depends on?

it is the freebsd way

oh and reboot ofc

yeah, sounds reasonable

sounds like you could shrink that into a few MBs

well maybe more than a few

./boot is 350MB on my lapotp

but the /boot can just be remounted actually anyways

yeah, in a traditional setup you mount both pools and symlink the bootpool into place so you don't have two copies and kldload can still work

better yet, this would be pcbios compatible too!

i'm interested in hearing about how your experiments go if you decide to start fucking with it

why symlink it?

I see no reason I cant just mount the /boot partition from the other pool into the reroot

because you still have to have a /boot path within the bootpool, so you end up with /boot -> /bootpool/boot

hmmm.....

I thought the reroot would unmount that all though?

mounting it there might just work, but I think there was an ordering concern of some sort

> The system kills all processes, unmounts all filesystems,

yes, but then the zpool rc script would probably bring it back depending on how your cache file is setup

but the zpool rc script would mount the zpool partitions no?

oh wait no, it tries to mount all zfs partitions ugh

but if zroot has no /boot...

ah fuck

this is probably where 3:00am starts to be a challenge :-)

so with default zfs partitioning /boot is stored within zroot/ROOT/default

so I would also need to partition off the /boot on the unencrypted install, then it can just mount the zroot's empty /boot dir

on reroot

unless im missing something?

but yeah good point, sleepy time

yeah, i don't know, heh

it's end of day here and i'm running on like 3 hours of sleep, so i'm also losing my ability to reason a bit

hah

goodnight

night

Reinhilde: on debian it worked, then I installed freebsd for speed

Reinhilde: I remember I access fan speed with acpi

which make sense

secondly I cannot have memtest86 check all memory, it only get half memory and 2 on 4 cpus

On debian you were able to access the fan speed? _Posterdati_

memtest86 is the image provided by freebsd with pkg

Reinhilde: yes I was, I had psensor

Reinhilde: the machine was so slow with debian

Reinhilde: freebsd performs better on this i5

right

_Posterdati_, you don't have to highlight me with every message

_Posterdati_, it starts to get annoying after about the second message in a row

_Posterdati_, so maybe please don't

uh ok

right, that out of the way: `pensor` here refers to what? a command? some kernel-related API?

er, `psensor`, s key broken

oh, high level, GUI based sensor reader

it uses lm-sensors

to my knowledge that is linux-specific. do you know what underlying driver was in use on the linux side to pull that data?

I do not recall

acpi for sure, and coretemp

and some asus specific driver

if you `kldload aibs` as root, does anything show up under `sysctl dev.aibs.` that would be of interest

sysctl: unknown oid 'dev.aibs'

you already kldload'ed aibs?

yes

k. then just at a first approximation the driver in question may not exist for freebsd

ok

what was the motherborad again?

Asus K55VD

latest bios 411

wait is this a laptop

yes it is

try `kldload acpi_asus`

ok

thouh i'm not sure what that'll do, and the nodes would be under hw.acpi.asus

likely won't give you fan spede access

same error

nothing at all under `sysctl hw.acpi.asus`?

nothing

did you install the OS under a BIOS type firmware or UEFI

this is one of the first laptop with uefi

if the machine can use UEFI but you used BIOS the CSM may be incomplete

ah

how can I check that?

are you asking how you can check whether you installed via CSM?

efibootmgr as root would probably error

it returns indos

infos

Boot to FW: false

BootCurrent: 0000

Timeout: 0 seconds

BootOrder: 0000, 0001

+Boot0000* FreeBSD

Boot0001* CD/DVD Drive

please don't paste multi line program output into irc

for long pastes - this one's thankfully over already, use a pastebin, such as debian's pastezone paste.debian.net

ok

(there's nothing in the TOS that requires the pastes to be related to a debian irc channel, so it shuold be fine for us here)

and to see how you booted sysctl machdep.bootmethod

UEFI

no idea?

nimaje, ah thank

_Posterdati_, yeah the driver probably doesn't exist. hopefully you're ok without your fan speeds for now

temps are a bit higher for the cpu

that can happen, it's possible you were finding debian slow because you'd done some scheduler tweaks to keep the cpu colder?

346 K

I did nothing

not even frequency scaling

oily josh teh third of syria palaestina, 73°C?

yes

it is a bit high

do you have thermal paste

yes

changed six months ago

fan was cleaned

and sometimes I use air to flush dust

okay

so the machine just runs hot

because fan does not change speed

at boot it goes high

then slow down during boot

there are dev.cpu.X.coretemp.tjmax = 105.0C

but I cannot change it

it says read only parameter

yes

it is

now I placed the laptop over a fancoil hatch :)

at that temperature the machine should switch off automatically to protect itself.

temp is decreasing

ok

40 C

now

should be an acceptable temporary workaround

:(

cannot work with it :)

oh

what is dev.cpu.X.coretemp.delta ?

does `sysctl -d dev.cpu.X.coretemp.delta` tell nothing?

yes it does

a changing number from 59 to 63

now 65

`-d` stands for describe; i seem to recall that delta was meant to be the "delta between current temperature and Tjunction.max"

Delta between TCC activation and current temperature

is this intel cpu?

hello

as coretemp is working, it has to be intel, for amd it would be amdtemp

you can't change built tjmax. what i suggest ( if the CPU is 6th or newer generation ) is to set epp to 100 - that's maximum powersave

i.e dev.hwpstate_intel.%cpunumber.epp = 100 ( and then for all cores )

if, then, maximum performance needed i set all to "0"

it works quite well

powerd, as well as powerdxx do not work with hwpstate driver and it is useless to set them, if tried at all

There is no mention here of etcupdate/merging of etc files... How should that happen now with pkgbase? docs.freebsd.org/en/books/handbook/cutting-edge/#pkgbase

pkg has a three way merge facility, I think with @config in the plist

nimaje: I believe that three way merge has failed me. I don't know why yet. dan.langille.org/2026/08/07/freebsd…pdated-overwritten-no-merge-attempt

I was absolutely shocked that I couldn't mount a UDF disk image - I burned a few hours on this until I found this bug: bugs.freebsd.org/bugzilla/show_bug.cgi?id=120989

but it looks like someone is working on it now, but really didn't occur to me that something that's been around for so long wasn't supported.

you can blame Sylve I guess, because when working on converting a Windows VM from BIOS to UEFI I had to pull things off the windows install DVD and... oops. Had to resort to the mac for that.

spork_css_: probably weak workaround to mount, but assume "7z x" could works there... ;D

dvl: did it drop a /etc/pam.d/sshd.pkg{new,save} file in?

dvl: it does try a three-way merge, but if that fails then it's supposed to install the 'new' version as foo.pkgnew and let you reconcile the differences

it would be convenient if it told you there are relevant files and to run find /etc /usr/local/etc -name '*.pkgnew' -ls to see the files

kevans: No /etc/pam.d/sshd.pkg{new,save} found. :/

kevans: for any of the jails, on any host.

there any way to enlarge the max pf table name limit? 31 chars is tough to squeeze into

should really be doubled

scoobybejesus: this cannot happen in /usr/local, but agreed on principle

It really should be added to the Handbook documentation. Almost every 3rd party guide/how-to for pkgbase includes it.

I'm surprised it's not there already (might just be a lingering needed approval, though)