Evening all

dautor8518050: I figured it out, it was nat64/dns64 creating ipv6 addresses where none existed. truss is still great.

lua is pretty slow though

realdeimos my favourite tool! It's also amazing for teaching and learning.

What happened at 5:20 UTC? Is it only me or was there ~1000 leaves and joins?

Rerouting at libera HQ

laptop setup is coming along nicely, I think. I bit the bullet and went with vector fonts, since I can dial in the exact size I want: matrix-client.matrix.org/_matrix/me…matrix.org/dJkoHktrDrtbWCIgxJaeGhMk

not sure I'm going to stick with this font, I find the spacing between letters to look pretty off, like the weirdly large space between r and a in my hostname

dwm is my favourite wm. Now I want to go back to it...

tm512 which font is that?

dautor8518050: it's monoid larsenwork.com/monoid

Title: Monoid

available in ports

before ditching it I might check out the "tight" variant of the font which I'm pretty sure I saw got installed

Hi, I recently have trouble configuring net/samba416 from ports - via make config, regardless I always get "Skipping 'config' as NO_DIALOG is defined"

I'm noticing that suspending seems to interact weirdly with uptime on FreeBSD. I'm guessing it just pauses uptime, but the reported uptime when I shutdown seems lower than I'd expect

either way, definitely different from Linux, where uptime actually reports how long it's been since I turned the machine on, including time spent suspended

Hello

Can somebody tell me in what package the libz.so.6 belongs?

I just updated a jail and it broke with: ld-elf.so.1: Shared object "libcrypto.so.111" not found, required by "pkg"

and: ld-elf.so.1: Shared object "libz.so.6" not found, required by "indexinfo"

bitblt: libz.so should be part of base

is this about a broken upgrade process?

Yes

I see, well then you should revert to the last working state and retry

I'm a newbie so I'm trying to bring all the pieces together on how freebsd works, but for starters my huge mistake was trusting the truenas core ui to do a jail update instead of `iocage update`ing it my self

if you're doing this via freebsd-update and also have ZFS available you'll have a BE which you can switch to

megaTherion: BE?

Boot Environment

try bectl list

First of all this is about a jail update, but as far as I understand the principle should be the same?

yes the principle is exacly the same

except you obviously won't require a kernel update & reboot

megaTherion: ld-elf.so.1: Shared object "libcrypto.so.111" not found, required by "libzfs.so.4"

which versions did you upgrade exacly?

bitblt: did you just upgrade from 13.x to 14.0?

that's what it looks like

sound like you already deleted the old shared libs before you upgraded the rest

crest_: I don't really know what the truenas core ui actually does with the jail 'update' button (but for sure I wont be trusting it again)

can you still run freebsd-version -kru

and uname -a

uname -a: FreeBSD radarr 13.1-RELEASE-p2 FreeBSD 13.1-RELEASE-p2 n245412-484f039b1d0 TRUENAS amd64

freebsd-version does not exist in the container

jail* (sorry linux habbits)

so they stripped down the userland beyond what is reasonable on a storage server^^

bitblt: you cant upgrade to 14.x if your host is 13.x

this is not supported

megaTherion: I did not intent to

the annoying truth is that 13.1 has been end of life for a while so expect a bumpy ride

I thought the jail 'update' button only did minor updates

I see, well then just rollback to a prior snapshot

problem solved then

if it ran env PAGER=cat freebsd-update fetch install it should just install security and bug fixes

but it appears to do more

he's not using FreeBSD, he's using a *NAS fork which is out of the scope of FreeBSD vanilla anyways

how familiar with freebsd are you? do you want to understand the problem or just get it working again?

crest_: Great question, I really want to get familiar with freebsd

if you have a snapshot from before the upgrade (not update) rolling back would the easy way to undo it all, but that would include any data you changed in the jail since

Last night I actually deep dived into some stuff before coming here, like iocage etc.

crest_: another HUGE mistake I did is that I did not take any snapshots prior hitting the update in the jail

bitblt: the cleanest way is to go back to a state which works, but if you want you can copy the missing libs out of the host into the jail

its a dirty fix, could work

so my previous snapshot is 4 months old

did their jail frontend do it for you?

crest_: yes, never using it again

I'm now trying to understand all the internals

and manage these things by shell

(I believe I'm pretty experienced with linux, but some basic knowledge around freebsd is trying to settle in my brain from last night now)

does /bin/sh still work in the jail?

crest_: yes

pkg-static works too

crest_: I have a shell inside the jail

but you don't have /bin/freebsd-version?

/bin/freebsd-version: Command not found.

I do have freebsd-update though

the problem is that freebsd 13.1 is end of life which means that the update servers may no longer have everything you need cached

how big is your truenas system (cpu cores, ram)?

crest_: Yeah I kind of got that when `iocage fetch` only showed me 13.2-RELEASE

crest_: Intel(R) Xeon(R) CPU E3-1220 v3 @ 3.10GHz, 16G ECC RAM, 4+4TB storage in zfs mirror + 128G boot pool

there is a good chance that a freebsd 13.2 userland will (mostly) work on a 13.1 kernel, but it would only be by accident

because you're not supposed to run a newer userland on an older kernel (only the other way around is supported)

crest_: ok, lets play

take a snapshot now if you haven't already

that way you can go back if you make it even worse

do you have any services you can't/won't stop still running in the jail?

nop, no problem

I broke my uptime after 325 days, at this point its all about recovery without loosing past 4 months

run sockstat -l -4 -6 to make sure nothing is listening on IP sockets that you forgot about

are you willing to share which service(s) the jail provided?

after you took the snapshot you can try this

big uptimes only suggest that you haven't updated a long time :)

crest_: yes, its a jail that has radarr, similar thing happened to my sonarr,bazarr etc jails

crest_: I just did `iocage snapshot radarr`, next step?

env PAGER=cat freebsd-update --currently-running 13.1-RELEASE -r 13.2-RELEASE upgrade

assuming you're stuck somewhere between freebsd 13.1 and 13.2

Looking up update.FreeBSD.org mirrors... ld-elf.so.1: Shared object "libcrypto.so.111" not found, required by "libprivateldns.so.5"

can you run file -s /bin/sh to learn more about your current userland

librypto.so.111 and libz.so.6 that are missing are major pain points now

ld-elf.so.1: Shared object "libz.so.6" not found, required by "libmagic.so.4"

when running file -s /bin/sh

run ls /.zfs/snapshot

it should show you your snapshots

ok I have a REALLY old one

2022-12-09-22:07:16

that's fine

found libz.so.6 and libcrypto.so.111

I suppose I copy them from the snapshot to the root fs?

run ls /.zfs/snapshot/$name/lib

crest_: yeah, lots of libs there

you can copy the missing from the old snapshot's /lib into the messed up /lib

I just did

pkg now works

freebsd-version too??

it would be kind of pointless because freebsd-version itself contains the userland version number

I messed up

I was on the host

so no in the jail I dont have the relevant libs

can you run mount -v inside the jail?

neither a zfs snapshot

tank/iocage/jails/radarr/root on / (zfs, local, noatime, nfsv4acls)

or zfs list -t snapshot | grep radarr on the host

you can also try to copy the libraries from the host /lib into the jails /lib

crest_: I tried copying from the host but I get

cp: /mnt/tank/iocage/jails/radarr/root/lib/libz.so.6: Read-only file system

you can run the update from the host

with freebsd-update -b <path_to_jail_root>

babz: what is the difference between iocage update and freebsd-update?

babz is right assuming truenas doesn't point freebsd-update to their own updating servers etc.

iocage is what's called a jail manager. a piece of software to manage multiple jails

it's one of the large ones (from the complexity point of view)

host does not have a freebsd-update

zsh: command not found: freebsd-update

freebsd itself has the mechanisms required to run jails, but almost no policies pushed on the user

bitblt: truenas probably removed it because it would be too tempting for a user to bypass their high level upgrade workflow

which would result in truly broken host systems

their design is to take full charge of the host system. you're not supposed to mess with the host. if you want be in control put it in a jail or virtual machine

which is a nice idea until their jail manager exploded into your face

crest_: Ok so I suppose we are back into trying to add the missing libraries libz.so.6 and libcrypto.so.111 into the jail

tried copying them from the host, I get read-only file system

can you run mount -v | grep /jails on the host

loots of stuff

crest_: I suppose you are interested in this:

/mnt/tank/iocage/releases/13.1-RELEASE/root/lib on /mnt/tank/iocage/jails/radarr/root/lib (nullfs, local, noatime, read-only, nfsv4acls, fsid 8bff0c2929000000)

I did this:

cp /.zfs/snapshot/2022-12-09-22:07:16/lib/libz.so.6 /mnt/tank/iocage/releases/13.1-RELEASE/root/lib/

okay the jail manager (iocage) has used nullfs (the linux equivalent would be a bind mounted directory) for the jail

the nullfs mount is read-only to allow them to share a single copy of the userland among multiple jails

crest_: I suppose they are using it to make some kind of container base / overlays

yes but only on a per mountpoint basis

without a dynamic unionfs (because there is no usable in-kernel unionfs)

i'm not familar enough with iocage to know how exactly they mount the writable filesystems on top of that

i assume there will be either a bunch of zfs filesystems mounted into the jail in the relevant places or a single one with writeable nullfs mounts to piece it together

Ok, some progress

ld-elf.so.1: Cannot open "/usr/local/lib/libcrypto.so.11"

thats a (broken) reference to the libcrypto provided by the openssl package (not the one in the base system)

if you have pkg working you can try pkg install -g 'openssl1*'

don't blindly press y

crest_: I have pkg-static working

but check what it wants to change if you get that far

can you run pkg-static install pkg to reinstall the package manager?

crest_: reinstalled pkg, still needs libcrypto.so.11

to be expected

run pkg search openssl

(by the way, really thank you for going though this with me, learning a lot of tricks)

I suppose we need openssl111-1.1.1w

now run pkg info | grep openssl

(yes there are more elegent ways to query the installed packages but this is easier)

I just installed openssl111, pkg is alive

Shall I run pkg update?

yes

and pkg upgrade?

start with pkg update

crest_: works

that updates your local copy of the package repo meta-data

yep done

as in which packages are supposed to exist on the repo mirrors

(similar to debian's apt update I suppose)

there is a difference between freebsd and most linux distros

freebsd splits the system into base (kernel + userland) and ports/packages

the ports tree is a collection of build instructions to fetch, compile and package software

Ok

I suppose that everything I install with pkg install <simplename> is a userland package

from the official repos?

crest_: also, shall I run upgrade now?

bitblt: pkg are 3rd party packages

official in the sense that each port has a maintainer and deals with the upstream

megaTherion: But they live in the freebsd repos / mirrors yeah?

yes

pre-compiled packages which are on official mirrors

they are however, not part of the distribution

bitblt: back

can you run file -s /bin/sh inside the jail now?

crest_: /bin/sh: ELF 64-bit LSB pie executable, x86-64, version 1 (FreeBSD), dynamically linked, interpreter /libexec/ld-elf.so.1, for FreeBSD 13.1, FreeBSD-style, stripped

is that from inside the jail or on the truenas host?

crest_: from the jail

at least that part of of your corrupted userland is from FreeBSD 13.1

crest_: so, what's next?

the proper way to do it would be upgrade your host first, butthat could be problematic if truenas is still stuck on 13.1

you could try to use the iocage interface on the host to prepare a 13.2 jail template and switch your existing jail to it

As far as I know it is even older than that, latest stable version in their site is: 'TrueNAS CORE 13.0-U6.1'

their release is a mix of a newer freebsd kernel with older parts on top of that

uname -a tells you among other details which kernel version you're running

crest_: I suppose now the next step is something in the lines of 'iocage upgrade radarr -r 13.2-RELEASE'

that could work, but it wouldn't be a supported configuration

lets see

i would also recommend that you install the package bsdadminscripts2

for its pkg_libchk script

which scans for missing libs

inside the jail I suppose

some packages like the go compiler will produce false positive because they include test elf files

yes

it finds all elf executables installed via pkg and tries to locate the shared libs required

if it finds any missing libs it will tell you which package contains the executeable, the path and the missing libs

e.g. i just found this on a lab machine

nfs-over-tls-1.2: /usr/local/sbin/rpc.tlsclntd misses libssl.so.111

nfs-over-tls-1.2: /usr/local/sbin/rpc.tlsclntd misses libcrypto.so.111

nfs-over-tls-1.2: /usr/local/sbin/rpc.tlsservd misses libssl.so.111

nfs-over-tls-1.2: /usr/local/sbin/rpc.tlsservd misses libcrypto.so.111

I think we are good after the manual cp of libz.so.6 and libcrypto.so.111

only these missing

radarr-4.7.5.7809_2: /usr/local/share/radarr/bin/libMonoPosixHelper.so misses libiconv.so.2

radarr-4.7.5.7809_2: /usr/local/share/radarr/bin/libcoreclrtraceptprovider.so misses liblttng-ust.so.0

did the jail upgrade to freebsd 13.2 go through?

nop because the radarr package has a dependency on libssl.so.11

that does not exist

i don't see how iocage upgrade to care about that

of course you as user care about the service the jail is supposed to provide

it stopped with that error though

intersting

the easiest way to fix that would be to delete the package from the jail an install it again

because during the installation it will also install the dependencies

if you want to make sure the packages exist you can run pkg install -f -F radarr first

-f to force it to work on already installed packages, -F to only fetch the (latest version of) the packages

or if you want to upgrade everything pkg upgrade -F

i think the usage of iocage plugins also complicates the whole setup

that only updates the packages not the base system

oh you're using plugins^^

yes that does complicate things

if i knew I would just setup everything in vanilla jails

yes I'm using this plugin:

Title: iocage-plugin-index/radarr.json at master · ix-plugin-hub/iocage-plugin-index · GitHub

which in turn uses this repo:

Title: GitHub - freenas/iocage-plugin-radarr

the plugins are a nice idea, but they didn't invest the effort to support them

and I suppose the problem in iocage upgrade is that the plugin has radarr as a dependency

so as part of the upgrade process it tries to install that too

i would try to delete the package before the upgrade

either it reinstalls it or it doesn't in the later case you can reinstall it manually

i don't really know how the package upgrade process happens, but I'm kinda afraid because I see that the program lives in /usr/local/radarr

because it looks to me like the radarr plugin only the port forwarding and runs a trivial post_install.sh script

and its data live in /usr/local/radarr/data

Title: iocage-plugin-radarr/post_install.sh at master · freenas/iocage-plugin-radarr · GitHub

take a snapshot of you jail datasets first *shurg*

it's already broken and if it gets worse you can go back

it's only a media downloader after all

why do I always forget the snapshots

because few file systems truly support them

maybe because I never lived in a powerfull fs as zfs

heck on real freebsd you can even create a new boot env, start it as a jail, run a system upgrade inside the jail and reboot the system into the new boot env

for minimal downtime upgrades

if fails you just boot the old boot env and try again

assuming you have access to the system console (e.g. ipmi, video+keyboard, serial port)

it doesn't help you with cheap budget hosters that don't provide a (trustworthy) way to access the system text console

yes, I'm starting to see the power in all of it

crest_: So I tried upgrade to 13.2-RELEASE in my bazarr jail, here is the log: bsd.to/ctvB

Title: dpaste/ctvB (Plain Text)

crest_: update seemed to work, but the jail base does not seem to be updated?

when i still used truenas it had fairly good documentation on these things

i don't know if they maintained that quality when they moved from their own jail manager to iocage

now they have nothing

either where to click in the gui

or an obvious pointer to the iocage upgrade

truenas.com/docs/core/13.0/uireference/jailspluginsvms ?

Title: Jails, Plugins and Virtual Machines |

and truenas.com/docs/core/13.0/coretutorials/jailspluginsvms/jails ?

Title: Jails |

i remember them going into more detail *sigh*

is it just me, or any one find it "odd" that /usr/ports/UPDATING hasn't seen an update since 20231229 ? I just check the GIT, and it's the same, or has there been nothing of note recently to add

jimmiejaz: they have been talking about splitting up libc and some bsdinstaller efforts.. with a sprinkle of gpart and bsd labels.. but i think it is still work in progress

so not surprised it has not been updated since that date

those are all src efforts, not ports

oh crap, i misread you are right kevans

maybe there are no updates that make it worth updating UPDATING

but omg, this repo hasn't been updated in DAYS! it must be abandoned! i must complain on reddit instead of helping!

Demosthenex, Your comment torments me because I see that happening too often. It's satire but it is also a documentary.

rwp: you think it's bad in a package manager? try common lisp libraries

s/DAYS/YEARS/ fewer commits == mature, not dead.

feel my tormet.

I feel it! I truly feel it. It is a shared torment.

waaah i got an awesome os/program/library/framework for FREE! i demand premium support! i'll pay you in EXPOSURE

gah! wait! c?! who writes in C? i demand you recode this in rust or $favnewlang asap!

god i've done this too long. i hate modern computing

This code is so obsolete. No one has made a commit in the last several hours. It's obviously dead. Let me take over the project so that I can rewrite it entirely and make it into a MODERN thing with MODERN features. Let me alienate everyone else who uses it. Let me leave the project in disgust after breaking almost everything about it.

Just to let you know that I do feel your pain.

OMG! it's not modern because it doesn't have a blinding white background and variable spaced fonts!

holy hannah, it was an observation, not a cry for help. I just don't think I've ever seen that file not updated in a couple months since.. at least 4.0 was released. But flame on, I got thick skin.

jimmiejaz, You did not realize it but you tripped and spilled lemon juice into our open wounds. Ouch! It's okay. It's just the state of things.

they are delayedin their conversion of that file into git-mark-up-down-gui-font-terminaled-asciidoc!

damn i'm salty

jimmiejaz: have you noticed breaking changes, that should be in UPDATING?

That's the purpose of the file, to keep track of breaking changes; if you haven't noticed any, and there's been no additions, it's possible there's been no breaking changes.

It's also possible that they've gone unnoticed, but in that case it might be because something broke that nobody's using, so it's kinda hard to do anything about that when upstream doesn't supply a test harness and there's no standardized test harness.

Hello folks

hey cheezy

hi

I'm in a bit of trouble, I got X11 running well on my x86 FreeBSD installation w/ KDE. I would prefer running Wayland using my AMD 5700 XT GPU, however SDDM seems to crash as soon as I log in to a Wayland session. I'm a bit stumped as to how to debug this, since KWin on Wayland doesn't write Xorg logs, nor anything else I could find. Does anyone have some pointers for me?

Or might it be possible that SDDM with the Plasma Wayland option does not start KDE as outlined in the handbook (e.g. dbus launch --exit-with-x11 ck-launch-session startplasma-wayland)?

can you try just starting wayland without the sddm

For experiment's sake, I also tried with KDE6 - no dice unfortunately

Hi - naïve question - do AMD cpus report estimated wattage if you are not on battery/laptop hardware?

another beginner question - how can I view boot logs from a previous failed boot? eg equivalent of journalctl

or maybe... does freebsd even record logs from a boot that goes into KP before reaching login?

now that iX systems has given up on FreeBSD does would it be too much trolling (on top of a trademark lawsuit waiting to happen) to name the TrueNAS CORE fork TrulyFreeNAS?

.oO( i shouldn't switch tabs mit sentence )

*mid

gn8