meena remember me asking if you freebsd-update production boxes or update the template and redeploy? well same with pkg upgrade, you don't do that on production boxes either right?

Hi

where's the guide to major upgrade ?

that will be my first one

Title: FreeBSD Handbook | FreeBSD Documentation Portal

debdrup: thanks

just noticed that 14.0 is not out

i need to upgrade to 13.2

i missed the 13.2 announcement

how to get notified ?

The freebsd-announce@ mailing list: lists.freebsd.org/subscription/freebsd-announce

Title: FreeBSD Mailing lists: subscription for freebsd-announce

There's also some RSS feeds at the bottom of the website.

thanks

hi! i'm having issues installing xwayland-devel

DBG(1)[1401]> curl> fetching pkg.FreeBSD.org/FreeBSD:13:amd64/la…ll/xwayland-devel-21.0.99.1.495.pkg

< HTTP/1.1 404 Not Found

Title: FreshPorts -- x11-servers/xwayland-devel: X11 server as Wayland client (development snapshot)

this is 13.2-RELEASE-p1

(latest, not quaterly)

any idea what's going on?

full debug logs: paste.sr.ht/blob/7aaae62154be3429a5d1710745e4599087c0aed3

i seeing version xwayland-devel-21.0.99.1.499 ( not *.495 )

maybe, it is needed to update pkg database first

i've done a pkg update right before

it's been happening for 1 or 2 days (it's breaking the Sway CI)

emersion: maybe a dependency is requesting that specific version

emersion: do you have a proxy cache in front?

no proxy

(this is builds.sr.ht)

yeah I figured

hm

that means packagesite.pkg is not in sync with the packages available

which cannot happen, so it means you somehow get an outdated packagesite.pkg this is weird

should i just wait then?

eh

let me check the cluster

emersion: test here I really have .499

a minimal reproducer is builds.sr.ht/~emersion/job/1032719

Title: build #1032719 - failed

from my 13 machines

I don't understand how you end up with .495

I have a builds.sr.ht account

let me check

you can resubmit that manifest, and then SSH into the machine to inspect it

or ask me to run commands for you

I can do it

hum

I bet this is quarterly

hm, it shouldn't be…

Title: ~sircmpwn/builds.sr.ht: images/freebsd/genimg - sourcehut git

ah

how do we get output from the SAT solver?

meena: DEBUG_DEVEL=4, but that won't help here

emersion: I think I know

emersion: if I ssh connect

sudo pkg update -f

I got the right xwayland

so the cache thinks it's up-to-date but it's not?

yes

but this si weird

I don't know where you do getch the bad packagesite.pkg from

is this based on modification time?

yes

url + mtime

if url changes

we re-generated images from the script linked above each week

it forces a update

re-generate*

and we do nothing else to images

I am reading

a hack could be to add pkg update -f in /etc/rc.local

so at each boot it ensures it has the lastest update

another way

probably cleaner would be

rm /var/db/pkg/repo-FreeBSD.sqlite

so the first time you call pkg install something it will install the missing bits

s/install.*/update the metadata/

right now you have the metadata from the creation of the images and somehow it considers it is to date

hm, i'd rather not workaround this

i guess i'll just disable freebsd for now

:(

I use it :D

see if it fixes itself in a week

hum

maybe I should remove the mtime

and set the time of the packagesite.pkg file into the metadata of the repo

(I am the author of pkg)

is there a way to check the bad mtime?

from the local cache

could this be due to e.g. NTP?

(an alternative to mtime would be ETag, but that requires HTTP server support)

pkg it fetching the mtime from /var/db/pkg/repo-FreeBSD.sqlite

and say "If-Modified-Since: thesaidmtime"

what trying to fetch the packagesite.pkg

let me try something else

in my case:

> GET /FreeBSD:13:amd64/latest/packagesite.pkg HTTP/1.1

If-Modified-Since: Fri, 28 Jul 2023 19:04:20 GMT

< Last-Modified: Fri, 28 Jul 2023 16:55:38 GMT

2h sounds like too much time for a NTP related issue

where does the mtime come from in the cache?

does pkg set the mtime from the Last-Modified header?

or is it just the current time when the file is downloaded?

yes

from the Last-Modified header

so I bet the mtime is lost at the creation of the image

so if I want pkg to be more robust I need to store this date inside the db metadata instead of relying on the filesystem

hm, i wonder how this could happen

we just bootstrap a filesystem and mount it

I don't know :D

so FS metadata should be exactly the same…

but I can confirm the date on the fs is the one in If-modified-since

-rw-r--r-- 1 root wheel 57643008 Jul 28 19:04 /var/db/pkg/repo-FreeBSD.sqlite

it is impossible this was the date of the packagesite at the time of creation of the qcow

does pkg open the file in read-write mode, when reading it?

no

at least it should not :D

double checked, it does not

Title: pkg/libpkg/repo/binary/update.c at master · freebsd/pkg · GitHub

t is a bit weird here, it's set to meta and then overwritten by the db's?

but that's not the cause of the bug, since the bug is about the db's mtime

emersion: this only happens if you run pkg update or pkg upgrade

and if you are root

(root being checked elsewhere)

suggestion: use TIMEVALUE_LARGE in github.com/freebsd/pkg/blob/master/libpkg/fetch_libcurl.c#L466

Title: pkg/libpkg/fetch_libcurl.c at master · freebsd/pkg · GitHub

arg, so i have a small biz cisco switch, and was using lacp with freebsd 13. it keeps flapping and having terrible response, even when i turn off all the options

until i removed 1 of the connections, now it works fine

emersion: done (for timevalue_large) would be in the next pkg

could this cause mtime bump? github.com/freebsd/pkg/blob/master/libpkg/repo/binary/update.c#L603

Title: pkg/libpkg/repo/binary/update.c at master · freebsd/pkg · GitHub

which goes through here github.com/freebsd/pkg/blob/master/libpkg/repo/binary/init.c#L336

Title: pkg/libpkg/repo/binary/init.c at master · freebsd/pkg · GitHub

because of the W_OK

emersion: that would be new

this code has been working without any mtime issue since it has been introduced very long ago

I agree this is suboptimal and could be greatly improve

I need to rewrite this old code since like forever

it was designed for a use case which never took off and is clearly over complicated for what it does ;)

emersion: I think I do have the bug!

this is a regression from the switch to libcurl

I will fix it in the afternoon

I am able to reproduce it locally now

oh nice

let me know if i can help

sure

emersion: that thing bapt suggested with removing the SQLite db isn't really a workaround, imo, it's something most Linux distros would do too. To make the image smaller, and cleaner, guarantee the first package install will just work

but if i can avoid downloading the index each time a package is built, it's a good thing imho

each time a buiold manifest is submitted*

build*

image size is not a concern here

getting the image up and ready as fast as possible is our priority

OTOH, most of the time the user will fetch packages, which will probably take a lot of time compared to refreshing the index

everything is a tradeoff

emersion: how often to you update the base image?

each week

then you approach is sane yes

keeping the db

ack!

as long as I fix the bug ;)

yeah, if we don't update the image often, then there's a good chance it'll get outdated and re-downloaded anyways

aha :P

man, i wish bapt would fix my bugs so quickly…

emersion: fixed

what should I use to credit you on the commit log ?

meena: which bug from you is not fixed?

Reported-by: Simon Ser <contact⊙ef> maybe?

sure

or whatever convention the project uses

bapt: freebsd/pkg #2169

thanks a lot!

Title: pkg fails to validate rsa signature on 14.0-CURRENT · Issue #2169 · freebsd/pkg · GitHub

2169 – zephyr port disagrees with Kerberos causing compilation error bugs.freebsd.org/bugzilla/show_bug.cgi?id=2169

meena: this is not a bug this is expected

I haven't written the comment yet

why is it expected??

openssl3 fallout,

isn't that a regression then?

no

or unfixable one

things signed by pkg build on openssl3 can sign and be verified on pkg build with openssl 1

so what does that mean? do i have to change the way I'm signing packages?

aka sign from current it will work for everyone

oh… wild

sign from openssl1 it will not work for current

we are not forward compatible here

this is due to a bad usage of openssl since like forever (only for the pubkey, not the fingerprint)

and openssl3 does not allow anymore to abuse its API is this area

meaning we cannot recreate validate old signatures anymore when built with openssl3

now pkg properly use this API

so we are backward compatible but not forward compatible

fingerprint usage was and remains ok

and why can't we properly use that API on OpenSSL 1.1?

emersion: FYI: freebsd/pkg 0ec04db

Title: curl: ensure curl asked for the filetime of the remote file. · freebsd/pkg@0ec04db · GitHub

thanks for reporting

<3

I'll check for the open issues and will probably issue a 1.20.5 today or tomorrow

are characters like ~ and . special bash characters or only characters that are interpreted by unix command?

maybe to make it a bit more robust we could check whether the time we gte back from CURLINFO_FILETIME_T is -1

and set fi->mtime to 0 in that case?

or do we already check for -1?

(just to make things easier to debug if the HTTP server doesn't return Last-Modified for instance)

emersion: yes that was my plan

the return form CURLINFO_FILETIME_T was CURLE_OK

I would have expected an error

right, curl doesn't return an error sadly

Title: curl/lib/getinfo.c at 47a3e6e577b019b8dfce8d3f8df764a8dd427fd2 · curl/curl · GitHub

just -1 and CURLE_OK

the example code in the docs explicitly check filetime >= 0

yup

DBG(1)[91624]> Impossible to get the value from Last-Modified HTTP header

now it will be easier to debug ;)

should probably not be a debug info but more a warning

nice

Title: curl: warn loudly when not able to provide remote Last modification time · freebsd/pkg@942b46c · GitHub

meena: freebsd/pkg 942b46c

Title: curl: warn loudly when not able to provide remote Last modification time · freebsd/pkg@942b46c · GitHub

looks good

fun it seems python3 -m http.server does not report Last-modified apaprently

it sends the header field for me…

< Server: SimpleHTTP/0.6 Python/3.11.3

< Last-Modified: Mon, 22 May 2023 15:02:21 GMT

(not for directory listings, as expected)

strange the unit tests complains about it

ah no actually the regression tests shows another real issue

:^)

I was still looking for Last-Modified if I received a 304 :D

aha

meena remember me asking if you freebsd-update production boxes or update the template and redeploy? well same with pkg upgrade, you don't do that on production boxes either right?

polyex: i usually find pkg upgrade less troublesome, but other people disagree

i'm gonna switch from quarterly to latest repo too, so when i build a new server image it has latest versions, then on server disallow freebsd-update and pkg upgrade, keep immutable in production

cool

question is, in case i forget later my decision, can i somehow disable freebsd-update and pkg from being run?

you can remove them from the image, but that's not gonna stop anyone who is determined. also doing that disables, or rather breaks, periodic(8) scans for vulnerabilities

damn i don't wanna break anything

well, you can also disable those scans, and only have a few dedicated machines where they run, but chances are, if you're rebuilding images, you will be quick enough to catch CVEs

I don't think anything in periodic runs freebsd-update?

security runs pkg check and pkg audit

crest: as I just saw your wireguard devd / rc.d scripts, why do you use resolvconf … -x ? that breaks use cases where the vpn should just be an additional network

nimaje: because that's what wg-quick does and i emulated it's invocation of resolvconf

if you don't want this use a PostUp hook

parser supports multiline scripts including here docs

nimaje: this should work:

PostUp = resolvconf -a %i.wg << EOF

PostUp = nameserver 8.8.8.8

PostUp = EOF

PreDown = resolvconf -d %i.wg

wait better don't use .wg because it could conflict with the dns up/down automation

ah, yes, I remember, I wanted to create a bug report against wg-quick, as they use -x too, but only address being ::/0,0.0.0.0/0 should have that exclusive effect

i decided against implementing saveconfig because i didn't want to make the script even longer

and i refuse to implement the insane daemon mode mucking with the routing table

a shell script "parsing" `route monitor` isn't only to linearly scan the routing table on every change is flawed in so many ways

just because some warez kiddie couldn't be bothered to learn the correct way to do policy based routing using multiple routing tables or domains before starting a torrent client </rant>

maybe i could add some resolvconf args to pass through similar to Sticky?

Sticky (configurable via the wireguard config file or the file mode on the config file) tells the rc.d script to leave the interface around similar to $cloned_interfaces

e.g. sysrc cloned_interfaces+=gre0:sticky

would create a gre0 interface that sticks around even after a service netif stop gre0

in case you want to reference it somewhere e.g. a ipfw or pf configuration

stealing the sticky bit for this is a dirty trick, but it allows sharing unmodified wireguard configuration files with other operating systems

nimaje: the PostUp/PreDown hook cover your needs?

or would say something like SharedDNS = 8.8.8.8,example.org be a lot better for your usecase?

maybe ExclusiveDNS/SharedDNS/PrivateDNS with DNS as a short alias for ExclusiveDNS?

corresponding to the -x, nothing and -p?

So. . . I was playing with glances on a FreeBSD VPS I have. . . and it prints this line below the hostname:

Cloud instance 56f2106129679db187c38ecb36af5417

Where does that get read from?

I thought of sysctl, procfs, or /var/run/dmesg.boot, but all no. . .

from cloud-init if you use it?

I was guessing motd?

I dont' use cloud-init.

And motd is just the OS version string and what else I put there.

what's the exact context of the message?

glances is like a top(1) on steroids, written in python. .. uses shutil modules.

In its default config, in the upper left corner, it prints the full hostname, and below that it says what I pasted above.

CrtxReavr: grep -r "Cloud.*hostname" /etc /usr/local/etc

ooops

grep -r "Cloud.*instance " /etc /usr/local/etc

I'd expect it to be from sysctl or kenv

Found it.

curl -s 169.254.169.254/latest/meta-data/instance-id

Standardish cloud meta-data convention.

IT's an OpenStack API thing.

But many other things use it too.

new OpenSSL vulnerability / release

"Due to the low severity of this issue we are not issuing new releases of

OpenSSL at this time."

crest: look, reading is hard, okay

meena: i was just confused looking for the new release i missed

and wanted to safe others from following the same path

crest: just saw that DNS is documented as using resolveconf -x for some reason, no idea why you would want to use the dns of some vpn in the case of using vpns as additional networks, there you just want to give the dns a part of the hierarchie

nimaje: imo it depends on your usecase, but i don't want to break the *semantics* of a wireguard configuration by redefining the meaning of the "DNS" property to suite your (or my) usecase a little better

what is implemented should be compatible to wg-quick(8) to avoid fragmenting the ecosystem. we don't need an other tower of babel like situation (one IPsec is enough)

+1

yeah, DNS is documented as using resolveconf -x so you should too (but I still think it was a bad idea when they decided that)

you're welcome to try and convince upstream otherwise, we just need to coordinate on stuff like that

when i connect my laptop to $corporate VPN but don't use their DNS I can't access shit on their network

That is a typical issue with VPNs. Private DNS space too.

If one does not want to route all DNS through the VPN, or have several VPNs concurrently, it is possible to configure nameservers such as unbound to forward specific domains to specific upstream nameservers.

^ that's what I do

and specific reverse zones

<3: CFLAGS+= -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang