ghoti: xmllint with --xpath can extract it

Are there plans to support the openat() flag O_XATTR anytime soon ß

s/ß/\?/

does freebsd configure mtu automatically? and if so – through what mechanisms (dhcp option 26, …?)

freebsd does path mtu discovery out of the box

which has issues

Any recommendations how to setup xfce DE on jail? i have 14.2 host with KDE DE but i want xfce in jail. i have .xinitrc file which does launch kde at bootup but i dont have .Xauhority file. if i create xauthority file then i am unable to boot even on the host. so i dont create it. All i want is an xfce DE on jail.

i'm starting a debian vm in bhyve on freebsd. weird thing. if i give it 6 cpu cores and 1500GB disk, installer finishes and reboots into new system. if i give it 12 cpu cores and 3600GB disk, installer finishes but then reboots to grub> prompt?

demido: it does not even attempt to load the kernel/initrd?

doesn't seem to. no errors in the vm's log (using vm-bhyve pkg)

oh and another change i make, 16GB ram when it works, 48GB when it doesn't

ok i narrowed it down. i tried switching 1 thing at a time to the working config and it's the disk size

1500GB disk0_size works, but 3600GB reboots into grub prompt

Odd. Does it default to a different type of partition table (BIOS vs GPT) between the two?

no clue how can i check?

If you boot from the installer iso, you can check the disk

(or however you get into your installer, really)

If size is the factor, I'd put $5 that the 1.5T disk gets legacy label, and 3.6T GPT, and your system apparently doesn't like GPT in a VM

what you think solution is then?

demido: not using a single huge ass disk.

why's that so bad?

It's not terribly bad per se. Just... I don't like it. I don't like extending disks. LVM makes it unnecessary anyway.

I also have a huge dislike of partitions of any kind. My server didn't have any, laptop doesn't have any on internal devices.

Small disk for OS and swap, and fixed-size extra disks for data. Need more? Add a new disk, pvcreate, vgextend, lvresize, bob's your uncle.

There is a noticeable performance gain to be had on physical machines if you use multiple smaller disks vs one big, but on VMs I suppose it's negligible.

ok it took forever but i have good info now. trying to install debian in a bhyve vm on freebsd. with 2048GB disk install finishes, reboots into new system. with >2048GB disk install finishes, reboots to grub prompt. what's causing that and how can i fix without resorting to multiple disks?

does anybody know a definitive "here's what you need to know about swap" article for freebsd?

I'm trying to figure out why I have 72% swap usage on a system with 128GiB RAM, after building from sources a couple of times

this system shouldn't need any swap at all

$ doas top -Sjw -mcpu -oswap -b

last pid: 42; load averages: 0.33, 0.18, 0.21 up 0+17:57:27 12:01:03

205 processes: 3 running, 200 sleeping, 2 waiting

CPU: 6.4% user, 0.0% nice, 0.6% system, 0.0% interrupt, 93.0% idle

Mem: 20G Active, 32G Inact, 62G Laundry, 7679M Wired, 104K Buf, 2760M Free

apologies!

someone found out what happens when you don't use a pastebin ;)

I found wiki.freebsd.org/Memory and added a bunch of links (mostly by markj) to the end of it. this is enough for me to dig through.

so, ~ 60% of my swap disappeared when I got rid of /usr/obj as tmpfs

I would have naively expected a tmpfs not to be swapped out, I would have expected a ENOSPACE or similar.

but its clearly documented in tmpfs(4) that swap will be used.

let's try these builds *without* swap

ok, so its all clear now.

its tmpfs, I will redo the builds without swap, and keep an eye on total memory.

I guess we get close to max mem, and then tmpfs is flushed to disk

it took forever but i have good info now. trying to install debian in a bhyve vm on freebsd. with 2048GB disk install finishes, reboots into new system. with >2048GB disk install finishes, reboots to grub prompt. what's causing that and how can i fix without resorting to multiple disks? i tried using uefi loader but that only works with

graphics="yes" which requires vnc, but i wanna stay with a pure console connection

i think github.com/churchers/vm-bhyve/wiki/…Console-Output-with-the-UEFI-loader is my issue (found at churchers/vm-bhyve #398) will those work for debian?

demido: any reason why you want to stick with a single disk?

to learn

Unless you know for certain that your data will never grow beyond the initial size, that is

Because LVM exists for a reason, and makes the physical disk underneath irrelevant

I had unix systems with well over a thousand devices connected - and yet only a dozen or so filesystems

to learn

You can learn LVM, then. Instead of eighties tech. :°)

i'll learn what i choose instead of what you choose thx

¯\_(ツ)_/¯

phryk: looks like it's initialised by the interface somehow, when the interface is created, and is interface-dependent. it can also be set later on with dhcp

using pkg can you show all packages use libfmt

grep -r 'devel/libfmt' /usr/ports/ <--- or is that better?

quickrr: use `pkg info -r libfmt`

dch: very nice, thanks :)

demido: share your vm-bhyve conf? my old debian9 with usefi works without console, I just used graphical console for the installer.

demido: but yes, the grub doc you posted should work just fine on debian too

dch i want the opposite of you, i want to use console and NO vnc

it's just loader="uefi", graphics="yes", and i can use vnc to install and start it. but i want to use vm console for install and start, no vnc at all

demido: thats not what I said, I only used vnc for the install, and afterwards can skip it. the doc you linked to looks like it will work, ..

demido: but it requires modifying grub config after install.

this (to me) looks like a debian/ubuntu issue rather than a specific bhyve one

p5r.uk/blog/2020/instaling-debian-over-serial-console.html covers altering the installer to cater for that case

is 'update-grub' a freebsd pkg?

demido: no. all of this stuff requires hacking debian. either after install, or on the debian .iso you downloaded

it would be nice if all OS could be installed via (serial) console

sheesh. just to be able to use a large vm disk dang

duno if you saw what i learned but 2048GB is the max that grub works seamlessly with. more than that and that's when i ran into problems

demido: is this a GPT format disk, or MBR? the uefi loader + GPT format should be able to deal with very big disks

not sure. i just used debian installer defaults. i'm using uefi loader now instead of grub and it works, but i have to start it with graphics="yes"

haven't figured out uefi sans vnc yet

huh, bugs.launchpad.net/ubuntu/+source/grub2/+bug/1666853

lol sitting around for years, 'nice'

demido: when you boot via bhyve without graphical console, do you get (at any stage) a terminal?

if i vm install with grub and >2048GB, vm console works and i can get through installation. but at end of debian install it wants to reboot so i do that and it boots to grub> prompt

instead of the OS

so 2 separate issues, (1) debian can't use large disks for the root fs, that we can't fix here obviously.

and you want to be able to install bhyve via serial consolel, that we probably can help

i got debian vm running with a 4TB disk  for root fs tho

i just had to use loader="uefi", graphics="yes" and vnc

is grub using the serial console?

would that do it?

ya, works great, as long as disk is <=2048GB

but i need my vm to have 4TB disk

how does graphics let you use 4TB?

I'm not connecting these

back

ya, works great, as long as disk is <=2048GB, but i need my vm to have 4TB disk

how does graphics let you use 4TB?

these dots don't connect for me

demido: we seem to be going in circles here. (1) is a debian issue, we can't fix that for you. See the above bug I linked.

dch: I prefer spirals

inbound spirals, converging on a solution

i'm using vm-bhyve. in my vm config, if i use grub loader, i can run vm console for install and using installed vm. i just gotta keep disk <=2048GB. to use a bigger disk, i gotta set vm config to loader="uefi" and graphics="yes" but then i can't use vm console, i gotta connect with vnc and i don't want that. i want console still

you need loader=uefi without graphics?

I've never used vm-bhyve so I'm not sure where to dig

ya but when i try to boot that, it says something like no video detected, using blind mode

what does?

the screen says that after i run vm console

is it vm-bhyve, grub or linux?

what screen?

the text on the screen. using loader="uefi" not grub

well i'm gonna crash it's late af. ty for trying to help me dch i'm prolly too tired to comprehend

demido: if you are booting with uefi than wait for vm-bhyve v1.6.2

version 1.6.0 is broken

the fix is ongoing

if you are booting linux with grub, vm-bhyve should work fine

[tj], jmnbtsls1E: thanks for the info. :)

hm, entering capability mode breaks rusts backtrace printing, afaik it uses libunwind, what do I have to call before cap_enter() so that it works properly?

nimaje: can you provide ktrace output? i can't think off the top of my head what libunwind would be doing that would make capsicum unhappy

it doesn't make capsicum unhappy, it just doesn't show function names and file locations (just <unknown> instead)

oh, hm. can you reproduce that in C using backtrace + backtrace_symbols by using libexecinfo?

that also uses libunwind

libexecinfo alone definitely doesn't look capsicum friendly

neither is my spincter

*sphincter

backtrace_symbols immediately wants to open() the binary, which wants both sysctl lookup and a subsequent open(2)

that will probably cause the unknown names then

still, i'd expect the actual unwinding to work, it's just the symbol resolution that would be broken then

I think everything else seems fine, though I'm not really sure how this works if you need to cross into a DSO or something

tbf they didn't say that the unwinding didn't work, just that it doesn't resolve names and locations

yeah

yeah, the unwinding works, but just some addresses doesn't help really much to debug stuff

not a whole lot you can do there, unfortunately; you could make it work, but you'd need to re-work libexecinfo a bit at a minimum

it wants to re-open() the binary fd every time you call backtrace_symbols(), I would suspect you could bake in a global fd and add a function to preload the exe binary, and use that if it's not -1 -- otherwise keeping the current open_self() calls

I don't think you'd want to remove the open_self() at runtime call just to avoid changing applications and to allow some flexibility to not burn an fd at the cost of not much complexity

0x0.st/8HFT.c is a bit better than what rust produces, it doesn't get the main function name, but figures out that it comes from the binary and it prints libc stuff

0x0.st/8HFQ.rs a rust equivalent

this seems to work well: termbin.com/2r1u

maybe it'll never grow anything else needed in a sandbox, but as a PoC

whoops, that has a major flaw in that it will close(self_fd) at the end of backtrace_symbols_fmt; easy fix

How can I find all snapshots that have changes to a given path?

I'd rather something faster than grepping zfs-diff

i don't think that there is

Hi, in a Jail, if I run "nc -lv 5000" it hangs, instead of showing "Listening on [0.0.0.0] (family 0, port 5000)"

martinrame what kind of jail networking? inherit? new? or vnet?

inherit

antranigv: allow.raw_sockets; ipv4=inherit; ipv6=inherit;

antranigv: I don't really need ipv6 here.

antranigv: this is not a vnet jail btw.

what makes you think it's hanging and not just listening? you can check.. i don't think base netcat prints anything on listen. you might have a port installed that's giving you that output.

martinrame with inherit, it inherits the host's network stack. as long as no one on the host listens on 5000, it should work fine. can you check the sockets using sockstat ?

isley: because I tried telnetting from the host and it doesn't respond.

antranigv: from inside the jail?

martinrame in case of inherit, it should not matter.

antranigv: termbin.com

antranigv: termbin.com/0y8m

yeah, base nc doesn't print a message about listening

even with -v it'll just print when a client connects

kevans: you are right. I tested on linux and it prints the message, but not on FreeBSD.

you've got an nc trying to do DNS stuff, is that this one?

It looks like a firewall issue. If I run nc -l 5000 from the host I can telnet to it, but not from the jail.

sorry, from the jail it also works. I used nc to try to debug a Phoenix (port 4000) issue, there it hangs.

so the original report is wrong and it's really just when trying to connect to Phoenix? (which seems to be some kind of web framework?)

nothing listening, just connecting?

oh. maybe "not nc at all, just phoenix and nc was to make sure bind/listen was fine"

kevans: yes, nc binds ok.

i know i am going to be asking a loaded question here: I need to get a GPU for my ollama bhyve and was wondering.. is there a "list" of GPUs that work with bhyve?

all i have been able to find is a AMD list.. which is great but curious if there is another "official" lishttps://wiki.freebsd.org/Graphics/AMD-GPU-Matrix

voy4g3r2: I use a rtx 3090, dell OEM and it works great. Some nvidia gpu's have a problem with resetting so you need to restart the whole machine if your restart the VM

you will also need to apply a small patch to bhyve if you want a nvidia gpu

12 or 6 gig model?

not bad price point either in my 200-300 range budget

24gig, but amd cards might work without any hacks

just make sure you cpu/mobo combo supports passthrough

getz: will the motherboard say passthrough or the like? i have a i3 and this board: microcenter.com/product/662283/B760…Intel_LGA_1700_microATX_Motherboard and so far i have been able to do a lot of things with bhyve and it.. just not GPU passthrough

voy4g3r2: *disclaimer* i don't know a lot about freebsd but if I've understood what you want to do correctly, i think the thing that your motherboard needs to support is "IOMMU". that should at least give you something to google and see whether it looks plausible.

mzar know when vm-bhyve 1.6.2 is coming?

the patch is ready, if driesm@ pushes it fast then by the beginning of next week 1.6.2 will be in official repo

mzar ok ty! there any place i can read about what it's going to address? to see if it's involved with th eprob i'm having

demido: you can buid it from ports, update patch is on the FreeBSD bugzilla

demido: see PRs 273560 and 284254 on Bugzilla

hm maybe. ok i'll collect more info on this prob and share in an hour or two. i wanna get all the details right

crab: thanks for the tip.

looks like my 500w power supply will not work with these beast video cards..