anyone going to be at FOSDEM?

is it a problem to get console messages that the box is out of swap space? it's a box with 192gb ram, 4gb default swap space, ufs. all it does is run 2 vms, each given 64gb ram, leaving base OS with 64gb

i run the vms with -5 nice fwiw

is it as simple as i need to increase the swap from 4gb or do i need to tune something ?

im trying to customize my bsdinstall installerconfig preamble to make a bigger than default swap. so i changed export PARTITIONS="da0" to export PARTITIONS="da0 GPT { auto freebsd-ufs /, 16G freebsd-swap }" and the installation ran but i booted into system and top reported swap as 512mb. what did i do wrong?

i reversed the order of the 2 partitions and that worked. so i'm gonna run my workload with 16gb swap and see if swap errors go away

hi

after all that

uh, demido you could increase the swap if you need to record kernel crashes, but otherwise 4gb is plenty

rtprio why am i getting out of swap space console messages then?

well, do you have 4gb or do you have 512mb?

also, what is the system doing

i can run a base system with 1GB or so of ram, so why is base OS left with basically 0 memory free when running 2 vms that each get 64gb?

maybe you came in late lemme paste...

is it a problem to get console messages that the box is out of swap space? it's a box with 192gb ram, 4gb default swap space, ufs. all it does is run 2 vms, each given 64gb ram, leaving base OS with 64gb. i run the vms with -5 nice fwiw

so by my math, you should have 192-(128) of ram left?

so i woulda thought 64gb for base OS would be plenty but instead it has like 4gb free, swap full, console messages about being out of swap

ya should

4104M Free in sudo top -a

zfs system?

100% Inuse swap

no ufs

can you paste top, preferably with the 2 vms showing?

86G active, 56G inact, 38G laundry

yep sec

the 2 vms are both using 50G mem of their 64G allotment btw

system has 64 logical cores, 30 to each of the 2 vms, 4 left for host

fwiw i'm spinning up a 2nd identical system with only 2 changes, no -5 NICE on vms, and 16G swap instead of the default 4G. will be another hour or 2 till they are done

not sure man

if you have space to spare, since you're on ufs you could add a swapfile

think freebsd has a mem leak or smth?

i'd be curious to see how much swap you use before it levels off

well i'll see what this 16G of swap does. if i fill it i'm gonna try it again with a 3rd machine sized to 64G swap

if it still fills it i'll know something's up

sure, but the swapfile would get you answers a bit more quickly, i suspect

i'm using vm-bhyve or whatever that tool is

yes, i assumed

do you know how to check if my swap is being used in a bad way? i know it's fine to use swap for infrequently accessed stuff, but it's bad if it's constantly being accessed hot (right?)

ps -f but when i do it on my system i don't really understand it

do the vm's themselves use swap? historiucally speaking

afaik no

you've got a lot of inactive pages in an os that you're simulating

the_oz_ well the vms are debian and they have 1G swap inside them

the os may be inactive

and the 1G debian swaps are fullish

Hmmm.

ya all the inactive mem being used made me think maybe bhyve was leaking mem or something

as if the 64G to each vm isn't being contained

well at least with my 2-8gb vms nothing like that has ever happened

think giving 30 cores to each vm is too much and somehow shitting ram?

i don't see how; again, i might just give the system some swap and see how much it consumes before stopping

ok :/ i'll do that and report back ty

just use a swapfile, while it's going

you can add without rebooting?

truncate -s 20G /usr/swapfile

swapon /userr/swapfile

run it as root or?

er, swapon -E /usr/swapfile

yes, as root

i'll try 100G just to give it room

swapon: ioctl(DIOCGDELETE): Inappopriate ioctl for device \n swapon: /usr/swapfile: block device required

uh'

oh shit you still have to md it

what's md?

it turns a file into a block

how do i do that?

mdconfig -t vnode -f /usr/swapfile

swapon /dev/md0

swapon -E /dev/md0 or no -E?

no -E

oh shit

i knew something was wrong damn

do i ctrl+c ?

i don't know what's happening

i ran swapon -E /dev/md0 :(

did it not already return? what's swapinfo show?

it might need to be in another window

ya it was hung. i terminted it then ran the command right, immediately returned

swapinfo shows the original 4G and now the /dev/md0

it probably seemed hung, when in fact the system was trying to swap to it right away

i added the 100G swap file to the 2nd system that's in the process of spinning up to see what happens with it too

swap file is starting to get used

2nd box down to 28G Free and swap is starting to get used. up to 96M used

26G Free, up to 774M swap used

i'm tempted to try disabling swap. like why the heck is base OS using >64G ram

22G Free, 4G swap used

i'll stop spamming channel. i'll just report numbers in a couple hours when 2nd server is done spinning up

live sysadminning is the action packed content I live for

:D

:)

oooo

im using virtio-blk and virtio-net for the vms, maybe they leak?

really feels like bhyve is leaking mem

hi all ... . skraito here ... .

after installing wpa_supplicant , will wpa_supplicant system service which under rc.d directory be generated automatically

xxy

hi

you love freebsd ? and wpa ?

which version you use

We only publish until version 14.1

i can't find wpa_supplicant under rc.d directory, 14.0

so don't believe if you get 14.2

type find / | grep -i wpa

try it

ask your government to release 14.1

We [ 0day (xc) Our ] the one that code all ... .

is grandpa and grandma that Code freebsd

they got it from Us ... .

when i am in singapore at 2005

that's so old but still stable

harvar-oday: how did you know where do i come from?

i come from ?

ask my wife yourself skraitow Lord Jesus Christ

i am in close wisdom so i don't know where are you from

which country are you from xxy

guess

don't know

may be america

i also wish i had been born in america

why

where are you from

china

oh you are from china that's good

what do you want china

are you from government

i already quit UN so if you want you can whisper me

but We charge 700 unknown max x 30 billion

skraitow: try not to behave like a 14 year old this time

what do you mean by that Alver

i am skraito Kid

how are you

is Us [ 0day (xc) Our ] that code freebsd so don't claim Our code

We already release it

please stay ~ on topic there’s a -social chan if you need to banter

you get it from Us skraito and skraitow which is Lord Jesus Christ at singapore 2005 ... .

how are you alver and dch

what your question about freebsd

We are the coder

ask me anything if you guys want to know

^^

sorry Kid

is skraitow here now ( Lord Jesus Christ skraito wife ) 14.2 grandma will release later grandma is testing skraito code ... .

interesting

from skraito now

which version did you use

14.1 is running with s41lb

try it Kid

wireless is working too

yeah

grandpa already try it at s41lb

is 14.1 with everything work even wireless

so if you want version 14.2 ask my Wife yourself is her that hold it Kid

she definitely having fun with it

We don't care We got k-line or not

per k-line is 7 trillion dollar

Don't ever claim Our code

from [ 0day (xc) Our ]

We open source it but remember is Us that code it ... .

so go modify and release your own and make your own company Kid ... .

modify or not modify just claim Now

We already got pay ... .

this is better than reddit, i picked a good night to not be able to sleep

ya that was some good content. even better than my live sysadmining. btw 2nd system still spinning up and 4G Free, 8G swap Used

still feels like bhyve is leaking mem

no reason swaps should be this hot

back all skraito here

remember

s41lb laptop really run on 14.1

so try it and let me know

yea but what about my bhyve memory leak issue

This feels like 1998 all over again

why did time loop back

ketas: scriptkiddies talking all 1337 and h4x0r on IRC :°)

where's my mirc

that's how i learned to code too

hi all

anyone run into bhyve leaking mem in 14.2? got a ufs system with 192G ram. all it does is run 2 bhyve vms that each get 64G ram. by the time the 2 vms are each using 50G ram, the host system is out of mem and already using G's of swap

ketas

you love mirc ?

that's nice

demido: where does the ram go in host?

14.2 ?

how you get it ?

not Even Us release it ... .

ketas no clue. here's top of top termbin.com/1xif

you must be running fake AstaraOS

when i turn on swap in top (W) it shows basically 0 for everything

hmm

*shrugs* Run two vms on a box and never seen them max out their allocated mem. Might be the VM os

sbr how could it be the vm OS? does bhyve not enforce mem limits on the guests?

64+64=128

i give 64G to each vm, the vm shouldn't be able to use a drop more than that in freebsd host

ya that leaves 64G for freebsd host and i don't see where it's going

demido: but the host has to manage filesystem memory too

but with ufs that's a lot? i thought only zfs used a lot

the 2 vms do do a lot of file storage/io

doesn't it give fs cache away on pressure?

not swapping 100%

how can i check?

and ya it should

Yes, that it should, I don't know what else is going on. Just rebooted this box for 14.2 update, was up for ~6 months and had allocated ~9/32gb of ram

weird eh

ZFS will cache agressively if allowed and not stick to ram limit of the VM

ya but this is ufs

I have only a cursory understanding of FreeBSD memory info, wiki.freebsd.org/Memory has a brief overview, and some expert links at the bottom

also, any tmpfs usage on this host will be backed by swap if memory runs low

so the 2 vms have ~50G resident out of the 64G allocated. host system's 64G is totally used up and ~12G swap is actively used. 'cool'

really feels like something isn't right in mem mgmt

demido: is laundry mem (top) staying this high?

ya i have 2 boxes configured more or less the same and teir laundry are 28G and 22G

ridcully ^

is Laundry only on ufs systems or? because just looked at my zfs system and it doesn't have ANY Laundry listed in sudo top -a

it's a freebsd 14.1 system tho, so maybe 14.2 has a mem leak or bug?

forums.freebsd.org/threads/how-to-f…emory-leak-on-14-1-release-p4.95014 seems related

hi all

lists.freebsd.org/archives/freebsd-current/2023-July/004027.html seems related too

i hope to upgrade freebsd from 14.1 to 14.2 , when i run command "freebsd-update upgrade -r 14.2-RELEASE" , it prompt "The following files are affected by updates. No changes have

been downloaded, however, because the files have been modified

locally:

is anyone willing to help a GhostBSD user? I asked in their channel but im not sure how active it is

hi all

We already say xxy no one release 14.2

check yourself

how do you can update it anyway

ask skraitow yourself if you don't believe me

which is Lord Jesus Christ

skraitow-kline:https://www.freebsd.org/releases/14.2R/installation/#upgrade

i see

then try it then since is written there

:)

What is the equivalent of /proc/net/nf_conntrack in FreeBSD which shows which connections are flowing through a gateway?

psionic: fw dependant commands

psionic: with pf, pfctl -ss does it

with ipf, ipnat -l shows it

fuck knows what ipfw had

can't find it

hopefully it's enough

ipfw two types of nat even

maybe the prob i'm having is that bhyve vm image files are on the 1 root partition. maybe i need to create 2 separate raid arrays and dedicate 1 to vm disk images

i'm thinking all of the vm disk image activity, being in the host system's root partition, is 'leaking' lots of disk cache work to the host OS

demido: I don't think that will change anything useful, and its an awful lot of work without more data

dch well any other ideas what's causing this prob and how to solve it?

the big question is why does memory usage climb over time, and what behaviour triggers this

sure

and why doesn't laundry or inactive get freed

are the VMs freebsd ones? running maybe zfs inside?

debian, ext4

are trims in the VM getting through to the host?

ya there has to be some kinda leaking going on

i'm using virt-blk and virt-net vm drivers fwiw

but I would start reading tuning(7) carefully to start with, and brush up on those memory links I pasted earlier, they are dense/excellent reading

ok this is interesting...

and maybe ask in the #bhyve channel for some suggestions too

i rebooted the 2nd server, started the vms back up, and now reinitializing their data payloads. as they spin up, host system Free mem disappeared predictably. then a bit of laundry appeared (10M) BUT now finally i see Inact being consumed, and swap is still unused

that's promising, tho would be preferable to not need a reboot to cause it

down to 57G Inact from a peak of 68

124G Active, which is more sensible. 56G resident for both vms, then a bit extra for host OS

54G Inact, 12M laundry

swap still unused

ok swap is now getting used, 95M

I skimmed tunefs(8) but don't see anything really useful for your situation.

tyvm

Hey, if I want to know how much memory (phys and swao( being used by each jail, is the system keeping track of that and perhaps putting it in a sysctl, or should I parse output of `ps`?

ghoti: man rctl, you need to enable it & reboot

it won't add up precisely, memory accounting never does. filesystem-related resources will likely not show up in the way that you want, but might be possible with per-jail zfs datasets

ghoti: klarasystems.com/articles/controlli…esource-limits-with-rctl-in-freebsd is a great intro

not going below 55G Inact. laundry at 4G. Free ram at 4G. swap has 3G used

gonna have to try debian host with kvm vms and see what that's like. this doesn't seem right

dch: that looks like exaact what I was hoping for. Much better than parsing `ps aojail,rss,vsz`. Thank you!

ghoti: yw

Hi all, having connectivity issues only on FreeBSD (doesn't matter - clean install, or truenas, whatever) so: ping 1.1.1.1 -- nothing; from gateway: ping <freebsd_ip> -- timeout. From freebsd: ping <any device in the network> -- ok; from some other device: ping <freebsd_ip> -- ok.

FreeBSD sees ICMP request but no reply to gateway.

I have literally no idea what's going on. Is that special freebsd network stack, or something else.

Anyone?

it looks like you go blackholed, or blockholed

s/go/got

but why?

Really do not understand. Dealing with this issue for two days already.

And the server is needed urgently.

check the net for duplicate addrs

oh damn, yes! it works.

I did check for duplicate addresses, haven't found any, was looking for other possible causes but no.

rosmi_th: please use deductive reasoning, solve, thoubleshoot by sniffing traffic, you will find the culprit

Removed every single IP from ARP table - done

mzar: yes, i did that all.

But I was looking for issues in wrong places.

look for them on the wire, use tcpdump

I did that as well.

Anyway, all works now. Thanks.

My health is a bit fucked now, working & falling down sometimes, lol. Didn't notice duplicate IP (at all) - removed all and all works.

so the issue solved

you don't have to thank us, express you grattitue toward FreeBSD commpunity by helping others and advocating

I'll donate to FreeBSD foundation then <3

Does this counts? :)

ha... maybe they will make some more stickers for Fosdem BSD booth, so probably yes. Thank you

<3

A bit off-topic. Why do you use FreeBSD?

I am used to, feel confident with it.

rosmi_th: there will be a lot of answers

Well, just few reasons.

And, like.. why not Linux? (i do not use it)

"well I use both"

I use it on my laptop, and kinda.. i do not feel confident with it.

rosmi_th: Linux let me down a few times, it was a long time ago, maybe he's more stable now

he's not

s/he/ti

it

:-)

One of biggest reasons - too bloated, complicated..

systemd, for example.. I don't say it's "bad" or something, I just do not need 99% of it's functions, I need init system to initialize ..

rosmi_th: no worries, as a community with have nothing agains Linux

I do have Gentoo currently, and it's most near to FreeBSD stability and bloat-less, but still not my thing.

Most of linux distributions feel like.. "beta version"

too many problems

I don't know how people use it, maybe I am just idiot, but I want system that works out of the box with no need to do anything other than using it.

Linux is used by the enterprise, FreeBSD not much; we have Gentoo committer in our local BSD User Group, he installed FreeBSD, probably out of curiosity

.3

.3

FreeBSD is wholesome, the best thing I've ever used, but the main problem - it doesn't have everything.

For desktop? sure, enough.. for server? well, it cannot run <too big list>

?

Let's say I want to run deepseek ai here

OK

Or what you didn't understood?

I meant "there's no enough ports for some uses"

What is the soundserver people on FreeBSD use?

ALSA?

OSS, pulseaudio if someone is weird

Isn’t there a need for pulseaudio or pipewire on freebsd?

?

afaik in Linux there is a need for those because otherwise you can’t manage applications individually

afaik, yes

I don't need sound on my laptop, so I have no idea.

Hm, okay. I got T420 which must fully work with FreeBSD.

If I can run winbox and ledger live with Linuxulator - i will go for it.

ahhh that’s sick!

I need sound for work and stuff

janvhs: I do not, not on this machine.

What's the best way to pull a variable out of the rc system? I don't know where it's set, but I want the current value of $jail_li st. From `service jail rcvar`, I only get _enable. There must be a better way than using grep,

ghoti: sysrc might be able to do it. For _enabled I use 'if service blah enabled; then foo; fi

bdrewery: looks like sysrc can pull out the variable as long as I name it explicly. So I would have to, for example, already know that `jail` requires `$jail_list`. I guess this will do for now, though it woule be nice if I could ask it for jail_* or rcvar could include 'jail_list'.

I've seen references to /etc/rc.conf.d/network as a place to put ifconfig assignments. I failed to get that to work. `mv network netif` succeeded. Is that the right file for that?

It seems to be an approach well suited to automated configuration (e.g. ansible, puppet, etc).

any ideas on my bhyve mem leak issue?

demido: I missed the original details, but I'm hoping there is a PR.

Yea, based on reading cgit.freebsd.org/src/tree/libexec/rc/rc.d/netif?h=releng/14.2 - yeah, that's the right place

dvl what in that file leads you to believe it's the source of my issue?

demido: I was referring to my own stuff.

ah

demido: I know the feeling.. so often people do not tag the recipient, and the message gets lost in the chaff.

maybe i'll try putting this dell r430 into HBA mode and using zfs

ya

demido: That sounds like a really good idea.

dvl: but you can add extra general rc.conf files and there are those old per-if files under /etc... if all of that fails, it's option to just patch the init too

ketas: Thanks. That sounds like an awful lot of work - i.e. I don't want to patch.

there should be plenty of custom config around there already

don't most people do like rc.conf for generic setup and rc.conf.local for locals settings

literally what it's for

or, if you wanted direct ifconfig lines...

it's also option to write init scripts :)

Yes, to /etc/rc.conf and /etc/rc.conf.local - however, I am automating the configuration via Ansible - it is easier to put all of snmpd config into /etc/rc.conf.d/snmpd

ketas: ^ and I'm also taking that approach for ifconfig etc (/etc/rc.conf.d/netif)

did that make ton of small files?

never used ir

it

oh rc.conf.d used $name

i also use rc.conf.d/*, best way

demido: definitely do the zfs thing, you can change the caching / metadata behaviour on a per-dataset level i.e. for each of the VMs

has anyone successfully gotten freebsd to serve home directories via nfs to a linux client running a gnome gui? When I try the gnome desktop hangs on login

Happy Sabbath Day All ... . from skraito and Lord Jesus Christ ... .

ave discordia o/

dch ya i haven't tried it yet so i don't know if it'll solve my problem but either way, UFS is effectively dead

total abandonware

hi all happy sabbath day from Equinix ... . thank you for hosting with Us ... .

kirk just recently committed ufs date fix that pushed times past 2100, at which point, he jokes, he will be long dead and you can't yell on him anymore

ufs1

:p

exact date is February 7, 2106

equinix-owner you're colo'd in an equinix facility?

i am the owner yohanes patra with skraitow Lord Jesus Christ demido

welcome to host with Us ... .

it's him again?

Our network is 1.6 tbit now

who kertas ?

too many bad reviews of using this raid card in hba mode for zfs so i either find a solution with freebsd and ufs, or switch host to debian

Howdy all. I'm rather new to freebsd. I have a fresh install and I'm trying to install the git port. First time I tried it gave a bunch of "stopped" errors, so I tried again and now it seems to be in an infinite loop on a few packages. For example I have "staging for py311-openssl-24.1.0.1" and then under it is "something something depends on

something else - not found". Has been doing this for hours. Same thing happened when I tried to install the emacs port. Any ideas what I'm doing wrong?

I tried both with the installer-provided ports tree and tried a fresh install and removed the ports tree and redownloaded with the pkg version of git, then removed git and etc etc.

What ketas said about the date commit: cgit.freebsd.org/src/commit/?id=111…44301da39d7b7459c784230e1405e8980f8

yeah :)

MountainMan1312: right, you have to build all of the dependencies, which can be a quite lengthy list

(once you get into their dependencies, and those one's dependencies, etc.)

im trying the leaky bhyve setup with 1 change, putting the vm storage on its own partition

kevans: I knew it would be a long build (rust/llvm) but I'm pretty sure I'm seeing exactly the same dozen or so items come across the screen in a loop

rust, py311-cryptography, py311-openssl, py311-sphinx, py311-requests, etc.

MountainMan1312: actually building, or things that just enumerate them as dependencies that are maybe found, maybe not found?

the latter will likely happen a lot, some of the ones you've listed are incredibly common dependencies of others

it says they're not found. I wouldn't think much of it if it weren't the same few over and over. Seems like it's making no progress whatsoever