ok this is weird. was in the 3rd of 3 bhyve vms on a freebsd host, in the debian guest OS running sudo mkfs.ext4 /dev/vdb1, and it froze along with my ssh connection to vm host. so i look at console and it says g_dev_taste: g_dev_taste(zvol/zroot/vm/vm1/disk1.img) failed to g_attach, error=6, same for vm3/disk0.img, and same for vm3/disk1.img. then

swap_pager indefinite wait butter bufobj 0, blkno 6360 size 4096. what's up?

192GB of swap

maybe there is a drive issue comes to mind, but I am sure others might be better informed

swap_pager messages are common with filled up partitions as I remember

forums.freebsd.org/threads/help-nee…_pager-indefinite-wait-buffer.67840 seems mirrored swap over so many drives (8) is bad

so we have a great redundant disk array with zfs raid10 and we get to have the system crash just because 1 drive fails and it has swap on it, because we can't mirror swap reliably. what do

so from that forum link, either my swap is too big (192GB) or i shouldn't have gmirror swap over so many drives (8) or who knows what lol. maybe i'll try 0g swap and see if that runs to just avoid the swap BS entirely

demido: How much RAM and disk/storage space do you have? Is SWAP even doing anything for you on that system?

192GB ram, like 6.5T usable after the raid10 zfs eats whatever

well i was running into an issue with swap being used 100% because i was trying to run bhyve vms on it

demido: Bhyve was forcing things into swap?

i got a stable config rn with 16GB swap, mirrored, and 2x64GB vms. but now i'm gonna try 0 swap

or forcing swap you could say, because shutting down vms then swap drained

hrm

so i added the wired_memory="yes" option to vm-bhyve so they don't swap

that seems to help

but it seems like the freebsd host needs like 48G just to itself to be happy, which feels like a lot for a zfs arc <1G and doing nothing but running 2-3 vms

demido: I have no seen this on my Bhyve systems. But, I've very likely running very different VM's than you.

How many VM's and what does disk I/O look like?

In fact, there are very few systems I've ever had that even tap into swap.

2 or 3 vms depending on the config im trying. either 2x64G each or 3x48G each

a fair bit of disk io

Especially since RAM has gotten so large in the recent years. It's seems swap would nearly be useless on a system today.

but i have primarycache = metadata on zroot/vm that's why arc is so small

oh and the disk_dev for the vms is zvol-sparse

ya and using wired_memory="yes" im not trying to overcommit anything

zvol-sparce should be fine. Of course, you could test with something else, I suppose. Not sure if that would make a difference. ARC being really high is normal if you don't limit it.

ZFS does like it's RAM.

Which I'm happy to provide if I'm not using it for anything else.

ya i usually set arc_max sysctl but didnt even bother with this system since it stays under 1G

see any prob with that?

just brought duplicate server up with no swap configured. excited to try it out and see what blows up next

I do not. And I'd also be weary of setting a limit anyway since I'm unsure how VM's how granted their RAM and cache. It could limit the VM's? I do use Bhyve daily, but because I haven't run into this issue, I've never dug too deep. Sorry.

I would assume that the swap usage would be slowing things down, though. Unless, you're using tuned NVME disks or something? Even then, I'd imagine RAM is quicker. SWAP hasn't helped me out once in the last 15-20 years.

sas

brb phone

Then you almost certainly do not need a swap setup.

Even exhausted RAM would be faster than swap at that point.

if you have 128G of ram in the HOST you shouldn't be allocating 64G on two VMs in my humble opinion. That is just asking for trouble. You should do more like 60G for memory per VM giving the host at least something left to work with

back

SponiX when did i say that? i have 192GB in this system

damn what a nightmare. so i started server up with a new install, this time no swap. and when trying to mkfs within the guest vm, everything (guest vms and host OS) lock up, console message is geom_eli: crypto request failed (enomem)

memory exhausted, and yet there's 88G of ram free and swap is disabled. and vms are all wired_memory

so the story with bhyve is, even when you have enough ram, you don't have enough ram, because there's never enough ram for the bhyve pig to run

So, if you say that you have free (unused) ram and system is complaining about being out of memory, instead of whining here, you should file a bug. Because it clearly sounds like bug.

Then there is a chance things will improve.

Hello! I am looking to set up root-on-zfs on an SSD, but I am looking to actually boot the system from a separate drive, specifically a USB flash drive. The reasoning for this is that I have an old server who's BIOS does not support NVMe, but I want my root to be an NVMe SSD in an NVMe-to-PCIe-slot adaptor. The server has an internal USB port, and I intend to plug a flash drive into that for it to

boot from.

How can I achieve this? I found a forum thread on it, but it basically just ends in telling the user that that isn't what they want; they didn't have the same use case need as me, however.

rather just complain ty tho

appledash that would need some custom setup because the default setup is built assuming you have system on the boot drive.

appledash your boot media needs to have /boot directory and its content, because thats where bootloader files and the kernel + modules are. once you get the bootloader running from it, and have loaded the kernel into the memory, you need to pass information about actual rootfs to kernel and that might be tricky part.

NVME boot support [as long as I'm aware] is implying UEFI boot.

These machines UEFI boot but it's very very old UEFI :p

no surprise there. have you checked for firmware updates btw?

Doing a quick search I can't find anything, and when searching the server model and NVMe boot people recommend the flash drive approach as well

Some people say to use "Clover Bootloader" though which may be a bootloader that can read NVMe drives on its own and just chain to the FreeBSD loader present on the NVMe drive?

That may be a better approach than trying to split the FreeBSD /boot

I do not see they provide NVME driver

hm, they claim that clover does have nvme driver...

Does your system setup allow to specify drivers to be loaded?

Hello everyone. I have a HP Proliant dl380p g8 with FreeBSD installed in one of the drives in the front bay. Thing is that I've got a 256GB card and I want to move the OS from the hard drive (1.2TB) to the SD card (256GB). It is ZFS. I want to clone the entire zroot pool (including all the filesystems, not only zroot/ROOT/default. But everything in the zroot pool). So then I can boot from the SD

card (I already created the boot partition). What is the best way to do this?

I read about zrepl, zfs clone and zfs send but zfs send did not clone *everything* I wanted. Also I'd prefer not to reinstall FreeBSD as I already made some configuration

svragv: zfs send will send the whole pool if requested

I see, with -R right?

svragv: yes

svragv: don't forget to install bootloader and set bootfs property on the bootable pool

mzar: ~ [prometheus] % zfs send -R zroot | zfs recv -F zroot2

Error: Unsupported flag with filesystem or bookmark.

cannot receive: failed to read from stream

hello, I'm trying to build a 2016Q2 ports tree in a poudriere 10.3 jail and I'm getting an error with pkg: tons of chown: /wrkdirs/usr/ports/ports-mgmt/pkg/work/pkg-1.7.2/docs/pkg-add.8: Operation not permitted

any idea?

Okay I made a recursive snapshot " zfs snapshot -r zroot@today3" and I did "zfs send -R zroot@hoy3 | zfs recv -F zroot2" let's see if this works

Unsupported flag means difference in feature flags, check zpool get all output

'zpool get all'

svragv: it will work if you'll do it in right way, please take a look here docs.freebsd.org/en/books/handbook/zfs

appledash UEFI has 2 types of programs, drivers and applications. driver will extend the firmware features (assuming it will work properly), so you can try to load nvme driver provided by clover (you have probably found tachytelic.net/2020/10/dell-powered…all-boot-pci-nvme/?linkId=120151914 or similar). and then you can see if firmware boot manager or freebsd bootloader can now see the nvme disks. I

do not know if clover itself does support chainloading.

Is there some way to set sysvshm=new; for a podman container? I read that it uses jails but I cant find anywhere to configure it per container

tsoome_: That's a good idea, I will investigate :)

getz: Hey! How are you doing ? not yet, it is being discussed how to standardise from specs perspective (github.com/opencontainers/wg-freebs…b/main/docs/proposals/PROPOSAL_A.md). Good news this is coming to a conclusion soon, and soon it will be implemented in the diff runtimes supported so far on FreeBSD like: ocijail (github.com/dfr/ocijail) by dfr@ and runj (github.com/samuelkarp/runj) by

Samuel Karp. Please do report issues, requests and findings here: github.com/oci-playground/freebsd-podman-testing/issues .

getz: and if you are interested in contribute and collaborate on getting this further I would an eye on: github.com/FreeBSDCloudTech ;)

s/in/to

Thanks a lot! I'll check it out!

I notcied that podman-compose works quite well, just needs the argument --podman-run-args="--os=linux" and it uses the linuxulator for running most docker containers

I've never tried them before but thought it was worth a try now that we podman available :)

getz: 👊😎

mnour_bsd: do you know if theres a reason why podman-compose is not in ports? Otherwise I'll try packaging it, I've been wanting to get into that more

getz: No I don't, so go ahead! 👊 😎

sweet!

getz: I see docs.podman.io/en/v5.1.1/markdown/podman-compose.1.html

getz: sorry I am not on my FreeBSD laptop at the moment, what happens when you run podman compose command as described in that link 👆 ?

getz: do you get an error ?

haha, okay that seems to be it :)

I felt like it was something obvious missing

thanks again mnour_bsd 👊

getz: wait, the podman compose needs a compose driver, so it does need docker-compose or podman-compose

oh yeah and now I got it installed so it works

getz: "podman compose is a thin wrapper around an external compose provider such as docker-compose or podman-compose."

getz: cool!

I might have tried running it before and it complained that it couldn't find docker-compose etc

great! then I do still get to package podman-compose :D

getz: cool! looking forward to give it a try 👊

hi, everyone! any OracleDB lover here, using it on FreeBSD? which versions please? by cage or?

does it work on FreeBSD ?

mzar: there was an old version before 2010, possible to work within a freebsd cage

yes, but IIRC it was only working somewhere between 1995 and 2005

Last time I used it I had to run it in a linux vm. I won't say my love for it is particularly strong, for the record.

yea, freebsd is not the place for OracleDB but as a test/play. I checked the oracle.com/database/technologies/or…le-database-software-downloads.html, there is no 21c for Solaris/AIX even :-( hopefully they are only late, because Oracle runs best on Solaris/AIX imho... I mean not di vxwarlock: sorry, any particular reason to run OracleDB in a VM instead of the main Linux host with better

speed?

sorry for extra words, I missed to correct/remove them

oracle.com/artificial-intelligence | clapont | oracle.com/tr/a/ocom/docs/solaris114datasheet-5024156.pdf > I don't know, I can't see any reason to choose this mountain.

has anyoneelse noticed SYN flood attacks on IPv4? I keep scanning for SYNfloods and filling the firewall with entries (/24's) but it keeps coming. Some of them are in Brazil, and whois is going slow. slack is practically shuit down. IPv4 pings are 1/2 sec. IPv6 is doing ok though

basically doing netstat -an and looking for SYN_RCVD then awk/sed the source IP into a /24, paste as firewall rule.

this botnet has to be HUGE!

hello shbrngdo, long time no see!

hi - you seeing the SYN flood?

we have net-mgmt/fastnetmon in ports, it works nice with BGP router

looking, thanks

nope, I fight with DoSes using FastNetMon

mnour_bsd: I submitted it to ports now if you'd like to try it freebsd/freebsd-ports #343

I think by the time I learn how to set up and use FastNetMon it will be done. strangely the SYN flood has been affecting DNS lookups... Someone on the radio complained about internet response.

that's why I am wondering if anyone else is seeing this

well looks like DNS is flaky - debian.org resolves, freebsd.org resolves, twiiter.com and x.com do NOT resolve, slack.com is having trouble resolving, and so on

google also worked. maybe is just me but I think this is bigger. does not affect IPv6 though...

I think the entire 177/8 network is involved (lacnic) - strange. Nobody else is seeng this?

what type of attack is that ?

indeed, AS52688 triggers some traffic, maybe they are feeding/teaching their new AI model ?

Learned today my company has the entire .pw TLD RPZ'd.

getz: Awesome! 👊 😎

mnour_bsd: do you know if there's an irc channel for freebsd oci/containers?

getz: yup! #freebsd-jails

sweet, thanks

getz: 👊

ok i got a solution to this neverending mem swap bhyve issue: disabled swap, didn't enable GELI encryption. so now i have 192G ram box with 3 48G ram wired_memory=yes vms, leaving 48G ram for host, zfs on host primarycache=metadata on zroot/vm with 1.5G arc and 35G free ram. finally running stable. no console messages, no swap, all good. the 1

thing i didn't try that might have worked but i don't care is the same config but with swap that isn't mirrored, but then if the swap disk fails the system crashes so i sidestepped that with no swap

ek ^

reads more like a workaround to me and like geli, bhyve and maybe zfs have some issues working correctly together

mzar - SYN flood. I have blocked probably 100m /24 network blocks, still seeing some, but DNS is affected (somewhat interm,ittently)

ya agreed

random checks [whenwhois does not gripe about no record found) show IP addresses in south america like the 1777/8 net block. Just randym 'syn' [packets from zillions of IP addresses.

ipfw can block the ports but my bandwidth is apparenytly still affected. let's hope ATT can fix it

How's your firewall blocking them? Silent drop?

ipfw deny ip

seems klike it stopped, but I also just added another alias address to my public facing IP (I have a block of 4).

got bad, could not use DNS, so added new IP, dns started working. and also no more SYN flood...

Did you make a new friend recently?

lol

yeah really. actually not doing anything different. I have been seeing lame attempts to circumvent the mail server to do relaying for months, though. I kept blocking the IPs. This thing is new.

well ic3.gov will get a list of IP netblocks that were doing the SYN flood, if their web site evr responds

What's more likely to be in use on a given FreeBSD installation, doas or sudo?

Or something else? su?

i think it depends on the administrator

that's more of a religious question than anything

the ports tree has both doas and sudo available for install

Sure, I realize it's a personal preference.

I don't use FreeBSD, I use OpenBSD, so I just use doas these days.

But I wonder what others who are actively using FreeBSD use, or prefer.

im guessing, in general, sudo is gonna have the highest 'market share' of the 3

but you can certainly install doas if you'd like: freshports.org/security/doas

Yeah, thanks.

Anyone here use newsyslog to pass the "R" flag to allow something like this in palce of the pid "/bin/pkill -USR1 -u root -U root -x obhttpd"

What does the `R' flag do? I don't see it mentioned in the man page.

I see a `Z', `B', `M', and `F', flag, but no mention of `R'.

R if this flag is set the newsyslog(8) will run shell

command defined in path_to_pid_cmd_fil

Which man page?

newsyslog

Doh, ignore me...

I thought I was in #openbsd. :-)

ha ,well in openbsd you dont need to do this "run this" works

in frreebsd they say you need ZR "/bin/kill blah blah" but it does not seam to work

newsyslog: illegal pid file or signal in config file: