02:48:58 ok this is weird. was in the 3rd of 3 bhyve vms on a freebsd host, in the debian guest OS running sudo mkfs.ext4 /dev/vdb1, and it froze along with my ssh connection to vm host. so i look at console and it says g_dev_taste: g_dev_taste(zvol/zroot/vm/vm1/disk1.img) failed to g_attach, error=6, same for vm3/disk0.img, and same for vm3/disk1.img. then 02:48:59 swap_pager indefinite wait butter bufobj 0, blkno 6360 size 4096. what's up? 02:49:13 192GB of swap 03:54:26 maybe there is a drive issue comes to mind, but I am sure others might be better informed 03:55:15 swap_pager messages are common with filled up partitions as I remember 03:55:25 https://forums.freebsd.org/threads/help-needed-kernel-swap_pager-indefinite-wait-buffer.67840/ seems mirrored swap over so many drives (8) is bad 04:01:56 so we have a great redundant disk array with zfs raid10 and we get to have the system crash just because 1 drive fails and it has swap on it, because we can't mirror swap reliably. what do 04:04:24 so from that forum link, either my swap is too big (192GB) or i shouldn't have gmirror swap over so many drives (8) or who knows what lol. maybe i'll try 0g swap and see if that runs to just avoid the swap BS entirely 04:17:30 demido: How much RAM and disk/storage space do you have? Is SWAP even doing anything for you on that system? 04:21:59 192GB ram, like 6.5T usable after the raid10 zfs eats whatever 04:22:21 well i was running into an issue with swap being used 100% because i was trying to run bhyve vms on it 04:23:02 demido: Bhyve was forcing things into swap? 04:23:03 i got a stable config rn with 16GB swap, mirrored, and 2x64GB vms. but now i'm gonna try 0 swap 04:23:25 or forcing swap you could say, because shutting down vms then swap drained 04:23:36 hrm 04:23:38 so i added the wired_memory="yes" option to vm-bhyve so they don't swap 04:23:48 that seems to help 04:24:21 but it seems like the freebsd host needs like 48G just to itself to be happy, which feels like a lot for a zfs arc <1G and doing nothing but running 2-3 vms 04:25:19 demido: I have no seen this on my Bhyve systems. But, I've very likely running very different VM's than you. 04:25:28 How many VM's and what does disk I/O look like? 04:25:52 In fact, there are very few systems I've ever had that even tap into swap. 04:26:23 2 or 3 vms depending on the config im trying. either 2x64G each or 3x48G each 04:26:31 a fair bit of disk io 04:26:34 Especially since RAM has gotten so large in the recent years. It's seems swap would nearly be useless on a system today. 04:26:43 but i have primarycache = metadata on zroot/vm that's why arc is so small 04:26:52 oh and the disk_dev for the vms is zvol-sparse 04:27:08 ya and using wired_memory="yes" im not trying to overcommit anything 04:27:57 zvol-sparce should be fine. Of course, you could test with something else, I suppose. Not sure if that would make a difference. ARC being really high is normal if you don't limit it. 04:28:18 ZFS does like it's RAM. 04:28:32 Which I'm happy to provide if I'm not using it for anything else. 04:33:39 ya i usually set arc_max sysctl but didnt even bother with this system since it stays under 1G 04:33:45 see any prob with that? 04:34:33 just brought duplicate server up with no swap configured. excited to try it out and see what blows up next 04:35:35 I do not. And I'd also be weary of setting a limit anyway since I'm unsure how VM's how granted their RAM and cache. It could limit the VM's? I do use Bhyve daily, but because I haven't run into this issue, I've never dug too deep. Sorry. 04:36:59 I would assume that the swap usage would be slowing things down, though. Unless, you're using tuned NVME disks or something? Even then, I'd imagine RAM is quicker. SWAP hasn't helped me out once in the last 15-20 years. 04:38:17 sas 04:38:19 brb phone 04:40:23 Then you almost certainly do not need a swap setup. 04:40:42 Even exhausted RAM would be faster than swap at that point. 05:45:46 if you have 128G of ram in the HOST you shouldn't be allocating 64G on two VMs in my humble opinion. That is just asking for trouble. You should do more like 60G for memory per VM giving the host at least something left to work with 06:10:18 back 06:10:41 SponiX when did i say that? i have 192GB in this system 07:29:46 damn what a nightmare. so i started server up with a new install, this time no swap. and when trying to mkfs within the guest vm, everything (guest vms and host OS) lock up, console message is geom_eli: crypto request failed (enomem) 07:32:41 memory exhausted, and yet there's 88G of ram free and swap is disabled. and vms are all wired_memory 07:33:57 so the story with bhyve is, even when you have enough ram, you don't have enough ram, because there's never enough ram for the bhyve pig to run 08:15:31 So, if you say that you have free (unused) ram and system is complaining about being out of memory, instead of whining here, you should file a bug. Because it clearly sounds like bug. 08:16:16 Then there is a chance things will improve. 08:19:00 Hello! I am looking to set up root-on-zfs on an SSD, but I am looking to actually boot the system from a separate drive, specifically a USB flash drive. The reasoning for this is that I have an old server who's BIOS does not support NVMe, but I want my root to be an NVMe SSD in an NVMe-to-PCIe-slot adaptor. The server has an internal USB port, and I intend to plug a flash drive into that for it to 08:19:02 boot from. 08:19:24 How can I achieve this? I found a forum thread on it, but it basically just ends in telling the user that that isn't what they want; they didn't have the same use case need as me, however. 08:19:41 rather just complain ty tho 08:25:39 appledash that would need some custom setup because the default setup is built assuming you have system on the boot drive. 08:28:47 appledash your boot media needs to have /boot directory and its content, because thats where bootloader files and the kernel + modules are. once you get the bootloader running from it, and have loaded the kernel into the memory, you need to pass information about actual rootfs to kernel and that might be tricky part. 08:35:09 NVME boot support [as long as I'm aware] is implying UEFI boot. 08:39:10 These machines UEFI boot but it's very very old UEFI :p 08:40:28 no surprise there. have you checked for firmware updates btw? 08:54:07 Doing a quick search I can't find anything, and when searching the server model and NVMe boot people recommend the flash drive approach as well 08:55:15 Some people say to use "Clover Bootloader" though which may be a bootloader that can read NVMe drives on its own and just chain to the FreeBSD loader present on the NVMe drive? 08:55:15 That may be a better approach than trying to split the FreeBSD /boot 09:00:03 I do not see they provide NVME driver 09:00:13 https://sourceforge.net/p/cloverefiboot/code/HEAD/tree/Drivers/ 09:04:04 hm, they claim that clover does have nvme driver... 09:10:59 Does your system setup allow to specify drivers to be loaded? 09:54:43 Hello everyone. I have a HP Proliant dl380p g8 with FreeBSD installed in one of the drives in the front bay. Thing is that I've got a 256GB card and I want to move the OS from the hard drive (1.2TB) to the SD card (256GB). It is ZFS. I want to clone the entire zroot pool (including all the filesystems, not only zroot/ROOT/default. But everything in the zroot pool). So then I can boot from the SD 09:54:45 card (I already created the boot partition). What is the best way to do this? 09:55:14 I read about zrepl, zfs clone and zfs send but zfs send did not clone *everything* I wanted. Also I'd prefer not to reinstall FreeBSD as I already made some configuration 09:59:40 svragv: zfs send will send the whole pool if requested 10:00:46 I see, with -R right? 10:03:18 svragv: yes 10:04:49 svragv: don't forget to install bootloader and set bootfs property on the bootable pool 10:16:53 mzar: ~ [prometheus] % zfs send -R zroot | zfs recv -F zroot2 10:16:55 Error: Unsupported flag with filesystem or bookmark. 10:16:57 cannot receive: failed to read from stream 10:17:18 hello, I'm trying to build a 2016Q2 ports tree in a poudriere 10.3 jail and I'm getting an error with pkg: tons of chown: /wrkdirs/usr/ports/ports-mgmt/pkg/work/pkg-1.7.2/docs/pkg-add.8: Operation not permitted 10:17:22 any idea? 10:18:23 Okay I made a recursive snapshot " zfs snapshot -r zroot@today3" and I did "zfs send -R zroot@hoy3 | zfs recv -F zroot2" let's see if this works 10:27:52 Unsupported flag means difference in feature flags, check zpool get all output 10:28:06 'zpool get all' 10:28:51 svragv: it will work if you'll do it in right way, please take a look here https://docs.freebsd.org/en/books/handbook/zfs/ 10:31:43 appledash UEFI has 2 types of programs, drivers and applications. driver will extend the firmware features (assuming it will work properly), so you can try to load nvme driver provided by clover (you have probably found https://tachytelic.net/2020/10/dell-poweredge-install-boot-pci-nvme/?linkId=120151914 or similar). and then you can see if firmware boot manager or freebsd bootloader can now see the nvme disks. I 10:31:43 do not know if clover itself does support chainloading. 11:03:24 Is there some way to set sysvshm=new; for a podman container? I read that it uses jails but I cant find anywhere to configure it per container 11:08:30 tsoome_: That's a good idea, I will investigate :) 11:13:39 getz: Hey! How are you doing ? not yet, it is being discussed how to standardise from specs perspective (https://github.com/opencontainers/wg-freebsd-runtime/blob/main/docs/proposals/PROPOSAL_A.md). Good news this is coming to a conclusion soon, and soon it will be implemented in the diff runtimes supported so far on FreeBSD like: ocijail (https://github.com/dfr/ocijail) by dfr@ and runj (https://github.com/samuelkarp/runj) by 11:13:39 Samuel Karp. Please do report issues, requests and findings here: https://github.com/oci-playground/freebsd-podman-testing/issues . 11:15:37 getz: and if you are interested in contribute and collaborate on getting this further I would an eye on: https://github.com/FreeBSDCloudTech ;) 11:15:51 s/in/to 11:19:41 Thanks a lot! I'll check it out! 11:20:15 I notcied that podman-compose works quite well, just needs the argument --podman-run-args="--os=linux" and it uses the linuxulator for running most docker containers 11:20:36 I've never tried them before but thought it was worth a try now that we podman available :) 11:22:15 getz: 👊😎 11:25:58 mnour_bsd: do you know if theres a reason why podman-compose is not in ports? Otherwise I'll try packaging it, I've been wanting to get into that more 11:26:49 getz: No I don't, so go ahead! 👊 😎 11:27:00 sweet! 11:29:52 getz: I see https://docs.podman.io/en/v5.1.1/markdown/podman-compose.1.html 11:30:46 getz: sorry I am not on my FreeBSD laptop at the moment, what happens when you run podman compose command as described in that link 👆 ? 11:31:01 getz: do you get an error ? 11:36:46 haha, okay that seems to be it :) 11:36:58 I felt like it was something obvious missing 11:38:02 thanks again mnour_bsd 👊 11:38:19 getz: wait, the podman compose needs a compose driver, so it does need docker-compose or podman-compose 11:38:49 oh yeah and now I got it installed so it works 11:38:59 getz: "podman compose is a thin wrapper around an external compose provider such as docker-compose or podman-compose." 11:39:03 getz: cool! 11:39:08 I might have tried running it before and it complained that it couldn't find docker-compose etc 11:39:24 great! then I do still get to package podman-compose :D 11:39:52 getz: cool! looking forward to give it a try 👊 14:50:11 hi, everyone! any OracleDB lover here, using it on FreeBSD? which versions please? by cage or? 14:54:33 does it work on FreeBSD ? 15:21:36 mzar: there was an old version before 2010, possible to work within a freebsd cage 15:23:00 yes, but IIRC it was only working somewhere between 1995 and 2005 15:25:10 Last time I used it I had to run it in a linux vm. I won't say my love for it is particularly strong, for the record. 15:51:46 yea, freebsd is not the place for OracleDB but as a test/play. I checked the https://www.oracle.com/database/technologies/oracle-database-software-downloads.html, there is no 21c for Solaris/AIX even :-( hopefully they are only late, because Oracle runs best on Solaris/AIX imho... I mean not di vxwarlock: sorry, any particular reason to run OracleDB in a VM instead of the main Linux host with better 15:51:52 speed? 15:52:45 sorry for extra words, I missed to correct/remove them 16:11:12 https://www.oracle.com/artificial-intelligence/ | clapont | https://www.oracle.com/tr/a/ocom/docs/solaris114datasheet-5024156.pdf > I don't know, I can't see any reason to choose this mountain. 17:46:42 has anyoneelse noticed SYN flood attacks on IPv4? I keep scanning for SYNfloods and filling the firewall with entries (/24's) but it keeps coming. Some of them are in Brazil, and whois is going slow. slack is practically shuit down. IPv4 pings are 1/2 sec. IPv6 is doing ok though 17:48:49 basically doing netstat -an and looking for SYN_RCVD then awk/sed the source IP into a /24, paste as firewall rule. 17:49:29 this botnet has to be HUGE! 17:49:29 hello shbrngdo, long time no see! 17:49:47 hi - you seeing the SYN flood? 17:49:49 we have net-mgmt/fastnetmon in ports, it works nice with BGP router 17:50:06 looking, thanks 17:50:11 nope, I fight with DoSes using FastNetMon 17:57:45 mnour_bsd: I submitted it to ports now if you'd like to try it https://github.com/freebsd/freebsd-ports/pull/343 18:19:55 I think by the time I learn how to set up and use FastNetMon it will be done. strangely the SYN flood has been affecting DNS lookups... Someone on the radio complained about internet response. 18:20:47 that's why I am wondering if anyone else is seeing this 18:38:32 well looks like DNS is flaky - debian.org resolves, freebsd.org resolves, twiiter.com and x.com do NOT resolve, slack.com is having trouble resolving, and so on 18:39:23 google also worked. maybe is just me but I think this is bigger. does not affect IPv6 though... 18:48:37 I think the entire 177/8 network is involved (lacnic) - strange. Nobody else is seeng this? 18:50:58 what type of attack is that ? 18:55:07 indeed, AS52688 triggers some traffic, maybe they are feeding/teaching their new AI model ? 19:04:53 Learned today my company has the entire .pw TLD RPZ'd. 19:12:49 getz: Awesome! 👊 😎 19:15:08 mnour_bsd: do you know if there's an irc channel for freebsd oci/containers? 19:15:25 getz: yup! #freebsd-jails 19:15:31 sweet, thanks 19:15:48 getz: 👊 19:37:06 ok i got a solution to this neverending mem swap bhyve issue: disabled swap, didn't enable GELI encryption. so now i have 192G ram box with 3 48G ram wired_memory=yes vms, leaving 48G ram for host, zfs on host primarycache=metadata on zroot/vm with 1.5G arc and 35G free ram. finally running stable. no console messages, no swap, all good. the 1 19:37:06 thing i didn't try that might have worked but i don't care is the same config but with swap that isn't mirrored, but then if the swap disk fails the system crashes so i sidestepped that with no swap 19:39:02 ek ^ 19:44:10 reads more like a workaround to me and like geli, bhyve and maybe zfs have some issues working correctly together 19:47:21 mzar - SYN flood. I have blocked probably 100m /24 network blocks, still seeing some, but DNS is affected (somewhat interm,ittently) 19:48:21 ya agreed 19:49:04 random checks [whenwhois does not gripe about no record found) show IP addresses in south america like the 1777/8 net block. Just randym 'syn' [packets from zillions of IP addresses. 19:50:01 ipfw can block the ports but my bandwidth is apparenytly still affected. let's hope ATT can fix it 20:00:28 How's your firewall blocking them? Silent drop? 20:15:11 ipfw deny ip 20:15:48 seems klike it stopped, but I also just added another alias address to my public facing IP (I have a block of 4). 20:17:08 got bad, could not use DNS, so added new IP, dns started working. and also no more SYN flood... 20:18:31 Did you make a new friend recently? 20:18:59 lol 20:35:52 yeah really. actually not doing anything different. I have been seeing lame attempts to circumvent the mail server to do relaying for months, though. I kept blocking the IPs. This thing is new. 20:37:07 well ic3.gov will get a list of IP netblocks that were doing the SYN flood, if their web site evr responds 20:50:33 What's more likely to be in use on a given FreeBSD installation, doas or sudo? 20:50:45 Or something else? su? 20:52:28 i think it depends on the administrator 20:52:40 that's more of a religious question than anything 20:52:51 the ports tree has both doas and sudo available for install 20:53:11 Sure, I realize it's a personal preference. 20:53:21 I don't use FreeBSD, I use OpenBSD, so I just use doas these days. 20:53:31 But I wonder what others who are actively using FreeBSD use, or prefer. 20:53:50 im guessing, in general, sudo is gonna have the highest 'market share' of the 3 20:54:08 but you can certainly install doas if you'd like: https://www.freshports.org/security/doas 20:54:31 Yeah, thanks. 21:06:55 Anyone here use newsyslog to pass the "R" flag to allow something like this in palce of the pid "/bin/pkill -USR1 -u root -U root -x obhttpd" 21:08:25 What does the `R' flag do? I don't see it mentioned in the man page. 21:08:53 I see a `Z', `B', `M', and `F', flag, but no mention of `R'. 21:09:11 R if this flag is set the newsyslog(8) will run shell 21:09:13 command defined in path_to_pid_cmd_fil 21:09:24 Which man page? 21:09:32 newsyslog 21:09:37 Doh, ignore me... 21:09:41 I thought I was in #openbsd. :-) 21:09:57 ha ,well in openbsd you dont need to do this "run this" works 21:10:24 in frreebsd they say you need ZR "/bin/kill blah blah" but it does not seam to work 21:10:44 newsyslog: illegal pid file or signal in config file: