00:01:31 i don't even dare log into my bank unless they do the same, i just use the phone app 00:01:36 hooray for the open internet, i guess 00:02:48 I have had no trouble logging into ebay.com from FreeBSD. Not a problem so far. 00:03:46 I normally use 'bot' as UA, some sites give better versions they want to give those search engine crawlers 00:04:48 Remembered a reference Dan Langille posted that included a list of things to turn off from periodic in jails, and probably useful elsewhere too. https://gist.github.com/dlangille/ce60ac76b69f267a3f1de33495a338fc 00:04:49 Title: Periodic things to turn off in FreeBSD jails · GitHub 00:05:20 rwp: I'm still able to log into ebay even when using librewolf with the enhance privacy protections. Though some websites like amazon still think I'm a bot and require me to verify I'm not. 00:05:39 rwp: Perhaps we should summarize that and submit a PR, grouping those items into ...something. 00:05:47 rwp: and by we, I hope you. 00:05:48 My user-agent would look weird as f**k to them 00:06:00 rwp: i also had no problem logging in, until i tried to buy something and they suspended me, and then told me that "any related accounts" will also be suspended, so now i'm scared to log into my work account... 00:06:18 * rwp chuckles (with dvl) :-) 00:07:25 I had that dlangille reference because it is still in my queue to read, understand, grok completely, and then go fix up all of my systems. I still need to do that. 00:09:10 I have bought a lot of things off eBay using my FreeBSD Firefox. Just verified now that I do not override User-Agent there. (User-Agent Switcher plugin FTW!) 00:09:41 it probably didn't help that i made a new account 00:10:26 when poudriere attacks 12:10AM up 4 days, 8:24, 2 users, load averages: 13.01, 12.17, 8.73 00:11:24 Hmm, doesn't the browser ports automatically build them using "Linux" as the OS default user-agent? 00:12:20 kenrap: this one (that i pasted above) has both linux and freebsd in, which is weird. i don't know if it's qtwebengine doing that... 00:13:32 Current pkg installed Firefox gives me "Mozilla/5.0 (X11; FreeBSD amd64; rv:121.0) Gecko/20100101 Firefox/121.0" which I just pulled from the log just now. 00:14:29 my librewolf user-agent shows: Mozilla/5.0 (X11; FreeBSD amd64; rv:109.0) Gecko/20100101 Firefox/117.0 00:14:30 yeah, ff's is much more esensible 00:14:44 this one is from qutebrowser 00:18:06 Well, I was definitely wrong on both accounts thinking that the obscure user-agent was about fingerprint resisting and that "Linux" is an automatic default for all browsers built in ports in general. 00:21:41 I don't know why any site would use User-Agent as anything more than a mild hint but some do. It's insane. 00:22:39 It is insane: https://github.com/fingerprintjs/fingerprintjs 00:22:40 Title: GitHub - fingerprintjs/fingerprintjs: Browser fingerprinting library. Accuracy of this version is 40-60%, accuracy of the commercial Fingerprint Identification is 99.5%. V4 of this library is BSL licensed. 00:51:17 Is it possible for linux and freebsd to use the same swap partition? 00:52:25 technically yes, but possibly they won't like that they use different partition types 00:57:47 me: "well, this vm provider only offers linux, i'll use zfs on there and i'm sure it'll be fine" 00:57:52 my vm: 00:57:54 2!fuchsia /home/lexi# zfs destroy -r backup/data 00:57:54 Connection to fuchsia.eden.le-fay.org closed by remote host. 00:58:30 here we go building lang/rust again... 00:59:10 jbo: i feel your pain [02] 01:08:43 lang/rust | rust-1.75.0 build 01:07:54 72.2% 8% 01:00:01 i need to get around to submitting enough PRs that i don't need to build ports from source anymore 01:00:06 not sure whether you truly do. because before I left the office I turned off my 32-cores 256 GB RAM build server so now I'm stuck building on a stupid desktop machine 01:00:30 what do you mean, lw? you can always just use an overlay. should be unrelated to PRs entirely. 01:00:37 jbo: i have devel/electron25 in my tree, so i do 01:00:56 not sure how submitting more PRs helps with that? 01:01:05 no, these are different problems 01:01:33 1) electron takes ages to build, so i feel your pain, 2) i need to build ports because the default options are stupid, but we can fix that with subpackages, like enabling jack in pulseaudio 01:01:59 but i have to send patches to enable subpackages where needed 01:02:25 aye 01:02:48 hmm, for some reason my repo also has devel/electron25 in it. I better go hunting 01:03:00 > electron25-25.9.8_2 Build cross-platform desktop apps with JavaScript, HTML, and CSS 01:03:03 it's signal-desktop for me 01:03:07 oh, that's right! 01:03:08 fricking hate signal 01:03:27 the guy who created it claims to be an anarchist but threatens legal action against anyone who makes a third-party implement of their protocol 01:03:31 how the fuck does that work 01:03:39 the sentence of "Build cross-platform desktop apps with JavaScript, HTML, and CSS" is just enought of a hint that something like that shouldn't exist IMHO :D 01:04:03 "here are technologies that are purposefully designed not be used for desktop applications put together to make desktop applications" 01:04:27 oh this is fun (on my linux vm) fuchsia.eden.le-fay.org login: [ 249.856720] Kernel panic - not syncing: System is deadlocked on memory 01:05:26 i guess 2GB RAM is not enough to destroy a zfs dataset? 01:07:01 lw, I still haven't figured out why syncoid does not succeed on anything other than the first sync for me 01:07:42 jbo: i'm fighting with syncoid right now (it's what caused all this) but at least it's not syncoid's fault... 01:08:16 lw, one "problem" is that the progress bars of each snapshot rarely reach 100% 01:08:18 then I get 01:08:19 mbuffer: error: outputThread: error writing to at offset 0x0: Broken pipe 01:08:29 and cannot mount '/storage/sanoid_test/beefy02/poudriere/jails/140Ramd64-main-ref': failed to create mountpoint: Permission denied 01:09:02 but one some datasets it works 01:09:05 .__. 01:12:58 jbo: broken pipe sounds like the ssh connection died, anything on the remote? 01:13:26 no idea about permission denied though, i guess you're sending as non-root and using zfs delegated administration? i've never tried that 01:13:27 lw, it's a very stable NAS type machine doing exactly nothing. never had _any_ issues with it. also on the same network just one switch away 01:13:36 yes @ second 01:14:12 that's a poudriere dataset, i guess it has a mountpoint set in /poudriere or something and you don't have permission to create it... 01:14:19 (you = the non-root user) 01:14:30 yeah I'm using a poudriere dataset to test sanoid/syncoid :D 01:14:58 hrm, is it worth a dedicated build host (for poudriere, and the world) ? 01:15:13 lw, well on the destination host: 01:15:15 zfs allow storage/sanoid_test/beefy02 01:15:20 user jbo create,mount,receive 01:15:25 rtprio: like a dedicated computer? imo, no, unless you're doing constant back-to-back builds 01:15:35 rtprio: a dedicated VM though, sure 01:15:44 rtprio, I have three separate dedicated build hosts .__. 01:16:10 jbo: i have never used delegated admin but i guess you need to find a way to make it not send the mount attribute 01:16:30 lw, agreed 01:16:34 jbo: ... or create /poudriere and make it owned by you but that doesn't sound like a good solution (especially if you back up >1 host) 01:16:58 jbo: you have 3 computers dedicated to building ports? how many do you maintain?? 01:17:43 lw, I really want this sanoid/syncoid situation to work for non-root users. 01:18:00 lw, I have a "production poudriere server", a "development poudriere server" and a "lol I hate my life poudriere server" 01:18:01 jbo: i might play with this when i have some free time in which case i'll let you know what i learn 01:18:06 to be fair, I usually just run two. 01:18:08 other than that i probably can't be much help though 01:18:26 jbo: did you try setting canmount=noauto on the entire hierarchy? 01:18:32 it would be a vm 01:18:44 I guess the packages have usually been enough for me 01:19:08 rtprio: a vm to build packages sounds completely reasonable to me. i don't do that myself but like, why not 01:19:09 rtprio, running poudriere in a VM is fine. however, you can also ditch the VM and just run poudriere in a jail. works well too. the reason why I run one poudriere in a VM is because kernel forward compatibility "restrictions" 01:19:49 rtprio, if you have no reason not to use official packages then keep doing that. poudriere is fun and works great but rabbit hole mechanics apply. 01:19:52 rtprio: although note that because poudriere creates its own jails, you don't need like a "14 build host" and "15 build host", just create jails of the appropriate version and put COMPAT_FREEBSDxx in your kernel 01:20:52 lw, the canmount=noauto would be for the dataset on the dest host, right? 01:21:03 jbo: yeah, set it on the dest host (and hope it doesn't get overwritten) 01:21:54 lw, what would override it potentially? 01:22:22 jbo: well, if syncoid notices the property is "wrong" compared to the source system and resets it. i don't know off hand if it does this 01:22:30 (i don't back up my poudriere filesystems) 01:23:28 well that would apply to a non-poudriere fs too 01:23:41 I'm just using the poudriere ds because that is a decent test and I don't care if I trash it :p 01:23:45 i do backup my root pools and never noticed / getting mounted over the backup server's root 01:24:24 so what does the mountpoint property show on your destination host dataset? 01:24:27 # zfs get mountpoint zroot/ROOT/default 01:24:27 NAME PROPERTY VALUE SOURCE 01:24:27 zroot/ROOT/default mountpoint / received 01:24:31 5!fuchsia /home/lexi# zfs get mountpoint backup/hemlock/zroot/ROOT/default 01:24:31 NAME PROPERTY VALUE SOURCE 01:24:32 backup/hemlock/zroot/ROOT/default mountpoint /backup/hemlock/zroot/ROOT/default default 01:24:43 first = source, second = backup host 01:25:00 so it seems like it's not sending the property 01:25:33 hmm... so the mountpoints on my destination host datasets are all set to something other than what the source host has - which is good. 01:25:56 maybe more relevant: 01:26:02 I remember the first time I was dealing with ZFS send|recv I ended up with a broken dest host because I was recursively snapping/sending zroot which caused the remote host to boot from the wrong dataset... 01:26:02 [11!] root@hemlock /etc/periodic/daily 01:26:02 # zfs get canmount zroot/ROOT/default 01:26:02 NAME PROPERTY VALUE SOURCE 01:26:02 zroot/ROOT/default canmount noauto received 01:26:07 6!fuchsia /home/lexi# zfs get canmount backup/hemlock/zroot/ROOT/default 01:26:07 NAME PROPERTY VALUE SOURCE 01:26:07 backup/hemlock/zroot/ROOT/default canmount on default 01:26:45 but then, if it doesn't send canmount *or* mountpoint, i don't understand why you had this problem to begin with 01:26:54 because the mountpoint would just be default on the backup host, right? 01:28:14 just checked, it's indeed default on the dest host 01:28:19 hm 01:28:35 any ideas? :/ 01:29:05 jbo: is it possible that 'zfs send' of /storage/sanoid_test/beefy02/poudriere/jails created it with root as owner, and now you can't create the child filesystem because... it's owned by root? 01:29:37 lw, well I mean when I zfs list on the dest host I see datasets created/send by syncoid 01:30:21 jbo: i'd suggest asking the sanoid people, i know jim salter (who wrote it) works for klara sometimes so they're probably open to support freebsd 01:30:52 I'd assume that this is just something stupid that I would have liked to be figured out by now :p 01:31:51 lw, here's my syncoid line: syncoid --no-privilege-elevation --no-sync-snap --recursive zroot/poudriere jbo⊙111:storage/sanoid_test/beefy02/poudriere 01:32:15 i've never tried it as non-root so i don't really have any idea how it works :-/ 01:32:29 you could try now :p 01:32:41 i cannot because i have to wait for my backup vm provider to upgrade the memory... 01:33:14 i thought 2GB might be okay for zfs but it turns out it's really not :-d 01:33:37 my datacenter remote host that does zfs recv has 4GB 01:34:01 my current situation is, "zfs destroy backup/old-fs" causes a kernel panic 01:34:14 soooo i'd rather not touch it until that's sorted :-) 01:34:21 a real kernel panic?! 01:34:29 not just OOM killer? 01:35:03 hm, I throught under 4GB you should tune for performance and under 1GB you have to tune to have zfs running properly 01:36:30 jbo: fuchsia.eden.le-fay.org login: root 01:36:31 Password: 01:36:31 fuchsia:~# zfs destroy -r backup/data 01:36:31 [ 1891.754792] Kernel panic - not syncing: System is deadlocked on memory 01:36:31 [ 1891.754804] CPU: 0 PID: 1 Comm: init Tainted: P O 6.6.9-0-virt #1-Alpine 01:37:06 hey i hear fuchsia is pretty cool. start rewriting freebsd in rust wen? 01:39:41 i might replace this vm with a proper server at some point but for now i just asked them to add another 6GB of memory 01:40:05 lw, you can get a VM from a provider that offers zfs recv as a service :) 01:40:30 jbo: are any of those providers not terrible? this is a small company run by a guy i know on irc... 01:40:43 rsync.net 01:40:44 just saying 01:40:59 they're pretty huge, that puts me off 01:41:20 also i pay £1/150GB for disk space, which is more than most backup services, but cheap compared to a lot of VM providers, and i can run my own OS 01:41:37 (well, any OS as long as it's linux, but still) 01:41:49 which brings me to a question: how do you backup a zfs-native-encrypted dataset over zfs send|recv ? 01:42:26 i use a separate encryption key on the source host and the backup host. and yes, this is not a good idea if you are encrypting secret data... for my use-case it's fine 01:42:38 but i believe you can tell syncoid to use zfs send -x which sends the raw encrypted data 01:42:47 can I zfs send the encrypted dataset? so it's still encrypted on the dest host? 01:42:52 yes 01:42:56 neat 01:43:10 if you send it with -x, the destination host can't decrypt it at all, the only information it has is that the dataset exists 01:43:12 so it's like live backups? 01:43:23 obviously you can't mount the dataset in that case 01:43:45 (well unless you copy the encryption key to the backup host) 01:44:04 nah I just want to have a retrievable copy - not accessible on the dest host 01:44:06 alepzi: that's how zfs backups works in general, you end up with a copy of the source filesystem on the destination that you can access normally 01:44:25 jbo: this is again something i've never tried, but i might play around with it at some point 01:44:33 lw, so far I have only used passphrases for zfs encryption anyway. might want to look into using a yubikey tho 01:45:06 i just use /etc/zfs/data.key. which... yeah, this is not secure, but the only reason i use encryption is so i can throw out old disks without wiping them 01:45:06 I was a bit "meh" to learn that I can't "properly" use a yubikey with KeepassXC 01:45:30 lw, that sounds like a terrible idea (the "without wiping them" part) 01:45:40 jbo: why? they're encrypted, no one can read the data 01:45:44 afaik the password is used to protect a key that gets generated when you decide to do encryption 01:46:03 lw, maybe not now 01:46:13 not now? what do you mean 01:46:13 encryption gets broken all the time down the road 01:46:21 eh 01:46:34 if someone cares enough about my secret data that they break AES-256 to read it, they can have it 01:46:42 more power to them 01:46:52 :D 01:46:53 like i said, the data on these disks is not that secret, if it is, do not do like i do :-) 01:47:32 mbuffer: error: outputThread: error writing to at offset 0x0: Broken pipe 01:47:32 mbuffer: warning: error during output to : Broken pipe 01:47:41 :< 01:48:08 warning: cannot send 'zroot/poudriere/data/packages@autosnap_2024-01-11_02:05:59_daily': signal received 01:48:08 warning: cannot send 'zroot/poudriere/data/packages@autosnap_2024-01-11_02:05:59_hourly': Broken pipe 01:48:31 [00:57:55] [03] [00:56:42] Finished lang/rust | rust-1.75.0: Success 01:48:36 at least that... slowest rust build yet. 01:48:51 [02] 00:13:15 devel/electron27 | electron27-27.2.1 build 00:08:35 8.9% 2.6% 01:48:56 lw, any ideas or hints regarding the broken pipe? 01:48:56 why is it building electron27 01:48:59 did they upgrade signal? 01:49:18 lw so do you know how ZFSBOOT_GELI_KEY_FILE works? like how i can generate one? 01:49:18 jbo: broken pipe must be a symptom of something else, are there no errors before that? or something on the remote host (in messages or auth.log or something)? 01:49:27 alepzi: never used geli, sorry 01:49:39 isn't that what does the zfs disk encryption you talkeda bout? 01:49:40 alepzi: although if it's like zfs, i would dd from /dev/random 01:50:00 i'm using zfs native encryption, that's different from zfs-on-geli which is an older method 01:50:14 (older but not necessarily worse, it has some advantages) 01:50:14 how do i use zfs native encryption with unattended bsdinstall? 01:50:22 i don't know, sorry 01:50:28 how do i use it period then plz? 01:50:32 or how do you use it? 01:51:10 geli is device/disk level encryption. zfs is dataset level encryption 01:51:18 hang on, my desktop is so slow when ports is building 01:51:41 alepzi: https://klarasystems.com/articles/openzfs-native-encryption/ 01:51:42 Title: OpenZFS Native Encryption | Klara Inc 01:51:59 lw, broken pipe does not show up anything on the dest hosts auth.log 01:52:08 which type is enabled by ZFSBOOT_GELI_ENCRYPTION? 01:52:13 lw, first it goes "Accepted public key", then "received disconnect" then "disconnected" 01:52:25 alepzi: since it has GELI in the name i'm guessing that is for zfs-on-geli 01:52:36 ya 01:52:47 ok i'll read that link thx 01:53:21 also: https://forums.freebsd.org/threads/geli-vs-zfs-encrypted-dataset.84721 01:53:22 while i'm reading it any of you guys switched from geli to zfs native encryption and regretted it for this or that? 01:53:22 Title: ZFS - GELI vs ZFS encrypted dataset | The FreeBSD Forums 01:53:28 nod 01:54:15 i think the main downside of zfs native encryption is it exposes dataset names. so if you have a dataset called mypool/illegal-anarchist-bomb-recipes, don't encrypt that with zfs 01:54:30 storage/media/adult :D 01:54:54 does it slow disk i/o, use more cpu, anything disadvantages like that? 01:55:09 geli encrypts the entire disk, so without the encryption key you can't even tell there's a zfs dataset on it 01:55:19 alepzi: not that i know of, both should use AESNI for encryption on modern CPUs 01:55:32 one feature I like of zfs-native-encryption over geli is that you can ad-hoc load and unload keys 01:55:39 ok well that cuts out the whole geli layer so prolly smart for me to switch to 01:55:49 alepzi: however one benefit of zfs native encryption (as i was discussing with jbo earlier) is you can transparently send the encrypted dataset to a remote system without decrypting it 01:55:53 oh hmm 01:56:14 for example on my laptop I only load keys of datasets I need, then unload them again. 01:56:25 that does sound pretty cool 01:56:26 can't easily do that with GELI afaik unless you have more hardware 01:56:39 fire up the web browser -> unload dataset keys :p 01:56:53 yeah, geli can only decrypt the entire pool 01:56:55 so kinda seems like native zfs encryption is better in every way except dataset name leak 01:56:58 you'd need separate pools to do that 01:57:13 lw, can you have more than one pool on a drive with geli? 01:57:33 jbo: i've never tried but couldn't you apply geli separate to two partitions? that's how encrypted swap works 01:57:46 dunno 01:57:51 just do geli on partition level, not disk level 01:57:52 hence I"m asking :p 01:58:11 i think geli can work on any geom device and a partition is a geom device... 01:58:13 yeah always encrypt your swap for sure :D 01:58:20 that makes sense 01:59:09 lw, that mediaelch PR probably has to wait. my desktop is awfully slow at testporting 01:59:38 jbo: no rush, i only use mediaelch when i download entirely legal, public domain films from reputable websites 02:00:40 lw, is there a "zfs way" of checking whether the src and dest datasets are "equal"? 02:00:48 ok why is poudriere making my wayland so sad 02:00:48 something like mtree but then to check whether sanoid/syncoid did the thing 02:00:58 wayland is making itself so sad :p 02:01:02 * jbo ducks 02:01:06 what the heck 02:01:07 Jan 11 01:43:07 ilythia kernel: swap_pager: out of swap space 02:01:07 Jan 11 01:43:07 ilythia kernel: swp_pager_getswapspace(28): failed 02:01:07 Jan 11 01:49:16 ilythia kernel: swp_pager_getswapspace(27): failed 02:01:07 Jan 11 01:55:29 ilythia kernel: swp_pager_getswapspace(11): failed 02:01:16 better that than a broken pipe 02:01:25 i have 32GB RAM and it's not even close to used 02:01:40 Mem: 3772M Active, 14G Inact, 2903M Laundry, 9971M Wired, 1559M Buf, 1228M Free 02:01:40 ARC: 3835M Total, 1480M MFU, 1025M MRU, 30M Anon, 65M Header, 1213M Other 02:01:45 wtf is 14GB inact doing? 02:01:59 you might want to read about memory management of the FreeBSD kernel 02:02:11 jbo: will that give me a magical sysctl to fix this problem? 02:02:22 https://wiki.freebsd.org/Memory 02:02:23 Title: Memory - FreeBSD Wiki 02:02:54 lmao what is this https://www.le-fay.org/tmp/30d/YaSCFT.txt 02:03:01 swap is 100% full, memory is empty 02:03:45 lw, is there a "zfs way" of checking whether the src and dest datasets are "equal"? 02:04:04 jbo: i'd probably do like diff =(zfs get all mypool) =(ssh myhost zfs get all mypool) 02:04:11 but i guess that's not what you mean by "zfs way" 02:04:29 I'm looking for zfs-diff I guess :p 02:04:39 i don't think that exists 02:04:42 it does 02:04:50 does it? why did you ask then :-P 02:05:01 oh this shows all differences 02:05:06 i thought you just wanted properties 02:05:06 because the manual says "between two snapshots of a given filesystem" 02:05:11 but that is not what you asked, i am bad at reading 02:06:02 jbo: this wiki page is not helpful, why do i get 'out of swap space' errors when i have 14GB inactive? 02:06:17 lw, because your swap is full according to your own reports 02:06:44 Mem: 3783M Active, 14G Inact, 2896M Laundry, 9753M Wired, 1558M Buf, 930M Free 02:06:47 root@ilythia /h/lexi# swapoff /dev/nda0p2 02:06:47 swapoff: /dev/nda0p2: Cannot allocate memory 02:06:51 ... 02:07:31 i blame the vmm 02:07:58 jbo: anyway if 14GB of RAM is Inact why would it matter if swap is full? 02:08:12 isn't inact meant to be basically pages on the freelist 02:09:14 inactive is not "free". that would be more your clean queue 02:09:23 this reminds me of early FreeBSD ZFS where it didn't free ARC pages properly and you'd run out of memory, except i limited ARC to 8GB 02:09:25 the inactive queue holds pages that need to be passed to the backing storage first 02:10:13 jbo: are you sure? the wiki says it's "pages evicted from the buffer cache", in which case surely they can't be pending writes to the backing store 02:10:16 and AFAIK inactive pages don't become free pages immediately. instead, inactive pages are flushed, then become cache pages and only then free pages (if it ever happens) 02:10:41 in any case something is definitely wrong here, because i'm barely using any memory and yet something is causing out of memory errors 02:10:44 everything I say is AFAIK 02:11:05 top says 3.2GB active, which is 10% of total memory... 02:11:06 I haven't studied this in a while 02:11:24 lw, any chance that you're getting screwed up by TMPFS with insufficient memory? 02:11:37 no tmpfs here, i disabled it in poudriere 02:11:53 that must be fun when building electron 02:12:14 it's not too bad, takes about 4 hours 02:12:50 wut? is that a clean build or with cache? 02:13:09 maybe it's my ~5MB/s of background NFS I/O causing pages to not be freed 02:13:19 jbo: clean, with ccache it's like 50 minutes 02:13:20 sounds like somebody is streaming 02:15:36 i might post this to a list because i'd like to know what's going on. not sure what the right list is though... -current? 02:15:47 or -questions 02:15:47 are you running current? 02:15:50 yes 02:15:54 then I'd say so. 02:16:06 although i'm pretty sure i've seen this on releng/14.0 as well 02:17:00 the current ml has the benefit of having more developers with "intimite" knowledge of VMM (I assume) 02:22:25 interestingly there are similar messages from earlier while i wasn't running poudriere: Jan 3 18:26:54 ilythia kernel: swap_pager: out of swap space 02:22:58 anyone see more kernel panics when switching to zfs native encryption from geli? 02:23:59 yes, but only because of PR#275306 which was fixed in (i think) 14.0-p3 02:24:00 275306 – 14.0-RELEASE: ossl(4) causes data corruption on encrypted ZFS filesystems/volumes https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=275306 02:24:20 alepzi, what are these random questions about? 02:24:29 wow 02:24:38 well, in that case i didn't switch "from geli" since i've never used geli 02:24:41 well, that sounded more negeative than necessary 02:24:41 jbo why assume they're random? 02:25:28 not sure if it's 14 or zfs but lately it's felt a lil buggy in freebsd and zfs land 02:25:41 i think .0 releases are usually a lil rough though 02:28:18 lw, based on what I understand, syncoid is "by default" not setting the mountpoint on the dest host 02:28:48 jbo: that matches my limited experience 02:29:43 lw, why does the dest host dataset need allow mount then? 02:29:48 "then" 02:29:50 I mean in general 02:30:08 you'd think that it can replicate the dataset without mounting 02:31:11 jbo: my suspicision is, say the filesystem is /foo and the child filesystem is /foo/bar, and you've sent /foo and /foo is owned by root, now you can't can't create /foo/bar? but this doesn't really make sense since the zfs send for /foo should have included the mountpoint... so... idk 02:32:00 not feeling confident running this on a production server yet :/ 02:33:01 maybe one day I'll understand how zfs mountpoints truly behave 02:34:20 I'm currently also in the process of ditching the jail management solution I used in favor of just raw jail handling 02:34:40 today in skynet news | 1 Jan 11 Jamie Landeg-Jones ( 19) BSDforge is alive. 02:36:12 lw, the mediaelch testport for qt5 just failed - did you testport that? 02:36:29 jbo: i did, can you show the log? 02:36:50 lw, on which target (version, arch)? 02:36:52 i can't testport it right now since i'm bulding normal ports (you know, electron) 02:37:03 jbo: uh... i guess 15-current, amd64? but whatever you have is fine 02:37:54 well as a committer I have to test reasonable/sane configurations which is why submission-to-commit can actually take such a long time and why its nice to have a whole separate beefy build server 02:38:13 oh, i meant, can i see the build log from the failure you mentioned 02:38:17 lw, the qt5 flavor failed on 14.0-RELEASE on amd64: https://bsd.to/17L6/raw 02:38:18 Title: 17L6 02:38:21 ty 02:38:34 you're probably missing the buildutils 02:38:54 oh that's weird, i'm sure i fixed that for qt6, maybe qt5 needs a different USE 02:39:08 buildtools:build and qmake:build if memory serves right 02:39:10 yes 02:39:22 https://cgit.freebsd.org/ports/commit/?id=b9be86ddf6fec059e7e05ae0d7f791b788861d7b 02:39:24 Title: ports - FreeBSD ports tree 02:39:34 no, it doesn't use qmake, it uses cmake 02:39:49 still needs the build tools 02:39:49 does it still need qmake though, that's weird 02:40:03 yeah, see the commit I made & linked. that's also a cmake based build 02:40:15 jbo: ok, i'll test a fix once this poudriere is done 02:40:47 i'm sure i tested it with testport though because i said in the PR: "fix testport errors" 02:40:54 maybe that doesn't build both qt6 and qt5 version 02:41:20 Ominous: https://lists.isc.org/pipermail/bind-announce/2024-January/001242.html 02:41:21 Title: Pre-announcement of BIND 9 security issues scheduled for disclosure 17 January 2024 02:41:40 CrtxReavr: this is going to be yet another nsupdate issue, i bet 02:41:55 lw, you have to test all flavors 02:42:05 jbo: i thought testport did that. but clearly not 02:42:14 lw, if you don't specify the flavor it will build the default flavor 02:42:21 (i did test qt5 manually but i probably had builttools installed locally) 02:42:27 :) 02:43:36 lw, you're running wayland, right? 02:43:39 i am 02:43:46 I assume that is not nvidia hardware then? 02:44:07 jbo: no, Radeon RX 6800 XT 02:44:23 never had a non-intel or non-nvidia GPU. does that work well? 02:44:46 it has worked flawlessly in freebsd (except for that loader(8) issue), but i haven't tried any real 3D apps on it yet 02:44:49 I adore that nVidia provides FreeBSD drivers themselves officially. no CUDA tho :( 02:45:13 i've been meaning to try Factorio or X4 but i always have other stuff i need to be doing, like these darned ports 02:46:07 I can land your ports more quickly if you test then more thorougly in the future :) 02:47:19 well ok, i mean i didn't even know testport existed when i submitted these 02:47:44 that's fine - it's a learning process. nothing wrong with that. that's why I tell you :) 02:47:54 i guess if i was a ports export i could just commit them myself, but i'm not, so i do what i can :-) 02:47:56 now you also know that testport doesn't build all flavors :p 02:48:00 s/export/expert 02:48:26 that's a long way to go - gotta start somewhere tho :) 02:48:33 jbo: i didn't mean i waste time waiting for you, i mean i waste time fixing the ports because of my own errors 02:48:53 nothing wrong with that. gotta learn stuff. 02:49:35 but also i've been submitting a lot of patches for the new version of net-im/toot recently 02:49:57 which is actually pretty interesting 02:50:47 ... also i submitted an update patch for the current net-im/toot and before it was applied a new version was released, so i made a patch for that, heh 02:51:37 that's a good start :) 02:52:14 wait, the came out wrong 02:52:37 i mean the author of net-im/toot is making a new, unrelated app which i've been submitting packages for, and in the mean time i also made some patches for the existing port 02:52:48 the former patches are not freebsd related 02:58:25 jbo: so here's a question, how do you become a ports committer? i've submitted a few patches for src but i don't see any benefit to having a src commit bit... but being able to commit to ports would handy. is it just a matter of learn how things work then asking? 03:01:02 lw, as far as I can tell its much, much easier to become a src committer than a ports commiter. AFAIK there are something like 4000 committers but only 165-ish are ports committers. the rationale is that screwing up in src is less disasterous than screwing up in ports 03:01:38 hmm 03:01:57 i guess it's true that my src patches have been accepted much more easily than ports commits 03:02:23 lw, in my case it was engaging in FreeBSD community stuff for the better part of 10 years, contributing to ports in terms of PRs and patches and stuff for a few years and maintaing a healthy amount of ports (although that one is low in my case). plus I am an "embedded developer" so I played the "I touch a lot of software that is not commonly used" card 03:02:28 port is basically a rube goldberg machine in the end 03:02:51 jbo: 10 years? no wonder there are only 165 committers 03:03:23 there are no hard rules on this AFAIK. but there is a vote even after you pass the basic "does it make sense?" bar 03:03:34 my recommendation: just keep going. don't try to force anything. 03:03:44 i'm not pushing, i'm just curious 03:03:50 that's the correct approach :) 03:03:52 I'm still under mentorship and I don't expect that to change any time soon either. 03:04:27 my blood pressure is still skyrocketing ever time I push to the repo :p 03:04:49 i set up a git fetch hook to mention you on irc every time you push to ports 03:04:54 (this is lie, i did not do that) 03:04:58 hah :p 03:05:06 yeah that was an almost-shit-my-pants-again moment indeed 03:05:43 there has to be a better way to do this stuff 03:05:52 what stuff? 03:06:12 i submitted a PR for a port recently because it wrongly installed a binary setuid root, and it was completely ignored until an unrelated non-committed submitted a PR which a patch which ended up being committed 03:06:32 s/non-committed/non-committer 03:06:51 jbo: ports stuff, like not putting so much load on a small number of people 03:07:17 lw, small number of people is probably very intentional in this case. 03:07:19 lw, you can always mail to the ports ML to bring stuff to attention 03:07:21 i think (i don't mean this in a negative way at all) that ports committers are pretty overloaded 03:07:39 stuff easily gets missed on bugzilla especially if its not properly assigned, tagged and stuff. we do luckily have the triage team tho :) 03:07:45 i did mail -ports about something else (devel/py-urwid has been broken for months) and got zero replies 03:07:56 again, not complaining. but it's not a great situation 03:08:13 FreeBSD is certainly not perfect - we all try to do our best tho. 03:08:39 python stuff is it's own story entirely. you can mail python@ for stuff if necessary 03:09:08 i know it's not perfect, i'm not complaining, it just feels like maybe there's a better way to allocate available resources? idk 03:09:11 based on my whole-life experience, I think the "active" FreeBSD ports team is doing an excellent job 03:09:25 it's always a trade off tho :) 03:09:41 this is really just a response to learning there's only 165 ports committers, i thought there would be more 03:11:03 oh, fixed my swap problem Swap: 18G Total, 2048M Used, 16G Free, 11% Inuse 03:11:10 at least until it decides that's not enough either 03:11:20 don't quote me on the exact number. but when I was given the commit bit I was congratulated with "welcome to an extremely exclusive club" :D 03:11:29 might be 168 now, 172, dunno. 03:11:39 it was around 165 when I became one. 03:12:10 so for example like, the issues in my ports have been because they're new ports and i did them wrong, but i'm fairly sure i could update an existing port with fewer issues 03:12:18 being a ports committer is not what I would consider a fun activity btw. It's just my way of contributing / "giving back" 03:12:25 mainly because someone else already did the hard part 03:12:46 i certainly wouldn't expect it to be fun, i just want to fix issues on my freebsd systems :-) 03:12:55 lmao if i wanted something fun i'd choose literally any git repo other than ports.git 03:13:14 you can help a lot without a commit bit. being a commiter just puts the blame and pressure on you. PRs that contain patches for existing ports are alway, always highly appreciated. 03:14:46 not to spill any secrets here but I think it's also self explanatory that committers are picking which PRs they handle. if you're a submitter with a good reputation you'll get much quicker turn around. 03:15:25 i'm used to people hating me but i don't think i've submitted enough PRs for that quite yet 03:16:50 it's more about the quality of the patches I'd say 03:16:51 well i do have 16 open PRs. but uh, i think most of those are ones i cc'd myself on 03:17:45 imagine if commercial users of freebsd hired 10 ppl to do nothing but fix bugs freebsd would get so good 03:17:55 on a more src-oriented note i don't understand why we can't get this committed: https://github.com/freebsd/freebsd-src/pull/957 03:17:56 Title: libkrb5: avoid crash if MD4 is not available by llfw · Pull Request #957 · freebsd/freebsd-src · GitHub 03:18:03 it's broken, upstream will never be updated, the fix is trivial 03:19:07 alepzi: they do? netflix, klara, darpa, chelsio, amazon, iXsystems, plenty of other companies pay people to work on freebsd 03:19:25 and sony :p 03:20:09 ilythia /s/main (main)> git log|grep 'Sponsored by'|head -5000|uniq -c|sort -nr|head -10 03:20:09 37 Sponsored by: Netflix 03:20:09 33 Sponsored by: Juniper Networks, Inc. 03:20:09 30 Sponsored by: The FreeBSD Foundation 03:20:09 27 Sponsored by: The FreeBSD Foundation 03:20:09 26 Sponsored by: Serenity Cyber Security, LLC 03:20:09 25 Sponsored by: Netflix 03:20:10 24 Sponsored by: Juniper Networks, Inc. 03:20:10 24 Sponsored by: Kumacom SAS 03:20:11 22 Sponsored by: Netflix 03:20:11 21 Sponsored by: Innovate UK 03:20:33 ('the freebsd foundation' includes a lot of donation from companies like sony) 03:20:33 lw, to round off this conversation: just keep going and don't feel limited by not being able to commit yourself. everything you can contribute you can without that ability. 03:21:14 wait, that was a bad command 03:21:30 ilythia /s/main (main)> git log|grep 'Sponsored by'|head -5000|sort|uniq -c|sort -rn|head -10 03:21:30 1528 Sponsored by: The FreeBSD Foundation 03:21:30 648 Sponsored by: Netflix 03:21:30 352 Sponsored by: Klara, Inc. 03:21:30 343 Sponsored by: Rubicon Communications, LLC ("Netgate") 03:21:30 238 Sponsored by: Juniper Networks, Inc. 03:21:30 173 Sponsored by: Arm Ltd 03:21:31 166 Sponsored by: Beckhoff Automation GmbH & Co. KG 03:21:31 109 Sponsored by: Beckhoff Automation GmbH & Co. KG 03:21:32 104 Sponsored by: DARPA 03:21:32 99 Sponsored by: Chelsio Communications 03:22:04 how about not spamming the channel? 03:22:05 that's 3760 out of 5000 commits that were sponsored, well over 50% 03:22:51 lw, regarding your GitHub pull request: 1\ I think that whole GitHub story is a bit controversial, 2\ Personally (!!!) I would have extendend that krb5_set_error_message() message to say why (i.e. because MD4 is lacking) 03:23:01 jbo: no harm when it's this quiet 03:23:53 If one squints really hard, you might be able to see a sony commit in there somewhere ;) 03:24:11 haha 03:25:07 nice use of unix piping tho, lw 03:25:43 it's still not perfect because of the whitespace, but if i try it again you'll probably murder me 03:25:51 nah :p 03:25:58 but I assume that will be some sed WOL 03:26:15 i was thinking awk '{print $3-}' 03:26:37 i guess freebsd awk doesn't like that syntax 03:28:01 https://www.le-fay.org/tmp/30d/QRD3eZ.txt same result basically 03:29:18 i wonder why DARPA submits so many patches 03:30:30 oh, that's CheriBSD 03:32:07 lw wow cool 03:34:28 the main difference between freebsd and linux is linux has a lot of users who sell it to customers (redhat, canonical, ...) while freebsd tends to have users who use it internally (netflix, klata, juniper, ...) 03:34:46 so the 'end user' experience in freebsd might be less polished but that doesn't reflect on the internals 03:36:41 like, netflix is not sending patches to make wayland better 03:36:54 (well idk, maybe they are, but i imagine that's not their main focus) 03:37:58 Hmm, would Netflix use wayland in any way for their business internals? 03:38:23 That would be interesting to know 03:38:37 kenrap: perhaps some of their freebsd-focussed employees run it on their desktop? 03:38:49 i have no idea though 03:39:52 heh: 499e84e16f56013e24fb69ae8ecfe75180e8d704 copyright: Bump the copyright date. Sponsored by: Netflix 03:40:14 i doubt netflix cares about the copyright date so i guess they just pay imp@ to do stuff 03:40:32 <_xor> I wish Wayland would get more love, but from the opinions I've gotten so far, the API is a pain in the neck. 03:40:43 <_xor> ...and I run Wayland on my desktop. 03:40:56 in fact all of imp's commits seem to be sponsored by netflix 03:41:25 _xor: i know netbsd complained about it because they didn't want to implement libinput... 03:41:47 which, is fair, they have their own input system, why should they clone linux's? 03:42:08 lw: netflix technically uses a specialized FreeBSD system derived from CURRENT, so your idea is not farfetched at all. 03:44:44 my main concern about wayland is that GNOME is going to dominate it with their CSD crap and whatever else and ruin it for normal users 03:46:28 well, the X11 API also sucks, is Wayland at least an improvement? 03:47:14 zwr: yes and no, it's an improvement for systems that use KMS/DRI (which includes freebsd) but it means the compositor (window manager) has to manage all the OS-specific stuff so niche platforms like netbsd get screwed unless they implement the Linux APIs 03:47:37 ... or write their own compositor, which isn't really a great solution 03:49:43 it's fair to dislike it because it's another aspect of the Linux Leviathan, which is the reason we have timerfd in freebsd now 03:50:01 posix is dead, all that matters is being compatible with linux, the microsoft of unixes 03:57:53 well adding 16GB of swap space has apparently fixed my 'out of swap space' errors but i don't really understand why 03:58:17 i still have 11GB of Inact memory 04:00:40 to be honest, timerfd has the benefit that you can poll() (or any better "wait on many fds" function) for a timer or file or anything else that works via fds and create something like a reactor program, and that's a design that works really well for many kinds of programs and scales well with threads up to a point. you can rig other APIs like clock_nanosleep() to do the same thing, but it means spawning a 04:00:46 thread which is a waste of resources. microsoft actually got this right with more or less everything being a HANDLE there. 04:01:45 zwr: but you can already do this with kqueue EVFILT_TIMER 04:01:46 but yes, Linux generally has inferior API design to the BSDs, and it pains me whenever a BSD implements an inferior Linux function for Linux compatibility. Most recently, NetBSD got getrandom() when they already had the superior arc4random() 04:02:24 the functionality of timerd is fine *for linux* because their epoll() api is so limited, but freebsd already has an API to do that... but we have to implement the Linux API because Linux is the standard 04:15:00 anyone here work with iocage at all? i am getting confused.. i create a jail, with an ipaddress, then i ssh into this new ip address...but it does not ssh into the machine.. 04:21:11 nevermind.. 04:21:20 iocage console for now 04:24:32 * lw wonders why traceroute6 doesn't show IP addresses 04:24:53 ah, you need -l 04:25:02 but this should be the default for consistency with traceroute 04:27:05 probably done due to v6 address lengths 04:28:37 there's basically no difference in practice https://www.le-fay.org/tmp/30d/vUeM5r.txt 04:31:30 i might send a patch to make this default 04:37:44 https://github.com/freebsd/freebsd-src/pull/1023 04:37:45 Title: traceroute6: remove -l flag by llfw · Pull Request #1023 · freebsd/freebsd-src · GitHub 04:46:49 meena: i don't remember if we talked about this, but can non-committers submit patches on phab? and if so how do you know who to add as reviewers? 05:20:46 lw: yes, you can create account and submit reviews 05:22:01 there are some rules in there that would possibly add the reviewers (if someone is interested in the parts you modify), otherwise check last committers who modified it, and/or send a request to hackers@/net@? 06:28:57 hello o/ how do I go about debugging leaking nmbclusters? I have a pair of bird bgp routers running 13-STABLE from about a month ago and the active router is leaking mbuf clusters like crazy 06:30:48 I didn't have metrics for mbufs until yesterday so I can't say for sure when this began, but I strongly suspect it was when I updated in december, from 13-STABLE-384a885111ad (december 21, 2023) to 13-STABLE-2cbd132986a7 (december 19, 2023), where I also upgraded bird2 from 2.0.11 to 2.14 06:31:04 &8 06:31:31 ugh, I mean from 13-STABLE-384a885111ad (december 21, 2022) to 13-STABLE-2cbd132986a7 (december 19, 2023) 06:37:01 I am looking for a way to identify what might be causing this, and hopefully get it fixed, what are some things people usually do when hunting mbuf cluster leaks? is it possible to see a list and what allocated them? 06:57:21 tykling: how many/which network related src commits were between those dates? is it possible to try 2.0.11 with 384a885111ad ? 06:58:39 (or bisect) 07:41:40 lw: what yuripv said 09:33:44 If a process is in D state, how can I check *what* is it waiting for? 09:49:26 antranigv: usually the function it's sitting in should give you a clue, so really, just the usual 10:10:32 meena what do I do if I can't even kill -9 a bhyve vm? 12:19:37 kill -30  ;-) 12:38:53 Kalten wait, is that a joke or a real signal? :D 12:44:21 antranigv: /usr/src/sys/sys/signal.h interesting: 30 is SIGUSR1—it could be, that it is just an unhandled signal terminating most? 13:36:43 https://www.bsdnow.tv/541 13:36:44 Title: BSD Now 541: Learning and Teaching 13:49:46 antranigv: So, did that work? 14:15:57 Yo. noob to bsd. Is there a simple way to skip bootloader menu (unless holding some key) and show a custom boot splash? I found some info on splash but it seems outdated and not supporting UEFI etc. 14:25:02 Is it a bad practice to take a backup of a running bhyve VM instance (actually doing tar on the 'disk1.img' VM disk while it's running)? Should the VMs be stopped before doing so? Any data loss possibility, otherwise? 14:25:19 lw: is this related to your Kerberos complains, https://freshbsd.org/freebsd/src/commit/cb350ba7bf7ca7c4cb97ed2c20ab45af60382cfb ? 14:25:20 Title: FreeBSD / src / cb350ba / kerberos: Fix numerous segfaults when using weak crypto - FreshBSD 14:25:24 If "powering them off" is needed for a healthy backup procedure, and in case of dozens of VMs running, how would you handle that? 14:26:54 tercaL: yes, that's bad practice not just on bvhyve. Generally, the recommended way to do this is to freeze a VM, then take a snapshot, then unfreeze it. that generally takes maybe 5 seconds altogether 14:28:30 meena: Oh, thanks. "Freezing" a VM? What's its switch? 14:29:04 I just know vm poweroff, start, stop, destroy and so on. 14:29:21 dunno how to do that in bhyve 😬 14:34:16 tercaL: zfs snapshot of the running volume (from the host) works well in that context 14:35:53 dunno if it's recommended or not 14:36:14 prob not but ive not seen any ill effects on fast hardware 14:36:15 hmm, even though splash(4) only supports the old syscons(4), vt(4) is capable of a "graphics mode" which makes me thing it's possible to have a boot splash. But I guess that requires a splash-like reimplementation for vt to begin with *shrugs* 14:37:37 tercaL: some folks run incremental backups every 15 mins on running systems 14:38:21 but backing up a disk? power it down every time 14:38:55 disk1.img that is. use zfs volumes for the backing store of the guest 14:39:55 recovering from "instant" snapshot of a volume would be the same thing as if your machine lost power at that very moment 14:42:29 would that imply bad news for the snapshot? 14:42:49 the filesystem is suposed to remain coherent 14:43:05 that doesnt mean it's content is supposed to make sense 14:45:55 the reason im asking is because I've seen people writing they're incrementally snapshotting at short intervals. I'm presuming they're able to recover if where they're snapshotting from becomes unusable. They are not interrupting services 14:47:12 depends what you are doing 14:48:37 ie if you snapshot while an update is running, don't expect the backup to boot 14:50:27 there must have been testing for this 14:53:11 ive not seen it mentioned that either a live system should not be snapshotted or, if not that, that the system needs to be quiet, or how quiet. But that might just be down to not reading all of whats needed 14:54:41 i mean it's lore that a system needed to be off or quiet due to the inconsistent data risk but i thought copy-on-write was meant to eliminate that 14:56:47 once again, it's equivalent to pulling the plug, but without the electrical/mechanical damage 14:58:37 yeah i understand. but i thought zfs works round the consequences. 15:01:58 ^^same here 15:02:12 does anyone have experience driving a fan through GPIOs? This is for a raspberry pi? 15:03:10 I plan to create a script to gather cpu temp and speed up/down the fan 15:44:49 so i'm trying bhyve. and trying to passthru a nic device but having no luck. bhyve won't boot and just exits with status 4 15:45:01 any ideas? 15:56:14 i set vmm_load="YES" in boot loader, and pptdevs="2/0/0", then -s 5:0,passthru,2/0/0 to bhyve 15:58:17 i see bhyve.log has Unable to setup memory (12) 15:59:07 Buy moar rams. 16:00:21 right reduced memory from 14gb to 12gb, boots now. weird that it only happened after i added passthru :/ 16:02:25 I was joking. . . sorta. 16:02:37 well it worked lol 16:02:54 But the memory allocated to passthrough, likely couldn't be paged (swapped), which is likely the reason. 16:03:01 sysctl -a | grep iommu 16:04:30 https://paste.mozilla.org/zTepkrgK 16:04:31 Title: Mozilla Community Pastebin/zTepkrgK (Bash) 16:04:49 for i in $(sysctl -a | grep iommu | cut -d: -f1) ; do sysctl $i ; done 16:05:16 lol what :p 16:05:35 (it didn't really do anything different) 16:06:04 Well, not output wise, but. . . somehow it seemed less ghetto. 16:06:32 depends what kinda ghetto you live in i suppose 16:06:44 Though, for that matter: sysctl hw.iommu 16:07:01 either way. is my pastebin missing something and i can't use iommu? 16:08:07 looks fine 16:09:19 CrtxReavr: it only lists keys in hw.iommu 16:09:40 my passthru device doesn't work though. it's a windows guest and it says "cannot start" in device manager :/ 16:10:05 babz, as does 'sysctl -a | grep iommu' 16:10:56 hmm reboot "kinda" helped. no more error, but now says network cable unplugged (it's a nic) 16:11:15 no, it also matches hw.vmm.iommu 16:11:19 but whatever 16:11:34 Not on my system. . . but whatever. >=] 16:14:20 could be wrong device i guess, passing thru all nics :p 16:17:24 can i get 'vncviewer' for freebsd? or something else i can run from command line in "kiosk mode" 16:20:09 as i understand getting gpu passthru to work is a hassle? 16:20:18 especially with windows 16:21:50 omg ethernet works 16:23:49 I have a FreeBSD vm that keeps on freezing under heavy php load, I can't put my fingers on it 16:24:04 when it freezes, I see under top many processes in STATE: *vm pa 16:24:15 what does that do ? 16:35:37 man top 16:38:18 STATE is the current state (one of “START”, “RUN” (shown as “CPUn” on SMP systems), “SLEEP”, “STOP”, “ZOMB”, “WAIT”, “LOCK”, or the event on which the process waits) 16:40:03 * means it's wating for a lock 16:40:52 how can I troubleshoot this ? There's literally nothing in the logs 16:40:55 the box just seizes up 16:41:06 not in console, /var/log/messages 16:41:54 in esxi I see the vm taking like 12Ghz of cpu 16:46:19 i think vm_pa is for vm_page_grab 16:47:08 I would be suspecting memory resource exhaustion stress. (Just checking but you don't have swap on zfs do you?) 16:47:54 I would reduce the memory resource footprint for the system. Perhaps add more swap so that it can survive long enough to at least log something. 16:48:20 But the problem might be hardware. The machine might just be crashing due to hardware problems. And then that is not a software problem to solve. 16:48:25 Good luck! 16:50:29 the physical host itself has plenty of ram, the vm sits @ Mem: 7667M Active, 1795M Inact, 1727M Laundry, 2760M Wired, 1245M Buf, 2002M Free 16:50:45 swap isn't even in use 16:51:37 the whole system starts being sluggish, then eventually freezes up completely. If I kill the http connections, it comes back eventually in ~2-3 minutes 17:01:08 last1, Hmm... No idea here. Sorry. Hopefully more clues will present itself. Perhaps set up a remote logging syslog to capture final bits? 17:01:46 I would keep a terminal logged in with a "tail -F /var/log/messages" at least with the hope of seeing something when it happens but it sounds like that would not be helpful in this case. 17:02:04 I would keep a terminal logged in with top running and hoping that it might show a clue. 17:02:27 I realize these are pretty brute force and ignorance approaches to debugging this. I am hoping someone else will have better ideas for you. 17:03:32 That 2 minute recovery though may be a clue. The "Internet Lifetime of a Packet" is 2 minutes. And you said if you kill the http then this happens. 17:04:06 Just brainstorming here but is it possible you are getting an attack abuse from the Internet that is attacking your web server and it is consuming all networking resources? 17:04:42 Perhaps a SLOWLORIS attack? https://en.wikipedia.org/wiki/Slowloris_(computer_security) 17:04:43 Title: Slowloris (computer security) - Wikipedia 17:05:07 no, it's not an attack 17:05:10 I have been seeing those hitting my servers on and off rather routinely. 17:05:23 it's the apache/php processes that keep on taking resources 17:05:46 What limits have you set on those processes? Perhaps reduce those further? 17:07:24 what do you mean by limits ? 17:07:45 I am always tuning MaxRequestWorkers to prevent the network from overwhelming my web servers with abuse. 17:08:21 Especially if it is using PHP which consumes significant memory for every interpreter module. 17:09:59 I'll be honest here and admit that my Apache+PHP servers are all running Linux and not FreeBSD. But have plans to change that in the future. But on Debian/Ubuntu /etc/apache2/mods-available/mpm_prefork.conf needs to have MaxRequestWorkers adjusted from the default there of 255 down to something the system has memory for or it will OOM Killer out trivially when the net abuse hits it. 17:10:54 How large that can be varies depending on how much ram the server has available. Maybe 32 is a good limit if it has a lot of ram. Maybe only 12 on a small 1GB Linode VM. (I like Linode btw). 17:11:02 oh, I'm way past that 17:11:50 I always thump on my own servers using "ab" the apache-benchmark tool to stress a machine myself. Effectively attacking myself. But then I can watch the machine and see how it reacts. 17:12:12 How it reacts to the inevitable abuse that will come from the hostile Internet. And then I tune things until it handles it okay. 17:13:06 Honestly the best way to harden Apache from network abuse is to switch to Nginx. Nginx is shrugging off the Slowloris attacks and hardly noticing for example. But Apache I am still tuning around and sometimes needing to restart as needed reactively. 17:13:45 These servers are sitting behind CloudFlare -> Nginx -> Apache 17:13:52 with mpm_event running 17:14:13 they aren't hitting any of the limits from Apache workers nor php-fmp 17:14:16 *fpm 17:14:41 CloudFlare will protect you against a lot of these types of problems, probably should protect against Slowloris completely. But not all of them. Won't protect against MaxRequestWorkers with PHP being too high for example. You might still be running out of memory there. 17:15:14 Good to hear you are using php-fpm. FTW! 17:15:41 That's a different set of tuning needed there though. Can still be an issue. I would still look at it. 17:16:38 This is where I like "htop" and the bar graph of memory use at the top. There are other tools too and some others might be better. Maybe others here will suggest their favorite way of monitoring memory resource use? 17:16:55 I would keep an htop running in a terminal and keep an eye on it when it is failing and see what that is showing. 17:17:55 I would implement system trend monitoring such as with Munin or Cacti or Grafana monitoring and see if you see bad things happening. 17:18:49 I am still using the old Munin which I am happy with but all of the cool kids are using Grafana today. 17:19:01 alright, I've installed htop 17:19:08 maybe that will shed a bit more light 17:19:55 That bar graph of memory at the top is why I like it. Green is userland and then other colors to the right are buffer cache and other things. If you see the green bar graph consume all of the memory that is a bad thing. 17:20:15 do you have access to the hypervisor, or is it a managed vm ? 17:20:26 in any case: do you have access to a serial console ? 17:21:10 it's my hardware top to bottom, I have access to everything 17:21:23 there are no other messages in the console 17:22:09 when the issue starts happening, it's so badly frozen, I can't even do a reset from the hypervisor 17:22:20 esxi prints: failed to reset, lol 17:22:34 The hypervisor is still responsive to you though? 17:22:37 yes 17:22:49 this is 13.2 btw 17:22:52 13.2-p8 I think 17:26:26 it doesn't seem to be an exhaustion of resources though, it's as if something locks up 17:41:15 Since your hypervisor system is still responsive that would lead me to believe that it is not a hardware problem but something in software. I guess that's good. Because then the solution would be a software solution too. Good luck! 19:41:33 Significant pkg upgrade day today! My desktop wants 263 upgraded with 75 reinstalled, plus a ghostscript upgrade from 9 to 10 removing and adding a few. All looks okay. 19:42:32 My servers generally want a more modest 27+22 today. 19:43:35 Most of that on the servers is for perl module packages too. So actually pretty tame there. 19:56:05 damn. sucking up all that bandwidth should be fun! 19:57:53 It's unfortunate that most of the bandwidth is packages telling me the port has no maintainer and asking us to please help. :-( 20:19:37 rwp: I feel very happy you. It's always a pleasant feeling seeing numerious updates :) 20:20:10 *happy for you 22:09:01 i have built the drm-510-kmod from ports, and have done `kldload radeonkms`; i have made a xorg.conf file which refers to it. when i startx i get. Failed to load module (module does not exist) No drivers available, no screens found 22:53:44 is it good practice to set 600 on any files .bashrc sources? also is it good to set 700 ~ to make our whole home dirs private? 22:58:02 alepzi: who you sharing that computer with? 22:59:23 but does that even matter? because what if it's just me, but then some other account gets hacked into somehow 22:59:43 where do i put firmware so that the module which requires it will find it? 22:59:49 i guess i don't see the benefit to ever leave ~ open to reading 23:00:00 I'm guessing random semi-trusted daemons / programs. I do 700 on home as a precaution. 23:00:17 ah ok so i'm not crazy 23:00:29 I'm guessing they are a potential threat* 23:00:34 do you also set 600 for files bashrc sources? or just 700 on ~ and that's it 23:00:48 However, it won't help for untruested programs running as your user. 23:01:00 I do not use bash, so no. 23:02:05 which sh do you use dautor? 23:03:22 I use fish for interactive shell. 23:03:54 back in the old days, people couldn't finger you and read your .project/.plan if your ~ was 700, but you're probably not giving anything up now 23:04:49 Yeah... It's probably a default that should change one day to a more secure option. 23:07:32 i am going down the crazy rabbit hole of ports and have a question.. i am going to be testing this out in a bastille jail first and once i got this operational.. is there a recommended approach to move to host? i figure backups are good.. then is there like a handbook entry that discusses this in more detail 23:11:32 mfisher: yea i feel like there should be a better way for ppl to share with other users and daemons than having ~ be group or world read 23:16:30 like /var/share/alepzi 23:33:30 eww pf what a pain 23:33:36 this is a complex beast 23:35:28 anyone have exposure to bastille (container manager) and be able to have the host.. work as normal and "lockdown" the constainers? https://docs.bastillebsd.org/en/latest/chapters/networking.html this documentation explains locking down host and container.. down to JUST ssh.. but then basically i can not use anything.. like smb 23:35:30 Title: Network Requirements — Bastille 0.10.20231125-beta documentation 23:59:00 I'd interpret that as just as an example of how you can set up SSH for per-container direct access, not instruction that you can't expose containerized services using other firewall configuration