01:46:04 /quit 03:34:12 ketas: ketas 03:41:17 mia cpet came 03:41:36 ketas: hi 03:48:05 ketas: not mia i just dont like libera 03:48:23 ketas: the whole # for every chat topic upsets me 03:56:09 a what 03:56:30 #freebsd has one # only :p 03:57:57 yea but if we start talk talking about purple bunnies people get upset 03:58:09 and redirect to #freebsd-purple-bunnies 03:59:31 who colored the bunnies anyway 03:59:38 some asshole 03:59:43 special easter ones? 03:59:45 named ketas 03:59:57 is it like bikeshed 03:59:57 ketas: msg 04:00:17 yourve been summoned 04:03:44 ˜/43 04:06:33 44 11:28:01 does FreeBSD bootloader only show the Boot Environments option if there are multiple boot environments? I did root on ZFS, and bectl is showing default and so forth, but I wasn't seeing Boot Environments in the bootloader 11:32:13 ...nevermind, just realized that the first freebsd-update takes cares of this 11:34:00 hodapp: yep 11:34:47 don't rely on freebsd-update, create BE's on your own 11:35:12 perhaps we can add command switch to freebsd-update to skip BE creation 11:35:32 the plan was to create them on my own (going to try to clone an existing system into this), but first I wanted to verify that the boot loader handled them at all 11:36:13 yes, it works fine 11:37:10 bectl create newbename - that's all 11:37:17 no need to clone anything 11:37:53 I meant "clone" in the more generic sense, not as in ZFS clone - I've a FreeBSD installation outside of this ZFS pool, and I'm aiming to get it inside of one 11:38:44 that's why I use FreeBSD: it has seemless BE support.... but everything else is failrly lagged behind ;-p 11:39:09 BEs are ZFS thing 11:39:20 yes, I know that much 11:40:29 if you want to create new BE from scratch, let's say install 14.3 into one BE and CURRENT into another one, it's supported, but you have do it by hand 11:40:48 have to go, bbl 11:40:51 later 12:01:50 good morning 14:18:18 hm, so if I understand right how this works, I could: boot to my shiny new root-on-ZFS install, make a new boot environment, mount all the FSes from that, cp/untar my existing FreeBSD install from the other drive over top of this 14:18:35 and this would be relatively non-destructive since my original BE would still be around, in case I mangle something in the copy 15:53:30 hodapp: That's doable if you maintain the bootloader. Sure. 15:56:39 ek: what's involved in maintaining the bootloader? I'm not as familiar with this on the FreeBSD side 15:57:40 Just keep the bootloader in tact on the new zfs-on-root install. Don't overwrite that, and you should be fine. 15:58:59 Or, snapshot it to be safe and roll back if it gets goofy. 16:01:15 Yeah, planning on snapshotting regardless until I have things kosher 16:01:38 I don't yet have a good understanding of what resides in the filesystem, and what is in ZFS settings/metadata 16:03:35 There's a lot of information out there if you're curious. At the moment, I wouldn't worry too much about metadata. 16:03:56 Just get your datasets backed up and transferred and you should be all set. 16:04:15 Settings you can take a look at using "zfs get all path/to/dataset" 16:04:16 I meant as far as what metadata pertains to booting/environments, e.g. I know there is bootfs property 16:04:39 Ah. Gotcha;. 16:05:36 It's not terribly involved (for the most part.) Maybe take a peek at https://wiki.freebsd.org/BootEnvironments 16:09:57 Hey, I've got a bit of an interesting network issue and I lack experience with this. I've got a system on OVH, and its networking is pretty simple. I've got a bridge and I've got a jail on this bridge, and I've got a VNET jail using epairs on it. The a side of the epair is on the bridge, and the b side is in the jail. Now the complication: I've got this random floating IP, and they say "it's attached to 16:10:03 the host, so use the host's gateway". I can't set that as the default route inside the jail until I can reach it, so I route add -host thatgateway -iface my internal epair, the b side, since that's what I can say. 16:11:20 I'll grab a pcap in a while when I get the chance, but I'm curious if there's some reasonable way to structure this. 16:14:40 Alright, so, pcap on the bridge, I'm seeing a bunch of: ARP, Request who-has tell , length 28 16:14:54 hodapp: you can create new BE with "bectl create newbe", it will be clone of the runninb be or begin with empty dataset: 1. create new ZFS dataset zroot/ROOT/newbe 2. set mountpoint=/ and canmount=noauto for this dataset 3. populate it with files 3. reboot and enjoy 16:15:54 mzar: just cloning from running should already set up the various mount points, right? 16:16:24 bectl create clones and promotes this dataset to be potential rootfs 16:16:35 that 16:16:40 ...that's what I mean 16:16:48 thare are no really any mounpoints except / 16:16:59 BE is usually plain / 16:17:25 thought it had /usr, /usr/local, /var... 16:17:41 so they wont't be cloned 16:18:03 uness exist on root dataset 16:18:06 thought -r did that 16:19:07 I am not using -r but perhaps it's for people who have nested something inside root dataset 16:19:30 like zroot/ROOT/default/anotherdataset 16:20:19 do you have /usr on different dataset ? 16:20:47 I did default root-on-ZFS setup, but not booted into it now so can't look 16:21:09 let's say homes and for example zroot/var/log can be shared between BEs 16:22:04 but I'm just going off of what bectl man page says with 'Boot Environment Structures' 16:22:13 ha.. you have to check it, learn everything from hier(7), do some experimentation and you will become expert 16:24:42 no worries, our community is strong, we are still runnig FreeBSD since ZFS is tightly integrated with this and appreciate pjd@ work on ZFS implementation, tsoome@ work on boot loader enhancemtents and kevans@ work on scripting BE booting with lua 16:25:12 did it I miss anything? please correct me 16:55:25 the default root-on-ZFS setup is built in a way that most of the system is shared between BEs 16:55:42 post update to 14.2, trying to pkg update yields "pkg: Setting ABI requires setting OSVERSION, guessing the OSVERSION as: 1400000" 16:56:02 freebsd-version -kru gives 14.2-RELEASE-p1, 14.2-RELEASE-p1, 14.2-RELEASE-p3 16:56:30 uname -UK gives "1402000 1402000" 16:57:00 How do I get it to set OSVERSION according to what it really is 17:07:36 sysctl kern.version: 17:25:58 well, only the major version matters for pkgs ABI, so the gussed value should be ok-ish th 18:10:47 hey, is it possible to convert a mirror vdev (2 disks) into 2 vdev mirrored 2 disks? 18:11:59 today I have 2 disks. they are part of a pool, single vdev. 18:12:43 I have additional 2 disks (C and D). I would like to have C and D as a single vdev mirror, to be part of that pool 18:13:25 ending up with a pool with 2 vdevs (each vdev being a mirror =A+B and C+D) 18:14:05 is there a way to make that expansion (add new vdev) on the fly? 18:22:46 cybercrypto: Absolutely. 18:22:51 Oops. 18:24:58 cybercrypto: Absolutely can be done on the fly. 18:30:26 cybercrypto: so you are starting from mirrored zpool and want to have pool consisting of two mirrored vdevs ? 18:33:30 in either way, it can be done on the fly, like ek said 18:47:10 eki: I lost connection. back now. 18:47:32 eki: I certainly missed your messages. 18:49:05 cybercrypto: I only replied that you can certainly add additional vdevs on the fly to add a secondary (or additional) disk mirror. 18:51:12 ek: to realize that setup I would simply use the zpool add mypoolname newdiskc newdiskd? 18:53:14 cybercrypto: I suppose it depends. Are you trying to add a new 2-disk mirror to the same zpool? 18:53:41 I didn't completely follow exactly what you were requesting. I just know there isn't much you can't do on the fly with ZFS. 18:55:00 Oh, yes. You are adding to the same pool. 18:55:39 It was weather or not you wanted all 4 disks to be mirrors or just A+B (as vdev) and C+D (as vdev) which I wasn't sure. 18:57:06 To just create an additional 2-disk mirror for the two new disks in the same zpool, you can use: zpool add 18:57:19 Ex: zpool add poolname mirror disk1 disk2 19:06:16 it depends, one can have in a poo1 either 1. a morrior of two vdevs consisting of two drives each or 2. two vdevs consisting of 2 mirrored drives each, the capacity will be the same, the latter is more common setup, described above by ek 19:11:23 mzar: thanks. I am not concerned about capacity. I will use mirror vdev with 3 disks each. (each vdev have fault tolerancy of 2 any 2 disks). My question is related on expansion of that pool using new disks. 19:12:57 ek: yes, thats is what I am trying to clarify. 19:14:09 cybercrypto: Okay. I wasn't sure what you meant by "expansion of the pool." 19:14:25 But, if you just want to add a secondary mirror, it's pretty straight forward. 19:14:56 mypool today = 3 disks mirrored (single vdev) 19:15:06 Keep in mind that the secondary mirror will not be a mirror of the first mirror. Just a new set of disks that will mirror each other. 19:16:18 the pool I am trying to achieve = vdev-mirror-0 (3disks) + vdev-mirror-1 (new 3 disks) 19:16:50 Okay. Then "zpool add mypool mirror disk1 disk2 disk3" is what you're looking for. 19:18:15 ek: yes. thats what I would think how to realize that setup. 19:19:21 Excellent. Have fun! 19:19:36 thank you all, cheers 19:20:58 cybercrypto: Sure thing. 19:21:24 And, the beauty of ZFS is, if that setup turns out to not be what you want, you can destroy and try something else. All on the fly. 19:21:54 So long as you don't mess up your current mirror setup, you can really break anything setting up the new disks. 19:22:04 s/can/can't/ 19:25:44 yay ZFS ~(^o^)~ 19:26:57 badkat: indeed :) 19:51:47 cybercrypto: please don't hesitate to read manuals, this excellent FreeBSD handbook chapter by A. Jude https://docs.freebsd.org/en/books/handbook/zfs/ and do some experimentation 19:52:48 cybercrypto: TL;DR: truncate -s 10G drive{1,2,3,4} && zpool create tank mirror ./drive1 ./drive2 mirror ./drive3 ./drive4 19:53:06 this way you will learn a lot 19:53:47 test, do experiment, break testing setups, and you will learn from this 19:56:18 cybercrypto: echoing mzar, and MWL's and Jude's "Mastery: ZFS" and "Mastery: Advanced ZFS" books supplement above-mentioned handbook part well if you want to dive deeper. 20:02:09 cybercrypto: That said, these were written back when FreeBSD used ZFS stack ported from OpenSolaris and before FreeBSD switched to OpenZFS (or implementing parts of ZoL? not sure here tbh) but I believe that all the content from these books applies. It's mostly high-level and userland commands stuff which has not changed. 20:04:47 cybercrypto: ZFS is flexible, but one tip: use "zpool labelclear" when moving drives between vdevs, mirrors and pools 20:12:33 might as well zero it out, who knows if i still works 20:12:45 then you would 20:14:08 if drive was correctly removed/detach it shouldn't be necessary 20:19:21 mzar: You don't really need a drive to be "correctly detached" and a proper shutdown. ZFS is self-repairable. No need for `fsck`. Importing filesystems between different OSs works flawlessly. Back when FreeBSD had extremely shitty support for my WiFi card (AX200 chip) with only 2.4GHz support, I used to dual-boot my ThinkPad with FreeBSD and Debian on a 2TB NVMe. I had a small UFS partition for 20:19:23 FreeBSD, EXT4 for Debian and kept my data on a large ZFS. Whichever OS I used I was just importing the ZFS stuff and it "just worked". The single downside was that there was no ZFS-native encryption and I was unable to use FreeBSD's geli nor Linux' LVM. 20:19:27 that man that I watch yt and I see this black filth fucking black should be kept as slaves as they are not allowed to speak in public bro I feel so sorry nazis diddnt win ww2 and put these all where they brelong in the stables or crematories. Why do we feed these filthy fucking niggers belong to jail instead of just gassing them. Worthelss as their kind are booming uselesss babies and commiting crimes. 20:20:24 Only thing worse than a nigger is a nigger defender 20:20:56 regis: it was about "zpool labelclear" not shutdown 20:22:10 psionic: are you a FreeBSD user ? 20:27:54 mzar: Got it. I read it along "unexpected power loss". That said - ZFS is resilient in such cases. I've worked with large-ish UFS arrays years ago, where a power loss resulted in 2-3 hours of customer support responding to customers with apologies and asking ops whether we can "just cancel this fsck thing". Sadly, higher-ups there made such decision a few times and it didn't go well. They got a 20:27:56 quick fix and customer support got "weird tickets" shortly after. 20:28:14 Please poke staff in #libera-hotline if they return with that behavior. 20:28:30 Bahhumbug: about ZFS ? 20:28:34 Bahhumbug: I got it 20:29:11 er. fuck 20:29:15 *lol* 20:29:17 Bahhumbug: The stats thing in #libera's topic didn't come back with anyone. 20:29:52 regis: Sorry, tab fail 20:31:56 mason: To be blunt we're rarely on stats p. 20:32:01 regis: ZFS works like hell, but to make it even more resilent you need to run zfsd(8), do you ? 20:32:30 mzar: zfsd just does reporting 20:32:45 gAy_Dragon: Ah, you did this? Not a mechanism based on what I quoted on priv sent to mzar? 20:33:22 it was not, we dont actively scan PMs for most things (generally just dox information) 20:33:53 it was me being a bit tired, missing someone else did something, and apparently failing to read 20:34:18 This is 7th Monday, also known as Monday Eve. 20:35:55 has anything bad happened here ? regis: was your connection affected ? 20:36:21 mzar: nah, just an unexpected reconnect ;) 20:37:04 ha.. klined, nice 20:55:56 mzar: Back to the topic at hand: I'm using ZFS as "ZFS on root" since became a viable option on FreeBSD 9 (CLI setup like it's 1995 and RedHat or 2000 and Gentoo, before ZFS was a thing offered via bsdinstaller). My experience with data loss and overall recoverability is that as long something is written onto the filesystem, it just "is". No need for `sync` and a proper OS shutdown procedure 20:55:58 (which on FreeBSD kinda requires `sync` with UFS). It's wonky with some fast storage used as ZFS SLOG but that's an edge case. 20:56:30 s/bsdinstaller/bsdinstall/ 20:58:38 regis: no worries, same here, I am also addicted to ZFS FreeBSD user and former RedHat user 21:06:12 mzar: I've switched a small-ish hosting company from keeping user data on UFS to ZFS. All this stuff was still rsync'd to backups sporting Debians with DRBD (I don't know it well but they appeared to be certain about having snapshots with their disaster recovery plan). Long story short: it went flawlessly. And whenever they had a Samsung HDD go nuts and had syslogd flooded with HDD CRC errors or 21:06:14 whatnot, they pulled it live, replaced, and only then just replaced in via `zpool` commands. Much less hassle than their earlier `gmirror` setup, and in case if the system would die for any reason (usually a hardware failure), there's no hours-long `fsck` after you boot it back up. 21:06:41 regis: neat, unfortunately, FreeBSD is still a bit lagged behind 21:07:51 mzar: I only joined this company due to being compatible with and knowledgeable with OS they were using. 21:08:06 mzar: That was some 14 years ago. 21:08:42 can one still find and apply for such a job now ? 21:11:04 probably not, skilled devs are transitioning to better paid jobs, can AI fill this gap contributing to FreeBSD project ? 21:13:54 if you've not written it, why should we read it? 21:14:29 who did ? 21:14:57 mzar: I know a few here in Poland but it's a shit pay and with small businesses get an overload of irrelevant tasks doing which you don't progress. You need to find the time to do the tasks which are relevant to how you want to evolve. Not worth it. Startups stuck as it's usually chaos and any proof of concept becomes a product. Corporations pay shit but you can easily find a place to learn and do 21:14:59 what you want. Choose your poison. 21:15:54 s/stuck/suck/ 21:17:10 regis: i will not drink this hemlock cup 21:23:05 mzar: tl;dr every options destroys in in some way. It's either a burnout at a startup where you need to stay for a year to grab stock options and it's (usually) a year of pure chaos, established small company with shit pay and no street cred unless you do something really incredible there and NDA allows you to brag about it later, or corpo where it's easy to just DM people on Slack/Teams/whatever 21:23:07 to get onboard their project and eventually get noticed if you do something well. 21:36:36 I'm hoping someone can help me with a funny routing problem I'm trying to conquer: https://bpa.st/5MWQ 21:38:20 At home I have vnet jails that have no problem running dhclient and getting an address, but they're on the same network as the host. This OVH "additional IP" is on a whole 'nother /24 but I'm told that if I get packets to my host's default gateway, they'll be handled properly. The trick is that I can't ping the jail from the host, nor the host or the gateway from the jail. 21:51:39 mason: you need to add an address to the bridge on the host and add appropriate routes to your host and your jail 21:52:41 satanist: So both the bridge and the epair in the jail have E.F.G.H? 21:54:06 no you need to have an address on the bridge acting as gateway for your jail 21:54:32 The bridge has an address - A.B.C.D - and that works fine for the host. 21:54:50 I tried setting A.B.C.D as the gateway for the jail, but was stumped. 21:56:25 this might also work, but you still need to add a route to A.B.C.D on your jail and a route to E.F.G.H on your host 21:57:03 mzar: regis: ek: many thanks. 21:57:04 Would the host-side route just be: route add -host E.F.G.H -iface bridge0 then? 21:57:27 And the jail-side route: route add -host A.B.C.D -iface epair0b ? 21:57:45 yes 21:57:57 and then also in the jail: route add -net default A.B.C.D 21:58:05 correct 21:58:06 I'll try it again. Pretty sure that was in there at one point. 21:59:11 don't forget to enable routing on the host with gateway_enable="YES" / net.inet.ip.forwarding=1 21:59:28 I've had that variously on and off as well. Let me set all that up and I'll make a noise 22:00:10 I'm also not sure if this works on freebsd (it does on linux), you might need also add an address to the bridge (which is imho a cleaner setup) 22:00:51 Yeah, I'm kind of thinking I can do this in a fairly straightforward way with namespaces on Linux. But I'm hoping not to give up on getting it to work in FreeBSD. 22:02:27 something like ifconfig bridge0 inet E.F.G.I/32 (might work better when you use the complete /24 on both sides 22:02:32 ) 22:06:48 All of that set, and still something's not quite right. Running pcaps on both sides, and I see packets trying to reach the floating IP through the host, but not getting through. Oddly, the jail's pcap only sees some random multicase traffic. 22:07:52 multicast* 22:08:48 have you your external interface in the bridge? 22:08:57 A.B.C.D, yes. 22:09:07 And the a-side of the epair. 22:09:48 I'm going to do some testing with different addressing and see if I can verify that traffic can flow correctly between host and jail without any real addresses in play. 22:10:26 I feel somewhat like somehow this is fighting me because the floating IP is out of nowhere. My home set-up uses epairs and has no problem, but's a unified address space. 22:10:36 sorry my explanaiton only works with routing not bridging your uplink 22:11:02 hrm 22:11:39 So, home equivalent server, I don't need net.inet.ip.forwarding set, and there's no special routing on the host side to get things to the jails. 22:11:59 I'm using what's essentially https://wiki.freebsd.org/MasonLoringBliss/JailsEpair 22:13:15 yes when you bridge your uplink and your jails alltogether you don't need to set net.inet.ip.forwarding 22:14:43 My understanding is that if things are throwing packets for the floating IP at my host, since that's a bridge, the backing epair will get that traffic and be able to respond. This has to be some oddity with routing. 22:14:44 but doing this might bring other problems, for exaple your uplink might only deliver your traffic to the mac of your nic 22:15:29 Hm, so, does the MAC matter once the bridge gets the packet? 22:16:00 I'm assuming that once the bridge has it, at that point it cares about the IP. 22:16:01 it depends 22:17:26 no the bridge works on layer 2, the ip-address doesn't matter (in most cases) 22:18:27 Do I need to manually inject the jail's MAC into my arp table or something, then? 22:18:38 it depends 22:19:32 Hrm, hrm, work is picking up. I'll dive back into this once my shift is over. (I cover weekends sporadically.) Thank you for your pointers thus far! If I figure it out or get stumped further I'll let you know. 22:19:58 I would recommend to remove the uplink nic from the bridge and use proper routing 22:21:00 Alright, I can certainly try that. 22:21:04 Thank you. 23:28:35 Hooray, weekend coverage is done and I get tomorrow off. I'm going to split the networking and try a pure routing approach now. 23:32:10 Is there an easy way to select what installed kernel gets used? 23:32:27 <-- goes to have a look at the handbook to see if this is covered 23:32:41 SponiX: see man loader.conf, specifically the kernel option 23:37:24 hmm, I installed a "minimal" and "nodebug" kernel with pkgbase yet I don't find them in /boot/ 23:49:14 I'm surprised *how* difficult it is to find any kind of documentation on running pipewire on FreeBSD.