15:48:54 jperkin: changed our tls-cert today and got a cert error while doing pkgin ug (we an rsyned clone of pkgsrc to speet up the updates :-). Might be because the intermediate from GlobalSign changed on 29.01.2024 and it is not yet in the pkgsrc trust store? Will it be included in the next updates? 15:50:53 The link to GlobalSign Intermediates: https://support.globalsign.com/ca-certificates/intermediate-certificates/alphassl-intermediate-certificates 15:51:19 yeh almost certainly, we're at 20231115. 15:55:55 jperkin: Can you say when the next trunk updates will land? 15:57:30 no, but I'll take a look at it tomorrow 15:58:00 in the meantime you should be able to skip the validation with SSL_NO_VERIFY_PEER=1 in your environment 15:58:31 jperkin: Thx a lot for your help! 16:14:31 hello all, I remember that The max vCPU is limited for 32 for Bhyve VM on SmartOS, if I want to create a Bhyve VM with 96 cores on SmartOS, I have to changed the SmartOS source code and re-compile, so my question is where defined the limitation? (I have a machine with 128 core/256 thread (AMD 9754), I need to change it ), a big thanks 16:30:05 You just need to change VM_MAXCPU in usr/src/uts/intel/sys/vmm.h 16:30:44 Has anyone here tried the igc(4D) build on their Intel I225/I226 systems? 16:31:05 (ahh fixed) 16:36:11 andyf: Thank you 16:37:02 tozhu - it's limited because several structures in the kernel are sized based on that, so there is extra memory overhead for every VM. Eventually I imagine that things will be sized dynamically. 16:38:56 With 32 vCPUS, sizeof (struct vm) = 0x49c8 16:39:07 With 64, sizeof (struct vm) = 0x8eb0 16:41:52 Thank you very much, I checked, current is set to 64 by default 16:42:23 I think 64 cores for a VM is okay for most case 16:42:29 thanks again 17:40:11 Is there any tricks to running a PXE / DHCP server in a bhyve virtual machine ? I can see it offer DHCP but my machine isn't accepting and booting 17:40:34 you need allow_dhcp_spoofing 17:40:38 I did that 17:40:58 You also have allow_ip_spoofing? 17:40:59 [DHCP] Offering to boot ac:1f:6b:9e:df:47 17:41:07 yeah, I set all of them allow 17:41:11 in adminui* 17:41:28 all includes mac spoofing? 17:41:38 restricted traffic? 17:41:54 So just to be clear, your dhcp *server* is the one in bhyve? 17:42:25 And when you tcpdump/snoop you can see the dhcp request come in and dhcp offer go out? 17:43:27 yea, it might be something on my server side cause I am seeing DHCP coming in 17:43:32 and the offer go out so it should be working* 17:44:16 Ok, next step, I would snoop from the gz on the physical interface to see if you see it exiting. 17:44:46 If the kernel is dropping it due to some anti-spoof protection then you won't see the packet leave. 17:44:49 Just wanted to make sure there wasnt anything with BHYVE that limits it 17:45:13 It might also be good to use like scapy or something from a running host to send dhcp request packets and see what kind of response you get. 17:46:14 Well, I can't say for certain that there's not. pmooney would be the person who would be able to say for sure. But as far as I know there's nothing inherent in bhyve or viona that will prevent it from being a dhcp server. 17:48:04 Side question: Any reason you're not running the DHCP server in an os zone? 17:48:59 ahhh im dumb https://github.com/danderson/netboot/blob/main/pixiecore/README.booting.md 17:49:18 Pixiecore proxies the DHCP offers 17:50:19 when I ran this in my KVM setup for testing, i didn't realize libvirt was actually doing the dhcp and not netboot* 18:13:40 https://gist.github.com/Smithx10/e930b24ba46cf69ea6fca908ed6e0a98 18:14:27 Looks like I'm sending out an address but the machine isn't getting passed that 19:14:28 What I'm learning right now is 19:14:39 EFI PXE Clients on network cards is really awesome. 19:20:06 i sense.... sarcasm 19:20:08 :) 19:23:25 jbk: you know the pain. 19:28:03 firmware is terrible :) 19:28:37 maybe oxide can sell a openfirmware 100gb NIC 19:29:10 or vendors can have an option to flash to some open firmware 19:29:11 what a dream 19:29:23 If my server would come with NICs that didn't hate me./ 19:42:11 i suppose (if i was really bored and had the time, i might consider it as a lab experiment), I suppose given the programming manual for a card, you could probably write your own, though how to safely flash it and such might be an interesting exercise on its own 19:43:20 i mean, at the end of the day, there's basically a bare bones NIC driver and then the PXE code that uses it... though there might also be some other bits to initialize the hardware that a NIC doesn't need to concern itself with 19:59:00 Mellanox connectX5 even booting off the ipxe.iso im getting "No configuration methods succeeded" really fun 20:06:55 like trying to troubleshoot apache :) my experience has been at least 80% of the time, it never actually tells you why something doesn't work and you just have to fumble around in the dark until it magically does 20:28:59 "denied" 20:29:00 why? 20:29:02 "denied" 20:29:07 *turns logs up to 11 20:29:09 why? 20:29:11 "denied" 20:30:15 pretty much 20:30:38 like there's someone that's running a custom php app that was written for an older version of php 20:30:54 the problem they're running into now though is that the version of openssl is too old to suppor tls1.3 20:31:07 but the version of openssl that support that is too new for the version of apache that's running 20:31:33 and upgrading apache (installed to a separate dir) it refuses to actually run the php code 20:31:42 with no indication of why at all 20:31:46 which is frustrating 20:32:03 (the app is strictly internal, and not reachable from the internet) 20:33:51 servers should come with a way to turn off the 1tb of dram 20:34:13 like give me 8gb so it doesn't take 45 minutes to boot 20:42:47 People awake 20:43:16 Smithx10: Probably some startup check? 20:43:35 Major issue I have is to boot with DAS/HBA, it takes forever 20:43:42 or training DIMMs 20:44:23 It's way faster for me to unplug the SAS cable, boot the machine with internal zones only. Then connect the SAS and do import tank after the system is up and running 20:44:25 at least with EFI it's easier to control the boot order so you don't have 'let's wait 30s for this PXE interface' (repeat for each NIC in the system), now let me try each HBA 20:44:54 granted an old Sun E10K laughs at your puny 45min boot time :P 20:45:12 SuperMicro X10 and X11 mostly 20:45:17 if you enabled full diagnostics (granted that was generally only done as a post-install burn-in test), it could take > 24 hours to boot 20:45:50 48 hours did not suffice in this setup 20:46:24 Not sure what hangs; good thing I found out I could simply detach external SAS cable to NetApp 4246 in order to have the system come up. 20:58:07 nahamu: pretty sure I might be hitting https://github.com/ipxe/ipxe/issues/1091 20:58:34 applied your commit to main and and doing build now to embed into pixiecore 20:58:39 Wish me luck "_" :P 21:00:19 @TyrfingM1olnir how many disks in the DAS? 21:09:40 in the the? :) 21:19:26 hello, i was wondering if it's possible to run a firewall in a zone and have it act as a gateway for a mixture of zones and bhyve vm's? or should i run the packet filter from the hypervisor itself? 21:20:35 if this is a pretty common use-case could someone point me to some recent docs or reference implementations? thanks! 21:21:47 s/hypervisor/global zone/ 21:27:06 You can do both. 21:28:57 I use a native zone as a router & NAT (it could be a FW but I approach that problem differently). It's on OmniOS, so you may need some extra SmartOS magic (forwarding requires some lnk-layer protections to be disabled..) 21:33:24 You can also, if you want a different/more-familiar-admin, use a BHYVE VM (again with some SmartOS link-protections disabled) to run some FW appliance. 21:38:23 ok thanks danmcd! i think i'll need to spend some time experimenting with etherstubs then 21:39:34 i'm looking to replace a freebsd hypervisor where i use pf and nginx for controlling access to private instances. i'm hoping i can do this more cleanly with smartOS. at least that's what i'll find out :) 21:41:03 nahamu: yep.... that was it "_"