00:38:12 Anybody ever start up smartos on one of these Xeon phi full systems? https://www.ebay.com/itm/185584398452 00:38:41 (Note, expensive old HW, not gonna buy, but interesting) 00:39:50 I think it will work, but I've never tried it myself. 13:36:38 danmcd: to confirm this is the platform that I should add from the latest release right? https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/smartos.html#20221215T000744Z 14:42:36 also, I never moved our us-east- HNs to pool booting, do you have that tutorial somewhere or the instructions Think the HNs were on an older platform when I went to do it last time 15:35:39 Yes it's the right release. 15:37:02 As for a tutorial, I don't know if we have an explicit tutorial (maybe @bahamat published one?), but I can tell you that: 15:37:16 1.) I recommend a dedicate boot pool (not zones) for HNs. 15:38:30 2.) It's very important to know if your HN is BIOS booting or EFI booting. If EFI booting, the bootable pool must have an ESP on it (i.e. created with `zpool create -B...`. 15:39:04 3.) The bootable pool must be a simple, single-vdev one (single-disk, mirror, raidz... no slog or l2arc). 15:39:19 I recommend a dedicate boot pool (not zones) for HNs. what is the fear here? 15:40:00 The fear is that most production nodes have multi vdevs (slog or L2ARC) so they aren't bootable. 15:40:11 I may be conflating HN disk layout with CN disk layout, however. 15:40:35 Also most pools were created WITHOUT -B meaning no ESP, meaning no way to migrate to EFI booting. 15:40:58 ahhh I think we went through that 15:41:05 Went through what? 15:41:19 creating a new usb stick 15:41:26 and porting over so it could boot efi * 15:41:26 Oh... to be a loader one? 15:41:28 yea 15:41:52 Ahh, thanks. One CANNOT piadm activate unless the USB key is loader. 15:42:08 MY concern isn't with the USB key having loader (it MUST or else you can't migrate). 15:42:29 My concern is with the pool-to-be-blessed-as-bootable having an ESP (EFI System Partition) on it for EFI booting. 15:44:19 Were I deploying an HN in production myself, I'd Mirror a pair of larger SSDs for `zones` , and mirror a pair of smaller SSDs for `bootpool`. Basically what I have on Kebecloud's HN but mirrored-for-redundancy. 15:44:45 I'm neither subtle nor clever in this regard, I know. 15:44:49 Gotcha 15:44:54 we just have 1 big raidz2 15:45:21 https://gist.github.com/Smithx10/febbbdae4603531259c75a1949d2e140 15:46:10 In theory that's a bootable pool, BUT (and it's a big BUT) if that pool was created w/o `zpool create -B` you're stuck only being able to BIOS boot, not EFI boot. 15:47:14 is that a zone attr? 15:47:21 errr Pool attribute* 15:48:14 No. 15:48:22 But you can invoke: prtvtoc /dev/rdsk/ 15:48:29 to see if it looks like it has an ESP. 15:48:41 Paste coming for what one with an ESP looks like... 15:49:31 https://gist.github.com/danmcd/d0bc2ad56805e11059b9e29f0a608dc8 15:49:55 See how partition 0 is 256MB and starts at sector 256 ? 15:50:26 THAT is the ESP (which is a DOS filesystem you can inspect with `mount -F pcfs...`). 15:50:40 Sectors 0-255 are the MBR for BIOS. 15:54:14 I think headnode layout is always mirror. It doesn't prompt you for the layout on a headnode, only on standalone SmartOS. 15:54:34 And for other compute nodes you can specify at setup time. 16:00:37 @Smithx10 added a without-ESP example. 16:00:42 Where does sparc/solaris load the kernel image from? 16:01:16 Or more accurately, whatever the system firmware was called on sparc 16:09:43 It's been a LONG time since I dealt with SPARC. I believe with a ZFS pool SPARC too loads from /platform/sun4/sparcv9/. 16:10:07 @copec you probably wanna ask that on #illumos and someone with more SPARC experience (like ptribble) can answer better.