00:02:54 <tozhu> danmcd: ping
03:42:44 <danmcd> Hang on...
03:47:56 <danmcd> We have a long-lived open bug on "libc ignores fadvise" but that's it.
15:58:52 <Agnar> I'm currently trying to PXE boot a bhyve VM, the boot file is grubx64.efi, it also downloaded a linux kernel (11M) and now a livefs image (90M) via tftp - which is terrible slow. looking on the host system running a pretty recent OpenIndiana I see that VM being 100% in systemcalls in mpstat, single thread...
15:59:54 <Agnar> transfer rate is around 10kb/s
16:00:17 <Agnar> (on a 10GB LAN)
16:01:09 <Agnar> and I did some speed tests from another server by just using a binary get via tftp on the exact same files, the get finishes in a few seconds
16:01:39 <Agnar> so it's obviously something grubx64.efi doing with bhyve
16:10:23 <Agnar> hey wacki!
16:10:54 <wacki> Hi Agnar!
16:21:04 <Agnar> and I already map my vnic to bhyve with -s 6:0,virtio-net-viona,bootstrappub0, so it should be fast
16:32:27 <jclulow> Agnar: TFTP is unfortunately extremely inefficient
16:33:14 <jclulow> the way the emulated NIC works you usually end up having to do something expensive like a system call
16:33:20 <jclulow> but out of the VM
16:33:52 <Agnar> jclulow: ok, so this is an expected behaviour 
16:34:01 <jclulow> I wouldn't say it's expected per se
16:34:09 <jclulow> Just that I'm not surprised it isn't fantastic
16:34:18 <Agnar> hehe
16:34:20 <jclulow> If it's really only 10KiB/s
16:34:23 <Agnar> ok, gotcha
16:34:26 <jclulow> That's pretty bad haha
16:35:03 <Agnar> yeah, dlstat on the vnic shows 50kb/s every 5 seconds
16:35:07 <jclulow> 10 * 1024 = 10240 bytes, divide by 1500 MTU (let's assume) is ~6.82 frames / sec
16:35:20 <jclulow> that's, what, 146msec between frames
16:36:07 <jclulow> If I were a betting man I'd say it's not using interrupts
16:36:14 <jclulow> but just spinning busy looping polling for frames
16:36:24 <jclulow> which at least probably explains the high CPU
16:36:43 <jclulow> But this is all obviously guesswork
16:37:08 <jclulow> I would probably get a packet trace of the TFTP session and look at it to see what it's doing
16:37:20 <Agnar> and yepp, I just wondered since oxide demo'd openshift on their rack and I wondered if they also used UPI installation with PXE for it, or the assisted installer of openshift from disk images
16:37:37 <jclulow> I think we would generally try to do the latter, with the images
16:37:57 <jclulow> PXE booting makes sense for hardware but if you have a better way to get install media onto your systems, physical or virtual, you should do that instead
16:38:18 <Agnar> jclulow: it's not better, as it is not hands-off
16:38:30 <jclulow> It can't be made hands-off?
16:38:34 <jclulow> Through some automation?
16:38:35 <Agnar> the PXE way is hands-off
16:38:43 <Agnar> jclulow: nah, stupid web assistent ;)
16:38:46 <jclulow> heh
16:38:51 <jclulow> Well I'm not that close to the openshift stuff
16:39:07 <Agnar> jclulow: be happy, I get paid for training it ;)
16:39:13 <jclulow> ha
16:39:21 <jclulow> My proximity is intentional :P
16:39:30 <Agnar> hehehe
16:40:04 <jclulow> I ported a Spotify client to OmniOS on the weekend to avoid needing to use a Linux machine, if that's any indication :P
16:40:21 <Agnar> but I do run an OpenShift already in bhyve on OI and people always freak out when I tell them I run their containers in pods on openshift in bhyve in containers on illumos ;)
16:40:30 <jclulow> haha
16:40:51 <Agnar> which runs very well when installed
16:41:30 <Agnar> and the X4-2 just eats around 210W, which is also very good for what it runs
16:41:39 <jclulow> Yeah not bad
16:42:16 <Agnar> can't afford an oxide rack...(yet ;))
16:42:30 <Agnar> (will look at ebay in 10y ;)
16:44:50 <Agnar> oh and by the way...are you a bit familiar with the pptadm and ppt driver for bhyve? I tried to pass through a fc hba into a vm, but ppt always matches all cards. the limited docs say I could specify a path in ppt_matches but that doesn't seem to work...
17:03:27 <Agnar> ha, told grub to load the kernel from http ;)
17:03:35 <Agnar> 79MB/s ;)
17:44:12 <tsoome_> grub2 tftp blocksize is 1024, thats part of the reason. and the value is built in, so you can not change it from config file.
17:46:14 <tsoome_> loader defaults to 1428, fallback on error to 512 and max value is 9008.
17:54:59 <gitomat> [illumos-gate] 16581 nvme cannot hold locks when calling into blkdev -- Robert Mustacchi <rm⊙fo>
17:54:59 <gitomat> [illumos-gate] 15929 d_qactive is a red herring -- Robert Mustacchi <rm⊙fo>
19:26:04 <Agnar> tsoome: yeah, I saw that in snoop...grub is terrible...
21:01:29 <patrick2> I have a nice LCD panel SunRay 3 or 3i ; does anyone have tips on installing it on OmniOS or a LX or OpenSolaris based zone ?
21:03:34 <otis> sunray need sunray software (and also firmware)
21:03:40 <otis> SUNWsrs
21:05:45 <patrick2> is that open source or will I have to search for it?
21:07:37 <ptribble> We (more specifically Agnar) had a SunRay booted off illumos on our stand at FOSDEM a few years back
21:08:43 <ptribble> https://docs.openindiana.org/handbook/sunray/
21:13:30 <patrick2> thank you
21:13:47 <patrick2> if it works out, I can get a few more, clean, for cheap
21:14:07 <patrick2> so could have 1 homelab server and put everything on it, then put SunRays all over the house
21:14:26 <patrick2> less hassle/duplication of stuff getting installed