danmcd: ping

Hang on...

We have a long-lived open bug on "libc ignores fadvise" but that's it.

I'm currently trying to PXE boot a bhyve VM, the boot file is grubx64.efi, it also downloaded a linux kernel (11M) and now a livefs image (90M) via tftp - which is terrible slow. looking on the host system running a pretty recent OpenIndiana I see that VM being 100% in systemcalls in mpstat, single thread...

transfer rate is around 10kb/s

(on a 10GB LAN)

and I did some speed tests from another server by just using a binary get via tftp on the exact same files, the get finishes in a few seconds

so it's obviously something grubx64.efi doing with bhyve

hey wacki!

Hi Agnar!

and I already map my vnic to bhyve with -s 6:0,virtio-net-viona,bootstrappub0, so it should be fast

Agnar: TFTP is unfortunately extremely inefficient

the way the emulated NIC works you usually end up having to do something expensive like a system call

but out of the VM

jclulow: ok, so this is an expected behaviour

I wouldn't say it's expected per se

Just that I'm not surprised it isn't fantastic

hehe

If it's really only 10KiB/s

ok, gotcha

That's pretty bad haha

yeah, dlstat on the vnic shows 50kb/s every 5 seconds

10 * 1024 = 10240 bytes, divide by 1500 MTU (let's assume) is ~6.82 frames / sec

that's, what, 146msec between frames

If I were a betting man I'd say it's not using interrupts

but just spinning busy looping polling for frames

which at least probably explains the high CPU

But this is all obviously guesswork

I would probably get a packet trace of the TFTP session and look at it to see what it's doing

and yepp, I just wondered since oxide demo'd openshift on their rack and I wondered if they also used UPI installation with PXE for it, or the assisted installer of openshift from disk images

I think we would generally try to do the latter, with the images

PXE booting makes sense for hardware but if you have a better way to get install media onto your systems, physical or virtual, you should do that instead

jclulow: it's not better, as it is not hands-off

It can't be made hands-off?

Through some automation?

the PXE way is hands-off

jclulow: nah, stupid web assistent ;)

heh

Well I'm not that close to the openshift stuff

jclulow: be happy, I get paid for training it ;)

ha

My proximity is intentional :P

hehehe

I ported a Spotify client to OmniOS on the weekend to avoid needing to use a Linux machine, if that's any indication :P

but I do run an OpenShift already in bhyve on OI and people always freak out when I tell them I run their containers in pods on openshift in bhyve in containers on illumos ;)

haha

which runs very well when installed

and the X4-2 just eats around 210W, which is also very good for what it runs

Yeah not bad

can't afford an oxide rack...(yet ;))

(will look at ebay in 10y ;)

oh and by the way...are you a bit familiar with the pptadm and ppt driver for bhyve? I tried to pass through a fc hba into a vm, but ppt always matches all cards. the limited docs say I could specify a path in ppt_matches but that doesn't seem to work...

ha, told grub to load the kernel from http ;)

79MB/s ;)

grub2 tftp blocksize is 1024, thats part of the reason. and the value is built in, so you can not change it from config file.

loader defaults to 1428, fallback on error to 512 and max value is 9008.

[illumos-gate] 16581 nvme cannot hold locks when calling into blkdev -- Robert Mustacchi <rm⊙fo>

[illumos-gate] 15929 d_qactive is a red herring -- Robert Mustacchi <rm⊙fo>

tsoome: yeah, I saw that in snoop...grub is terrible...

I have a nice LCD panel SunRay 3 or 3i ; does anyone have tips on installing it on OmniOS or a LX or OpenSolaris based zone ?

sunray need sunray software (and also firmware)

SUNWsrs

is that open source or will I have to search for it?

We (more specifically Agnar) had a SunRay booted off illumos on our stand at FOSDEM a few years back

thank you

if it works out, I can get a few more, clean, for cheap

so could have 1 homelab server and put everything on it, then put SunRays all over the house

less hassle/duplication of stuff getting installed