lol is this like when zfs tells you file <0x0> is corrupted, but everything on the dataset still seems to work and you have no idea what it's talking about

current Hipster kernel seems broken (i.imgur.com/IzcK8IT.png) - known?

My last good beadm seems 9/28.

arekinath: lol, Ive got some pools with corrupt blk devices.... and the users never say a thing lol

Good afternoon. Guten Nachmittag. Bon après-midi. Добрый день.

trn: that looks like your boot archive is hosed, you should be able to boot an old BE, and rebuild a boot archive in a new be by mounting that BE and doing bootadm update-archive -R /path/to/new/be/mount

richlowe: I've got no problem booting any old BE - when I update it again though, and nothing special, just a "simple" `pkg update`, I end up in the same situation.

richlowe: I can actually boot into an older BE without any problem, but the problem happens again just doing a standard update with IPS (`pkg update`, nothing special) and trying to boot into the new BE.

right, try updating the boot archive manually and see if that fixes it for that BE

if so, you need to work out why doing it automatically isn't working properly.

I've not tried to look into it further.

Apparently, it's happening to just me :)

so the bootadm command I gave you should rebuild it from another BE

you can add '-v' to make it verbose, if that helps

but mysterious ELF complaints from early boot like that are almost always a corrupt boot-archive

it's very difficult to _build_ an object that broken, much easier to have the boot-archive be truncated

I'll give it a shot. I've never had any major problems with OI lately. I'm just wondering what could have changed.

I usually update my OI images ever 2 weeks or so (I use them for CI/CD).

richlowe: Interestingly, I see an ATA error (informational, read error) in my log. But that's a bit odd.

(it's actually a disk is that is being served to the VM over iSCSI)

so perhaps the problem is something in virtio drivers, or further up the chain.

richlowe: I just tried your suggestion of bootadm update-archive before booting into the new be, same error.

hm

then I don't know what's going on

I don't think toomas did anything to the loader or any of that recently

I'm going to try to verify all the packages first, see if there is any other corruption or something odd.

toasterson: is this familiar?

It's trivial for me to bring up a completely new image as well, so it's not a showstopper. But I'd feel like that's admitting defeat.

I'll also actually look at the kernel file size and such, and try bootadm verbose too, after I verify the installed packages and filesystems.

Thanks for the suggestions.

All packages verified without errors.

I'm doing an update and then I'll see what bootadm verbose says, etc.

richlowe: Everything seems to be mostly normal. i.imgur.com/NFYv5MP.png

Sorry for the images, using a SPICE console.

that looks normal yes, except it not having had the etc/system.d file

I don't know, sorry :\

Well, well.

It booted into the new be fine this time.

I'm at a loss to explain it, because I didn't do anything differently.

And the verification didn't need to fix anything. I'll run pkg fix again now, and I'll scrub the whole pool.

If there is anything I can provide that might be helpful, let me know. I'll keep that particular older be around for awhile.

hm, hash with length 0x0?

No errors after scrubbing the pool.

Who can I complain to that OI's gdb requires python37? :)

I hate having three Python-3's installed...

Also, speaking of weirdness. I haven't tested it lately, but strip seems to totally destroy go binaries.

But Go in general seems to work really well on illumos these days, I'm really pleased with it.

looks like someone just did: illumos.org/issues/15091

→ BUG 15091: strip(1) rips go binaries apart (New)

oh dear XD

Is OmniOS well supported?

I don't know the status of it, but quite a few of the principals hang around here.

I had a problem with libuv and andyf here had debugged it and fixed it before I even had a chance to look into it. :)

I use OmniOS on my VPS that uses illumos - so I'd hope so.

trn: me :) gdb probably needs a rebuild then. Let me have a look

it looks like go binaries are incorrect, but I'm not sure that's why we destroy them

and I'm looking at older binaries, it's possible newer binaries are incorrect-er

one for nahamu to fix for us in go, and then see if we still screw it

seoul: I use OmniOS LTS r151038 for a number of production services. Security patches are promptly available and I've not had things break because of upgrades.

I've been very happy with it

Sounds great, thank you.

toasterson: Ah, thanks. I saw that migration from python-35 is in progress. But besides some system components that are using 3.5, the only item I have that has a Python 3.7 dependency is gdb.

Everything else is 3.9

trn: yeah looks like gdb had an update when we where working with python 3.7 and before 3.9