release-20220922 has been announced. Folks who serve NFS (zone or global) should make sure their hostname is in name services or else mountd won't start.

maybe soon(ish) the zfs alias stuff will be in too :)

Unlikely...

I talked to both Alex and Josh about it, and superficially it seems easy, but the more you use it, the more lies you need to tell in order for things to work.

well it'd probably need some plumbing for smartos zones

It's like a fractal of edge cases.

but it could be used for the 'data' dataset

But it's not just the dataset. You need to fabricate a pool.

and how often does anything run a zpool command inside a zone?

Plenty often.

for what? the only thing you can do with _zpool_ in a zone is basically 'list' or 'status'

list shows size, compress ratio, all the properties, etc. What to do with them?

status shows device geometry.

Then there's zpool iostat

and why would you be looking at that from within a zone vs. the gz (where none of this applies)

But so many people do.

In JPC we had customers that were monitoring disks, and would open tickets with us if there were errors.

Like, we already knew and were handling it. But they'd open a ticket anyway.

But like what all breaks if we just shut that stuff off inside a zone? No idea, because it's always been there.

if it's a new brand (it's not like it does this automatically -- you have to specify an alias in the zonecfg)..

As in vmadm create brand=not-joyent?

there's already dones for kvm and bhyve.. could call it 'mnxio' :)

(e.g. joyentv2) or such..

Yeah, but if the difference between an mnx brand and joyent brand is that a ton of zfs utilities are broken, nobody will use it.

At any rate, it's extremely unlikely that it'll be in SmartOS first. If it shows up in gate, we'll make it usable in SmartOS.

will a newer platform image show more in 'imgadm avail' or is the list always up-to-date and trying to use images requiring newer platform will just fail?

do you mean 'piadm avail' ?

...platform just fail

imgadm

there are minimum platform requirements for images

Ah.

not sure how it is used

Probably in Triton for scheduling.

well, if a lx-zone requires more support from the platform...=)

jesse_ : afaik 'imgadm avail' gives you a current list - including all available images - regardless of minimum platform requirements - those are only checked while provisioning an instance - you will not be able to provision it if your platform is older than the minimum platform required for that image

so... updating the PI won't help me and I'll have to roll my own lx image (urgh)

expect cursing here about the PI built-in scripts for different distros=)

jesse_: What is it that you're wanting?

something new linux

(not feeling like doing updates to it anytime soon)

Is there a distro called literally "something new"?

probably ubuntu 22.04, as I've been using it in other contexts lately (due to native zfs support)

bahamat, not-centos-not-ubuntu-20.04

ubuntu 22 shouldn't be too hard

I wonder if I have the energy to get almalinux running

that's what I'd most likely run on lx in the long run

I had some problems with ubuntu 21.10, but no idea what they were

I assume I'll find out if 22.04 fixed those=)

how is the lx image build repo nowadays? it didn't do something the last time and it was easier to do all manually

There's two parts to it.

First there's this: github.com/TritonDataCenter/lx-images

That's a fork of OmniOS's LX image repo, but there's currently no changes in ours.

Then there's this: github.com/TritonDataCenter/smartos-lx-img-builder

which takes a tarball produced by the lx-images repo and converts it to a SmartOS image.

it already has ubuntu 22.04, so it'll probably work without much hassle

Yeah, we should be able to just run the conversion and be done.

bahamat, github.com/TritonDataCenter/smartos-lx-img-builder: readme should probably tell in what environment to run it (gz/zone?) and steps to get the target/debug/smartos-lx-img-builder?

I agree.

And since I need to re-figure that out, I'll write it down this time :-)

oh, it's rust=)

IIRC, it works fine in a zone

The main requirement is a delegated dataset.

having img-builder as an img wouldn't be a bad idea?

Maybe.

my previous solution used docker (OCI) images as base and just edited their filesystems

Yeah, I think that's what these actually do as well.

easy way to get plenty of application images without re-doing the dockerfiles

last time I checked omnios did get the container tars from the releases and did the base image properly

my solution was the less 'proper' one, as there was no shared parent of OS image

uh, no, and yes

omnios uses dockerfiles to setup the images=)

bahamat, for smartos-lx-img-builder: a description what it does/adds to the images would be nice

Yeah, I'm going to add that.

"adds joyent service to systemd" etc.

smartos really needs native podman support

(I ditched docker-the-runtime-and-command couple of years ago and after using podman for a while all these dockerfile/smartos image systems are a major pita...)

vmadm will always be the primary way to run instances on SmartOS.

It's necessary for Triton

But we've always wanted to make it easier to run lx containers with it.

jesse_: give this a try:

If that works out for you, I'll publish it.

I haven't tried it at all, I just ran the conversion.

bahamat, not the instace things

bahamat, to build images

*not to instance

basically a little bit better zlogin

Well then podman needs to have a vmadm (or at least zonecfg/adm) backend for running them

yeah, that's why I said "smartos really needs native podman support"=)

What you mean is podman needs native smartos support

It's going to have to be on them to want SmartOS support.

There's no way we can shove it down their throats if they don't want it.

you can "podman create from scratch" that gives you an image with no fs (smartos: /native there?). then just "podman $ctr <any shell command>" to run commands in the context of root-of-that-image without any services running, and "podman mount $ctr" to mount the fs (in smartos just /zones/$ctr/root/)

mostly the second part is thing I miss

Are you saying this works now?

it works in linux with podman

(hence, "smartos really needs native podman support")

I'm seeing myself running more and more bhyve just because the build-an-image tooling is so bad on smartos

So far it doesn't seem like there's anything that SmartOS can do to support podman.

It seems like it's the other way around.

like you said: podman devs won't do anything

so, "smartos really needs native podman support", mentioned here

So I don't really know what you're asking for.

that's probably part of the problem

you don't even see the problem

(that was just a neutral statement)

It's not that I don't see the problem. I don't see how you're suggesting smartos be modified to accommodate podman.

suggesting getting smartos support into podman

How exactly does smartos need to behave differently for that to happen?

it's not 'smartos' development, but then again bhyve wasn't, either?

I don't really think that's the same thing.

smartos devs need to add the support. Possibly have some tuning of being able to run things "in a zone" without the zone being 'running'

it's the same thing, see: bhyve

see: java

see: node

see: ...

Ok, well you're free to send me a pull request any time you want.

...I was expecting this=)

I can't tell if you're asking me to modify smartos or if you're asking me to modify podman.

if I had the time to do all the runtime support on smartos, I'd start with openjdk

...which I've done

and I no longer have the time for

I'm not trying to be obstructionist. Like, let's say I agree wholeheartedly and am ready to sit down and write code...what is it supposed to *do*?

I'm asking podman to work on smartos as image-building tooling

Then I think what you're asking for is for the podman developers to add support for running on SmartOS.

I think the first step would be to get podman on some linux and see how it works and do some exploration to see what I'm even talking about

And like I said, even if I went through all the work of adding SmartOS support to podman, I doubt they'd be willing to merge it just because I asked.

there are (at least parts of) podman that are services that can be added

the build thingy probably less so, it's probably married to linux namespaces to some degree

(I haven't looked)

It would need to add zones as a backend.

looking from other angle, zones backend would be need to be added

That's...exactly what I said.

it's from the opposite direction

you said podman needs to add one, I said smartos needs to add one

Smartos already has a zones back end

not with the interface required for podman

if there even exists one yet

if it doesn't it needs to be added

etc.

So you're suggesting zones be modified or replaced with something that's compatible with linux namespaces for the purpose of running podman on SmartOS?

no

in interface glue in between podman and smartos zones

as +

lx brand zones are already that glue.

they are the _runtime_

But, lx doesn't support nesting.

(and not a really good one at that)

what I'm talking about is actually _making_ the zones

this has always (since docker was released) been the achilles heel of smartos

there is no easy way to make the images

(there is a way, but it's not something that's used easily)

in real-life environments I've seen, there are tens of different containers, when zones act more like the hw/virtual machines the containers are deployed on

(real-life linux/container environments)

Well, an image is a zfs dataset.

So it's just a matter of having a dataset and dumping files onto it.

but the tooling is not there

let's take an example, how will you build and image that is this dockerfile: github.com/TritonDataCenter/lx-imag…lob/master/centos-stream/Dockerfile

that's like 3-4 podman commands

Ok, fine, but modifying the OS so that it behaves like a completely different OS is a tremendous undertaking.

(and yes, the commands _are_ that dockerfile, it's just two different toolings doing the same thing)

it's not about the os

And trying to convince a 3rd party that they should support my OS is also not easy.

because the podman interface is 'instantiate an empty container', 'copy files to a container', and 'run commands like you were in the container'

you're not convincing them

you're doing the work

that's what I call 'adding a native support'

(you being the abstract people developing smartos)

Yes, then I send them a pull request and they say "thanks but no thanks"

then you make the things that they didn't want an interface for support for other platforms

submit that and ship your implementation in the PI

I don't know if you've ever maintained an open source project before, but generally people don't like people who are not part of the project suggesting large new subsystems.

example: you now

yes, I know

You still haven't given any suggestions on how to change SmartOS so that podman just works unmodified.

I've given up on using several oss software because their devs are poondering their own orb instead of looking around what's needed

bahamat, 'left as an exercise for the readers', as in if I did, it'd be in the form of a pull request

All you've said so far is "make it work". Ok, what does that entail?

'I have not looked into it to give you steps'

I'm not your project manager

I can give you input on what is needed or would be goood

Even "looking into it" is going to take days to weeks at minimum.

hours to days

yes

and it would need someone to either to get approval on it or just do it

Sounds pretty simple, then, why not volunteer for it?

but for to that happen, someone has to give the idea of 'this would be really good'

I'd love to, but my first priority would be to get openjdk to run again

ptribble has taken care of that. We've got working openjdk17 builds.

and the easiest way is to just ditch smartos and use the time to make podman's network stack work even slightly

bahamat, does it have ZGC?

if it doesn't, it's pretty much useless for production use

because a bhyve with linux openjdk will dance around it

(no, it's not easy, did dig into it)

If it's a native part of openjdk, then I think it probably works, but I wasn't involved with that work so I can't say for certain.

it's native part

but there is only win and linux implementations of it

and the whole build system will fight you to do #ifdef implementation

if you copy it, you'll on the hook to manually do every single small commit yourself

so, I know what I'm talking about

podman is just some node(? possibly go?) interfaces implemented

not €"#%€&%//%&€%# memory mappings in-a-hurry magicks

It's go

(and the platform-specific parts of ZGC are not in-a-hurry afaik)

but it was more dramatic to claim that=)

(and most of the openjdk problems are because of how they use c++ and their build system, but there's not much one can do about that)

bahamat, ah, if you have podman-angst, try adding buildah support=)

(it's the build part podman uses)

bahamat, and only now I realize what I should have been using all this time... Obviously I use buildah commands for everything, but I just keep copy-pasting the prefix of the shell lines)

(but podman uses internally buildah to run dockerfiles etc.)

to add to the trinity, adding imgadm support to skopeo...

(podman/buildah/skopeo is the redhat response to docker going pay-to-run-production, afaik)

obviously the lines above were triggered of me copy-pasting lines from existing buildah scripts and realizing the prefix is buildah, not podman...

lists.freebsd.org/archives/freebsd-jail/2022-May/000129.html <- there seems to be an effort to port podman and buildah to freebsd...

it'll still probably need smartos-specific massaging

In the vmadm man page for "cpu_cap" it says "percentage of a single CPU that can be used by the VM". If a CN has simultaneous multi-threading enabled, does "CPU" refer to a CPU core or a thread on the core?

not sure if the zone model can handle 'run command in zone context without really starting up the zone'

blackwood821, afaik, thread

jesse_: That's what I was assuming, hoping someone can confirm it

I _think_ I've seen that with zones and bhyves with high load

(monitoring from gz)

Is there anything in the global I can run to confirm that for a particular zones?

zone*

prstat -Z?

(and see how much a much-loaded zone gets)

on an idle host... I don't know

jesse_: `prstat -Z -z <zone_uuid>` gives me one entry for the zone so would the percentage in the "CPU" column be the percentage of CPU of all the CPUs on the system?

just "prstat -Z" will give you all load on the system

may help or not help

but what I remember looking at the numbers is 100% per thread

bahamat, to get back to podman, eventhough that's not what i'm looking for, there are runc and crun implementations of the interface to actually run the container; implement either or see how to add your own=))

(vmadm will do for my needs, most likely, eventhough it sucks like docker in setting bootup-sequence)

jesse_: Ok thanks. docs.tritondatacenter.com/private-cloud/troubleshooting/cpu-usage seems to agree: "The percentage is the total across all CPUs (psrinfo). So, a value of 200 is equivalent to 2 virtual CPUs (a virtual CPU is either a core or a hyper-thread)."

blackwood821, good that my memory agrees =)

Your memory serves you well

runc is just a very thin wrapper around making a bunch of system calls.

What you're actually proposing is adding linux namespaces to illumos.

no, what I'm proposing is lying about them in relative terms like with lx

and adding support in relation too that

s/too/to/