It boots from disk again, after a... zpool destroy; zpool create ... ; all to carve out some space...

Wait is the slog for a second pool on other disks

yes

Boot, rpool, zil, are all on the NVMe device (small-ish SSD)

Spinning rust for (mostly) backups etc. Favoring low power and cost over perforance.

Fair enough

Builds of illumos gate on the NVMe SSD are faster than my old system (down from about 1h30m to about 35m). And at less than half the power and in nearly complete silence.

That's neat. What kind of CPU does it have?

The setup I wrote about on developers.

I think I figured out SoL. Looks like it's just the baud rate. needs to be 115200 starting in loader, staying that way into the OS ec.

I think I figured out SoL. Looks like it's just the baud rate. needs to be 115200 starting in loader, staying that way into the OS etc.

(Oh yeah, no typo corrections here:)

Could someone please write up how to arrange for a getty to run on all of: wscons, ttya, ttyb (regardless of which is the "console"). I think SmartOS did that.

Would be nice to have the service instances defined (at least) even if disabled by default.

emailed that.

I replied to your mail

hrmm.. macOS still doesn't seem to handle the restart of smb/server too well :(

i had been testing my smb+mDNS stuff just with a VM on my home server.. but decided to build it and actually run my home server w/ it (serving SMB out of a zone).. and switched over from afp

aside from that, seems to work minus it only seems to check/advertise the time machine flag on startup.. so gotta fix that

Does anyone who uses vanilla wireguard and experience reviewing bash have time to review WireGuard/wireguard-tools #17/files ?

I'm guessing there's a bug or two lurking in there.

FYI, @jbk, for time machine disks, I use the simple work around of "connect to server" in Finder first, after which the time machine "select disk..." button will offer to use what you mounted in Finder.

That method does not require mdns or bonjour or whatever for locating. That stuff is handy, but not absolutely necessary.

jbk this would be neat.

I have been using time machine on top of illumos smb for years now:)

yeah, this is just a 'quality of life' thing..

it's probably more useful for small/home use, but still just a nice bit of integration

Any SMB share on an illumos system can function as a Time Machine backup disk by explicitly providing the host/IP and name of the share in the client Time Machine configuration. This setting is purely informational to aid in discovery on a client system.

is what I put in the man page as part of the change

(obviously can be wordsmithed/changed once i throw it up for review)

I probably need to add a note if you're using netatalk that the tm option can interfere with it

i don't think there's a way to conditionally reject a mount option in the current share api

i.e. 'don't allow it to be set if netatalk is already advertising the adisk service'

tsoome: if you want to play with it (obviously w/ the usual 'work in process' caveats), there is a branch out there you can pull the changes

it's not squashed, but also just rebased if off master, and it's just a handful of commits

certainly any feedback you can give would be good

oh, nice.

one thing I do still want to do based off my own testing on it so far is see if there is a conflict with the mDNS stuff, I think that could be logged better (and clearer)

My main concern the last time I saw an attempt at making the SMB server advertise shares on startup or share was about the flexibility of it. I'd like to see this use something like a configurable run script with some provide environment variables, perhaps similar to the tree map/unmap scripts.

In a similar vein, we could really use logon/logoff scripts for SMB. Windows has these...

i guess the question though is what sort of flexibility to you think is needed?

Enough so we don't have to "wire in" the publication mechanisms?

that sort of defeats the purpose then though

honestly trying to do stuff like that with scripts in my experience is clunkly and error prone

I'm not doing a deep technial dive in IRC. You can call me sometime about it, or write something up.

mDNS is a part of the OS, this is an opt-in integration

and it's not even up for review yet

(and something people have wanted enough to write pages to do it in a klunky/ad-hoc fashion)

this does it better, and more seamless

lack of share discovery is among the things that has made my cifs-y life hard

but it's not the major thing, which is windows being a pain in the ass.

I've been vaguely disturbed that gwr holding my hand has been a necessary step in the last 3 times I used this stuff outside of a big enterprise environment though.

i had thought about something like svc.ipfd but for multicast -- a service's manifest could supply static info to advertise via mDNS when it's enabled

but not sure how much stuff is out there outside of smb/time machine that'd make use of it

so if your concern is normal people, of which, technically, I am one, I think your best bet is to get a stock illumos system, and a stock win 10 home, win 10 pro, win 11 ..., and make that work smoothly from soup to nuts

and doesn't work well for information that's more dynamic in nature

I think you're currently nuts, when the soup is awful too

well I can at least handle the soup (or do that later)

@tsoome around?

yes

I've figured out some of what's going on with IPMI/BMC console redirection to COM2 (ttyb) on that AsRock X570D4U board.

ou?

It turns out IPMI access (via ipmitool) works for access to ttyb (with the rate set correcty, meaning 115200)

9600,8,n,1 for life

The surprise was that, unlike older systems I've used, the loader never prints anything on ttyb, nor does it appear sensitive to input.

Every board with IPMI to a com port I've seen used 115200. Sorry, @richlowe :)

despite the console variable setting?

So, @tsoome, does that seem right that the loader only deals with the wscons?

I didn't change anything except ttyb-mode

if you have OS running, you can: tr '\0' '\n' < /system/boot/environment | grep console

and grep tty too

console=text, os_console=ttyb

ok, that means, loader was only using console=text, and asked kernel to use ttyb

I changed console to ttyb in the loader menu.

so first output (and input) should work on ttyb when kernel is sstarting

Right, that works. Again, on other systems, I could interact with the loader on any or all of text, ttyb

On this system, ttyb is inactive until the OS starts.

but could you get 'grep tty' too

No appearance other than in os_console

even no ttyb-mode ?

(and ttyb-mode)

no ttya-mode?

That stuff is there.

Should console have a list of values, comma separated?

comma or space

I'm actually interested if ttyb-spcr-mode was set

tr '\0' '\n' < /system/boot/environment |grep ttyb

os_console=ttyb

ttyb-ignore-cd=true

ttyb-mode=115200,8,n,1,-

ttyb-pcidev=

ttyb-rts-dtr-off=false

thats UEFI boot?

I think so. It boots from NVMe.

How can I tell for sure?

in that environment should be efi-version

The setup menus for boot refer to UEFI

gwr: I usually just do this for systems with a BMC and SOL: gist.github.com/jclulow/b76a9f1add72d66f75f734042f22ceea

Most boards I have used lately have also required: enabling serial redirection of the BIOS menu at all, and then making sure you disable redirection _after_ the BIOS menu so that it doesn't stomp on the serial port later

They're all different, and basically shoddy, but that is generally what works (at least once you figure out whether it's tty[a-d])

I could not find any options about that for this BIOS.

That is somewhat surprising

ok, when you reboot, it would be interesting to get efi-show -g global -v ConOut (or ConOutDev). it seems, your system does not list serial port as part of ConOut...

I assumed, the sol would add serial to console devices list...

Perhaps the firmware is rubbish

Well, more rubbish than usual, that is

It is pretty strange for the system to have a SOL-capable BMC, though, and not be able to use it from the BIOS

That is always a possibility.

of course, it may be worth to check if there is an update.

yeah

which may make it better or worse!

of course I can just add check for ACPI SPCR as we have in bios version.

Can we have some mode where, if prodded, we just do I/O to the serial port directly

I assume we used to do that for BIOS boot

the mode should be set by firmware.

Well, sure, but firmware is trash

OK, setting console="text,ttyb" works well.

which is why the loader prompt doesn't work properly on about half the EFI systems I end up using

That lets me interact with the loader during boot.

The OS works fine because we've deleted the firmware by then :P

gwr: That's good

The firmware is AMI BIOS, which is pretty widely used. Don't know if it's trash or not. :)

It definitely is

if we detect serial port from ConOut, we should use the port settings. (in bios version, we get port settings from SPCR)

there was actually "funny" bug, hyper-v arm version will hung the system if you attempt to change serial port speed with UEFI serialio api.

(serial was used for console).

tsoome: gist.github.com/jclulow/f45821c657e8b5317847ecc2a84a846c

on this system there is an 80%+ chance that a keystroke will be randomly eaten

so

I don't think we should be trusting the firmware

there you have serial + video

sure, but I just want the serial console to work, like it does under grub

jclulow: did anyone ever fix cscope, we didn't right?

richlowe: err

I mean probably not, but I also don't remember the specifics

so that good cscope still found everything it should, and didn't get mangled by function pointers and random confusion.

ConOutDev is list of all possible devices, but your ConOut has actually 2 serial port s and video...

something about flex v. lex?

oh, that thing

with the lex

all I know is I have cscope-fast which sucks, and cscope which sucks differently

and an ever shortening tether

yeah I think you found it was using some totally different hard coded set of ... whatever match rules, for different lexers?

that is all I remember yeah. I thought one of us was going to try and do somtehing, but then everyone got busy

but I held out a slim hope you fixed it angrily as part of being busy but forgot to tell me ):

tsoome: Right but if I "set console=ttya" I expect it to work reliably and it does not

everything seems to do it live, like LSP. Which is cool, but not the main use case

EFI firmware is fundamentally untrustworthy, we need to be able to ignore whatever is wrong with it and just use the serial device

tsoome: (before I forget) is there an upstream for EFI guids + names in loader? or do we just add them as desired?

there's a few from the more recent UEFI standards that'd be nice to not have to look for by the GUID :)

if you have patch, I can push it for FreeBSD

Looks like ttymon is doing 115200,p,7,1 rather than ...n,8,1. what's the right thing in ttydefs to fix that?

The ttydefs man page is not all that much help.

How did you get there

like, what configuration have you done so far

Rebooted, selecting ttyb as console

loader env has console="text,ttyb"

Did you set os_console

Most of the time, I'll look at the KVM to see if it's crashed or something. Just when I'm actually debugging stuff or whatever, I'll boot with console=ttyb

I would not do that

No. I let the loader do that based on the menu.

I'm not you :)

ttymon should pick ttyb-mode if ttyb is used for console

I know, but you're setting yourself up for pain haha

This all worked fine on other servers I've used.

One problem with "serial console all the time" (in typical lab environments) is that people see a black screen in the kvm and assume it's "dead".

That's a people problem

gwr what does ttyb-mode have in prtconf -v output?

But if you can't fix the people, it would be pretty easy to have a startup service that does "echo the system is not dead do not reboot it >/dev/wscons" :P

echo "I feel happy! I feel happy" > /dev/wscons

lol

Or run a getty :)

Sure but you're then back to over-constrained territory if you want to sometimes use console=ttyb dynamically

prtconf doesnt

as I outlined in the mail

prtconf doesn't show anything about the mode for ttyb

Oh, maybe I missed it the way I grepped the output.

make sure you have -v

you also need grep -A1 or whatever to get the ttyb it and the value

*bit and

name='ttyb-mode' type=string items=1

value='115200,8,n,1,-'

ok, so, your ttymon *should use 8 bit, not 7

assuming you have the bits for illumos#14608

I'm not sure what happened, but now the SOL shows:

roc?co??ole?lo?i?:??

roc?co??ole?lo?i?:??

roc?co??ole?lo?i?:?

I mean, if the os is recent enough, ttymon will pick ttyb-mode *if* ttyb is used for console.

right which it is not here

because you said console="text,ttyb"

so the system console is wscons

No, on my system right now, wscons is blank, and console is ttyb

um, if he did set console in menu, then os_console was set

Exactly.

Ah, so what is os_console set to in prtconf

name='os_console' type=string items=1

value='ttyb'

That's from the loader menu interaction. os_console is not set in bootenv.rc etc.

ok

I can certainly see some of the expected letters haha

Just try try things: rebooted. Interaction with loader looks fine.

does stty -a confirm you have 7 bits?

(over serial)

Yeah how do you know it is 7 bits

Looks OK while kernel is writing startup stuff

As soon as ttymon starts, it's trash. Hm...

Just guessing based on what I see.

the console is not a tty until autopush has run

which might also be relevant

does stty cs8 fix it?

see the charmingly named /etc/iu.ap

# stty -a </dev/term/b

speed 115200 baud;

rows = 0; columns = 0; ypixels = 0; xpixels = 0;

csdata UTF-8

eucw 1:0:0:0, scrw 1:0:0:0

intr = ^c; quit = ^\; erase = ^?; erase2 = ^h; kill = ^u;

eof = ^d; eol = <undef>; eol2 = <undef>; swtch = <undef>;

start = ^q; stop = ^s; susp = ^z; dsusp = ^y;

rprnt = ^r; flush = ^o; werase = ^w; lnext = ^v;

status = ^t;

parenb -parodd cs7 -cstopb -hupcl cread -clocal -loblk -crtscts -crtsxoff -parext

-ignbrk brkint ignpar -parmrk -inpck istrip -inlcr -igncr icrnl -iuclc

ixon -ixany -ixoff imaxbel

isig icanon -xcase echo echoe echok -echonl -noflsh

-tostop echoctl -echoprt echoke -defecho -flusho -pendin iexten

opost -olcuc onlcr -ocrnl -onocr -onlret -ofill -ofdel

it would be good to pastebin/gist the big output if you could

cs7 certainly looks unwanted.

indeed its cs7 there

sorry. What's "big"?

more than a line or two

how is ttymon started?

Is it just in the usual way under svc:/system/console-login:default ?

yes

(did you end up adding any new console-login instances)

nope

on my OI, it has -l console -- this means it should pick console entry from /etc/ttydefs

booooo: ctfdump: failed to open file /usr/lib/saf/ttymon: File does not contain CTF data

yeah what does... svcs -p console-login:default | awk '/ttymon$/ { system("pargs "$(NF-1)); }'

tell you

yeah, this fixes it: stty cs8 -parenb </dev/term/b

and in addition to console entry, we pick up ttyX-mode line, and set the speed.... um, well, perhaps, if we pick ttyX-mode, we should set all settings, not just speed.

yeah, I have never seen anything like this. It feels vaguely like either something was done incorrectly, or something does a lookup and gets confused

This system is a nearly brand-new install, so I haven't done anything relevant other than what I mentioned above.

ttydefs does not actually set cs8 for console, so the number of bits et al should come from terminal defaults

default_termios does have cs8

may want to check ttymodes in options.conf

I tried to add " -parenb cs8" to the flags in ttydefs, and restarted the service. Something is still setting cs7 parenb

anyhow, its 1am here, i better get sleep...

FWIW any testing that was done on SmartOS for 14608 probably didn't test the change

none of the ttymons on SmartOS use /dev/console

(which is unfortunate but true)

gwr: what does... svcs -p console-login:default | awk '/ttymon$/ { system("pargs "$(NF-1)); }'

tell you?

I am suspicious of our use of "sane" in ttydefs, and the DEFAULT Gdef in ttymon, which seem like they default to "reasonable values" including CS7

and I believe we fall into the default case if we can't find the right one in ttydefs, _and also_ we're on the console or ttymode is NULL

I don't see where "cs7 parenb" is coming from.

it's probably coming from the "sane" entry in ttymon

I would debug if, and if so how, you end up using "sane"

console:115200 hupcl opost onlcr:115200::console

see sttytable.c:121

(see also other entries in that table, but "sane" looks like the one that we'd potentially fall back onto)

you have "sane" in some ttydefs entries, though not the ones you should use, you also have "sane" in the DEFAULT struct Gdef in tmglobal.c

The svc cfg and ttymon args show the "console" label for /etc/ttydefs so I'm stumped.

I also tried adding "cs8 -parenb" to the entry. Did not help.

I've just said a whole lot of stuff you should check that doesn't involve ttydefs.

right because omething is going wrong in get_speed() of ttymon it seems

or merge_flags()

Unfortunately ttymon doesn't have CTF

hm. OK. Just weird, as this works fine on the old machine...

Your old machine is on what bits

Slightly older OI

but you've obviously editted stuff (like that console line), so it's hard to really do things while not being you.

is slightly older before or after Jul 13 21:54:45 2022

SunOS sm20 5.11 illumos-d9be5d44a9 i86pc i386 i86pc

that's sep 25th

October 13th!

oh yeah, commit date

but yes that should have 14608

on _that_ system what do you get "prtconf -v | grep -A1 console"

Sept. 2022?

yes 2022

so it is recent enough to have 14608

name='console' type=string items=1

value='ttyb,text'

and os_console?

nothing

hmm

does this tell you a path on that system: mdb -ke '*rconsdev::dev2snode | ::print struct snode s_vnode | ::print vnode_t v_path'

if you remember how, and god knows I don't right now, could you anonymous dtrace for an ioctl setting these modes?

I don't think you would need to do that

I think just restarting the service instance would be enough?

(from, say, an SSH session)

it could be, I've suffered here recently for unrelated reasons so I'm paranoid

You could also run the ttymon by hand

under DTrace or truss

it just feels like if we see _where_ it's coming from, we could backtrack to how.

Uhm. I haven't tried to use the tty console on sm20 (my old server) in a couple updates. It's apparently not working right now.

It used to... but I guess we can't use that as a reference right now.

is it not working in the same way?

SOL connects but is unresponsive.

notably CS7|PARENB also appears to be, in at least some ways, kernel defaults too

yeah probably hard coded in asy or comes from options.conf or whatever

or both!

so finding if it's coming from ttymon, or just being left to default internally would also be good to know.

Oh, right. It's booted to use kbd/vga right now.

I can reboot it to use ttyb...

but you said it has ttyb,text as the console setting

right.

I wonder if we split on the comma or if we just treat that as invalid and fall back to the framebuffer

I don't see how it could be using the framebuffer console unless we're fumbling that

That one is an (old!) supermicro

BIOS redirection etc.

sure, but if the OS was told "ttyb,text" for console and no os_console value, then either it should split on the comma and just use the "ttyb" bit, or we are fucking that up somehow and it tried to match on "ttyb,text" which is an invalid value so we default to "text"

Oh my... I forgot. That's still on grub. :)

Ah well, that's totally busted now :)

That uses the really OLD Jave ap. for KVM, so it's hard to deal with.

My old system was configured to prefer ttyb and that seems to still work right up until getty runs.

I'm sure that used to work, but I don't get on the console often enough to know when it stopped working.

It does seem odd that os_console is not getting set.

Well if it is grub there is no special logic to set it I believe

Presumably it is coming from /boot/grub/menu.lst or /boot/solaris/bootenv.rc or something in that case?

or ... /rpool/boot/menu.lst ?

I don't remember how GRUB works haha

At least on there, the tty mode remains correct. Output to /dev/console reads OK.

but there is no ttymon?

Not on wscons, and the one on ttyb does not seem to be working

when you say output to /dev/console reads OK, where do you see it

If I echo stuff to /dev/console I see it on the SOL

ah ok

that is pretty odd!

what if you "svcadm restart console-login:default"

I may have had enough of dealing with broken s/w today :)

or broken systems, or misconfigured or whatever.

well please file bugs when you figure it out

Tried setting os_console="ttyb" in the loader, and then the ttymon for /dev/console works fine. Very strange...

I guess not all parts are getting their setting the same way, or something like that.

on the old system or the new system

And yes, something apparently changed to break grub console selection, but I doubt anyone cares.

On the new.

did the loader environment previously have os_console set in it

when you did the menu version

Not in the bootenv.rc or conf.d no, but it was in the env. given to the OS.

yeah /system/boot/environment is what ultimately matters

to the kernel and eventually ttymon

I believe

When I added it to conf.d/serial instead of letting the loader add that, then the ttymon works.