i have a bash alias that mounts and encrypted (loopback) file. this worked fine until i upgraded my vm instance from base-64-lts 21.4.1 to 23.4.0. now, it doesn't bother waiting for me to enter the encryption password and just fails to attach the loopback. the alias is this: alias myMount "mount \$(lofiadm -a /enc/file/path -c aes-256-cbc) /mnt/path"

if i avoid nesting the commands, the alias works as expected and waits for password input on the lofiadm command: lalias workingMount "lofiadm -c aes-256-cbc -a /enc/file/path /dev/lofi/2 && mount /dev/lofi/2 /mnt/path"

unfortunately, the working way doesn't play nice if something else is using /dev/lofi/2

lofiadm comes from the platform rather than the image so it's hard to see how that would affect things here

I'd have a look at any surrounding infrastructure that may have changed at the same time

looking at lofiadm source in the illumos, nothing seems to have changed except for the Makefile in the last few years: github.com/illumos/illumos-gate/tree/master/usr/src/cmd/lofiadm

i verified that bash works fine w/ nested tty read blocking with other commands...

ok.. i just spun up the old vm instance and it exhibited the correct behavior.. weird

bash, mount and lofiadm are all the same binaries provided by the platform which is constant between the two vm instances

would termcap differences cause this?

ok.. looking at the termcap info using infocmp, there is a difference in the rin value... the old value (that causes the original nested lofiadm to work) is just `rin@` whereas the new vm value is `rin=\E[%p1%dT` which doesn't work. off to learn about termcaps....

razamatan: So I wouldn't dig into termcap first for this.

There are a bunch of changes in the kernel lofi driver. So can you share the two platforms you changed between?

Also... is your bash coming from platform (/bin/bash) or pkgsrc (/opt/local/bin/bash)?

I ask because of the failure mode...

"now, it doesn't bother waiting for me to enter the encryption password and just fails to attach the loopback."

This seems like a can't-reach-the-tty problem.

Platform bash won't change unless you change the platform, which I can't tell if you did that or not, given you also said you updated the zone's image from "base-64-lts 21.4.1 to 23.4.0."

Knowing if you changed platforms as part of the pkgsrc base is helpful, so if you did, also let us know the platform versions.

If you use pkgsrc bash, it COULD be something there related to $(SHELL-COMMAND) processing that changed

danmcd: from platfrom? it's /usr/bin/bash in both cases

Yeah, so that's platform bash alright. Did you change platforms when you updated your zone's pkgsrc base?

danmcd: i am actually running a single platform image (joyent_20240125T000404Z) with the old and new vm that can repro this problem

Ahhh. Interesting.

very

set -x bash output is the same flow

So the old pkgsrc image on this same PI works, but the new pkgsrc image does not?!?

correct

These running concurrently on the same physical machine?

binaries wise, they're all the same, afaiu.. to me, it seems there's something in the vm images that's causing the tty read to fail in the new vm

yes

Damn....

rmustacc: yeah.. the termcap diff was a red herring

So tell me, what does "pldd `pgrep lofiadm`" say (run it in the same zone) while your lofiadm is asking for a password on the one that works?

ALSO:

what does "vmadm get $VM | grep fs_allowed" say for both zones?

danmcd: fs_allowed is the same between them: "fs_allowed": "ufs,pcfs,tmpfs",

Had to check.

pldd? Does it have anything in /opt being linked in? I doubt it, but I have to ask.

non-working new zone pldd: 38341: lofiadm -c aes-256-cbc -a /zshares/archive/.private /dev/lofi/10

/usr/sbin/lofiadm

/lib/libc.so.1

/usr/lib/libpkcs11.so.1

/lib/libcryptoutil.so.1

/usr/lib/security/pkcs11_softtoken.so.1

/usr/lib/libsoftcrypto.so.1

/lib/libavl.so.1

/lib/libmd.so.1

/lib/libgen.so.1

working old zone pldd is the same output for the paths. only difference is the pid (39409 instead of 38341)

the uuid of the two base images is: 85d0f826-0131-11ed-973d-2bfeef68011c and 8adac45a-aca7-11ee-b53e-00151714048c

Wait... both ask for the password? I thoguht you said the new one doesn't?

i had to use the workaround approach (non-nested &&) to capture the pgrep lofiadm

Oh.

otherwise, it dies too fast

Yeah... the workaround works... WHY it works is the question. CLearly it's not some /opt library lofiadm is bringing in for some bizarre reason.

Is your privilege set the same in both zones?

yes

So literaly `ppriv $$` on invoking shells on both zones show the same thing.

Damn.

danmcd: outside of the bash pid being different, the diffs show nothing else

Damn.

So if you utter "lofiadm -c aes-256-cbc -a /enc/file/path" all by itself on a CLI you fail on the 23.4 zone?

(Trying to minimize the reproduction...)

no.. it only fails when i nest it inside of a mount: mount $(lofiadm -c ... -a ...) /mnt/path

Ahhhh so

echo $(lofiadm -c....)

succeeds on the 21.4, fails on 23.5 ?

23.4 I mean ?

doing it by itself or doing the workaround of serializing the lofiadm and mount works fine

i'll do the echos

yeah.. echo repros

actually.. this may be due to sudo

WHA?

You're using sudo?

hold on...

I saw no mention of sudo.

yes.. i omitted it b/c i thought it couldn't possibley repro

this is my mistake

1000x apologies

Oh no worries. I'm glad I could help narrow it down.

so the main delta in terms of binaries is sudo which does live in the vm image

really, thank you

YW, glad to help.

Gonna go install a USB-key now downstairs. I have to bring this NUC back online after a weekend of failed experiments.

i'll have to figure out why sudo is so bad... visudo in the new vm image doesn't seem to handle the term correctly...

gah.. the fix is to disable use_pty in sudo

the just enabled it by default in the last few months. sorry for the noise folks, and thanks again for the help danmcd. also, thanks in general for a great os!

YW, and don't forget props to jperkin and rmustacc as well.

yep