I have a weird issue in a multiDC deployment...I setup the ufds replication 2 years ago and it was working fine. At somepoint the admin network routes on our dark-fiber got AFU'ed in our IGP and AZ2 couldn't contact AZ1. It seems that all of the sdcperson objects have propogated from AZ1 to AZ2 previously but for some reason the only user that isn't propogating to AZ2 is the poseidon user for manta.

Although all is synced fine in AZ3. Has anyone ever seen this behavios in UFDS? Or could it be that I since I haven't run the manta setup steps in AZ2 that the posiedon user simply isn't going to be replicated yet? I do not remember running anything in AZ3 besides installing the manta zone on the headnode. So I am not sure why it is replicating there.

Side note: /opt/smartdc/ufds/bin/ufds-reconcile-data reports that all data is in sync on all 3 AZ's in the ufds0 zone on each headnode.

But `sdc-useradm search poseidon` only returns results in AZ1 and AZ3

This is `barfield` BTW. For some reason the nickServe isn't letting me update my nick today.

Yeah, there's a bug (maybe fixed now, I don't remember) where the manta install script would add poseidon to the local UFDS instead of to the primary.

If you're only missing it in one DC, then you're going to have problems.

I see, so maybe I did run it in AZ3 and it is pulling from the local primary?

I guess UUID's would identify this

The way UFDS replication (well, *all* ldap replication) works is by sending differential changes. There's no "reconcile everything" feature in LDAP.

right that makes sense

The only way to fix it is by using ldap commands directly in the local UFDS.

the UUID is the same in both AZ1 and AZ3 it just doesn't exist in AZ2

The local UFDS in the replication zone in AZ2?

In the DC where it's missing.

Thanks man!

Thats easy enough.

I was worried I had a bigger issue then that

BTW I found the issue with my rabbitmq zone

Was the external NIC in sdc-napi causing the issues. I had to delete it and unreserve the IP address. Then when I bounced the headnode net-agent updated napi and rabbit came right up

I suspect that someone was aware of the UFDS bug at install time and attempted to correct it by removing the offending users, but then forgot to add the fixed one in to AZ2.

That would make sense

It would've been me but I've slept a lot since then

Drank alot too :)

Is the Linux CN project still active by chance?

As for rabbit, ISTR there being a discussion about the extra IP being an issue.

We're working on a new milestone release for LinuxCN.

I updated all of the SAPI VM UUID's in /instances ... serveral had params.server_uuid set to 2 different missing server ID's.

Things are a bit in flux because of the LXD announcement. We haven't decided what to do about that yet.

there was a 0.1 incus release earlier, I think

Have you guys looked into `harvestor` as a potential pi?

What LXD announcement? I must have missed it

nueroserve: is this on github?

No. I don't know what harvestor is.

Its essentially a minimal container OS designed to back k8s

Then it's unlikely to fit well with Triton

We'd likely have to remove a ton of stuff until it resembles what we need.

I figured as much

I was just curious...seemed like it might make a good candidate

What distro currently backs the linux CN?

Debian

oh nice

my favorite anyway

well for linux that is

thansk

I've got poseidon recreated in AZ2 but I cannot seem to add the "dclocalconfig" attr so that it is displayed when you run sdc-useradm get UUID

Is this something that should be populated from sapi instead?

replace-attr, add-attr etc completes successfully. but get never shows that aspect

oh wait I may have just figured it out

Nah I didn't. I thought that I could add it with a json manifest to sdc-ufds.

So I am assuming that dclocalconfig on a ufds object comes from the local ufds master and not from the remote

The remote appears to nest all of the localconfigs on a user as a json array []

*dclocalconfigs