hrm...

after updating to recent (past 2-3 weeks) smartos, I no longer am able to incrementally build

before if I say was working on a driver, I could bldenv illumos.sh; dmake -e install the driver into the proto area, rm proto/buildstamp, and gmake live to get a new dated image

now that always does a full illumos build

(so 5 minutes now turns into 40)

not sure why yet

Hi folks. I'm working on getting the first Triton cloud set up for my company, and hit a small snag toward the end of building the headnode. I'm running the post-setup command to create common-external-nics and am getting this error:

[root@headnode (us-west-1) ~]# sdcadm post-setup common-external-nics

sdcadm post-setup common-external-nics: error: No backends available in pool 251ffee2-5788-431c-89e5-ca6910058bb6 (napi.us-west-1.emeraldbroadband.net)

sdc-healthcheck reports that everything is up.

Hmm, sdcadm self-update also fails: [root@headnode (us-west-1) ~]# sdcadm self-update --latest

sdcadm self-update: error: Binder service seems to be down. Please review it before proceeding: sapi client error: No backends available in pool 6cf5082f-bf3b-40f3-a758-c50d2d2bda9d (sapi.us-west-1.emeraldbroadband.net)

Seems like it's just sdcadm that's acting up so far - sdc-healthcheck works, as does sdc-login (e.g., sdc-login sapi). But every subcommand of sdcadm fails. Any ideas on what I could try?

How big is your public address pool? `external` needs at least enough addresses to support the zones in question.

It's a /22

May be nameserver related. I had added two nameservers during the install process. The commands were succeeding sporadically, and I noticed that /etc/resolv.conf on the HN includes the binder address and the two I added. I removed the two that I had specified and commands seem to be working now.

Odd, though. Why does the installer ask for DNS if it breaks things?

jdt: check your moray service to see if it's running properly

Okay, will do. sdc-healthcheck indicated it was fine, but I didn't dig deeper than that.

Also, if you have an external name server that replies with anything other than NXDOMAIN for your triton dns suffix, that's going to cause a problem.

Oh, got it. Definitely do have that.

I'll rebuild it with a dedicated suffix.

Oh, you do?

Oh, well not the full "us-west-1"

Querying from here, it doesn't look like you do, but sometimes things are different internally.

So, the DC is "us-west-1.emeraldbroadband.net" and that wouldn't resolve to anything. But "emeraldbroadband.net" does.

That's fine

Cool

but if you have a record like *.emeraldbroadband.net that will reply for anything, that will cause a problem

Okay, yeah, I think we're good there.

Actually, I take that back. I do get NS records: gist.github.com/justindthomas/1f654d955c61f14bae26add5149d5d94

(internally)

And 10.10.60.103 is the DNS server you configured?

Yeah, one of them.

OK, yeah. On line 7, it says status: NOERROR

That's what's wrong.

Okay, I'll find a way around that. Thanks for the tip.

If you do the same query from your workstation with 8.8.8.8 as the DNS server you'll see the status is NXDOMAIN, that's what we'd expect.

Got it.

NOERROR means the operation was successful with a valid reply.

NXDOMAIN means "Uhhhh, dunno, maybe ask a different dns server"

So what we'd expect is to get NXDOMAIN, then the query is automatically retried against the next server, which will eventually land on binder, which will give us the correct response.

If I follow that through to the specified server (.69), I get a REFUSED because that server isn't configured to respond to requests from my new subnets.

I do get a NXDOMAIN from that server for us-west-1 when I query it from an allowed subnet.

Ok, so maybe that's all you're missing then.

I'll try that.

Is there a way to adjust those nameservers post-install? I'm still getting NOERROR on the first level recursers, but I can get proper NXDOMAIN responses from the referred servers (which also recurse for internal subnets).

I can adjust the config on the usbkey, but do I also need to adjust anything in the database?

It's on the usb and in sapi

And it might be on anything that has an external interface

Okay, thanks.