15:17:33 danmcd: I believe if I'm reading this correctly, we are hitting that postgres coredump even on "genunix: [ID 107833 kern.notice] #015illumos Version joyent_20221215T000744Z 64-bit" https://gist.github.com/Smithx10/decb76decbc515de550204279d61a602 15:25:29 jinni illumos#14892 15:25:30 https://www.illumos.org/issues/14892 15:25:36 Smithx10: ^^^ 15:26:19 May I have the dump, please? 15:26:30 doing that now* 15:27:13 Thx. 15:28:19 04:35 ETA 15:33:39 ping me unicast when ready. (And pardon slowness today, twisted my ankle causing a full-fall down the stairs. The bruises are epic.) 15:34:55 ouch 15:35:06 i cut my thumb with an exacto knife the other week 15:35:13 got lluuuuuucccky 15:35:56 Ouch! Lucky is nice, however. (We're starting to sound like Billy Crystal's and Christopher Guest's janitors from the 1984-1985 season of SNL.) 15:37:30 No dying, you two. We need you. 15:37:45 how many stairs? 15:38:00 very dangerous.... hope you werent carrying anything 15:44:27 1.) 10 +/- 2 (garage stairs, on to concrete floor... handled my fall okay to avoid head injury)> 15:44:33 2.) Not carrying anything. 15:44:50 My right ankle and I have a torrid relationship dating back to 1991. 19:03:57 I ran out of disk in a zone, changed the quota but am getting "svcs: Could not bind to repository server: repository server unavailable. Exiting." 19:04:01 what's the fix? 19:10:17 are svc.startd and svc.configd running in the zone? 19:11:03 i believe the error is referring to svc.configd, so it might be enough to kill it if it's running and svc.startd _should_ restart it 19:11:09 https://gist.github.com/Smithx10/8b504af388f91b12859ab6f787a6b6b1 19:11:56 you could try killing svc.startd -- init should restart that :) (or just reboot the zone) 19:12:37 it may be possible the smf database is unhappy due to the fs being full 19:13:02 in which case there's a recovery process to restore it to an earlier repository snapshot 19:13:46 looks like vmadm stop doesn't work 19:13:59 stop -f stopped it 19:14:29 (logging into the console of the zone should give you the steps if the smf repository is unhappy) 19:18:18 svc.configd exited with status 102 (database initialization failure) : ( 19:19:20 googling lead me to /lib/svc/bin/restore_repository 19:20:02 it should tell you that on the console (zlogin -C) 19:20:03 but yeah 19:20:07 that's the thing to run 19:20:27 it'll show you the available snapshots (basically backup copies of the sqllite db) 19:20:40 pick whichever one makes the most sense (probably the most recent snapshot) 19:20:49 it'll restore, then reboot 19:21:00 and you should be good to go 19:23:27 picked the latest ./crosses fingers 19:24:08 yay, thanks jbk 20:39:56 TIL. I've never had to do that. Usually running out of space and nuking SMF just needs a quota bump and a reboot. 20:49:38 i think the version of sqlite in illumos has some bugs where it's not always consistent on disk 20:49:41 which can necessitate that 20:49:57 unfortunately it's so old, trying to do any sort of bug search is pretty daunting 20:50:58 i suspect it'd likely be better w/ a newer version, but that brings in it's own set of issues (upgrade concerns, changes in behavior that might be significant, ...) 20:51:18 nothing insurmountable, but not sure how much appetite there'd be for it 20:56:35 Especially when it's usually fine.