13:36:09 [illumos-gate] 15611 rtld: suggest explicit braces to avoid ambiguous 'else' -- Toomas Soome 13:43:56 hmmm.. i do wonder if it might be useful to make pageout related i/os a higher priority than other writes 13:46:33 +1, to avoid pageout_deadman panic 14:51:56 it looks like (from looking at one such dump) that they just get lumped into the sync write queue 14:52:30 so they could compete with other sync writes 15:04:09 Hello all. Sorry if this has already been asked, but is there documentation that explains the directory structure of illumos-gate? I know that "uts" is the directory that contains the kernel sources. However, I don't recall what "uts" stands for & I have no idea what "cb" means for the directories named "ucbcmd" & "ucblib". A little help? 15:06:05 Try https://illumos.org/books/dev/layout.html 15:06:32 I believe it's 'unix timesharing system' and 'university of california berkeley 15:20:26 ^^^ 15:25:15 originally sun shipped with a BSD based os (back in the 80s I guess?) -- aka SunOS 4.x, then in the 90s, they moved (or I guess worked with AT&T?? something like that) on SystemV which became 'Solaris 2', and because it would be too simple otherwise, SunOS 4.x because solaris 1.x and the system-v based OS became solaris.2 (but the uname bits showed SunOS 5.x) 15:25:20 something like that 15:26:37 and so for compatability reasons, versions of commands that were different between sunos 4.x and solaris had the older sunos 4.x bits put into the ucb directories 15:27:23 so the ucb stuff is more or less the BSD versions (from that era -- don't confuse with modern BSDs) of commands 15:43:00 danmcd (or anyone else that looks at dumps)... I've written a little dcmd that takes an hrtime_t as an input and basically gives you the delta with mdb_gethrtime() (so 'now' on a live system, more or less the panic time on a dump -- mdb_gethrtime() is often a few ms later than panic_hrtime in a dump) 15:43:17 Oooh. 15:43:20 e.g. 15:43:33 > 0x2862546361::hrdelta -h 15:43:33 00:01:36.15787773 15:43:53 it seems like maybe useful as an mdb built-in ? 15:45:10 (i've been deep, deep insides the bowels of zfs's vdev queueing and the zio pipeline and i found myself doing the math constantly and though it might be useful as a command) 15:50:34 nothing like scratching a real-world itch. 15:52:17 You may want a way to specify what the comparison is against, but seems like it'd be useful depending on what you're doing. 15:58:07 [illumos-gate] 15590 smbd becomes unresponsive on memory allocation failures -- Gordon Ross 17:29:39 [illumos-gate] 15655 Build scripts should tolerate make -w -- Bill Sommerfeld 17:44:52 [illumos-gate] 15605 elfedit: clean up warning gags -- Toomas Soome 17:49:23 if you have a need for mdb stuff, I think it's always good to have it, it can be extended, etc. 17:50:02 (I think explicit comparison is good now though, and probably easy to add?) 17:51:49 ADS_Sr: 'uts' - UNIX timesharing system, what the business side of AT&T called what everyone else called 'sys/', 'ucb*' - UC Berkerely, but in this case really SunOS 4.x, 'psm' - platform specific module, once was the future, but then stopped being the future, kind of sits there half baked, 'stand' - standalones, like the booter etc, that run without an OS (on system's where that's more useful), not a thing on x86. 18:05:10 I have no idea how I came to spell Berkeley like that, sorry alanc. 18:05:30 [illumos-gate] 15651 bhyve mishandles periodic RTC timer -- Patrick Mooney 18:06:19 You might as well do "sgs" for compleness. 18:06:55 It is in the dev guide. 18:07:07 It's the software generation suite, which was the AT&T name for the collection of utils at various points. 18:12:06 system, not suite, I think. 18:12:25 as a name, it made more sense when it had the compiler in it, etc, too. 18:12:30 but then Sun decided the compiler was a separate product. 18:12:34 and here we are 18:12:40 (I'm not sure whether they were the first to do that?) 18:18:37 sommerfeld_: I have an observation I didn't want to derail your 'make -w' for. The massive towers of egrep we form in nightly are -- in circumstances -- _really_ slow. 18:19:33 I _think_ because during the build obviously we fill nightly.log sequentially, and never read it, so the head of it is long gone from any cache before we get to reading it back, and then we effectively throttle it by the speed of each successive egrep, but I'm not sure, because I've not been able to catch it being awful except during a build. 18:19:40 where my worst result is in the minutes. 18:25:06 (and that might just be innately how long it will take because of circumstances) 18:25:11 I just enjoy sharing a mystery around. 18:31:31 at least nightly.log compresses well, if on a dataset with it so configured 18:33:34 yes, I noticed that recently while impatiently awaiting a build's completion. 18:38:17 I think the slow one is actually the "grep : $SRC/${INSTALLOG}.out" 18:41:31 sommerfeld: would fgrep/`grep -F` help there? 18:42:02 actually the several such. danmcd: not quite because there are regexps involved 18:42:45 You mentioned $SRC/${INSTALLOG}.out... thought that was a pattern. 18:42:52 I know the other egreps need the 'e'. 18:43:20 * danmcd sometimes misses agrep 18:43:26 start looking at line 193 18:46:45 I see 5 passes over ${INSTALLOG}.out 18:47:28 And there's this: `egrep ":" $SRC/${INSTALLOG}.out |` 18:47:34 THAT can be fgrep, no? 18:47:43 fgrep is not necessarily faster than egrep 18:47:52 benchmark it 18:48:15 it comes with less semantic overhead 18:48:41 Fair enough. But the 5 passes... THAT could be one egrep expression if done right, no? 18:48:57 (ugg, prob. not) 18:49:24 or one awk? 18:49:27 The output goes to a bunch of different places, though. I'm wondering if we wouldn't be better off with "make install | tee -a $LOGFILE | awk 18:49:55 with the awk script splitting the bits & pieces off as needed 18:51:09 (build errors, build warnings, build noise, and the outright go-directly-to-mail-msg checks for fatal link errors and oversized bootblocks. 18:51:40 that would probably be more cache friendly 18:51:50 certainly more cache friendly. 18:54:08 Oh, I bet the 'grep "bootblock image is .* bytes too big" $SRC/${INSTALLOG}.out' is dead code. 18:54:43 (on a different subject, and nothing new, but again disappointed that the various allocation functions in zfs either don't allow the caller to specify allocation flags, nor change underlying allocation behavior based on what's requested 18:54:48 aside from data vs. metadata 18:57:59 That line in nightly is the only hit in the gate for "bytes too big" and "bootblock image". 19:00:34 nbjoerg: less semantic overhead but (at least historically) egrep was just plain more efficient even for fixed strings. 19:06:17 I wouldn't be surprised if some old versions are not using BM or so ;) 19:10:11 sommerfeld: a lot are dead code, etc. 19:23:15 If anyone is looking for places to optimize in the full build, consider the huge set of make pattern rules that come from $SRC/uts/.../Makefile.rules etc. 19:24:05 the only thing stopping me dealing with that is the merge problems. 19:24:26 Most of those only needed to be centralized like that so that we could run a global lint on all the kernel code. Now that we don't do that anymore, I'd love to see all the Makefile.files and Makefile.rules stuff split up and distributed around somewhere more local to the sources they related to. 19:24:29 my plan is to split things up so that common bits are actually _common_ for the ARM gate, and hopefully reduce a lot of that in the process. 19:24:57 but everyone downstream would get a potentially hellish merge, so I've been thinking about it in detail first 19:25:07 because it doesn't actually hurt build times as much as it seems (or didn't, when I measured it) 19:25:11 The current situation cause make to stat a _whole_ _lot_ of files that won't exist. (that ain't free) 19:25:40 Depend how well your FS does with ENOENT. 19:26:01 and honestly, the idea of writing up an IPD -- which I'd have to do -- terrifies me, because everyone and their dog has a peanut gallery opinion about this stuff 19:26:10 and I have no energy to talk about gmake, cmake, ninja, or whatever. 19:26:30 and yet "Why do all this and not ..." is a perfectly valid question 19:26:38 it's just one I'd rather throw myself into a lake than answer. 19:26:51 Incremental improvement is really the only sensible way forward, not "Let's switch to (insert favorite framework)' 19:27:08 anyway, it depends heavily on my cross-compilation IPD/work 19:27:14 which you'll notice I haven't sent out yet either :) 19:27:21 chicken. 19:27:24 :) 20:35:21 yeah, setting aside the possibilities of a build system overhaul or replacement, fixing the target rule stuff in uts has been a serious temptation 20:36:00 (because of all the unecessary stat()s) 20:36:22 there are also some VFS improvements we could make on that front as well, I think 20:39:00 well, I can shuffle some steps of things I want to do around maybe so that that's closer to the top 20:39:05 but I'm not sure how closer to the top is enough. 20:39:27 the critically important bit to me is the absolute lack of sharing in there stops 20:39:28 didn't mean to make it an ask for re-prioritization 20:39:37 yes 20:39:40 but the 2nd stage is probably not much worse after that. 20:39:55 most of the makefiles know what objects they need, and where they are 20:40:07 we don't need to put all the rules in one place so they're shared by literally everything 20:40:26 well, that's the fun bit, they're shared by "literally everything", except across platforms, where that sharing is useful. 20:40:30 and instead they're duplicated. 20:41:13 so I want to shared the shared bits, and then slice the files up as step 2 20:41:42 for various reasons, but especially that no matter how you do step 2, everyone downstream doing actual work has one hell of a merge 20:42:10 I can talk about it more another time, if you're interested (and I'd have to write it up anyway, so you'd definitely have time for an opinion) 20:44:25 looking at the egrep thing, it appears that " egrep 'warn:|warning:' " is a pathological regexp for egrep. 20:45:06 compared to /warn(ing)?:/ even? 20:45:36 trying desperately to remember finite automata 20:53:19 time egrep -i 'warn:|warning:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 20:53:19 [2] 801286 exit 1 egrep -i 'warn:|warning:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 20:53:19 egrep -i 'warn:|warning:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 147.50s user 0.15s system 99% cpu 2:27.66 total 20:53:32 time egrep -i 'warn(ing)?:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 20:53:33 [2] 801299 exit 1 egrep -i 'warn(ing)?:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 20:53:33 egrep -i 'warn(ing)?:' /z/ws/illumos-wformat/usr/src/install-nd-i386.out 1.40s user 0.07s system 99% cpu 1.476 total 20:56:57 wow 20:58:49 yeah. 21:05:37 I filed https://www.illumos.org/issues/15662 21:05:38 → BUG 15662: nightly uses pathological regexp to find warnings. (New) 21:08:14 .oO 21:09:18 We should probably also look at improving egrep there at some point. That is pretty bad. 21:12:01 gnu grep does much faster (with "bad" regexp) 21:15:43 it's not using spencer regex 21:17:29 yep, reading cached log does not help our grep:( 21:18:08 https://speakerdeck.com/0x3c3e/codeql-plus-dtrace-equals-memory-disclosure-vulnerabilities-in-xnu?slide=32 21:18:31 I suspect that Apple fixed a CVE in DTrace and did not share the fix with anyone. Correct me if I am wrong. 21:19:39 I will send an email to the security email. 21:52:19 Thank you @ryao 22:35:18 [illumos-gate] 15648 Attempting to create multiple link-local IPv6 addresses leaves subinterfaces behind -- Andy Fiddaman 23:17:05 rmustacc: it looks like grep just uses the libc regcomp/regexec