[illumos-gate] 18134 Panic in tem module due to stale queue_t -- Jason King <jason.brian.king⊙gc>

this is a bit in the weeds, but I'm trying to locate a value (start of a struct) on the stack of a crashed command... would the frame address seen with ::stack be set before or after the initial locals are setup?

i.e. given the frame address of the function, should i be looking at lower or higher addresses for the start of the struct

or maybe to put it a bit differently.. pastebin.com/8MXaCaBP -- would `8045918,2060::dump` be the right incantation to look at the stack usage of `scf_iter_next_value` ?

I have some clues on the contents, so I _think_ i've found it, but the values are puzzling (making me wonder if i've missed something)

um, frame address is highest and stack pointer is lowest normally, I suppose it appears the same in trace...

hrm.. ok.. I don't know my 64-bit constants well enough :)

i think something took (uint64_t)-1, printed it (as 1844.....) then converted it to a double and printed the resulting string

Current illumos-gate on Intel 7 Series/C210 xHC reports super-speed on xhci, but throughput is only ~108MB/s raw. Is this expected for illumos scsa2usb/xhci/BOT/UASP behavior, or are there tunables/known-good controllers/bridges that might improve this (3x as fast on old macbook)

Dixie_F: I can report many headaches regarding using xhci with Illumos

Just with pass-thru or in general?

in general. the worst combo for me is zfs pool on xhci attached usb-nvmi enclosure.

hrm... i suppose it's probably ok, but digging into this deeper and seeing the lack of parameter binding in libscf saddens me

also makes me twitch a bit

if you're meaning that re: sql, I'm not sure it existed when they did it

sqlite2 is kinda rough

could be

we just convertd smf to sqlite3 and i saw this after the export/import on reboot on a system

but it made me start looking more at the svc.configd code

since the schema is using INTEGERS

I guess I probably need to see if I can find the row in the v2 db

oracle and sommerfeld already discovered that data does not survive the v2/v3 switch without manual fixups

sommerfeld: I think maybe you had even some code that might start to fix them?

yeah. it's a long-backburnered project, though.

jbk: main issue is that sqlite stores signed integers, while SMF stored unsigned integers...

Jerry did some work internally on it... i pointed him and the stuff you had seen as there was a need to get off of sqlite2

in this case, it looks like somewhere -1 is getting turned into ULLONG_MAX and then into a double and then into a string

causing svccfg import to crash

(or trip an assert)

yeah. I fixed it in sqlite2 shell on the way out of sqlite2

but tbh, it's too close to end of day friday to dig too deeply today :)

I lost my long weekend last weekend due to a customer issue, so it can wait until monday :)

i'm trying to get back to the i40e upstream now that the other things that got in front of it are resolved

(I still want to slap the intel engineers that designed the DMA engine for their NICs with a dead fish or two)

jbk, you can always hire me to make a custom SQLite version, or an extension [not sure if the extension mechanism allows to create new types] for you; it shouldn't cost you too much.

jbk: save this URL until Monday: code.illumos.org/c/illumos-gate/+/4254

→ CODE REVIEW 4254: sqlite: hack up sqlite2 shell to dump tables in ways more compatible with sqlite3 (NEW)

Better would be to just do it, then apply for a job at the SQLite company, forgot the name of it. Maybe they'll want to pay for it retroactively?

I think the issue is that sqlite2 is dumping values from INT64_MIN to UINT64_MAX without quotes; sqlite3 then converts the values between INT64_MAX and UINT64_MAX to float.

(sqlite2 was storing them all as strings; sqlite3 converts unquoted numbers to either binary integers or floating point)

ahh.. i'm guessing if i look at the exported v2 db, i'll probably see that value in there...

based on that...

IIRC, you can explicitly declare columns as integers and floating point, but you'll have to do the right incantation when you create the .db file to make that work.

the existing schema in svc.configd does do that (all the columns are INTEGERs)

but it sounds like it's an issue with doing the export and then importing

But do you do the right incantation to /force it/ when you create the .db file in the first place?

since it sounds like v3 kinda ignores the column types

and just associates a type with each cell

I'm off the clock, you need to pay me to look it up in the manual for you.

so the same column can have all sort of fun types in its value if you enjoy pain

jbk: sqlite3 is squishy with types unless you enable STRICT - see sqlite.org/stricttables.html

yeah, I think I had to use 'strict' for the prototype of sqlite3+idmap, if I wanted to stay sane

And I thought there was an option when the .db file was created to force it; but I may have misremembered the manual.

Now <jbk> can also enjoy this public humiliation. imdb.com/name/nm18494641/mediaviewer/rm1173343234/?ref_=nmmi_mi_1_4

I think some sqlite options like that are sticky

maybe only foreign key obeyance?

STRICT is part of the schema, so it's sticky on a per-table basis. WAL is sticky. Foreign keys enforcement is annoyingly not sticky - you need to reenable via pragma on every session.

everybody likes databases :)

sqlite3 is a great database with a bunch of compat baggage.

hadfl: if only more people used them properly :)

and RTFM

Of course, nobody builds Sqlite3 commands or libraries on their own, and enables STRICT by default; I'm pretty sure I did that once or twice on Windows, but forgot how.

early on in my career, I dealt with a system that used a database (informix, but any other sql db would have been the same) where they just created a table with like 8 VARCHAR(128) columns

and to store data would basically concatenate values into a string and then store it in one of those columns

I'm not old enough to have used Informix; I did install DB2 on my IBM Thinkpad once, but couldn't use the laptop afterwards. IIRC I had to buy a new HDD to fix the problem.

jbk, are you sure you're not talking about DOS Dbase III or something?

I'm very sure

(if you really want to know, it was a Siemens FFP (fast feature platform))

Then I'm pretty sure Dbase III copied their protocols C:

so like columnA would have 'foo|1234|bar|9999|true' as a value

instead of having separate columns for each item

I've never been a fast feature platform, but I've spent several weeks doing CREATE INDEX to make a website go faster. We don't talk about which website.

well that's the thing, you really can't index that :)

there was other things, like when doing bulk inserts, committing after each row IIRC

No, nobody can just index by the slow query log automagically.

but it was all behind a propritary API where the only language supported was perl5

jbk, Informix didn't have BEGIN ; and COMMIT ; ?

oh it did

jbk did you do any 4GL? My first real job involved Informix Online and 4GL..

jsut teh software using it was using the database quite poorly

I once implemented shift-jis to utf-8 in perl5; the code was complete bullshit, but I had to do it the way I did, because code pages in perl5 are complete bullshit anyway.

so loading like 10,000 rows took _hours_

when it should have taken maybe a few minutes at most

Pretty sure it could have taken milliseconds with a modern SQL server, and the proper PostgreSQL setup; I know, I did it about ten years ago.

but not to single them out too much, such SQL anti-patterns are unfortunately not as rare as they should be

this was like in 98-99 running on a sparc 20

Nobody reads the slow query logs, and programmers are taught not to CREATE INDEX; that's because they believe it'll make the system slow. Too many indexes makes the system slow, of course.

jbk, and you didn't have the budget for 10,000 sparc 20s? Usually when a system is slow, it's always a budget problem.

it was a self contained system with the software from siemens

Then always blame Siemens. Don't blame yourself for their budget problems in the late 80s.

andyf: naah.. though I know some people that have...

but the point is SQL databases have had immense amount of effort put into making them pretty good products, and just treating it like some dumb datastore is going to do you and your products a great disservice

jbk, a Japanese programmer who does -- or did -- everything in perl5? I have an ex-coworker who may fit the bill; we went out to eat sushi yesterday.

(but it's sadly a lesson people keep learning the hard way)

Usually they don't learn it at all; the website gets so slow the company goes bankrupt and the programmers move on to a different website, and repeat the same configuration errors.