18:58:38 <vetal> jclulow: Here is with shared disks between two nodes -  (case when only one connection to each node)
18:58:49 <vetal> jclulow: https://gist.github.com/gusev-vitaliy/69f7063b67a3100425f9cdf6a2fad06b#file-gistfile1-txt
20:33:31 <tsoome> is there sort of "obvious" cause for vmem_hash_delete() panic with "bad free" message? 
20:33:55 <tsoome> like there are 'freeing into free cache' and such....
20:39:26 <richlowe> the address you're trying to free isn't in a vmem segment
20:39:49 <richlowe> that is, we hashed `addr` and walked the bucket, and it wasn't there
20:46:33 <tsoome> yep. that much was easy to see:)
20:46:55 <richlowe> Hey, I'm not the one who asked the question
20:48:38 <tsoome> nono, I meant no harm:) but yes, I guess it really has no obvious answers anyhow
20:52:53 <tsoome> seeing it from reap, so the kmem_cache_free() and friends have not spotted anything wrong  
21:38:27 <sommerfeld> by default, kmem_cache_free() doesn't do a lot of checking on the fast path.  
21:41:58 <sommerfeld> if it's a reproduceable crash, try the workload with kmem debugging enabled.
22:20:50 <tsoome> yep, thats the plan. there has been 2 cases, but not sure if it is reliably reproducible.
22:24:52 <richlowe> you could turn the lighter-weight kmem debugging on for longer periods of time, even if the heavier pieces would be a problem, too
22:25:27 <richlowe> (I've mostly found the checking wasn't a major concern, but the logging was problematic)
22:25:50 <richlowe> unfortunately, the logging is also often invaluable to fix what the checking has found :)