[illumos-gate] 15086 ccid mistakenly thinks F_AUTO_PARAM_ATR means params are required -- Lexi Wilson <lexi⊙cn>

is there anyone how is familiar with the Solaris 11.4 mdb command ::deadman_journal? I try to investigate a Solaris 11.4 guest hang on Omnios Bhyve and I have a crash dump initiated by set kernel parameter snooping=1.

this shows deadman_journal

vmcore.5> ::deadman_journal

  CPU  LOG#        TIMESTAMP  INTRACTV     NRUN           THREAD

    0   299        21871650d      4000        2 fffffffffc235b00

    1   298        227588bba      4000        0 ffffe33000f7dac0

    1   299        225d5a7aa      4800        1 ffffe33000e5bac0

    2   299       dc1b40dcd9      4000        0 ffffe33000cfaac0

    3   299        225d8d4b9      4000        2 ffffe33000bffac0

havent used it. name is kind of self explanatory, I'd check the thread stacks where NRUN column entries are not 0, also LOG# column would seem to suggest there are log entries for deadman?

in general, deadman fires on resource starvation - the immediate suspect would be disk IO (being the slowest resource there).

cgr: Does "::help deadman_journal" work?

Also, if it's related to "snooping", that's generally something getting in the way of clock() I think, rather than some of the ZFS I/O or pageout deadmans

The timestamp of CPU 2 is much different, could be normal?

vmcore.5> ::help deadman_journal

NAME

  deadman_journal - display the deadman journal

SYNOPSIS

  ::deadman_journal [-v] [-c <CPU>]

DESCRIPTION

  Displays the contents of the deadman journal.

OPTIONS

    -c <CPU>    display content for the given CPU only

    -v          report interrupt statistics in nanoseconds

ATTRIBUTES

  Target: kvm

  Module: genunix

  Interface Stability: Unstable

some data here: pastebin.com/45UHaAhS

well, the ffffe33000c29ac0::findstack at line 120 is interesting and is indeed pointing towards time management as you are getting panic from cyclic_expire()

some data more data form threads on CPU 1 here: pastebin.com/wy33kjhi

hrm..

from the mac man pages, it suggests for MAC_CAPAB_RINGS, for TX rings, you can specify a group count of '0' but with > 1 rings, correct?

looking at mac_init_rings(), it appears that will cause the mr_rget() func to be called with a group_index of -1

which doesn't appear to be a value specified in the man pages

so not sure if this is a mac bug or a man page bug

(my assumption was that for such a scenario, the group index passed to mr_rget() would be '0')

[illumos-gate] 18261 want apic_id in cpu_info kstat -- Andy Fiddaman <illumos⊙fn>