i'm trying to run omnios under smartos. the vm is starting, and vnc is listening (and allows connections) on the assigned port, but none the vnc clients I've tested get past protocol version negotiation.

i was suspecting bootrom setting of "uefi" to be the issue but that should be ok. Any ideas?

i figured it out -- was allocating too much ram (it was 4G less than available, so not sure why that was too much)

FYI: if you happen to have the problem where USB 3 devices don't show up after a reboot (until you unplug and replug them) you should take a look at: illumos.topicbox.com/groups/developer/T7b412760d78e8a22