09:27:02 <tozhu> hello all, I have created a process set for 0-31 64-95 by command: psrset -c 0-31 64-95  ,   and returned process set id: 1   , now I’m going to bind VM and it’s process under the process set 1, then I executed the command in GZ as root user:  psrset -e 1 vmadm start VM_UUID ; I find the memory is not allocate under the lgroup 1 (process 0-31 64-95 is in this group), but the memory allocated under lgroup 2 (cpus 32-63 96-127) ;    and postgres in the VM 
09:27:02 <tozhu> very slow it had 96GB shared_buffers setting, I checked prstat, the LAT is usually larger than 70, what’s the problem? is there anything incorrect?   I have set echo 'pg_contig_disable/W 1' | mdb -kw  and echo 'mpss_coalesce_disable/W 1' | mdb -kw
14:48:27 <bahamat> tozhu: If you've got 100% of your cpu pinned to one NUMA domain and 100% of your memory pinned to another NUMA domain, you're going to see bad latency all around.
14:48:37 <bahamat> Try putting both in the same NUMA domain.
14:49:27 <tozhu> but there is only a postgresql database running
14:50:04 <bahamat> That doesn't matter. You've hard set it so that it needs to cross the NUMA boundary to do *anything* and *everything*.
14:50:22 <tozhu> so, I’m not sure what’s the root cause, should I bound IRQ? why memory is allocated from othere numa node
14:50:48 <bahamat> I'm not sure either because I generally don't use this aspect of illumos.
14:51:13 <bahamat> But having your memory pinned to the opposite cpus is the worst possible configuration.
14:51:13 <tozhu> for pg_contig_disable and mpss_coalesce_disable should be set to 1 to disable something?
14:51:26 <bahamat> I don't know about any of that.
14:51:40 <tozhu> I have enable NUMA in BIOS/UEFI setting, 
14:51:53 <bahamat> I just know that pinning them opposite is the most wrong way to do it.
14:54:54 <tozhu> I want to create a process set for 0-31 64-95 which are under the same numa node, then run the command psrset -c 0-31 64-95, return 1; then run psrset -e 1 vmadm start VM_UUID, then the application in the VM should allocate memory from the numa node for process set 1 ?
14:58:15 <bahamat> I don't know the mechanism you need to make sure that memory is allocated in the same NUMA domain. You'll have to read the documentation for that.
14:59:05 <bahamat> As I said, I don't usually get involved in that kind of thing. But I do know that what you've described will definitely induce an extreme level of latency for all operations.
15:10:41 <tozhu> okay, thank bahamat
15:11:00 <tozhu> have a nice weekend :)
15:59:08 <danmcd> @tozhu ==> I wonder if plgrp(1) might be able to help?  Maybe reassign your proc's CPU lgrp to match the memory?
15:59:24 <danmcd> Also for deeper reading:  https://illumos.org/books/lgrps/lgrps-ebook.pdf
16:00:20 <tozhu> danmcd: Thank you very much, I’ll had a try next monday, have a nice weekend :)
16:23:50 <tozhu> one more question: how to install node-libuuid ?  I have get error say har-validator⊙5 is no longer supported   :(      I just need the version had process.dlopen() function ; thanks in advance
16:57:39 <danmcd> I honestly don't know that one.  Perhaps pkgsrc?
18:30:30 <pjustice> I know nothing, but maybe you want node-libuuid-v2, tozhu?
23:51:17 <tozhu> pjustice: Thank you