hello all, I have created a process set for 0-31 64-95 by command: psrset -c 0-31 64-95 , and returned process set id: 1 , now I’m going to bind VM and it’s process under the process set 1, then I executed the command in GZ as root user: psrset -e 1 vmadm start VM_UUID ; I find the memory is not allocate under the lgroup 1 (process 0-31 64-95 is in this group), but the memory allocated under lgroup 2 (cpus 32-63 96-127) ; and postgres in the VM

very slow it had 96GB shared_buffers setting, I checked prstat, the LAT is usually larger than 70, what’s the problem? is there anything incorrect? I have set echo 'pg_contig_disable/W 1' | mdb -kw and echo 'mpss_coalesce_disable/W 1' | mdb -kw

tozhu: If you've got 100% of your cpu pinned to one NUMA domain and 100% of your memory pinned to another NUMA domain, you're going to see bad latency all around.

Try putting both in the same NUMA domain.

but there is only a postgresql database running

That doesn't matter. You've hard set it so that it needs to cross the NUMA boundary to do *anything* and *everything*.

so, I’m not sure what’s the root cause, should I bound IRQ? why memory is allocated from othere numa node

I'm not sure either because I generally don't use this aspect of illumos.

But having your memory pinned to the opposite cpus is the worst possible configuration.

for pg_contig_disable and mpss_coalesce_disable should be set to 1 to disable something?

I don't know about any of that.

I have enable NUMA in BIOS/UEFI setting,

I just know that pinning them opposite is the most wrong way to do it.

I want to create a process set for 0-31 64-95 which are under the same numa node, then run the command psrset -c 0-31 64-95, return 1; then run psrset -e 1 vmadm start VM_UUID, then the application in the VM should allocate memory from the numa node for process set 1 ?

I don't know the mechanism you need to make sure that memory is allocated in the same NUMA domain. You'll have to read the documentation for that.

As I said, I don't usually get involved in that kind of thing. But I do know that what you've described will definitely induce an extreme level of latency for all operations.

okay, thank bahamat

have a nice weekend :)

@tozhu ==> I wonder if plgrp(1) might be able to help? Maybe reassign your proc's CPU lgrp to match the memory?

Also for deeper reading: illumos.org/books/lgrps/lgrps-ebook.pdf

danmcd: Thank you very much, I’ll had a try next monday, have a nice weekend :)

one more question: how to install node-libuuid ? I have get error say har-validator⊙5 is no longer supported :( I just need the version had process.dlopen() function ; thanks in advance

I honestly don't know that one. Perhaps pkgsrc?

I know nothing, but maybe you want node-libuuid-v2, tozhu?

pjustice: Thank you