[maka]

I'm running a channel case of 1.5 M cell devided on 4 processors on 2 machines.

I noticed I have a bad loading factor on the machine (gstat command). I checked the performance graphs of Ganglia cluster toolkit, and noticed that the momory cashed is not the same on both machines. Is it because only the pressure equation is solved in parallel. The first figure is for the machine containing the master process. Can any body notice what did I do wrong? Thanks.








------------------------------
The output of /proc/cpu:
processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 5
model name : AMD Opteron(tm) Processor 248
stepping : 10
cpu MHz : 2191.242
cache size : 1024 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 pni syscall nx mmxext lm 3dnowext 3dnow
bogomips : 4308.99
TLB size : 1088 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp
--------------------------------------

The output of decomposePar:

Processor 0
Number of cells = 393216
Number of faces shared with processor 1 = 12288
Number of faces shared with processor 3 = 12288
Number of boundary faces = 14336

Processor 1
Number of cells = 393216
Number of faces shared with processor 0 = 12288
Number of faces shared with processor 2 = 12288
Number of boundary faces = 14336

Processor 2
Number of cells = 393216
Number of faces shared with processor 1 = 12288
Number of faces shared with processor 3 = 12288
Number of boundary faces = 14336

Processor 3
Number of cells = 393216
Number of faces shared with processor 2 = 12288
Number of faces shared with processor 0 = 12288
Number of boundary faces = 14336
--------------------------------------------------

Regards,
Maka

[eugene]

All the equations are solved in parallel, but the pressure equation is by far the most communications intensive because of the number of internal loops it performs.

From your cpu_idle graphs it doesnt seem like there is a lot wrong with your setup. I would expect two dual cpu Opterons linked via Gbit ethernet to perform at around 80% efficiency given your decomp. This should translate into a per machine load of around 1.6, which is what you are getting.