[christian_kunkelmann]

Dear all,

I am running OpenFOAM 1.5 on a Debian machine with 2 quadcores. Is it possible to limit a simulation to 4 CPUs even if the number of subdomains is larger than 4?

Thanks in advance.
Christian

[fra76]

You must simply decompose on 4 subdomains and run on 4 processors, no matter how many cores you have in your machine.

Hope this helps,
Francesco

[gschaider]

Or if you want to limit your OpenFOAM-run to 4 CPUs (while doing other work with the remaining 4 cores) you might want to have a look at the man-page of the taskset-utility (or google arround for "setting the cpu-affinity of a process")

Although I don't know why you might run something in parallel with more subdomains than you have processors (except for debugging parallel stuff)

Bernhard

[christian_kunkelmann]

Francesco, thanks for the answer.

The background is that I use an adaptive mesh. Therefore it could happen that one process is much heavier than all the others (because the mesh was strongly refined in this region). A possible solution would be to decompose in a higher number of subdomains and let the "easier" processes run all together on one core.

I know that the processes will be distributed on the CPUs even if the number of subdomains is higher than the number of CPUs.

However, what if I want to decompose in, say, 8 subdomains and then run on 4 CPUs even though there are 8 CPUs in the machine (because I am not the only one to run on this machine)?

Thanks alot in advance.

Christian

[christian_kunkelmann]

Bernhardt,

thanks for the suggestions. I will have a close look at that.

Christian

[fra76]

Christian,
if you decompose on "n" processors, you must run on "n" processors. How can you solve your problem ignoring half of it?
About load unbalancing, my opinion is that the best option is to rebalance when the unbalancing becomes unacceptable. I don't know in details how to do it on the fly in OpenFOAM, but I would follow this strategy.

Francesco

[christian_kunkelmann]

Francesco,

the whole computational domain is solved, of course. I just let each CPU solve a more than one subdomain.

Thanks for the suggestion about rebalancing on the fly when the unbalancing becomes unacceptable. I had already thought about that. However, before trying this I wanted to try the other way (which was easier to realize). If the performance suffers to much, I will have to re-decompose during the calculation.

Christian

[fra76]

Ok, I understand.
You want to run 8 processes (with mpirun -np 8) but limit these processes to use only 4 cores of your machine, right?
This moves the issue from openfoam to the operating system. As Bernhard said, you must use "taskset" to force each process (8 different PIDs in your case) to use only a given list of cores (let's say, from 0 to 3). Check "man taskset" for the syntax.
However, I would avoid using this approach. 8 processes on 4 cores will be much slower than a proper 4 processes simulation. There are many reasons for this. As a quick test, try to run the same simulation on 8 cores (on your 8 cores machine) and then on 16 cores, on the same machine, and check the run time.

Hope this helps,
Francesco

[olwi]

Hi Guys,

Rebalancing after mesh refinement interests me. Doing it on the fly even more... Is that supported in OpenFOAM, somehow?

I'm currently putting the mesh refinement of interDyMFoam into a custom solver, and the load balancing is definately an issue. Any pointers appreciated.

Thanks,
Ola