[tok3rat0r]

I'm currently working on a 3D overset-grid hydrodynamic simulation with overInterDyMFoam from OpenFOAM v1706. I have the mesh set up, and the simulation runs, but I'm having problems figuring out a 'recipe' for parallel decomposition.

Each time I try a different decomposition method/number of processes, the stability and speed of the calculation seem to change pretty much at random. I've found a 12-processor hierarchical decomposition which runs reasonably well, but I'd really like to use more of the computational resources at my disposal (up to 64 cores per node with 2 nodes available)!. I tried a Scotch decomposition into 24 processes, which also seemed to start running well enough, but the overset analysis/mesh.update() stage took a painfully long time (well over 200 s, as opposed to 93 s with the 12-process hierarchical decomposition). A 16-process hierarchical decomposition was catastrophically unstable, and various other combinations have either been too unstable or run too slowly.

Are there any rules of thumb for effective decomposition of overset cases?

[Yann]

Have you tried a scotch decomposition on 12 processors?

You can't compare a 12 processors hierarchical decomposition with a 24 proc scotch one because you have no idea if the speedup/slowdown is due to the decomposition method or something related to your hardware.

[Hussamsababha]

Scotch is often not recommended in overset mesh. I'd recommend heirechal