[yeharav]

Hi,

I posted in a different thread, but the title (and openfoam version) are no informative so I decided to open a new thread.

The domain is a simple box and
I use a hierarchical decomposition into 32 processors.

I use my own solver based on the buoyant boussinesq solver.
The simulation runs for a while with adaptable time step.
The average time step is approximately 0.23.

At ~50sec the parallel solver breaks.
That is, courant number jumps to 1e11, the same with the pressure.
When the simulation breaks I also tried using small fixed time steps (1e-3) in parallel, but to no avail.

I reconstruct and run in single and it passes. Then I decompose and run in parallel
and it runs for a while before it breaks (reaches 70).

I have tried to increase the diffusion and change the decomposition but it didn't change
the problem (just moved it to different times).

I can post the configuration files (and the solver) if it helps.

[FlyingCat]

Hello,

I have the same problem (I think) on OpenFOAM-dev and OpenFOAM-4.0.

I think there is a problem with the decomposition. It seems that the decomposition of OpenFOAM is not very strong.

On a little case, the calculation was lower with more processors (even though it is possible if this is a little mesh). I changed the method of decomposition and I used scotch method, and it ran very well, quicker than with one processor. So it depends of the method of decomposition (in the decomposeParDict).

Now, I have the same problem with a bigger mesh but with scotch method. It is quicker with one processor than with 16.

Try to change the method, the problem could come from there.

[yeharav]

Hi,

Thanks, Yes it is a good idea.

My investigations so far have led me to think that it is the mesh quality,
but it could be the mesh decomposition as well.

When I refine my mesh and limit Co to 0.05 (with adaptive time step) things work
(but very slowly).

[FlyingCat]

Yes, perhaps it is both. It is because of the mesh quality that the decomposition is bad. But in my case, I have already 700K cells and it is a structured grid, so I do not think it is the mesh quality (at worst, it is possible for the refinement but I will see).

[yeharav]

Did you get this behaviour with a native OF solver (which one?)
Did you use swak4foam?

[FlyingCat]

Yes, it is with fireFoam. And no, I do not use swak4foam.

[FlyingCat]

See this thread :

Parallel Performance of Large Case

Do you use the GAMG solver ? I will try to run my case without this solver to see the results.

EDIT: I changed the solver but it is the same, still very low in parallel. I tried a lot of things concerning the method of decomposition and it does not work neither.