[WhiteW]

Hi,
I'm trying to run in parallel an anlysis with a MRF zone and mixing plane interface, I'm using Foam extend 4.1.
However I'm not having advanteages in parallelize the analysis.
I read that for cyclicGGI zones and Mixing Plane I need to use the patchConstrained Method.
My decomposeParDict is (in the case of 4 processors):

numberOfSubdomains 4;

method    patchConstrained;

globalFaceZones
(
    RUCYCLIC1Zone
    RUINLETZone
    RUCYCLIC2Zone
    RUOUTLETZone
    GVOUTLETZone
    DTINLETZone
    DTCYCLIC1Zone
    DTCYCLIC2Zone
);

patchConstrainedCoeffs
{
    method            metis;
    numberOfSubdomains    4;
    patchConstraints
    (
        (RUINLET 0)
        (GVOUTLET 0)
        (RUOUTLET 1)
        (DTINLET 1)
    );
}

metisCoeffs
{
    processorWeights
    (
        1
        1
        1
        1
    );
}


distributed     no;


// ************************************************************************* //

The scalability of the analysis seems very bad, the fastest run is using one core.
I run the multi-processor analysis with the comand:

mpirun -np 8 MRFSimpleFoam > log.MRFSimpleFoam_np08

The attached scalability analysis is refered tot he MRFSimpleFoam axial Turbine tutorial, however the same scalability can be observed with all the other axialTurbine tutorials of Foam extended.

Do you have some advise to speed up the analysis by decomposing the domain? Am I doing something wrong in the decomposeParDict setting?

Thanks,
WhiteW

[Santiago]

Originally Posted by WhiteW

Hi,
I'm trying to run in parallel an anlysis with a MRF zone and mixing plane interface, I'm using Foam extend 4.1.
However I'm not having advanteages in parallelize the analysis.
I read that for cyclicGGI zones and Mixing Plane I need to use the patchConstrained Method.
My decomposeParDict is (in the case of 4 processors):

numberOfSubdomains 4;

method    patchConstrained;

globalFaceZones
(
    RUCYCLIC1Zone
    RUINLETZone
    RUCYCLIC2Zone
    RUOUTLETZone
    GVOUTLETZone
    DTINLETZone
    DTCYCLIC1Zone
    DTCYCLIC2Zone
);

patchConstrainedCoeffs
{
    method            metis;
    numberOfSubdomains    4;
    patchConstraints
    (
        (RUINLET 0)
        (GVOUTLET 0)
        (RUOUTLET 1)
        (DTINLET 1)
    );
}

metisCoeffs
{
    processorWeights
    (
        1
        1
        1
        1
    );
}


distributed     no;


// ************************************************************************* //

The scalability of the analysis seems very bad, the fastest run is using one core.
I run the multi-processor analysis with the comand:

mpirun -np 8 MRFSimpleFoam > log.MRFSimpleFoam_np08

The attached scalability analysis is refered tot he MRFSimpleFoam axial Turbine tutorial, however the same scalability can be observed with all the other axialTurbine tutorials of Foam extended.

Do you have some advise to speed up the analysis by decomposing the domain? Am I doing something wrong in the decomposeParDict setting?

Thanks,
WhiteW

you forgot the -parallel flag when running your code.!! basically you are running 8 serial instances of the MRFSimpleFoam solver!

[WhiteW]

Hi Santiago,
thanks for your reply.
I did not used the -parallel option because it did not work well in my OF.

using:

mpirun -np 2 MRFSimpleFoam -parallel >log.MRFSimpleFoam

the code stops (without errors) after these prints:

Time = 1

BiCGStab:  Solving for Ux, Initial residual = 1, Final residual = 0.00286063, No Iterations 1
BiCGStab:  Solving for Uy, Initial residual = 1, Final residual = 0.00320029, No Iterations 1
BiCGStab:  Solving for Uz, Initial residual = 1, Final residual = 0.00224373, No Iterations 1
BiCGStab:  Solving for p, Initial residual = 1, Final residual = 0.0484594, No Iterations 25
BiCGStab:  Solving for p, Initial residual = 0.377316, Final residual = 0.0180105, No Iterations 47
time step continuity errors : sum local = 0.2369, global = -0.0946146, cumulative = -0.0946146
Initializing the mixingPlane interpolator between master/shadow patches: GVOUTLET/RUINLET

The two processor are active but the simulation does not procede.

using:

mpirun -np 3 MRFSimpleFoam -parallel >log.MRFSimpleFoam

I get the following error:

Create time

Create mesh for time = 0

Initializing the GGI interpolator between master/shadow patches: RUCYCLIC1/RUCYCLIC2
Initializing the GGI interpolator between master/shadow patches: DTCYCLIC1/DTCYCLIC2
Initializing the mixingPlane interpolator between master/shadow patches: RUOUTLET/DTINLET
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------

The only decomposition that works is with 4 processor:

mpirun -np 4 MRFSimpleFoam -parallel >log.MRFSimpleFoam

and actually is the faster one.
What could be the cause of the other erros?
Thanks,
WhiteW