CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM

Parallel running stops when sample point sets z=(0,0) using LES

Register Blogs Community New Posts Updated Threads Search

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   May 31, 2023, 11:57
Default Parallel running stops when sample point sets z=(0,0) using LES
  #1
New Member
 
Teresa Sun
Join Date: Jun 2021
Posts: 21
Rep Power: 5
Teresa Sun is on a distinguished road
Dear Foamer,

Can anyone tell me what's going on?

Simulation case: circular cylinder at Re=3900 using LES with the customized model (minimum-dissipation model).

The problem happened when sampling along the central line in a streamwise direction, i.e., y=(0,0) and z=(0,0)
It works with no problem while running on a single core. But the simulation stops while running in parallel on 128, 64 and 32 cores using simple or scotch decomposition methods. There is no error message and the simulation doesn't exit.

This problem can be solved by changing z=(0,0) to z=(0,1e-5), (using a small non-zero value).

Why this problem occurs and why it is solved in this way?

Code:
controlDict

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  6                                     |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    object      controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application     RKSymLESFoam;

//startFrom       startTime;
startFrom       latestTime;
startTime       0;
stopAt          endTime;
endTime         500;
deltaT          0.002;
writeControl    adjustableRunTime;
writeInterval   20;
purgeWrite      50;
writeFormat     ascii;
writePrecision  8;
writeCompression off;
timeFormat      general;
timePrecision   6;
runTimeModifiable yes;

//Not implemented in PISO
adjustTimeStep  yes;
maxCo           0.5;
maxDeltaT       0.1;

// ************************************************************************* //

functions
{

    inMassFlow
    {
        type            surfaceFieldValue;
        libs            ("libfieldFunctionObjects.so");
        enabled         true;
        timeStart       100;
        writeControl    outputTime;
        writeInterval   1;
        log             true;
        writeFields     false;
        regionType      patch;
        name      in;
        operation       sum;
        fields
        (
            phi
        );
    }
    outMassFlow
    {
        type            surfaceFieldValue;
        libs            ("libfieldFunctionObjects.so");
        enabled         true;
        timeStart       100;
        writeControl    outputTime;
        writeInterval   1;
        log             yes;
        writeFields     false;
        regionType      patch;
        name            out;
        operation       sum;
        fields
        (
            phi
        );
    }
    fieldAverage
    {
        type            fieldAverage;
        libs            ("libfieldFunctionObjects.so");
        enabled         true;
        timeStart       100;
        writeControl    timeStep;
        //writeInterval 100;

        fields
        (
            U
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }

            p
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }
        );
    }

    forceCoeffs_object
    {
        // rhoInf - reference density
        // CofR - Centre of rotation
        // dragDir - Direction of drag coefficient
        // liftDir - Direction of lift coefficient
        // pitchAxis - Pitching moment axis
        // magUinf - free stream velocity magnitude
        // lRef - reference length
        // Aref - reference area
        type forceCoeffs;
        libs ("libforces.so");
        //patches ("body1" "body2" "body3");
        patches ("cylinder");

        pName p;
        Uname U;
        rho rhoInf;
        rhoInf 1.0;

        //// Dump to file
        log true;

        CofR            (0.0 0 0);
        liftDir         (0 0.5 0);
        dragDir         (0.5 0 0);
        pitchAxis       (0 0 0.5);
        magUInf         1.0;
        lRef            1.0;         // reference lenght
        Aref            3.1415926;         // reference area 1 for 2d
        writeControl    timeStep;
        timeStart       100;
        writeInterval   1;
    }

    minmaxdomain
    {
        type            fieldMinMax;
        libs            ("libfieldFunctionObjects.so");
        enabled         true;
        mode            component;
        timeStart       100;
        writeControl    timeStep;
        writeInterval   1;
        log true;
        fields (p U nut);
    }


    turbulenceFields1
    {
        type            turbulenceFields;
        libs            (fieldFunctionObjects);
        fields          (I devReff nut nuEff L);
        //fields          ( k  R I devReff nut nuEff L);
        executeControl  writeTime;
        timeStart       100; // approx 2x flow-throughs
        writeControl    writeTime;
    }


    Q1
    {
        type            Q;
        libs            (fieldFunctionObjects);
        timeStart       100; // approx 2x flow-throughs
        writeControl    writeTime;
    }

    momErr
    {
        type            momentumError;
        libs            (fieldFunctionObjects);
        timeStart       100; // approx 2x flow-throughs
        writeControl    writeTime;
    }
    vorticity1
    {
        type            vorticity;
        libs            (fieldFunctionObjects);
        timeStart       100; // approx 2x flow-throughs
        writeControl    writeTime;
    }

    sample
    {
        type                sets;
        libs                (sampling);
        interpolationScheme cellPoint;
        setFormat           raw;
        executeControl      writeTime;
        writeControl        writeTime;

        //extract the profile along y axi x=constant,z=0m
        Ymin                    -3;
        Ymax                    3;
        Z                       0;
        D                       1;
        yPoints                 80;

        sets
        (
            X_by_D_1_06
            {
                type    face;
                axis    y;
                start   (#eval{1.06 * $D} $Ymin $Z);
                end     (#eval{1.06 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_1_54
            {
                type    face;
                axis    y;
                start   (#eval{1.54 * $D} $Ymin $Z);
                end     (#eval{1.54 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_2_02
            {
                type    face;
                axis    y;
                start   (#eval{2.02 * $D} $Ymin $Z);
                end     (#eval{2.02 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_6
            {
                type    face;
                axis    y;
                start   (#eval{6.0 * $D} $Ymin $Z);
                end     (#eval{6.0 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_7
            {
                type    face;
                axis    y;
                start   (#eval{7.0 * $D} $Ymin $Z);
                end     (#eval{7.0 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_10
            {
                type    face;
                axis    y;
                start   (#eval{10.0 * $D} $Ymin $Z);
                end     (#eval{10.0 * $D} $Ymax $Z);
                nPoints $yPoints;
            }
            X_by_D_centralLine
            {
                type    midPoint;
                axis    x;
                start   (-1   0 0);
                end     (20.0 0 0.00001);
                nPoints 200;
            }

        );

        fields
        (
                k nut p pPrime2Mean R I devReff Q
                vorticity
                U UMean UPrime2Mean grad(U)        
        );
   }

};


Code:
fvSchemes
/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  6                                     |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    object      fvSchemes;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

ddtSchemes
{
    //default         Euler;
    default         backward;
    //default         CrankNicolson 0.0;
}

gradSchemes
{
    default         Gauss linear;
    grad(p)         Gauss linear;

    //default        leastSquares;

    //default        cellMDLimited leastSquares 0.5;
    //default        cellMDLimited Gauss linear 1;
    //grad(U)        cellMDLimited Gauss linear 1;
}

divSchemes
{
    default         none;

    div(phi,U)      Gauss linear;

    //div(phi,U)      Gauss filteredLinear;
    //div(phi,U)      Gauss limitedLinearV 1;
    //div(phi,U)      Gauss LUST unlimitedGrad(U);
    //div(phi,U)      Gauss linearUpwind grad(U);

    //div(phi,U)      Gauss linearUpwindV default;
    //div(phi,U)      Gauss linearUpwindV grad(U);

    div((nuEff*dev2(T(grad(U))))) Gauss linear;

    //div(phi,k)                Gauss linearUpwind default;
    //div(phi,omega)    Gauss linearUpwind default;

    //div(phi,k)                Gauss linear;
    //div(phi,omega)    Gauss linear;

    //div(phi,k)                Gauss upwind;
    //div(phi,omega)    Gauss upwind;

}

laplacianSchemes
{
     default         Gauss linear corrected;
}

interpolationSchemes
{
    default         linear;
}

snGradSchemes
{
    default         limited 1;
}

// ************************************************************************* //
Code:
fvSolution

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  6                                     |
|   \\  /    A nd           | Web:      www.OpenFOAM.com                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    object      fvSolution;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

solvers
{
/*
    p
    {
        solver          PCG;
        preconditioner  DIC;
        tolerance       1e-06;
        relTol          0;

        minIter         2;
    }
*/

    p
    {
        solver           GAMG;
        tolerance        1e-6;
        relTol           0.01;
        smoother         GaussSeidel;
        nPreSweeps       0;
        nPostSweeps      2;
        cacheAgglomeration on;
        agglomerator     faceAreaPair;
        nCellsInCoarsestLevel 100;
        mergeLevels      1;

        minIter         2;
    }

    pFinal
    {
        solver          PCG;
        preconditioner  DIC;
        tolerance       1e-06;
        relTol          0;

        minIter         3;
    }

    U
    {
        solver          PBiCG;
        preconditioner  DILU;
        tolerance       1e-08;
        relTol          0;
        minIter         3;
    }

/*
    U
    {
        type            coupled;
        solver          PBiCCCG;
        preconditioner  DILU;
        tolerance       (1e-08 1e-08 1e-08);
        relTol          (0 0 0);
    }
*/


    UFinal
    {
        solver          PBiCG;
        preconditioner  DILU;
        tolerance       1e-08;
        relTol          0;
        minIter         3;
    }


}

PISO
{
    nCorrectors     2;
    nNonOrthogonalCorrectors 1;
    pRefCell        0;
    pRefValue       0;
}

// ************************************************************************* //
Attached Files
File Type: txt blockMeshDict.txt (10.2 KB, 0 views)
Teresa Sun is offline   Reply With Quote

Old   May 31, 2023, 12:06
Unhappy
  #2
New Member
 
Teresa Sun
Join Date: Jun 2021
Posts: 21
Rep Power: 5
Teresa Sun is on a distinguished road
Intended to post this in the sub-forum OpenFOAM Running, Solving & CFD.

How to edit it or delete
Teresa Sun is offline   Reply With Quote

Reply

Tags
les, parallel running, sample


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
[snappyHexMesh] reconstructPar after running snappyhexmesh in parallel fidu OpenFOAM Meshing & Mesh Conversion 11 October 20, 2020 10:01
Something weird encountered when running OpenFOAM in parallel on multiple nodes xpqiu OpenFOAM Running, Solving & CFD 2 May 2, 2013 05:59
Running in parallel Djub OpenFOAM Running, Solving & CFD 3 January 24, 2013 17:01
Kubuntu uses dash breaks All scripts in tutorials platopus OpenFOAM Bugs 8 April 15, 2008 08:52
[Gmsh] Gmsh and samplesurface touf OpenFOAM Meshing & Mesh Conversion 2 December 10, 2007 03:27


All times are GMT -4. The time now is 12:20.