[MaryBau]

Hi everyone;

I have been having problems with the time average values of my simulations when I ran them in the cluster that I have always used. This problem started a couple weeks ago, but before that they were running fine. I talked to the people in charge of the cluster, they think it might be an OpenFoam issue. I am using OF v2.1.0

The simulations seem to run fine, I do not get any error message, but looking at the results, it seems that the fieldAverage is not working. I did not notice any big difference in the amount of time it takes for the simulations to run before and after this problem. And the output shows that the averages are being calculated.

I ran exactly the same case in my local computer and in the cluster and I get very different results. The only difference in these simulations is the number of cores I use, 6 at my local computer and 32 cores in the cluster, but it should not be important. I am attaching two plots of these results. My local computer shows results that are smooth and "averaged" and the cluster plot shows results that do not seem to be averaged.

Any ideas of what could it be the problem? or what could have changed in the cluster to make the fieldAverage stop working properly?

This is what I put in controlDict:

functions
(
    fieldAverage1
    {
        type            fieldAverage;
        functionObjectLibs ( "libfieldFunctionObjects.so" );
        enabled         true;
        cleanRestart    false;
        outputControl   outputTime;
        resetOnOutput   false;
        timeStart       7.0;
        fields
        (
            U
            {
                mean        on;
                prime2Mean  on;
                base        time;//or iteration
            }
        );
     }
......

Thanks a lot,

Mary

[wyldckat]

Greetings Mary,

Well, the number of sub-domains does matter if there is a division in a location where it isn't working as intended between processor domains.

In addition, the only bug that seems to have been reported regarding "fieldAverage" for OpenFOAM 2.1 is this one: http://www.openfoam.org/mantisbt/view.php?id=818 - but it refers to continuing the simulation from another time location.

Assuming you cannot share the case, here are a few questions:

1. Does your case have any special boundary conditions? Such as cyclic patches?
2. What decomposition method are you using? Have you tried the "simple" or "hierarchical" methods?
3. How many cells does your case have got?
4. Why kind of simulation is it being performed?
5. You said that the problem only started some time ago. What else has changed, besides this result?

Best regards,
Bruno

[MaryBau]

Hi Bruno;

Thanks for replying. The bug sounds a lot like what I am getting in my results.

So the problem is in the totalTime in fieldAverageProperties when the "timeStart" option is used in fieldAverage, right? I guess the work around will be start the simulation up to a certain time without averaging and then stopped it. Then add fieldAverage with a cleanRestart in controlDict and restart the simulation this time with the averaging. Am I correct? will this avoid the problem?

And just to answer your questions:

-I am running a simple channel flow case but with a hybrid model.
1. Cyclic on inlet/outlet and laterals.
2. Decompostion: hierarchical. I had some instabilities at the subdomain boundaries when using scotch, that was making my simulations crashed.
3. I have less than 1 million cells in all my cases.
4. Channel-flow with high Re to simulate the earth atmosphere.
5. I guess the problem started when I added the "timeStart" flag in fieldAverage and/or varied the time at which I started the averaging and the error became more evident.

Thanks a lot,

Mary

[wyldckat]

Hi Mary,

Originally Posted by MaryBau

So the problem is in the totalTime in fieldAverageProperties when the "timeStart" option is used in fieldAverage, right? I guess the work around will be start the simulation up to a certain time without averaging and then stopped it. Then add fieldAverage with a cleanRestart in controlDict and restart the simulation this time with the averaging. Am I correct? will this avoid the problem?

In theory, yes, that should avoid the problem. I say in theory, because OpenFOAM sometimes has a couple of surprises for us.
For example, the folder "uniform" that usually appears in the time folders will register the time it is meant for, so you should also erase that folder when you continue your case from 0.

I haven't looked into the attached file you provided, since the problem seems to have be solved this way. Nonetheless, if it does not, then let me know what steps to take to run the case and I'll give it a try with the latest OpenFOAM 2.2.x vs the old 2.1.0.

Best regards,
Bruno

[MaryBau]

Hi Bruno and all;

I am still trying to figure out this average issue that I mentioned on my previous posts.

Since there is a bug in "fieldAverage" in v2.1., I started a simulation up to a certain time (without averaging). Then add fieldAverage in controlDict and restart the simulation this time with the averaging. The average seems to be working fine like this.

But now my problem is that all the instantaneous fields give me very different results if I stopped the simulation (due to the average issue) or if I do not stopped it. See the attached images for the value of the sub grid turbulence kinetic energy when the simulation is stopped and when it is not, there are taken at the same timeStep. All the instantaneous fields seem to show the same behaviour. I have tried a couple of different cases and stopping at different times...

I am saving as binary with a writePrecision of 12. Any ideas of why this is happening?

Thanks again and have a good weekend,

Mary

[wyldckat]

Hi Mary,

Honestly, I believe that you should either try to build OpenFOAM 2.1.x or 2.2.2 or 2.2.x.

I've given a really quick look into the case you've provided and I wasn't expecting so many special requirements by the case, namely the libraries needed and the custom solver.

If you can provide a slightly modified tutorial case from OpenFOAM 2.1.1 in which this is reproducible (i.e., simple and without special requirements), I can run it in any and all OpenFOAM versions, to ascertain in which version it works properly.

Best regards,
Bruno

[MaryBau]

Hi Bruno and all;

It took me a long time to figure this out and therefore to reply. I ended up installing v2.2.2 and I am running my simulations with it now. So the average problem is fixed.

The other problem that I had with the instantaneous fields that give me very different results if I stopped/rerun the simulation turned out to be a bug in my turbulence model code. But it had nothing to do with the average and/or OF version.

Just want to make sure the answer is in the forums in case someone has the same problem.

Thanks for your help,

Mary

[MaryBau]

Hi again;

I do not think the fieldAverage bug mentioned by Bruno on post #2 is completly fixed in v2.2.2. I had verified it and I thought it was fixed, but apperently I was wrong and I am still having issues with averaging the fields.

At least when I use pimpleFoam with and adjustableRunTime, the value of the "totalTime" in uniform/fieldAveragingProperties file is:

totalTime = deltaT(in controlDict)+time step

instead of just:

totalTime = time step

so if the value of deltaT(in controlDict) is really big compared to the actual time step in the simulations, the averages are biased.

I have implemented the averages a bit different in my solver, but I just wanted to let you guys know.

Have a nice day,

Mary

[mechem]

I use OpenFOAM 2.3 and time averaging does not work for MPPICFoam/Goldschemidt when putting below code to its controldict. Any idea?

 fieldAverage1
    {
        type            fieldAverage;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   outputTime;

        fields
        (
            U.air
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }

            p
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }
        );
    }

[wyldckat]

Greetings Charlie,

It does work. It creates the fields "U.airMean", "U.airPrime2Mean", "pMean" and "pPrime2Mean".

Best regards,
Bruno

[mechem]

But it doesn't create for me!
Maybe somewhere else of controlDict should be changed. All of controlDict:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application     MPPICFoam;

startFrom       startTime;

startTime       0;

stopAt          endTime;

endTime         5;

deltaT          2e-4;

writeControl    runTime;

writeInterval   0.01;

purgeWrite      0;

writeFormat     ascii;

writePrecision  6;

writeCompression uncompressed;

timeFormat      general;

timePrecision   6;

runTimeModifiable yes;

 fieldAverage1
    {
        type            fieldAverage;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   outputTime;

        fields
        (
            U.air
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }

            p
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }
        );
    }

// ************************************************************************* //

[wyldckat]

Hi Charlie,

Looks like you didn't carefully read the very first post on this thread. It clearly shows that this function object must be placed inside a "functions" entry:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application     MPPICFoam;

startFrom       startTime;

startTime       0;

stopAt          endTime;

endTime         5;

deltaT          2e-4;

writeControl    runTime;

writeInterval   0.01;

purgeWrite      0;

writeFormat     ascii;

writePrecision  6;

writeCompression uncompressed;

timeFormat      general;

timePrecision   6;

runTimeModifiable yes;

functions
{
    fieldAverage1
    {
        type            fieldAverage;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   outputTime;

        fields
        (
            U.air
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }

            p
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }
        );
    }

}
// ************************************************************************* //

Best regards,
Bruno

[mechem]

Originally Posted by wyldckat

Hi Charlie,

Looks like you didn't carefully read the very first post on this thread. It clearly shows that this function object must be placed inside a "functions" entry:

/*--------------------------------*- C++ -*----------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      controlDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

application     MPPICFoam;

startFrom       startTime;

startTime       0;

stopAt          endTime;

endTime         5;

deltaT          2e-4;

writeControl    runTime;

writeInterval   0.01;

purgeWrite      0;

writeFormat     ascii;

writePrecision  6;

writeCompression uncompressed;

timeFormat      general;

timePrecision   6;

runTimeModifiable yes;

functions
{
    fieldAverage1
    {
        type            fieldAverage;
        functionObjectLibs ("libfieldFunctionObjects.so");
        enabled         true;
        outputControl   outputTime;

        fields
        (
            U.air
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }

            p
            {
                mean        on;
                prime2Mean  on;
                base        time;
            }
        );
    }

}
// ************************************************************************* //

Best regards,
Bruno

You are right. I forgot to add function.
Thanks.

[kiddmax]

Hi Mary,

I saw your attached file "averageCluster.pdf ". The plot is very beautiful and professional. May I ask you what plot software do you use? gnuplot or others??

Regards
Ye

[MaryBau]

Hi Ye,

A couple months later, but I just saw your question, sorry. I used Matplotlib to do the plot.

Cheers