[mkhm]

Dear foamers,

I want to use the mapFields pre-processing utility of OpenFOAM to map fields from a coarse mesh into a much refined mesh. The problem is that this seems to take too much time. I think that is not normal (more than 10 hours in serial). So I tried to speed up things by running the mapfields in parrallel. But my tries were not successful. My command line is :

mpirun -np$NSLOTS mapFields case_1 -case case_2 -sourceTime latestTime -parallel> log


and I tried to get inspiration from :
mapFields taking too long


I tried different variants of this command as :

mpirun -np$NSLOTS mapFields case_1 -case case_2-sourceTime latestTime -parallel> log

mpirun -np$NSLOTS mapFields case_1 -case case_2 -mapMethod direct -sourceTime latestTime -parallel> log

mpirun -np$NSLOTS mapFields case_1 -case case_2 -mapMethod direct -sourceTime latestTime -fields '(U T p)' -consistent -noLagrangian -parallel> log

mpirun -np$NSLOTS mapFields case_1 -case case_2 -mapMethod direct -sourceTime latestTime -fields '(U T p)' -consistent -parallel> log

I have :
Invalid option: -parallel,-noLagrangian, -fields
as errors.

Is someone could help me ?
I guess that as in the above mentioned link, they don't use the same version than me, I got other error message like
Invalid option: -fields. However, what I want to do is very basic. Just run mapFields in parallel to speed up.

I would be glad for your help.
thanks in advance,
Mary

[yambanshee]

I've had success with map fields in parallel using the following script (extracted from a submit file for a hpc) to run on OF5

mpirun -np $nproc -machinefile $PBS_NODEFILE mapFieldsPar -consistent -parallel -case mid-ke mid-ke-wallfunc > map.out

The main differences I note between our scripts is the use of mapFields vs mapFieldsPar. I have moved over to using mapFieldsPar even when doing single core mapping due to better stability.
Further more, you seem to be missing a space between -np and $NSlots
Lastly, if you think there might be a version difference, please include what version you are using

[mkhm]

Originally Posted by yambanshee

I've had success with map fields in parallel using the following script (extracted from a submit file for a hpc) to run on OF5

mpirun -np $nproc -machinefile $PBS_NODEFILE mapFieldsPar -consistent -parallel -case mid-ke mid-ke-wallfunc > map.out

The main differences I note between our scripts is the use of mapFields vs mapFieldsPar. I have moved over to using mapFieldsPar even when doing single core mapping due to better stability.
Further more, you seem to be missing a space between -np and $NSlots
Lastly, if you think there might be a version difference, please include what version you are using

Thanks for your answer. I am using the versions 4.x, 4.1 and 2.3.0 of OpenFoam. I can not find "mapFieldsPar" in none of them. Which version of OpenFoam are you using ?


Best regards,
Mary

[Geon-Hong]

Originally Posted by mkhm

Thanks for your answer. I am using the versions 4.x, 4.1 and 2.3.0 of OpenFoam. I can not find "mapFieldsPar" in none of them. Which version of OpenFoam are you using ?


Best regards,
Mary

You can find it in OF-5 or later versions.
You can also find it in OF+ versions, i.e. v1906

[louisgag]

I run a case in simpleFoam and map into a pimpleFoam case.
I have an AMI interface but I don't use it in simpleFoam.
I have both in simpleFoam and pimpleFoam periodic boundaries (cyclic).
They both run in parallel.


I'm also getting a terribly slow mapping.
Using as inspiration both this thread and that from mapFields taking too long did not really make things faster.
There is no way I get mapFieldsPar to run: I always get a seg fault.
I tried both AABB and LOD.


Only mapFields works, and it is always extremely slow, even when using mapNearest and consistent (mapFields does not allow the direct method).


So only something like this (with any mapMethod) works:

runApplication mapFields  -mapMethod mapNearest -parallelSource -parallelTarget -sourceTime latestTime -consistent  ./tmpCase

[AlxB]

Hello,
I have an issue using mapFieldsPar.

here is the command line I use in an Allrun script:

runParallel mapFieldsPar simple -case pimple -sourceTime latestTime -consistent -decomposeParDict

(ps: I've also tried without the -decomposeParDict option)

'pimple' and 'simple' are the two subfolders of the case file.
the Allrun script is located in the main folder.

everything is running smoothly except the mapFieldsPar operation.
Here is the Allrun log:

Running decomposePar on ../run/AA01p/simple
Running renumberMesh (16 processes) on ../run/AA01p/simple
Restore 0/ from 0.orig/ for processor directories
Running checkMesh (16 processes) on ../run/AA01p/simple
Running patchSummary (16 processes) on ../run/AA01p/simple
Running decomposePar on ../run/AA01p/pimple
Running renumberMesh (16 processes) ../run/AA01p/pimple
Restore 0/ from 0.orig/ for processor directories
Running checkMesh (16 processes) on ../run/AA01p/pimple
Running patchSummary (16 processes) on ../run/AA01p/pimple
Running potentialFoam (16 processes) on ../run/AA01p/simple
Running setFields (16 processes) on ../run/AA01p/simple
Running simpleFoam (16 processes) on ../run/AA01p/simple
Error getting 'numberOfSubdomains' from 'system/decomposeParDict'
Running mapFieldsPar (1 processes) on ../run/AA01p
Running reconstructParMesh on ../run/AA01p/simple
Running reconstructPar on ../run/AA01p/simple
Running pimpleFoam (16 processes) on ../run/AA01p/pimple
Running reconstructParMesh on ../run/AA01p/pimple
Running reconstructPar on ./run/AA01p/pimple

and the mapFieldsPar says:

--> FOAM FATAL ERROR: (openfoam-2012)
attempt to run parallel on 1 processor

    From static bool Foam::UPstream::init(int&, char**&, bool)
    in file UPstream.C at line 289.

FOAM aborting

#0  Foam::error::printStack(Foam::Ostream&) at ??:?
#1  Foam::error::exitOrAbort(int, bool) at ??:?
#2  Foam::UPstream::init(int&, char**&, bool) at ??:?
#3  Foam::argList::argList(int&, char**&, bool, bool, bool) at ??:?
#4  ? at ??:?
#5  __libc_start_main in /lib/x86_64-linux-gnu/libc.so.6
#6  ? at ??:?
[oli-t7610:31119] *** Process received signal ***
[oli-t7610:31119] Signal: Aborted (6)
[oli-t7610:31119] Signal code:  (-6)
[oli-t7610:31119] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7f5505cf83c0]
[oli-t7610:31119] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xcb)[0x7f5505b3718b]
[oli-t7610:31119] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x12b)[0x7f5505b16859]
[oli-t7610:31119] [ 3] /opt/OpenFOAM/OpenFOAM-v2012/platforms/linux64Gcc63DPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam5error11exitOrAbortEib+0x1f0)[0x7f5506591360]
[oli-t7610:31119] [ 4] /opt/OpenFOAM/OpenFOAM-v2012/platforms/linux64Gcc63DPInt32Opt/lib/openmpi-4.0.3/libPstream.so(_ZN4Foam8UPstream4initERiRPPcb+0xf24)[0x7f55058e5ab4]
[oli-t7610:31119] [ 5] /opt/OpenFOAM/OpenFOAM-v2012/platforms/linux64Gcc63DPInt32Opt/lib/libOpenFOAM.so(_ZN4Foam7argListC1ERiRPPcbbb+0x6cd)[0x7f55065ba6ad]
[oli-t7610:31119] [ 6] mapFieldsPar[0x429c98]
[oli-t7610:31119] [ 7] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3)[0x7f5505b180b3]
[oli-t7610:31119] [ 8] mapFieldsPar[0x42bdb7]
[oli-t7610:31119] *** End of error message ***
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node oli-t7610 exited on signal 6 (Aborted).
--------------------------------------------------------------------------

then I decided to locate the mapFieldsPar operation under the 'pimple' folder and I tried to run the following commands but got the following outputs...:

runParallel mapFieldsPar ../simple -case -sourceTime latestTime –consistent
returns: Expected 1 arguments but found 2

runParallel mapFieldsPar -consistent -case ../simple -sourceTime latestTime 
returns: Expected 1 arguments but found 0

runParallel mapFieldsPar -parallelSource -case ../simple -parallelTarget -sourceTime latestTime –consistent
returns: Expected 1 arguments but found 0
             Invalid option: -parallelSource
             Invalid option: -parallelTarget

runParallel mapFieldsPar -case ../simple -consistent -sourceTime latestTime
returns: Expected 1 arguments but found 0

please may I have some help to understand where the problem could come from ?

thank you

[louisgag]

The error seems quite clear: you have to define the number of processors in the decomposeParDict

[AlxB]

Thanks Louis,
here is my decomposeParDict:

FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    object      decomposeParDict;
}

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 16;

method          hierarchical;

simpleCoeffs
{
    n               (4 2 2);
    delta           0.001;
}

hierarchicalCoeffs
{
    n               (4 2 2);
    delta           0.001;
    order           xyz;
}

manualCoeffs
{
    dataFile        "cellDecomposition";
}

may be something wrong in it ?

[AlxB]

Hello,

is there anyone able to run mapFields with runParallel in an Allrun file ?

runParallel mapFields

So far it has not been able to catch the correct number of processors as defined in the " decomposeParDict

"Error getting 'numberOfSubdomains' from system/decomposeParDict'
Running mapFields (1 processes) on ..

is there a way to do it differently ?


thank you

[Geon-Hong]

Originally Posted by AlxB

Hello,

is there anyone able to run mapFields with runParallel in an Allrun file ?

runParallel mapFields

So far it has not been able to catch the correct number of processors as defined in the " decomposeParDict

"Error getting 'numberOfSubdomains' from system/decomposeParDict'
Running mapFields (1 processes) on ..

is there a way to do it differently ?


thank you

Why don't you execute "mpirun" command instead the runParallel script function directly?

[AlxB]

Hello,
thank you for your answer.

I got eventually some success with the following command:

runParallel mapFieldsPar ../<sourceCase> -consistent -parallel -sourceTime latestTime

Now I have a new problem:

I change the boundary conditions for that new case with an increase in rotational velocity for an object ($spinSpeed rotatingWallVelocity) and an increase in velocity for the domain inlet. (uniform $flowVelocity vector)

I start, for that new case, with a fresh 0.orig folder, immediatly copied as a 0 folder, and then proceed with mapFieldsPar to initialize the fields with the results from the previous simulation.

The updated velocity values are in the 0.orig and 0 folders.

However, when starting to run I can check that :

. the rotating wall velocity has been correctly updated
. the inlet velocity has not been correctly updated

Is there a trick to make it working ?

Should I add another command to impose the inlet velocity values ? (like setFields or other ?)

Below the U folder:

#include        "$FOAM_CASE/0.orig/include/initialConditions"
dimensions      [0 1 -1 0 0 0 0];

internalField   uniform $flowVelocity;
boundaryField
{
    #includeEtc "caseDicts/setConstraintTypes"
	inlet
	{
		type  	  fixedValue;
		value 	  $internalField;
	}
       object
       {
                type            rotatingWallVelocity;
		origin          ( 0 0 0 );
                axis            ( 0 1 0 );
		omega           $spinSpeed;
       }
}

Thank you for your help