[Carlen]

Hi foamers,

I am facing a problem of mapFields taking too long to map fields from a fine mesh to a coarser mesh. I am running simulation using sprayFoam, and want to coarse the mesh to achieve faster solution. So I executed:

mpirun -n 4 mapFields ../aachenBomb-fine -sourceTime latestTime  -consistent -parallel

in the aacheBomb-coarse directory and it is running without errors. However, it gets stuck at here for four hours:

 Create meshes

Source mesh size: 42025    Target mesh size: 42025


Consistently creating and mapping fields for time 0.0004

Creating mesh-to-mesh addressing for region0 and region0 regions using cellVolumeWeight
    Overlap volume: 4e-05
Creating AMI between source patch walls and target patch walls using faceAreaWeightAMI
    AMI: Creating addressing and weights between 19762 source faces and 19762 target faces
    AMI: Patch source sum(weights) min/max/average = 1, 1, 1
    AMI: Patch target sum(weights) min/max/average = 1, 1, 1
    interpolating mut
    interpolating rho
    interpolating PaSR:kappa
    interpolating k
    interpolating H2O
    interpolating PaSR:dQ
    interpolating alphat
    interpolating N2
    interpolating O2
    interpolating dQ
    interpolating p
    interpolating T
    interpolating epsilon
    interpolating U

    processing cloud "sprayCloud"

Anybody knows what is wrong?

kind regards,
carlen

PS: I am using OF2.3.1

[wyldckat]

Quick answer: http://www.openfoam.org/mantisbt/view.php?id=1326#c3694

Originally Posted by mattijs

The parallel mapFields uses conservative mapping ('cellVolumeWeight') by default which is slow since it has to do full tet-tet overlap calculations. The old behaviour is like the 'direct' method. Does using this method help?

[Carlen]

Hi Bruno,
Thank you for kindly moving my thread to an appropriate section. I will try the direct method and let you know if the execution time is reduced.

Best regards,
carlen

[Carlen]

Hi bruno,

I have tried using direct mapMethod and now it gets to this state much faster (2s):

Source time: 0.0004
Target time: 0

Create meshes

Source mesh size: 42025	Target mesh size: 42025


Consistently creating and mapping fields for time 0.0004

Creating mesh-to-mesh addressing for region0 and region0 regions using direct
    Overlap volume: 4e-05
Creating AMI between source patch walls and target patch walls using directAMI
    AMI: Creating addressing and weights between 19762 source faces and 19762 target faces
    AMI: Patch source sum(weights) min/max/average = 1, 1, 1
    AMI: Patch target sum(weights) min/max/average = 1, 2, 1.0022
    interpolating mut
    interpolating rho
    interpolating PaSR:kappa
    interpolating k
    interpolating H2O
    interpolating PaSR:dQ
    interpolating alphat
    interpolating N2
    interpolating O2
    interpolating dQ
    interpolating p
    interpolating T
    interpolating epsilon
    interpolating U

    processing cloud "sprayCloud"

However, it still remains at such a state for way too long. More strangely, if I change the executing line from:

mpirun -n 4 mapFields ../aachenBomb-twoparcels -mapMethod direct -sourceTime latestTime -consistent -parallel

to

mpirun -n 4 mapFields ../aachenBomb-twoparcels -mapMethod direct  -consistent -parallel

and I modify the startTime in the controlDict of the source case to 0.0004. it finishes almost immediately but without showing

processing cloud "sprayCloud"

in the terminal window.I guess the sprayCloud properties are not mapped.

How can I solve this problem?

[Carlen]

Hi everyone,

I think I found a solution for this problem, I now type in

mpirun -n 4 mapFields ../aachenBomb-twoparcels -mapMethod direct -sourceTime latestTime -consistent -noLagrangian -parallel

Source time: 0.0004
Target time: 0

Create meshes

Source mesh size: 42025	Target mesh size: 42025


Consistently creating and mapping fields for time 0.0004

Creating mesh-to-mesh addressing for region0 and region0 regions using direct
    Overlap volume: 4e-05
Creating AMI between source patch walls and target patch walls using directAMI
    AMI: Creating addressing and weights between 19762 source faces and 19762 target faces
    AMI: Patch source sum(weights) min/max/average = 1, 1, 1
    AMI: Patch target sum(weights) min/max/average = 1, 2, 1.0022
    interpolating mut
    interpolating rho
    interpolating PaSR:kappa
    interpolating k
    interpolating H2O
    interpolating PaSR:dQ
    interpolating alphat
    interpolating N2
    interpolating O2
    interpolating dQ
    interpolating p
    interpolating T
    interpolating epsilon
    interpolating U

End

Finalising parallel run

It runs without mapping the lagrangian fields which is actually what I wanted in the first place (not allowing sprayParcels to be mapped)

Cheers,

[Carlen]

Carlen again,
I found that it is problematic to run simulation for the mapped case without mapping the lagrangian fields. it gives so many errors that I cannot find a solution for.

[[A[carlen@localhost aachenBomb-mapFields]$ mpirun -carlenSprayFoam -parallel
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.1                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.1-262087cdf8db
Exec   : carlenSprayFoam -parallel
Date   : Mar 30 2015
Time   : 15:14:11
Host   : "localhost.localdomain"
PID    : 13054
Case   : /home/carlen/OpenFOAM/carlen-2.3.1/carlenVOFLPT/run/aachenBomb-mapFields
nProcs : 4
Slaves : 
3
(
"localhost.localdomain.13055"
"localhost.localdomain.13056"
"localhost.localdomain.13057"
)

Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0


Reading g
Creating combustion model

Selecting combustion model PaSR<psiChemistryCombustion>
Selecting chemistry type 
{
    chemistrySolver ode;
    chemistryThermo psi;
}

Selecting thermodynamics package 
{
    type            hePsiThermo;
    mixture         reactingMixture;
    transport       sutherland;
    thermo          janaf;
    energy          sensibleEnthalpy;
    equationOfState perfectGas;
    specie          specie;
}

Selecting chemistryReader chemkinReader
Reading CHEMKIN thermo data in new file format
[2] #0  Foam::error::printStack(Foam::Ostream&)[0] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
[2] #1  Foam::sigFpe::sigHandler(int) at ??:?
[0] #1  Foam::sigFpe::sigHandler(int) at ??:?
[2] #2  
 at ??:?
[0] #2  
[2]  at sigaction.c:?
[2] #3  double Foam::max<Foam::fvPatchField, double>(Foam::FieldField<Foam::fvPatchField, double> const&)[0]  at sigaction.c:?
[0] #3   at ??:?
[2] #4  double Foam::max<Foam::fvPatchField, double>(Foam::FieldField<Foam::fvPatchField, double> const&)Foam::dimensioned<double> Foam::max<double, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&) at ??:?
[0] #4   at ??:?
[2] #5  Foam::multiComponentMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::correctMassFractions()Foam::dimensioned<double> Foam::max<double, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&) at ??:?
[2] #6  Foam::multiComponentMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::multiComponentMixture(Foam::dictionary const&, Foam::List<Foam::word> const&, Foam::HashPtrTable<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> >, Foam::word, Foam::string::hash> const&, Foam::fvMesh const&) at ??:?
[0] #5  Foam::multiComponentMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::correctMassFractions() at ??:?
[2] #7  Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::reactingMixture(Foam::dictionary const&, Foam::fvMesh const&) at ??:?
[0] #6  Foam::multiComponentMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::multiComponentMixture(Foam::dictionary const&, Foam::List<Foam::word> const&, Foam::HashPtrTable<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> >, Foam::word, Foam::string::hash> const&, Foam::fvMesh const&) at ??:?
[2] #8  Foam::heThermo<Foam::psiReactionThermo, Foam::SpecieMixture<Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > >::heThermo(Foam::fvMesh const&, Foam::word const&) at ??:?
[0] #7  Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::reactingMixture(Foam::dictionary const&, Foam::fvMesh const&) at ??:?
[2] #9  Foam::psiReactionThermo::addfvMeshConstructorToTable<Foam::hePsiThermo<Foam::psiReactionThermo, Foam::SpecieMixture<Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > > >::New(Foam::fvMesh const&, Foam::word const&) at ??:?
[0] #8  Foam::heThermo<Foam::psiReactionThermo, Foam::SpecieMixture<Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > >::heThermo(Foam::fvMesh const&, Foam::word const&) at ??:?
[2] #10  Foam::autoPtr<Foam::psiReactionThermo> Foam::basicThermo::New<Foam::psiReactionThermo>(Foam::fvMesh const&, Foam::word const&) at ??:?
[0] #9  Foam::psiReactionThermo::addfvMeshConstructorToTable<Foam::hePsiThermo<Foam::psiReactionThermo, Foam::SpecieMixture<Foam::reactingMixture<Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > > >::New(Foam::fvMesh const&, Foam::word const&) at ??:?
[2] #11  Foam::psiReactionThermo::New(Foam::fvMesh const&, Foam::word const&) at ??:?
[0] #10  Foam::autoPtr<Foam::psiReactionThermo> Foam::basicThermo::New<Foam::psiReactionThermo>(Foam::fvMesh const&, Foam::word const&) at ??:?
[2] #12  Foam::psiChemistryModel::psiChemistryModel(Foam::fvMesh const&) at ??:?
[0] #11  Foam::psiReactionThermo::New(Foam::fvMesh const&, Foam::word const&) at ??:?
[2] #13  Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::chemistryModel(Foam::fvMesh const&) at ??:?
[2] #14  Foam::ode<Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > >::ode(Foam::fvMesh const&) at ??:?
[0] #12  Foam::psiChemistryModel::psiChemistryModel(Foam::fvMesh const&) at ??:?
[2] #15  Foam::psiChemistryModel::addfvMeshConstructorToTable<Foam::ode<Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > >::New(Foam::fvMesh const&) at ??:?
[0] #13  Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > >::chemistryModel(Foam::fvMesh const&) at ??:?
[2] #16  Foam::autoPtr<Foam::psiChemistryModel> Foam::basicChemistryModel::New<Foam::psiChemistryModel>(Foam::fvMesh const&) at ??:?
[0] #14  Foam::ode<Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > >::ode(Foam::fvMesh const&) at ??:?
[2] #17  Foam::psiChemistryModel::New(Foam::fvMesh const&) at ??:?
[0] #15  Foam::psiChemistryModel::addfvMeshConstructorToTable<Foam::ode<Foam::chemistryModel<Foam::psiChemistryModel, Foam::sutherlandTransport<Foam::species::thermo<Foam::janafThermo<Foam::perfectGas<Foam::specie> >, Foam::sensibleEnthalpy> > > > >::New(Foam::fvMesh const&) at ??:?
[2] #18  Foam::combustionModels::psiChemistryCombustion::psiChemistryCombustion(Foam::word const&, Foam::fvMesh const&) at ??:?
[0] #16  Foam::autoPtr<Foam::psiChemistryModel> Foam::basicChemistryModel::New<Foam::psiChemistryModel>(Foam::fvMesh const&) at ??:?
[2] #19  Foam::combustionModels::laminar<Foam::combustionModels::psiChemistryCombustion>::laminar(Foam::word const&, Foam::fvMesh const&) at ??:?
[0] #17  Foam::psiChemistryModel::New(Foam::fvMesh const&) at ??:?
[2] #20  Foam::combustionModels::PaSR<Foam::combustionModels::psiChemistryCombustion>::PaSR(Foam::word const&, Foam::fvMesh const&) at ??:?
[2] #21  Foam::combustionModels::psiCombustionModel::adddictionaryConstructorToTable<Foam::combustionModels::PaSR<Foam::combustionModels::psiChemistryCombustion> >::New(Foam::word const&, Foam::fvMesh const&) at ??:?
[0] #18  Foam::combustionModels::psiChemistryCombustion::psiChemistryCombustion(Foam::word const&, Foam::fvMesh const&) at ??:?
[2] #22  Foam::combustionModels::psiCombustionModel::New(Foam::fvMesh const&) at ??:?
[0] #19  Foam::combustionModels::laminar<Foam::combustionModels::psiChemistryCombustion>::laminar(Foam::word const&, Foam::fvMesh const&) at ??:?
[2] #23   at ??:?
[0] #20  Foam::combustionModels::PaSR<Foam::combustionModels::psiChemistryCombustion>::PaSR(Foam::word const&, Foam::fvMesh const&)
 at ??:?
[0] #21  Foam::combustionModels::psiCombustionModel::adddictionaryConstructorToTable<Foam::combustionModels::PaSR<Foam::combustionModels::psiChemistryCombustion> >::New(Foam::word const&, Foam::fvMesh const&) at ??:?
[0] #22  Foam::combustionModels::psiCombustionModel::New(Foam::fvMesh const&)[2]  at ??:?
[2] #24  __libc_start_main at ??:?
[2] #25   at ??:?
[0] #23  

[2]  at ??:?
[localhost:13056] *** Process received signal ***
[localhost:13056] Signal: Floating point exception (8)
[localhost:13056] Signal code:  (-6)
[localhost:13056] Failing at address: 0x1f400003300
[localhost:13056] [ 0] /lib64/libc.so.6[0x3621c326a0]
[localhost:13056] [ 1] /lib64/libc.so.6(gsignal+0x35)[0x3621c32625]
[localhost:13056] [ 2] /lib64/libc.so.6[0x3621c326a0]
[localhost:13056] [ 3] carlenSprayFoam(_ZN4Foam3maxINS_12fvPatchFieldEdEET0_RKNS_10FieldFieldIT_S2_EE+0xdc)[0x449b5c]
[localhost:13056] [ 4] 
[localhost:13054] [23] carlenSprayFoam[0x42dffa]
[localhost:13054] [24] /lib64/libc.so.6(__libc_start_main+0xfd)[0x3621c1ed5d]
[localhost:13054] [25] carlenSprayFoam[0x4338e5]
[localhost:13054] *** End of error message ***
[localhost:13057] *** An error occurred in MPI_Pack
[localhost:13057] *** reported by process [140187236761601,140733193388035]
[localhost:13057] *** on a NULL communicator
[localhost:13057] *** Unknown error
[localhost:13057] *** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
[localhost:13057] ***    and potentially your MPI job)
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 13054 on node localhost exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------

How can I solve this problem?

[Carlen]

Hi everyone,
its carlen again. It is hilarious I have been talking to myself and get the solution by myself . Anyway, I figured out how to solve the problem. By looking into the 0 folder, I realise that only T, p, U fields are needed for such a simulation. So I type in:

mpirun -n 4 mapFields ../aachenBomb-twoparcels -mapMethod direct -sourceTime latestTime -fields '(U T p)' -consistent -noLagrangian -parallel

and it only maps the required fields for sprayFoam. Then simulation the runs perfectly fine. Hope this is helpful to those who have the same problem.

Cheers,
Carlen

[Carlen]

Now you can see the pressure field is identical for both the source and target case!

[Carlen]

it turns out that I have to use

-mapMethod cellVolumeWeight

because "direct" method requires both the source and target to be identical. It is still time consuming for large cases. I wonder if OpenFoam has a faster implementation of such a process.

carlen

[wyldckat]

Hi Carlen,

Many thanks for sharing each step you've done!

As for a faster method, have you tried using "-mapMethod mapNearest"? It should be faster, although it will strongly depend on how related the fine mesh is to the coarse mesh, i.e. if the fine mesh is simply the coarse refined for some or all cells in a 2x2x2 division for each cell.

Best regards,
Bruno