[Zlatko]

Hi,

I have a problem with simpleFOAM parallel sover using fluent polyhedral mesh.

A tetrahedral mesh was created in GAMBIT and imported into Fluent, where computational domain was converted to polyhedral cells and rotational periodic boundary conditions were defined for two faces. The case was run without any problems in Fluent.

I wanted to use the same mesh with OpenFOAM simpleFoam solver, so I:

1. used fluent3DMeshToFoam utility to create OpenFOAM mesh,
2. applied createPatch tool to define rotational cyclic boundary conditions following this thread,
3. decomposed the domain into 4 partitions using decomposePar utility.

New mesh was checked with checkMesh utillity and everything was OK.
First, I started simpleFoam serial solver and it converged well with the expected results. Then, I wanted to use parallel solver, but it crashed with error:

$ mpirun -np 4 simpleFoam -parallel 2>&1 | tee run.log
      
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.0-f5222ca19ce6
Exec   : simpleFoam -parallel
Date   : Sep 18 2014
Time   : 11:12:40
Host   : "lfdt"
PID    : 17682
Case   : /work/cyclic-1phase-laminar
nProcs : 4
Slaves : 
3
(
"lfdt.17683"
"lfdt.17684"
"lfdt.17685"
)

Pstream initialized with:
    floatTransfer      : 0
    nProcsSimpleSum    : 0
    commsType          : nonBlocking
    polling iterations : 0
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

[3] #0  Foam::error::printStack(Foam::Ostream&)[2] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
[3] #1  Foam::sigSegv::sigHandler(int) at ??:?
[2] #1  Foam::sigSegv::sigHandler(int) at ??:?
[3] #2   in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) at ??:?
[2] #2   at ??:?
[3] #4  Foam::polyBoundaryMesh::updateMesh() in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&)[1] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
[3] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) at ??:?
[2] #4  Foam::polyBoundaryMesh::updateMesh() at ??:?
[1] #1  Foam::sigSegv::sigHandler(int) at ??:?
[3] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) at ??:?
[2] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) at ??:?
[1] #2   at ??:?
[3] #7   at ??:?
[2] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&)
 at ??:?
[2] #7   at ??:?
[1] #4  Foam::polyBoundaryMesh::updateMesh()[3]  at ??:?
[3] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #9  
 at ??:?
[1] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&)
[2]  at ??:?
[2] #8  __libc_start_main[3]  at ??:?
[lfdt:17685] *** Process received signal ***
[lfdt:17685] Signal: Segmentation fault (11)
[lfdt:17685] Signal code:  (-6)
[lfdt:17685] Failing at address: 0x3e800004515
 at ??:?
[1] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #9  [lfdt:17685] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f061b6b0c30]
[lfdt:17685] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7f061b6b0bb9]
[lfdt:17685] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f061b6b0c30]
[lfdt:17685] [ 3] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x223) [0x7f061c8be663]
[lfdt:17685] [ 4] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x176) [0x7f061c8c49b6]
[lfdt:17685] [ 5] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam8polyMeshC2ERKNS_8IOobjectE+0xc05) [0x7f061c90c325]
[lfdt:17685] [ 6] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f061e1a1e99]
[lfdt:17685] [ 7] simpleFoam() [0x41aaf7]
[lfdt:17685] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f061b69bec5]
[lfdt:17685] [ 9] simpleFoam() [0x41c640]
[lfdt:17685] *** End of error message ***

 at ??:?
[1] #7  
[2]  at ??:?
[lfdt:17684] *** Process received signal ***
[lfdt:17684] Signal: Segmentation fault (11)
[lfdt:17684] Signal code:  (-6)
[lfdt:17684] Failing at address: 0x3e800004514
[lfdt:17684] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f9d15520c30]
[lfdt:17684] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7f9d15520bb9]
[lfdt:17684] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f9d15520c30]
[lfdt:17684] [ 3] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x21b) [0x7f9d1672e65b]
[lfdt:17684] [ 4] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x176) [0x7f9d167349b6]
[lfdt:17684] [ 5] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam8polyMeshC2ERKNS_8IOobjectE+0xc05) [0x7f9d1677c325]
[lfdt:17684] [ 6] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7f9d18011e99]
[lfdt:17684] [ 7] simpleFoam() [0x41aaf7]
[lfdt:17684] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f9d1550bec5]
[lfdt:17684] [ 9] simpleFoam() [0x41c640]
[lfdt:17684] *** End of error message ***
[1]  at ??:?
[1] #8  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[1] #9  
[1]  at ??:?
[lfdt:17683] *** Process received signal ***
[lfdt:17683] Signal: Segmentation fault (11)
[lfdt:17683] Signal code:  (-6)
[lfdt:17683] Failing at address: 0x3e800004513
[lfdt:17683] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7fa0073e2c30]
[lfdt:17683] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7fa0073e2bb9]
[lfdt:17683] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7fa0073e2c30]
[lfdt:17683] [ 3] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x21b) [0x7fa0085f065b]
[lfdt:17683] [ 4] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x176) [0x7fa0085f69b6]
[lfdt:17683] [ 5] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libOpenFOAM.so(_ZN4Foam8polyMeshC2ERKNS_8IOobjectE+0xc05) [0x7fa00863e325]
[lfdt:17683] [ 6] /home/OpenFOAM/OpenFOAM-2.3.0/platforms/linux64GccDPOpt/lib/libfiniteVolume.so(_ZN4Foam6fvMeshC2ERKNS_8IOobjectE+0x19) [0x7fa009ed3e99]
[lfdt:17683] [ 7] simpleFoam() [0x41aaf7]
[lfdt:17683] [ 8] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fa0073cdec5]
[lfdt:17683] [ 9] simpleFoam() [0x41c640]
[lfdt:17683] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 3 with PID 17685 on node lfdt exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
$

Does anyone have an idea what could be wrong?

Best regards.

Zlatko

[Zlatko]

Update: If boundary type cyclic is changed to type wall in files 0/{U,p} and constant/polyMesh/boundary, then parallel simpleFoam runs without problem.
Is it possible that decomposePar has a problem with cyclic boundaries?

[tomf]

Hi,

For coupled patches you should keep them on the same processor when decomposing using:

preservePatches (cyclic_1 cyclic_2);

in the decomposeParDict

Where cyclic_1 and cyclic_2 are the names of your cyclic patches.

Regards,
Tom

[Zlatko]

Originally Posted by tomf

For coupled patches you should keep them on the same processor when decomposing using:

preservePatches (cyclic_1 cyclic_2);

in the decomposeParDict

Where cyclic_1 and cyclic_2 are the names of your cyclic patches.

Tom, thank you very much. Now it works.

Best regards,

Zlatko