[MacGyver]

When running my case, this error is triggered:
log.interFoam:

 
[xs024:17484] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17484] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17488] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17488] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17486] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17486] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17483] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17489] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17482] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17483] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17489] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17482] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17485] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17485] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17487] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_ofud: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
[xs024:17487] mca: base: component_find: unable to open /share/openfoam/2.1.x/OpenFOAM/ThirdParty-2.1.x/platforms/linux64Gcc44/openmpi-1.5.3/lib/openmpi/mca_btl_openib: libibverbs.so.1: cannot open shared object file: No such file or directory (ignored)
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.1.x                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.1.x-bae608187e52
Exec   : interFoam -parallel
Date   : May 07 2012
Time   : 15:51:21
Host   : "xs024"
PID    : 17482
Case   : /path/OF/myBC_21_scotch
nProcs : 8
Slaves : 
7
(
"xs024.17483"
"xs024.17484"
"xs024.17485"
"xs024.17486"
"xs024.17487"
"xs024.17488"
"xs024.17489"
)
Pstream initialized with:
    floatTransfer     : 0
    nProcsSimpleSum   : 0
    commsType         : nonBlocking
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Allowing user-supplied system call operations
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time
Create mesh for time = 0
[7] #0  Foam::error::printStack(Foam::Ostream&)[3] #0  Foam::error::printStack(Foam::Ostream&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[3] #1  Foam::sigSegv::sigHandler(int) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[7] #1  Foam::sigSegv::sigHandler(int) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DP/liOpt/lib/libOpenFOAM.so"
[3] #2  b/libOpenFOAM.so"
[7] #2  ???? in "/lib64/libc.so.6"
[7] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/lib64/libc.so.6"
[3] #3  Foam::processorPolyPatch::updateMesh(Foam::PstreamBuffers&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[7] #4  Foam::polyBoundaryMesh::updateMesh() in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[3] #4  Foam::polyBoundaryMesh::updateMesh() in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[7] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[3] #5  Foam::polyMesh::polyMesh(Foam::IOobject const&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[7] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so"
[3] #6  Foam::fvMesh::fvMesh(Foam::IOobject const&) in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libfiniteVolume.so"
[7] #7   in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libfiniteVolume.so"
[3] #7  mainmain in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/bin/interFoam"
[3] #8  __libc_start_main in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/bin/interFoam"
[7] #8  __libc_start_main in "/lib64/libc.so.6"
[3] #9  Foam::regIOobject::writeObject(Foam::IOstream::streamFormat, Foam::IOstream::versionNumber, Foam::IOstream::compressionType) const in "/lib64/libc.so.6"
[7] #9   in "/share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/bin/interFoam"
[xs024:17485] *** Process received signal ***
[xs024:17485] Signal: Segmentation fault (11)
[xs024:17485] Signal code:  (-6)
[xs024:17485] Failing at address: 0x6510000444d
[xs024:17485] [ 0] /lib64/libc.so.6 [0x2b806c1a4bd0]
[xs024:17485] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x2b806c1a4b45]
[xs024:17485] [ 2] /lib64/libc.so.6 [0x2b806c1a4bd0]
[xs024:17485] [ 3] /share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so(_ZN4Foam18processorPolyPatch10updateMeshERNS_14PstreamBuffersE+0x2da) [0x2b806b35c0da]
[xs024:17485] [ 4] /share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so(_ZN4Foam16polyBoundaryMesh10updateMeshEv+0x2b1) [0x2b806b3616c1]
[xs024:17485] [ 5] /share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libOpenFOAM.so(_ZN4Foam8polyMeshC2ERKNS_8IOobjectE+0xd16) [0x2b806b3b4c26]
[xs024:17485] [ 6] /share/openfoam/2.1.x/OpenFOAM/OpenFOAM-2.1.x/platforms/linux64Gcc44DPOpt/lib/libfiniteVolume.so(_ZN4Foam6fvMeshC1ERKNS_8IOobjectE+0x19) [0x2b806a2ccbb9]
[xs024:17485] [ 7] interFoam [0x424d90]
[xs024:17485] [ 8] /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b806c191b54]
[xs024:17485] [ 9] interFoam(_ZNK4Foam11regIOobject11writeObjectENS_8IOstream12streamFormatENS1_13versionNumberENS1_15compressionTypeE+0x111) [0x421809]
[xs024:17485] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 3 with PID 17485 on node xs024 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------

The case is started with:
decomposePar -force
foamJob -p interFoam


Decomposition method "scotch" was applied. When using another method (like "hierarchical") no problem occured.
DecomposeParDict:

 
numberOfSubdomains 8;
//method          hierarchical;
//method          ptscotch;
method    scotch;
simpleCoeffs
{
    n               ( 2 2 1 );
    delta           0.001;
}
hierarchicalCoeffs
{
    n               (1 2 4);
    delta           0.001;
    order           xyz;
}
metisCoeffs
{
    processorWeights ( 1 1 1 1 1 1 1 1 );
  // 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1);// 1 1 1 1 1 1);
}
manualCoeffs
{
    dataFile        "";
}
 
distributed     no;
roots           ( );

I use a Mesh created by snappyHexMesh with cyclic Boundaries. OF-Version 2.1.x.

Anyone encountered this problem and knows a solution?
Thank you!!

[JulytoNovember]

Try changing metisCoeffs to scotchCoeffs

[falke126]

Originally Posted by MacGyver

I use a Mesh created by snappyHexMesh with cyclic Boundaries. OF-Version 2.1.x.

Anyone encountered this problem and knows a solution?
Thank you!!

for SHM meshes you have to use ptscotch!