[paul.m.edwards]

Hi,

I am having problems running on a large number of processes. I have a reconstructed mesh that I first decompose and then run. The case is the motorBike example where I have increased the cell count. I am having problems with some decompositions and particularly when the number of ranks is high (e.g. over about 400). The error I get is the following (here from running checkMesh):

8<----------------------------------------------------------------------
[0]
[0]
[0] --> FOAM FATAL IO ERROR:
[0] error in IOstream "IOstream" for operation operator>>(Istream&, List<T>&) : reading first token
[0]
[0] file: IOstream at line 0.
[0]
[0] From function IOstream::fatalCheck(const char*) const
[0] in file db/IOstreams/IOstreams/IOstream.C at line 114.
[0]
FOAM parallel run exiting
[0]
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
---------------------------------------------------------------------->8

I have attached a debugger and this is the stack trace for a process that is raising the error:

8<----------------------------------------------------------------------
#0 Foam::IOstream::fatalCheck (this=0x7fff59e07238,
operation=0x472514 "operator>>(Istream&, List<T>&) : reading first token")
at db/IOstreams/IOstreams/IOstream.C:114
#1 0x000000000043a5dd in Foam:perator>> (is=..., L=...)
at /OpenFOAM-orig/OpenFOAM-2.3.x/src/OpenFOAM/lnInclude/ListIO.C:54
#2 0x00002b1ad01842fc in Foam:perator>> (is=..., L=...)
at /OpenFOAM-orig/OpenFOAM-2.3.x/src/OpenFOAM/lnInclude/ListIO.C:86
#3 0x00002b1ad017d0be in Foam::List<Foam::labelList>::List (this=0x7fff59e072d8, is=...)
at /OpenFOAM-orig/OpenFOAM-2.3.x/src/OpenFOAM/lnInclude/ListIO.C:40
#4 0x00002b1ad017e9af in Foam::Pstream::combineScatter (comms=..., Value=..., tag=1, comm=0)
at /OpenFOAM-orig/OpenFOAM-2.3.x/src/OpenFOAM/lnInclude/combineGatherScatter.C:215
#5 0x00002b1ad0186125 in Foam::combineReduce (Value=..., cop=..., tag=1, comm=0)
at /OpenFOAM-orig/OpenFOAM-2.3.x/src/OpenFOAM/lnInclude/PstreamCombineReduceOps.H:102
#6 0x00002b1ad13f5cdb in Foam::Pstream::exchange (sendBufs=..., recvBufs=..., sizes=..., tag=1, comm=0,
block=true) at db/IOstreams/Pstreams/exchange.C:82
#7 0x00002b1ad13f47e4 in Foam::PstreamBuffers::finishedSends (this=0x7fff59e07760, block=true)
at db/IOstreams/Pstreams/PstreamBuffers.C:89
#8 0x00002b1ad15ea4cf in Foam::globalMeshData::initProcAddr (this=0x28499d0)
at meshes/polyMesh/globalMeshData/globalMeshData.C:113
#9 0x00002b1ad15f9514 in Foam::globalMeshData::updateMesh (this=0x28499d0)
at meshes/polyMesh/globalMeshData/globalMeshData.C:2740
#10 0x00002b1ad15f4847 in Foam::globalMeshData::globalMeshData (this=0x28499d0, mesh=...)
at meshes/polyMesh/globalMeshData/globalMeshData.C:1761
#11 0x00002b1ad1648450 in Foam:olyMesh::globalData (this=0x7fff59e08068) at meshes/polyMesh/polyMesh.C:1211
#12 0x0000000000451ff4 in main (argc=3, argv=0x7fff59e089c8) at checkMesh.C:156
---------------------------------------------------------------------->8

Does anyone have any hints for where to look to fix this problem? I cannot tell whether there has been a problem with decomposition - in stack frame 8 it is sharing processor patches with neighbours. Is there a way to validate the patches (although I assume this is checkMesh which I cannot run!).

Any help would be appreciated.

Thanks,
Paul