[donno]

Can anyone tell me how to solve this problem?
it is ok for a serial run,but it stops with segentation fault when i try to run parrallel in a cluster. my openfoam version is 2.1.1.

Cloud size= 800
Courant Number mean: 0.555556 max: 0.555898
deltaT = 0.0634921
DILUPBiCG:  Solving for Ux, Initial residual = 0.363798, Final residual = 1.84647e-09, No Iterations 7
DILUPBiCG:  Solving for Uy, Initial residual = 0.4546, Final residual = 6.04328e-09, No Iterations 7
DILUPBiCG:  Solving for Uz, Initial residual = 0.441025, Final residual = 3.78583e-10, No Iterations 6
DICPCG:  Solving for p, Initial residual = 0.00655311, Final residual = 0.000447691, No Iterations 3
DICPCG:  Solving for p, Initial residual = 0.000444716, Final residual = 3.87554e-05, No Iterations 29
time step continuity errors : sum local = 1.2429e-09, global = -3.07084e-10, cumulative = -3.07334e-10
DICPCG:  Solving for p, Initial residual = 0.00138002, Final residual = 9.84069e-05, No Iterations 3
DICPCG:  Solving for p, Initial residual = 9.84319e-05, Final residual = 9.50433e-09, No Iterations 747
time step continuity errors : sum local = 3.0458e-13, global = 5.6887e-16, cumulative = -3.07334e-10
DILUPBiCG:  Solving for epsilon, Initial residual = 0.0288261, Final residual = 1.75545e-09, No Iterations 7
DILUPBiCG:  Solving for k, Initial residual = 0.66385, Final residual = 9.60895e-10, No Iterations 9
DILUPBiCG:  Solving for C, Initial residual = 0.0620355, Final residual = 2.92867e-09, No Iterations 8
DILUPBiCG:  Solving for T, Initial residual = 0.137049, Final residual = 1.90345e-10, No Iterations 6
Time = 0.119048

ExecutionTime = 277.13 s  ClockTime = 279 s

Moving particles.

[3] [5] #0  Foam::error::printStack(Foam::Ostream&)#0  Foam::error::printStack(Foam::Ostream&)[1] [4] #0  Foam::error::printStack(Foam::Ostream&)#0  Foam::error::printStack(Foam::Ostream&)[7] [2] #0  Foam::error::printStack(Foam::Ostream&)[6] #0  #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)--------------------------------------------------------------------------
An MPI process has executed an operation involving a call to the
"fork()" system call to create a child process.  Open MPI is currently
operating in a condition that could result in memory corruption or
other system errors; your MPI job may hang, crash, or produce silent
data corruption.  The use of fork() (or system() or other calls that
create child processes) is strongly discouraged.  

The process that invoked fork was:

  Local host:          node65 (PID 29186)
  MPI_COMM_WORLD rank: 4

If you are *absolutely sure* that your application will successfully
and correctly survive a call to fork(), you may disable this warning
by setting the mpi_warn_on_fork MCA parameter to 0.
--------------------------------------------------------------------------
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/ in "/public2/home/nl/OpenFOAM in "/public2/home/nengyuan/limao8 in linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #1  Foam::sigSegv::sigHandler(int)"/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64Gcc in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/lin/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #1  9/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #1  DPOpt/lib/libOpenFOAM.so"
ux64GccDPOpt/lib/libOpenFOAM.so"Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int)[1] #1  
[5] #1  Foam::sigSegv::sigHandler(int)Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #1  Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[7] #1  Foam::sigSegv::sigHandler(int) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[1] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[7] #2   in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #2   in "/lib64/libc.so.6"
[4] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[1] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[5] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[6] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[2] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[7] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/lib64/libc.so.6"
[3] #3  double Foam::particle::trackToFace<Foam::solidParticle::trackingData>(Foam::Vector<double> const&, Foam::solidParticle::trackingData&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #4  Foam::solidParticle::move(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #5  void Foam::Cloud<Foam::solidParticle>::move<Foam::solidParticle::trackingData>(Foam::solidParticle::trackingData&, double) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #6  Foam::solidParticleCloud::move(Foam::dimensioned<Foam::Vector<double> > const&) in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #7  
[1]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[1] #8  __libc_start_main in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #7  
 in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #7  
[4]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[4] #8  __libc_start_main in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #7  
 in "/lib64/libc.so.6"
[1] #9  
[5]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[5] #8  __libc_start_main[2]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[2] #8  __libc_start_main[6]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[6] #8  __libc_start_main in "/lib64/libc.so.6"
[4] #9  
[7]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[7] #8  __libc_start_main[3]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[3] #8  __libc_start_main in "/lib64/libc.so.6"
[5] #9  
 in "/lib64/libc.so.6"
[2] #9  
[1]  in "/public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong"
[node65:29183] *** Process received signal ***
[node65:29183] Signal: Segmentation fault (11)
[node65:29183] Signal code:  (-6)
[node65:29183] Failing at address: 0x3f9000071ff
[node65:29183] [ 0] /lib64/libc.so.6(+0x32920) [0x7f3c81738920]
[node65:29183] [ 1] /lib64/libc.so.6(gsignal+0x35) [0x7f3c817388a5]
[node65:29183] [ 2] /lib64/libc.so.6(+0x32920) [0x7f3c81738920]
[node65:29183] [ 3] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam8particle11trackToFaceINS_13solidParticle12trackingDataEEEdRKNS_6VectorIdEERT_+0x19e) [0x47dcce]
[node65:29183] [ 4] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam13solidParticle4moveERNS0_12trackingDataEd+0x14d) [0x477d9d]
[node65:29183] [ 5] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam5CloudINS_13solidParticleEE4moveINS1_12trackingDataEEEvRT_d+0x417) [0x565577]
[node65:29183] [ 6] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong(_ZN4Foam18solidParticleCloud4moveERKNS_11dimensionedINS_6VectorIdEEEE+0x42b) [0x4afc0b]
[node65:29183] [ 7] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong() [0x42a0d7]
[node65:29183] [ 8] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f3c81724cdd]
[node65:29183] [ 9] /public2/home/nl/OpenFOAM/li-2.1.1/platforms2/linux64GccDPOpt/bin/chuanreqiudong() [0x428559]
[node65:29183] *** End of error message ***
 in "/lib64/libc.so.6"
[6] #9  
--------------------------------------------------------------------------
mpirun noticed that process rank 1 with PID 29183 on node node65 exited on signal 11 (Segmentation fault).
--------------------------------------------------------------------------
[node65:29181] 6 more processes have sent help message help-mpi-runtime.txt / mpi_init:warn-fork
[node65:29181] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

[be_inspired]

Hi,

you could check several things:

• If previously the simulation works correctly in parallel until some iteration or crash at first moving particle, it can tell you where to look for.

• If you use AMI, be sure that patches are not divided between different processors.Change decomposition method according to that.
• Check memory request when moving particles. Maybe some of the nodes is out of memory or swap memory is low of a memory module is damaged.
• You say that it works in serial. Are you running an own solver that maybe is not correctly coded to be launched in parallel?

[donno]

It is ok now when i set a smaller deltaT 0.005.