[timdrake]

I am putting together a cluster of 20 physical computers, each containing 4 cores. Right now I am trying to get the master and the first node (acl-n1) working. I am using MPI and NFS.

From my understanding, the openfoam9 dir should be in my shared folder OpenFOAM. I have updated the bashrc in openfoam9/etc/bashrc to correspond with the new dir.

Some context info:
$PATH
/home/mpiuser/.local/bin:/path/to/openmpi/bin:/home/mpiuser/OpenFOAM/ThirdParty-9/platforms/linux64Gcc/gperftools-svn/bin:/opt/paraviewopenfoam56/bin:/home/mpiuser/OpenFOAM/ThirdParty-9/platforms/linux64Gcc/cmake-*/bin:/home/mpiuser/OpenFOAM/mpiuser-9/platforms/linux64GccDPInt32Opt/bin:/home/mpiuser/OpenFOAM/site/9/platforms/linux64GccDPInt32Opt/bin:/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/bin:/home/mpiuser/OpenFOAM/openfoam9/bin:/home/mpiuser/OpenFOAM/openfoam9/wmake:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin


$LD_LIBRARY_PATH
/home/mpiuser/OpenFOAM/ThirdParty-9/platforms/linux64Gcc/gperftools-svn/lib:/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/lib/paraview-5.6:/opt/paraviewopenfoam56/lib:/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/lib/openmpi-system:/home/mpiuser/OpenFOAM/ThirdParty-9/platforms/linux64GccDPInt32/lib/openmpi-system:/usr/lib/x86_64-linux-gnu/openmpi/lib:/home/mpiuser/OpenFOAM/mpiuser-9/platforms/linux64GccDPInt32Opt/lib:/home/mpiuser/OpenFOAM/site/9/platforms/linux64GccDPInt32Opt/lib:/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/lib:/home/mpiuser/OpenFOAM/ThirdParty-9/platforms/linux64GccDPInt32/lib:/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/lib/dummy

Shared folder that is mounted correctly on acl-n1: /home/mpiuser/OpenFOAM

$FOAM_INST_DIR=/home/mpiuser/OpenFOAM
$FOAM_RUN=$FOAM_INST_DIR/mpiuser-9/run
$machines is:
master
acl-n1

I copied the tutorials dir to $FOAM_RUN, and im trying to run the airFoil2D case:

mpirun --hostfile $machines -np 5 simpleFoam >> log

this is the output when ran:

--------------------------------------------------------------------------
mpirun was unable to find the specified executable file, and therefore
did not launch the job.  This error was first reported for process
rank 4; it may have occurred for other processes as well.

NOTE: A common cause for this error is misspelling a mpirun command
      line parameter option (remember that mpirun interprets the first
      unrecognized command line token as the executable).

Node:       acl-n1
Executable: /home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/bin/simpleFoam
--------------------------------------------------------------------------

Why is this wrong if it is shared in the NFS??

If I specify where simpleFoam is exactly:

mpirun --hostfile $machines -np 5 $FOAM_INST_DIR/openfoam9/platforms/linux64GccDPInt32Opt/bin/simpleFoam >> log

then it gives a new error:

/home/mpiuser/OpenFOAM/openfoam9/platforms/linux64GccDPInt32Opt/bin/simpleFoam: error while loading shared libraries: libPstream.so: cannot open shared object file: No such file or directory
--------------------------------------------------------------------------
Primary job  terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun detected that one or more processes exited with non-zero status, thus causing
the job to be terminated. The first process to do so was:

  Process name: [[49322,1],4]
  Exit code:    127
--------------------------------------------------------------------------

Why cant it find the library? simpleFoam can run just fine on each computer when there are 4 processes, but when i try 5 it doesnt work, no matter what computer i run it from.

Please help, I have been tearing out my hair for weeks on this!!