[OpenFOAM.com] Install OpenFOAM v1906 on a cluster with CentOS/RHEL 7

herbert0709 · August 21, 2019, 02:37

Hey guys

I am trying to install the openFOAM on the university cluster and i have been following this link for reference.

https://www.openfoam.com/download/install-source.php

and every thing seems right until i run the command: foamInstallationTest and this is the error message displayed saying 'icoFoam not installed: (i apologize for the bad formatting of the post, it is actually my first post)

Code:


Basic setup :
-------------------------------------------------------------------------------
OpenFOAM:            OpenFOAM-v1906
ThirdParty:          ThirdParty-v1906
Shell:               bash
Host:                monarch-login2
OS:                  Linux version 3.10.0-957.5.1.el7.x86_64
-------------------------------------------------------------------------------


Main OpenFOAM env variables :
-------------------------------------------------------------------------------
EnvironmentVariable   FileOrDirectory                          Valid      Crit
-------------------------------------------------------------------------------
$WM_PROJECT_USER_DIR  /home/zzha518/OpenFOAM/zzha518-v1906      no        no
$WM_THIRD_PARTY_DIR   ...y81/zzha518/OpenFOAM/ThirdParty-v1906  yes       yes
-------------------------------------------------------------------------------


OpenFOAM env variables in PATH :
-------------------------------------------------------------------------------
EnvironmentVariable   FileOrDirectory                          Valid Path Crit
-------------------------------------------------------------------------------
$WM_PROJECT_DIR       .../ny81/zzha518/OpenFOAM/OpenFOAM-v1906  yes  yes  yes

$FOAM_APPBIN          ...06/platforms/linux64GccDPInt32Opt/bin  no        yes
$FOAM_SITE_APPBIN     ...06/platforms/linux64GccDPInt32Opt/bin  no        no
$FOAM_USER_APPBIN     ...06/platforms/linux64GccDPInt32Opt/bin  no        no
$WM_DIR               ...zzha518/OpenFOAM/OpenFOAM-v1906/wmake  yes  yes  often
-------------------------------------------------------------------------------


OpenFOAM env variables in LD_LIBRARY_PATH :
-------------------------------------------------------------------------------
EnvironmentVariable   FileOrDirectory                          Valid Path Crit
-------------------------------------------------------------------------------
$FOAM_LIBBIN          ...06/platforms/linux64GccDPInt32Opt/lib  yes  yes  yes
$FOAM_SITE_LIBBIN     ...06/platforms/linux64GccDPInt32Opt/lib  no        no
$FOAM_USER_LIBBIN     ...06/platforms/linux64GccDPInt32Opt/lib  no        no
$FOAM_EXT_LIBBIN      ...v1906/platforms/linux64GccDPInt32/lib  yes  yes  maybe
$MPI_ARCH_PATH        --------- env variable not set ---------            yes
-------------------------------------------------------------------------------


Software Components
-------------------------------------------------------------------------------
Software  Version    Location  
-------------------------------------------------------------------------------
flex      2.5.37     /bin/flex                                                 
wmake     1906       ...zha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/wmake/wmake
gcc       4.8.5      /bin/gcc                                                  
g++       4.8.5      /bin/g++                                                  
gzip      1.5        /bin/gzip                                                 
tar       1.26       /bin/tar                                                  
icoFoam   *** not installed ***
          CRITICAL ERROR

-------------------------------------------------------------------------------


Summary
-------------------------------------------------------------------------------
Base configuration ok.
The foam installation contains 1 critical error(s).

Review the output for warning messages and consult
the installation guide for troubleshooting.

Done

I am not sure what causes the issue and what should i be doing?

Thank you very much in advance!!!!

wyldckat · August 25, 2019, 09:12

Quick question: In that page you provided, it gives a link to the Build page: https://www.openfoam.com/code/build-guide.php - did you follow the instructions given on that page?

herbert0709 · August 26, 2019, 23:37

Quote:

Originally Posted by wyldckat

Quick question: In that page you provided, it gives a link to the Build page: https://www.openfoam.com/code/build-guide.php - did you follow the instructions given on that page?

Hello Bruno
Thanks for replying.
Yes i follow everything on the build page and when i try the command 'foamSystemCheck' it says the following:

Code:

bash: foamSystemCheck: command not found...

And when i try to source the bashrc file, the following is shown:

Code:

bash: mpicc: command not found...
No completion added for /home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/bin

But, after this i can use the 'foam' command to go to the OpenFoam installation directory.

So i am not sure what is the cause of the problem

Thank you very much !!!

wyldckat · August 27, 2019, 19:01

Quick answers:

The command foamSystemCheck will only work after you activated the OpenFOAM shell environment, namely after you source the "bashrc" file.
The message:
Code:
```
bash: mpicc: command not found...
```
is because there is no Open-MPI ready to be used... check if you have any module with Open-MPI ready to be used by running:
Code:
```
module avail
```
I expect it's something like "openmpi-x86_64" or "mpi/openmpi-x86_64" that is available in your system. If you do have it, you can load it by running, for example:
Code:
```
module load mpi/openmpi-x86_64
```
then source OpenFOAM's "bashrc" file once again.
The message:
Code:
```
No completion added for ...
```
is being shown because OpenFOAM-v9106 hasn't been compiled yet.
Then to compile OpenFOAM-v1906, use the following commands:
Code:
```
./Allwmake -j 4 -log=log.make
```
where 4 is the number of cores to use to build/compile OpenFOAM.
The output is sent to the file "log.make" which you can check its contents on another terminal, e.g. by running:
Code:
```
tail -F log.make
```
This log file is critical for diagnosing any compilation problems.

herbert0709 · August 27, 2019, 21:25

Hello Bruno

Thank you for your reply !

When i tried your suggestion, i found out that my university's cluster does not have 'openmpi-x86_64' available, so i loaded a one of existing openmpi which is 'openmpi/1.10.7-mlx'

then i tried to compile the openFOAM with ./Allwmake and i get the following warning:

Code:

/home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so: undefined reference to `MPI_Finalize'
/home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so: undefined reference to `ompi_mpi_op_sum'
/home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so: undefined reference to `MPI_Buffer_attach'
/home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/lib/openmpi-system/libPstream.so: undefined reference to `MPI_Group_free'
collect2: error: ld returned 1 exit status
make[3]: *** [/home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/platforms/linux64GccDPInt32Opt/bin/rhoPorousSimpleFoam] Error 1
make[2]: *** [rhoPorousSimpleFoam] Error 2
make[1]: *** [rhoSimpleFoam] Error 2
make: *** [compressible] Error 2

By the looks of error message, i am not sure if it is to do with the openmpi that i did not use the right one or something else i am not doing correctly. So the Openmpis that is available in the cluster is as follows:

Code:

openmpi/1.10.3-gcc4-mlx
openmpi/1.10.3-gcc4-mlx-cuda75
openmpi/1.10.3-gcc4-mlx-verbs
openmpi/1.10.3-gcc4-mlx-verbs-cuda75
openmpi/1.10.3-gcc5
openmpi/1.10.3-gcc5-mlx
openmpi/1.10.7-1.mlx
openmpi/1.10.7-intel
openmpi/1.10.7-mlx(default)
openmpi/3.1.4-mlx
opennmt-py/0.7.0

And i am so sorry if i am asking very obvious questions, but i have zero experience in doing jobs in cluster previously.

Thank your very much for your patience!

wyldckat · August 28, 2019, 19:21

Quick answer: The commands you should execute are as follows:

Code:

module load openmpi/1.10.7-mlx
source $HOME/OpenFOAM/OpenFOAM-v1906/etc/bashrc

(I'm assuming the path on the second line works in your installation, but if not, adjust accordingly)
Then check what error messages you get. If you only get the error about "no completion added", then Open-MPI should be ready to be used with OpenFOAM.

If so far so good, then run:

Code:

foam
./Allwmake -j 4 -log=log.make

However, if sourcing OpenFOAM's "etc/bashrc" file gave the error about mpicc or some other error, then run:

Code:

echo $MPI_ARCH_PATH
ls -l $MPI_ARCH_PATH
ls -l $MPI_ARCH_PATH/include

and tell us what it gives you.

herbert0709 · August 29, 2019, 04:28

Hi Bruno

Thanks for replying!

When i load the openmpi and source the bashrc, there was nothing happening in the command window so i waited for a hour or two and found out there was still nothing. and i can not type any command either except ctrl+C to terminate the process.

Code:

[zzha518@monarch-login1 ~]$ source /home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/etc/bashrc

I am not sure if it is suppose to take that long to source the bashrc or anything has gone wrong again?

But the following is the what i get when i call the $MPI_ARCH_PATH

Code:

[zzha518@monarch-login1 ~]$ echo $MPI_ARCH_PATH
/usr/local/openmpi/1.10.7-mlx

Code:

[zzha518@monarch-login1 ~]$ ls -l $MPI_ARCH_PATH
total 13
drwxr-xr-x 2 chines monashuniversity 70 Mar 14  2018 bin
drwxr-xr-x 2 chines monashuniversity  7 Mar 14  2018 etc
drwxr-xr-x 6 chines monashuniversity 25 Mar 14  2018 include
drwxr-xr-x 4 chines monashuniversity 83 Dec  3  2018 lib
drwxr-xr-x 4 chines monashuniversity 84 Nov 30  2018 lib2
drwxr-xr-x 5 chines monashuniversity  5 Mar 14  2018 share

Code:

[zzha518@monarch-login1 ~]$ ls -l $MPI_ARCH_PATH/include
total 97
-rw-r--r-- 1 chines monashuniversity    255 Mar 14  2018 mpi-ext.h
-rw-r--r-- 1 chines monashuniversity   2712 Mar 14  2018 mpif-config.h
-rw-r--r-- 1 chines monashuniversity  10974 Mar 14  2018 mpif-constants.h
-rw-r--r-- 1 chines monashuniversity   1954 Mar 14  2018 mpif-externals.h
-rw-r--r-- 1 chines monashuniversity    188 Mar 14  2018 mpif-ext.h
-rw-r--r-- 1 chines monashuniversity   2715 Mar 14  2018 mpif.h
-rw-r--r-- 1 chines monashuniversity   7740 Mar 14  2018 mpif-handles.h
-rw-r--r-- 1 chines monashuniversity   2122 Mar 14  2018 mpif-io-constants.h
-rw-r--r-- 1 chines monashuniversity   1207 Mar 14  2018 mpif-io-handles.h
-rw-r--r-- 1 chines monashuniversity   2955 Mar 14  2018 mpif-sentinels.h
-rw-r--r-- 1 chines monashuniversity   1152 Mar 14  2018 mpif-sizeof.h
-rw-r--r-- 1 chines monashuniversity 166835 Mar 14  2018 mpi.h
-rw-r--r-- 1 chines monashuniversity  16055 Mar 14  2018 mpi_portable_platform.h
drwxr-xr-x 2 chines monashuniversity      4 Mar 14  2018 mpp
drwxr-xr-x 3 chines monashuniversity      3 Mar 14  2018 openmpi
drwxr-xr-x 3 chines monashuniversity      4 Mar 14  2018 openshmem
-rw-r--r-- 1 chines monashuniversity  19913 Mar 14  2018 pshmem.h
-rw-r--r-- 1 chines monashuniversity  13158 Mar 14  2018 pshmemx.h
-rw-r--r-- 1 chines monashuniversity   1384 Mar 14  2018 shmem-compat.h
-rw-r--r-- 1 chines monashuniversity   1692 Mar 14  2018 shmem.fh
-rw-r--r-- 1 chines monashuniversity  21648 Mar 14  2018 shmem.h
-rw-r--r-- 1 chines monashuniversity  12515 Mar 14  2018 shmemx.h
drwxr-xr-x 3 chines monashuniversity     23 Mar 14  2018 vampirtrace

Regards

herbert0709 · September 1, 2019, 20:41

Hi Bruno

It seems that it was the server to have a technical issue from my last post to not respond.

And this is what i get when i run the process again when the cluster is normal again.

So when i try to load the openmpi-1.10.7 i give the following message:

Code:

[zzha518@monarch-login2 ~]$ module load openmpi/1.10.7-1.mlx
    DO NOT USE THIS MODULE> IT IS FOR TESTING ONLY
[zzha518@monarch-login2 ~]$ source /home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/etc/bashrc
bash: mpicc: command not found...
bash: mpicc: command not found...

So i changed to another openmpi (openmpi/1.10.3-gcc5). then i source the bashrc and get the following:

Code:

[zzha518@monarch-login2 ~]$ module load openmpi/1.10.3-gcc5
[zzha518@monarch-login2 ~]$ source /home/zzha518/ny81/zzha518/OpenFOAM/OpenFOAM-v1906/etc/bashrc
mpicc: error while loading shared libraries: libslurm.so.30: cannot open shared object file: No such file or directory
mpicc: error while loading shared libraries: libslurm.so.30: cannot open shared object file: No such file or directory
mpicc: error while loading shared libraries: libslurm.so.30: cannot open shared object file: No such file or directory

Any suggesting on where should i proceed next?

Thank you very much!!

Regards

August 25, 2019, 09:12		#2
wyldckat Retired Super Moderator Bruno Santos Join Date: Mar 2009 Location: Lisbon, Portugal Posts: 10,981 Blog Entries: 45 Rep Power: 128	Quick question: In that page you provided, it gives a link to the Build page: https://www.openfoam.com/code/build-guide.php - did you follow the instructions given on that page? __________________ OpenFOAM: FAQ \| Getting started Forum: How to get help, to post code/output and forum guide Read this before sending me PM

August 27, 2019, 19:01		#4
wyldckat Retired Super Moderator Bruno Santos Join Date: Mar 2009 Location: Lisbon, Portugal Posts: 10,981 Blog Entries: 45 Rep Power: 128	Quick answers: The command foamSystemCheck will only work after you activated the OpenFOAM shell environment, namely after you source the "bashrc" file. The message: Code: bash: mpicc: command not found... is because there is no Open-MPI ready to be used... check if you have any module with Open-MPI ready to be used by running: Code: module avail I expect it's something like "openmpi-x86_64" or "mpi/openmpi-x86_64" that is available in your system. If you do have it, you can load it by running, for example: Code: module load mpi/openmpi-x86_64 then source OpenFOAM's "bashrc" file once again. The message: Code: No completion added for ... is being shown because OpenFOAM-v9106 hasn't been compiled yet. Then to compile OpenFOAM-v1906, use the following commands: Code: ./Allwmake -j 4 -log=log.make where 4 is the number of cores to use to build/compile OpenFOAM. The output is sent to the file "log.make" which you can check its contents on another terminal, e.g. by running: Code: tail -F log.make This log file is critical for diagnosing any compilation problems.

August 28, 2019, 19:21		#6
wyldckat Retired Super Moderator Bruno Santos Join Date: Mar 2009 Location: Lisbon, Portugal Posts: 10,981 Blog Entries: 45 Rep Power: 128	Quick answer: The commands you should execute are as follows: Code: module load openmpi/1.10.7-mlx source $HOME/OpenFOAM/OpenFOAM-v1906/etc/bashrc (I'm assuming the path on the second line works in your installation, but if not, adjust accordingly) Then check what error messages you get. If you only get the error about "no completion added", then Open-MPI should be ready to be used with OpenFOAM. If so far so good, then run: Code: foam ./Allwmake -j 4 -log=log.make However, if sourcing OpenFOAM's "etc/bashrc" file gave the error about mpicc or some other error, then run: Code: echo $MPI_ARCH_PATH ls -l $MPI_ARCH_PATH ls -l $MPI_ARCH_PATH/include and tell us what it gives you.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Other] Basic questions about OpenFOAM cluster running and installing	Fauster	OpenFOAM Installation	0	May 25, 2018 16:00
[OpenFOAM.org] OpenFOAM Cluster Setup for Beginners	Ruli	OpenFOAM Installation	7	July 22, 2016 05:14
How to setup a simple OpenFOAM cluster?	TommiPLaiho	OpenFOAM Installation	3	October 27, 2013 16:15
New OpenFOAM Forum Structure	jola	OpenFOAM	2	October 19, 2011 07:55
unable to install openfoam 1.7 using apt-get	maazhmd	OpenFOAM Installation	7	July 24, 2011 12:18