[KarthickRajkumar]

Dear Foamers,

I am using foam-extend 3.2 for some time now. But I have never used cluster for running the cases.
I am facing some problems which I would like to share and could probably solve through your comments!

Well!! I want run (a serial job) my simulation as a normal user in Rocks cluster - cent OS 6.8 version, installed version: Foam-extend-3.2.
So, let me explain what I did before posting the troubles I am facing

1. I copied the cavity tutorial case to my working directory
2. Inside the case folder, I created a script file cavityscript.sh with PBS commands as below

#PBS -N testInstall
#PBS -l nodes=1:ppn=1
#PBS -l walltime=01:00:00  

##move into cavity directory
cd /home/y0080051/foam/y0080051-3.2/cavity

##source environmental variables
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc
module load openmpi-1.10-x86_64
blockMesh
icoFoam

1. I learnt that we have to load module files in order to use foam-extend-3.2! But should we need to create the module file on our own, because I do not see foam-extend-3.2 module though "module avail" command in the cluster which has foam-extend-3.2 installed. If I have to create it myself, how would I do this?

After all, I see that loading modules is basically loading all environmental variables, which I think the following line from the script file should do it.

Could anyone explain me why should load module if I include this?!

##source environmental variables
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc

2. Other than loading foam-extend-3.2 module, do I need to load any other modules? I see openmpi module is required only for parallel job, so I may not require that! Can anyone tell me what all should I need to load to run a serial job?

3. I submit the file using the following command

qsub -cwd cavityscript.sh

The error that I face always are "icoFoam command not found" and "blockMesh command not found"!

But if I type "icoFoam -help" and "blockMesh -help" in the terminal with loaded alias command for foam-extend-3.2, I can see the usage commands of those. Then, why does it not work if I give those via a script file.

Could anyone help me out from this!
Any help would greatly be appreciated!

Many thanks,
Karthick Rajkumar

[wyldckat]

Greetings Karthick,

I don't know if you've managed to solve this problem or not, but here are a few answers that should at least answer part of your questions. And many thanks for providing all of the necessary details to diagnose this!

Originally Posted by KarthickRajkumar

2. Inside the case folder, I created a script file cavityscript.sh with PBS commands as below

#PBS -N testInstall
#PBS -l nodes=1:ppn=1
#PBS -l walltime=01:00:00  

##move into cavity directory
cd /home/y0080051/foam/y0080051-3.2/cavity

##source environmental variables
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc
module load openmpi-1.10-x86_64
blockMesh
icoFoam

1. I learnt that we have to load module files in order to use foam-extend-3.2! But should we need to create the module file on our own, because I do not see foam-extend-3.2 module though "module avail" command in the cluster which has foam-extend-3.2 installed. If I have to create it myself, how would I do this?

After all, I see that loading modules is basically loading all environmental variables, which I think the following line from the script file should do it.

Could anyone explain me why should load module if I include this?!

##source environmental variables
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc

Modules are a nice way to load and unload software from the shell environment. For example, if you want Open-MPI 1.10 but loaded 1.08 by mistake, you can unload it with something like:

module unload openmpi-1.08-x86_64

This ensures (or at least tries to) that the environment is properly loaded with the correct software versions.

The source command is loading the foam-extend 3.2 environment, in a way that is not as clean as the module system, but does its job correctly when it comes to loading its environment.

Now, what I can see wrong with the order you've used is that it should be switched around, namely it should be in this order:

##source environmental variables
module load openmpi-1.10-x86_64
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc

This is because foam-extend needs to know what Open-MPI is installed.

For more details on the topic of shell environment for OpenFOAM/foam-extend, read this page: http://openfoamwiki.net/index.php/In...with_the_Shell

Originally Posted by KarthickRajkumar

2. Other than loading foam-extend-3.2 module, do I need to load any other modules? I see openmpi module is required only for parallel job, so I may not require that! Can anyone tell me what all should I need to load to run a serial job?

This is where things get a bit tricky. I depends on how foam-extend 3.2 was installed on the cluster. You need to contact the system administrator and ask how "/state/partition1/apps/OpenFOAM/foam/foam-extend-3.2" can be used properly within the cluster. Because there might be other modules that need to be loaded, but I have no way to properly guess without being able to do a full step-by-step check of what is installed in that cluster, starting by checking what this command gives:

module avail

Originally Posted by KarthickRajkumar

3. I submit the file using the following command

qsub -cwd cavityscript.sh

The error that I face always are "icoFoam command not found" and "blockMesh command not found"!

But if I type "icoFoam -help" and "blockMesh -help" in the terminal with loaded alias command for foam-extend-3.2, I can see the usage commands of those. Then, why does it not work if I give those via a script file.

The problem here is that the file system paths on the connection machine that you are accessing to, may have different paths for the machines within the cluster.

You can try the following script to try and diagnose how the shell environment looks like from within the cluster:

#PBS -N testInstall
#PBS -l nodes=1:ppn=1
#PBS -l walltime=01:00:00  

##move into cavity directory
cd /home/y0080051/foam/y0080051-3.2/cavity

##source environmental variables
 module load openmpi-1.10-x86_64
source /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/etc/bashrc

export > shell_environment.txt 2>&1
find /state/partition1/apps/OpenFOAM/foam/foam-extend-3.2/ -type f > installed_files.txt 2>&1

Two files will be created inside your case directory:

• "shell_environment.txt" - this contains the shell environment that was loaded within the shell that the job ran.
• "installed_files.txt" - this contains a complete list of the files that exist within the foam-extend 3.2 installation.

If you can attach these files, then please zip them first and then attach them on your next post.


Best regards,
Bruno

[KarthickRajkumar]

Dear Bruno,

Thanks for the well written reply. I talked to the system administrator and I was able to run the simulations in the cluster.

Now I am in the final stage, documenting the work!

In general, your messages in the forum were of great help to my work and many thanks again!!

Regards,
Karthick