[wyldckat]

Hi Aditya,

OK, I think I've figured it out, although there was an addition detail I should have asked for.

Anyway, let's try this:

1. First, you need to edit the file "~/.bashrc" and change this:
   

   # .bashrc
   
   # Source global definitions
   if [ -f /etc/bashrc ]; then
       . /etc/bashrc
   source $HOME/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc
   fi
   
   # User specific aliases and functions
   
   export PATH=/usr/lib64/openmpi/bin/:$PATH 
   export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH
   export ParaView_DIR=/home/aditya/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/ParaView-4.1.0
           export PATH=$ParaView_DIR/bin:$PATH
           export PV_PLUGIN_PATH=$FOAM_LIBBIN/paraview-4.1

   To this:
   

   # .bashrc
   
   # Source global definitions
   if [ -f /etc/bashrc ]; then
       . /etc/bashrc
   fi
   
   # User specific aliases and functions
   
   module load mpi/openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin
   
   source $HOME/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc

   This modified version should have everything that's necessary for things to work well, without having to define all of the values you had.
2. Let's define Fedora's Open-MPI installation as the one to be used with OpenFOAM. You can do this by running the following commands:
   

   foam
   echo "export WM_MPLIB=SYSTEMOPENMPI" >> etc/prefs.sh

3. Now, let's update the shell environment, by running:
   

   wmSET

4. Finally, we'll need to run Allwmake to build some of the missing pieces. Run the following commands:
   

   foam
   ./Allwmake > make.log 2>&1

   It shouldn't take too long, since it should only have to build Scotch and "libPstream.so". I think that it should take somewhere between 5 and 20 minutes to build.
5. Then run again the last command:
   

   ./Allwmake > make.log 2>&1

   This will get us a summary of the build, making it easier to diagnose any problems.

Finally, try running decomposePar on your case and then try to run it in parallel.
If it doesn't work, then please run the following commands:

foam
pwd
gzip < make.log > make.log.gz

Then attach the file "make.log.gz" to your next post. The file is located at the path indicated by the second command, namely shown by pwd.

Best regards,
Bruno

[dsn]

Hi Bruno,

I was trying to go through the steps and I got an error as shown below after the second step where

foam
echo "export WM_MPLIB=SYSTEMOPENMPI" >> etc/prefs.sh

was used. The error is as follows,

[aditya@localhost OpenFOAM-2.3.0]$ echo "export WM_MPLIB=SYSTEMOPENMPI" >> etc/prefs.sh

[aditya@localhost OpenFOAM-2.3.0]$ wmSET
Cannot open configuration file /home/aditya/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/openmpi-1.6.5/share/openmpi/mpicc-wrapper-data.txt
Error parsing data file mpicc: Not found

Looks like some file is missing

[wyldckat]

Hi Aditya,

I thought that you had installed Open-MPI... OK, then you can install it by running these commands:

1. Switch to root mode:
   

   su -

2. Install the missing packages for the system's Open-MPI:
   

   yum install openmpi openmpi-devel

3. Exit root mode:
   

   exit

Then start a new terminal window and continue from step #4, from my previous post.

Best regards,
Bruno

[dsn]

Hi Bruno,

I am really grateful for your patient replies. Inspite of the error, I continued with the other steps just to see what will happen and then I used the decomposePar on a trial case.

and it worked. The output is shown below.

[aditya@localhost depthCharge2D_basecase]$ decomposePar
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.3.0                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.3.0-f5222ca19ce6
Exec   : decomposePar
Date   : Apr 07 2015
Time   : 19:21:39
Host   : "localhost.localdomain"
PID    : 3704
Case   : /home/aditya/OpenFOAM/OpenFOAM-2.3.0/tutorials/multiphase/compressibleInterFoam/laminar/depthCharge2D_basecase
nProcs : 1
sigFpe : Enabling floating point exception trapping (FOAM_SIGFPE).
fileModificationChecking : Monitoring run-time modified files using timeStampMaster
allowSystemOperations : Disallowing user-supplied system call operations

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time



Decomposing mesh region0

Create mesh

Calculating distribution of cells
Selecting decompositionMethod simple

Finished decomposition in 0 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 3200
    Number of faces shared with processor 1 = 80
    Number of faces shared with processor 2 = 40
    Number of processor patches = 2
    Number of processor faces = 120
    Number of boundary faces = 6520

Processor 1
    Number of cells = 3200
    Number of faces shared with processor 0 = 80
    Number of faces shared with processor 3 = 40
    Number of processor patches = 2
    Number of processor faces = 120
    Number of boundary faces = 6520

Processor 2
    Number of cells = 3200
    Number of faces shared with processor 0 = 40
    Number of faces shared with processor 3 = 80
    Number of processor patches = 2
    Number of processor faces = 120
    Number of boundary faces = 6520

Processor 3
    Number of cells = 3200
    Number of faces shared with processor 1 = 40
    Number of faces shared with processor 2 = 80
    Number of processor patches = 2
    Number of processor faces = 120
    Number of boundary faces = 6520

Number of processor faces = 240
Max number of cells = 3200 (0% above average 3200)
Max number of processor patches = 2 (0% above average 2)
Max number of faces between processors = 120 (0% above average 120)

Time = 0

Processor 0: field transfer
Processor 1: field transfer
Processor 2: field transfer
Processor 3: field transfer

End.

Does it mean that it just worked for this one case or does everything look to be ok?

[dsn]

Just to verify that openmpi is installed, i used the yum install command and the terminal told me that its already installed.

[aditya@localhost ~]$ su
Password: 

[root@localhost aditya]# yum install openmpi openmpi-devel
Loaded plugins: langpacks, refresh-packagekit
Dropbox                                                  |  951 B     00:00     
dl.atrpms.net_el6-x86_64_atrpms_stable                   | 3.0 kB     00:00     
google-chrome                                            |  951 B     00:00     
updates/19/x86_64/metalink                               | 4.0 kB     00:00     
google-chrome/primary                                      | 1.9 kB   00:00     
google-chrome                                                               3/3
Package openmpi-1.6.5-1.fc19.x86_64 already installed and latest version
Package openmpi-devel-1.6.5-1.fc19.x86_64 already installed and latest version
Nothing to do

[dsn]

But when I run the case using the parallel run command as shown below i get an error again..

[aditya@localhost depthCharge2D_basecase]$ mpirun -np 4 compressibleInterFoam -parallel > log
--------------------------------------------------------------------------
Sorry!  You were supposed to get help about:
    opal_init:startup:internal-failure
But I couldn't open the help file:
    /home/aditya/OpenFOAM/ThirdParty-2.3.0/platforms/linux64Gcc/openmpi-1.6.5/share/openmpi/help-opal-runtime.txt: No such file or directory.  Sorry!
--------------------------------------------------------------------------
[localhost.localdomain:04210] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in file runtime/orte_init.c at line 79
[localhost.localdomain:04210] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in file orterun.c at line 694

[wyldckat]

Hi Aditya,

Sorry, busy week and I haven't had a chance to test things any sooner.

I forgot a detail in the instructions I gave in post #21... step #3 is wrong
The correct commands to be executed (for step #3) are these:

wmUNSET
module load mpi/openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin
source $HOME/OpenFOAM/OpenFOAM-2.3.0/etc/bashrc

Or simply start a new terminal/console window.

Then please follow again from step #4 and onward, on post #21.

Best regards,
Bruno

[dsn]

Hi Bruno,

When I use that command, I get the following error..

[aditya@localhost OpenFOAM-2.3.0]$ wmUNSET

[aditya@localhost OpenFOAM-2.3.0]$ module load mpi/openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin
mpi/openmpi-x86_64(5):ERROR:150: Module 'mpi/openmpi-x86_64' conflicts with the currently loaded module(s) 'mpi/mpich-x86_64'
mpi/openmpi-x86_64(5):ERROR:102: Tcl command execution failed: conflict		mpi

Did I screw up anything anywhere?

Thanks!!

[wyldckat]

Quick question: Do you need the module "mpi/mpich-x86_64" to be loaded?

[dsn]

Hi,

yeah I wish to use mpich as my default parallel compiler. I have that installed in my system already.

If that can be possible.

THanks a ton once again!!!

[wyldckat]

Hi Aditya,

Originally Posted by dsn

yeah I wish to use mpich as my default parallel compiler. I have that installed in my system already.

You should have said that last week. There was no clue among the data I asked for that MPICH was the MPI toolbox you are using in your system.
Unfortunately I don't have enough time before next weekend to test which instructions are best for you to use MPICH instead of Open-MPI. All I know is that you now have a serious conflict of MPI toolboxes, which is complicating everything

From the top of my head, what you need to do is:

1. Uninstall the two packages I said before that you should install, namely by running as root:
   

   yum remove openmpi openmpi-devel

2. Run as normal user:
   

   foam
   echo "export WM_MPLIB=MPICH" > etc/prefs.sh

3. Code similar to this:
   

   module load mpi/openmpi-x86_64 || export PATH=$PATH:/usr/lib64/openmpi/bin

   must be removed from the "~/.bashrc" file, because it's creating an unnecessary conflict with MPICH.
4. And I guess you need this:
   

   module load mpi/mpich-x86_64

   in the "~/.bashrc" file, before it sources OpenFOAM's "bashrc" file.
5. And then start a new terminal/console window, so that you can have a clean and working shell environment.
6. Then you need to run Allwmake once again, as I instructed in previous posts.

Good luck! Best regards,
Bruno

[adityadsn]

Hey Bruno,

SOrry for the confusion. I am not particular about the MPI toolbox. Can you please tell what needs to be done to have the openmpi only as the default MPI toolbox?

Shall I uninstall MPICH toolbox. will that remove the conflicting commands?

Regards,
Aditya

[wyldckat]

Originally Posted by adityadsn

Shall I uninstall MPICH toolbox. will that remove the conflicting commands?

If you didn't have MPICH installed, the instructions I've written in the past posts should have worked as intended.

[adityadsn]

I will try that out and let you know. Thank you so much.

Can i use the command shown below to remove it? I apologise for asking such basic questions!!

yum remove mpich-x86_64

Regards,
Aditya

[dsn]

Hi,

I removed mpich and tried using the previous commands with the necesary modifications suggested by you. But it still doesnt work..

I have attached the make.log.gz as instructed.

[wyldckat]

Quick answer: I forgot that since you had both MPICH and Open-MPI installed in the system, there was a very weird collision of files. Try the following commands, to see if they fix the problem, since they do a clean up and then rebuild what's missing:

foam
rm $FOAM_LIBBIN/openmpi-system/libPstream.so
rm $FOAM_EXT_LIBBIN/$FOAM_MPI/libptscotch*
rm -r $WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/scotch*
wclean all src/Pstream
./Allwmake > log.make 2>&1

Hopefully this will make your installation operational once again.

[dsn]

Hi,

There seems to be an error for the libscotch command as follows,

[aditya@localhost OpenFOAM-2.3.0]$ rm $FOAM_EXT_LIBBIN/$FOAM_MPI/libptscotch*
rm: cannot remove ‘/home/aditya/OpenFOAM/ThirdParty-2.3.0/platforms/linux64GccDPOpt/lib/openmpi-system/libptscotch*’: No such file or directory

Regards

[wyldckat]

Originally Posted by dsn

There seems to be an error for the libscotch command as follows,

OK, then that seems it wasn't built in the first place. Proceed with the commands after that one.

[dsn]

Hi Bruno,

Everything seems to be running perfectly now. I even tried running it in parallel and everything worked out well!!!!

THank you so much!!! I was so stuck with all these problems since a long time!!

Is there any forum or tutorial where I can get a detailed understanding of how OF interacts with the shell and etc, so that i can atleast know whats happening

Regards,
Aditya

[wyldckat]

Originally Posted by dsn

Is there any forum or tutorial where I can get a detailed understanding of how OF interacts with the shell and etc, so that i can atleast know whats happening

Quick answer: http://openfoamwiki.net/index.php/In...with_the_Shell