[Kolchmob0711]

Dear all,

first of all, I'm really proud to have a forum like this one and till now I could solve every of my problems by reading the threads carefully.
But now I reached a point, at which I stuck...
I build a small cluster computer (4 nodes) for my lab and we already run some software like ANSYS and NUMECA on it. Therefor we decided to use the MPICH2 (Version 1.4.1) as mpi-environment and everything works fine. The only problem is with OpenFoam. I know OpenFoam uses as standard the OpenMPI and I also know that it is possible to make OpenFoam with MPICH2 (but how?)
Till now I changed the statement at /etc/bashrc to MPICH and in /etc/config/settings.sh I tried to set the names under mpich for my case. Then I went to main folder and made ./Allwmake...But it doesn't work, when I run a job in parallel I get an error message, that something went wrong with Pstream.
It would be really nice if somebody of you can provide me a simple line-by-line tutorial to fix that problem (and I don't have the possibility to switch to OpenMPI)

Greets
Kolchmob

[jherb]

These are the changes, I did to use MPICH2:

in $WM_PROJECT_DIR/etc/bashrc:

export WM_MPLIB=MPI-MVAPICH2

in $WM_PROJECT_DIR/etc/config/settings.sh add to

case "$WM_MPLIB" in

...

MPI-MVAPICH2)
    export MPI_HOME=/usr/lib64/mpi/gcc/mvapich2/ # you probably have to change this
    export MPI_ARCH_PATH=$MPI_HOME
    _foamAddPath $MPI_ARCH_PATH/bin
    _foamAddLib     $MPI_ARCH_PATH/lib64
    export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/mvapich2
    ;;

in $WM_PROJECT_DIR/wmake/rules/linux64Gcc (if you use 64bit linux and gcc, else you have to chose the correct directory) add the file mplibMPI-MVAPICH2 with the following content:

PFLAGS     = -DMPICH_SKIP_MPICXX
PLIBS      = -L$(MPI_ARCH_PATH)/lib64 -lmpich -lrt -lopa
PINC       = -I$(MPI_ARCH_PATH)/include

This worked for OpenFOAM 2.3.1. Your paths (and perhaps also the compiler flags) might be different but this should help you for the start.

But I guess you have to recompile OpenFOAM.

[Kolchmob0711]

Hi Joachim,

thanks a lot for your help...now it's working. The missing step was to create that mplib file in rules-directory. I recompiled only the Pstream and for me that was fine. I already started the first simulations on the cluster :-)))
Thx
Greets

[yyz987]

Hi Kolchmob,

I am a beginner of OpenFoam, but I need to installing OpenFoam using MPICH2 just as you did. It will be very grateful of you if you can share your specific configuration in doing this job on this forum. My English is not well good, I hope you can understand what I mean.

Greets
YYZ

[Kolchmob0711]

Hello yyz987,

sorry for my late response but I was on vacation...
What i did:
1. Change the bashrc: uncomment export WM_MPLIB=SYSTEMOPENMPI and use export WM_MPLIB=mpich instead
2. Adapt the entries in etc/config/settings.sh to your mpich2 installation (Check with echo commnad if all the enviromentals are well defined)
3. Add the file in wmake/rules /linux64Gcc (see the post of Joachim)
4. For me it works to make a ./Allwmake in src/Pstream
5. It should work then...perhaps you have to check with sudo update-alternatives --config mpirun, which mpi flavor (I guess you also have openmpi) should be use

Add also in your ~/.bashrc WM_MPLIB=MPICH if you defined an alias like of240

Greets,
Kolchmob

[yyz987]

Hi Kolchmob,

Thanks for your line-by-line tutorial first. I come from China, and I'm not familiar with ubuntu command. I'm trying to configure OpenFoam as you did. My ultimate goal is not to make OpenFoam with MPICH2, but to make CFDEMcoupling software with MPICH2. I'm also very interested in the small cluster computer (4 nodes) of your lab. I want to make a small cluster computer for my lab and run my CFDEMcoupling software on it. I want to know if it is a must to use MPICH2 instead of OpenMPI if I'm going to make a small cluster computer for my lab. And chould you give me some help about what I should learn as a start.

Greets
YYZ

[Kolchmob0711]

Hi yyz987,

The answer is: it depends...some 'commercial' software is compiled with mpich some another with openmpi.
I've never heard about this software (CFDEM), but I read a little bit about it and so I guess it is OpenMpi because of its opensource (you can also recompile it with another mpi-flavor). I think for more detailed informations you should contact the customer.
Regarding to the cluster itself: https://help.ubuntu.com/community/MpichCluster and https://www.digitalocean.com/communi...-vps-instances.
This two tutorials help me (as a normal engineer) to setup the cluster. As job scheduler you can use http://www.openlava.org/

Greets
Kolch

[yyz987]

Hi Kolchmob,

CFDEMcoupling is a hybrid software using OpenFoam and LIGGGHTS. LIGGGHTS is
a DEM simulation software. If you want to know more details, you can see the introduction on http://www.cfdem.com. CFDEMcoupling is using the same mpi-flavor as OpenFoam(OpenMPI), but there may be some differences between them. I am a new learner, and I am still working hard to study it...

Greets
YYZ

[ettaka]

Hello,

I tried compiling OpenFOAM to our cluster. I had a problem:

In file included from /export/modules/apps/mvapich2/2.1/gnu/include/mpi.h:2237:0,
from ptscotchDecomp.C:125:
/export/modules/apps/mvapich2/2.1/gnu/include/mpicxx.h:2737:34: error: conflicting declaration of C function ‘void MPI::Init(int&, char**&)’
extern void Init(int &, char **& );
^
/export/modules/apps/mvapich2/2.1/gnu/include/mpicxx.h:2736:13: note: previous declaration ‘void MPI::Init()’
extern void Init(void);
^
/export/modules/apps/mvapich2/2.1/gnu/include/mpicxx.h:2739:45: error: conflicting declaration of C function ‘int MPI::Init_thread(int&, char**&, int)’
extern int Init_thread(int &, char **&, int );
^
/export/modules/apps/mvapich2/2.1/gnu/include/mpicxx.h:2738:12: note: previous declaration ‘int MPI::Init_thread(int)’
extern int Init_thread(int);
^
This is what I did to make it work:

diff --git a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C
index 29f1f6b..c8a3acd 100644
--- a/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C
+++ b/src/parallel/decompose/ptscotchDecomp/ptscotchDecomp.C
@@ -118,11 +118,11 @@ License
 #include "OFstream.H"
 #include "globalIndex.H"
 #include "SubField.H"
+#include <mpi.h>

 extern "C"
 {
     #include <stdio.h>
-    #include <mpi.h>
     #include "ptscotch.h"
 }

See also:
http://lists.mcs.anl.gov/pipermail/m...ry/006405.html

[popstar0426]

Hi, Kolchmob:
I installed mpich2 in ubuntu with "sudo apt-get install mpich2"
Do you know where is the MPI_HOME for mpich2?

Best regards!

[HerbertPrater]

what is my IP address provides free lookup tool to check location of your IP Address Data is gathered through several GEO IP data providers Just enter an IP and check location