First of all, I thanks the people that create OpenFOAM to make it free software. I have been looking for a good piece of open software like this for some time. I intend to use OpenFOAM in CFD teaching, thesis development and research projects.

I have just begin to explore the OpenFOAM capabilities and I intent to run it in a Rocks cluster (Makalu, 3.3), which already has MPICH 1.2.6 compiled with gcc 3.2.3 and the Sun Grid Engine sheduler (SGE 5.3). I have already installed OpenFOAM in a personal account and run a tutorial iteratively in the frontend.

Does anyone has any experience in running OpenFOAM in a Rocks cluster? I would like to receive any hints on how to run it in parallel using the existing gcc-MPICH environment and then in batch mode using SGE.

Thanks in advance.

Do you have to use MPICH? We prefer LAM particularly on Linux where MPICH behaved very badly on large cases. Also gcc-3.4 is much better than gcc-3.2, do you have to use 3.2?

Dear Henry,

I do not have to use it but it seems to be a better solution in the long run because MPICH compiled with gcc is standard in the Rocks distribution. New Rocks releases will probably update gcc but they will probably continue to use MPICH and not LAM. Are you familiar with the Rocks distribution?

No I am not familiar with Rocks. We use SuSE 9.1,2 and always use the latest gcc and lam versions. I don't like to be constrained by the compiler version supplied with the operating system, it is rarely optimal for CFD. The last time I tried MPICH I was unable to run my large cases because of the bugs in the linux version so I moved over to lam which ran fine and gave better performance so I have stuck with it.

However OpenFOAM can now be compiled with any version of MPI (or PVM for that matter), it's just a matter of compiling the appropriate Pstream library with the comms library you want to use.

First, I am not an expert in parallel environments. I am basically an user that has to maintain his own clusters running without an IT professional - a third world problem - who considers Rocks a manageable solution.

ROCKS is a cluster distribution made to deploy and manage easily large clusters, having some in the top 500 built with it. I agree that not having the latest compiler may impose some limitations but ROCKS was made to run large problems. Thus, I plan to give its MPICH environment a try.

I have not read the OpenFOAM programmer manual yet for the details of OpenFOAM compilation. I will study it and then I will probably post some questions about its recompilation. I think you could help me a lot at that time. Thanks.

[maka]

PVM compiled OpenFOAM:

However OpenFOAM can now be compiled with any version of MPI (or PVM for that matter), it's just a matter of compiling the appropriate Pstream library with the comms library you want to use.

Can any one please, describe how to compile OpenFOAM with PVM? This has the advantage that PVM is a good friend of OpenMosix load balancing library (in contradictory to MPI). For detail you read:

http://howto.x-tend.be/openMosix-Summer2004/x146.html

Thanks,
Maka

[hjasak]

Have a look at:

/home/hjasak/OpenFOAM/OpenFOAM-1.2/src/Pstream

Here, there is the protocol-dependent part of the Pstream class, currently implementing the dummy, mpi and the long-untouched pvm library.

First, you will need to set tPVM as the communications library in:

OpenFOAM-1.2/.cshrc

Look for the block called


# Communications library
# ~~~~~~~~~~~~~~~~~~~~~~
if ($WM_MPLIB == MPICH) then


and add an "else if" block for PVM

Then, revive the code in the Pstream section above and the rest should just work.

I'd appreciate the working setup for the .cshrc file and updates for the pvm Pstream once you get it working - thanks.

Enjoy,

Hrv