[jvn]

I'd like to pose a question concerning preferred platforms and Linux distros. What would you consider optimal, very well posed, or definitively validated for a parallel implementation of OpenFOAM? When I search on-line, I'm inundated with Intel vs. AMD, SuSe vs. Redhat, etc. I figured that the code authors would be the authority on this. For reference, I'm thinking about cases on the order of 5-10 million cells. What have you found agreeable in these respects?
Thanks again,
JVN

[mattijs]

Re. hardware the AMD Opteron seems to be really good. Because of the nature of the code it is quite dependent on random memory access speed and the AMD seems to behave quite well.

For 5-10M cells you'd probably run 8 processors each with say 2Gb of memory. You'll need one machine with lots of memory to do the decomposition / reconstruction (ca. 2k per cell).
Make sure that you have lots and lots of swap space - not because we're running such big jobs but because it is very convenient to suspend jobs to run something more urgent inbetween.

We have no recent experience with any other interconnect than standard 100M/giga ethernet. If you're going for any specialist hardware make sure their MPI links with OpenFOAM (since their stuff usually comes precompiled and you don't get the sources)

As for Linux distros we stick with Suse since that is what we know but I guess any fairly recent one is ok.

This is only our experience. We would love to hear from other peoples experience.

[sergei]

Not much experience with FOAM, but with other CFD & FEM codes fairly enough.
At the moment, AMD 64 seems to be faster (but again, this may be code and problem dependent) and cheeper than Intel.
We have used (and still using) SuSE and RedHat (enterprise)recent distros (don't see principal difference), in cluster (beowulf-like) and sun grid enviroment (latter seems to be slower) running mpich,lam. 100m/Gigabit network
At the moment have 4Gb per node (2 processors).(didn't run problems over 2 mln cells yet).
However, looking for upgrade, with a bit more memory because of pre-processing (mesh generation can be cpu & memory intensive)

[grtabor]

I've just bought a 2x2GHz Opteron machine, 8GB RAM, produced by the company DNUK. Installation - no problems. I did my usual trick of running icoFoam on the lid driven cavity case, blinked, and missed the calculation. Very impressive machine - I look forward to using it for real!

[aike]

We are also in the process of evaluating OpenFOAM and conducting some benchmarks to determine parallel performance and speedup on Linux clusters. We have tested a problem (VOF surface capturing) mostly ~ 1.3 million cells up to 16 processors and also 1.9 million on 4 procs. These tests were conducted on IA32 systems, currently investigating performance on 64 bit platforms(AMD Opterons) and dual processor IA32 nodes.