[hrn]

Hi,

I have for several years been installing OpenFOAM on an IBM P575+ supercomputer with AIX 5.3 and g++ 4.4.3. I have developed a patch for both OpenFOAM-1.7.1 and a patch for Thirdparty-1.7.1, so that OpenFOAM can be used on our supercomputer.

However, where the tutorials/interFoam/damBreak example from OpenFOAM 1.5.x from February 24, 2010 works just fine, the tutorials/multiphase/interFoam/ras/damBreak from OpenFOAM 1.7.1 gives me INF as result. See the attached file. As you can see, the change happens quite suddenly for Min(alpha1) that goes from a normal number to -4.07145e+155 in a single time-step

Are you interested in my patches?
Do you have an idea to what could be the problem?

Best regards,

Henrik

[wyldckat]

Greetings Henrik and welcome to the forum!

My experience in changing OpenFOAM code to make it work in supercomputers is pretty much none, since I don't have access to one

But I do have some experience in adapting OpenFOAM code to make it work in Windows and Linux.

So, my question is: what are the primary modifications you have done to the code? Were they:

• make it build statically or how library loading is done;
• some changes to make it work with the system's MPI;
• some changes to make it work with the file system;
• change how memory/object pointers are handled (including static and const type casts);
• change how mathematical operations are made (e.g. vector operations).

AFAIK, only the last two should affect operationability of the RAS model library.

The other possibility is that you're getting a bug in OpenFOAM 1.7.1 that has already been fixed in 1.7.x. So, it would be good if you could test running the same cases you have in a dual or quad core x86(_64) machine to validate if it should work or not... although you might have already done this, but you didn't tell us

So, before we get to the patches, if you could provide some more general information about your changes, it would helpful for either me or someone else in the forum community to help you!

Best regards,
Bruno

[hrn]

Greetings Henrik and welcome to the forum!

Thank you. Just to make it clear: I am a system administrator and not a person who knows a lot about CFD.

My experience in changing OpenFOAM code to make it work in supercomputers is pretty much none, since I don't have access to one

We have an additional minor IBM P575+ supercomputer for testing purposes that we could give the developers behind OpenFOAM access to.

• make it build statically or how library loading is done;

Yes, there are actually quite a few loader options in the patch that I no longer remember the purpose of. I will try to compile OpenFOAM 1.7.1 without them and see what happens.

• some changes to make it work with the system's MPI;

Yes, one must use IBM's version of MPI on AIX. There is no alternative.

• some changes to make it work with the file system;

No

• change how memory/object pointers are handled (including static and const type casts);

Yes, the transFunc define in floatScalar.H does not work on AIX for the functions j0, j1, jn, y0, and y1, so for these special functions, I had to use aixTransFunc:

#define transFunc(func)            \
inline Scalar func(const Scalar s) \
{                                  \
    return ::func##f(s);           \
}

#define aixTransFunc(func)         \
inline Scalar func(const Scalar s) \
{                                  \
    return ::func(static_cast<double>(s)); \
}

• change how mathematical operations are made (e.g. vector operations).

No

The other possibility is that you're getting a bug in OpenFOAM 1.7.1 that has already been fixed in 1.7.x. So, it would be good if you could test running the same cases you have in a dual or quad core x86(_64) machine to validate if it should work or not... although you might have already done this, but you didn't tell us

I have also installed OpenFOAM 1.7.1 and a recent version of 1.6.x on 2 Linux clusters. I have tested this case on all versions without any problems there. I will also try the latest version of 1.7.x and tell you how that goes, but I doubt that this part of the code has been changed, since there are no problems with it on Linux AFAIK. We have users who for years have used other parts of OpenFOAM on our supercomputer, apparently without encountering any problems with it.

Best regards,

Henrik

[wyldckat]

Hi Henrik,

Originally Posted by hrn

We have an additional minor IBM P575+ supercomputer for testing purposes that we could give the developers behind OpenFOAM access to.

Then you better ask OpenCFD for support, since they're the developers behind OpenFOAM

Originally Posted by hrn

Yes, one must use IBM's version of MPI on AIX. There is no alternative.

I assume you only needed to change/add the linking rules for that MPI, without changing the OpenFOAM library Pstream. But still, this probably isn't the reason.

Originally Posted by hrn

Yes, the transFunc define in floatScalar.H does not work on AIX for the functions j0, j1, jn, y0, and y1, so for these special functions, I had to use aixTransFunc:

#define transFunc(func)            \
inline Scalar func(const Scalar s) \
{                                  \
    return ::func##f(s);           \
}

#define aixTransFunc(func)         \
inline Scalar func(const Scalar s) \
{                                  \
    return ::func(static_cast<double>(s)); \
}

I suggest testing this with gcc 4.3.3 and 4.4.3, with both OpenFOAM 1.6.x and 1.7.1. Something might have changed from gcc 4.3.3 to 4.4.3 that will require some additional tweaking of your code for using aixTransFunc. For example, before OpenFOAM 1.7.1, OpenFOAM did build with gcc 4.5.0, but there was an incompatibility in the code that made some applications break, only because gcc had changed some optimizations from 4.4 to 4.5. So something similar might have happened for your modifications as well.

Best regards,
Bruno