[Carlen]

Hi foamers,

I am experiencing a problem when using pow() function for parallel run

I want to calculate the diametre of an identified droplet using the code below

pD = dropletV[i]*6.0/mathematical::pi;
dropletD[i] = Foam::pow((pD), 1/2.0);

This works perfectly with only one CPU core, however it crashes halfway in the parallel run.

I went back to check every value of dropletD array and they are all reasonable. So I figured a way to test the pow() function by manually putting in a calculated pD value and it worked fine. However, whenever I put in pD the simulation crashes. Can anyone help me with this problem? is there another alternative for pow() function?

[wyldckat]

Quick answer: Try using the square root instead?

dropletD[i] = Foam::sqrt(pD);

[Carlen]

Originally Posted by wyldckat

Quick answer: Try using the square root instead?

dropletD[i] = Foam::sqrt(pD);

Actually, putting 1.0/2.0 was a way I used to test the pow function. I want to evaluate dropletD[i] = Foam:ow(pD, 1.0/3.0) to find the diametre of the droplet. It seems to be the only way to find "to the power of 1/3". It crashes every time I try to run it in parallel. Any idea how to solve the problem?

[alexeym]

Hi,

Can you post the error solver produces in parallel run? If you really think it has something to do with pow function, next question will be: where do you get dropletV array? How you iterate over the array?

[sharonyue]

I also tried with this, there is a simple way to reproduce this problem:

Add this into interFoam:

double asdfasfd = Foam::pow(alpha1[0],1.0/2.0);

it crushed in parallel mode, but runs smoothly in serial. This is logfile:

Courant Number mean: 0.0369924 max: 0.281874
Interface Courant Number mean: 0.00198018 max: 0.264548
deltaT = 0.00473396
Time = 0.0263302

MULES: Solving for alpha1
Phase-1 volume fraction = 0.130194  Min(alpha1) = -1.24957e-55  Max(alpha1) = 1
MULES: Solving for alpha1
Phase-1 volume fraction = 0.130194  Min(alpha1) = -1.18514e-55  Max(alpha1) = 1
sdfsdfsd
sdfsdfsd
sdfsdfsd
sdfsdfsd
[3] #0  Foam::error::printStack(Foam::Ostream&) at ??:?
[3] #1  Foam::sigFpe::sigHandler(int) at ??:?
[3] #2   in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #3   in "/lib/x86_64-linux-gnu/libm.so.6"
[3] #4  pow in "/lib/x86_64-linux-gnu/libm.so.6"
[3] #5  
[3]  at ??:?
[3] #6  __libc_start_main in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #7  
[3]  at ??:?
[q-virtual-machine:03985] *** Process received signal ***
[q-virtual-machine:03985] Signal: Floating point exception (8)
[q-virtual-machine:03985] Signal code:  (-6)
[q-virtual-machine:03985] Failing at address: 0x3e800000f91
[q-virtual-machine:03985] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f58302c3c30]
[q-virtual-machine:03985] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7f58302c3bb9]
[q-virtual-machine:03985] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x36c30) [0x7f58302c3c30]
[q-virtual-machine:03985] [ 3] /lib/x86_64-linux-gnu/libm.so.6(+0x15a95) [0x7f583087ea95]
[q-virtual-machine:03985] [ 4] /lib/x86_64-linux-gnu/libm.so.6(pow+0x1c) [0x7f583088f9ac]
[q-virtual-machine:03985] [ 5] interFoam() [0x4278e6]
[q-virtual-machine:03985] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f58302aeec5]
[q-virtual-machine:03985] [ 7] interFoam() [0x42af5a]
[q-virtual-machine:03985] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 3 with PID 3985 on node q-virtual-machine exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------
q@q-virtual-machine:~/OpenFOAM/OpenFOAM-2.2.x/tutorials/multiphase/interFoam/laminar/damBreak$ mpirun -np 4 interFoam -parallel
/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | OpenFOAM: The Open Source CFD Toolbox           |
|  \\    /   O peration     | Version:  2.2.x                                 |
|   \\  /    A nd           | Web:      www.OpenFOAM.org                      |
|    \\/     M anipulation  |                                                 |
\*---------------------------------------------------------------------------*/
Build  : 2.2.x
Exec   : interFoam -parallel
Date   : Feb 22 2015
Time   : 10:46:33
Host   : "q-virtual-machine"
PID    : 4044
Case   : /home/q/OpenFOAM/OpenFOAM-2.2.x/tutorials/multiphase/interFoam/laminar/damBreak
nProcs : 4
Slaves :

To bo noticed that, by the Info I can see that the solver can calculate this pow() function perfectly. but when the solver get into Ueqn or pEqn. it blows up. However, this value got from pow() function is not related with flow field.


But this works:

double asdfasfd = Foam::pow(4,1.0/2.0);

volScalarField asdsdfd = Foam::pow(alpha1,1.0/2.0);

Hopes this infomation will make this problem more clear.


BTW, all of these cases can be launched without any problem in serial mode.

[Carlen]

Thank you Dongyue,

Your supplementary comment does make my question clearer

[wyldckat]

Greetings to all!

I have to say that I cannot find enough information in the examples you two have provided, in order for me to properly test this and to be 100% certain of what I'm going to write, but my guess/estimate is this:

1. "Foam:: pow" was designed for field calculations. This means that all of the mumbo-jumbo needed for running an application in parallel, is carefully embedded within "Foam:: pow" and all other similar mathematical functions in OpenFOAM.
2. This to say that "Foam:: pow" will take care of processing only the data relevant to each processor, depending on how the data is handled for each processor. Synchronization will also be taken care of automagically.
3. Now, the operation that seems to be problematic seems to be related to accessing only the value for one specific cell in a field array. This means that there is a risk that the function will try to synchronize a field operation when it shouldn't, because there is only one cell that matters.
4. Therefore, my guess here is that you should instead be using something like this:
   

   double asdfasfd = ::pow(alpha1[0].value(),1.0/2.0);

   This way it will only operate on the numbers themselves and use the standard mathematical C/C++ "pow" function, instead of OpenFOAM's own "pow" implementation, which is parallel-ready.

If this doesn't make much sense, then please, provide a full and clear example, not just small snippets of code

Best regards,
Bruno

[Carlen]

Hi Bruno,

Thank you for your suggestion. I have experienced another problem when integrating your line of code in my solver.

error: request for member ‘value’ in ‘(& alpha1)->Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh>::<anonymous>.Foam:imensionedField <double, Foam::volMesh>::<anonymous>.Foam::Field<double>::< anonymous>.Foam::List<double>::<anonymous>.Foam::U List<T>:perator[]<double>(cellI)’, which is of non-class type ‘double’
if (alpha1[cellI].value() > 0.8)

it says value is of non-class type of ‘double' and the compilation fails. Any suggestion to solve this problem?

we just simply added "volScalarField asdfs = Foam:ow(alpha1[0], 1.0/2.0) just above the runtime.write() line of interFoam.C and the solver gave the aforementioned error. The case we used to test the line of code is the standard dambreak tutorial case.

[wyldckat]

Hi Carlen,

Sorry, I was drawing ideas directly from what I remembered and ".value()" is actually only applicable for "dimensionedScalar" and similar classes/objects.

Nonetheless, I've fortunately managed to reproduce the problem and it's fairly simple... remember the error message stating something about "sigfpe"? Well, it means this: http://en.wikipedia.org/wiki/SIGFPE#SIGFPE

If you haven't figured it out yet, then here's the demonstration in source code:

• This works just fine:
  

          if(alpha1[0]>=0.0)
          {
              double cookies = ::pow(alpha1[0], 0.5);
              Info<< "cookies: " << cookies << endl;
          }

• This doesn't:
  

          //if(alpha1[0]>=0.0)
          {
              double cookies = ::pow(alpha1[0], 0.5);
              Info<< "cookies: " << cookies << endl;
          }

Why? Because the root of any negative value gives a non real value, namely it gives a complex value. And since interFoam isn't in the business model of simulating with complex/imaginary numbers, there is no special handling of such values; i.e. everything should be done in the real mathematical plane.

And here I was imagining that it could have something to do with OpenFOAM's parallel coding mechanisms... and it was all due to a matter of understanding mathematics.

Oh, as to why "alpha1" would have negative values? Well, in my tests, the values were negative, but they were very small, very near 0.0... therefore, they were computationally/numerically as good as 0.0.
As for why it would only crash in parallel? Well, you were "lucky", in the sense that probably the values were only this bad for "alpha1[0]" in processor 2, because this cell was actually somewhere in the middle of the domain, while in serial mode, the "alpha1[0]" is always near the wall or floor (i.e always "1.0").

Best regards,
Bruno

[Carlen]

Thank you very much Bruno!

Your method perfectly solves my problem. I very much appreciate you taking time to answer my question.

Best regards
Carlen