[Arman_N]

Hello everyone,

I am using foam-extend 4.0 and have created a custom boundary condition based on constantAlphaContactAngle and it works fine with standard interFoam solver.
When I use this BC with a custom solver based on interFoam, it works fine with some simple mesh geometries, but crashes with some slightly more complex meshes and gives the error:

Segmentation fault (core dumped)

I tried putting some Info statements inside the BC code to find where the problem is, and it turned out to be where the BC wants to switch between different boundary patches (i.e. it loops over upperWall, but won't go through to lowerWall and crashes).


I later recompiled to Debug mode to find out where the crash happens, and surprisingly, the code works fine with no errors. But still crashes when I switch back to Opt mode. So I couldn't find where the problem stems form.

Now the problem is this: running in debug mode is very slow and I also couldn't manage to run it in parallel, where I get the following error:

--------------------------------------------------------------------------
It looks like opal_init failed for some reason; your parallel process is
likely to abort.  There are many reasons that a parallel process can
fail during opal_init; some of which are due to configuration or
environment problems.  This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):

  opal_shmem_base_select failed
  --> Returned value -1 instead of OPAL_SUCCESS
--------------------------------------------------------------------------

So each of my simulations takes too long to finish, which is not ideal to remain like this for the rest of the project.

Does anyone have any idea on where the problem could be, so I could run it successfully in Opt mode or at least parallelize it in Debug mode?

Many Thanks
Arman

[Arman_N]

OK, the problem with parallel run was because of incompatibility between the installed openmpi version and the system's own openmpi. It was solved when I changed:

${WM_MPLIB:=OPENMPI}

to

${WM_MPLIB:=SYSTEMOPENMPI}

in foam-extend-4.0/etc/bashrc directory and then recompiled foam-extend.

However, the problem with the Opt mode still remains and I have to run my simulations in Debug mode (which is still not as fast).

I would appreciate if anyone could give any help to resolve this issue.
Thanks.

[Daniel_Khazaei]

Hello,

"segmentation fault" error means that you are accessing a memory location that does not belong to the variable you are working with.
Can you describe how do you implement the looping over boundaries?
You are probably accessing an element beyond the array.

Regarding the BC seemingly working fine in debug mode and failing in Opt, does it produce reasonable result?
I don't have enough information about the differences between debug and Opt modes except for some basic aspects, but there is definitely something wrong in your code. In debug mode everything runs slower and probably your variable is still in memory and hence you don't get "segmentation fault" error.

Regards,
D. Khazaei

[Arman_N]

Hi Daniel,

Thanks for your response.

The part of my code for looping over the boundary in AlphaContactAngleFvPatchScalarField is like this:

{
    // Lookup T
    const volScalarField& T = this->db().objectRegistry::lookupObject<volScalarField>("T");

    // Lookup T on boundary patch
    const fvPatchScalarField& boundaryT = T.boundaryField()[patch().index()];

    // Define contact angle (theta0)
    scalarField theta0(size(), scalar(0));

    // Loop over theta0
    forAll(theta0, i) //equivalent to for (int i=0; patch.size()<i; i++)
    {
        // theta0 as a function of T on boundary patch
        scalar omega = (boundaryT[i] - lowT_) / (highT_ - lowT_);
        theta0[i] = min(max(omega * thetaHighT_ + (1 - omega) * thetaLowT_, scalar(0)), scalar(180));
    }

    return theta0;
}

I have checked the array sizes (theta0 and boundary patch) and also the counter (i), and they all seem to be equal. Therefore, I don't know where it wants to access an element beyond an array.

The simulation results (produced in Debug mode) "seem" reasonable, and the BC "apparently" does what it is supposed to do. But I can't exactly verify how reliable the results are.

Moreover, I don't think it is only the matter with Debug mode keeping the variables in memory, as the code also works fine in Opt mode with some simple meshes (i.e. 2D tube).

[Daniel_Khazaei]

Originally Posted by Arman_N

I have checked the array sizes (theta0 and boundary patch) and also the counter (i), and they all seem to be equal. Therefore, I don't know where it wants to access an element beyond an array.

The simulation results (produced in Debug mode) "seem" reasonable, and the BC "apparently" does what it is supposed to do. But I can't exactly verify how reliable the results are.

Moreover, I don't think it is only the matter with Debug mode keeping the variables in memory, as the code also works fine in Opt mode with some simple meshes (i.e. 2D tube).

As you say it runs fine on simple cases even on Opt mode, what do you mean by simple case? Does it mean completely uniform grid with the same number of faces on all boundaries?

Originally Posted by Arman_N

The part of my code for looping over the boundary in AlphaContactAngleFvPatchScalarField is like this:

From the limited information you have provided and without knowing what else you have changed in the code, I can only think of two suspects here:

1- How the theta0 is returned?
2- On more complex cases, boundaryT[i] and theta0[i] some how don't have the same size.

Can you attached a simple reproducible case?

[Arman_N]

As you say it runs fine on simple cases even on Opt mode, what do you mean by simple case? Does it mean completely uniform grid with the same number of faces on all boundaries?

Yes, exactly. In some limited cases with completely uniform regular grids it works fine, while in other - equally uniform - grids it crashes.

1- How the theta0 is returned?

theta0 is the contact angle and it is returned exactly the same as other BCs in the AlphaContactAngleFvPatchScalarField family (like timeVaryingAlphaContactAngleFvPatchScalarField, for example). And just to mention, the provided code is the only part which is changed inside the BC file.

2- On more complex cases, boundaryT[i] and theta0[i] some how don't have the same size.

Well, this is the part I'm confused about. How would this happen when they both get their sizes from the size of the same boundary patch?
And as I said, I have checked their sizes before. The code loops fine through the first boundary patch (i.e. upperWall), and their sizes are printed out to be equal. But then it won't even go to the next boundary (i.e. lower wall) and crashes before it has to check their sizes (the Info statement prior to it will not be shown). Furthermore, I don't think that this is something that Debug mode would be OK with to allow the program run without giving any errors.

Can you attached a simple reproducible case?

Well, this is a bit tricky (since there are several other parts of the problem which should be attached as well), but I would try to send a simple case.



Personally, I think this might be a bug in foam-extend in Opt mode which is resolved in Debug mode. Anyways, I'm not a pro in this.

Thanks again

[Gerry Kan]

Dear Arman:

When a code is compiled with the Debug option, optimized options are turned off. This is more likely why your code works in Debug mode.

You could try recompiling the code in Release mode with -O0 first, and increase the optimization level until the seg fault reappears.

I also don't know if OpenFOAM introduces more robust exception handling in Debug mode, which makes the compiled code more tolerant to, say, coding errors. A quick (but not conpletely certain) way of checking is to see if your results in those problematic cases still make sense.

For amything deeper you might need a debugger.

Gerry.

[Arman_N]

After months, I looked through this issue again and this time I was finally able to figure out what was wrong with my case, with the help of the hints given in these links: here and here

I found out that if I set nOuterCorrectors to 0 in PIMPE settings, the code works fine with no errors also in the Opt mode. Therefore, the issue must be in the re-discretization of the momentum equation.

The problem was here:

const volScalarField& T = this->db().objectRegistry::lookupObject<volScalarField>("T");

with the use of referencing for the definition of T. I copied this from a similar case without really considering its actual meaning. So that was my mistake.

When the outer correction loop takes place, either the indexing sequences or the whole fields change, while the reference to T is still pointing to something that is not there anymore. Therefore, when I defined T without referencing (by simply removing the &), the problem went away!

I think that in Debug mode, the variables are kept in memory during the outer correction loop and that is the reason why the reference to T still had a meaning and the code worked fine in Debug mode. However, this doesn't seem to happen in the Opt mode.

Thanks to Daniel and Gerry for their help and insights.

Cheers.