[andlog]

Hi all,

I've noted a problem with finite area solvers in parallel using foam-extend-4.0.

I copied both liquidFilmFoam and surfactantFoam tutorials, adding in the system folder the decomposeParDict from incompressible/simpleFoam/pitzDaily tutorial.

After blockMesh and makeFaMesh, I've decompose the case using decomposePar without a hitch.

Finally, I've tried to run the case in parallel typing "runParallel surfactantFoam 4 &", getting this error:

/*---------------------------------------------------------------------------*\

| =========                 |                                                 |

| \\      /  F ield         | foam-extend: Open Source CFD                    |

|  \\    /   O peration     | Version:     4.0                                |

|   \\  /    A nd           | Web:         http://www.foam-extend.org         |

|    \\/     M anipulation  | For copyright notice see file Copyright         |

\*---------------------------------------------------------------------------*/

Build    : 4.0-4fad65ce7cac

Exec     : surfactantFoam -parallel

Date     : May 02 2017

Time     : 13:03:33

Host     : ManDybHig_clust

PID      : 5971

CtrlDict : "/stoc/andrea/OpenFOAM_EXT/andrea-4.0/tutorial/FAM/planTransport/system/controlDict"

Case     : /stoc/andrea/OpenFOAM_EXT/andrea-4.0/tutorial/FAM/planTransport

nProcs   : 4

Slaves : 

3

(

ManDybHig_clust.5972

ManDybHig_clust.5973

ManDybHig_clust.5974

)



Pstream initialized with:

    nProcsSimpleSum   : 0

    commsType         : nonBlocking

SigFpe   : Enabling floating point exception trapping (FOAM_SIGFPE).



// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

Create time



Create mesh for time = 0



[0] 

[0] 

[0] --> FOAM FATAL ERROR: 

[0] otherProcNo:2 : illegal communicator -1650115904

Communicator should be within range 0..0

[0] 

[0]     From function PstreamGlobals::checkCommunicator(const label, const label)

[0]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.

[0] 

FOAM parallel run aborting

[0] 

[1] 

[1] 

[1] --> FOAM FATAL ERROR: 

[1] otherProcNo:0 : illegal communicator -1104586048

Communicator should be within range 0..0

[1] 

[1]     From function PstreamGlobals::checkCommunicator(const label, const label)

[1]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.

[1] 

FOAM parallel run aborting

[1] 

[2] 

[2] 

[2] --> FOAM FATAL ERROR: 

[2] otherProcNo:0 : illegal communicator -886773056

Communicator should be within range 0..0

[2] 

[2]     From function PstreamGlobals::checkCommunicator(const label, const label)

[2]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.

[2] 

FOAM parallel run aborting

[2] 

[3] 

[3] 

[3] --> FOAM FATAL ERROR: 

[3] otherProcNo:2 : illegal communicator 1469405888

Communicator should be within range 0..0

[3] 

[3]     From function PstreamGlobals::checkCommunicator(const label, const label)

[3]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.

[3] 

FOAM parallel run aborting

[3] 

--------------------------------------------------------------------------

MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD 

with errorcode 1.



NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.

You may or may not see output from other processes, depending on

exactly when Open MPI kills them.

--------------------------------------------------------------------------

I get the same error with all the finite-area-method tutorials. I haven't found any suggestion for this.


The above procedure successfully run using foam-extend-3.2 instead.
Therefore, it should be a bug related to the last version (foam-extend-4.0).

Any idea what this is and how to fix it?

Best,

Andrea

[hjasak]

Let me have a look and get back to you - this looks like some recent development.

Apologies,

Hrv

[hjasak]

Hi Andrea,

My bad: simple bug in comms initialisation. I have fixed it now; if you need it urgetnylu, please drop me a line.

The bug fix will be pushed asap.

Apologies,

Hrvoje

[andlog]

Hi Hrvoje,

Thank you so much for your prompt reply.
I' m currently in a development stage, so I can wait for the fix.

Thank you again,

Andrea

[mseka]

Hi Andrea,

I am wondering whether you are working on the finite area or not? I am interested in applying finite area on a problem but there is not enough documentation on how to modify the solver for other applications in a proper way, even the original Ph.D. thesis Z. Tukovi´c, Finite volume method on domains of varying shape, is not available online.

I am wondering whether you have any documents or would like to discuss a bit on the problem?

[hjasak]

Hi,

Sorry, I didn't realise the PhD Thesis by Tukovic is not online. We will fix this asap- it should be on

http://www.fsb.hr/cfd

under Dissemination.

There is also a paper in Coumputers and Fluids:

A moving mesh finite volume interface tracking method for surface tension dominated interfacial fluid flow
Ž Tuković, H Jasak - Computers & fluids, 2012

https://scholar.google.co.uk/citatio...J:hqOjcs7Dif8C

[hjasak]

My bad: here's the link to prof. Zeljko Tukovic PhD Thesis

http://foam-extend.fsb.hr/wp-content...c_PhD_2005.pdf

[ziad]

Cloned fe-4.0 from the git repository (git://git.code.sf.net/p/foam-extend/foam-extend-4.0) and this error is still happening with the finite area tutorials and cases.

/*---------------------------------------------------------------------------*\
| =========                 |                                                 |
| \\      /  F ield         | foam-extend: Open Source CFD                    |
|  \\    /   O peration     | Version:     4.0                                |
|   \\  /    A nd           | Web:         http://www.foam-extend.org         |
|    \\/     M anipulation  | For copyright notice see file Copyright         |
\*---------------------------------------------------------------------------*/
Build    : 4.0-f500917045c0
Exec     : liquidFilmFoam -parallel
Date     : Dec 09 2017
Time     : 08:51:04
Host     : lazarus
PID      : 13964
CtrlDict : "/home/ziad/foam/ziad-4.0/run/tutorials/finiteArea/liquidFilmFoam/cylinder/system/controlDict"
Case     : /home/ziad/foam/ziad-4.0/run/tutorials/finiteArea/liquidFilmFoam/cylinder
nProcs   : 2
Slaves : 
1
(
lazarus.13965
)

Pstream initialized with:
    nProcsSimpleSum   : 0
    commsType         : nonBlocking
SigFpe   : Enabling floating point exception trapping (FOAM_SIGFPE).

// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //
Create time

Create mesh for time = 0

[0] 
[0] 
[0] --> FOAM FATAL ERROR: 
[0] otherProcNo:1 : illegal communicator 1
Communicator should be within range 0..0
[0] 
[0]     From function PstreamGlobals::checkCommunicator(const label, const label)
[0]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.
[0] 
FOAM parallel run aborting
[0] 
[1] 
[1] 
[1] --> FOAM FATAL ERROR: 
[1] otherProcNo:0 : illegal communicator 1
Communicator should be within range 0..0
[1] 
[1]     From function PstreamGlobals::checkCommunicator(const label, const label)
[1]     in file db/IOstreams/Pstreams/PstreamGlobals.C at line 72.
[1] 
FOAM parallel run aborting
[1] 
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 1.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
[lazarus:13959] 1 more process has sent help message help-mpi-api.txt / mpi-abort
[lazarus:13959] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages

[ziad]

Turns out the repository is not up to date yet but the next release will have the fix. In the meantime you can download the updated attached file in src/finiteArea/faMesh and recompile the library from src/finiteArea (wmake libso).

Many thanks to Prof Jasak and the foam-extend development team at the University of Zagreb for making the fix available