[Hisham]

Dear Foamers

I use the #codeStream feature to set up an initial field. This works fine for running on a single core. However, after decomposing the case it does not work in parallel and gives an error that has nothing to do with the field in question. It actually took me all day to figure out that removing the #codeStream fixed the case.

So can anyone kindly confirm whether it is possible or not to use #codeStream to set initial fields for parallel runs?

Best regards,
Hisham El Safti

[gschaider]

Originally Posted by Hisham

Dear Foamers

I use the #codeStream feature to set up an initial field. This works fine for running on a single core. However, after decomposing the case it does not work in parallel and gives an error that has nothing to do with the field in question. It actually took me all day to figure out that removing the #codeStream fixed the case.

So can anyone kindly confirm whether it is possible or not to use #codeStream to set initial fields for parallel runs?

Best regards,
Hisham El Safti

Are you sure that your initialization routine is correctly parallelized? For instance if you have local variables then chances are good that you might need some strategically place reduce-operations.

[Hisham]

Hello Bernhard,

Actually I think your remark definitely points what's wrong. I'll try to give more info. I'm developing a solver in which I have an object (derived from a new class) assigned to each cellZone. The objects are contained (in the solver) in a pointer list (the pointers are for a mother class). Each cellZone might have an object from a different child class.

The need for this is that each child class has its own implemented calculations (model). To save the numbers from the time step calculations (to allow for run continuation) an IOField<scalarField>, an IOField<vectorField> and an IOField<symmTensorField> are used to store numerical values (in solver) from the objects and are reread if present. The advantage of this is that not all objects need to save the same number of variables.

So I think maybe there is a lot of traffic between processors ... I really have little ideas about how to reduce this. I would appreciate your advice. I can also email you the solver and a case if you're interested!

Thanks in advance
and best regards
Hisham El Safti

[niklas]

what's the error?

in etc/controlDict, is this set?
fileModificationChecking timeStamp;

[Hisham]

Hello Niklas

In controlDict:
fileModificationChecking timeStampMaster;

The case runs in serial OK. The error in the parallel run happens in:

volSymmTensorField strain
(
    IOobject
    (
        "strain",
        runTime.timeName(),
        mesh,
	IOobject::NO_READ,
        IOobject::NO_WRITE
     ),
     0.5*twoSymm(fvc::grad(D))
);

The error is (using 8 cores):

[6] #0  Foam::error::printStack(Foam::Ostream&)[4] #0  [5] #0  Foam::error::printStack(Foam::Ostream&)Foam::error::printStack(Foam::Ostream&)[2] #0  Foam::error::printStack(Foam::Ostream&)[3] #0  Foam::error::printStack(Foam::Ostream&) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #1  Foam::sigFpe::sigHandler(int) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #1  Foam::sigFpe::sigHandler(int) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #1  Foam::sigFpe::sigHandler(int) in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[2] #2   in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[4] #2   in "/opt/openfoam in "/lib/x86_64-linux-gnu/libc.so.6"
[4] #3   in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #1  Foam::sigFpe::sigHandler(int)201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #1   in "/lib/x86_64-linux-gnu/libc.so.6"
[2] #3Foam::sigFpe::sigHandler(int)  Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > Foam::operator*<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > const&)Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > Foam::operator*<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > const&) in  in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[5] #2   in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[4] #4  "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[3] #2   in "/opt/openfoam201/platforms/linux64GccDPOpt/lib/libOpenFOAM.so"
[6] #2   in "/lib/x86_64-linux-gnu/libc.so.6"
[3] #3  
 in "/lib/x86_64-linux-gnu/libc.so.6"
[6] #3  [4]  in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[4] #5  __libc_start_mainFoam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > Foam::operator*<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > const&)Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > Foam::operator*<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > const&) in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[3] #4   in "/lib/x86_64-linux-gnu/libc.so.6"
[4] #6   in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[2] #4   in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[6] #4  


 in "/lib/x86_64-linux-gnu/libc.so.6"
[5] #3  [3]  in "/home/hisham/OpenFOAM[2]  in "/home/hisham/OpenFOAM/hisham-2.0.1/platforms/linux64GccDPOpt/bin/geotechFoam"
[2] #5  __libc_start_mainFoam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > Foam::operator*<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh>(Foam::GeometricField<double, Foam::fvPatchField, Foam::volMesh> const&, Foam::tmp<Foam::GeometricField<Foam::SphericalTensor<double>, Foam::fvPatchField, Foam::volMesh> > const&)[4]  in "/home/hisham/Op/hisham-2.0.1/platforms/linux64GcenFOAM/hisham-2.0.1/platforms/linux64GccDcDPOpt/bin/geotechFoam"
[3] #5  __libc_start_mainPOpt/bin/geotechFoam"
[numubuntu-System-Product-Name:27284] *** Process received signal ***
[numubuntu-System-Product-Name:27284] Signal: Floating point exception (8)
[numubuntu-System-Product-Name:27284] Signal code:  (-6)
[numubuntu-System-Product-Name:27284] Failing at address: 0x3e900006a94
[numubuntu-System-Product-Name:27284] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x33d80) [0x7f943d10bd80]
[numubuntu-System-Product-Name:27284] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7f943d10bd05]
[numubuntu-System-Product-Name:27284] [ 2] /lib/x86_64-linux-gnu/libc.so.6(+0x33d80) [0x7f943d10bd80]
[numubuntu-System-Product-Name:27284] [ 3] geotechFoam(_ZN4FoammlINS_15SphericalTensorIdEENS_12fvPatchFieldENS_7volMeshEEENS_3tmpINS_14GeometricFieldIT_T0_T1_EEEERKNS6_IdS8_S9_EERKSB_+0x321) [0x4b4db1]
[numubuntu-System-Product-Name:27284] [ 4] geotechFoam() [0x44c6e9]
[numubuntu-System-Product-Name:27284] [ 5] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xff) [0x7f943d0f6eff]
[numubuntu-System-Product-Name:27284] [ 6] geotechFoam() [0x429bb9]
[numubuntu-System-Product-Name:27284] *** End of error message ***
--------------------------------------------------------------------------
mpirun noticed that process rank 4 with PID 27284 on node numubuntu-System-Product-Name exited on signal 8 (Floating point exception).
--------------------------------------------------------------------------

Hisham

[niklas]

what happens when you switch to using timeStamp instead of timeStampMaster?

[niklas]

does this work? (it should)
volSymmTensorField strain = 0.5*twoSymm(fvc::grad(D));

the only problem I can think of is that you're trying to register all processor-meshes on the same
timeName()/mesh database.

[Hisham]

I will check that now. But it is worth mentioning that there is a timeVaryingMapped BC that is distributed to several processors. I found here that this could be a problem (actually the case has ran without the codeFixed (using the timeVaryMap BC) but I had to modify the mesh and afterwards it didn't work ... Further modifications of the mesh does not help!