|
[Sponsors] |
Sharing large scalarList among multiple processors |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
August 25, 2017, 06:06 |
Sharing large scalarList among multiple processors
|
#1 |
Member
hannes
Join Date: Mar 2013
Posts: 47
Rep Power: 13 |
Hi All,
I'm currently looking for a solution to efficiently share data in a parallel simulation. The problem is as follows: In a custom (temperature) boundary condition I want to interpolate the temperature based on several parameters. The data for the interpolation (list of scalars) is read from a file during initialisation of the bc. This list is rather large (36Mio values). My current implementation works fine, but it reads and stores this list for every thread which makes it very slow in the beginning and causes trouble with memory when running on 20 or more processors. Instead, we thought of using a pointer to the list and reading the list only on the master processor but somehow I can't make the code run properly. And here comes the problem. The relevant constructor of the BC currently looks like this: Code:
Foam::myTemperatureFvPatchScalarField:: myTemperatureFvPatchScalarField ( const fvPatch& p, const DimensionedField<scalar, volMesh>& iF, const dictionary& dict ) : fixedValueFvPatchScalarField(p, iF), phiName_(dict.lookupOrDefault<word>("phi", "phi")), airTempIntTblName_(dict.lookup("airTempIntTblName")), airTempIntTbl_ ( airTempIntTblName_, this->patch().boundaryMesh().mesh(), IOdictionary ( IOobject ( airTempIntTblName_, this->patch().boundaryMesh().mesh().time().constant(), this->patch().boundaryMesh().mesh(), IOobject::MUST_READ, // must exist, otherwise failure IOobject::NO_WRITE, // dict is only read by the solver false // register ) ), true //initialise only ), dataPointer_(), ... { Pout << " in constructor " << endl; ... if (Pstream::master()) { Pout << "in master loop ... " << endl; uniformMatrixInterpolationTable<scalar> foo ( uniformMatrixInterpolationTable<scalar> ( IOobject ( airTempIntTblName_, this->patch().boundaryMesh().mesh().time().constant(), this->patch().boundaryMesh().mesh(), IOobject::MUST_READ, // must exist, otherwise failure IOobject::NO_WRITE, // dict is only read by the solver true ) ) ); scalarList tblData=scalarList(foo.size(),Zero); scalarList *dataPtr; forAll(foo,i) { tblData[i]=foo[i]; } dataPtr = &tblData; dataPointer_.set(dataPtr); Pout << "dataPtr " << *dataPtr << endl; } // Pstream::scatter(dataPointer_,Pstream::blocking); } Code:
autoPtr<scalarList> dataPointer_; Code:
End *** Error in `buoyantSimpleFoam': double free or corruption (out): 0x00007ffd10596a30 *** Aborted (core dumped) Code:
[1] in constructor [0] in constructor [0] in master loop ... [0] dataPtr 32{800} [1] [1] [1] --> FOAM FATAL IO ERROR: [1] incorrect first token, expected <int> or '(', found on line 0 the word 'dictionary' [1] [1] file: IOstream at line 0. [1] [1] From function Foam::Istream& Foam::operator>>(Foam::Istream&, Foam::List<T>&) [with T = int] [1] in file /home/ubuntu/OpenFOAM/OpenFOAM-4.1/src/OpenFOAM/lnInclude/ListIO.C at line 148. [1] FOAM parallel run exiting [1] Code:
g++ -std=c++0x -m64 -Dlinux64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof -O3 -DNoRepository -ftemplate-depth-100 -I/opt/openfoam4/src/finiteVolume/lnInclude -I/opt/openfoam4/src/TurbulenceModels/incompressible/lnInclude -I/opt/openfoam4/src/TurbulenceModels/compressible/lnInclude -I/opt/openfoam4/src/turbulenceModels/compressible/lnInclude -I/opt/openfoam4/src/TurbulenceModels/turbulenceModels/lnInclude -I/opt/openfoam4/src/transportModels/compressible/lnInclude -I/opt/openfoam4/src/thermophysicalModels/basic/lnInclude -IlnInclude -I. -I/opt/openfoam4/src/OpenFOAM/lnInclude -I/opt/openfoam4/src/OSspecific/POSIX/lnInclude -fPIC -c myTemperatureFvPatchScalarField.C -o Make/linux64GccDPInt32Opt/myTemperatureFvPatchScalarField.o In file included from /opt/openfoam4/src/OpenFOAM/lnInclude/Pstream.H:352:0, from /opt/openfoam4/src/OpenFOAM/lnInclude/mapDistributeBase.H:75, from /opt/openfoam4/src/OpenFOAM/lnInclude/Field.C:30, from /opt/openfoam4/src/OpenFOAM/lnInclude/Field.H:405, from /opt/openfoam4/src/OpenFOAM/lnInclude/labelField.H:39, from /opt/openfoam4/src/OpenFOAM/lnInclude/primitiveFields.H:37, from /opt/openfoam4/src/OpenFOAM/lnInclude/pointField.H:36, from /opt/openfoam4/src/OpenFOAM/lnInclude/edge.H:40, from /opt/openfoam4/src/OpenFOAM/lnInclude/edgeList.H:32, from /opt/openfoam4/src/OpenFOAM/lnInclude/PrimitivePatch.H:56, from /opt/openfoam4/src/OpenFOAM/lnInclude/primitivePatch.H:35, from /opt/openfoam4/src/OpenFOAM/lnInclude/polyPatch.H:43, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatch.H:39, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatchField.H:47, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatchFields.H:29, from myTemperatureFvPatchScalarField.H:55, from myTemperatureFvPatchScalarField.C:26: /opt/openfoam4/src/OpenFOAM/lnInclude/gatherScatter.C: In instantiation of ‘static void Foam::Pstream::scatter(const Foam::List<Foam::UPstream::commsStruct>&, T&, int, Foam::label) [with T = Foam::autoPtr<Foam::List<double> >; Foam::label = int]’: /opt/openfoam4/src/OpenFOAM/lnInclude/gatherScatter.C:227:70: required from ‘static void Foam::Pstream::scatter(T&, int, Foam::label) [with T = Foam::autoPtr<Foam::List<double> >; Foam::label = int]’ myTemperatureFvPatchScalarField.C:303:52: required from here /opt/openfoam4/src/OpenFOAM/lnInclude/gatherScatter.C:184:27: error: no match for ‘operator>>’ (operand types are ‘Foam::IPstream’ and ‘Foam::autoPtr<Foam::List<double> >’) fromAbove >> Value; ^ /opt/openfoam4/src/OpenFOAM/lnInclude/gatherScatter.C:184:27: note: candidates are: In file included from /opt/openfoam4/src/OpenFOAM/lnInclude/string.H:48:0, from /opt/openfoam4/src/OpenFOAM/lnInclude/word.H:42, from /opt/openfoam4/src/OpenFOAM/lnInclude/wordList.H:41, from /opt/openfoam4/src/OpenFOAM/lnInclude/patchIdentifier.H:38, from /opt/openfoam4/src/OpenFOAM/lnInclude/polyPatch.H:42, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatch.H:39, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatchField.H:47, from /opt/openfoam4/src/finiteVolume/lnInclude/fvPatchFields.H:29, from myTemperatureFvPatchScalarField.H:55, from myTemperatureFvPatchScalarField.C:26: /opt/openfoam4/src/OpenFOAM/lnInclude/char.H:49:10: note: Foam::Istream& Foam::operator>>(Foam::Istream&, char&) Istream& operator>>(Istream&, char&); ... 1. Is the approach using pointers in this case reasonable or is there a better alternative? 2. Is there anything wrong in the way the pointer is used. (see the error message for a single thread run above) 3. Why doesn't it run in parallel? I really appreciate any comments and thoughts regarding the above and thanks a lot! Hannes |
|
August 28, 2017, 04:51 |
|
#2 |
New Member
Join Date: Dec 2012
Posts: 13
Rep Power: 14 |
Hi,
I'm struggeling with a similar question and would appreciate any hints on scattering a smart pointer over processors while only the master initializes it. |
|
Tags |
autoptr, parallel, pstreams |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Calculate field and patch averages in multiple zones, large mesh | buffi | OpenFOAM Post-Processing | 2 | July 21, 2022 06:45 |
Socket 2011-3 processors - an overwiew | flotus1 | Hardware | 10 | June 17, 2017 10:47 |
timeVaryingMappedFixedValue: large data mapping on complex inlet boundaries | vkrastev | OpenFOAM Pre-Processing | 7 | June 2, 2016 16:50 |
Large Epsilon 3D | lepsilon | Main CFD Forum | 0 | July 10, 2012 11:43 |
64-bit processors for home computing | Ananda Himansu | Main CFD Forum | 2 | March 16, 2004 13:48 |