CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Programming & Development

ObjectRegistery in Parallel!

Register Blogs Community New Posts Updated Threads Search

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   December 16, 2013, 05:16
Default ObjectRegistery in Parallel!
  #1
Senior Member
 
Hisham's Avatar
 
Hisham Elsafti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 257
Blog Entries: 10
Rep Power: 17
Hisham is on a distinguished road
Dear Foamers,

I have a situation where I have a dictionary written by the (top level) solver and it is read from the mesh objectRegistry in a class. Works well in serial. Nevertheless, in parallel I want to assign the dictionary preparation to one process (e.g. master process, normally that happens by itself and assigned to a random process) and I want this dictionary to be accessible by all processes.

So where should I start reading, better still, how do I do it?

Thanks in advance
Hisham
Hisham is offline   Reply With Quote

Old   December 16, 2013, 16:46
Default
  #2
ngj
Senior Member
 
Niels Gjoel Jacobsen
Join Date: Mar 2009
Location: Copenhagen, Denmark
Posts: 1,903
Rep Power: 37
ngj will become famous soon enoughngj will become famous soon enough
Hi Hisham,

Does the standard approach not work for you? It has never failed me to do something like this:

Code:
IOdictionary dictName
(
    IOobject
    (
         "dictName",
         runTime.timeName(),
         mesh,
         IOobject::MUST_READ,
         IOobject::NO_WRITE
    )
);
Obviously, now you should be able to access this dictionary anywhere by the following command:

Code:
const dictionary& ref2Dict = mesh.thisDb().lookupObject<IOdictionary>("dictName");
Also in parallel.

Could you tell a bit more about your problems/errors?

Cheers,

Niels

P.S. Be aware that it should only enter into your data base once! Otherwise OF will have problems with cleaning up correctly, because of several objects with the same name.
__________________
Please note that I do not use the Friend-feature, so do not be offended, if I do not accept a request.
ngj is offline   Reply With Quote

Old   December 17, 2013, 04:08
Default
  #3
Senior Member
 
Hisham's Avatar
 
Hisham Elsafti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 257
Blog Entries: 10
Rep Power: 17
Hisham is on a distinguished road
Hi Niels,

Of course the standard approach (as you explained) works for values that are read from files. The problem is the dictionary in question is calculated and filled by the solver, hence, there is an instance on each process. The dictionary values depend on other IO objects (i.e. scalarField and vectorField ) that are read from the constant directory. So what happens is that only one instance of the nodes reads the fields (I don't know on what criteria but it's not the master), makes the calculation and fill the dictionary. So if another node needs the dict, it has its own "empty" dict. So I thought there may be a way to sync dicts across nodes in case of such randomness. It is (to be honest) most likely a miss-use from my part.

Waiting for an answer yesterday I changed the design and the dict is filled inside another class (a new relaxationScheme) because the original class is all const (a new waveTheory class) and cannot hold any data for further steps. I now have other problems . More on that soon!

Thanks a lot for your help.

Best regards,
Hisham
Hisham is offline   Reply With Quote

Old   December 17, 2013, 06:07
Default
  #4
ngj
Senior Member
 
Niels Gjoel Jacobsen
Join Date: Mar 2009
Location: Copenhagen, Denmark
Posts: 1,903
Rep Power: 37
ngj will become famous soon enoughngj will become famous soon enough
Good morning,

Do you need to do it each and every time step? Otherwise, you could do it as a pre-processing step in line with how I construct waveProperties based on waveProperties.org.

If you need to do it each time step, i.e. updating the dictionary, then I suggest that you do something along the following lines:

1. Inside a scope, construct the dictionary and write it to <case>/constant. Note not to write it to <case>/processor<number>/constant. As you leave the scope, the dictionary is deleted in the database.

2. On all processors, read the dictionary as any ordinary dictionary.

Note that the writing should be done inside the following if-statement:

Code:
if (Pstream::master())
{
    // Construct and write the dictionary
}
Actually, the above scope with for the master processor should be sufficient to get it deleted in the database. Also note that all the other processors will/should wait for Pstream::master() to finish his/her job with the construction of the dictionary.

Cheers,

Niels

P.S. A new wave theory sounds interesting Can you give me a hint on its nature?
__________________
Please note that I do not use the Friend-feature, so do not be offended, if I do not accept a request.
ngj is offline   Reply With Quote

Old   December 17, 2013, 06:22
Default
  #5
Senior Member
 
Hisham's Avatar
 
Hisham Elsafti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 257
Blog Entries: 10
Rep Power: 17
Hisham is on a distinguished road
Hi Niels

That was really nice to know. Thanks for the effort!

The idea is having a pre-defined velocity and surface elevation fields for the relaxation zone. So the values are read then interpolated in time and space (so points can stay independent of mesh). So a new wave theory class is needed to introduce the values instead of calculating from a real theory. Because waveTheory is const, the reading/interpolation of data has to be done somewhere else. Further, disk operations must be kept to a minimum or else ... Using a new relaxationScheme (inherits spatial) actually solved that and now the read point IO fields (and the dicts) are distributed on processors according to the node's sub-mesh.

The time of sims is acceptable for a short relaxation zone. Nevertheless, I am still debugging some segfault errors that occur from treatment of unfound keys in the dict.

Best regards,
Hisham
Hisham is offline   Reply With Quote

Old   December 17, 2013, 19:19
Default
  #6
ngj
Senior Member
 
Niels Gjoel Jacobsen
Join Date: Mar 2009
Location: Copenhagen, Denmark
Posts: 1,903
Rep Power: 37
ngj will become famous soon enoughngj will become famous soon enough
Hi Hisham,

Ok, it sound as if you make a variation over tbe spatialInterpolation relaxation scheme. Is that too expensive for you?

Secondly, if you run into this type of const problems, and you are sure of what you are doing, then you can always define your class variables as e.g.

Code:
mutable vectorField myVar_;
This allows for the modification of the variable even though it is modified in a const method. You will find the mutable keyword all over the place in e.g. the mesh classes.

Kind regards,

Niels
__________________
Please note that I do not use the Friend-feature, so do not be offended, if I do not accept a request.
ngj is offline   Reply With Quote

Old   December 18, 2013, 02:14
Default
  #7
Senior Member
 
Hisham's Avatar
 
Hisham Elsafti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 257
Blog Entries: 10
Rep Power: 17
Hisham is on a distinguished road
Hi Niels,

Quote:
Originally Posted by ngj View Post

Ok, it sound as if you make a variation over the spatialInterpolation relaxation scheme. Is that too expensive for you?
Now that was a soon addition for me to learn about ... sounds nice :-)
Nevertheless, the application I have is different if I understand correctly!

What I am doing is linking a far field model with a near field 3D model. So the results coming from the far-field model (2D) should be interpolated in time and space.

Quote:
Originally Posted by ngj View Post
Secondly, if you run into this type of const problems, and you are sure of what you are doing, then you can always define your class variables as e.g.

Code:
mutable vectorField myVar_;
This allows for the modification of the variable even though it is modified in a const method. You will find the mutable keyword all over the place in e.g. the mesh classes.
That's a good tip. It always pays to take that extra C++ mile.

Best regards,
Hisham
Hisham is offline   Reply With Quote

Old   December 18, 2013, 03:33
Default
  #8
ngj
Senior Member
 
Niels Gjoel Jacobsen
Join Date: Mar 2009
Location: Copenhagen, Denmark
Posts: 1,903
Rep Power: 37
ngj will become famous soon enoughngj will become famous soon enough
Good morning,

That sounds like a nice coupling. Basically, you should be able to couple your approach with the interpolation. It is merely a matter of correct behaviour of the waveTheory. Have you build your own far-field model?

Alternatively, you could have the two models running simultaneously, so you do not have to read/write from/to the disk. One of my colleagues from TU Denmark (BT Paulsen) did that. I have tried finding a link for his thesis, but I could not. Please send me your email address in a private message, and I will send you a copy.

Kind regards,

Niels
__________________
Please note that I do not use the Friend-feature, so do not be offended, if I do not accept a request.
ngj is offline   Reply With Quote

Old   December 18, 2013, 08:42
Default
  #9
Senior Member
 
Hisham's Avatar
 
Hisham Elsafti
Join Date: Apr 2011
Location: Braunschweig, Germany
Posts: 257
Blog Entries: 10
Rep Power: 17
Hisham is on a distinguished road
Hi Niels,

I had the privilege of meeting Bo last summer in Nantes. We discussed some ideas about this project but my ideas were not yet consolidated. I would appreciate a copy of the PhD of course!

The idea is to have constant wave conditions (on desk) that you can test for different structural configurations (3D mainly). This approach should also makes it trivial to adapt the input from any far field model (did I mention it is one-way) as time and position of input data is not an issue. So for every time step the relaxation scheme loads the relevant input data if needed (not every time step), interpolate for space if new data are loaded else interpolate only for time. Then, final data are filled in a registered dictionary that is read by the wave theory directly from the registry (cough cough pointers) without writing/reading the dictionary to disk. For this project, the far field is a 2D moving piston (v. long) wave flume to reproduce wave conditions generated in physical experiments. So, it is way cheaper to read input data from desk rather than wait on another (rather) expensive model and adapt time steps. What I meant by disk IO operations was the need to read the dict for each call to function in the waveTheory class because of the const decelerations.

It has compiled and is now in testing phase!

Best regards,
Hisham
Hisham is offline   Reply With Quote

Reply

Tags
dictionary, mpi, objectregistery, parallel


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
simpleFoam parallel AndrewMortimer OpenFOAM Running, Solving & CFD 12 August 7, 2015 19:45
Can not run OpenFOAM in parallel in clusters, help! ripperjack OpenFOAM Running, Solving & CFD 5 May 6, 2014 16:25
simpleFoam in parallel issue plucas OpenFOAM Running, Solving & CFD 3 July 17, 2013 12:30
parallel Grief: BoundaryFields ok in single CPU but NOT in Parallel JR22 OpenFOAM Running, Solving & CFD 2 April 19, 2013 17:49
Parallel Computing Classes at San Diego Supercomputer Center Jan. 20-22 Amitava Majumdar Main CFD Forum 0 January 5, 1999 13:00


All times are GMT -4. The time now is 09:50.