[evrikon]

Hello forum,

I have computed a some large case (16 M cells) and I am going to postprocess it. Unfortunately I have some troubles with utilities foamToVTK and sample. It seems to me that I have not enough memory on the node where I start these utilities. Here is the error message:

------------------
Time = 1000
new cannot satisfy memory request.
This does not necessarily mean you have run out of virtual memory.
It could be due to a stack violation caused by e.g. bad use of pointers or an out of date shared
-----------------

The problem is that I have only 16 Gb RAM per node and it seems to be not enough (16M cells in my case). I tryed to find a workaround:

1) In every case I restricted the amount of necessary data fields to as small as possible:

foamToVTK -latestTime -fields "(p)"

In this way I can get the pressure field. However, for the velocity field I cannot because I need more RAM.

2) Ok, I thought if foamToVTK needs more RAM I try sample utility. But I had no success. It crashes with the same error message.

3) I tryed to use the option -parallel with foamToVTK. But it seems to me that it only works in the distributed fields, i.e. it creates the VTK directories in processor0, processor1 etc.

4) For sample -parallel I get the strange error:
"given patch field does not correspond to the mesh. Field size: 20753 mesh size: 20754"


5) What really works is foamToVTK -noInternall. In that way I produced the pressure pictures on the patches. I still need to produce a sample line at least and sample surface (nice to have).

Does anybody had any more ideas how to proceed? The problem apperars always if large cases are proceeded. One needs a node with large RAM for postprocessing. Or does anybody know how to overcome this restriction?

Oleksiy

[gschaider]

If this is sufficient for you, you could use mapFields to map your results to a coarser grid of the same geometry and then postprocess that

[deepsterblue]

Takuya's FOAM reader for Paraview 3.x allows post-processing for decomposed cases. You shouldn't have to reconstruct the case, but rather, post-process in parallel. This is, after all, what paraview was designed for, isn' it?

[evrikon]

Thank you on both suggestions. Especially a parallel native reader for OpenFoam seems to be very interesting.

[lakeat]

Originally Posted by gschaider

If this is sufficient for you, you could use mapFields to map your results to a coarser grid of the same geometry and then postprocess that

I don't think so, my experience is if you can't go with foamToVTK (be it a scalar or vector), then you cannot make mapFields work.

[sandy]

Hi Daniel,

Did you solve this problem? I met the same error informations when I run " parafoam " with my 4.67 million grids. Please help out.

Sandy

[lakeat]

How many ram do you actual have? You can't do it with just a 12G ram

[sandy]

DELL T3500 -- 24G ram, one cpu and six cores . What about it, you think?

[lakeat]

Originally Posted by sandy

DELL T3500 -- 24G ram, one cpu and six cores . What about it, you think?

I think this is far from enough, even for a scalar field. Sorry. You need to find a larger ram.

[arjun]

Originally Posted by lakeat

I think this is far from enough, even for a scalar field. Sorry. You need to find a larger ram.

you can post process 4.7 million of data with fieldview in 2Gb ram (32bit machine).

24G is too much (may be not enough for paraview though).

[lakeat]

Sorry, my bad.

I thought she has 46.7 million mesh.
Ok, 4.67m is not a big mesh, your RAM is good to handle it, if you meet any difficulty with uprime2mean, just pick it out, or you can first use foamtovtk to transform them to vtk format, then there would be no problem.

Arjun, fieldview is not free!

[arjun]

Originally Posted by lakeat

Sorry, my bad.

I thought she has 46.7 million mesh.
Ok, 4.67m is not a big mesh, your RAM is good to handle it, if you meet any difficulty with uprime2mean, just pick it out, or you can first use foamtovtk to transform them to vtk format, then there would be no problem.

Arjun, fieldview is not free!

yaa that is true.

Last few days I have been working on creating a cfd solver that could handle large meshes. A solver that could handle 1 billion or so cells. (3.6billion is my largest so far). And have done lots of experimentation.

Post processing large data is really a big problem.

My solution (which i will be working on implimenting in next few months) is that let the cfd solver handle post processing. If solver can run the calculation it can do post processing too.

Here is bit of explanation to above comment. We as a user mostly concerned about say contour plots on mid plane or some planes. Animations or them. Vector plots on planes or some well defined surfaces.
Iso surface values etc etc.

Now the point to note that even though data is very very large, all these plots require data that could be written as a very small file. For example if you want iso surface of say vorticity. Solver can calculate that and export it is set of triangles in a file that paraview can read and load. (say vtk format). You have to just load that file and export the image that you want.

[lakeat]

3.6 billion, that's huge! Are you from fieldview?
I remember I have to borrow a 128G ram to post process my results, and even then i have to use vtk outputs.

And sorry, I failed to follow you, files like uprime2mean is a very huge file in it self, after it is constructed, it will larger than 15G compressed already for your 3.6billion mesh I guess, and it's just a tensor field with six components, and you also have to build its connection with points coordinates info, in all, how could you make it smaller.

I see no better way, to process this tensor field except you decompose the domain into some small parts.

[lakeat]

Nowadays, US nsf has some computing resources available to apply and also they are aware the need to visualize the large data files. So they have some machines with very big rams etc., and they can render the contours on a web browser, and I think this is also a nice idea. I don;t know what about in euro and japan, do you have the same resources or not?

[sandy]

However, how can I reconstruct my datum distributed in every propessor by parallel? Can I directly foamtovtk ?

If I run "reconstruct", my ram is not enough too ....

[lakeat]

pls read:

reconstruct -help

[arjun]

Originally Posted by lakeat

3.6 billion, that's huge! Are you from fieldview?
I remember I have to borrow a 128G ram to post process my results, and even then i have to use vtk outputs.

And sorry, I failed to follow you, files like uprime2mean is a very huge file in it self, after it is constructed, it will larger than 15G compressed already for your 3.6billion mesh I guess, and it's just a tensor field with six components, and you also have to build its connection with points coordinates info, in all, how could you make it smaller.

I see no better way, to process this tensor field except you decompose the domain into some small parts.

Data will always be partitioned.
The idea is that if you can run the calculations, you can use the same number of processors to do the post processing. So each processor is responsible for the data it produces.

What i want to do is that make each processor do the post processing, at the moment paraview is one possibility (fieldview is out because can not afford 500 licenses of it).

The problem i am facing with paraview is that while cfd solver can run the job, paraview is asking for much more memory and hence can not post process data even with same number of processors.


You are correct though, when case is large, things are very different, data sizes are huge. All these are challenges we need to overcome.

[sandy]

Originally Posted by lakeat

pls read:

reconstruct -help

Hi lakeat, I get the error information again. Maybe I should try:

Danie -help

?

[wyldckat]

Greetings to all!

I'm just stepping in to point out the typo... it's reconstructPar, not just reconstruct.

In the terminal (at least if you are using bash) you can confirm the existence of an application by pressing the Tab key to auto-complete the name. For example:

reconstr<Tab key>
reconstructPar

In case it doesn't show the rest, try pressing the Tab key twice to see if it shows a list of visible applications that start with the same characters.

Best regards,
Bruno

[arjun]

just an update to my comment.


I implemented iso surface generation part. It is working well. So now for generating animation of Q criteria for 3billion cells case, I only need 1 processor paraview. Post processing files are very small and thus, i can save them with high frequency and it also means quality of animation would be much better than what I would get from saved calculation data.


Sweet success. :-D