[pkr]

I am working on nonnewtonianIcofoam. I refined the mesh using refineMesh utility. The current mesh structure is as follows:

Mesh stats
points: 14884
internal points: 0
faces: 29042
internal faces: 14182
cells: 7200
boundary patches: 6
point zones: 0
face zones: 0
cell zones: 0

Overall number of cells of each type:
hexahedra: 7180
prisms: 0
wedges: 0
pyramids: 0
tet wedges: 0
tetrahedra: 0
polyhedra: 20


I would like to increase the number of cells and faces in this mesh to a large value (say 200K). Please suggest the way to do it. Can it be done using blockMesh? It will be great if I can get some kind of help. Thanks!!

[akidess]

Yes, it can be done. Edit the file constant/polymesh/blockMeshDict. See the user guide on how to do it.

[pkr]

Hey Anton,

Thanks for your response. Good to see reply from TUD, I was associated with TUD for last 2 years.

1. As I understand, I can edit the blockMeshDict. But it will require adding the vertices, edges etc. by hands. I am looking for an automated way to create a very refined mesh which increases the size of ldu matrix, and hence leads to a computational intensive case. Please suggest.

2. I can refine the mesh repeatedly using refineMesh. But after some iterations of refineMesh, the checkMesh utility starts reporting problem with the generated mesh. Please suggest a way to create a large mesh without using refineMesh utility.

Thanks in Advance !!

[MartinB]

Hi pkr,

here are two python scripts and an OpenFOAM case that might be useful: it's just a simple tube, but you can easily increase the number of elements. First, you can increase the mesh_density parameter, second you can increase the length of the tube.

Have a look at line 172 of "pyCylinder.py": you can increase length and mesh density here. You must define an appropriate path to your file system here, too.

Start the script with:

python pyCylinder.py

A blockMeshDict is created in the "constant/polyMesh" folder. Mesh it with blockMesh and start simpleFoam...

By the way I recommend to use simpleFoam instead of nonNewtonianIcoFoam. Getting a steady state solution is simple, fast and reliable. You should get all information you need for your computation time analysis. Despite of that it would be easy to switch to nonNewtonianIcoFoam, too.

The usage of the script can be automized further, p.e. by making a loop changing the case path name and mesh size. The other files for completion of the case can be copied to the new directories (i.e. system, constant and 0 folders). OpenFOAM can be started automatically in each loop, and the times for PCG solver can be found in the simulation log file....

Do you have different processors to run your cases on? It would be nice to have some benchmarks, if you are analyzing computation times anyway...

Hope the scripts are useful,

Martin

[pkr]

Hi Martin,

The scripts provided by you are really very useful. I am analyzing OpenFoam for different multi-core architectures. I don't have any benchmarks at this moment, but I will let you know once I get some.

Following queries:
1. I would like to know if the similar script can be used for other applications in the tutorial? As I see the folder 0 differs from application to application. Please suggest the way to incorporate it to other applications like multiphase, heat transfer etc.

2. In the script, changing of both density as well as length leads to increase in number of cells. How is changing density different from the changing length?

Thanks !!

[MartinB]

Hi,

ad 1.) Since the script only provides a blockMeshDict file it is independent from the application. The mesh can be used by any OpenFOAM solver.

A very typical way to set up an OpenFOAM case is:
- Select the solver you want to use.
- Copy the case structure for the selected solver from the tutorials directory. The case structure mostly consists of "0", "system" and "constant".
- Insert the blockMeshDict from the script into "constant/polyMesh" and generate the mesh with the blockMesh command.
- Next you must adjust the files in the "0" directory, so that each file has correct boundary conditions defined for the patches from your blockMeshDict. In my case these are "inlet", "outlet" and "wall".
- Check the settings for material laws, turbulences etc. in the "constant" directory.
- Adjust the controlDict file in the "system" folder to your needs. Tune the "fvSchemes" and "fvSolution" in "system" folder if necessary.

In your particular case you are going to spend much time in "fvSolution", since this is the place to deal with the linear solver settings.

After you have defined the case for your selected solver once you can copy your case to your next project. Only minor adjustments will be necessary to do, once you got your standard cases. You can easily write a script to copy the complete case structure for each new test run you want to perform. If you only want to change the mesh size, only one file will differ for each run.

ad 2.)
The mesh density parameter applies to one quarter of the tube as you can see in the attached image. A value of 10 will provide 10 cells in x and y direction of the block. To have a different number of cells in one of the direction change line 87.

For the number of cells in z direction let's have a look at lines 86 to 91:

        d = self._mesh_density
        d2 = d
        z_level_1 = int(self._l / (((2*math.pi*self._r)/(d*4.0))))+1
        
        sum_cells = 5 * d * d2 * z_level_1
        print "Number of cells: %i" %(sum_cells)

The z_level_1 formula provides the number of cells in z direction, so that the elements on the wall have a nearly squared form.

Instead of this automatism you can add another parameter to the script that takes the number of elements in z direction. Same is true for the d2 parameter mentioned above.


Here are some thoughts of mine, feel free to ignore them ;-) :

Since your objective is to examine the behavior on multi core architecture I would concentrate on only one or two solvers. Most solvers behave very similar concerning the underlying hardware, so solving the same pressure equation in dozens of different solvers don't provide useful information.
Further more I would drop all turbulence related stuff. It can take too many time only to get the simulation converging.

I would not use time discretized solvers either. The time steps have great influence on stability, the mesh density limits the time steps, the time for solving the equation systems depends on the chosen time steps.... too complicated...

So my favorite would be simpleFoam with laminar and steady state flow, using a high viscosity Newtonian fluid. May be you can add a scalar transport to the solver to get a rather simple additional equation system.

To benchmark the pure effect of hardware architecture I would only change length of the tube. The simulation itself will stay in a physically meaningful range.

You should consider of using the OpenFOAM-1.6-ext version. It has some very interesting AMG solver variants. There are papers dealing with the performance mentioned here:
http://www.cfd-online.com/Forums/ope...41dev-svn.html


Looking forward to your benchmarks

Martin

[pkr]

Hi Martin,

Have you worked with parallel decomposed cases?

I am trying an experiment on interFoam/laminar/damBreak case. The number of cells in the generated mesh is 955000. To run in parallel, the mesh decomposition is done using metis decomposition.

When decomposed and running for 4 cores (quad-core Xeon E5620 ), it works perfectly fine.

On changing the decomposed case to 2-cores, After some time of running, the systems hangs and displays the following error: "mpirun noticed that process rank 0 with PID 3758 on node exited on signal 11 (Segmentation fault)."

Both of the cases (2-cores and 4-cores) work fine for lower number of cells (< 600000 cells).

Please suggest. Thanks!!

[lindstroem]

Hi pkr,

just for your interest, I'm using the decomposition method metis for parallelization as well and quite often on 2 cores with more than 1 000 000 cells without any problem. Maybe you want to upload the case, so I could give it a try?

Greetings

[pkr]

Originally Posted by lindstroem

Hi pkr,

just for your interest, I'm using the decomposition method metis for parallelization as well and quite often on 2 cores with more than 1 000 000 cells without any problem. Maybe you want to upload the case, so I could give it a try?

Greetings

Hi lindstroem,

Thanks for your response. I have attached the case, can you please try and see.

Thanks

[pkr]

Hi lindstroem, Can you also upload your case for me to try?

Thanks

[lindstroem]

Hi pkr,

Create time

Time = 0
Create mesh

Calculating distribution of cells
Selecting decompositionMethod metis

Finished decomposition in 1.89 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
    Number of cells = 477656
    Number of faces shared with processor 1 = 606
    Number of processor patches = 1
    Number of processor faces = 606
    Number of boundary faces = 38720

Processor 1
    Number of cells = 477344
    Number of faces shared with processor 0 = 606
    Number of processor patches = 1
    Number of processor faces = 606
    Number of boundary faces = 38680

Number of processor faces = 606
Max number of processor patches = 1
Max number of faces between processors = 606

Processor 0: field transfer
Processor 1: field transfer

End.

no problems in decomposition your geometry... Don't know whats wrong with your method. maybe use the 4-core PC and try decomposing for 2 cores there?! Maybe you should monitor your system (-> RAM) while you decompose your case.

Greetings

[pkr]

Hi lindstroem,

Thanks for your reply. I don't have any problem in decomposition, the problem comes when I run the case after decomposition.

Thanks.

[lindstroem]

Sorry, got that wrong. I started a computation (with unphysical boundary conditions, but I had no time to set up a "realistic" case) and it ran for 10 minutes. I can try your boundary conditions if you send them (maybe the whole case-directory).

Greetings

BTW: Changing the decomposition method sometimes helps as well.. e.g. try scotch instead of metis. But I dont know if this helps in your case..

[ehsankf]

Hi
I need the computational domain with layers of quads that have about equal lengths (a=b=c). From the bottom to the top, each layer has quads of increasing size.I there any way to create it in OF.

[Yosmcer]

Hello,

To refine the mesh starting from a blockMeshDict file, you can change the number of cell per block directly at the creation of the block:

blocks
(
hex (0 1 2 3 4 5 6 7) // vertex numbers
(10 10 10) // numbers of cells in each direction
simpleGrading (1 2 3) // cell expansion ratios
);

To make it easy, you can define a parameter at the beginning of your blockMeshDict:

n 10;

blocks
(
hex (0 1 2 3 4 5 6 7) // vertex numbers
(n n n) // numbers of cells in each direction
simpleGrading (1 2 3) // cell expansion ratios
);

If you need some calculus, it can be better to generate the blockMeshDict with a M4 file (calculus can be done in blockMeshDict, but as far as I tried, it generate one source code file in C++ per calculus and compile each of theses files, that makes at all a longer time than asking to perl to give you directly the answer).

I'm not an expert, but it is how I do to refine my meshes (I include some parameters in my M4 file).
The good thing is that if you do it well, you have full control of your mesh.

Hope it can help.