[Astan]

Hello everyone, I would kindly ask you what the Max Local Cells and Max Global Cells parameters refer to.
Reading on the internet I realized that Max Global Cells represents the maximum number of cells in the phase of castellation (before eliminating the portion of domain we are not interested in).
As for Max Local Cells, I did not understand what it refers to and how it could affect the number of cells generated and the quality of the mesh..

I noticed that by modifying Max Global Cells, it greatly increases the RAM used, but I do not understand how to set these parameters to have the best results in terms of mesh quality.

In my simulation i have available 32 GB RAM and 4 cores.

I thank you in advance for the time spent for reading the message and for any answers!

[Tobi]

Hi,

maxLocalCells within the snappyHexMesh file tells you how much cells you can have per core.

maxGlobalCells within the snappyHexMesh file tells you how much cells you can have in total.

Assuming that you don´t decompose your background and use the application in serial, both are equal; it is obvious because the number of cells within the mesh represent (i) the maxLocalCells for the serial processor and (ii) the maxGlobalCells in the whole domain. However, for parallel meshing, it can occur that in one processor domain you have a high refinement based on a lot of details. Therefore, it can occur that you get much more cells (based on splitting) on one processor compared to others. Thus, if you will reach the maxLocalCells on that processor, it will stop refining even though, you would like to refine further or the maxGlobalCells is still not reached. If the maxGlobalCells value is reached, the whole castellated process will stop - no refinement will be done in any case.

Back to your meshing step. If you realize, that an increase of maxGlobalCells will use more cells, this is actually based on the fact that there are still cells to be refined (based on your refinement settings).

[Astan]

Hi Tobi,


Thanks for the reply, it was very clear and exhaustive!

Just one last clarification I would kindly ask.

I set MaxGlobalCells to 12000000, I set castellatedMesh true, snap true, addLayers false and I noticed that the number of cells reached is equal to 15000000, to what do you attribute this gap with respect to the value of MaxGlobalCells?

Is there a law to set MaxLocalCells once MaxGlobalCells, the number of Cores used and the available RAM are fixed?

Thanks in advance for any answers!

Astan

[Tobi]

I have to make an additional statement. The maxLocalCells is used additionally for balancing in the parallel run (as given in the description). Thus, if any processor reaches this limit, it starts to try to balance the workload based on the keyword maxLoadUnbalance. However, balancing is expensive and therefore, an unbalanced system might be better (that´s why the maxLoadUnbalance value is around 10 %). So I am sorry that I did not add this critical detail.

Furthermore, the maxGlobalCells is just an approximation value. I never checked the deviations of cells to that value, but if you set 1000 cells, preferably you will get more cells. However, this setting is used to prevent freezing your local computer or cluster; e.g., running out of memory. Sometimes I run into an overload of memory (snappyHexMesh needs a lot of memory) and after I start swapping, its almost over - a hard restart has to be done.

To your question. I cannot give you any information. My meshes are small < 2.000.000 cells (128 GB Memory)

[Astan]

Thanks you very much! You helped me a lot with your advice. Now it is more clear.

Greeting
Astan

[akidess]

Originally Posted by Astan

Hi Tobi,

I set MaxGlobalCells to 12000000, I set castellatedMesh true, snap true, addLayers false and I noticed that the number of cells reached is equal to 15000000, to what do you attribute this gap with respect to the value of MaxGlobalCells?

snappy checks the mesh count vs maxGlobalCells after each refining step, not during. Assume you have 1 cell, and maxGlobalCells=3 and your requested refinement level is 3. snappy will do one refinement to 8 cells (level=2), then notice that maxGlobalCells is reached and stop refining.

[srikrishnan_balasubramani]

Originally Posted by Tobi

I have to make an additional statement. The maxLocalCells is used additionally for balancing in the parallel run (as given in the description). Thus, if any processor reaches this limit, it starts to try to balance the workload based on the keyword maxLoadUnbalance. However, balancing is expensive and therefore, an unbalanced system might be better (that´s why the maxLoadUnbalance value is around 10 %). So I am sorry that I did not add this critical detail.

Furthermore, the maxGlobalCells is just an approximation value. I never checked the deviations of cells to that value, but if you set 1000 cells, preferably you will get more cells. However, this setting is used to prevent freezing your local computer or cluster; e.g., running out of memory. Sometimes I run into an overload of memory (snappyHexMesh needs a lot of memory) and after I start swapping, its almost over - a hard restart has to be done.

To your question. I cannot give you any information. My meshes are small < 2.000.000 cells (128 GB Memory)

Hi,

I have question on this maxLocalcells and maxGlobalcells. From your previous answer, I get that we can set maxGlobalcells to a high value based on our requirement and refinement level. Coming to maxLocalcells, you have mentioned that it is the cells per core. Is there any method to find out how many cells one core can handle or is it based on trial and error we find ???
Because I ran out of memory when i try to refine my geometry at a particular region. And I have to somehow refine a particular region.

So based on your answer I come under this conclusion that to refine a geometry either I can increase the maxLocalcells till value until the core runs out of memory or I can increase the number of cores (assuming I have access to clusters) used.

Is this correct ??

Many Thanks,
krishna

[Kalabagh]

Originally Posted by srikrishnan_balasubramani

Hi,

I have question on this maxLocalcells and maxGlobalcells. From your previous answer, I get that we can set maxGlobalcells to a high value based on our requirement and refinement level. Coming to maxLocalcells, you have mentioned that it is the cells per core. Is there any method to find out how many cells one core can handle or is it based on trial and error we find ???
Because I ran out of memory when i try to refine my geometry at a particular region. And I have to somehow refine a particular region.

So based on your answer I come under this conclusion that to refine a geometry either I can increase the maxLocalcells till value until the core runs out of memory or I can increase the number of cores (assuming I have access to clusters) used.

Is this correct ??

Many Thanks,
krishna

Hi Krishna,


I don't know so much how to answer your question, but those two settings are just limits used for avoiding chrashes or freezings of the machines. A correct mesh generation should not activate those limits. To find a good value there's nothing you can do but to start with a low refined mesh, see the statistics, run an expanded checkMesh and use a viewer (Paraview) to describe what problems does it have. From this point is preferable to aplicate refinements and changes in the setup based in what you have encountered (if not, you can easily produce an innefficient mesh).


Also, don't go crazy with the nº of CPU's because all the power you gain on running in parallel, you will loose it during the solver run because of the needed communication between CPU's.


Thanks.

[Kalabagh]

Regarding with this,


Do anybody know if the log of snappyHexMesh informs of when maxGlobalCells or maxLocalCells has been activated?? There's any way to know from it's activation??


Thanks.

[Tobi]

Yes, there is some line written if you exceed the limit. It tells you that "No further guys are refined as limit of cells is reached".

Shell refinement iteration 2
----------------------------

No cells marked for refinement since reached limit 20000.
Determined cells to refine in = 0.02 s

maxLocalCells determines for decomposed methods if the number of cells are re-distributed.