|
[Sponsors] |
[Other] bad setSize while building up a too large mesh ? |
![]() |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
![]() |
![]() |
#1 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear foamers,
I am trying to build up a mesh of about 600 millions of cells (3396*3396*50) and when I run blockMesh I get the following error : " Creating cells Creating points with scale 1 --> FOAM FATAL ERROR: bad set size -835122496 From function List<T>::setSize(const label) in file /usr/local/OpenFoam/OpenFOAM-2.0.x/src/OpenFOAM/lnInclude/List.C at line 322. FOAM aborting " The procedure I use work for the building of a four time smaller mesh, of about 150 millions of cells (1698*1698*50), but give the error above for the larger case. Does anyone have an idea of what should be done to solve this problem ? Thanks by advance for any help. Laurent |
|
![]() |
![]() |
![]() |
![]() |
#2 |
New Member
Anirban Jana
Join Date: Apr 2010
Location: Pittsburgh, PA, USA
Posts: 19
Rep Power: 16 ![]() |
Hi Laurent,
I just encountered the same error. Were you able to overcome this problem? My guess is that this is a case of overflow in a variable of type int. For example, for large meshes, the variables holding the number of points, faces, cells etc are susceptible to overflow. The solution may be to switch to "long int" when this happens. @ OpenFOAM developers: Should this be submitted as a bug report? Thanks Anirban |
|
![]() |
![]() |
![]() |
![]() |
#3 | |
New Member
Albert Pinto
Join Date: May 2013
Posts: 16
Rep Power: 13 ![]() |
Hello Laurent,
This questions of yours appears to have not gotten a response from any of the other fellow FOAMers. Did you manage to solve this problem? The problem seems to be a bit more involved than merely promoting "int" to "long int" in the label.H header file in primitives/ints/label (by defining FOAM_LABEL64 or setting FOAM_LABEL_MAX to a value greater than 2,000,000,000). Cheers, Albert Quote:
|
||
![]() |
![]() |
![]() |
![]() |
#4 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear Jans, Dear Abracurcix,
I didn't manage to slve this problem yet ; instead I dealt with smaller meshes. However I will try again soon so thank you for your threads I'll look cqrefully about this issue of long int. I'll post a threads as soon as (hopefully) i succed in working with lqrger meshes. Best regards, Laurent |
|
![]() |
![]() |
![]() |
![]() |
#5 |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,978
Blog Entries: 45
Rep Power: 128 ![]() ![]() ![]() ![]() ![]() ![]() |
Greetings to all!
If my memory doesn't fail me, the estimate is that 3396*3396*50 cells will require around 575 GB of RAM! A machine to test this is rather difficult to come by for testing such a big case. Let alone the time it takes to run in a single core for generating it... Nonetheless, if your machines do have that amount of RAM or more, I do strongly suggest that you report this on the bug tracker: http://www.openfoam.org/bugs/ Best regards, Bruno
__________________
|
|
![]() |
![]() |
![]() |
![]() |
#6 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear wyldckat,
I do use a fat node with 2TB RAM. But i encountered this bug with an old version of OF (2.0.1 I think), so I'll try again with 2.2 in the following days and then do a bug reports if I still get the same error. Best regards, Laurent |
|
![]() |
![]() |
![]() |
![]() |
#7 |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,978
Blog Entries: 45
Rep Power: 128 ![]() ![]() ![]() ![]() ![]() ![]() |
Greetings to all!
I've given this a try and I found out that there are several pieces of source code in OpenFOAM that are not prepared for "FOAM_LABEL64" ![]() The steps I've taken were:
![]() So, once again, since you guys have the machines with tons of RAM and are interested in the solution, please report this on the bug tracker: http://www.openfoam.org/bugs/ Best regards, Bruno
__________________
|
|
![]() |
![]() |
![]() |
![]() |
#8 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear all,
The bug is reported (ID 0000903 on the mantis). Thank you all for your answers Cheers Laurent |
|
![]() |
![]() |
![]() |
![]() |
#9 |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,978
Blog Entries: 45
Rep Power: 128 ![]() ![]() ![]() ![]() ![]() ![]() |
Greetings to all!
I haven't had the time and opportunity to look into this myself, but in the following bug report there are some instructions that might help: http://www.openfoam.org/mantisbt/view.php?id=1033 By the way, it's possible that OpenCFD won't fix the problem unless someone sponsors the fix and subsequent testing. Best regards, Bruno
__________________
|
|
![]() |
![]() |
![]() |
![]() |
#10 |
Member
Matthew J. Churchfield
Join Date: Nov 2009
Location: Boulder, Colorado, USA
Posts: 49
Rep Power: 18 ![]() |
Laurent,
Were you ever able to build very large meshes, or successfully compile with -DFOAM_LABEL64? |
|
![]() |
![]() |
![]() |
![]() |
#11 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Matthew,
I checked again with the version 2.0.x of OpenFOAM : it still does not work. With parallel meshing however one can build up relatively large meshes, with 100 millions number of mesh cells or more. See for example the following paper : http://authors.elsevier.com/a/1PsPx2OInFfaV Best regards, Laurent |
|
![]() |
![]() |
![]() |
![]() |
#12 |
Senior Member
Andrea Pasquali
Join Date: Sep 2009
Location: Germany
Posts: 142
Rep Power: 17 ![]() |
Dear all,
I found the same problem with the integer size in OpenFOAM. I recompiled OpenFOAM with the extension "-DFOAM_LABEL64" checking and correcting the pieces of code where the compilation gives me error. Now it works but I noticed that OpenFOAM in 64 requires 30% memory more than in 32 (running the same application with the same grid size). Should I check something else during the compilation? Or are there any other suggestions? Best regards Andrea
__________________
Andrea Pasquali |
|
![]() |
![]() |
![]() |
![]() |
#13 | |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,978
Blog Entries: 45
Rep Power: 128 ![]() ![]() ![]() ![]() ![]() ![]() |
Greetings Andrea,
Quote:
The problem of using 64-bit "labels" is that this means that in fact it uses 64-bit integers, instead of 32-bit integers. This means that all integers used in the code will use up 8 bytes per value, instead of 4 bytes. Therefore, consider yourself lucky it's only consuming more 30% RAM and not 50% ![]() One solution for this would require diagnosing all parts of the code and to isolate the situations where "2^31 = 2147483648" items (both negative and positive) is more than enough, i.e. where more than 2 billion cells/faces/points/hash lists are necessary. Also keep in mind that this can scale up to needing matrices for the equations with roughly as many entries over at least 2 dimensions... Keep in mind that OpenFOAM relies heavily on indexing points, cells and faces, hence using all labels at the same bit-length. Another solution would be to rely on hybrid mathematical libraries, such as GMP: https://gmplib.org/ - in theory, it could be able to handle 48-bit mathematics, which would equate to "2^47 = 1.407374884×10¹⁴" items. problem is that storage and access to 48-bit data isn't very "standard", which means that it would possibly require some strong efforts in having an intermediate interpretation layer between OpenFOAM's way of doing things and GMP's way of doing things. There was some discussion about this sometime ago... let me check... here you go: http://www.cfd-online.com/Forums/ope...precision.html - although that was more focused on the floating precision and not integer precision... The bottom line is: if you need to go into the 64-bit realm, be sure to have enough RAM to pay the price ![]() The other solution would be to handle your mesh in parallel, so that each partition doesn't go over the 32-bit limit. Best regards, Bruno |
||
![]() |
![]() |
![]() |
![]() |
#14 |
Retired Super Moderator
Bruno Santos
Join Date: Mar 2009
Location: Lisbon, Portugal
Posts: 10,978
Blog Entries: 45
Rep Power: 128 ![]() ![]() ![]() ![]() ![]() ![]() |
Greetings to all and a Happy New Year!
For those who aren't keeping track of the bug report, it has been fixed yesterday!
![]() Using this repository should roughly be used similarly to the git repository 2.3.x. Best regards, Bruno |
|
![]() |
![]() |
![]() |
![]() |
#15 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear all,
I have just tested the building of a mesh of ~600 millions cells with the Dev version of OpenFOAM on the fat node of CALMIP* (UVPROD, 2TB RAM) and it works fine. I will make a more detail return on this issue later on this year. Greetings, *http://www.calmip.univ-toulouse.fr |
|
![]() |
![]() |
![]() |
![]() |
#16 |
Member
Laurent Orgogozo
Join Date: Mar 2011
Location: Toulouse
Posts: 33
Rep Power: 15 ![]() |
Dear all,
With the development version of OpenFoam*, we have managed to make a scaling curve from 400 to 3200 cores on a case with a 1.2 billion cells mesh. We experienced super-linearity until 3200 cores. The runs have been done on EOS**, the new cluster of CALMIP. The building and the decomposition of the mesh were performed on an UVPROD cluster, with ~3TB RAM avalaible (the requests were using maximum 1.5 TB requests, i.e. half of the machine). This result has been presented in the beginning of may 2015 at the Maison de la Simulation in Saclay, France. You may find at the url below the pdf of this seminar (billion mesh cells scaling results : slides 20-22). http://www.maisondelasimulation.fr/en/semmod.php?a (scroll to may 2015 to find the presentation) Cheers, Laurent *https://github.com/OpenFOAM/OpenFOAM-dev **http://www.calmip.univ-toulouse.fr/s...php?article388 |
|
![]() |
![]() |
![]() |
![]() |
#17 |
New Member
zurich
Join Date: Apr 2022
Posts: 3
Rep Power: 4 ![]() |
To resolve the problem, simply you could set geometry of domain to involve negative values further positive ones. For instance, set the mesh box to start from (-10,-10,1-0) to (100,100,100).
***This may deals with problem*** /*--------------------------------*- C++ -*----------------------------------*\ ========= | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox \\ / O peration | Website: https://openfoam.org \\ / A nd | Version: 10 \\/ M anipulation | \*---------------------------------------------------------------------------*/ FoamFile { format ascii; class dictionary; object blockMeshDict; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // convertToMeters 1; x1 20; x2 150; x3 22; x4 20; x5 60; x6 50; h1 100; // elevation of the upper patch h2 3; h3 25; h4 5; d 100; // d is the depth in y direction normol to the plan vertices ( name v0 ( 0 0 0 ) // 0 node [QUOTE][/this locate box first vertices 0n (0,0,0) leads the problem ] name v1 ( $x1 0 0) // 1 node name v2 ( $x1 $h1 0) // 2 node name v3 ( 0 $h1 0) // 3 node name v4 ( #calc"$x1+$x2" $h2 0) // 4 node name v5 ( #calc"$x1+$x2" $h1 0) // 5 node name v6 ( #calc"$x1+$x2+$x3" $h3 0) // 6 node name v7 ( #calc"$x1+$x2+$x3" $h1 0) // 7 node name v8 ( #calc"$x1+$x2+$x3+$x4" $h3 0) // 8 node name v9 ( #calc"$x1+$x2+$x3+$x4" $h1 0) // 9 node name v10 ( #calc"$x1+$x2+$x3+$x4+$x5" $h4 0) // 10 node name v11 ( #calc"$x1+$x2+$x3+$x4+$x5" $h1 0) // 11 node name v12 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h4 0) // 12 node name v13 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h1 0) // 13 node name v14 ( 0 0 $d) // 14 node name v15 ( $x1 0 $d) // 15 node name v16 ( $x1 $h1 $d) // 16 node name v17 ( 0 $h1 $d) // 17 node name v18 ( #calc"$x1+$x2" $h2 $d) // 18 node name v19 ( #calc"$x1+$x2" $h1 $d) // 19 node name v20 ( #calc"$x1+$x2+$x3" $h3 $d) // 20 node name v21 ( #calc"$x1+$x2+$x3" $h1 $d) // 21 node name v22 ( #calc"$x1+$x2+$x3+$x4" $h3 $d) // 22 node name v23 ( #calc"$x1+$x2+$x3+$x4" $h1 $d) // 23 node name v24 ( #calc"$x1+$x2+$x3+$x4+$x5" $h4 $d) // 24 node name v25 ( #calc"$x1+$x2+$x3+$x4+$x5" $h1 $d) // 25 node name v26 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h4 $d) // 26 node name v27 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h1 $d) // 27 node ); blocks ( hex (v0 v1 v2 v3 v14 v15 v16 v17) (5 20 20) simpleGrading (1 1 1) //block1 hex (v1 v4 v5 v2 v15 v18 v19 v16) (20 20 20) simpleGrading (1 1 1) //block2 hex (v4 v6 v7 v5 v18 v20 v21 v19) (5 20 20) simpleGrading (1 1 1) //block3 hex (v6 v8 v9 v7 v20 v22 v23 v21) (5 20 20) simpleGrading (1 1 1) //block4 hex (v8 v10 v11 v9 v22 v24 v25 v23) (10 20 20) simpleGrading (1 1 1) //block3 hex (v10 v12 v13 v11 v24 v26 v27 v25) (10 20 20) simpleGrading (1 1 1) //block4 ); boundary ( front { type patch; faces ( (v0 v1 v2 v3) (v1 v4 v5 v2) (v4 v6 v7 v5) (v6 v8 v9 v7) (v8 v10 v11 v9) (v10 v12 v13 v11) ); } Back { type patch; faces ( (v14 v15 v16 v17) (v15 v18 v19 v16) (v18 v20 v21 v19) (v20 v22 v23 v21) (v22 v24 v25 v23) (v24 v26 v27 v25) ); } inlet { type patch; faces ( (v0 v3 v17 v14) ); } outlet { type patch; faces ( (v13 v12 v26 v27) ); } ground { type wall; faces ( (v0 v1 v15 v14) (v1 v4 v18 v15) (v4 v6 v20 v18) (v6 v8 v22 v20) (v8 v10 v24 v22) (v10 v12 v26 v24) ); } upperPatch { type patch; faces ( (v2 v16 v17 v3) (v2 v5 v19 v16) (v5 v7 v21 v19) (v7 v9 v23 v21) (v9 v11 v25 v23) (v11 v13 v27 v25) ); } ); // ************************************************** *********************** // but *** This may solve the problem *** /*--------------------------------*- C++ -*----------------------------------*\ ========= | \\ / F ield | OpenFOAM: The Open Source CFD Toolbox \\ / O peration | Website: https://openfoam.org \\ / A nd | Version: 10 \\/ M anipulation | \*---------------------------------------------------------------------------*/ FoamFile { format ascii; class dictionary; object blockMeshDict; } // * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * // convertToMeters 1; x_0 -10; y_0 -10; x1 #calc"20+$x_0"; //20; x2 150; x3 22; x4 20; x5 60; x6 50; h1 #calc"100+$y_0"; //100; // elevation of the upper patch h2 #calc"3+$y_0"; //3; h3 #calc"25+$y_0"; //25; h4 #calc"5+$y_0"; //5; d 50; //100; // d is the depth in y direction normol to the plan vertices ( name v0 ($x_0 $y_0 0 ) // 0 node this solve the problem x_0 -10; y_0 -10; name v1 ( $x1 $y_0 0) // 1 node name v2 ( $x1 $h1 0) // 2 node name v3 ( $x_0 $h1 0) // 3 node name v4 ( #calc"$x1+$x2" $h2 0) // 4 node name v5 ( #calc"$x1+$x2" $h1 0) // 5 node name v6 ( #calc"$x1+$x2+$x3" $h3 0) // 6 node name v7 ( #calc"$x1+$x2+$x3" $h1 0) // 7 node name v8 ( #calc"$x1+$x2+$x3+$x4" $h3 0) // 8 node name v9 ( #calc"$x1+$x2+$x3+$x4" $h1 0) // 9 node name v10 ( #calc"$x1+$x2+$x3+$x4+$x5" $h4 0) // 10 node name v11 ( #calc"$x1+$x2+$x3+$x4+$x5" $h1 0) // 11 node name v12 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h4 0) // 12 node name v13 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h1 0) // 13 node name v14 ( $x_0 $y_0 $d) // 14 node name v15 ( $x1 $y_0 $d) // 15 node name v16 ( $x1 $h1 $d) // 16 node name v17 ( $x_0 $h1 $d) // 17 node name v18 ( #calc"$x1+$x2" $h2 $d) // 18 node name v19 ( #calc"$x1+$x2" $h1 $d) // 19 node name v20 ( #calc"$x1+$x2+$x3" $h3 $d) // 20 node name v21 ( #calc"$x1+$x2+$x3" $h1 $d) // 21 node name v22 ( #calc"$x1+$x2+$x3+$x4" $h3 $d) // 22 node name v23 ( #calc"$x1+$x2+$x3+$x4" $h1 $d) // 23 node name v24 ( #calc"$x1+$x2+$x3+$x4+$x5" $h4 $d) // 24 node name v25 ( #calc"$x1+$x2+$x3+$x4+$x5" $h1 $d) // 25 node name v26 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h4 $d) // 26 node name v27 ( #calc"$x1+$x2+$x3+$x4+$x5+$x6" $h1 $d) // 27 node ); blocks ( hex (v0 v1 v2 v3 v14 v15 v16 v17) (5 20 20) simpleGrading (1 1 1) //block1 hex (v1 v4 v5 v2 v15 v18 v19 v16) (20 20 20) simpleGrading (1 1 1) //block2 hex (v4 v6 v7 v5 v18 v20 v21 v19) (5 20 20) simpleGrading (1 1 1) //block3 hex (v6 v8 v9 v7 v20 v22 v23 v21) (5 20 20) simpleGrading (1 1 1) //block4 hex (v8 v10 v11 v9 v22 v24 v25 v23) (10 20 20) simpleGrading (1 1 1) //block3 hex (v10 v12 v13 v11 v24 v26 v27 v25) (10 20 20) simpleGrading (1 1 1) //block4 ); boundary ( front { type patch; faces ( (v0 v1 v2 v3) (v1 v4 v5 v2) (v4 v6 v7 v5) (v6 v8 v9 v7) (v8 v10 v11 v9) (v10 v12 v13 v11) ); } Back { type patch; faces ( (v14 v15 v16 v17) (v15 v18 v19 v16) (v18 v20 v21 v19) (v20 v22 v23 v21) (v22 v24 v25 v23) (v24 v26 v27 v25) ); } inlet { type patch; faces ( (v0 v3 v17 v14) ); } outlet { type patch; faces ( (v13 v12 v26 v27) ); } ground { type wall; faces ( (v0 v1 v15 v14) (v1 v4 v18 v15) (v4 v6 v20 v18) (v6 v8 v22 v20) (v8 v10 v24 v22) (v10 v12 v26 v24) ); } upperPatch { type patch; faces ( (v2 v16 v17 v3) (v2 v5 v19 v16) (v5 v7 v21 v19) (v7 v9 v23 v21) (v9 v11 v25 v23) (v11 v13 v27 v25) ); } ); // ************************************************** *********************** // in the second blockMeshDict file x_0 -10; y_0 -10; first vertices start name v14 ( $x_0 $y_0 $d) |
|
![]() |
![]() |
![]() |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
[snappyHexMesh] very bad quality snapped mesh | federicabi | OpenFOAM Meshing & Mesh Conversion | 18 | September 26, 2018 10:33 |
[ICEM] 2D hybrid mesh (unstructured mesh highly dependent on structured mesh parameters) | shubham jain | ANSYS Meshing & Geometry | 1 | April 10, 2017 05:03 |
[snappyHexMesh] Snappyhex mesh: poor inlet mesh | Swagga5aur | OpenFOAM Meshing & Mesh Conversion | 1 | December 3, 2016 16:59 |
[mesh manipulation] Importing Multiple Meshes | thomasnwalshiii | OpenFOAM Meshing & Mesh Conversion | 18 | December 19, 2015 18:57 |
Problems of Duns Codes! | Martin J | Main CFD Forum | 8 | August 14, 2003 23:19 |