foam-extend-4.1 release

hjasak · November 4, 2019, 06:36

The new version of foam-extend has been released following extensive development and testing and is available for download:

https://sourceforge.net/projects/foa...am-extend-4.1/

The foam-extend project is a fork of the OpenFOAM® open source library for Computational Fluid Dynamics (CFD). It is an open project welcoming and integrating contributions from all users and developers.

In total, the release consists of 1450 commits since the last release

Some major development features:

Block-coupled pressure velocity solver for steady and transient simulations of incompressible turbulent fluid flow. Fully implicit handling of porosity and MRF in block-coupled solvers

This is the final release of complete functionality for pressure-based implicit block-coupled solvers steady and transient incompressible turbulent flows. The pressure and momentum equation are solver in a single linear block with 4x4 coefficients, with full implicit support for multiple frames of reference and porosity. The linear system is solved using the block-coupled AMG solver (see below).

The code supports implicit interfaces without transformation such as GGI. For fully implicit treatment of symmetry plane boundaries, please use the blockSymmPlane boundary condition on velocity.

Consistency of p and U boundary conditions is necessary. The code does not support indeterminate form of pressure matrices, meaning that zero gradient boundary condition on the complete periphery of the domain is not allowed. At least a single boundary pressure reference point is required. Consistent treatment of inletOutlet velocity boundary condition requires the equivalent pressure boundary condition to be specified as outletInlet.

The speed-up compared to the segregated solution comes from significant change in relaxation factors. A typical relaxation factor on velocity is 0.95; for trivial meshes, academic problems and appropriate choice of convection discretisation the solver can also operate without relaxation on U, but for industrial cases this is not recommended. Typical relaxation factors on turbulence variables is 0.9 or 0.95, depending on the complexity of the case. Further improvement may be achieved using block-coupled turbulence models (see below).

Significant effect on coupled solver performance is achieved using appropriate linear algebra settings. It is recommended to use the block-AMG solver with the block-SAMG coarsening and ILUC0 smoothers.

Expected performance of the coupled solver compared to the segregated solver for the same steady state case is a factor of 3 in execution time, at a cost of using triple the amount of RAM memory, due to the storage of the coupled matrix.

Parallel scaling of the coupled solver on a large number of processors is significantly better than the equivalent segregated solver, as the number of MPI messages is reduced, with messages of larger size. The solver is tested to the levels of hundred of millions of cells and thousands of cores.

In transient simulations, the coupled solver gives advantage over the segregated solver because of its accuracy and because p-U coupling is not dependent on the time-step size or maximum CFL number in the domain. It is recommended for use in large LES-type simulations, where there is a significant difference between the mean and max CFL number. Outer iterations in the transient solver can be enabled but are typically not necessary.

For details of the coupled solver and AMG solver technology we recommend the following references:

Uroić, T., Jasak, H.: Block-selective algebraic multigrid for implicitly coupled pressure-velocity system, Computers&Fluids, 2018

Beckstein, P., Galindo, V., Vukčević, V.: Efficient solution of 3D electromagnetic eddy-current problems within the finite volume framework of OpenFOAM, Journal of Computational Physics, Volume 344, 1 September 2017, Pages 623-646

T Uroić, H Jasak, H Rusche: Implicitly coupled pressure–velocity solver OpenFOAM: Proceedings of the 11th Workshop, Springer, 249-267

Fernandes, C., Vukcevic, V., Uroic, T., Simoes, R., Carneiro, O.S., Jasak, H., Nobrega, J.M.: A coupled finite volume flow solver for the solution of incompressible viscoelastic flows, Journal of Non-Newtonian Fluid Mechanics, 2019

Immersed Boundary Surface Method. Support for turbulence, dynamic immersed boundary and adaptive polyhedral refinement on immersed boundary meshes

The new formulation of the Immersed Boundary Method (IBM) is a complete methodology rewrite of the work implemented in foam-extend-3.2 and 4.0. It was shown that the principle of near-body interpolation is not sufficiently powerful for the flexibility and accuracy required for practical engineering simulation. On suggestion of dr. Tukovic, the new method performs the actually cutting of the background mesh with the immersed boundary surfaces, modifying internal cells and faces and creating new intersection faces. The Immersed Boundary (IB) faces exist in their own patch and are not present in the face list belonging to the polyMesh.

Representation of IB in the background mesh is achieved by using the intersection faces of the surface mesh and cells on the background mesh. The resolution of the original surface mesh does not influence the accuracy of the IBM: this is only influenced by the background mesh. For cases of "unclean intersection", such as the surface mesh coinciding with the points or faces of the polyMesh, the error mitigation algorithm is implemented: the Marooney Maneouvre. This will ensure that the cut cell is geometrically closed (sum of face area vectors for the cell equals zero vector) under all circumstances.

The limiting factor of the IBM is the fact that a single background cell mesh can only be cut once. The limitation is mitigated by the use of adaptive mesh refinement, based on the distance to the IB surface, which is provided as a part of the package.

The background mesh for the IBM calculation can be of arbitrary type: polyhedral cells are fully supported. The IBM can be combined with other complex mesh operations and interfaces: moving deforming mesh, topological changes and overset mesh.

Post-processing of the immersed patch data is performed separately from the main mesh. Individual VTK files are written for each field in the time directory, due to the limitations of the current VTK export format.

The method is enabled to support moving deforming immersed surface, optionally operating on a moving deforming mesh.

IBM implementation operates correctly in parallel on an arbitrary mesh decomposition. Interaction of IBM and processor boundaries is fully supported.

For static mesh simulations, regular static mesh boundary conditions may be used on IBM patches; however, the surface data for IBM patches will not be exported for post-processing. To achieve this, IBM-specific boundary conditions may be used. IBM does not carry execution overhead compared to the body-fitted mesh on static mesh cases, beyond the calculation of original IBM intersection.

For dynamic mesh simulations, IBM-specific boundary conditions need to be used in order to handle the interaction of a moving deforming IBM and the background mesh, where the number of intersected cells changes during the simulation.

The best reference for the Immersed Boundary methodology currebly publicly available is:

Robert Anderluh: Validation of the Immersed Boundary Surface Method in Computational Fluid Dynamics, Master Thesis, Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, 2019

http://cfd.fsb.hr/wp-content/uploads...uhMSc_2019.pdf

Further publications are under way.

Overset Mesh Method. New automatic overset mesh fringe calculation algorithms.

Further development of the native implementation of overset mesh includes work on automatic fringe detection and fringe minimisation. Parallel fringe search algorithm and inter-processor fringe communication have been improved.

Polyhedral adaptive mesh refinement and coarsening, working on all cell types, in 2-D and 3-D.

A new adaptive mesh refinement and coarsening algorithm has been developed and deployed. The algorithm operates on arbitrary polyhedral meshes, offering refinement and coarsening of polyhedral cells. On hexahedral cell types, refinement is equivalent to 2x2x2 splitting of a hexahedron, while on polyhedra the algorithm regularises the mesh towards hex types. Mesh coarsening has been executed based on re-assembling the cells from previously refined clusters. pointLevel and cellLevel fields are no longer needed as a read-input and can be re-created from existing mesh structure. This allows coarsening of initial locally consistent refined meshes as received from external meshing tools.

In 2-D simulations, the adaptive mesh refinement and coarsening algorithm will correctly recognise the planar/wedge conditions and only operate in live directions.

Dynamic load balancing for parallel topologically changing meshes

A native implementation of dynamic load balancing is implemented as a low-level function of a topologically changing mesh. Load balancing as a function of a topoChangerFvMesh virtual base class, making it available for all types of topological changes (or as a response to external load imbalance for a static mesh). Implementation uses the tools developed for parallel decomposition/reconstruction with changes needed for Pstream communication. Balancing action is executed as a global decomposition, assemble of a-b migrated meshes (using decomposition tools), migration via Pstream communication and re-assembly at target processor (using reconstruction tools). Field data follows the same path, migrating with relevant mesh data. Load balancing is typically used with adaptive mesh refinement and is thoroughly tested for large parallel decompositions. Cases of "zero cell at processor" are fully supported; this allows the load balancing tool to be used for initial decomposition or reconstruction., which no longer relies to point/face/cell-ProcAddressing fields.

Linear solver and block linear solver improvements

In the search for significant performance improvements on meshes with coupled interfaces and large-scale HPC, significant work has been done on linear algebra. On preconditioners, Crout-type ILU preconditioners are implemented. For meshes where there is direct contact between face-neighbours of a cell (virtually all mesh structures, apart full-hex meshes), the diagonal based ILU preconditioning is incorrect, with consequences on solver performance. To replace this, Crout-type preconditioners and smoothers are implemented both for the segregated and block-coupled solvers. Variable-level fill-in ILU-Cp and zero fill-in ILU-C0 preconditioners are implemented, with several variants of preconditioning across processor boundaries. Performance testing of processor-aware ILU-type preconditioners is likely to continue for some time.

On linear solver methodology, major work has been done to improve the performance of the linear algebra package where a number of matrix rows (cells) is excluded from the simulations, such as immersed boundary and overset meshes. In particular, zero-group-handling in AMG coarsening is implemented. New agglomeration algorithms resulting from the work at Uni Zagreb have been implemented, including a smart cell clustering algorithm and a generalisation of the Selective AMG work by Stuben et al. Here, a coarse level of multigrid is created by equation selection (as opposed to agglomeration), based on priority criteria of equation influences. The algorithms have been generalised on non-M, non-symmetric and non-diagonally dominant matrices. Parallel handling of coarse level selective AMG interfaces, splitting the triple matrix product coarse level assembly operations to relevant processors has been implemented. The selective AMG (incredibly) shows theoretical convergence properties of 1-order-of-magnitude residual reaction per V-cycle (theoretically, W-cycle) even on industrial grade meshes.

The block-coupled solver implements both the equation clustering and equation selection operations on a block-matrix system using the appropriate norm of a block-coupled coefficient. The algorithms mirror the scalar version, and show remarkable convergence characteristics for a block system without diagonal dominance, such as implicitly coupled U-p block matrix. Again, theoretical convergence behaviour is indicated on industrial strength meshes.

For further information see:

Tessa Uroic: Implicitly Coupled Finite Volume Algorithms, PhD Thesis, Faculty of Mechanical Engineering and Naval Architecture, University of Zagreb, 2019

Major performance improvement for parallel overset and GGI interfaces

Performance improvement for GGI and related interfaces (partial overlap, mixing plane) in parallel execution has been implemented by distributing the work on all available processors.

Consistent SIMPLE and PISO segregated algorithms, where the solution is independent of time-step size or relaxation parameters

The final and validated version of the consistent handling of relaxation and time-stepping within the SIMPLE-PISO family of algorithms has been deployed. The code has been validated and shown to remove relaxation- and time-step dependent artefacts in steady and transient solutions

New formulation of buoyant Boussinesq approximation solver

Alternative formulation of the steady Boussinesq approximation solver for buoyant flows has been released, following community comments on the lack of accuracy and instability of the original formulation

Incremental development of the Finite Area Method and liquid film solver

All your contributions are highly welcome: New solvers, utilities and models; bug fixes; documentation. The many ways of contributing and the contribution process are described in detail at:

http://sourceforge.net/p/foam-extend...wToContribute/

Hrvoje Jasak

kmooney · November 6, 2019, 19:12

Not sure if its a temporary issue on the side of the host but the parmetis Package URL appears to be down.

tschenkel · November 21, 2019, 04:38

Brilliant. Thanks for all the effort that went in this, and the quick bug fix to the MPI/GAMG issue.

I am planning to roll this ouit to students and was wondering if the ubuntu1804 deb package will be updated on a regular basis.

If not, I'm happy to compile and package a tgz for the students myself, but the debian installation would make things a lot easier. So if there is going to be an updated deb soon, I may delay the rollout.

Thanks again.
T

Linxiangfeng · January 12, 2020, 19:01

Hi,
I am using the latest released OF now and testing its capability of immersed boundary method. However, 'bad cell cut' warning occurs when running the case. Does it matter or what can I do to avoid it?
Thanks you all.
regards.

Code:

Bad cell cut: volume = (1.03008 0.0341328) = 1.06421

Bad cell cut: volume = (1.08702 0.016779) = 1.10379

Bad cell cut: volume = (1.04602 0.0287138) = 1.07474

Bad cell cut: volume = (1.0217 0.0324708) = 1.05417

Bad cell cut: volume = (1.01828 0.0369687) = 1.05525

Bad cell cut: volume = (1.01267 0.000227666) = 1.01289

Bad cell cut: volume = (1.03638 0.0149395) = 1.05132

Immersed boundary blades info: nIbCells: 8325 nDeadCells: 1999 nIbFaces: 15201 nDeadFaces: 10114

hjasak · January 14, 2020, 06:41

Hi,

This is immersed boundary. The rule says that the cell can only be cut ONCE by an immersed boundary patch. Therefore, either you have 2 IB patches really close to each other - cutting the same cell - or you have a lot of details on the IB surface that cannoe be captured on the background mesh.

The IBM algorithm will work correctly, but this indicates a mis-match between the background grid and IB surface. You can do the following:
- look at the surface to see what causes the error
- use uniform or adaptive refinement to put more cells into the problem region. There are tools like refineImmersedBoundaryMesh to do this
- have a look at your STL to make sure it is reasonably clean.

In conclusion: safe to proceed, but with care.

Hrv

Linxiangfeng · January 14, 2020, 18:00

Quote:

Originally Posted by hjasak

Hi,

This is immersed boundary. The rule says that the cell can only be cut ONCE by an immersed boundary patch. Therefore, either you have 2 IB patches really close to each other - cutting the same cell - or you have a lot of details on the IB surface that cannoe be captured on the background mesh.

The IBM algorithm will work correctly, but this indicates a mis-match between the background grid and IB surface. You can do the following:
- look at the surface to see what causes the error
- use uniform or adaptive refinement to put more cells into the problem region. There are tools like refineImmersedBoundaryMesh to do this
- have a look at your STL to make sure it is reasonably clean.

In conclusion: safe to proceed, but with care.

Hrv

Hi, Hrv
Thank you so much. It is so glad to receive your reply. You have done a great job in the development of OF. I will look into my case following your advice.

Thanks again.
Regards,
Xiangfeng

time- · January 17, 2020, 06:28

Hello Hrvoje,
is there any documentation about overset workflow in foam-extend-4.1?
I think your overset implementation is different from OpenCFD's one, am I correct?
Is there any interpolation schemas description and oversetMeshDict syntax docs?

Bye
Mattia

sai193 · February 28, 2020, 11:53

Hello!

I recently made the switch to foam-extend to try out the block-coupled pU solver.

I am now trying to run a case that contains both a MRF zone and a porous zone and was wondering how to go about this in foam-extend. Is there a way to add a MRF or porous source to simpleFoam or pUCoupledFoam via fvOptions as you can do in OpenFOAM?

Also, how would you go about running a transient simulation using the block-couple pU solver?

Finally, I was wondering if 'cyclicAMI' can be used in foam-extend or if ggi/cyclicGgi should be used instead as in the MRFSimpleFoam tutorials?

Thanks to all in advance!
Kind regards,
Sergio

hjasak · March 1, 2020, 09:44

Hi,

You have he MRFPorousFoam solver that does all that.

Hrv

sai193 · March 5, 2020, 10:22

Thank you very much for the response!

I had a go at using MRFPorousFoam but ran into issues at the ggi/cyclicGgi interfaces:

For example, the images below are a capture of the axialTurbine_ggi case from $FOAM_TUTORIALS/incompressible/MRFSimpleFoam/, ran using MRFSimpleFoam and MRFPorousFoam,

… and a snippet of the log output showing continuity error and increased flux through the interfaces.

The only changes made to the MRFPorousFoam setup were the added Up solver, blockSolver, fieldBounds and reduced under-relaxation in the fvSolution file, the coupled turbulence model and the div(U) term in fvSchemes. I ran the case using various settings for linear solvers/turbulence model/ boundary conditions/schemes, all without much effect.

My own test case consisting of a propeller blade at hover inside a (non-conformal) MRF zone shows the difference between my coupled and segregated solver runs a little clearer:

(It looks as is if the airflow is being reversed..)

I was wondering if you are familiar with this behaviour and if so, what changes to the setup or mesh you think would allow me to run the tutorial case using the coupled solver?

The only other post I could find mentioning issues with the ggi interface is from a thread several years ago: pUCoupledFoam with Multiple Reference Frames (MRF).

Thank you again!

Sergio

sai193 · March 5, 2020, 11:24

Thank you very much for the response!

I had a go at using MRFPorousFoam but ran into issues when using ggi/cyclicGgi interfaces.

For example, in the attachments image 1 shows a screen capture of my runs of the axialTurbine_ggi tutorial case (from $FOAM_TUTORIALS/incompressible/MRFSimpleFoam/) using MRFSimpleFoam and MRFPorousFoam.

Image 2 is a snippet of the log output showing continuity error and increased flux through the interfaces with the coupled solver.

The only changes made to the MRFPorousFoam setup were the added Up solver, blockSolver, fieldBounds and reduced under-relaxation in the fvSolution file, the added div(U) term in fvSchemes and the coupled turbulence model. I also tried using various settings for boundary conditions/RAS properties/linear solver settings and schemes, all without much effect.

My own test case (images 3 and 4), consisting of a propeller blade inside a (non-conformal) MRF zone shows the difference between my coupled and segregated solver runs a little clearer (it looks as if the airflow is being reversed..).

I was wondering if you are familiar with this behaviour and if so, whether you have any suggestions on how to adapt the tutorial case to make it run successfully using MRFPorousFoam?

The only other post I could find mentioning issues with ggi in this context is from a thread from several years ago: pUCoupledFoam with Multiple Reference Frames (MRF).

Thanks again for your help!
All the best,
Sergio

zjz · April 20, 2020, 11:59

Hi, prof Jasak:
I'am using the immersed boundary method in openfoam-extend 4.1 to calculate the flow field of a NACA0012 foil. But I got some errors ,such as:

1.
External flow
Immersed boundary ibNACA info: nIbCells: 358 nDeadCells: 739 nIbFaces: 223 nDeadFaces: 3144
--> FOAM Warning :
From function void Foam::immersedBoundaryPolyPatch::calcCorrectedGeom etry() const
in file immersedBoundaryPolyPatch/immersedBoundaryPolyPatch.C at line 1381
Minimum IB face area for patch ibNACA: 0. Possible cutting error. Review immersed boundary tolerances.
Reading field U
////

2.
Calculating divSf
Face areas divergence (min, max, average): (0 2e-05 3.44861e-10)
--> FOAM Warning :
From function writeIbMasks
in file writeIbMasks.C at line 166
Possible problem with immersed boundary face area vectors: 2e-05
Open cell 216149: 1.41421e-05 gamma: 1
Open cell 216854: 1.41421e-05 gamma: 1
Open cell 217551: 2e-05 gamma: 1
Open cell 217552: 1.41421e-05 gamma: 1
Open cell 218249: 1.41421e-05 gamma: 1
Open cell 223199: 6.7868e-07 gamma: 0.932174
Open cell 223208: 6.77995e-07 gamma: 0.932218
Open cell 223216: 6.78226e-07 gamma: 0.932194
Open cell 223225: 6.79081e-07 gamma: 0.932106
Open cell 223230: 6.80099e-07 gamma: 0.932024
Open cell 223234: 6.80846e-07 gamma: 0.931924
Open cell 223253: 6.2886e-07 gamma: 0.967803
Open cell 223255: 6.8835e-07 gamma: 0.931207
Open cell 223263: 6.92602e-07 gamma: 0.930771
Open cell 223277: 6.31157e-07 gamma: 0.967062
Open cell 223295: 6.33853e-07 gamma: 0.965989
Open cell 223300: 7.35851e-07 gamma: 0.926549
Open cell 228749: 1.41421e-05 gamma: 1
Open cell 229451: 2e-05 gamma: 1
Open cell 229452: 1.41421e-05 gamma: 1
Open cell 230154: 1.41421e-05 gamma: 1
Open cell 230849: 1.41421e-05 gamma: 1

I have tried to adjust the background grid and the STL grid. But it can't remove these errors.
I was wondering if there are some relations between the background grid and the STL grid, and I hope you could give me some suggestions.

regards.

jantheron · April 26, 2020, 05:30

I am having trouble running the parallel overset tutorials on foam-extend 4.1

I installed it on two systems, both running Ubuntu 18, and get the same MPI error during the runApplication phase (segmentation faul on processor 3).

There is an earlier error in the mergeMeshes procedure:

--> FOAM Warning :
From function Foam::forces::forces(const word&, const objectRegistry&, const dictionary&, const bool)
in file forces/forces.C at line 209
No fvMesh available, deactivating

but that does not seem to terminate the run.

Has anyone come across this, or have been able to run this tutorial successfully?

Jan Theron

Kombinator · May 31, 2020, 21:52

Dear Hrvoje,

Do I understand correctly that AMG linear solver does not work with a specific coupled matrix as one present in conjugateHeatFoam? Is there any possibility to use other linear solvers for the coupled matrix in the conjugateHeatFoam besides these:

Code:

Valid asymmetric matrix solvers are :

3
(
BiCG
BiCGStab
smoothSolver
)

Thank you.

idrees khan · September 6, 2020, 08:53