[arjun]

This is a small note introducing FVUS (Finite Volume Utilities and Solvers) / Wildkatze.

This is a thread I wish to maintain to update about FVUS/Wildkatze for the people who wish to follow its update and feature additions.

NOTE: The default version that could be downloaded from http://www.dravvya.co.in is usable till 200000 cells which I feel is good enough size for small size calculations.

NOTE: Windows version does not do User Coding now because this part for windows needs to be re-written and its on low priority as of now.

NOTE: Since its always being tested and changes are coming in, the solver will be updated as soon as some important feature is added or some bug is fixed.

NOTE: Linux version of the code does partitioning in parallel, the windows version uses different partitioner and thus can only do serial partitioning. (graph collected to single processors and results are distributed back). (This i clearify because I just saw this thread Mesh too big for memory. How to perform decomposition in parallel? and i think its worth clearifying it). Like other solvers like Fluent, starCCM+ the paritioning happens when case is loaded and only 1 single mesh file is saved. This means user can save and restore calculation with different number of processors and does not have to run decompose differently).


Introduction

I started to work on FVUS project in month of January 2015. The aim was to provide framework for multi-region, multi-phase simulations. In nutshell one can view it to be an alternative to already existing finite volume solvers like Fluent, CFX, StarCCM+, OpenFOAM or any such already existing solver.

In last two years the FVUS has been one man effort. In two years of its development framework has gone through various changes and it has been constantly tested. FVUS is written in C++ while its helper GUI is written in QT. GUI is made open source code. (started as side project in August 2015 to help setting up calculations but now has grown bigger project). FVUS is MPI based parallel solver.

As name suggests FVUS is finite volume framework for generalized research. FVUS also provides collection of physics models to carry out most common simulations in case user decides to use FVUS as multi-physics solver. Off course it is possible to use FVUS as combination of a multi-physics solver and solver for customized general research.
FVUS thus promises user to take care of most of finite volume related task like loading the mesh, creating user variables, user settings for user models, saving data for restart or exporting user variables for post processing (these are few examples of things FVUS will do for user).
This nature of FVUS that it can be both a multi-physics solver and a customized research tool, puts FVUS at unique positio.

[arjun]

General Philosophy of FVUS


It starts with a very general question how to perform a multi-physics simulation involving one or more phases on/in a multi-region grid/mesh. The question is further expanded into asking how to perform some physics on some of the regions and some other physics on entirely different set of regions.



Natural answer to these questions is that we define sets of phases and sets of regions and we define physics model to be solved over these phase sets and region sets. This allows us to apply a physics model Physics1 on Phase-Set1 and Region-Set1, while a model Physics2 could be applied on Phase-set2 and Region-Set2. Here Phase-Set1 and Region-set1 could have common phases and common regions with Physics-Set2 and Region-Set2.


This is exactly what FVUS does. A very general outline of FVUS process is:
1. Read mesh information
2. Define region sets on which user intend to apply physics or general models. (User could as well ignore some of the regions if he wishes. It is quite flexible framework).
3. Define phases and assign them material if needed. (In built database provide common materials like air water etc).
4. Define phase sets on phases defined.
5. Define physics models on phase sets and region sets created.
6. Save simulation (simulation tree or .stree).
7. Perform simulation using wildkatze (solver for FVUS project is called wildkatze, a German language word for wildcat).


Because of this design FVUS is very flexible. For example one could define SST-K Omega turbulence model on region1 and Spalart Almaras turbulence model on region2, while same time solving flow model on set of region1 and region2. This in itself does not sound much interesting but it implies that some of the difficult problems in CFD like Conjugate Heat Transfer are defined trivially in this framework where heat transfer is calculated on bigger set of regions while flow is solved only in flow regions.



One could off course decide not to use any of the physics models provided by FVUS and just use User Coding model. This way user could only use all the goodies provided by FVUS for finite volume framework and use User Coding model to do his specialized research models.



Special attention is given to developing user coding to provide better access to solver internal data and to provide better user coding framework for performing tasks easily.

[arjun]

FVUS Solver


Steady/Transient
FVUS Solver provides steady as well as unsteady simulations (Implicit Euler and 3 Time Level 2nd Order Scheme) with iterative as well as fractional (or hybrid) framework.

https://www.youtube.com/watch?v=aC6WzjTBNe8

Gradients
Gradient evaluation could be based on Green Gauss Method or Weighted Least Square Method which could be selected/changed in every individual model. (For example StarCCM or Fluent solver user can either use Green Gauss or Least Square but then that method is applied to all the models).



Linear Solvers
FVUS provides additive corrective multigrid method with three types of smoothers:
1. Gauss – Seidel smoother
2. Polynomial Smoother (Based on Chebyshev Polynomials)
3. Auto-smoother (Experimental smoother to tackle difficult systems, that checks and adjusts its behaviour based on convergence of linear system. Only recommended when there is difficulty in convergence since it is costly solver).

[arjun]

Motion and Immersed Boundary Methods

FVUS allows user to connect various regions via generalized grid grid interface where grids could be unmatched. Linear system for each model that uses region set containing GGI takes account of this connection and GGI is treated implicitly. (That is a single matrix is created for all the regions plus interfaces).

User at this moment can apply simple motion to any region (called rigid body motion) which means two or more regions can slide about each other (sliding interface).


https://www.youtube.com/watch?v=gX1PnqLO-R4

FVUS also has immersed boundary framework. Simple motion is also supported for immersed regions (that is rotation and translation)


https://www.youtube.com/watch?v=PcQrPSIpZR8

[arjun]

Flow Solver

Flow is solved with collocated SIMPLE method suitable for unstructured grids. Convection terms are treated with one of the three methods: Upwind, Second Order Upwind and CUBISTA (stable version of QUICK scheme)


Velocity and Pressure are coupled by delta-pressure dissipation method (similar to what FLUENT has) where as standard Rhie and Chow method could also be selected as an alternative (called Standard method).

For body force dominated flows user can change pressure interpolation from Momentum Coefficient Weighted (standard) to Body Force Weighted scheme. It is very similar to what ANYSYS Fluent recommends for multiphase models). This scheme is recommended for buoyancy dominated flows like natural convection (Bousenesq Compressiblity Method).
(See attached picture)

Flow solver also supports compressible flow for Ideal Gas based gas law. (User defined compressibility to be added in future).

Of course Gravity and Bousenesq Compressibility Method for adding body forces are also present and could be added as physics models.

Turbulence
For turbulent flows FVUS provides SST K-Omega, Spalart Almaras and Smagorinsky LES methods.



K Epsilon and Dynamic Smagorinsky would be added in near future.

[arjun]

Multiphase Methods

Current version of FVUS supports Mixture multiphase method based on algebraic drift flux model and VOF method.
For VOF method interface is resolved via HRIC or W-THINC schemes. CICSAM and STACs schemes also supported but are not active for user to select them. This is done because HRIC and THINC scheme provide very good performance.



Mixture Multiphase



https://www.youtube.com/watch?v=v_y8VejlklQ


https://www.youtube.com/watch?v=wf1XBDAzMb0


VOF


https://www.youtube.com/watch?v=dGMN_OeuO24


https://www.youtube.com/watch?v=3NRh-uVJ4Ac

[arjun]

Future and what to expect

Immediate concern for me at the moment is to provide an explicit type of overset mesh solver. This seems like easy to implement and allows flexibility in simulations where simple sliding mesh is not possible to use (intermeshed gears or mixers for example).


Second user defined vertex motion is easy to implement and hence expect soon to be available (one or two design issues to be solved related to GGI behaviour in case user moves vertices).

FVUS supports power law method for viscosity so enhancements around it to be expected (other similar models plus temperature dependent models).
Viscoelastic flows: I am interested in developing a solver dedicated to it (Have in mind a specialized solver). Development of it depends really on me able to understand log formulation for better stability. I wish to directly have this in solver (so expect it at some time in 2017).

Eulerian Eulerian multiphase – This is next but at the moment no-one is asking so its development really depends on me getting time from other things.

[sbaffini]

Dear arjun,

i have a couple of questions, which i don't know if you like to be posted here (if this was in a blog it would have been better, maybe).

The first one is about the multi-region/multi-phase arrangement of the solver. I still don't have a large experience nor i have found yet the time to start my own "large" project from scratch so, last time i had to implement multi-physics features i had to accept a compromise (because of time constraints and very bad previous code). In practice, if, for example, solving for turbulence just in a grid block, using laminar (no turbulence model) in another and, say, solid in another one, i ended up allocating the turbulence variable(s) on the whole grid. Actually, the main reason was that i had no grid block at all to use and only thinking about the logic of parallel exchanges differentiated by variables and "ideal" blocks was a nightmare (temperature, for example, is likely to behave differently from turbulence variables when considering coupled blocks; the elctromagnetic fields too; velocity is midway, as using it in solids might have sense for moving reference frames; displacements only have sense in solids, etc. etc.).

Now, the first question is, how did you arrange the solver logic?

Obviously, if I could develop from scratch, my first tought would be to work on the object orientation, so that each grid block has its own solver and each solver its own variables. What i found conflicting in this view, is the location (with respect to the other objects) of the parallel communicator. In my view, this tends to be a feature of a grid block because, otherwise, i might have a (global) variable, say the temperature, going beyond the limits of a grid block, which then is in contradiction with it being a component of it trough the solver. To avoid this, in turn, would require partitioning all the blocks separately (not necessarily all of them with all the processors) which, to me, seems an enemy of the parallel scalability.
I'd like to hear your opinion (or anybody else, if allowed) on this.

The second question is quite more specific and, on your suggestions, I created a specific post here:

On the immersed boundary method

[FMDenaro]

Paolo, I think your questions about the treatment of the IB on the boundary is intriguing, so why do you not open a new post?

[arjun]

Originally Posted by FMDenaro

Paolo, I think your questions about the treatment of the IB on the boundary is intriguing, so why do you not open a new post?

Yes. It would be good idea.
I have lot to say on it. Previously when i worked in japan I spent almost 3 years with it and did tackle (solve? ) some of the issues with immersed boundary.
some patented, some did not

http://patents.justia.com/inventor/arjun-yadav

I would like to outline what direction it might go with wildkatze in future and how i aim to solve various issues with immersed boundary frame work.
(Actually i am working on an explicit overset and next appication where wildkatze is going to be used will have rigidbody motion, sliding interface , overset and immersed boundary working together.

@Paolo,
I just woke up so will reply later when i regained sanity. Thank you for asking here because it clears up for others too.

[arjun]

Originally Posted by sbaffini

Dear arjun,


The first one is about the multi-region/multi-phase arrangement of the solver. I still don't have a large experience nor i have found yet the time to start my own "large" project from scratch so, last time i had to implement multi-physics features i had to accept a compromise (because of time constraints and very bad previous code). In practice, if, for example, solving for turbulence just in a grid block, using laminar (no turbulence model) in another and, say, solid in another one, i ended up allocating the turbulence variable(s) on the whole grid. Actually, the main reason was that i had no grid block at all to use and only thinking about the logic of parallel exchanges differentiated by variables and "ideal" blocks was a nightmare (temperature, for example, is likely to behave differently from turbulence variables when considering coupled blocks; the elctromagnetic fields too; velocity is midway, as using it in solids might have sense for moving reference frames; displacements only have sense in solids, etc. etc.).

Now, the first question is, how did you arrange the solver logic?

You have pointed out some of the problems, there are lot more issues to be taken care of.


1. How to add physics models such that they are applicable to the region set and phase set they are defined on without disturbing the behaviour of other models along with it.
2. How each model can add boundary conditions or options as development progresses. How the user is informed of them. What happens to versioning as the newer versions comes or older versions go out of scope. What happens if user loads old file with newer version and that model or option no longer exists.
3. How the variables are created and stored. What happens to variable arrays when say more halo cells are created as a result of say ggi interface that needs to be updated as mesh motion.


For parallel
4. How the mesh be loaded and paritioned. User can chose to run it with 4 processor and then suddenly changes his mind to run with 8 processor. The method of running decomposition and creating partitioned meshes and data files sounds too cumbersome (openFOAM).
6. How the restart files are saved and loaded.


Programming


7. Easy to code and development time shall be short. Even compilation time matters.
8. Easy to test and how unit tests are added and performed.
9. Since the code is closed code, it shall not be an hindrance to user who wants to add advanced work or his research. For example, Fluent StarCCM provide very limited user coding and doing very advance work is very difficult. OpenFOAM is open and allows user to do anything he wants but its difficult to find out for new comers and it takes time for someone to adjust and understand. Easier way would be better.


These are some of the issues, there are lot more interesting things but for now. ...


What we want is a software that is Agile or fluid enough to incorporate the future changes such that we shall be able to add or remove features from it.

So how these issues are handled in Wildkatze. Next post about it.

[arjun]

First have a look at the gui for a typical set up. See attached picture.


As user defines sets of regions and sets of phases and we wish to apply physics models based on it with given control options and boundary conditions, the issue of storage crops up first for both various variables and various options of models.
This piece of code shall clearify how wildkatze handles this. This is code where volume fraction models (mixture multiphase and VOF calculate fluxes for them).

void
VolumeFractionModelEquation::
ComputeFluxTerm::
visitInterior     ( FVFaces      & fvFaces )
{
  PhaseContinuum* adPhase = _model.getAdvectionPhase();

  if (!adPhase)
    return;

  StorageAndMemoryManager * stManAd =
      getSimulation().getStorageAndMemoryManager( *adPhase, getFiniteVolume  ());

  StorageAndMemoryManager * stManBndAd =
      getSimulation().getStorageAndMemoryManager( *adPhase, getFiniteVolume  () , fvFaces);


  StorageAndMemoryManager * stMan =
      getSimulation().getStorageAndMemoryManager( *getPhase(), getFiniteVolume());

  StorageAndMemoryManager * stManBnd =
      getSimulation().getStorageAndMemoryManager( *getPhase(), getFiniteVolume() , fvFaces);


  double *density  =     stMan      ->getDoubleVar("density");
  double *densAdF  =     stManBndAd ->getDoubleVar("eff-density");
  double *flux     =     stManBnd   ->getDoubleVar("flux");
  double *fluxAd   =     stManBndAd ->getDoubleVar("flux");


  int const nfaces =     fvFaces.size        ();
  int      *CL     =     fvFaces.cellsLeft   ();
  int      *CR     =     fvFaces.cellsRight  ();

  for (int f=1; f <= nfaces; f++)
    {
      flux[f]    =    fluxAd[f]/densAdF[f];
    }

  // if there is a drif velocity present we add the drift too
  double * xVelDrift    =     stMan ->getDoubleVar("x-velocity-drift");
  double * yVelDrift    =     stMan ->getDoubleVar("y-velocity-drift");
  double * zVelDrift    =     stMan ->getDoubleVar("z-velocity-drift");

  if (!xVelDrift)
    return;

  for (int f=1; f <= nfaces; f++)
    {
      int const cl    =     CL[f];
      int const cr    =     CR[f];

      double const UatFace = 0.5 * (xVelDrift[cl] + xVelDrift[cr]);
      double const VatFace = 0.5 * (yVelDrift[cl] + yVelDrift[cr]);
      double const WatFace = 0.5 * (zVelDrift[cl] + zVelDrift[cr]);

      flux[f]  += (UatFace * ax[f] + VatFace * ay[f] + WatFace * az[f]);
    }

}

In this code you can see that before the variable is used, we ask the simulation object to give storage manager for the volume and phase on which we are working. For the bounaries storage manager is asked for region, boundary name and phase name.
From the storage manager one can then ask various variables by their title ie pressure, x-velocity, y-velocity etc.


Good thing about it is that in user coding you can follow the exact same thing. User has access to simulation object and he can access any region there from the name and any phase from its name (or any other element by their name).


So user in user coding have the same access as a developer has to the various things.


Another issue to handle is what happens to the linear solver and how descretization shall take place.


For that each region has its own matrix that is local to it. When descretization takes place it takes place on matrix local to the region. Which is done in a class that visits each region.


After the descretization is done local to each region, a final matrix is assembled for that region set.
So for each model a final linear system is assembled and destroyed once its use is done. It could be done efficiently because only GGI related things needs to be added otherwise it is just like giving different numbers to each equation.


The code for that looks like this.

void
SolverModel::
constructSolver(  int  blkSize,  MultiLevLinearSolver * mlSolver  )
{

  if ( mlSolver )
    {
      _mlSolver    =    mlSolver;
    }
  else
    {
      delete           _mlSolver;
      _mlSolver    =    new MultiLevLinearSolver();
    }

  LinearSystemEquation(   *_mlSolver
                       , (*getSimulation()), getFvRegionSubSet(), getCurrentPhase()
                       ,   "apName",      "srcVar",   "coVar"
                       ,   1.0, blkSize
                       )
                      .
                      construct  ();

}

Another issue to handle is how the options are handled. For this each model has ObjectOptions class that stores them as key value pair. It could be made clear from this piece of code where user under-relaxation factors are accessed for KLES model.

      double urf  =  0.7;
      double eurf =  1.0;
      double tmin =  1E-8;
      double tmax =  1E+12;

      ObjectOptions* objOptions = getPhaseOption ( getCurrentPhase()->getName() );

      LinearSolverSettings userSetting;

      userSetting.verbosity    =     0;
      userSetting.maxCycles    =     50;
      userSetting.convergeTol  =     1.0E-3;
      userSetting.groupSize    =     2;


      if (objOptions)
        {
          urf                     =    objOptions->getDValue("klesmodel-urf"         );
          eurf                    =    objOptions->getDValue("klesmodel-explicit-urf");

          SolverSettingProvider().acquireSettings      ( "klesmodel", objOptions,   userSetting);
        }

      solveLinearSystem(userSetting, urf);
      applyCorrection( tmin,  tmax, eurf );

About the parallel exchange, for each region there is an exchange class that handles exchange for that region.
The following piece of code does exchange and then constructs the variable at GGI inferface.

      {
        exchange                  ( "les-kinetic-energy",    getCurrentPhase()  );
        constructAtInterfaces     ( "les-kinetic-energy",    getCurrentPhase()  );
      }

This is given to base model class and thus available to each model that is created.

[arjun]

I would say that download the default version and play with windows version because it is easy to run as no installation is required.

For the very first run, go to the tutorial folder and copy the wildkatze.exe there and double click to run.

Then you would see console for wildkatze. Three to four commands to run that test case.

process-file run.txt
iterate 250
export-ensight test
exit

Thats all one need for very first run.

To see the set up, double click fvusgui from windows folder.
Right click simulation object to open .stree file.

Note that when you run wildkatze it generates a configuration file. The first menu item in simulation node in fvusgui is to load this configuration file. This contains information about the models etc for this wildkatze version. GUI does not know of models until user loads this.

So good idea is first to load this file then load the stree file. Please look into docs for more detail.

[arjun]

Originally Posted by sbaffini

In practice, if, for example, solving for turbulence just in a grid block, using laminar (no turbulence model) in another and, say, solid in another one, i ended up allocating the turbulence variable(s) on the whole grid.

To add on previous comments. To solve this particular issue, the phase in wildkatze is something that holds memory for variables pertaining to that particular field. That also acts as phase in the sense we understand. So if phase air and phase flow are defined. You can chose to attach air properties to air phase and you can chose to leave flow phase without any material.
All the variables that shall live inside this flow phase will be allocated from models on the region set they are working. For example flow model will allocate velocity etc in flow phase if flow model is defined on this phase. But it will only do on the regions that comes under the region set this flow model is defined on.

In this way there are lots of places where this variable is not defined or allocated.

To solve the issue of dependence of phase properties on other phases, I have introduces concept of 'Phase to Phases' where user can define a main phase and then dependent or related phases. It is like 1 to many mapping.
This is useful when you are defining say mixture. For example in VOF there is flow phase that has properties calculated from from air phase and water phase.
user can define flow to air and water phase (phase to phases). Then tell VOF model that treat this as mixture.

Now flow model is defined on flow phase and VOF model is defined on set of air and water. When flow wants properties of flow phase it asks its 'observers' to provide it. VOF model on listening to this request computes the mixture properties and returns.

The same is done by introducing concept of 1 to many mapping for regions so we also have Region to Regions. This is useful at many places. For example in overset i will define a background region and then oversetting regions.

I hope it makes sense. There are lots of interesting things introduced in FVUS that you usually don't see in other cfd solvers.

[arjun]

With FVUS, I have introduced another concept to tackle some of the advanced issues that sometimes come up.

Partial regions.

In FVUS one can create partial regions from a full region. It is actually just a marker that has value between 0 to 1 of length equal to number of control volumes in that region. 0 means that that control volume is inactive.

At the moment solid marking algorithm marks and creates partial regions but in future we could have these partial regions created from VOF marker volume fractions or other means (user defined or results of calculations etc).

The idea is that you could be able to solve different physics models on these partial regions. Now imagine that you have a phase that moves with volume fraction marker and you want to solve something only on the part of region where this marker is 1, then you can create this partial region and apply model there.

This concept is not completely working at the moment but slowly it will be built upon and improve. (Because of 1 obvious issue of boundary conditions to this partial region framework needs more contemplation).

This partial region however opens up possibilities for many types of simulations. For example imagine that a control volume is occupied with 5 different materials and these material affect the porosity and properties like density viscosity etc. Or may be reacting to each others. These materials could be moving during the simulation where their shape is not fixed. All this could be handled nicely under these partial region concept.

[arjun]

Started chemkin type project for FVUS, it is called firekatze for chemical reactions and combustion.


Developed ODE stiff equation solver based on VODE solver for firekatze:

3-species kinetics problem

At t = 4.0000e-01 y = 9.851704e-01 3.386364e-05 1.479570e-02
At t = 4.0000e+00 y = 9.051848e-01 2.236635e-05 9.479280e-02
At t = 4.0000e+01 y = 7.150360e-01 9.154910e-06 2.849548e-01
At t = 4.0000e+02 y = 4.499532e-01 3.215835e-06 5.500436e-01
At t = 4.0000e+03 y = 1.829592e-01 8.928058e-07 8.170399e-01
At t = 4.0000e+04 y = 3.892164e-02 1.619023e-07 9.610782e-01
At t = 4.0000e+05 y = 4.927679e-03 1.980703e-08 9.950723e-01
At t = 4.0000e+06 y = 5.156252e-04 2.063552e-09 9.994844e-01
At t = 4.0000e+07 y = 5.159072e-05 2.063734e-10 9.999484e-01
At t = 4.0000e+08 y = 5.200706e-06 2.080293e-11 9.999948e-01
At t = 4.0000e+09 y = 5.176931e-07 2.070773e-12 9.999995e-01
At t = 4.0000e+10 y = 5.201122e-08 2.080449e-13 9.999999e-01

Along with it, combustion model for FVUS is on it way.

[arjun]

Added explicit version of Navier Stokes solver to deal with small time step sizes.

Here is basic ignition test for combustion

8e-4 x 8e-4 m grid dimentions, filled with air at 20mPa at 700K. The center is heated till 2200K and then source is switched off.

the animation shows the behaviour of pressure in closed chamber.

https://youtu.be/bjjyFjWwR48

[arjun]

Attaching validation results for CH4 - Air auto ignition.
Initial temperature is 1500K and Pressure is 1Bar.

Mechanism use is GRI 3.0 mechanism from http://www.cerfacs.fr/cantera/mechanisms/meth.php

The mechanism contains 325 reactions with 53 species.

The image shows comparison of rersults of Wildkatze and Cantera.

Further validation are on with reacting flow, so for those who are interested would be soon able to use this model.

[arjun]

Link to video version of tutorial for H2 auto ignition case that shows how to set up reacting flow problem.

This video also gives some idea of how to use FVUS/wildkatze in video form.

https://youtu.be/IV7Fb1Qy33c

Note: Reacting flow is not in currently available version ( in soon to be released version).

[FMDenaro]

Hello,
What about the developing stage of dynamic LES modelling?