[coffee_break]

I am involved in CFD as a part of my postgraduate thesis study. Since I have experimental results I am able to understand my CFD simulation is correct or not. Through this process, I have also learned that turbulence models or many other parameters affect the results tremendously (especially heat transfer). So, I wonder how CFD engineers in industry are sure that their simulation for complex systems is correct?

[sbaffini]

Just to be complete, let me mention that, in general, there are actually two aspects in this. They are typically summarised with the following two questions:

1) Am I solving the right equations? Which is your original question. We refer to this as validation.

2) Am I solving the equations right? Which is the other part of the story, but relevant as well. We refer to this as verification.

For what concerns the second question (I'll get back to the first in a moment), once you have picked up the models/equations you think are needed in your case, you can do two things to be sure that the software correctly solves the equations as promised.

The more general one is using the so called "Method of Manufactured Solution" (MMS). You might or not have an analytical solution for your set of equations, but you can put whatever function into the variables of your equations, work out all the derivatives, etc. If it was an actual analytical solution, then your equations will be identically satisfied (i.e., return 0 = 0), otherwise some source term S will be needed to make it happen.

In both cases now you have some functions (possibly with a source term S to insert into them) that must solve your equations within the promised accuracy of the solver. You then just solve the given problem with the given source term S (if any at all) and check that the error reduces as you expect when you reduce time or grid steps. The very nice thing of the MMS is that you can apply it to whatever geometry, grid and model equations, no matter how complex. So, if you are rigorous, a first step in solving a problem with a black box software would be to use the MMS on the exact geometry and set of equations you plan to solve. Note that, as you know the solution everywhere, you also know the b.c. everywhere, and you can convert them at wish between dirichlet or flux type.

I have to admit that, in practice, this is rarely done, if ever at all. However, I wanted to highlight this aspect because there is a lot of confusion on the topic in relation to open vs closed source codes. It is a scientifical bullshit that open source is better. What you need from a software is to know exactly what it solves and how it does it and methods to apply the MMS (i.e., user routines for source terms, initializations and boundary conditions). That is, even if you coded the software yourself, scientifically, you should apply the MMS on each different set of geometry/grid/bc/equations/settings.

Now let's say you already did MMS. This tells you nothing on your real problem (your actual boundary conditions and the resulting solution). The ideal is that you perform the computation with different time and grid steps and, if certain conditions are met, extrapolate the solution with Richardson. If such conditons are not met, but MMS worked great with the same grid and time steps, it means your actual problem requires finer ones. In this case you should go finer then. I have to admit again that, while more common than MMS, this is rarely done as well. People just pick up the best they can for the given computational resources and, if ever needed, just improve that in some place by compromising the solution somewhere else.

If everything worked as expected, you can be sure that you are solving the equations correctly. But, as you asked, were they the correct equations in the first place? Your only chance here is validation. You typically pick up a problem that has been experimentally tested (and data is available, of course) and is as close as possible to your original problem. Then, using the methods of the verification (it is indeed more common to use them for validation cases), you solve such problem with a set of equations that should, heuristically, give the best answer for such problem. If they don't, you either switch one or more models, you develop one (or more) yourself, or you just don't give an F if you can't do nothing. There are also a lot of peer reviewed articles on validation, so you typically don't do that at all, just pick up the same code and models of the article (probably just models if you can't switch code). If no realistic validation is possible at all, you just use heuristic and experience. Again, in reality, your mileage might vary here as well.

[aerosayan]

We don't. CFD is mostly numerical. Analytical and Experimental solutions are king.


In the most simplest terms, there are two sources of errors:

1. Solver is garbage, and the developers didn't validate it properly. But the salesman is good enough to get your university or employer to pay for it, and now you have to use it even though it gives wrong results.
2. User is not using the solver correctly, and using wrong models, discretization, turbulent schemes, timesteps, etc. This is most likely to happen.

While developing solvers, you validate them by comparing the results with analytical or experimental data. For example : I'm developing and validating my homebrewed supersonic-hypersonic compressible euler solver by running simulations on compression corners, and prandtl-meyer expansion corners meshes. Those cases have analytical solutions, and thus, I use them more than anything. If you validate your solver on multiple different analytical and experimental cases, then you can say that your solver is validated *but only for those cases*.


However, the user might be using the solver incorrectly, and thus they might get wrong results even though the solver is validated by the developers. This is the most realistic error that could occur. Thus, it's extremely important to use small scale experiments to validate your solutions, and progress step by step towards your final product. For example : You will do CFD on a small scale model of an aeroplane, and then do an experiment in wind tunnel, then go back to see if your CFD results are validated or not. If not, you fix your CFD model, then redo experiments until correct.


Experimental data is what matters at the end.

[Simbelmynë]

@aerosayan


I have a different view, in that there is no difference in hierarchy between a physical experiment and a numerical experiment. Both can be good and both can be garbage. In my research field it is impossible to conduct most physical experiments since there is no equipment that can manage it. Yet, it is possible to estimate behavior of the system to a reasonable degree of accuracy, using numerical experiments.


I think there are many analogs here. In most cases you purchase your experimental equipment (a microscope or a numerical code) which means that they are mostly black boxes. Poor settings in the numerical code will lead to poor results, as will poor settings in the experimental equipment and sample preparation (which in a sense is equal to setting a poor boundary condition in the numerical code).


The verification may be better with experimental equipment though (but not always the validation). Here I define verification as finding bugs in the code or in the equipment. Similar to what @sbaffini said, the verification part of any code/equipment requires a lot of testing and this points towards using the most popular codes rather than developing something (general purpose) yourself.


It is fully possible to use numerical experiments as validation for physical experiments, although it is not as common as the other way around.

[sbaffini]

Originally Posted by Simbelmynë

@aerosayan


I have a different view, in that there is no difference in hierarchy between a physical experiment and a numerical experiment. Both can be good and both can be garbage. In my research field it is impossible to conduct most physical experiments since there is no equipment that can manage it. Yet, it is possible to estimate behavior of the system to a reasonable degree of accuracy, using numerical experiments.


I think there are many analogs here. In most cases you purchase your experimental equipment (a microscope or a numerical code) which means that they are mostly black boxes. Poor settings in the numerical code will lead to poor results, as will poor settings in the experimental equipment and sample preparation (which in a sense is equal to setting a poor boundary condition in the numerical code).


The verification may be better with experimental equipment though (but not always the validation). Here I define verification as finding bugs in the code or in the equipment. Similar to what @sbaffini said, the verification part of any code/equipment requires a lot of testing and this points towards using the most popular codes rather than developing something (general purpose) yourself.


It is fully possible to use numerical experiments as validation for physical experiments, although it is not as common as the other way around.

Let me add that, when it comes to validation, either of a code with respect to an experiment or, as you mention, of an experiment with respect to a simulation, there is, typically, little scrutiny of the source of validation from the other side.

It is an experiment, it must be correct, thinks the numerical guy. It is a DNS, it must be correct, thinks the experimental guy. Hopefully they both are, but we tend to forget that they could not.

[Eifoehn4]

I would never trust a code framework, which does not at least has implemented:

• Code verification via MMS
• Regressiontests for each new feature
• Unittests for the most relevant routines
• A good documentation with various tutorials/examples

[JBeilke]

Most of the answers completely miss the original question:
"how would a CFD engineer in industry ..."

[sbaffini]

Originally Posted by JBeilke

Most of the answers completely miss the original question:
"how would a CFD engineer in industry ..."

Just summarizing my previous post:

Originally Posted by sbaffini

...
The more general one is using the so called "Method of Manufactured Solution" (MMS) ...

I have to admit that, in practice, this is rarely done, if ever at all ...

The ideal is that you perform the computation with different time and grid steps and, if certain conditions are met, extrapolate the solution with Richardson ...

I have to admit again that, while more common than MMS, this is rarely done as well. People just pick up the best they can for the given computational resources and, if ever needed, just improve that in some place by compromising the solution somewhere else ...

There are ... peer reviewed articles on validation, so you typically ... just pick up the same code and models of the article (probably just models if you can't switch code). If no realistic validation is possible at all, you just use heuristic and experience. Again, in reality, your mileage might vary here as well.

Maybe we can add that the CFD engineer in industry might end up with:

1) Whatever his company already uses or

2) Whatever the CFD code support team tells him to use

which, nonetheless, would still be based on points above (even if just heuristic and experience)

[LuckyTran]

Originally Posted by sbaffini

Just summarizing my previous post:



Maybe we can add that the CFD engineer in industry might end up with:

1) Whatever his company already uses or

2) Whatever the CFD code support team tells him to use

which, nonetheless, would still be based on points above (even if just heuristic and experience)

Can confirm that 1) and 2) occurs.


Also we should take a step back and remember that the vast majority of engineering analysis is still done with low order tools (0D and 1D models) on excel sheets. A flow network model is already very high tech.

Whatever the fancy 3D CFD fails to do, it's certainly still better at it than whatever crap I was using before.

That is to say, we might as well be discussing how engineers are sure of anything ever.

[piu58]

I made a bunch of calculations for industrial application. I knew in advance that my results would be use to change the process, mostly without additional experimental validation. So I had to ensure that the solution is correct within the given error level.

My advice:
- Make the simulation as simple as possible. I often simulate only "regions of interest" without meshing all of the area / technical installation
- Make the physics as simple as possible. Describe the effect of emitted variation (e.q. temperature dependency) in the discussion part of the result paper.
- Make a real good mesh. The physics dictates the mesh, not the geometry. Increase or decrease node density smoothly.
- If paramater variation is necessary (mostly it is) write a dedicated mesh generator to ensure the quality of the mesh and avoiding breaks in the node density.
Try to connect the simulation results with experimental data. That means calibrate the results against measuring available. Mostly they have something or can get something with less effort.
- Avoid 3D simulations. You get more accuracy from 2d results, perhaps with a correction term for 3d in comparison to an imperfect 3d solution.
- If possible, calculate with abstract (or rather without) coordinates. So the solution is valid for a whole class of problems.

All this makes only sense, if you know your tools. Simulation tools have pecularities, their usage need to be trained. There is no short cut here.

[JBeilke]

Originally Posted by piu58

- Avoid 3D simulations. You get more accuracy from 2d results, perhaps with a correction term for 3d in comparison to an imperfect 3d solution.

The first thing I learned in the world of industrial CFD was, that even a simplfied 3d geometry is much better than a 2d simulation for most of the cases.

One of the worst ongoing jokes in CFD is, that people do 2d steady state calculations and blame the turbulence models for the crappy results.