[JVJ]

Dear all


We are seeing variation in results when we run the same simulation in a workstation and a super computer. The difference even in the extend of one simulation predicting misfire and other predicting cycles with combustion. We have not just simulated one cycle we have simulated up-to 6th cycle. We are seeing this difference repeatedly.
Recently we have got a case where we are getting combustion on 32 core and not getting combustion in 16 core. When we contacted converge we were advised to use 32 core. We were not able to understand why this is happening. Is there anyone else facing similar issue?


With regards
Jubin V Jose & Hemant Thakur

IIT Madras
India

[nitesh.attal]

Hello Jubin and Hemant,


You are observing core dependency. The computational domain is made up of number of cell which are distributed in one case over the number of cores available on your workstation while in the other case on 32 cores. Furthermore, the numerical roundoffs on two different processors will be different which leads to small perturbations in the solution. Such perturbations are susceptible to grow over the course of the simulation and hence result into a different flow field, which inturn can lead to different combustion results. If your engine show cycle-to-cyle variation (in experiment), then CONVERGE is capable to capture them (small changes in flow field are bound to affect the combustion results in high CCV engines).


A sample size of 6 simulation cycles is not large enough to infer statistics from, it may so happen that the simulations on workstation and 32 cores if run long enough may result in statistically same results, say when run for say 20-40 cycles (Note that for high CCV engines, more engine cycles are need to be simulated to get a sample size that is large enough to gain confidence on the inferred population statistics, such as mean and COV).


If you wish to get repeatable results, then please use are repeatable solver setting (strong 1st order unwinding) so that such perturbations are damped before they can influence the flow field.



Please feel free to reach out to us to discuss this further.


Thanks,
Nitesh

Originally Posted by JVJ

Dear all


We are seeing variation in results when we run the same simulation in a workstation and a super computer. The difference even in the extend of one simulation predicting misfire and other predicting cycles with combustion. We have not just simulated one cycle we have simulated up-to 6th cycle. We are seeing this difference repeatedly.
Recently we have got a case where we are getting combustion on 32 core and not getting combustion in 16 core. When we contacted converge we were advised to use 32 core. We were not able to understand why this is happening. Is there anyone else facing similar issue?


With regards
Jubin V Jose & Hemant Thakur

IIT Madras
India

[JVJ]

Dear Mr. Nithesh


First of all, I really thank you for the very detailed explanation. We have also received a similar explanation from your Indian support team as well. This explanation gives us a clear idea of why these changes happen in the numerical simulation when we change the number of cores.



We using this predicted results for combustion chamber development and optimization. If the simulation is sensitive​ to the number of cores how confident we can be in the predicted results?
What are the best practice we can follow?



Also, I just posted this question here so as to bring these aspects into the notice of my peers working in the same area. Hope this helps others too.




--
With regards,
Jubin

[JVJ]

Dear Mr. Nithesh


I would like to bring to your notice that as per my understanding the reason for COV in an experiment and simulation are due to totally different reasons. So how confident we will be in the COV predicted?


In former it is due to all sort of uncertainty and lack of repeatability in all the engine processes repeating each cycle. In latter it is due to the solver used, numerical viscosity involved, probabilistic models used and other parameters involved in solving the equation.

[nitesh.attal]

Jubin,


The core dependency is a numerical artifact. The solution using any number of cores is still a valid solution.



Oversimplified -> Think of this a picking a set of colored balls from a bag, when you pick it using your left hand, you get a different sample (say 5 red 2 blue and 10 green) than when you pick using your right hand (say 5 blue 1 red and 11 green). But in each case you are getting a a ball, a valid solution.



What I am trying to illustrate is that the sequence of cycles you get will be different using different number of cores but each of individual solution will still be valid and for large enough sample size the two sets will be statistically same. In other words you will be getting the correct solution irrespective of the number of cores you use.


Now I agree that CCV in Engines can be caused from a lot of different factors and not all of them are accounted for in the simulation. But this does not mean that we will not see CCV in the simulation because some of the causes of CCV is included, such as resolving the flow, charge distribution, detail chemical kinetics, etc.



We have demonstrated that CONVERGE is capable to predict CCV and recommend you to read the following and similar papers.

############

Probst, Daniel M., et al. "Predicting Cycle-to-Cycle Variation With Concurrent Cycles in a Gasoline Direct Injected Engine With Large Eddy Simulations." Journal of Energy Resources Technology 142.4 (2020).
############



In summary, core dependency is a numerical but will still produce a valid set of cycles. For a large enough sample size, the statistics are invariant of number of cores used to generate the cycles. Finally, some (not all) of the causes of CCV in an experiment can be simulated and hence the simulation results will show CCV.



Thanks,
Nitesh

Originally Posted by JVJ

Dear Mr. Nithesh


I would like to bring to your notice that as per my understanding the reason for COV in an experiment and simulation are due to totally different reasons. So how confident we will be in the COV predicted?


In former it is due to all sort of uncertainty and lack of repeatability in all the engine processes repeating each cycle. In latter it is due to the solver used, numerical viscosity involved, probabilistic models used and other parameters involved in solving the equation.

[liang11]

Hello, what is repeatable solver setting, and how to set it? Thank you!

Originally Posted by nitesh.attal

Hello Jubin and Hemant,


You are observing core dependency. The computational domain is made up of number of cell which are distributed in one case over the number of cores available on your workstation while in the other case on 32 cores. Furthermore, the numerical roundoffs on two different processors will be different which leads to small perturbations in the solution. Such perturbations are susceptible to grow over the course of the simulation and hence result into a different flow field, which inturn can lead to different combustion results. If your engine show cycle-to-cyle variation (in experiment), then CONVERGE is capable to capture them (small changes in flow field are bound to affect the combustion results in high CCV engines).


A sample size of 6 simulation cycles is not large enough to infer statistics from, it may so happen that the simulations on workstation and 32 cores if run long enough may result in statistically same results, say when run for say 20-40 cycles (Note that for high CCV engines, more engine cycles are need to be simulated to get a sample size that is large enough to gain confidence on the inferred population statistics, such as mean and COV).


If you wish to get repeatable results, then please use are repeatable solver setting (strong 1st order unwinding) so that such perturbations are damped before they can influence the flow field.



Please feel free to reach out to us to discuss this further.


Thanks,
Nitesh