[Audrius]

Dear colleagues,

I need Your advices. I try to repeat experiment in which were investigated natural convection. For simulations, I use ANSYS Fluent 17.2 version and high performance computing cluster (SGI cluster with 240 parallel licences). I have high quality mesh, which consist of ~800'000 cells. For simulations, I use VOF explicit model. The time step size is variable (0.007-0.017 sec) and I need to simulate ~14400 sec. Even with this cluster and 240 parallel licences it is impregnable task. I tried to use NITA method, but then I use NITA residuals starts to diverge. The task running four days and at this moment is solved 1100 sec. What are Your advises?

Thanks.

[LuckyTran]

None. You have small time-steps and want a large simulation time. There are various ways to shave off some computational time here and then, like NITA, but what you need is a 10x-100x reduction.

[Audrius]

Dear LuckyTran,

could You explain that is 10x-100x reduction?

[LuckyTran]

Your problem is that you have small time-steps and need to solve a large time, so you need many time-steps. You need one or more orders of magnitude reduction in this problem. Other tricks will not give you this type of reduction. For example, your cell count is already low and doesn't receive much benefit from running on more parallel cores.

[Audrius]

Thank You.

[Audrius]

By the way, "I need one or more orders of magnitude reduction in this problem" You mean, that I must to set Spatial Discretization of pressure, momentum and etc. from First Order Upwind to higher order, for expample to Second Order Upwind? Or I must to do simplifications in my task? Thank You.

[juliom]

Sorry, but the only way you can overcome this issue is using an implicit approach. However, the implicit approach adds spurious modes. I would try to solve the problem from a dimensionaless approach. Therefore, you will not be attached to the exact domain size and instead you can solve a smaller model based on similarity approach.

[Audrius]

When I use implicit approach, I do not get convergence (10^-4), while in explicit approach everything is perfect at the same conditions. Here is domain: the box (height - 650 mm, width - 300 mm and lenght - 60 mm. Water level is 400 mm from the bottom, in the top of the part there is air (top wall - pressure outlet). In the bottom of the domain there is heater rod. And I need to analyze the phenomenon of natural convection. There is no possibility to make a simplifications in this domain.

[LuckyTran]

Originally Posted by juliom

Sorry, but the only way you can overcome this issue is using an implicit approach. However, the implicit approach adds spurious modes. I would try to solve the problem from a dimensionaless approach. Therefore, you will not be attached to the exact domain size and instead you can solve a smaller model based on similarity approach.

Going to dimensionless doesn't change anything. You still have x cells per length and some time delta relative to a flow time. Besides, Fluent is a dimensional solver anyway.

Originally Posted by Audrius

By the way, "I need one or more orders of magnitude reduction in this problem" You mean, that I must to set Spatial Discretization of pressure, momentum and etc. from First Order Upwind to higher order, for expample to Second Order Upwind? Or I must to do simplifications in my task? Thank You.

Discretization schemes add very little to the computational cost. Besides, going to second order upwind increases the cost, dose not decrease.

You need to find a way to take bigger time-steps & a way to simulate less time. Going to a coarser grid can help, because you can take larger time-steps. Why do you need such a long simulation time? Whatever the reason for that is, reduce it!

Originally Posted by Audrius

There is no possibility to make a simplifications in this domain.

Stubbornness won't get you anywhere! If you want to be stubborn and say I refuse to simplify my problem, then I say you are stuck with a long simulation.

[Audrius]

I need to simulate ~14400 sec because the experiment takes all this time period. The thermal power of the heater rod is small and water from low temperatures until boiling point increasing very slowly. Also I need to simulate water level changes in the pool, air moving above water, boiling and etc. And to compare my CFD results with experimental results from ISI publication.

I'm not stubborned, I thought about simplifications, also there is one possibility - to use Boussinesq case and don't use VOF model, but then the results will be far from reality. It just my opinion.

[juliom]

Sorry lucky train but I disagree with you. You can solve a smaller scale of the problem as long as you respect the dimensionless parameter such as Reynolds, Raleigh and so forth. However, the computational cost will be somehow similiae because the explicit formulation. Is restricted to the size of the smallest cell. If you want to reduce your domain and keep the same spect ratio then delta t will have the same order of magnitude. Increasing the size of the cell will help you but it will also depend on the turbulence model. Why don't you solve it using the steady state approach instead of the time dependent? My guess is that there are different options that you have to explore. Write taking a final decision.

[Audrius]

Dear juliom, in this case, I can't to solve steady-state, because I need to see how changing processes in this domain. For example, in the beginning, water temperature is 32 C, heater rod power 600 W, there is no heat losses through the walls except the top wall, which is pressure-outlet. In this case there is no heat balance and it means, that in steate-steat calculations I will see empty pool, because water will be evaporated. I still thinking that to do... I trying again to use Implicit solver. Also, I will to reduce size of cells near the heater rod.

[juliom]

I see, then It means that you are facing a very demanding computational problem. Your problem is more related to computation than fluid dynamics. My recommendation (again) is to reduce the magnitude of your problem. You said that you are using parallel computing and that is good. I usually write my own codes using MPI and OpenMP, because without parallel computing CFD is constrained to the computational resources.

[Audrius]

My high performance computing cluster make fast iterating, but the main problem is small time step.

[sina_mech]

I agree with LuckyTran, You don't receive any benefit of having high number of cores. The communication time between cores will adversely affect it. If you don't believe it, decrease the core numbers to 80 or even 60, and compare it!

About scaling the problem, you should consider the fact that dimensionless scaling will result in losing some details anyway, unless you have time/tools for calibration.

I would reduce the cell number to half of what it is now.

BTW, if you submit jobs using qsub, consider having less nodes, and higher ppn. That will give you higher speed! Why? because less communication between nodes! So for example if you wanna stick to 240 cores, go with something like

nodes=12:ppn=20

instead of

nodes=240:ppn=1

There is a big difference.

[Audrius]

Colleagues,

the problem isn't how many cores I use.. I can use 1, 12, 24 or 240... The problem is small time step, not number of cores used in this modelling.

You are right, I will try to reduce number of cells and if it not works, then I try to scale and so small domain. Thanks for the advices.

[Audrius]

So, the one way in this speed-up transient calculations is to reduce number of cells. Because time step is calculated by the smallest cell size.