[egonalter]

Hi,

I have modeled a hollow cylinder (or pipe) using a quasi 2D mesh in CFX. The boundary conditions are sinusoidal mass flow and pressure using an opening boundary (the mass flow boundary is specified as a source term, because openings can't have a mass flow boundary for some reason). The boundary layer is resolved using 20 nodes.

During solution, I monitor the time averaged mass flow and pressure at the boundaries, as well as the enthalpy flow. These monitor points converge pretty fast in a few cycles as it seems. However, the time averaged mass flow never becomes zero as it should. It is in the order of 10^-6 kg/s, where I need at least 10^-9 kg/s (the AC amplitude is 10^-3 kg/s). I use double precision of course. Time discretization is 2000/cycle.

The inner loop convergence is set to 10^-6 (MAX) and the conservation target is set to 10^-4 (max 200 steps).

So either this solution (or better the time averaged flow field) is not converged yet (even if it looks like that), or I made something else wrong (numerical errors maybe). At least I ran out of ideas.

Any help is welcomed!

Best wishes

[ghorrocks]

You should check the sensitivities of all adjustable parameters. In your case try a tighter convergence criteria, finer time step and finer mesh and see if it makes a difference.

As you already have 1 in 1000 accuracy on the mass flow, that means you are close. But to get it more accurate you need to carefully check the simulation parameters.

[egonalter]

Hi ghorrocks,

thanks for your quick answer. With the current settings, I often run against the max inner loop limit (200) already because of the reduced conservation target and I don't want to increase the max inner loops further. I should take a closer look on the development conservation balance during the inner loops though.

To give a bit more detail, the tube is 4 mm radius and 80 mm long. The oscillation frequency is 50 Hz. In radial direction I have 200 µm mesh resolution and 30 µm resolution in the boundary layer. In axial direction I use 1 mm resolution. This could be a problem when the flow direction reverses maybe.

On the other hand, each cycle takes a day to calculate and I need a few cycles (maybe 20) to get a (time-averaged) stable flow field and another few seconds of simulated time so that the heat flow is also stable (I didn't calculated this yet). So I just don't have enough (real) time to do trail-and-error.

I'm not sure what you mean in 1 in 1000 accuracy but I'm still three orders of magnitude away from my goal (10^-6 to 10^-9), which I think is not close but looks more like "hard to reach" for me.

I currently use the solvers Time Integral function on
frequency*areaInt(density*v)@boundary*360
to monitor the time averaged mass flow. I also wrote a script to do the same (but at an arbitatary Plane) in CFD-Post and the results are equal. I don't think it makes sense to use a different weighting, e.g. Favre, which is hard to implement.

Thanks!

[ghorrocks]

I often run against the max inner loop limit (200)

Do you mean the coefficient loops? If so then you are running far too many coefficient loop iterations. This suggests your time step size is far too big. Hence my comment about doing a time step sensitivity study - have you done this?

I would recommend using adaptive time stepping, homing in on 3-5 coeff loops per iteration. Make sure the max and min time step size are wide enough that you never hit them. Then the solver will find its own time step size.

[egonalter]

Originally Posted by ghorrocks

Do you mean the coefficient loops? If so then you are running far too many coefficient loop iterations. This suggests your time step size is far too big. Hence my comment about doing a time step sensitivity study - have you done this?

I would recommend using adaptive time stepping, homing in on 3-5 coeff loops per iteration. Make sure the max and min time step size are wide enough that you never hit them. Then the solver will find its own time step size.

well, I first reduced the convergence target to very low values (10^-7 for max residual and 10^-6 for conservation) and see at which coefficient loop number the solution accuracy does not longer improve. I wanted to find out where the numerical limit is. As it turned out, I need ~ 120 loops to reach 10^-4% P-Mass Imbalance and 10^-7 max momentum residual, which are both the lower limit. But sometimes I need more loops.

My current timestep size is fixed to 0.02s/2000 (10 µs), but I will try to lower it to 1 µs and check its influence on the loops required as you said. I think 5 loops is a bit small to get very low residuals, even if the timestep size is small - isn't it?

I will wait for one cycle to finish to see if the lowered convergence targets will affect the solution (or the time-averaged mass flow) and continue with reduction of timestep size.

Thanks!

[ghorrocks]

My current timestep size is fixed to 0.02s/2000 (10 µs), but I will try to lower it to 1 µs and check its influence on the loops required as you said. I think 5 loops is a bit small to get very low residuals, even if the timestep size is small - isn't it?

Where did 10us come from? What about 1us? Unless you actually did something to show this is the correct time step size then I assure you it is wrong.

Will 3-5 coeff loops give a small time step? Yes. Will that be necessary in my simulation for accuracy? Yes, that is why I suggested it.

Choosing a time step size which is way too big is noobie mistake #2. I see it all the time. Mistake #1 is too coarse a mesh, by the way.

[egonalter]

Originally Posted by ghorrocks

Where did 10us come from? What about 1us? Unless you actually did something to show this is the correct time step size then I assure you it is wrong.

Will 3-5 coeff loops give a small time step? Yes. Will that be necessary in my simulation for accuracy? Yes, that is why I suggested it.

Choosing a time step size which is way too big is noobie mistake #2. I see it all the time. Mistake #1 is too coarse a mesh, by the way.

Ok. Reducing the convergence criteria didn't changed the solution. So I changed the time step size to adaptive (with max loop target 8). This gave a time step size of 1.5 ns.

This is a reduction of ~4 orders of magnitude while the loops only reduced by a factor of 20 (from 120 -> 5). Hence the real time to simulate 10 cycles increased from 10 days to 5000 days. I'm not sure if the lower time step size is needed all over a cycle though. But lets just assume this, then the simulation would "never" finish.

Regarding the mesh resolution, the flow speed is max 1m/s and the mesh density in flow direction is 1 mm, so the courant number is much lower than 1 in any case. On the other hand, I need the spacial resolution in order to show the acoustic streaming inside the tube.

So why is the required time step size so small? Are there any ways to increase it?

[ghorrocks]

Why did you use 8? Trust me, 3-5 coeff loops per iteration is best for most simulations. I have been using CFX for over 20 years so I do have a pretty good idea about how it works....

OK, now we are getting somewhere. Now your time step is close to correct, but your convergence and mesh are just guesses. You need to establish the three key tunable parameters required to get the accuracy you require. The three tunable parameters are mesh size, time step size and convergence criteria.

If you use adaptive time stepping homing in on 3-5 coeff loops per iteration that will automatically choose the correct time step, so that means you now have 2 tunable parameters. Also note that the correct time step size will vary with convergence tolerance and mesh size, so having the time step automatically determined for different meshes and convergence criteria significantly simplifies the exercise.

Now generate a mesh with half the edge length in all dimensions and a mesh with double the edge length in all dimensions. For a 3D hex model this means the mesh will have approximately 8x and 0.125x the node count. Now you can see if your mesh needs to go finer or can go coarser.

Once you have done that, have a look at your convergence criteria. You have already said a tighter tolerance does not change anything, but what about a looser criteria?

You are looking for the loosest criteria and coarsest mesh which gives you the accuracy you require.

Once you have done this, you know the settings to perform an accurate simulation. Now you look at your computer resources and see what you need to do to run this simulation in a reasonable time. This might mean buying more computers or licenses, distributed parallel or buying some time on a cloud cluster.

But you do not start with an a simulation run time in mind, and then adapt the simulation to fit that run time

[egonalter]

First, thanks for your persistent support, Glenn!

Originally Posted by ghorrocks

Why did you use 8? Trust me, 3-5 coeff loops per iteration is best for most simulations. I have been using CFX for over 20 years so I do have a pretty good idea about how it works....

ok reduced to 5.

Originally Posted by ghorrocks

OK, now we are getting somewhere. Now your time step is close to correct, but your convergence and mesh are just guesses. You need to establish the three key tunable parameters required to get the accuracy you require. The three tunable parameters are mesh size, time step size and convergence criteria.

If you use adaptive time stepping homing in on 3-5 coeff loops per iteration that will automatically choose the correct time step, so that means you now have 2 tunable parameters. Also note that the correct time step size will vary with convergence tolerance and mesh size, so having the time step automatically determined for different meshes and convergence criteria significantly simplifies the exercise.

As mentioned before, the time step size is now down to 1.5 ns, which makes it impossible to get a cycle-averaged solution (150 days per cycle calculation time), so I can no longer decide if the solution improves or not. I decreased the convergence criteria a bit, but still around 15 ns time-step which is too small to get a result in a reasonable time.

Originally Posted by ghorrocks

Now generate a mesh with half the edge length in all dimensions and a mesh with double the edge length in all dimensions. For a 3D hex model this means the mesh will have approximately 8x and 0.125x the node count. Now you can see if your mesh needs to go finer or can go coarser.

Impossible to get any results as mentioned above. Reducing the mesh density would hide the regions I'm interested in (boundary layer and acoustic streaming), so I think this is not an option. Increasing density would likely require even more computation time. What I could think of is the to use a denser mesh in regions where the solution only hardly converges, in the hope that the time-step size will increase.

Originally Posted by ghorrocks

Once you have done that, have a look at your convergence criteria. You have already said a tighter tolerance does not change anything, but what about a looser criteria?

You are looking for the loosest criteria and coarsest mesh which gives you the accuracy you require.

Originally Posted by ghorrocks

Once you have done this, you know the settings to perform an accurate simulation. Now you look at your computer resources and see what you need to do to run this simulation in a reasonable time. This might mean buying more computers or licenses, distributed parallel or buying some time on a cloud cluster.

But you do not start with an a simulation run time in mind, and then adapt the simulation to fit that run time

So this looks like to be a dead end. I just didn't expected that a model with only 4000 elements would require a supercomputer for calculation. I will try to identify mesh regions which fail to converge. Thanks for your help!

[Opaque]

Would you mind elaborating on the following:

I have modeled a hollow cylinder (or pipe) using a quasi 2D mesh in CFX. The boundary conditions are sinusoidal mass flow

The above should be no problem. Now the confusing part

and pressure using an opening boundary (the mass flow boundary is specified as a source term, because openings can't have a mass flow boundary for some reason).

An opening pressure boundary condition should no problem either, but what do you mean by "the mass flow boundary is specified as a source term" ?

Once you set the pressure opening, there is nothing else to set in the problem nor anything needed for "boundary condition closure" of the Navier-Stokes equation.

I think the crux of your problem is in the last sentence.

[egonalter]

Originally Posted by Opaque

An opening pressure boundary condition should no problem either, but what do you mean by "the mass flow boundary is specified as a source term" ?

Once you set the pressure opening, there is nothing else to set in the problem nor anything needed for "boundary condition closure" of the Navier-Stokes equation.

because the problem has an oscillating nature, both openings of the pipe need to have an "opening boundary". One side is pressure and the other side is mass flow. CFX cannot add a mass flow boundary condition on an "opening boundary" (just velocity). The problem is not isothermal (in fact, there is a temperature gradient), so I prefer mass flow over velocity.

In this case, there should be no time averaged mean mass flow through any plane perpendicular to the flow, which I take as a measure to check if the solution is valid.

edit: link to model attached

https://1drv.ms/u/s!AqGZdxHNGrTehi_Bci1aTYCCtsje

[ghorrocks]

So this looks like to be a dead end.

I would not give up so easily....

You have not said anything about your convergence criteria. Your convergence criteria sounds tight, and probably can be loosened. Do a sensitivity check and find out. And if you run adaptive timestepping then it will automatically find the time step size for you, and it will be much bigger as you loosen the tolerance. So you simulation will speed up a lot if you can loosen the tolerance.

Opaque's comment is important too - I am not sure you are driving the flow appropriately either. Can you describe the device you are trying to simulate? Also show an image of your CFD geometry.

[egonalter]

sorry, I just returned from holidays...

Originally Posted by ghorrocks

Opaque's comment is important too - I am not sure you are driving the flow appropriately either. Can you describe the device you are trying to simulate? Also show an image of your CFD geometry.

the device is a thermoacoustic buffer tube (or pulse tube). I like to see an effect called Raleigh streaming which occurs in acoustics due to the interaction of the sound field with a wall.

The acoustic enthalpy flow H2 through a pipe with is given by
H2=cp m1 T1,
where m1 and T1 are the mass flow and temperature oscillation amplitudes, in my case about H2 ~ 5 W time-averaged.

The Raleigh streaming is a time-average heat convection and can be calculated by
Hr = cp m20 T,
where m20 is the time-average mass flow through a small surface perpendicular to the acoustic flow. Small surface because the streaming changes its direction when going from the wall to the center of the pipe. The time-average mass flow through the total surface perpendicular to the acoustic flow should be zero of course.

I expect m20 to be in the order of 1e-6 kg/s (acoustic mass flow amplitude is 1e-3 kg/s) resulting in a convective heat flow of ~ 1 W (cp = 5200 J/kg-K), T=300 K). So I *think* I need a conservation target of at least 1e-6/1e-3 = 0.001, better 10 or 100 times lower.

For the residual target, a similar assumption can be made (u_raleigh/u_acoustic is also 1e-3, so a residual target of 1e-4 should be sufficient).

Using default residual (1e-4) and default conservation target (1e-2), the required time step size is 4.1e-7, which is still very low, but maybe ok to simulate a single period. The velocity imbalance shown in the monitor is then 1e-4%. I guess I can translate this to a relative error of 1e-6, which is good.

[ghorrocks]

Your estimates of the required imbalance and residuals do not make sense to me. Imbalance is how well overall conservation is achieved, and residuals are how well the equations are solved. These are different parameters to the acoustic and m20 mass flows you talk about.

The normal way of establishing effective convergence criteria is to do a series of runs with significantly different convergence criteria (eg 1e-3, 1e-4, 1e-5) and do a sample simulation which tells you the error you get in each. Then you select the largest error you are happy to accept.

You probably can loosen your criteria, but you will only know if you check.

[egonalter]

Originally Posted by ghorrocks

Your estimates of the required imbalance and residuals do not make sense to me. Imbalance is how well overall conservation is achieved, and residuals are how well the equations are solved. These are different parameters to the acoustic and m20 mass flows you talk about.

The CFX nomenclature is sometimes confusing. In CFX-Pre, you can set the "conservation target". As far as I understand it, this is the conservation error (what flows out - what flows into a specific volume) divided by the total flow, e.g. what flows in. So I assumed what flows in is 1e-3 kg/s and the maximum error I can tolerate must be smaller than the the Rayleigh flow I'm looking for 1e-6 kg/s, hence my estimation for the conservation target.

The imbalance shown in the monitor is given in "%", so I naively assume this must be the conservation multiplied by 100. Wrong?

Originally Posted by ghorrocks

The normal way of establishing effective convergence criteria is to do a series of runs with significantly different convergence criteria (eg 1e-3, 1e-4, 1e-5) and do a sample simulation which tells you the error you get in each. Then you select the largest error you are happy to accept.

You probably can loosen your criteria, but you will only know if you check.

Yes, I will do this, but each check takes a few days...

[ghorrocks]

The conservation/imbalances is global conservation. For mass, it is the sum of all mass flows in and out, mass sources and sinks and checks that they balance. Internal flows do not contribute, only flows at the boundaries (and anything which creates or destroys mass). Similarly for momentum and the other scalars.

But as this is global conservation, it has little to do with the magnitude of the internal flows. That is why I say you cannot use the imbalances parameter to judge accuracy on flows inside the domain.

The convergence checks are slow if you run the whole simulation. So normally you either run a smaller or simplified case, or a single cycle rather than the full thousand cycles; or something else to speed it up. Likewise for mesh density studies and all the other checks - you do them on smaller versions of the whole model, and when you know the correct parameters you apply it to the big model, so you don't have to run the big model too much.

[egonalter]

Originally Posted by ghorrocks

The conservation/imbalances is global conservation. For mass, it is the sum of all mass flows in and out, mass sources and sinks and checks that they balance. Internal flows do not contribute, only flows at the boundaries (and anything which creates or destroys mass). Similarly for momentum and the other scalars.

But as this is global conservation, it has little to do with the magnitude of the internal flows. That is why I say you cannot use the imbalances parameter to judge accuracy on flows inside the domain.

Ok, so this is only for the boundary flow, and Rayleigh streaming does not cross the boundary.

But this also means, that if I do the mass-flow time-averaging over a boundary (which should be zero), I can expect an error in the order of "conservation target" * mass flow, e.g. 0.01*1e-3 kg/s. I'm asking because this is an important check for me if the solution is correct.

The problem is that I require an absolute error in mass-flow across the boundary of < 1e-8 kg/s. This is because the enthalpy flow induced by this mass flow is 5200 J/kg-K * 1e-8 kg/s * 300 K = 16 mW, which is the error I'm willing to tolerate (< 1% of the acoustic enthalpy flow). This would require a conservation target of 1e-5 if I'm right.

Originally Posted by ghorrocks

The convergence checks are slow if you run the whole simulation. So normally you either run a smaller or simplified case, or a single cycle rather than the full thousand cycles; or something else to speed it up. Likewise for mesh density studies and all the other checks - you do them on smaller versions of the whole model, and when you know the correct parameters you apply it to the big model, so you don't have to run the big model too much.

Yes, I meant a few days calculation time for a single period (20 ms) and having ~12k nodes. I also planed the first 100 periods with a much coarser mesh without boundary layer (and/or larger time-step size) and use the high-resolution model only in the last periods.

[egonalter]

ok, the first solution with a 1e-4 residual and 1e-2 conservation (all RMS) is finished. The flow field shows many velocity waves along in the pipe, which is a hint for me that the solution is not accurate. I expect the velocity field to be smooth. So I'm going reduce the residual to 1e-5 and wait...

[ghorrocks]

This sounds like a good approach.

A question: Are the velocity waves just start up transient effects which disappear after the simulation runs for while or are they evidence of a poorly converged simulation? If the 1e-5 simulation also has them it suggests they are startup transients and are therefore real.

[egonalter]

Originally Posted by ghorrocks

A question: Are the velocity waves just start up transient effects which disappear after the simulation runs for while or are they evidence of a poorly converged simulation? If the 1e-5 simulation also has them it suggests they are startup transients and are therefore real.

I did not restart the simulation but just continued where I stopped it, so there is already an initial temperature/velocity field. I only ran a single period. It is possible that this is just a transient effect, but I don't think so. I have seen this effect month ago already, when I kept residual target the default (1e-4). We will know more in a week or so.

Can you please comment on my statements regarding the conservation target?

Again, many thanks for your kind help and guiding!