[sakurabogoda]

I am doing a parallel computing, MPICH, for steady state simulation (Particle tracking).

My problem is, I am using 50 nodes, but it shows CFX does not allocate all memory,

total used free shared buffers cached
Mem: 66095868 20154548 45941320 0 63272 8845636

The simulation speed is much lower and roughly takes 6hrs to complete 100 iterations.

It can be slow due to the increase max. iterations, distance and time in particle tracking. But it cannot be this much of delay, as I could run same without particles less than 20min before.

But why CFX does not allocate all memory? not at-least 50%?

Any advises please....

[ghorrocks]

I do not think the problem has anything to do with memory allocation. I think the slow solution is due to complex particle tracking over so many nodes. I would have a look at the particle tracking options and see what options are available to make multi-processor computing more efficient.

[sakurabogoda]

Dear Glenn,
Thanks a lot for the advice,

I have tried to find any particle tracking options or multiprocessing options to speed-up the process, but could not find it yet.

I am doing a cyclone separator flow and with, (still running steady state) and particle tracking options are,

PARTICLE INTEGRATION:
First Iteration for Particle Calculation = 20
Iteration Frequency = 20
Option = Forward Euler
END
PARTICLE TERMINATION CONTROL:
Maximum Number of Integration Steps = 2750000
Maximum Tracking Distance = 1000 [m]
Maximum Tracking Time = 600 [s]
with this control parameters also particles exceeded distance limit yet. I have seen CFX-post particle tracking and it showed particles are still inside the cyclone.

There are small particles which should exit domain, they are also still inside the domain.

My problem is with this situation,
For complete 100 iterations it takes 1.5 days in steady state. Then if I use LES for 2E4 iterations, it will take many many days.

Can anybody please inform me whether am I following a wrong way? This is my first time deal with particles. I am using 1000 particles only.

Thank you.

[ghorrocks]

You cannot infer transient simulation time from a steady state model. They are totally different, especially for particle tracking.

You probably have a few particles which are going into a recirculation and making very long tracks going around and around. This can be helped by shortening the maximum track length - as long as you can do ths without loosing accuracy.

[sakurabogoda]

Dear Glenn,
Thanks a lot.

I always refer steady state result files in cfx post (previously without particles) and I saw stream lines had the actual pattern. So I though particle tracks also should be fully developed in steady state to use as initial values for LES. That is why I increased PARTICLE TERMINATION CONTROL OPTIONS.

According to you, I can use default values for steady state without considering particle fate types. Am I correct?

My question is if a particle exceeded distance state, then it ignores for further simulation. Then, this continuous to transient and then also this particle can be neglected. Isn't it?

Thanks again.

[ghorrocks]

I do not think you understood my point. You statement:

For complete 100 iterations it takes 1.5 days in steady state. Then if I use LES for 2E4 iterations, it will take many many days.

Is not correct. You cannot predict transient run time from a steady state run.

But if you are going to do a LES model with particle tracking you can be sure it will take a long time. LES is always a very computationally intensive approach.

In a steady state model the maximum integration length is a tuneable parameter which you can adjust. Find the best setting for you with a sensitivity analysis. But it does not apply in transient runs, the particle just advances the small amount in a time step for a transient run.

[sakurabogoda]

Dear Glenn,
I got it. Thanks a lot.

I saw when I am using particle tracking the CPU allocation is very low (35%), especially the time it is writing "Particle Diagnostics"(about 20%).

Tasks: 555 total, 25 running, 530 sleeping, 0 stopped, 0 zombie
Cpu(s): 18.8%us, 81.0%sy, 0.0%ni, 0.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16320580k total, 5221256k used, 11099324k free, 133732k buffers
Swap: 6160376k total, 0k used, 6160376k free, 392528k cached

Without particle tracking simulation, it normally allocate more than 85% CPU.

If I increase the "Iteration Frequency" in Particle coupling control will it make any inaccuracy of results?

(It is, if there are 100 total iterations to run, set iteration frequency to 50[2 times] or 100[at the end of simulation])

[ghorrocks]

Have you tried a run with the particle tracking settings at their defaults? Most of the time the defaults are best.

[sakurabogoda]

I haven't check Result files for default case, as many particles exceeded fate limits,

| Particle 1um | Entered domain : 300 |
| | Left domain : 182 |
| | Exceeded integration limit : 118 |

So I increased "PARTICLE TERMINATION CONTROL" values.

I have attached Particle track for 1um (which should be left domain) for following conditions.

Maximum Number of Integration Steps = 2750000
Maximum Tracking Distance = 1000 [m]
Maximum Tracking Time = 600 [s]

It shows still particles are rotating inside the cyclone.

[sakurabogoda]

I have done steady state run for default parameters of particle termination control values. It took only 30min to finish run, but with many particles exceeded distance/time and integration limits.

Particle track image is attached herewith.

My problem is, if I use this file as initial conditions of LES, will results be inaccurate?

Any helps Pls..

Thank you.

[ghorrocks]

Just because many particles are triggering termination criteria does not mean you should increase it. Before you increase anything like that, think whether this is real. Should those particles exit somehow? The simulation is showing they stay trapped (and hence the very long paths). Your LES model may change things, in that case the only approach is to do a LES model and accept the issue on the RANS model.

You have to assess the accuracy, it depends on how accurate you want to be of course. Turn the effect on and off (or at least change it significantly) and see if it makes a difference.

[sakurabogoda]

Dear Glenn,
Thanks a lot. Yes I did Steady state for default values and that finished within 30min. That means time spent bz of increased values.

I changed my way and used steady state simulation results without particles as initial conditions to LES. Yes, as you mentioned LES still does not indicate any exceeded limits.

Now LES is running for default values of Particle termination control.

But I have another problem now.

In LES particle diagnostics give (@ time=0.668 s),

+--------------------------------------------------------------------+
| Particle 1um | Entered domain : 1 |
| | Continue from last time step : 844 |
| | Waiting for next time step : 845 |
+--------------------------------------------------------------------+
| Particle 10um | Entered domain : 1 |
| | Continue from last time step : 913 |
| | Waiting for next time step : 913 |
| | Exceeded integration limit : 1 |
+--------------------------------------------------------------------+

My problem is I am using particles, 100 of 1um and 300 of 10um, but this indicate more than that.

I have calculated number rate and mass flow as follows (density=2768 kgm^-3),

Considering 1um,

Mass of 1 particle =1.45E-15 Kg
No. of particles = 100
Total mass of particles = 1.45E-13 Kg
Total simulation time = 10 s
Particle no. rate = 10 ^s-1
Mass flow rate = (Total mass of particles/Total simulation time) = 1.45E-14 kgs^-1

Is there any wrong with my calculations? I cannot find it.

Could you please help me?

Thanks a lot again for your kind helps.

[ghorrocks]

Read the documentation about particle modelling. Using a model without knowing how it works is very dangerous.

CFX does not model individual particles. It uses the modelled particle as representative of a population of particles. So each particle represents a large number of real particles.