[francescomarra]

Dear all,

I suspect a bug in the procedure that determines the transfer of particles between processor zones: as reported in the figure attached, particles un-physically accumulates on processor patch boundaries near solid walls.
The case regards a parallel simulation in a cylindrical channel with a strong swirl that drives the particles injected near the axis towards the walls.
In the figure two processor zones to highlight the problem are also indicated: accumulation of particles clearly happens along the processor zone boundaries.

Any hint on this problem or the indication of the routines mostly involved will be greatly appreciated.

Thank you in advance,

Franco Marra

[mattijs]

Do you have a testcase you can post?

[francescomarra]

Dear Mattijs,

thank you so much for the so fast reply.

I do not know how exactly I can post the test case. This simulation is relatively big (about 220'000 cells, the same order of magnitude of particles) and is running with 64 processors.
Probably this is not a testcase to immediately reproduce the problem. However, I can certainly send the necessary files to restart the run from the time step the figure represents.
Otherwise I should try to reproduce the problem on a much smaller testcase.

I can just add that I already have done several runs with the same code, slightly modified to include a source term in the momentum equation to simulate a periodic channel with particles. Also running in parallel, this problem never arose, but in this case processor patch interfaces are always aligned with the coordinate directions.

Let me know if you like I send you the files necessary to initialize and run the case together with the last time step to immediately see the problem.

Thank you and best regards,

Franco

[mattijs]

Can you email it to me? m dot janssens at opencfd.co.uk

[andersking]

We are also having problems with particles getting 'stuck' at processor boundaries.

It seems to be worse when communication is across more than one node, as well as being affected by the grid size. We have also seen particles getting stuck well away from walls.

As an illustration, this shows the stuck particles (in blue), as well as some of the processor boundaries.

Any help appreciated.

Regards,
Andrew

[BlueyTheDog]

My name is Andrew Lowe and I'm working with Andrew King, the previous post, on this problem, in fact I was the one who found it in our work. If anyone has any questions regarding this programme feature with regards to our work, or would like me to test any proposed fixes, please feel free to contact me.

Andrew

[mattijs]

Dear Andrews,

If you could post / send to my email (see above) a simple testcase that replicates the problem we can have a look at it.

The problem could be interpolation, truncation errors, mesh concavity or any other detail.

Regards,

Mattijs

[niklas]

I'd also like your case.

If I run on a nice blockMesh-generated mesh, I cant reproduce the problem.

If I use my utility perturbPoints to make the mesh nastier, I am able to produce
particles that gets stuck and I think the problem is related with this, the mesh quality.

continuing to investigate...

[ambros]

Hi all, I'm Fiorenzo Ambrosino and I'm working with Franco Marra on the topic he posted previously.
In fact the problem of the stuck particles between processor zones is not present in the other case that Franco mentioned. In this latter case the mesh size is very small and we get a good mesh quality. In the case reported in the figure posted by Franco the mesh is very coarse.
So, for our experience in this kind of simulations, the problem could be related to the mesh quality.

Regards,
Fiorenzo

[andersking]

I will try to prepare a small test case that highlights the problems.

Andrew

[mattijs]

Any luck creating a testcase?

Niklas: any success?

[niklas]

Im re-writing the tracking completly. A new algorithm called 'incredibly inefficient but straightforward',
which is sort of similar to what the previous algorithm was with triangle-decomposition, since
its dependent on the inCell-functionality, which I also rewrote to handle all types of cells.

works like this.

while (move)
{
    if ( inCell(endPosition) )
        move to endPosition
    else
        move to the face found in inCell
        switch to new cell
}

but as usual its the fiddling of how to treat positions on the face.

So no success yet, but I can feel im close, hrmm.....

[andersking]

Hi Mattijs,

I've have made a test case and attached it. To build the mesh use the makemesh script (wouldn't fit within the 97kB limit otherwise). the decomposeParDict is set for 16 processors, but it should display the problem with 4, (or maybe two).

Looking at it more closely, the particles seem to get stuck where the processor boundary is stairstepped, as in the attached image.

Hope this can shed some light on the issue.

Cheers,
Andrew

[mattijs]

Hi Andrew,

got the case. What solver do I run?

[Edison_Ge]

Originally Posted by mattijs

Hi Andrew,

got the case. What solver do I run?

Run into same problem when trying to implement a PDF method in OF. Any progress with the fix?
My experience is that the more complicated the processorpatch are generated the more likely the stuck particle to happen.
I'm using the latest 1.7.1 and dying to know if anyone has found a solution.

Edison

[ambros]

Hi, looking at the code in detail I figured out that there were a problem on calculating the face that the particle has to cross.
Findface function use lambda function to evaluate which face has to be crossed. Lambda takes into account the wallImpactDistance, the distance between particle center and particle surface that impact with a wall (= particle radius for sphere particle). I've changed lambda for fixing two bugs: the first fix is for considering particle radius bigger than the minimal distance between cell center and a wall; the second for preventing particles to stuck on the wall or processor/cyclic patch.

For the first bug I've removed the use of cell center instead of starting particle position; This seems to have no bad consequences for planar cell faces.
For the second bug, that is our case, in lambda the application of wallImpactDistance is applied for all kind of patch (also processor, cyclic and so on) not only for the wall.
So I changed lambda to do:

    if (!cloud_.internalFace(facei))
    {
        scalar CCf = mag((from - Cf) & Sf);

        label patchi = patch(facei);
        const polyPatch& patch = mesh.boundaryMesh()[patchi];
        if (isA<wallPolyPatch>(patch))
        {
            if ( CCf > static_cast<const ParticleType&>(*this).wallImpactDistance(Sf) )
            {
                  Cf -= static_cast<const ParticleType&>(*this)
                   .wallImpactDistance(Sf)*Sf;
            }
        }
    }
    scalar lambdaNominator = (Cf - from) & Sf;
    scalar lambdaDenominator = (to - from) & Sf;

I no longer have these bugs.

Hope I was helpful.

Fiorenzo

[niklas]

Originally Posted by ambros

For the first bug I've removed the use of cell center instead of starting particle position; This seems to have no bad consequences for planar cell faces.

Ooooo, if you have done what I think, this is not good.
This means that if the particle is lost it will never find it again.

As you say, on planar cell faces this will be less likely, but (and this is a big but, for convex cells)
this is used to screen which faces to check, since if the particle is inside the cell, the path from cell
centre to end position will cross the same faces as the path from particle start position to end position,
but if it is outside the cell, or even on border, this will no longer be true.

so this can then be used as a check for lost particle, since if cell centre -> end says we have face hit
and particle start -> end says no face hit, we know we have a problem.

now, it will just continue to move the particle and you will not see a problem cause you cant find out.

N

[ambros]

Hi Niklas, you're right but if I 've all planar faces I can't loose particles (or am I wrong?). I understand that blockMesh utility makes all planar faces, at least the internals, so I think that it's not a big mistake for blockMesh generated meshes (again, I'm not sure so feel free to correct me if I'm wrong).
BUT (and I think this is a big one but), this way, I fix the problem of rebounding particles with radius bigger than distance between cellcenter-wall, say it d1, that have starting_position inside the border cell and distance from wall bigger than d1. In this way it is solved also the case of rebounding particle in whatever cell not near wall when the radius is bigger than distance between thatcellcenter-wall.

I'll reread your useful publication "Particle tracking in unstructured, arbitrary polyhedral meshes for use in CFD and molecular dynamics" for better understand your argument in case I messed up.

Fiorenzo