[A_Pete]

Hi,

I was writing a small functionObject to monitor p and U values in selected cellSets. The sets are created with topoSet and then converted to cellZones with setsToZones. It does all run well in a serial simulation, but not if I try to run it in parallel using mpirun. It will then just think that the cellZones are empty.

What I did before, when having to handle cellZones, is that I executed the topoSet and setsToZones in every processor directory explicitly, which seemed to solve the problems. This time that did not solve the problem.

I have a guess that just one processor is looped over in my functionObject and not all of them, and that the processors don't communicate in this cellZone loop, as they would in a loop over a volume field.

const fvMesh& mesh(refCast<const fvMesh>(obr_));
const volScalarField& p(mesh.lookupObject<volScalarField>("p"));
const volVectorField& U(mesh.lookupObject<volVectorField>("U"));
const cellZoneMesh& cellZones(mesh.cellZones());

forAll(cellZones, zoneID)
        {
            const word& zoneName(cellZones.names()[zoneID]);
            const labelUList& cellZone(cellZones[zoneID]);

            scalar pSumme(0);
            scalar UXSumme(0);
            scalar UYSumme(0);
            scalar UZSumme(0);
            scalar count(0);

            forAll(cellZone, i)
            {
                pSumme += p[cellZone[i]];
                UXSumme += U[cellZone[i]].x();
                UYSumme += U[cellZone[i]].y();
                UZSumme += U[cellZone[i]].z();
                count++;
            }
            
            scalar pAvg = pSumme/(count+VSMALL);
            scalar UXAvg = UXSumme/(count+VSMALL);
            scalar UYAvg = UYSumme/(count+VSMALL);
            scalar UZAvg = UZSumme/(count+VSMALL);

            Info<< nl << zoneName << ": " << pAvg << tab 
                << UXAvg << tab << UYAvg << tab
                << UZAvg << nl << endl;
}

The problem is that my cellZone, which the parallel run gives me output for, seems to be empty and I did an "Info<< cellZone.size()" which verified that. So how can I access all the cellZones, when running in parallel.

The method I chose isn't the most elegant one, but I don't know how I can average in an easier way and I don't know if that even matters, because the cellZones can't be found.

Anyone having an idea?

[A_Pete]

Found a solution in an old thread in the forums:

Use the reduce() function to assure communication between the processors. In my case this has to be done on the averaged fields. An example is shown below:

reduce(pSumme, sumOp<scalar>());
reduce(count, sumOp<scalar>());

scalar pAvg = pSumme/(count+VSMALL);

That gave me the right values for an averaged p over all processors.

[yuhan1991]

Hi Pete,
Your solution really helps me a lot. Think you very much!