[chrisb2244]

So it seems like the dnsFoam packaged with OF-2.2.2 doesn't support parallel running. Initially, it looks like the kMesh class is the one which creates some problems, but presumably these could be solved with not too much effort?

My question is essentially - is there some other (deeper) reason why dnsFoam does not work in parallel? I wanted to ask before altering lots of files and ending up with many slightly edited classes, just to find that the solver is for some (as now unknown to me) reason completely incompatible with parallel solving.

If the problem is only with the generation (and distribution?) of a kMesh, then could someone perhaps advise me on whether I should use some sort of

if (Pstream::master())
{
    // create a kMesh, and distribute to the other processes
    // possibly with some sort of scatter(..) function?
}

or alternatively, try and generate a kMesh for each subsection of a mesh, and only allow 1D decomposition (which I think should be ok?)

I am looking at this so that I can attempt to simulate turbulent pipe flow, and I'm just unsure as to the best way to go about it.

Any help or suggestions would be appreciated.

[kmooney]

Have you tried running this on the newest OpenFOAM2.3.x release? The problem just just disappear for you.

[chrisb2244]

Dear Kyle,

Thank you for your suggestion - I spent yesterday getting OF-2.3.0 built and installed, so hopefully I can test dnsFoam again today.

In the case that it doesn't work as hoped, I did find another thread in this set of forums which had V&V studies for an 'ico_DNS' solver, which used dns with a gradP forcing, rather than the kMesh implementation used by OF-2.2.2's (and presumably other OFs') dnsFoam.

I'll post back here when I know if the changes to parallel processing and mesh handling change anything, but for others reading this, the ico_DNS solver is pretty useful:

http://www.cfd-online.com/Forums/ope...s-2-1-0-a.html

[kwardle]

Perhaps I am wrong, but I think it has been established here and elsewhere that the dnsFoam solver does not run in parallel due to the use of the fft for calculation of the turbulence forcing function. Others have correctly mentioned that DNS can be done for pressure driven flows (where turbulence forcing is not required) using other solvers, but this does not solve the problem if the goal is to simulate isotropic turbulence in a box---for which a parallel version of dnsFoam is needed unless some other turbulence forcing scheme can be used which does not rely on FFT (is there such a thing, even as an approximation to 'real' turbulence?). Alternatively, if an FFT library can be used which does not require a uniform 2^n mesh then perhaps the same forcing scheme can be employed and run in parallel.

So to my question, has anyone made any attempts to use the PFFT or PNFFT libraries available here https://www-user.tu-chemnitz.de/~mpi...ware.php#pnfft ? This would seem like a viable option and appears to be targeted to just this issue---calculation of FFT in parallel on irregularly spaced meshes (though perhaps still Cartesian, I haven't look close enough yet to determine the limitations of the algorithm).

Another possible alternative I can think of is to somehow compute the turbulence forcing FFT on a overlaid uniform mesh on the master node and then map this out to the processor meshes. This would perhaps create a bottleneck but might work OK if the cost of this step is a small fraction of the total computation.

Anyone out there tried to do something like either of these (PNFFT or overalid mesh hack)? I am trying to simulate droplet breakup dynamics for isotropic turbulence in a box using a DNSInterDyMFoam solver I have put together base on the turbulence forcing in dnsFoam. Unfortunately, I had not appreciated the serial limitation of the parent solver and am now trying to sort out how best to proceed...

[usama784]

Is there any update on whether dnsFoam can be parallelized or not?

[daron]

Originally Posted by kwardle

Perhaps I am wrong, but I think it has been established here and elsewhere that the dnsFoam solver does not run in parallel due to the use of the fft for calculation of the turbulence forcing function. Others have correctly mentioned that DNS can be done for pressure driven flows (where turbulence forcing is not required) using other solvers, but this does not solve the problem if the goal is to simulate isotropic turbulence in a box---for which a parallel version of dnsFoam is needed unless some other turbulence forcing scheme can be used which does not rely on FFT (is there such a thing, even as an approximation to 'real' turbulence?). Alternatively, if an FFT library can be used which does not require a uniform 2^n mesh then perhaps the same forcing scheme can be employed and run in parallel.

So to my question, has anyone made any attempts to use the PFFT or PNFFT libraries available here https://www-user.tu-chemnitz.de/~mpi...ware.php#pnfft ? This would seem like a viable option and appears to be targeted to just this issue---calculation of FFT in parallel on irregularly spaced meshes (though perhaps still Cartesian, I haven't look close enough yet to determine the limitations of the algorithm).

Another possible alternative I can think of is to somehow compute the turbulence forcing FFT on a overlaid uniform mesh on the master node and then map this out to the processor meshes. This would perhaps create a bottleneck but might work OK if the cost of this step is a small fraction of the total computation.

Anyone out there tried to do something like either of these (PNFFT or overalid mesh hack)? I am trying to simulate droplet breakup dynamics for isotropic turbulence in a box using a DNSInterDyMFoam solver I have put together base on the turbulence forcing in dnsFoam. Unfortunately, I had not appreciated the serial limitation of the parent solver and am now trying to sort out how best to proceed...

Dear kwardle,

did you find out any solution for parallelization of forced HIT by dnsFoam?