CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Programming & Development

Running in Parallel

Register Blogs Community New Posts Updated Threads Search

Like Tree3Likes
  • 1 Post By alexeym
  • 2 Post By alexeym

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 30, 2016, 20:47
Default Running in Parallel
  #1
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Hi All,

I am using scalarTransportFoam tutorial with some modification, i.e two PDEs with no flow. I have 3D domain 1 x 1 x 1 m^3 and I set internal mesh with initial value C=1 uniformly. When I run this solver on one processor, it works fine. But when I run it on 4 processors, I don't get the similar result. When i visualize in paraview, I observed that at time t=0 the value is C=1 everywhere (correct) but at the next time step say t=1 it turns to ZERO. I don't know why? Do I need to do something else before running in parallel. Application runs, but does not produce the correct result.

Do you have any suggestion?

Thanks.
cute is offline   Reply With Quote

Old   July 1, 2016, 03:47
Default
  #2
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
I fixed my previous problem by increasing allocated memory. By default it was 1G.

Now I am facing another problem. I divided my domain on 4 processors as (2 2 1). As initial setup, I placed biomass in processor 0, 1, and 2 leaving 3 as empty. I am expecting the biomass should spread in all directions and should occupy space in empty slot. But it is not. Please see figures (contour surface).

Why is it happening? and how to fix? Any suggestion.

Thanks.
Attached Images
File Type: jpg fig-1.jpg (63.2 KB, 33 views)
File Type: jpg fig-2.jpg (68.3 KB, 32 views)
cute is offline   Reply With Quote

Old   July 4, 2016, 03:32
Default
  #3
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Anyone has any suggestion. I am stuck.
Thanks.
cute is offline   Reply With Quote

Old   July 4, 2016, 06:01
Default
  #4
Senior Member
 
Hassan Kassem
Join Date: May 2010
Location: Germany
Posts: 242
Rep Power: 18
hk318i is on a distinguished road
Have you tried the same case on single processor? Just to confirm your expectations.
__________________
@HIKassem | HassanKassem.me
hk318i is offline   Reply With Quote

Old   July 4, 2016, 13:04
Default
  #5
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Yes, it is working good on single processor. I mentioned it in my first post.
cute is offline   Reply With Quote

Old   July 4, 2016, 14:00
Default
  #6
Senior Member
 
Hassan Kassem
Join Date: May 2010
Location: Germany
Posts: 242
Rep Power: 18
hk318i is on a distinguished road
I run quick few cases to try to reproduce the problem but I could not. Attached here a image for a case with zero velocity (no flow), pure diffusion. As you can see there is diffusion through the processor interface. Also I run ``laplacianFoam`` for the same case to double check and I get the same behaviour. Moreover, I tried to start with zero inside and one at the inlet and it worked as well. Probably we need to know more about your code and case setup to help.
Attached Images
File Type: png TField.png (18.3 KB, 15 views)
__________________
@HIKassem | HassanKassem.me
hk318i is offline   Reply With Quote

Old   July 4, 2016, 15:24
Default
  #7
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Thanks. Here is my decomposeParDict

Code:
FoamFile
{
    version     2.0;
    format      ascii;
    class       dictionary;
    location    "system";
    object      decomposeParDict;
}
// * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * //

numberOfSubdomains 4;

method          simple;
//method          scotch;

simpleCoeffs
{
    n               (2 2 1);
    delta           0.001;
}

hierarchicalCoeffs
{
    n               (2 2 1);
    delta           0.001;
    order           xyz;
}

manualCoeffs
{
    dataFile        "";
}

distributed     no;

roots           ( );
Let me know, what else you need to know.
cute is offline   Reply With Quote

Old   July 5, 2016, 04:31
Default
  #8
Senior Member
 
Hassan Kassem
Join Date: May 2010
Location: Germany
Posts: 242
Rep Power: 18
hk318i is on a distinguished road
``decomposeParDict`` looks fine. You may try to decompose in Z direction as well (2 1 2). It could give an indicator where is the problem come from. Are you currently using ``scalarTransport`` or your solver?
__________________
@HIKassem | HassanKassem.me
hk318i is offline   Reply With Quote

Old   July 5, 2016, 15:40
Default
  #9
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Thanks Hassan Kassem.

I used (2 1 2) partition, and still the same result. That is one of the region is completely empty. I am using my solver which is modified version of scalar transport equation. Here is my code snippet
Code:
while (simple.loop())
    {
        Info<< "Time = " << runTime.timeName() << nl << endl;

        while (simple.correctNonOrthogonal())
        {
            forAll(DX, cellI) {
                DX[cellI] = alpha.value()*X[cellI]*X[cellI];
            }
            
            solve(fvm::ddt(X) == fvm::laplacian(DX, X) + k*X);
            
        }

        runTime.write();
    }
cute is offline   Reply With Quote

Old   July 6, 2016, 03:38
Default
  #10
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,938
Rep Power: 38
alexeym has a spectacular aura aboutalexeym has a spectacular aura about
Send a message via Skype™ to alexeym
Hi,

1. What is wrong with DX = alpha*sqr(X) (or DX = alpha*X*X)? Why do you need loop?

2. Since you are using loop, DX values at processor boundaries are not updated. Try inserting DX.correctBoundaryConditions() after loop.
hk318i likes this.
alexeym is offline   Reply With Quote

Old   July 6, 2016, 03:48
Default
  #11
Senior Member
 
Hassan Kassem
Join Date: May 2010
Location: Germany
Posts: 242
Rep Power: 18
hk318i is on a distinguished road
I can't see the point of the loop as well but what worries me here why this behaviour is consistent even when he changes the decomposition. 🤔
__________________
@HIKassem | HassanKassem.me
hk318i is offline   Reply With Quote

Old   July 6, 2016, 07:55
Default
  #12
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Thanks for the reply.

(1) If I don't use loop, I get dimension problem. In my case DX is m^2/s, X is kg and alpha is dimensionless. If I use alpha*sqr(X), I get dimension problem, i.e. m^2/s = kg^2. The only way, I could figured out is to use loop. If you have some suggestion to fix this error, that would be easy solution.

(2) I tested DX.correctBoundaryConditions() on a quick run and it seems to worked out on initial testing.

(3) Since this solution works, I need to extend my model by adding another PDE. I don't know how to incorporate both equations. Is there any tutorial or test-case to solve coupled PDE? Please suggest. My preference is to add both discritized equation in a matrix and then solve. I don't want to solve using segregated way.
cute is offline   Reply With Quote

Old   July 6, 2016, 12:44
Default
  #13
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,938
Rep Power: 38
alexeym has a spectacular aura aboutalexeym has a spectacular aura about
Send a message via Skype™ to alexeym
1. Why can't you set dimensions of alpha to m2*s^-1*kg^-2? Why not use non-dimensioning coefficient X1 with dimensions kg and value 1 so you can calculate DX as alpha*sqr(X/X1)? If you still afraid of dimensions, you can use internalField() that is dimensionless, so DX.internalField() = alpha.value()*sqr(X.internalField()).

3. As you can see, nobody wants to answer "abstract" questions. Solution to your parallel run problem was proposed as soon as you posted your snippet. Post your equations.
hk318i and cute like this.
alexeym is offline   Reply With Quote

Old   July 6, 2016, 14:50
Default
  #14
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Thanks, I will try your solution and I did not think that way (feeling stupid).

Here are my equations that I want to solve:

ddt(C) = laplacian(d, C) - alpha*F(C)*X
ddt(X) = laplacian(DX, X) + F(C)*X - beta*X

where F(C) = C/(0.5+C) and d, alpha and beta are some real numbers.
cute is offline   Reply With Quote

Old   July 7, 2016, 18:46
Default
  #15
Senior Member
 
Alexey Matveichev
Join Date: Aug 2011
Location: Nancy, France
Posts: 1,938
Rep Power: 38
alexeym has a spectacular aura aboutalexeym has a spectacular aura about
Send a message via Skype™ to alexeym
Well, guess, if you would like to solve your equations in coupled manner, you have to write them in coupled manner, i.e. q = ||C, X||, and then write your equations in terms of q. Then you introduce volVector2DField and solve equation as usual, or just use volVectorField assuming third component as 0. For solution of vector equations, you can refer to any UEqn.H in incompressible solvers.

If you would like someone to rewrite your equations in vector form (and then post corresponding code), you have to explicitly ask for this. Also I think separate post could attract more interested persons.
alexeym is offline   Reply With Quote

Old   July 7, 2016, 19:55
Default
  #16
Member
 
anonymous
Join Date: Mar 2016
Location: Canada
Posts: 93
Rep Power: 10
cute is on a distinguished road
Thanks, I will post it separately.
cute is offline   Reply With Quote

Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
error while running in parallel using openmpi on local mc 6 processors suryawanshi_nitin OpenFOAM 10 February 22, 2017 21:33
Running mapFields with Parallel Source and Parallel Target RDanks OpenFOAM Pre-Processing 4 August 2, 2016 05:24
running OpenFoam in parallel vishwa OpenFOAM Running, Solving & CFD 22 August 2, 2015 08:53
Unconsistent parallel jobs running time arnaud6 OpenFOAM Running, Solving & CFD 4 February 10, 2015 12:42
Problems running in parallel - missing controlDict Argen OpenFOAM Running, Solving & CFD 4 June 7, 2012 03:50


All times are GMT -4. The time now is 05:04.