[xiaow_g]

Hi Bruno,
Thanks for your reply. I run snappyHexMesh in OpenFOAM-2.1.0 on Tianhe-1A, a supercomputer. So the memory shouldn't be a problem.
OMG! I will re-download a OpenFOAM-2.1.0 and try again, hope that it will work. Thank you again!
Best wishes!
Xiaow-g

Originally Posted by wyldckat

Greetings to all!

@xiaow_g: I know I've had a similar problem, where snappyHexMesh kept running at 100% in parallel and wouldn't continue, but I can't remember what the problem was. It might be a memory issue like Chris said, but it could also be a bug you might be triggering somehow.

It would be useful to know the Linux distribution and architecture (uname -m) you are using, as well how much RAM is available to your Linux installation (in case you are running it inside a virtual machine).

The other possibility is that you need to upgrade to OpenFOAM 2.1.x. Several bugs have been fixed since 2.1.0 was released and I vaguely remember having problems with snappyHexMesh in parallel with one versions that wasn't from git...

Best regards,
Bruno

[lovecraft22]

Hi, as I'm experiencing the same error (http://www.cfd-online.com/Forums/ope...ssorx-0-a.html) is there any way to set up a case from scratch in such a way that this error here is avoided without the need to run changeDict?

Thank you!

[levka]

Hello OF experts and users,
please assist me in struggling the following problem:

during run of icoFoam i included my utility:

....
        runTime.write();
    #include "utility.H";

        Info<< "ExecutionTime = " << runTime.elapsedCpuTime() << " s"
            << "  ClockTime = " << runTime.elapsedClockTime() << " s"
            << nl << endl;
......

"utility. H" consists of one loop, that prints values of U in cell centres along Y direction in the center of the channel (dimensions: x=0..6,Y=-1..1,Z=0..3; Nx=120,Ny=64,Nz=120):

scalar x,y,z;

forAll(mesh.cellCentres(),ii)
{
    x=y=z=0;
    x=mesh.cellCentres()[ii].x();
    y=mesh.cellCentres()[ii].y();
    z=mesh.cellCentres()[ii].z();

      if(x<3.10&&x>3.03&&z<1.58&&z>1.55)
    {
     Info<<"ii="<<ii<<" y="<<y<<" u="<<U[ii]<<endl;
    }

}

in single run output is perfect: 64 values of U

ii=315457 y=-0.996054 u=(-0.00620626 6.79366e-05 0.0108981)
ii=315585 y=-0.987859 u=(-0.00614572 -0.000243301 0.0243829)
ii=315713 y=-0.979032 u=(-0.00606576 -0.000949787 0.0380821)
ii=315841 y=-0.969524 u=(-0.00595958 -0.00210503 0.0521407)
ii=315969 y=-0.959283 u=(-0.00582333 -0.00378294 0.0664832)
ii=316097 y=-0.948252 u=(-0.00565297 -0.00606614 0.0810126)
ii=316225 y=-0.936371 u=(-0.00544439 -0.00904414 0.0956093)
ii=316353 y=-0.923574 u=(-0.00519362 -0.012812 0.110129)
ii=316481 y=-0.90979 u=(-0.00489682 -0.0174689 0.124403)
ii=316609 y=-0.894943 u=(-0.00455088 -0.0231161 0.13823)
ii=316737 y=-0.878951 u=(-0.00415365 -0.0298542 0.151383)
ii=316865 y=-0.861726 u=(-0.00370377 -0.0377802 0.163607)
ii=316993 y=-0.843174 u=(-0.00320252 -0.0469805 0.174613)
ii=317121 y=-0.82319 u=(-0.002653 -0.0575266 0.184092)
ii=317249 y=-0.801666 u=(-0.00206075 -0.0694652 0.191718)
ii=317377 y=-0.778482 u=(-0.00143554 -0.0828093 0.197153)
ii=317505 y=-0.753511 u=(-0.000789686 -0.0975293 0.200063)
ii=317633 y=-0.726614 u=(-0.000139452 -0.113541 0.200139)
ii=317761 y=-0.697644 u=(0.000496615 -0.130699 0.197118)
ii=317889 y=-0.66644 u=(0.00109615 -0.148783 0.190809)
ii=318017 y=-0.632829 u=(0.00163618 -0.167488 0.181132)
ii=318145 y=-0.596627 u=(0.00209476 -0.186432 0.168159)
ii=318273 y=-0.557634 u=(0.00245017 -0.205152 0.152146)
ii=318401 y=-0.515634 u=(0.00268635 -0.223128 0.133575)
ii=318529 y=-0.470396 u=(0.00279119 -0.239814 0.113173)
ii=318657 y=-0.42167 u=(0.00276001 -0.254686 0.0919138)
ii=318785 y=-0.369186 u=(0.002593 -0.267305 0.0709739)
ii=318913 y=-0.312656 u=(0.00229841 -0.277386 0.0516354)
ii=319041 y=-0.251767 u=(0.00188806 -0.284874 0.0351015)
ii=319169 y=-0.186184 u=(0.00137428 -0.289967 0.0222259)
ii=319297 y=-0.115543 u=(0.000771825 -0.293112 0.0131436)
ii=319425 y=-0.0394559 u=(0.000105613 -0.294848 0.0068923)
ii=839745 y=0.0394559 u=(-0.000582071 -0.295449 0.00134095)
ii=839873 y=0.115543 u=(-0.00122611 -0.294747 -0.00626435)
ii=840001 y=0.186184 u=(-0.00177453 -0.292263 -0.0177436)
ii=840129 y=0.251767 u=(-0.00220134 -0.287433 -0.0335546)
ii=840257 y=0.312656 u=(-0.00249681 -0.279854 -0.053115)
ii=840385 y=0.369186 u=(-0.00265327 -0.269405 -0.075247)
ii=840513 y=0.42167 u=(-0.00266883 -0.256237 -0.0985297)
ii=840641 y=0.470396 u=(-0.00254961 -0.240723 -0.121556)
ii=840769 y=0.515634 u=(-0.00230948 -0.223372 -0.143094)
ii=840897 y=0.557634 u=(-0.00196405 -0.204763 -0.162169)
ii=841025 y=0.596627 u=(-0.00153323 -0.185474 -0.178093)
ii=841153 y=0.632829 u=(-0.00103735 -0.166047 -0.190452)
ii=841281 y=0.66644 u=(-0.000497004 -0.146948 -0.199071)
ii=841409 y=0.697644 u=(7.11539e-05 -0.128564 -0.203973)
ii=841537 y=0.726614 u=(0.000650192 -0.111193 -0.205332)
ii=841665 y=0.753511 u=(0.00122649 -0.0950506 -0.203423)
ii=841793 y=0.778482 u=(0.00178946 -0.0802759 -0.198595)
ii=841921 y=0.801666 u=(0.00233001 -0.0669428 -0.191226)
ii=842049 y=0.82319 u=(0.0028412 -0.0550705 -0.181708)
ii=842177 y=0.843174 u=(0.00331821 -0.0446353 -0.170421)
ii=842305 y=0.861726 u=(0.00375808 -0.0355805 -0.15773)
ii=842433 y=0.878951 u=(0.00415926 -0.027827 -0.143969)
ii=842561 y=0.894943 u=(0.00452063 -0.0212799 -0.129439)
ii=842689 y=0.90979 u=(0.00484285 -0.0158363 -0.114408)
ii=842817 y=0.923574 u=(0.00512709 -0.0113894 -0.099104)
ii=842945 y=0.936371 u=(0.00537485 -0.00783297 -0.0837236)
ii=843073 y=0.948252 u=(0.00558836 -0.00506389 -0.0684298)
ii=843201 y=0.959283 u=(0.00576992 -0.00298402 -0.0533565)
ii=843329 y=0.969524 u=(0.00592204 -0.00150175 -0.0386111)
ii=843457 y=0.979032 u=(0.00604735 -0.00053254 -0.0242779)
ii=843585 y=0.987859 u=(0.00614837 -5.53222e-07 -0.0104258)
ii=843713 y=0.996054 u=(0.00622505 0.000147164 0.00250257)

But parrallel run for 6 proc outputs only 11 values!

ii=103398 y=-0.996054 u=(-0.00619963 6.84788e-05 0.0108932)
ii=103526 y=-0.987859 u=(-0.00613874 -0.000242565 0.0243827)
ii=103654 y=-0.979032 u=(-0.00605887 -0.000950786 0.0380804)
ii=103782 y=-0.969524 u=(-0.00595275 -0.00210967 0.0521367)
ii=103910 y=-0.959283 u=(-0.00581652 -0.00379168 0.0664762)
ii=104038 y=-0.948252 u=(-0.00564615 -0.00607851 0.0810015)
ii=104166 y=-0.936371 u=(-0.00543755 -0.00905979 0.0955933)
ii=104294 y=-0.923574 u=(-0.00518676 -0.0128303 0.110108)
ii=104422 y=-0.90979 u=(-0.00488997 -0.0174897 0.124375)
ii=104550 y=-0.894943 u=(-0.00454407 -0.0231361 0.138196)
ii=104678 y=-0.878951 u=(-0.00414708 -0.0298703 0.151337)

But i expect the same result as single run...Why it is not the same?

DecomposePar Dict the following:

numberOfSubdomains 6;

method          simple;

simpleCoeffs
{
    n               ( 1 6 1 );
    delta           0.00001;
}

hierarchicalCoeffs
{
    n               ( 1 2 2 );
    delta           0.001;
    order           xyz;
}

manualCoeffs
{
    dataFile        "cellDecomposition";
}

scotchCoeffs
{
}

Also the result is sensitive for the number of proc. i am using...

[wyldckat]

Greetings Lev,

It's quite simple:

• Info is for single process output on the master thread; or for each slave, when each slave is launched with it's own shell, which can be achieved by using foamJobDebug if I'm not mistaken.
  Run:

  foamJobDebug -help

  for more information.
• PInfo will gather all Info's and output on the master thread! (edit: apparently PInfo doesn't exist... I guess my memory isn't all that well...)

Best regards,
Bruno

[levka]

Hello, Bruno
unfortunately command "PInfo" give a compilation error as "PInfo was not declared in this scope ".
But the problem is not only at the screen output, but also i do not know how to order the code to analyze whole mesh that is spread to many processors (as is working with single run). And i need the results to be collected and to be written in the log file.
Like this command in single run

OFstream k(runTime.path()/"k.dat");
....
k<<"velocity "<<U[ii]<<endl;

Where can i read about this topics?

Regards
Lev

[wyldckat]

Hi Lev,

Sorry about that. Apparently PInfo doesn't exist. But I vaguely remember reading about it... weird

OK, by the description of what you're trying to do, it looks like to me that the probe or sampling utilities and/or function objects would be more suitable for what you want to do.
And then there is also "swak4Foam": http://openfoamwiki.net/index.php/Contrib/swak4Foam

Other than this... try searching this forum for more answers... I know they're in here somewhere...

Best regards,
Bruno

[wyldckat]

Greetings!

I saw just now the solution and remembered about this thread.

Originally Posted by wyldckat

Sorry about that. Apparently PInfo doesn't exist. But I vaguely remember reading about it... weird

Apparently my memory mangled the mnemonic incorrectly:

• It's not: Info + Pout - cout => PInfo
• It's: Info-Info + Parallel + cout => Pout

Kudos to mfmohdyasin:

Originally Posted by mfmohdyasin

Just figured out that I supposed to use "Pout" instead of "Info" to get output from each processor.

Best regards,
Bruno

[marco.müller]

Hi all,

I'm following up with the issue that decomposePar does not preserve some boundaries and later on the solver complains about missing boundary definitions.

Furthermore potentialFoam complains about the processor-boundaries.

My question is: Apart from the workarounds that have been constructed in the meanwhile to solve this: is there a out-of-box-solution for OpenFOAM 2.1.1 that I am using out of box on ubuntu???

my script looks like this (without reconstructing)

#!/bin/sh
# Source tutorial run functions
. $WM_PROJECT_DIR/bin/tools/RunFunctions

cp -r 0.org 0 > /dev/null 2>&1

runApplication blockMesh
cp system/decomposeParDict.simple system/decomposeParDict
runApplication decomposePar
runParallel snappyHexMesh 2 -overwrite -parallel

find . -type f -iname "*level*" -exec rm {} \;

ls -d processor* | xargs -i cp -r 0.org/* ./{}/0/ $1

runParallel potentialFoam 2 -initialiseUBCs -noFunctionObjects -writep

runParallel `getApplication` 2

Thanks very much!!
Marco

[wyldckat]

Greetings Marco,

Well... there are 2... 2.5 tricks that I know of:

1. The one used in the tutorial "incompressible/pisoFoam/les/motorBike/motorBike", which you seem to be basing your case on.
2. The one used in the case examples available here: http://code.google.com/p/bluecfd-sin...untimes202_211 - those rely on reconstructing the mesh and then decomposing all of it once again.
3. And the half-trick still relies on mesh reconstruction, but decomposes only the fields based on the existing decomposition - the following can be used to replace the tutorial "incompressible/pisoFoam/les/motorBike/motorBike/Allrun":
   

   #!/bin/sh
   cd ${0%/*} || exit 1    # run from this directory
   
   # Source tutorial run functions
   . $WM_PROJECT_DIR/bin/tools/RunFunctions
   
   # copy motorbike surface from resources folder
   cp $FOAM_TUTORIALS/resources/geometry/motorBike.obj.gz constant/triSurface/
   
   #make sure the write tolerance is upped for mesh reconstruction
   sed -i -e 's=writePrecision  6;=writePrecision  7;=' system/controlDict
   
   # Copy 0.org to 0
   cp -r 0.org 0
   
   runApplication blockMesh
   
   cp system/decomposeParDict.hierarchical system/decomposeParDict
   
   runApplication decomposePar
   mv log.decomposePar log.decomposePar.1
   
   cp system/decomposeParDict.ptscotch system/decomposeParDict
   
   runParallel snappyHexMesh 8 -overwrite -parallel
   
   find . -type f -iname "*level*" -exec rm {} \;
   
   #ls -d processor* | xargs -i cp -r 0.org/* ./{}/0/ $1
   
   runParallel renumberMesh 8 -overwrite
   
   #Need to reconstruct the mesh, in order to be able to re-decompose the fields
   runApplication reconstructParMesh -constant
   runApplication decomposePar -fields
   
   runParallel potentialFoam 8 -initialiseUBCs -noFunctionObjects
   
   runParallel `getApplication` 8
   
   # ----------------------------------------------------------------- end-of-file

Anything beyond this depends on how you defined your boundary condition patch names in the first place!

Best regards,
Bruno

[marco.müller]

Hey Bruno,

great help, thanks! I took version 2, which works fine.

Thanks
marco

[m.goudarzi]

Hi dear FOAMuser

I have 3 machin include 12 core. I used scotch method for 12 processor, its ok on 1 machine. but when I used 2 or 3 machine for more processor, it dosent work without any error or warning.

[wyldckat]

Originally Posted by m.goudarzi

I have 3 machin include 12 core. I used scotch method for 12 processor, its ok on 1 machine. but when I used 2 or 3 machine for more processor, it dosent work without any error or warning.

Quick request: Please provide more details, so that we can reproduce the same error. The details I need are as follows:

1. Which installation instructions did you follow for installing OpenFOAM?
2. Which version of OpenFOAM are you using?
3. Can you reproduce the same problem with one of OpenFOAM's tutorials?
   • For example, the tutorial "incompressible/simpleFoam/motorBike"?
4. Which are the exact commands you are using for launching the solver in parallel?

[m.goudarzi]

Hi dear Bruno
Thank you for your reply
actually i dont know how the OpenFOAM installed, This system with OpenFOAM has been gave me.
I use OpenFOAM-2.3.1.
I reproduce the problem with motor bike and have a same problem with scotch method.
when I using simple method ( for decompose) for cavity case, its work with 36 processor but didnt work with scotch method.
I try to run simple method for my 3D asymmetric diffuser but similar to scotch method it didnt work.
my command for run is:
mpirun -np 36 $HOME/OpenFOAM/OpenFOAM-2.3.1/foamExec pimpleFoam -parallel

Thanks
Marzieh

[m.goudarzi]

Originally Posted by wyldckat

Quick request: Please provide more details, so that we can reproduce the same error. The details I need are as follows:

1. Which installation instructions did you follow for installing OpenFOAM?
2. Which version of OpenFOAM are you using?
3. Can you reproduce the same problem with one of OpenFOAM's tutorials?
   • For example, the tutorial "incompressible/simpleFoam/motorBike"?
4. Which are the exact commands you are using for launching the solver in parallel?

In addition I use Gambit mesh for the geometry.

[wyldckat]

Quick answer: Sorry Marzieh, I've been having long work weeks, along with other responsibilities and I haven't managed during weekends to answer as many questions as I wanted here on the forum .

Let me refer you to this blog post of mine, which I forgot the other day to mention about it: Notes about running OpenFOAM in parallel

In addition, I forgot to ask the following questions:

1. What does the following command give you?
   

   mpirun --version

2. And what about this one?
   

   uname -a

[m.goudarzi]

Originally Posted by wyldckat

Quick answer: Sorry Marzieh, I've been having long work weeks, along with other responsibilities and I haven't managed during weekends to answer as many questions as I wanted here on the forum .

Let me refer you to this blog post of mine, which I forgot the other day to mention about it: Notes about running OpenFOAM in parallel

In addition, I forgot to ask the following questions:

1. What does the following command give you?
   

   mpirun --version

2. And what about this one?
   

   uname -a

Your welcome dear bruno. Thank you for taking the time to reply me.

1. mpirun version 1.8.2

2. Linux ubuntu00 3.13.0-35-generic #62~precise1-Ubuntu SMP Mon Aug 18 14:52:04 UTC 2014 x86_64 x86_64 GNU/Linux

[wyldckat]

Quick answer: I've search through my memories and tried to deduce the problem... and something doesn't add up.
This could be a problem with Open-MPI 1.8.2 or it could be due to a problem with incompatible MPI versions on each one of the 3 machines... but it's hard to prove that the issue is only reproducible if you decompose the mesh with scotch and run with pimpleFoam.

I need the following details:

1. What do the following commands give you?
   

   mpirun -np 36 $(which foamExec) which mpirun
   mpirun -np 36 $(which foamExec) mpirun --version
   mpirun -np 36 $(which foamExec) which pimpleFoam
   mpirun -np 36 $(which foamExec) wmakePrintBuild
   mpirun -np 36 $(which foamExec) ldd $(which pimpleFoam)
   mpirun -np 36 uname -a

2. What are the exact steps you take to prepare, mesh, decompose and run the case?
   • Because something seems to be missing or is being incorrectly used...

[Bashar]

Hi,
I am using openFoam 3.0.1 , and I am trying to run my case using the following steps:

$ decomposePar
its running ok.
$ mpirun –np 6 renumberMesh –overwrite -parallel

gives me the following error:

--------------------------------------------------------------------------
mpirun was unable to launch the specified application as it could not find an executable:

Executable: –np
Node: basharhpc

while attempting to start process rank 0.
--------------------------------------------------------------------------

I was using "mpirun -np" on the same machine with pimpleFoam with no problems but when I tried to use it with "renumberMesh" I get this error . This is the first time for me to use the "renumberMesh".

Bashar

[Persia]

Try -n instead of -np:

mpirun -n 6 renumberMesh -overwrite -parallel

[Bashar]

Originally Posted by Persia

Try -n instead of -np:

mpirun -n 6 renumberMesh -overwrite -parallel

Thanks alot for the help.
I just used

mpirun -n 6 renumberMesh -overwrite -parallel

and it works for me.My last question is that when I will run the case in parallel which one is more appropriate to use the -np 6 or the -n 6 , with mpirun? since I used the -n 6 , in renumberMesh?
Reagrds,
bashar