[vvqf]

dear all,

i am now solving a 2.5e6 cells case with parallel LES. I set the case as a normal non-parallel case, and did not use decomposePar utility.

it started running, but then stopped, and got 2 files:
mycase.decompout:

...
Nprocs : 1
Create time

Time = 0
Create mesh


Calculating distribution of cells
Selecting decompositionMethod metis


Finished decomposition in 10.26 s

Calculating original mesh data

Distributing cells to processors

Distributing faces to processors

Calculating processor boundary addressing

Distributing points to processors

Constructing processor meshes

Processor 0
Number of cells = 75600
Number of faces shared with processor 30 = 1730
Number of faces shared with processor 5 = 775
Number of faces shared with processor 1 = 2546
Number of faces shared with processor 28 = 309
Number of faces shared with processor 4 = 1343
Number of faces shared with processor 3 = 263
Number of faces shared with processor 2 = 2497
Number of boundary faces = 3065

Processor 1
Number of cells = 74647
...
...
Processor 31
Number of cells = 74255
Number of faces shared with processor 28 = 1979
Number of faces shared with processor 30 = 1844
Number of faces shared with processor 19 = 1637
Number of faces shared with processor 26 = 1033
Number of faces shared with processor 25 = 38
Number of boundary faces = 6027

Processor 0: field transfer
Processor 1: field transfer
...
...
Processor 32: field transfer
End.
...

Nprocs : 1
Create time
Create mesh
Time = 0

Reconstructing FV fields

Reconstructing volScalarFields

nuTilda
nuSgs
p
k

Reconstructing volVectorFields

U

Reconstructing volTensorFields

B

No tetFem fields

No lagrangian fields

End.

Removing single processor directories
\\end of file

mycase.solverout:

...
...
[31] Nprocs : 32
Create mesh, no clear-out for time = 0

Reading transportProperties

Reading field p

Reading field U

Reading/calculating face flux field phi

Selecting incompressible transport model Newtonian
Creating field Umean

Creating field R

Creating field Bmean

Creating field epsilonMean

Creating field pMean

Creating field pPrime2Mean

Reading probeLocations

Constructing probes

Starting time loop

Time = 5e-05

Mean and max Courant Numbers = 0.0520911 4.52888

\\end of file

Thank you for reading this long post, and
How can I get it run properly?

[eugene]

Hi VVqf,

I'm a bit confused. You say you didn't use decomposePar, but then you show the logs for decomposePar and reconstructPar and your LES code seems to be running in parallel : "[31] Nprocs : 32"

I take it your LES run is locking-up/crashing? First thing to do is to lower your timestep to 1e-5 or 5e-6. Then run the case in serial.

Really, I need more info to make any useful suggestions.

[vvqf]

Hi Eugene,

i forgot to mention, to run the case i used the command openfoam_hpc-1.2. I was told to do so by our administrator.
The whole command is:
openfoam_hpc-1.2 -np 32 oodles mycase

from the file, i guess, both decomposePar and mpirun are wrapped inside this command. I dont know the detail.
and the timestep 5e-05 is ok when running turbulent SST in CFX.

I will try you suggestion.
thank you.

[vvqf]

and before when I ran it in serial(on my own machine), I've got message as follows:


Creating field epsilonMean

Creating field pMean

Creating field pPrime2Mean

Reading probeLocations

Constructing probes

new cannot satisfy memory request.
This does not necessarily mean you have run out of virtual memory.
It could be due to a stack violation caused by e.g. bad use of pointers or an out of date shared library

Starting time loop

Time = 5e-05

Mean and max Courant Numbers = 0.0522919 4.52888
new cannot satisfy memory request.

[eugene]

"new cannot satisfy memory request" means you have run out of memory. I guess you're personal machine is 32bit? Then 2.6 million cells would be unlikely to run on it. You can try recompiling oodles with averaging disabled, might just allow you to run.

I have no idea what "openfoam_hpc-1.2" does, it must be something thats been cooked up by your sysadmins. As such it is impossible for me to comment on why it is failing.

CFX uses a coupled solver, which might be why you can run at higher CFL numbers. The problem is the PISO solver used by oodles in 1.2 will be unstable with central differencing above CFL ~ 0.9. That doesn't mean it will always blow up, but it is quite likely to.

To get more info from your runs when they fail add "2>&1" to your log pipe. This should redirect all error messages to the standard output and thence to your log. Ask your sysadmin about this for "openfoam_hpc-1.2". Perhaps you can get some more info this way, because what is currently available is simply not enough to make a diagnosis.