[Kissshot]

it's a simulation with 886w overset mesh and 253 bg mesh applying adaptive overset mesh refinement with 378 cores running.
when it ran about 4 hours in hpc, the output file is stuck in “re-partitioning” and no longer updates, even though the is running normally. If i kill it and restart it again, it also will stuck in about 4 hours.
what will cause this occur ? plz.

this is normal output:
TimeStep 8000: Time 1.560000e+00
40000 2.349517e+08 7.078061e+07 1.365317e+09 7.956696e+07 4.532384e+09 1.456996e+07 1.346901e-04
Mesh of the region bgFixed adapted.
Removing 12360 vertices from region bgFixed (11850 primaries)
Mesh of the region ov adapted.
Removing 17 vertices from region ov (13 primaries)
OVM 1 needs update

Updating two-level indirect region interface OVM1 between ov and bgFixed.

Re-partitioning
Initialization of star.segregatedflow.SegregatedFlowModel requires an additional pass...
Initialization of star.segregatedflow.SegregatedFlowSolver requires an additional pass...
Initialization of star.segregatedenergy.SegregatedFluidTemperatureMo del requires an additional pass...
Initialization of star.segregatedenergy.SegregatedEnergySolver requires an additional pass...
OVM 1 needs update

Updating two-level indirect region interface OVM 1 between ov and bgFixed.

40001 9.198809e+09 2.546202e+11 4.165964e+12 1.944885e+11 1.591007e+11 4.361700e+09 3.033863e-02

however, the output stuck at:
Re-partitioning

and there is no "Initialization of star.segregatedflow.SegregatedFlowModel..." and so on.

[FliegenderZirkus]

output file is stuck in “re-partitioning” and no longer updates, even though the is running normally

What makes you think that the simulation is running normally? Isn't it stuck? I doubt it's just the output file not updating. It can be many things but one of the first ones I'd check is memory consumption. It may be that your mesh adaptation creates so many cells that the sim no longer fits into memory. This can be checked by a Maximum memory report and monitor in Star-CCM+.

[Kissshot]

Originally Posted by FliegenderZirkus

What makes you think that the simulation is running normally? Isn't it stuck? I doubt it's just the output file not updating. It can be many things but one of the first ones I'd check is memory consumption. It may be that your mesh adaptation creates so many cells that the sim no longer fits into memory. This can be checked by a Maximum memory report and monitor in Star-CCM+.

Thanks a lot for your advice.
I will create a Maximum memory report and monitor to check memory consumption.

[theindianlel]

Originally Posted by FliegenderZirkus

What makes you think that the simulation is running normally? Isn't it stuck? I doubt it's just the output file not updating. It can be many things but one of the first ones I'd check is memory consumption. It may be that your mesh adaptation creates so many cells that the sim no longer fits into memory. This can be checked by a Maximum memory report and monitor in Star-CCM+.

Yeah, this could be the case. I had a similar experience recently when generating a mesh with large number of cells.

[Kissshot]

Originally Posted by FliegenderZirkus

What makes you think that the simulation is running normally? Isn't it stuck? I doubt it's just the output file not updating. It can be many things but one of the first ones I'd check is memory consumption. It may be that your mesh adaptation creates so many cells that the sim no longer fits into memory. This can be checked by a Maximum memory report and monitor in Star-CCM+.

I checked memory consumption. It increases to maximum memory of hpc as simulation runs, then the simulation stuck.
Is there any method to prevent memory comsumption from increasing to maximum value? For example, free the memory periodically?
Thank you in advance.

[Kissshot]

Originally Posted by theindianlel

Yeah, this could be the case. I had a similar experience recently when generating a mesh with large number of cells.

yes, the memory matters a lot. But, i think the memory should fluctuate around an average value, not keep increasing.

[FliegenderZirkus]

You'll need to share more details about your case and the cluster for us to be able to help further. What is the cell count and the reported memory consumption at start of the simulation and at the point when it freezes? How many HPC nodes do you use and what is the RAM installed on each of them?

[theindianlel]

Originally Posted by Kissshot

yes, the memory matters a lot. But, i think the memory should fluctuate around an average value, not keep increasing.

Yes. At one point the memory requirement keeps increasing until it reaches a maximum. I tracked the memory usage when some of my meshes were crashing. They crashed because of lack of memory. I am not aware of the entire meshing algorithm to tell you at what point this happens, though.

Another reason my meshes were crashing eventhough there was enough memory was because of my mpi version (I made sure star-ccm+ has access to more memory than it requires). This might not be useful to you but if someone comes across this thread with a similar issue, it might be helpful for them.