Very long time for saving the output (sim) file ...

Eike · December 7, 2011, 05:38

Hi,

everytime I use our cluster, it takes a very long time to save the XYZ@xyz.sim output file. Today my simulation finished at 4 am and now, 6 hours later the sim file is still 0 KB an the job is running. If I cancel the job, the file will be xyz GB and the next job will start. Btw, CPU usage at the cluster is 0% all the time.

Where should I search for the problem? At STAR-CCM+ or MS Windows HPC Server 2008R2? The STAR-CCM+ Support never heard about this problem ...

Best wishes
Eike

RobertB · December 7, 2011, 13:56

On stopping some jobs that I have been running interactively I have had the client be hundreds of iterations behind the server. Thus on stopping it can take a long time before it actually gives you back control (so that you can save for example). It slowly updates the client until it is at the same iteration as the server.

No idea why it only seems to happen on some models but it is annoying.

abdul099 · January 2, 2012, 20:40

When you cancel the job and there is an output file with a non-zero file size, it indicates, the file is just written slowly.
That might be due to hardware problems (like hard disks, network infrastucture etc) or due to software problems. Or due to utilisation of the system. Hard to say what causes the problem. Ever did a benchmark about disk performance? Especially when the disk is nearly full, the write performance drops dramatically.
Also did you ever check whether the system is swapping?

@RobertB: It is not recommended to stay connected to a server all the time when it's just running. That costs performance for updating the client - even when the client doesn't respond that fast, the server needs to send out the data. Best way is either to start simulations in batch mode or to disconnect when it runs. You still can connect from time to time to check whether it's still behaving like expected or not.

December 7, 2011, 05:38	Very long time for saving the output (sim) file ...	#1
Eike Member Eike Join Date: Sep 2011 Posts: 40 Rep Power: 15	Hi, everytime I use our cluster, it takes a very long time to save the XYZ@xyz.sim output file. Today my simulation finished at 4 am and now, 6 hours later the sim file is still 0 KB an the job is running. If I cancel the job, the file will be xyz GB and the next job will start. Btw, CPU usage at the cluster is 0% all the time. Where should I search for the problem? At STAR-CCM+ or MS Windows HPC Server 2008R2? The STAR-CCM+ Support never heard about this problem ... Best wishes Eike

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
OpenFOAM 1.7.1 installation problem on OpenSUSE 11.3	flakid	OpenFOAM Installation	16	December 28, 2010 09:48
1.7.x Environment Variables on Linux 10.04	rasma	OpenFOAM Installation	9	July 30, 2010 05:43
[blockMesh] BlockMesh FOAM warning	gaottino	OpenFOAM Meshing & Mesh Conversion	7	July 19, 2010 15:11
Version 15 on Mac OS X	gschaider	OpenFOAM Installation	113	December 2, 2009 11:23
DecomposePar links against liblamso0 with OpenMPI	jens_klostermann	OpenFOAM Bugs	11	June 28, 2007 18:51

December 7, 2011, 13:56		#2
RobertB Senior Member Robert Join Date: Jun 2010 Posts: 117 Rep Power: 17	On stopping some jobs that I have been running interactively I have had the client be hundreds of iterations behind the server. Thus on stopping it can take a long time before it actually gives you back control (so that you can save for example). It slowly updates the client until it is at the same iteration as the server. No idea why it only seems to happen on some models but it is annoying.

January 2, 2012, 20:40		#3
abdul099 Senior Member Join Date: Oct 2009 Location: Germany Posts: 636 Rep Power: 22	When you cancel the job and there is an output file with a non-zero file size, it indicates, the file is just written slowly. That might be due to hardware problems (like hard disks, network infrastucture etc) or due to software problems. Or due to utilisation of the system. Hard to say what causes the problem. Ever did a benchmark about disk performance? Especially when the disk is nearly full, the write performance drops dramatically. Also did you ever check whether the system is swapping? @RobertB: It is not recommended to stay connected to a server all the time when it's just running. That costs performance for updating the client - even when the client doesn't respond that fast, the server needs to send out the data. Best way is either to start simulations in batch mode or to disconnect when it runs. You still can connect from time to time to check whether it's still behaving like expected or not.