CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Software User Forums > OpenFOAM > OpenFOAM Running, Solving & CFD

OpenFoam logfile compression tip

Register Blogs Community New Posts Updated Threads Search

Like Tree4Likes
  • 4 Post By makaveli_lcf

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   April 23, 2009, 03:37
Default OpenFoam logfile compression tip
  #1
Senior Member
 
Dr. Alexander Vakhrushev
Join Date: Mar 2009
Posts: 256
Blog Entries: 1
Rep Power: 19
makaveli_lcf is on a distinguished road
Send a message via ICQ to makaveli_lcf
Hi all!

Making simulations in OpenFoam sometimes I get really huge log files. In some cases it can result in hard drive freespace running out! But all information can be useful afterwards and I cannot just redirect output to /dev/null.
So it is necessary to compress output on the fly. One solution in Linux OS is piping it to gzip/bzip2:
openfoamSolver 2>&1 | gzip > log.gz
But worst thing is that gzip blocks log.gz pipe for reading during simulation and one cannot get access to log data until task is finished! The solution is to use "compress" package. To obtain it in Ubuntu use
sudo apt-get install ncompress
Then it is possible
openfoamSolver 2>&1 | compress > log.Z
or make run script, containing
Code:
#!/bin/sh

checkMesh

if [ $? -eq 0 ]; then
    nohup openfoamSolver 2>&1 | compress > logfileName.Z &    
else
    exit -1
fi
Then you have a simulation running at background. Have a coffee break before coming back and analysing results!
__________________
Best regards,

Dr. Alexander VAKHRUSHEV

Christian Doppler Laboratory for "Metallurgical Applications of Magnetohydrodynamics"

Simulation and Modelling of Metallurgical Processes
Department of Metallurgy
University of Leoben

http://smmp.unileoben.ac.at
makaveli_lcf is offline   Reply With Quote

Old   January 18, 2011, 05:53
Default Continue with logfile
  #2
Member
 
José Rodrigues
Join Date: Jun 2010
Location: IN+/IST Lisbon
Posts: 53
Rep Power: 16
jose_rodrig is on a distinguished road
Hi! great post tip, btw..

My problem is a bit different though: I would like to know how it is possible to continue a logfile of a simulation that was interrupted and I wish to resume with the same logfile, without erasing the previous timesteps.

Thx

Jose
jose_rodrig is offline   Reply With Quote

Old   January 18, 2011, 06:35
Default
  #3
Senior Member
 
romant's Avatar
 
Roman Thiele
Join Date: Aug 2009
Location: Eindhoven, NL
Posts: 374
Rep Power: 21
romant is on a distinguished road
Quote:
Originally Posted by jose_rodrig View Post
My problem is a bit different though: I would like to know how it is possible to continue a logfile of a simulation that was interrupted and I wish to resume with the same logfile, without erasing the previous timesteps.
This can be done with
Code:
foamProgram >> logFile
the double error uses an already existing file. since the simulation will start from the last saved folder, you should delete everything that is written in the logfile after the folder save point.
__________________
~roman
romant is offline   Reply With Quote

Old   January 18, 2011, 06:39
Default
  #4
Member
 
José Rodrigues
Join Date: Jun 2010
Location: IN+/IST Lisbon
Posts: 53
Rep Power: 16
jose_rodrig is on a distinguished road
Great! many Thx

Regards
jose_rodrig is offline   Reply With Quote

Old   January 18, 2011, 10:15
Default Compress VTK files
  #5
Senior Member
 
Eelco van Vliet
Join Date: Mar 2009
Location: The Netherlands
Posts: 124
Rep Power: 19
eelcovv is on a distinguished road
Thanks for the compress tip!

In line with compression of data files, another question.
I commonly create output sample planes using the function object libfieldFunctionObjects.so. The problems is that the planes are only written in uncompressed ascii, so a lot of data is generated (especially for isocontour).
An example

Code:
PlanesVTK
{functions
(

      enabled   true;
      type surfaces;
      setFormat raw;
      surfaceFormat vtk;
      writeFormat binary;
      timePrecision 0;
      interpolationScheme cellPointFace;
      writeCompression compressed;
      functionObjectLibs ("libsampling.so");
      outputControl   timeStep;
      outputInterval  1;
      fields
      (
         magVorticity
      );
      surfaces
      (
         iso_magU0p5
          {
            type        isoSurfaceCell;
            isoField    magU;
            isoValue    0.8;
            interpolate true;
            regularise  false;
          }
      );
   } // END PlanesVTK is
);
creates asci data files of more then 40 MB, whereas I can compress it for almost 80%. The compression write option does not seem to work. Does somebody has an idea how to compress these kind of data output files during the run ?.
Thanks!

Regards,

Eelc
eelcovv is offline   Reply With Quote

Old   September 30, 2015, 09:14
Default
  #6
Member
 
matteo lombardi
Join Date: Apr 2009
Posts: 67
Rep Power: 17
matteoL is on a distinguished road
Hello,
have you had any luck exporting slices in vtk binary format?Thanks,
matteo
matteoL is offline   Reply With Quote

Old   September 1, 2016, 09:29
Default
  #7
Senior Member
 
Join Date: Mar 2015
Posts: 250
Rep Power: 12
KateEisenhower is on a distinguished road
Hello,

this is very interesting and may be of great helpt to me.
The problem is I don't really understand how you proceed with this compressed file. I mean normally one wants to extract data from this file during runtime (e.g. for plotting residuals). For doing this I need to extract the file and I am as far as before.
What am I missing here?

Best regards,

Kate

Quote:
Originally Posted by makaveli_lcf View Post
Hi all!

Making simulations in OpenFoam sometimes I get really huge log files. In some cases it can result in hard drive freespace running out! But all information can be useful afterwards and I cannot just redirect output to /dev/null.
So it is necessary to compress output on the fly. One solution in Linux OS is piping it to gzip/bzip2:
openfoamSolver 2>&1 | gzip > log.gz
But worst thing is that gzip blocks log.gz pipe for reading during simulation and one cannot get access to log data until task is finished! The solution is to use "compress" package. To obtain it in Ubuntu use
sudo apt-get install ncompress
Then it is possible
openfoamSolver 2>&1 | compress > log.Z
or make run script, containing
Code:
#!/bin/sh

checkMesh

if [ $? -eq 0 ]; then
    nohup openfoamSolver 2>&1 | compress > logfileName.Z &    
else
    exit -1
fi
Then you have a simulation running at background. Have a coffee break before coming back and analysing results!
KateEisenhower is offline   Reply With Quote

Old   September 11, 2017, 08:07
Default Edited the log file on a server
  #8
Member
 
Tarang
Join Date: Feb 2011
Location: Delhi, India
Posts: 47
Rep Power: 15
gtarang is on a distinguished road
Hello,
By mistake I have edited the solver.log output file in the running case. Now the case is running for 65 hours and it is scheduled to run for 300 hours more. The issue is solver.log file is not getting updated anymore. Although I am monitoring the residuals through residuals function object, but I want to see the number of PIMPLE iterations being used so that I can update my Courant Number. Can anyone suggest the way to restart the output log file?
gtarang is offline   Reply With Quote

Old   September 11, 2017, 11:18
Default
  #9
New Member
 
Oliver K
Join Date: May 2017
Posts: 15
Rep Power: 9
silencebreak is on a distinguished road
Quote:
Originally Posted by gtarang View Post
Hello,
By mistake I have edited the solver.log output file in the running case. Now the case is running for 65 hours and it is scheduled to run for 300 hours more. The issue is solver.log file is not getting updated anymore. Although I am monitoring the residuals through residuals function object, but I want to see the number of PIMPLE iterations being used so that I can update my Courant Number. Can anyone suggest the way to restart the output log file?
I would stop the simulation and restart the simulation with the latest timestep the log file says. So for example for time = 20.0 the log file is written so you delete all timesteps larger than 20 and restart the simulation. In controlDict you have to edit before restarting the simulation the startTime to latestTime.
silencebreak is offline   Reply With Quote

Reply

Tags
compression, logfile, openfoam


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
Superlinear speedup in OpenFOAM 13 msrinath80 OpenFOAM Running, Solving & CFD 18 March 3, 2015 06:36
64bitrhel5 OF installation instructions mirko OpenFOAM Installation 2 August 12, 2008 19:07
Adventure of fisrst openfoam installation on Ubuntu 710 jussi OpenFOAM Installation 0 April 24, 2008 15:25
OpenFOAM Debian packaging current status problems and TODOs oseen OpenFOAM Installation 9 August 26, 2007 14:50
OpenFOAM Training and Workshop Zagreb 2628Jan2006 hjasak OpenFOAM 1 February 2, 2006 22:07


All times are GMT -4. The time now is 16:00.