CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Main CFD Forum

Smallest binary file format to save large data

Register Blogs Community New Posts Updated Threads Search

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   June 2, 2008, 06:29
Default Smallest binary file format to save large data
  #1
Zonexo
Guest
 
Posts: n/a
Hi,

I need to save the u,v and p data output from my CFD code. I only 've limited space on my school's server so I need to conserve disk space.

Which data format allows me to read/write data with min size?

I'm currently using fortran binary. I hope the file size can go smaller. I'm also using tecplot to visualize. Its file size is smaller, but I can't read out the data (only write).

Is CGNS better? Btw, I'm using fortran and working in linux.

Thank you!
  Reply With Quote

Old   June 2, 2008, 20:50
Default Re: Smallest binary file format to save large data
  #2
Ananda Himansu
Guest
 
Posts: n/a
The fortran binary format (form='unformatted') you use already saves you much space over ASCII. The only further savings could come from compression algorithms. General binary data will not shrink much if compressed, unless your flowfield has contiguous regions of identical values (an unlikely possibility except for extensive freestream regions, or extensive no-slip surfaces, and even there you must store the velocity as block and pressure as a block, not cell-wise). The only savings in going from fortran binary (form='unformatted') to C binary (form='binary') is the removal of the end-of-record markers. This should normally be negligible savings.

I am guessing that the reason that the Tecplot files are smaller is that Tecplot may be saving the data in single precision versus the binary precision that you supply. Certainly, if your future use of the data does not require that you retain the full precision of your calculations so far, you can convert your data from binary double precision (64 bit?) to binary single-precision (32-bit?). This is a lossy conversion. You could alternatively shrink the data set by a more controllable lossy compression algorithm that gives you control over the precision of the output. Note that Tecplot supplies (in the tech support part of their website) the C code that will take your dataset in memory and write it out into the format that their Preplot utility creates. By studying the C code you can learn how to read from the format, and not just write in that format.
  Reply With Quote

Old   June 2, 2008, 21:25
Default Re: Smallest binary file format to save large data
  #3
Zonexo
Guest
 
Posts: n/a
Oh thanks Ananda!

I now understand why my tecplot file is smaller. I don't think it is necessary to store the values in double precision so guess I'll store my real(8) data in real(4). Then maybe I'll use 7z or rar to compress them.

  Reply With Quote

Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
paraview installation woes vex OpenFOAM Installation 15 January 30, 2011 08:11
Installation of OpenFOAM-1.6 on Ubuntu 9.10 marval OpenFOAM Installation 2 March 17, 2010 09:33
OF 1.6 | Ubuntu 9.10 (64bit) | GLIBCXX_3.4.11 not found piprus OpenFOAM Installation 22 February 25, 2010 14:43
input vertex data file format Rick FLUENT 3 March 6, 2006 03:39
error while compiling the USER Sub routine CFD user CFX 3 November 25, 2002 16:16


All times are GMT -4. The time now is 17:42.