|
[Sponsors] |
Smallest binary file format to save large data |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
June 2, 2008, 06:29 |
Smallest binary file format to save large data
|
#1 |
Guest
Posts: n/a
|
Hi,
I need to save the u,v and p data output from my CFD code. I only 've limited space on my school's server so I need to conserve disk space. Which data format allows me to read/write data with min size? I'm currently using fortran binary. I hope the file size can go smaller. I'm also using tecplot to visualize. Its file size is smaller, but I can't read out the data (only write). Is CGNS better? Btw, I'm using fortran and working in linux. Thank you! |
|
June 2, 2008, 20:50 |
Re: Smallest binary file format to save large data
|
#2 |
Guest
Posts: n/a
|
The fortran binary format (form='unformatted') you use already saves you much space over ASCII. The only further savings could come from compression algorithms. General binary data will not shrink much if compressed, unless your flowfield has contiguous regions of identical values (an unlikely possibility except for extensive freestream regions, or extensive no-slip surfaces, and even there you must store the velocity as block and pressure as a block, not cell-wise). The only savings in going from fortran binary (form='unformatted') to C binary (form='binary') is the removal of the end-of-record markers. This should normally be negligible savings.
I am guessing that the reason that the Tecplot files are smaller is that Tecplot may be saving the data in single precision versus the binary precision that you supply. Certainly, if your future use of the data does not require that you retain the full precision of your calculations so far, you can convert your data from binary double precision (64 bit?) to binary single-precision (32-bit?). This is a lossy conversion. You could alternatively shrink the data set by a more controllable lossy compression algorithm that gives you control over the precision of the output. Note that Tecplot supplies (in the tech support part of their website) the C code that will take your dataset in memory and write it out into the format that their Preplot utility creates. By studying the C code you can learn how to read from the format, and not just write in that format. |
|
June 2, 2008, 21:25 |
Re: Smallest binary file format to save large data
|
#3 |
Guest
Posts: n/a
|
Oh thanks Ananda!
I now understand why my tecplot file is smaller. I don't think it is necessary to store the values in double precision so guess I'll store my real(8) data in real(4). Then maybe I'll use 7z or rar to compress them. |
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
paraview installation woes | vex | OpenFOAM Installation | 15 | January 30, 2011 08:11 |
Installation of OpenFOAM-1.6 on Ubuntu 9.10 | marval | OpenFOAM Installation | 2 | March 17, 2010 09:33 |
OF 1.6 | Ubuntu 9.10 (64bit) | GLIBCXX_3.4.11 not found | piprus | OpenFOAM Installation | 22 | February 25, 2010 14:43 |
input vertex data file format | Rick | FLUENT | 3 | March 6, 2006 03:39 |
error while compiling the USER Sub routine | CFD user | CFX | 3 | November 25, 2002 16:16 |