Binary gives significant performance advantage (Mesh & Solve)
Posted April 28, 2017 at 06:08 by kindle
Well, binary or ascii. Haha
Quote:
Originally Posted by wyldckat
Greetings to all!
OK, only after 6 months did I manage to find enough time to run some tests of my own on a stable machine (mine isn't as stable 
).
The results are now available here: https://github.com/wyldckat/wyldckat...nce_Analysis_2
The summary results:
Want more performance out of OpenFOAM? Then please suggest proven file storage software/technology, instead of simply complaining about it
... And I'll go first
: LZO and LZ4 are high speed compression algorithms that offer impressive compression/decompression speeds, where the decompression is almost as fast as memcpy. These compression algorithms don't offer as much compression ratios as gzip and bz2, but for most cases, they could offer improved throughput to disk for writing such files when the data is in binary format.
Best regards,
Bruno
OK, only after 6 months did I manage to find enough time to run some tests of my own on a stable machine (mine isn't as stable ).The results are now available here: https://github.com/wyldckat/wyldckat...nce_Analysis_2
The summary results:
- If no file is written, it doesn't matter if it's configured to write in "binary" or "ascii".
- Example timings of running 200 iterations with simpleFoam and writing every 10 iterations:
- ascii precision 12: 184.65 s
- binary: 167.26 s
- Example timings of running 50 iterations with simpleFoam and writing every iteration:
- ascii precision 12: 86.62 s
- binary: 42.7 s
- Writing to disk did not affect performance, since memory cache came into play to assist in writing as fast as possible, due to the case not being all that big. We're talking about 3GB in files versus a machine with 24GB of RAM... hurray for memory cache
- Then comes the 6th diagnosis approach I took, where I wrote my own simplistic codes in C++ to write a double array of 5000000 (5 million) values to file in ascii and binary. Results:
- ascii: 3.102s - 52180111 (byte)
- ascii_sprintf_f: 3.911s - 69767509 (byte)
- ascii_sprintf_g: 2.609s - 52180111 (byte)
- binary: 0.142s - 40000000 (byte)
- And here's the kick to the nuts, if we run gzip on the resulting files for these 4 tests:
- ascii: 1.062s - 2452385 (byte)
- ascii_sprintf_f: 6.208s - 29881711 (byte)
- ascii_sprintf_g: 1.061s - 2452395 (byte)
- binary: 2.689s - 27629527 (byte)
Want more performance out of OpenFOAM? Then please suggest proven file storage software/technology, instead of simply complaining about it
... And I'll go first
: LZO and LZ4 are high speed compression algorithms that offer impressive compression/decompression speeds, where the decompression is almost as fast as memcpy. These compression algorithms don't offer as much compression ratios as gzip and bz2, but for most cases, they could offer improved throughput to disk for writing such files when the data is in binary format.Best regards,
Bruno
Total Comments 0
