[General] Reading Fluent gzip-compressed files to ParaView

larsschwarzer · May 16, 2014, 11:13

Hello everyone,
I've recently started working with ParaView to post-process my Ansys Fluent calculations. As my overall memory space is limited, I want to save my result files in compressed form (i.e., cas.gz and dat.gz). I have tried to read these into ParaView, but ParaView fails to find the corresponding data-file of the pair and ultimately crashes. A short example: There are two files named

my_file_name.cas.gz
my_file_name.dat.gz

I can select 'my_file_name.cas.gz' through the ParaView GUI, and it will recognize the file to be a Fluent file (appropriate reader is selected). ParaView will then go busy for several minutes and finally crash. In serial mode, there is no error message. I have tried parallel mode as well, which at least gives me an obscure message somewhere along the lines that there is no .dat-file.

Is there any way to force ParaView to invoke gunzip when reading the files? It works fine with the uncompressed files ('my_file_name.cas' and 'my_file_name.dat'), which I do not want to save permanently, though (disk space!).

The whole idea is to 'streamline' my workflow and not have to maintain a compressed and uncompressed file side by side, or have to manually unzip the file, e.g. by Fluent, post-process and then delete the file again. (I'm doing case studies and frequently have to re-evaluate older solutions in the light of new results)

I've checked this Forum, the greater Internet and the manual to no success. Any help on solutions (or workarounds) is appreciated.

Thanks and enjoy your weekend,
Lars

(Program versions: ParaView 4.1.0, Fluent 14.5.7)

wyldckat · May 16, 2014, 17:20

Greetings Lars and welcome to the forum!

Knowing which Operating System you're using would help, as ways to get around these kinds of issues can depend on the OS you're using.

For example, on Linux:

You could use a shell script (Bash or even Python) for unpacking the files to a temporary work folder, open them in ParaView and when ParaView closes, the script would delete the temporary folder.
You could use a specific FUSE module for opening files on the fly, as if they were uncompressed.

On Windows:

You could use a batch file or a Powershell script, to do the same as it was done in Linux with a shell script.
There might still be a solution similar to a FUSE module for Windows, where ZIP files are accessed on-the-fly as if they were USB drives.

If you need a more OS-agnostic way of doing this, for example, directly in ParaView, then it might get a bit tricky. This is because ParaView does not support uncompressing on-the-fly, unless the file format itself has an internal compression option. The way to try and do this would be to use a "Programmable Source" (see the "Sources" menu), coded in Python. This would essentially unzip the file, open with the dedicated reader and then erase the temporary file once the data was read into RAM. The complexity increases if you have transient data spread across several files...

Therefore, the remaining question is: which solution do you want to go with?

Best regards,
Bruno

larsschwarzer · May 19, 2014, 13:28

Hi Bruno,
thanks for answering! I'm using a Windows desktop/work station for post-processing. For now, it's all steady-state data.

The on-the-fly solutions sound neat, although the idea with a batch or script file might also do the trick. All I am trying to do is to avoid having to sit in front of my computer, press a few buttons, wait, press some other buttons, wait again, etc., etc.

If I get around to try some of your ideas this week, I'll post the results here.

Thanks,
Lars

cfdan · August 4, 2014, 23:55

extract both .gz files
rename them into my_file_name.cas and my_file_name.dat
open my_file_name.cas as fluent case file

larsschwarzer · August 8, 2014, 05:12

Hi cfdan,
thanks for your suggestion. That's more or less what I was doing before, but I was trying to avoid keeping the compressed and uncompressed files or creating a new uncompressed file every time I want to return to post-processing.

Recently, my supervisor made me drop the issue because other problems were more urgent. I'll try to get back to the topic, though, whenever I can spare a few minutes.

May 16, 2014, 11:13	Reading Fluent gzip-compressed files to ParaView	#1
larsschwarzer New Member Lars Schwarzer Join Date: Apr 2013 Posts: 8 Rep Power: 13	Hello everyone, I've recently started working with ParaView to post-process my Ansys Fluent calculations. As my overall memory space is limited, I want to save my result files in compressed form (i.e., cas.gz and dat.gz). I have tried to read these into ParaView, but ParaView fails to find the corresponding data-file of the pair and ultimately crashes. A short example: There are two files named my_file_name.cas.gz my_file_name.dat.gz I can select 'my_file_name.cas.gz' through the ParaView GUI, and it will recognize the file to be a Fluent file (appropriate reader is selected). ParaView will then go busy for several minutes and finally crash. In serial mode, there is no error message. I have tried parallel mode as well, which at least gives me an obscure message somewhere along the lines that there is no .dat-file. Is there any way to force ParaView to invoke gunzip when reading the files? It works fine with the uncompressed files ('my_file_name.cas' and 'my_file_name.dat'), which I do not want to save permanently, though (disk space!). The whole idea is to 'streamline' my workflow and not have to maintain a compressed and uncompressed file side by side, or have to manually unzip the file, e.g. by Fluent, post-process and then delete the file again. (I'm doing case studies and frequently have to re-evaluate older solutions in the light of new results) I've checked this Forum, the greater Internet and the manual to no success. Any help on solutions (or workarounds) is appreciated. Thanks and enjoy your weekend, Lars (Program versions: ParaView 4.1.0, Fluent 14.5.7)

May 16, 2014, 17:20		#2
wyldckat Retired Super Moderator Bruno Santos Join Date: Mar 2009 Location: Lisbon, Portugal Posts: 10,982 Blog Entries: 45 Rep Power: 128	Greetings Lars and welcome to the forum! Knowing which Operating System you're using would help, as ways to get around these kinds of issues can depend on the OS you're using. For example, on Linux: You could use a shell script (Bash or even Python) for unpacking the files to a temporary work folder, open them in ParaView and when ParaView closes, the script would delete the temporary folder. You could use a specific FUSE module for opening files on the fly, as if they were uncompressed. On Windows: You could use a batch file or a Powershell script, to do the same as it was done in Linux with a shell script. There might still be a solution similar to a FUSE module for Windows, where ZIP files are accessed on-the-fly as if they were USB drives. If you need a more OS-agnostic way of doing this, for example, directly in ParaView, then it might get a bit tricky. This is because ParaView does not support uncompressing on-the-fly, unless the file format itself has an internal compression option. The way to try and do this would be to use a "Programmable Source" (see the "Sources" menu), coded in Python. This would essentially unzip the file, open with the dedicated reader and then erase the temporary file once the data was read into RAM. The complexity increases if you have transient data spread across several files... Therefore, the remaining question is: which solution do you want to go with? Best regards, Bruno

August 4, 2014, 23:55		#4
cfdan New Member Join Date: Sep 2012 Posts: 23 Rep Power: 14	extract both .gz files rename them into my_file_name.cas and my_file_name.dat open my_file_name.cas as fluent case file

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[General] Error while reading VTK files in Paraview	d_ray	ParaView	12	January 24, 2018 09:00
Problem in running ICEM grid in Openfoam	Tarak	OpenFOAM	6	September 9, 2011 18:51
Recovering Fluent surfaces in ParaView	alemenchaca	FLUENT	6	August 11, 2011 16:52
OpenFOAM15 paraFoam bug	koen	OpenFOAM Bugs	19	June 30, 2009 11:46
Reading mesh files into Fluent from Gambit	Prabhu.B	FLUENT	4	July 16, 2002 20:03

May 19, 2014, 13:28		#3
larsschwarzer New Member Lars Schwarzer Join Date: Apr 2013 Posts: 8 Rep Power: 13	Hi Bruno, thanks for answering! I'm using a Windows desktop/work station for post-processing. For now, it's all steady-state data. The on-the-fly solutions sound neat, although the idea with a batch or script file might also do the trick. All I am trying to do is to avoid having to sit in front of my computer, press a few buttons, wait, press some other buttons, wait again, etc., etc. If I get around to try some of your ideas this week, I'll post the results here. Thanks, Lars

August 8, 2014, 05:12		#5
larsschwarzer New Member Lars Schwarzer Join Date: Apr 2013 Posts: 8 Rep Power: 13	Hi cfdan, thanks for your suggestion. That's more or less what I was doing before, but I was trying to avoid keeping the compressed and uncompressed files or creating a new uncompressed file every time I want to return to post-processing. Recently, my supervisor made me drop the issue because other problems were more urgent. I'll try to get back to the topic, though, whenever I can spare a few minutes.