Importing large .csv files to STARCCM+

kermelosh · August 6, 2021, 15:57

Hi,

I am importing a very large instantaneous velocity .csv file into my simulation file (> 5 GB) using the table import option. The problem is that it is taking a really long time to import the table (days). Is there another way of doing this in a time efficient way?

Thanks

khw

LuckyTran · August 7, 2021, 05:20

By any chance is it still reading? Like, have you seen it finish reading ever?

Default Star launches with a java heap space of 768 mb and shouldn't be able to read any .csv larger than that. But it would throw an error as soon as it runs out of heap space which should happen rather quickly if the .csv has been successfully read from the disk.

cwl · August 7, 2021, 14:28

LuckyTran, I'm curious - is there a way to increase the heap size? - Through the command line parameter maybe?
I could not find anything related in the user guide.

kermelosh · August 7, 2021, 15:57

LuckyTran, It is still reading. There is no error indication suggesting it run out of heap space.

LuckyTran · August 8, 2021, 22:42

It should only take about 5-10 minutes to pull 5GB from the disk. If it takes longer than that, then your job is stuck in limbo.

The maximum heap space you can increase by passing -jvmargs -Xmx6G into the command line to increase it to 6GB just as an example. It's not specific to Star but all java virtual machine launchers. What's unique in Star is that Star automatically launches using a max heap space of 786 megabytes. It is very uncommon that you need to stream a file larger than this in one go. But then you run into the guy that has a 5GB .csv...

August 6, 2021, 15:57	Importing large .csv files to STARCCM+	#1
kermelosh New Member agSilverMAN Join Date: Aug 2018 Posts: 6 Rep Power: 8	Hi, I am importing a very large instantaneous velocity .csv file into my simulation file (> 5 GB) using the table import option. The problem is that it is taking a really long time to import the table (days). Is there another way of doing this in a time efficient way? Thanks khw

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Workbench Parametric Runs - Obtaion .csv files Automatically	nikos_	FLUENT	3	April 3, 2017 04:14
Problems in compiling paraview in Suse 10.3 platform	chiven	OpenFOAM Installation	3	December 1, 2009 08:21
Format for importing Plot3d files into Star-CCM+	Meredith	Siemens	1	October 12, 2005 13:28
very large saved files	jelle	Phoenics	4	August 5, 2005 09:49
Results saving in CFD	hawk	Main CFD Forum	16	July 21, 2005 21:51

August 7, 2021, 05:20		#2
LuckyTran Senior Member Lucky Join Date: Apr 2011 Location: Orlando, FL USA Posts: 5,761 Rep Power: 66	By any chance is it still reading? Like, have you seen it finish reading ever? Default Star launches with a java heap space of 768 mb and shouldn't be able to read any .csv larger than that. But it would throw an error as soon as it runs out of heap space which should happen rather quickly if the .csv has been successfully read from the disk.

August 7, 2021, 14:28		#3
cwl Senior Member Chaotic Water Join Date: Jul 2012 Location: Elgrin Fau Posts: 438 Rep Power: 18	LuckyTran, I'm curious - is there a way to increase the heap size? - Through the command line parameter maybe? I could not find anything related in the user guide.

August 7, 2021, 15:57		#4
kermelosh New Member agSilverMAN Join Date: Aug 2018 Posts: 6 Rep Power: 8	LuckyTran, It is still reading. There is no error indication suggesting it run out of heap space.

August 8, 2021, 22:42		#5
LuckyTran Senior Member Lucky Join Date: Apr 2011 Location: Orlando, FL USA Posts: 5,761 Rep Power: 66	It should only take about 5-10 minutes to pull 5GB from the disk. If it takes longer than that, then your job is stuck in limbo. The maximum heap space you can increase by passing -jvmargs -Xmx6G into the command line to increase it to 6GB just as an example. It's not specific to Star but all java virtual machine launchers. What's unique in Star is that Star automatically launches using a max heap space of 786 megabytes. It is very uncommon that you need to stream a file larger than this in one go. But then you run into the guy that has a 5GB .csv...