|
[Sponsors] |
HDD Storage Slow in Parallel Read/Write reconstructPar |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
July 11, 2016, 14:46 |
HDD Storage Slow in Parallel Read/Write reconstructPar
|
#1 |
Member
james wilson
Join Date: Aug 2014
Location: Orlando, Fl
Posts: 39
Rep Power: 12 |
All,
I searched the forum for something related to my issue but couldn't find much specific to my issue. So i have a dual boot system and a shared HDD attached to the MOBO that is formatted to NTFS so that I can access data from windows as well as linux. When i run cases on the HDD, namely parallel but also serial, I notice there is a large hang time when writing larger quantities of data, e.g. reconstructPar. I should say, more hang time than what I would expect, even from large case files with a high cell count. When executing the reconstructPar command for larger cases 100K-1M or so cells, the utility will hang for anywhere from 10s to minutes before beginning to reconstructing the data. Even the transition from one time step to the next while reconstruction takes a good deal of time. Its so bad that I have restricted the number of parallel processors to use for the simulation because each additional processor drastically increases reconstruction time. When running these larger cases on my SSD, formatted to ext4, I dont see any excessive hang time (other than what would be expected when dealing with a large cell count and number of fields written). The questions I have are: 1. Has anyone experienced this with an HDD? 2. Does anyone have any suggestion on what file system type I should use for a cross OS compatible storage drive such that linux (ubuntu 14.04) can handle the read/write operations required by openfoam? 3. should I abandon the requirement to have a cross OS compatible file system so that I can maximize performance when running large OF cases on the drive? I will be purchasing a 2TB drive that has a higher spin speed and read/write speed. I just want to make sure I format everything properly before I make an investment. Thanks all for the views and any feedback, James |
|
July 12, 2016, 10:33 |
|
#2 |
Senior Member
Michael Prinkey
Join Date: Mar 2009
Location: Pittsburgh PA
Posts: 363
Rep Power: 25 |
reconstructPar is pretty I/O intensive. I would try copying the data over from NTFS to a native Linux filesystem and see if performance improves. NTFS support in Linux has been focused on "making it work" versus "making it fast." You will likely have better results with ext3/4 or xfs filesystem serving up the data.
and if you need speed, buy a 128 or 256 GB SSD will make more differences than anything else. Use it as your storage for the current run and archive to the spin disk when you are done. |
|
July 24, 2017, 13:54 |
|
#3 |
Senior Member
sandy
Join Date: Feb 2016
Location: .
Posts: 117
Rep Power: 10 |
reconstructPar can be done in parallel as well. it reduces time immensly. i have the utility. in case you guys need it
|
|
Tags |
file system type, hdd, openfoam, reconstructpar, slow write speed |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
The udf.h headers are unable to open- in VISUAL STUDIO 13 | sanjeetlimbu | Fluent UDF and Scheme Programming | 4 | May 2, 2016 06:38 |
Explicitly filtered LES | saeedi | Main CFD Forum | 16 | October 14, 2015 12:58 |
simpleFoam parallel | AndrewMortimer | OpenFOAM Running, Solving & CFD | 12 | August 7, 2015 19:45 |
the speed of parallel computation for over 3 million grid is very slow | liujmljm | SU2 | 7 | June 27, 2014 17:45 |
Coal combustion, big node, slow calculation, parallel | Whitebear | CFX | 1 | April 8, 2014 04:27 |