I am running a Fortran CFD application code PC with a 550mhz PIII and Windows NT. I want to know how much total time is used to i/o the hard drive during a typical run of the CFD code. Is there any way I can do this without modifying the code?

My aim is to determine if I can save significant time, in obtaining a solution, by getting a faster hard drive. BTW, I have the whole model in RAM during computations to minimize any hard disk io; any i/o is for interblock interpolation.

(1). I am not a system person, so when running UNIX job on workstation, I normally open another console window and type in "top" to display the CPU and the job(process) activities. (2). You can then monitor the process status every few seconds, and be able to see the CPU time, % usage, memory usage etc... (3). So, on Window/NT, if you can find the similar utility program, then you can do the same thing. In other words, try to find something which will display the real time process activities, including the CPU and memory usage. (4). Using the hard disk is not a good idea at all. It always creat emotional problem for the ANSYS program users in the office. It is always between the huge model size, finite computer memory size, slow computer models, project deadline, trying to put their job on someone's larger and faster computers, etc... (5). It is perfectly all right to present a coarse mesh model solution based on the available computer resources. If you model the problem properly, it still possible to use the coarse mesh solution (properly distributed mesh). (6). The large mesh size will create convergence problem, that is, the convergence will have much larger effect on the solution accuracy than the effect caused by the coarse converged solution. So, the properly distributed mesh and converged solution are two important factors.

In Windows NT, there is also a command called "top" to display the run job status. You can type the command in the dos prompt

In Windows NT if you right click on your task bar and choose task manager..... a gui will pop up that helps you to monitor your system resources

or

there may be an option for you to instruct your program to write this information in its output. In StarCD the program will record the cpu time and i/o time.

JD,the Task Manager gives NO info on hard drive activity regarding I/O time; and, the CFD program has no option to give that info.

Adam, the 'top' command does not work (for me) when invoked from the command window.

John, I have no choice in the matter of the use of the hard drive by the CFD code. It generates large temporary files that contain flow field, turbulent, and interpolation coefficients info for the block interface boundaries. Also Windows NT uses about 1GB of virtual memory for these jobs even when there is plenty of RAM not being used; I am baffled by this; why is this virtual memory being used? I'm sure that there is plenty of hard drive I/O when virtual memory is in use. I think that the main limitation in the use of PCs for CFD is the slow I/O of the hard drives.

(1). I think it is the programming philosophy behind the code you are using. (2). The I/O time for hard disk is always slow regardless of the machine or CPU. So, in old days, they have to charge the I/O time in addition to the CPU time in using the super-computer. (3). Well, if you can see the CPU running all the time ,say 98% , then it is not doing the I/O. (if you have two jobs on the same machine, then it will be 50-50 split, and the CPU time for one process will be say 48%.)

If you are running over your physical memory limit into virtual memory, then the OS is swapping your process into and out of the real memory from disk. Yes, swapping will kill you. I almost always stop swapping processes because they run, say, 20 times slower than those that stay in memory. So I'd guess you need more memory, not more disk.

Sometimes CFD codes will store certain bits and pieces of info on disk, e.g. inter-zone interpolation coefficents. But the designer of the CFD code will only store information that is relatively infrequently used....thus any i/o vs. memory-speed penalties will not be significant. I doubt your CFD code author would use disk improperly....most people writing programs are well aware that disk is much much slower than memory.

A real OS just caches all the temporary and scatch files as long as there is enough main memory. So even a poorly designed code might run fairly fast. If you insist on using NT you have to accept the limitations.

Joern, what is a 'real' OS?

Ed, you haven't read my post carefully. I said that NT uses virtual memory even when when much RAM (~400MB)is NOT being used.

Steve,

As JC Chien said, you need to monitor your CPU usage, I have been running Fluent on NT and if the CPU usage close to 100% (single-cpu) then it means all the CFD are stored in memory. If it is fully-coupled solver, then I think everything should be in core. But for segregated solver, each variables are solved iteratively/sequentially, I noticed after each iteration, there is a noticeable disk i/o for a very short period (several seconds). Then for a number of iterations, there is a relatively long disk-swap(tens of seconds). This happens no matter how big your memory is, that's just the way the software was designed. This disk i/o should be minimal, otherwise the code is really bad.

A complete guess. Using files is the default communication mechanism that always work (albeit slowly). The author of your code (you have not said who) is using separate streams for each block interface (or group of interfaces). The mechanism used to communicate sensibly exists on the platform the code was developed (e.g. unix) and does not exist on NT or has not been purchased or does not quite conform to specification or you have not configured the software to use it. Check for things like MPI, PVM, signals/sockets, pipes, TCP, etc... in the supporting documentation.

If that gets you know where, it should be easy enough for someone in your computer support group to show you how to connect up the streams in the environment of the program or how to use a chunk of memory as a temporary file system rather than the disk. This is straightforward in unix and is probably doable in NT if you can find someone who knows how.

Kai, are you saying that the CPU is not involved in transferring data from RAM to the hard drive?

yes, it surely need cpu, but because the slow disk i/o, the cpu nned not be dedicated for that process, meaning only upto, say 20% of cpu time will be used depending on the disk access time.

I did read it.

I don't know all the ins and outs of NT, but I assume that it may allocate VM to a process but doesn't actually USE it unless needed. Ideally (and practically), you don't want to use more physical memory than you really have. I merely meant to offer the advice that you check first, before anything else, that you are not swapping.

Try turning down virtual memory (set to 50Mb or something under Control Panel-System-Performance) and rebooting and run again. If you ARE swapping (NT kills your process with an "out-of-virtual-memory" message), then it's time to buy more physical memory, not a faster disk.

Under the NT Task Manager, go to View->Options->Select Columns, and pick Page Faults and Virtual memory --- this will enable you to get some idea of i/o activity. Also, total time spent in i/o will be essentially the "System" time if you time your process. Get the free CYGWIN tools for NT and run your process from the command line as"time executeable-name". Observe system time and compare to total time.

To address your first question, a "typical" CFD code spends about 0.1% of the total run time doing i/o. "Typical" CFD codes do not run in this "out-of-core" mode that you seem to be running in, and only write to disk periodically to save results.

Ed, it seems you are assuming parallel operation. This code has that capability in its unix version but not in the NT version. If I had a computer support group, I would not be asking questions on this discussion board . I know no one who have the combined competent knowledge of Windows NT and CFD.

For the pc there are Linux, Solaris and Unixware. Personally I prefer to use SGI/Irix.

The problem with the "very fast" disks is, that they are noisy and tend to crash faster than the slow ones.

Joern, if you are willing to donate $10,000-20,000, I'll buy an SGI or other Unix workstation . Unfortunately, our budget only allows PCs.