At work I have been running CFD for a few years on SUN workstations (Ultra's). Recently, I am getting the impression that you get 'more for your bucks' if you run CFD on pc platforms. It certainly seems that the cpu's are much cheaper, are developed quicker, and may even be significantly quicker. Memory also seems cheaper.

Initially I had concerns about pc platform stability, but the latest versions of linux seem to be the solution to this problem. Has anybody made the switch to pc's and can comment on their experiences (good or bad)? On a similar note... Looking in the pc mags etc. it would appear fairly simple and cheap to construct a multi cpu intel based computer. Has anybody done this and used it for large CFD problems? My driving force here is; why buy one top of the range cpu when 3 lower spec cpu chips would be quicker and cheaper (even considering data transfer penalties)? Has anybody tried such a contraption?

PS Sorry if this has already been discussed and I missed it.

We've had a 28 node PC cluster running for a few months now. The PC's are connected with two 100 mbit switches interconnected with 4 bundled channels. All components are 100% off-the-shelf products. We use the cluster as a compute server for CFD simulations.

So far we are extremely happy with both performance and stability. Aside from one intital harddisk crash we haven't had any problems - all 28 nodes have stayed up for more than 3 months now! This is better than our HP V-class server! We run Linux RedHat 6.0 on the cluster.

Performance wise... Noone wants to run on our HP J and V class machines any more! The cluster scales very well and is faster per node than our one-year old high-end HP workstations (HP J2240).

We have PIII-500 machines with 512 mb per node. Today you could buy AMD Athlon 800 MHz machines for the same price and get about twice the performance per CPU. I'm sure no workstation solution comes close to the price/performance of this. If you need top-perfromance you should connect the nodes with a Myrinet. However, IMHO the speed gain this gives can't warrant the extra cost. Standard fast ethernet is most often good enough and I think that it is wise to stay with the basic philosophy of only using standard off-the-shelf compononents (don't buy any fancy double CPU Xeon machines or so... you can never warrant the extra cost).

(1). To get the best results, often one has to have the best computer system to attract the best engineer in the world. (2). If cheaper computer system .not equal. cheaper CFD solution , then it does not make sense to invest time and effort to create and use cheaper systems. (3). In terms of a single software on a single computer system (multiple workstations or PCs), there is no practical problems in using different computers, as long as the software can fit and run on the new system. (4). On the other hand, a company with many engineering groups and hundreds of design software on the system, the change in computer systems and conversion (or different versions) of software will become a big problem. (5). The problem is in the design system integration as a whole. If the designer (CAD) is using a workstation with high speed graphics, then it does not make sense to use a PC or a network of PCs to do CFD. (6). I must say that in 3-D CFD, graphics is one of the most important area in the operation (including geometry, mesh, flow field results, animation.) Therefore, the system integration must look at the graphics capability as the most important factor in the hardware and the supporting software. (7). Because of computer games development, PC is getting high speed graphic cards at affordable cost. But the hardware graphic workstation has been around since early 80's, it is going to take a while to replace it by PC. (even the graphic standards and library were developed for workstations) (8). If you have a high speed main-frame computer, there will be a group of engineers to keep it running all the time. With thousands of workstations or PCs in the todays network environment, the failure rate is just thousands of times higher. The cost of computer down time becomes very expensive in the network environment. (if you have 28 PCs, there will be 28 hard drives, 28 CPUs, 28 sets of RAMs, Cards, etc.. At some point, it will become a problem to fix the hardware problem which can happen randomly in time.)

i think you'll find that today's high end pc offer more bang for the buck than workstations. especially compared to the suns you're working with now. linux works extremely well (as you can see from Jonas' post) at penn state we have similar pc clusters running linux and there very cost effective compared to supercomputers. if you want to put together a multiprocessor (say two) computer the best mag i think is computer shopper. you know the big thick one full of ads for generic parts. if you want to do this get a systems admin guy or your resident office compugeek to help you. we've had similar threads before so you can check the archives.

I have PII-PRO 200 machine with 512 Mb RAM and 2 CPU. NT is the operating system. So far (3 year) I have no problem with both performance and stability. I can run 3 job at the same time with no RAM and hard disk problem. Typical run takes 3-15 days depending on problems. The code is in serial form and used for furnace simulation.

nuray

I HAVE PII-450 MACHINE AND DECIDE TO BUY DOUBLE CPU XEON. IS THE SUCCESS OF AMD ATHLON FOR NUMBER CRUNCHING (NUMERICAL APPLICATIONS)OR GRAFICAL APPLICATIONS?

NURAY

(1). I can offer my experience in using PC for CFD development. (2). I have used Radio Shack/Model-III (a popular, 1MHZ DOS-model) for my Navier-Stokes code, in Basic and Fortran. Pen plotter graphics was written in Basic, separetely. Radio-Shack Fortran and Basic were used. (3). I have used Amiga/1000 (a multi-media computer, with Motorola CPU, multi-task GUI) for my compressible Navier-Stokes code, no graphics included. I have used Amiga for photo-realistic 3-D ray-tracing applications. Absoft Fortran was used. (4). I have used a Korean brand PC/286 to develop Navier-Stokes code and commercial prototype code with graphics, in DOS mode. A typical simple 2-D case ran 5 to 15 minuites. MSFortran was used. (MS is no longer supporting Fortran for several years.) (5). I have used Pentium/ 90~266, through AMD/k6-2/400 for Navier-Stokes development. (6). Over twenty years, I had to repair once on Radio-Shack computer for monitor problem, once to repair the external line socket on the Pentium computer for bend pins, and recently to replace the crushed disk drive, which ended up with a nearly complete upgrade to a AMD/k6-2/400 system. (7). So, from the code development point of view, there is no problem with PC, regardless of the brand, CPU, or nationality. (8). The modern commercial CFD codes are different. These codes require graphic support, which is always non-universal. And even the UNIX system is machine dependent. (9). The conclusion is: PC is all right for CFD development work. If you are trying to run PC-version of commercial CFD codes (which are ported from UNIX workstation side), there is no guarantee that the whole package will work smoothly. When you run into problem, the solution is always difficult to find. (No one is designing a PC just for CFD applications, yet. And even the Windows operating system is not bug-free.) (10). So, to use a commercial CFD code, you buy the machine and the system used by the vendor, to avoid un-necessary headache. And whether a job will run faster on a new CPU with a higher clock speed, requires actual testing of the code on the machine. Sometimes, it will run slower on a new CPU, but in most cases it will run faster, but not as fast as the clock speed ratio. (11). When you are still undecided, there are already PCs running at one gigaHZ.

The AMD Athlon has better floating point performance than a PIII or a Xeon at the same clock freq. We haven't done any own benchmarks yet, but I'd guess that it is say 20% faster.

I don't think that the small performance gain you get with a Xeon machine compared to the standard PIII can justify the extra cost. We have benchmarked PIIIs aganst Xeons on both in-house and commercial CFD codes and the difference is barely noticeable, while the cost is more than twice as much, but YMMW of course.

I'm doing 3-D grid generation and running a CFD solver simultaneously on a dual PIII 500Mz/896MB RAM machine and getting better performance than with SGI R10000 for 1/4 of the price.

(1). SGI is famous for its graphics. (2). If you are not doing graphics, I guess it is always cheaper to use PCs or other workstations.

I use graphics for grid gen (ICEM-CFD), and solution/grid visualization (Tecplot).

(1). Yes, these are professional codes. (2). The original Plot3d and Fast from NASA were developed on SGI machine. And as I remember, at one time, we had to transfer the result file and gird file from HP machine to a SGI machine just to do the 3-D plotting. (3). The pre- and post- processing software is definitely providing great help in getting the job done easier.