[philippose]

Hello again :-)!

Looks like I seem to be putting in a lot of threads this weekend :-)!

This time, its more to do with performance comparisons.... I know there are lot of you who run OpenFOAM on a variety of multi-processor environments, so... here is a question...

I was thinking of putting together a small cluster of linux systems (about 8 to 10 processor cores...i.e.... 4 to 5 dual cores), particularly with OpenFOAM in mind.

Which processor series would be the better choice from point of view of power consumption / performance?

From point of view of just price, the AMD X2 seems to still be pretty much ahead of the Intel Core2Duo processors....but then, the AMD X2s have only upto 2MB cache, while the Core 2 Duos have upto 4MB, etc..etc...

What kind of experiences have you people had regarding performance, and which processor series would be the wiser choice to make?

What kind of factors does one need to look into when making such an evaluation?

Have a nice Sunday!

Philippose

[cfdnovice]

Hi Philippose,

If I had the money to build an 8 to 10 core cluster, I'd think about the possibility of putting all the cores on the same board. You might find that an AMD NUMA system of 4 800-series Opterons will do the job better than a cluster of dual-core boxes.

Check out:

http://www.amdboard.com/tyan_s4882_opteron_board.html

and get some prices on some dual-core Opterons.

I'm not sure what the implications of the NUMA memory arrangement are when running OpenFOAM, but my guess is that it can be made to run nicely on such hardware, and the bandwidth and latency between processes would beat a gigabit ethernet cluster.

Anybody have any direct experience with NUMA and OpenFOAM?


Regards, Graeme.

[seang]

anyone tried the 8 processor Apple box?

[msrinath80]

Philippose, here is my 0.02$

Intel's dual/quad-core offerings are bullshit for High Performance Computing. I've seen and used their core 2 duos and their quads. Pathetic performance. The extra 2 MB L2 cache made no difference. I'm no AMD fan, but I do pretty well recognize class when I see it. AMD X2s do a more decent job thanks to their onbard memory controller and hypertransport technology. Opterons do an even better job. To be fair, I don't really find the idea of dual/quad cores appealing unless CPU manufacturers take the effort to put in extra memory bandwidth. But for the present, between Intel and AMD, I would definitely go for AMD. You can check my scaleup results for an 8-CPU (dual core, so 16-cores) AMD Opteron box as well as for the 2-CPU (quad-core, so 8 cores) Intel box on the Running / Solving / CFD section.

Aside: Ironically my latest notebook is Intel core 2 duo despite their shortcomings. The reason I got it is because it comes with an onboard GMA 950 graphics chipset that delivers good OpenGL performance for reasonable power consumption. Plus it supports free-software GPL drivers (high priority for me). Intel is good for laptop systems and supports free-software drivers. AMD, at present does not. I've tried testing the scale-up of OpenFOAM on my notebook, and I only get 1.25 speedup on most cases. The reason I suspect is because memory access is done through the extremely slow 667 MHz FSB.

[philippose]

Hey!!

Thanks a lot for all the suggestions and ideas :-)! I didnt know that Mac had an 8 processor system in the market!

And Srinath :-)! That was quite a clear set of suggestions I would say :-)! That would make Graeme's idea of a 4 CPU Opteron motherboard quite interesting! (Though that motherboard costs >1200$ !)

I was looking at the AMD website, and they have an Opteron Quad Core slated for release sometime around this month I think.... They are putting in 3 caches on the die.... L1 (64KB) directly on each core, L2 (512KB) one for each core, and an L3 (1024KB) which is common to all four cores, but with special load balancing and data routing stuff.

They have mentioned that it is some "direct connectivity" concept, with a Memory and IO controller embedded on the die, which makes things much more efficient and faster.

Any one has any details regarding these processors? From the looks of it it sounds very interesting.

Have a nice week ahead!

Philippose

[alberto]

I fount this test, which I read when looking for workstations.

http://techreport.com/reviews/2007q1/cpus/index.x?pg=1

We bought Intel based ones, and we are quite happy with them.

In general, pay attention to hyper-integrated systems, like 8 cores on a board or similar, mainly because they're newer (less tested), more prone to thermal issues and more vulnerable (if you have an issue, you stop working). This is not true on a conventional parallel system, where you can easily exclude a node ;-)
Plus, the increase in performance for higly demanding applications like CFD is rarely tested and the price might not be worth of.

I found a test using FLUENT on both Intel and AMD here:

http://www.intel.com/products/proces...xeon_brief.pdf


Regards,
Alberto

[seang]

The interesting thing about the 8 proc Mac box is the amount of memory it can accomodate, up to 32 GB.

[stephan]

hi,

the above link is from intel - i guess looking at the homepage of amd will show the opposite...
www.spec.org gives maybe a more realistic overwiew.
the only argument for buying intel is the price not the performance.
regards
stephan

[tehache]

Hi all,

My experience with OpenFOAM on a cluster of 16 boards with 2 2core Opteron on each blade (i.e. 4 cores/blade), connect by GB-Ethernet, tells me that OpenFoam scales very well up to 4 processes RUNNING ON THE SAME BLADE, but performance is lousy if processes are running on different blades. Obviously, network connection seems to be much more important than performance of the single cores.
Now trying to get gamma running, but dont know yet...

regards,

Thomas

[cfdnovice]

To Kuan:

As far as I can tell, the "8-processor Apple box" you mention is really a dual quad-core Xeon system. I'd expect CFD to be fairly heavy on the bus usage, so packing more and more cores on the one CPU is likely to lead to limited performance improvements.

To Phillipose:

Since you seem interested in more up-to-date hardware than the quad dual-core system I mentioned, and I agree that that mobo is expensive, you might look at:

http://www.tyan.com/product_board_detail.aspx?pid=496

The Tyan Thunder n3600QE (S4980) is a quad Opteron board using the 1207-pin socket, and it's claimed to be quad-core ready, though as I mentioned above, I doubt the increase in performance for CFD by going to quad cores would justify the cost. (I may be mistaken.)

You might wait for the quad-cores to come out in the hope of getting a price reduction on the dual-cores, then populate one of these with the (presumably cheaper) dual-core Opterons.

You'd want at least a 1kW power supply, and I think that board is a bit larger than your average ATX, so make sure it'll fit into the case. Maybe have the system built for you by someone reliable.


Regards, Graeme.