[Tobi]

Hi all,

i set up a Server for my numerical calculations with openfoam.

MB with two cpu sockets - opteron 12 cores @ 2400 MHz.

I want to insert 2x 32 gb Memory.
Memory is 4x8 gb per cpu so 16 gb per core.

Question is, if I should use an EEC or non-eec Memory?

Can someone give me an advice because the opinions are very different and EEC is much more expensive.

I think I ll overclock the CPU to 2.9 GHz.

Regards
Tobi

[pete]

ECC memory is not critical. We have run a cluster with >300 CPUs for soon 10 years and none of our compute nodes have ECC memory. ECC is critical for a server that can't fail (CFD Online's server has ECC memory), but for a CFD simulation what happens is at worst that your simulation crashes and you have to reboot a node before you restart the simulation.

[wyldckat]

Greetings to all!

Quick overview of what I can think about before going to sleep:

• ECC gives you peace of mind. If you have a run that needs to be finished in the next morning, but it crashed in the middle of the night... ECC would possibly have been worth it.
• I don't think you can overclock AMD Opteron CPUs, at least not on conventional dual socket server motherboards... but I could be wrong.
• ECC and overclocking don't mix... but I could be wrong here as well.
• Be careful about the specifications of the specific AMD Opteron CPU model, because of two details:
  • How many memory channels the CPU has got. There are models with only 2 memory channels, even if they handle DDR3. So be sure to buy CPUs with 4 memory channels.
  • 12 logical cores per CPU is not the same as 6 real cores per CPU, so be careful about this specific detail. Because when it comes to CFD, it's the real cores that count.

Best regards,
Bruno

[Tobi]

Hi all,

thanks for your replay.

1. I decided to use NON-EEC ram.
2. Overclocking with EEC is not possible (you are right bruno)
3. Overclocking Opteron - i am not sure but I am still not sure if I do that.

4. Cores - I know that the "real" cores are the true thing. For that I thought that the Opteron 6234 do have 12 real cores - but now its too late I bought the stuff today evening damn it.

So I still have 12 real cores ... well hmmm I informed me the last 2 weeks about things and stuff and the last 3 days additionally here in the forum ...

Well ... shit ...

PS: 4 channel support!



Well a better option would be send everything back (if possible) and buy a motherboard with 4 CPU ś and insert CPU 's with 4 or 6 real cores.
How can I see how many real cores I have? The problem that I have - at the official AMD website there is only "12 cores" nothing with real and virtual ...

[evcelica]

That cpu has 12 real cores......
edit: Actually, it seems a bit more complicated now that I've looked a bit:
http://www.anandtech.com/show/5058/a...nterlagos-6200

AMD doesn't use hyperthreading, that's only an Intel thing. So it is 12 core, but the cores share resources:

That is a six module (not core) CPU that runs 12 Threads. Each module can run 2x integer cores, for 2*6 = 12 real integer cores. But each module can only do one 256bit Floating Point thread.
Here is a quote from the discussion:
__________________________________________________ _________________________
Each module includes 2x integer cores, correct. But the floating point core is "shared-separate", meaning it an be used as two separate 128-bit FPUs or as a single 256 FPU.
Thus, each Bulldozer module can run either 3 or 4 threads simultaneously:
- 2x integer + 2x 128-bit FP threads, or
- 2x integer + 1x 256-bit FP threads
It's definitely a dual-core module. It's just that the number of threads it can run is flexible.
The thing to remember, though, is that these are separate hardware pipelines, not mickey-moused hyperthreaded pipelines.
__________________________________________________ _________________________

[Tobi]

Originally Posted by Irving Huseman

I don't think you can overclock AMD Opteron CPUs, at least not on conventional dual socket server motherboards... but I could be wrong.At least now I think so.

Hi,

well I do not have to overclock the CPU. I will read and inform you about it. If its not working its not a problem for me.


@evcelica: Thanks for your answer. Where can I find that information that the Opteron 6234 has 12 real cores?

Tobi

[wyldckat]

Greetings to all!

Just to be clear on this, when I said:

Originally Posted by wyldckat

• 12 logical cores per CPU is not the same as 6 real cores per CPU, so be careful about this specific detail. Because when it comes to CFD, it's the real cores that count.

I only meant by this is that I did not know which exact Opteron model it was, so I advised you to verify what kind of cores they truly were. This is because AMD also adhered to the concept of having shared calculation parts between cores, as described here:

Originally Posted by CapSizer

The current series of AMD CPU's use a slightly unusual design. The CPU's are built up from a number of "modules", each of which has two cores that share a single floating point processor. You can get into long debates about whether or not this is a brilliant way of putting together a CPU, but in fact it makes quite a lot of technical sense, even if it is not great for CFD use. Your 4170 CPU has two of these modules, therefore 4 cores in total, but only two floating point processors. This may not be as bad as it sounds. You can run 4-way parallel, and it will be significantly faster than running just two way parallel, but will probably not be twice as fast. What you should do is to run exactly that test ... set up an OpenFOAM case, and run it serial, 2-way, 3-way and 4-way. Then graph the number of iterations / minute, and post the graph here.

The attached graph should help explain - this is an "8-core" FX8150. Once you get to 4-way parallel, the speed flattens off, because you now have to start sharing floating point processors. You also get a saturation of the memory system, which is less noticeable when you underclock it.

Honestly, when I read this, it completely broke my heart, because I always thought that AMD was true to the denomination of CPU cores. But apparently they've had to give in to the concept of sharing computing cores.
In essence, for CFD, this only means a slight improvement in performance, because all it does is help schedule data to be computed by the real calculation units. And since scheduling also takes time, there is some improvement in all of this.

As for the Opteron 6234 specifications, it's a bit difficult to find some information on it:

• This site doesn't seem to be reliable enough:
  • Opteron 6234: http://www.cpu-world.com/CPUs/Bulldo...on%206234.html
  • FX-8350: http://www.cpu-world.com/CPUs/Bulldo...20FX-8350.html - it doesn't add up to what was stated in the quote above.
• The official page doesn't give enough information: http://products.amd.com/pages/Optero...f9=&f10=&f11=& - but 256bit FPU is a lot of bits!!!
• Good old Wikipedia: http://en.wikipedia.org/wiki/Opteron...roarchitecture - 12-core - Interlagos (6234, 6238):
  

  Multi-chip module consisting of two dies, each with three dual-core Bulldozer modules

• More information on each Bulldozer module: http://en.wikipedia.org/wiki/Bulldoz...Bulldozer_core
  

  • Two dedicated integer clusters
    • each one consists of two ALU and two AGU which are capable for total of 4 independent arithmetic and memory operations per clock and per cluster
    • duplicating integer schedulers and execution pipelines offers dedicated hardware to each of two threads which increase performance in some multi-threaded integer case
    • the second integer cluster increases the Bulldozer core die by around 12%, which at chip level adds about 5% of total die space[14]
  • Two symmetrical 128-bit FMAC (fused multiply–add capability) floating-point pipelines per module that can be unified into one large 256-bit-wide unit if one of integer cores dispatch AVX instruction and two symmetrical x87/MMX/SSE capable FPPs for backward compatibility with SSE2 non-optimized software

If you continue reading the wikipedia page on the last link, it gives the idea that if your motherboard is properly up-to-date at the BIOS level and if you properly compile OpenFOAM with the latest GCC 4.7 series, using the correct and dedicated build options for this series of processors, you should get a really very good performance out of these processors.

This is promising: http://developer.amd.com/community/b...es-processors/

Best regards,
Bruno

[Tobi]

Wow Bruno,

thanks for the information and your research. That sounds good - if I do not understand the things wrong

The topic with gcc compiler is new and I can not understand the advantages of the new 4.7 version.

But I am going to set up OpenFOAM with that gcc version.

Thanks additionally.
Tobi