[natem]

Once again some hardware questions with regards to investing in a new work station...
I did my best to browse through many of the past posts on this topic, but my case seems to be so special that it has not been covered exactly like this in the past.


I have two usage scenarios for my system, both of them can license-wise use an unlimited number of cores:
1. Running non-parallelized steady-state CFD calculations using PHOENICS (segregated solver and coupled solver). I always have several variants (typically 8-16) for each of my projects, so I still need a multi-core system to process those variants in parallel. Also the RAM usage is quite high with the coupled solver (2GB per 1 million cells * 10 million cell cases * # of variants = 20 GB RAM * # of variants)
2. Running well parallelized steady-state CFD calculations using OpenFOAM (segregated solver). RAM usage is lower (up to 10 GB RAM in total, assuming I won’t run the different variants in parallel but rather decompose each variant and run them sequentially [comment: does this make sense or should I run different variants in parallel, each on a single core, despite the good scalability of OpenFOAM?]).

Both scenarios will be relevant over the life time of the work station; I cannot tell which one will prevail. Obviously this makes the optimization of my new system quite hard. For scenario 1) I’d need fast single-core performance, for scenario 2) a higher number of (slower) cores could be advantageous. Furthermore scenario 1) requires a ridiculous amount of memory (up to 20 GB RAM per variant, which is calculated on a single core) whilst scenario 2) is less demanding with regards to this aspect.

I am currently leaning towards a dual-socket workstation with two Xeon E5-2687W (8 cores per CPU). This would give me the advantage of using up to 16 cores for scenario 2), furthermore I have four memory channels per CPU. Obviously I could use more cores for scenario 2), but all of the common Xeon processors I found had a lower single-core performance – which is important for scenario 1). The disadvantage for scenario 1) is that there would be single-socket systems with a slightly better single core performance (e.g. i7-3970X), but they don’t support enough RAM. The sad part is that even though the E5-2687W support up to 750GB RAM, I probably won’t manage to get more than 256 GB RAM (budget restrictions; furthermore I want to avoid overinvesting since it might turn out later that scenario 2) is prevailing). This in turn means that for scenario 1) I could only a maximum of 12 variants in parallel, so four cores would effectively be unused.

My questions to you:
1.) Do you think my choice of CPU seems reasonable with regards to above scenarios?
2.) I understand transfer rate and number of memory channels for RAM are critical for good CFD performance. Is there any advantage in getting 2133 MHz (or even faster) RAM, even though Intel states that E5-2687W only supports up to 1600 MHz (moreover I’d need to find a Mobo that supports this)?
3.) The question of ECC or non-ECC RAM: Opinions in this forum seem to vary, some say it’s only expensive and not necessary, other claim that it’s an absolute must with multi-socket systems. Could you please comment on that?

Please note that it has to be one single work station, so other clustering solutions or running two single-socket workstation in parallel is not an option. Also, I’d prefer to not get into the overclocking business (I’m unexperienced in that area and find it too risky to get things wrong), so just assume there will be no tuning in that respect.

Thank you very much for your advice.

Regards
Nate

[CapSizer]

Originally Posted by natem

Once again some hardware questions with regards to investing in a new work station...
1.) Do you think my choice of CPU seems reasonable with regards to above scenarios?
2.) I understand transfer rate and number of memory channels for RAM are critical for good CFD performance. Is there any advantage in getting 2133 MHz (or even faster) RAM, even though Intel states that E5-2687W only supports up to 1600 MHz (moreover I’d need to find a Mobo that supports this)?
3.) The question of ECC or non-ECC RAM: Opinions in this forum seem to vary, some say it’s only expensive and not necessary, other claim that it’s an absolute must with multi-socket systems. Could you please comment on that?

Well, you've said it, you are faced with conflicting requirements. E5 CPUs are very expensive, so in your case a small cluster, possible a "cluster in a box" built using several Core i7's would be more cost-effective, but if you either can't or won't go down that route, yes, the 8-core E5 is a good option.

I do not think that you will find a dual-socket Xeon board that will allow you to overclock the memory.

Check the motherboard documentation for the question of ECC.

[natem]

Thanks very much, CapSizer, for your valuable comments.

If I understand correctly, there'd only be an advantage in buying 2133 MHz (or faster RAM), if I was able to overclock the CPU (which is something I don't consider doing).

As for ECC vs. non-ECC: Is it really just a question of what the motherboard requires? I thought a motherboard that can handle ECC can also handle non-ECC. And then the question gets down to: Is there actually any real benefit / necessity in using ECC-RAM for CFD. Maybe you could briefly clarify this bit.

Thanks again!

[CapSizer]

Originally Posted by natem

If I understand correctly, there'd only be an advantage in buying 2133 MHz (or faster RAM), if I was able to overclock the CPU (which is something I don't consider doing).

As for ECC vs. non-ECC: Is it really just a question of what the motherboard requires? I thought a motherboard that can handle ECC can also handle non-ECC. And then the question gets down to: Is there actually any real benefit / necessity in using ECC-RAM for CFD. Maybe you could briefly clarify this bit.

The CPU and memory clocks are separate, and can be overclocked independently, although not normally for multi-socket systems.

From what I can gather, some dual-socket chipsets will allow you to use non-ECC RAM up to a certain amount. If you want to use more, all the memory must be ECC.

[natem]

Sorry I had been a little unprecise with regards to over-clocking - that is what I meant.

I'll probably have to consult with the MoBo manufacturer to see whether a large amount of non-ECC RAM (in total I am aiming for ~256GB) would be possible. I hope it's true what other members in this forum claim, when they say that ECC is not really necessary for most CFD applications.

Thanks once more!

[evcelica]

256GB non-ECC would not be possible. The MOBO probably has 8 slots for each CPU. and non-ECC only comes up to 8GB per module, so 128GB total.

If you are going to need a very large computer with a lot of RAM then the dual E5-2687W is probably the best you can get. And you will probably have to go with the ECC memory just for hardware reasons.

[natem]

Thanks a lot for this comment, evcelica.
I actually wasn't aware of this fact!