[evcelica]

I am purchasing two workstations for CFD (CFX) as well as FEA (ANSYS Mechanical)


OS will be windows 10.



I was leaning towards EPYC ROME 7302, but have seen a fair amount of literature stating INTEL should be better for ANSYS mechanical.

The mechanical benchmarks in this presentation are only for EPYC naples. And show lower performance than Intel. But so do the CFD benchmarks between INTEL and EPYC Naples. While the CFD benchmarks for INTEL vs ROME, show ROME on top, which has been proven around here as well.

http://www.ozeninc.com/wp-content/up...entation-1.pdf


The ANSYS hardware recommendations say Intel CPUs should be used because of AVX512 support.

https://www.simutechgroup.com/suppor...rdware-support


Also, most of the time I will not be using all cores on the node, I will probably only be solving on 8 cores (16 max). Anyone have any mechanical benchmarks for XEON GOLD vs EPYC ROME on a core to core basis? Any advice for picking between the two?


Thanks in advance,
Erik

[flotus1]

The ANSYS hardware recommendations say Intel CPUs should be used because of AVX512 support.

I have seen this statement multiple times in different publications. It is mostly a marketing statement born out of necessity.
With the move from Broadwell to Skylake, Intel Xeon CPUs gained AVX512 support. Which the direct competitor -Epyc Naples- didn't have.
On the other hand, Intel moved from 4 to 6 memory channels. The direct competitor -Epyc Naples- has 8 memory channels.
Now Intel marketing and their partners had the choice to attribute performance gains between the two generations to one of these two specs. Guess which one they went with, even in cases that are not compute-bound...

When scrolling through marketing presentations, it is also worth keeping in mind that there is some kind of partnership between Intel and Ansys. I don't know the exact nature of that partnership, but there are ties between the companies https://www.ansys.com/about-ansys/pa...el-corporation
So whenever I see the Ansys logo on one of these presentations, I apply more than a pinch of salt to the interpretation of the results presented.
I am not trying to construct any kind of conspiracy theory here. All I am saying: with enough knowledge about hardware, software and charts, you can convey any message you want, without presenting data that is outright wrong.
Intel (as well as AMD) is no stranger to such methods https://www.servethehome.com/intel-p...ng-benchmarks/

I can not back up my doubts by any performance tests of my own. Or by any benchmarks from Ansys, their page for mechanical benchmarks only lists Intel systems. I just want to encourage you digest marketing presentations with the proper amount of care.

But anyway, when using 16 cores max, you are not looking at a game-changing performance difference between two dual-socket systems. And with the latest refresh to Intels CPUs, price/performance has been improved by a lot. The results presented here are probably as large as the gap will get: Xeon Gold Cascade Lake vs Epyc Rome - CFX & Fluent - Benchmarks (Windows Server 2019)
I know which CPUs I would go with, but I am starting to think that I am becoming biased towards AMD.

[evcelica]

Just wanted to say thank you for the response Alex.

That CFX performance of EPYC is hard to ignore. 42% faster @ 8 cores!
But mechanical will be used more for these machines than CFX.



If I look @ these benchmark results from this site: Epyc 7551 vs 6850K; Ansys Mechanical Bench


I can compare them to each other and to ANSYS published data, and the EPYC (Naples) performs pretty badly if I am interpreting everything correctly:
XEON Gold just destroys everything. Attached is the plotted data for cg-1 and cg-2.
I am just guessing the the EPYCs were solves with 32 cores, which is a result from DUKE on this forum.

Why is XEON GOLD so much better than E5-v4? AVX512 or memory bandwidth? If the latter, why is EPYC Naples so poor then? I really with we could get one ROME benchmark to compare to the ANSYS benchmarks on a core per core basis.

How does anyone interpret these results?

[flotus1]

I won't even try to make any sense of the numbers posted in that thread. I'm an engineer, not a psychic

[arphy]

There is a detail comparison in the slide you mentioned. You just missed page 34.


In summary, the old EPYC (Naples) can be upto 1.4X, faster or slower when compare to GOLD 6148. They are priced the same now. For fluid it is definitely slower.


Rome is much ... much better than Naples (see p.34).

Let say, 15% improved in IPC and the frequency improved a lot. (7402 was increased by 40% from 7401). So we can say that, for the same equivalent model, Rome is 50% faster than Naples.



Still, if we look at your choice, the budget should be around 1,000 USD per cpu.


This is cleary shown on p.34 where the 7542 (3400USD) outperforms 22-core 6248(3000USD) by 20%. (Note* They got the 7542 frequency wrong)
Now, let us do a simple calculation.
According to the slide, we can conclude that 1.2XINTEL GOLD 20-core@2.5GHz = ROME 32-core@2.3GHz

Thus

1 CORE*GHZ (GOLD) = 1.23 CORE*GHZ (ROME).



For 7302, you have 48 CORE*GHZ, you'll need 31 CORE-GHz of Gold to match.
The lowest comparable gold 6226R will give you 46.4 CORE-GHz (Gold) which is a bout 50% faster than 7302.

The 6226R will cost you 500USD more per core. However. this would be around just1,000 out of 15K - 25K server.



I suspect that the 1 CORE*GHZ of 7302 should be better than 1 CORE*GHZ of 7452, due to memory bandwith/core is better.
Thus choosing 7302 may be anywhere from 10% - 30% slower than 6226R,with just 10% budget saved.


So at this time, 6226R is a better choice than 7302. The story might change again when ZEN3 comes out.

[evcelica]

Thanks for the info,
We ended up going with two of the dual XEON GOLD 6226R machines. Running Windows 10 Pro.
I'll run them in parallel with QDR Infiniband.

[evcelica]

Just an update, I was able to benchmark the dual XEON 6226R vs a Dual EPYC machine for a single ANSYS mechanical benchmark.
I used 16 cores on both machines, and the XEON was faster by 25%.
This is most surely due to AVX512 acceleration which only the XEON has.

[flotus1]

No offense, but there could be a multitude of reasons why one CPU runs the code faster than the other. AVX-512 is only one potential reason. I would be more reluctant to jump to such a conclusion.

[evcelica]

I don't think I'm jumping to a conclusion, but perhaps I didn't explain all my reasoning.

First off, ANSYS states Intel CPUs with AVX512 support are recommended for mechanical as it uses AVX512 acceleration.

I have benchmarked many machines in the past, and these XEONs are much faster than other machines for ANSYS mechanical when AVX512 acceleration is involved.
Much faster than they should be considering their clock speed, memory bandwidth etc.
During the mechanical benchmark I can see the clock speed for the XEONs is lower than its base frequency, which I believe is because it is using AVX512, which lowers the clock speed. The frequency is not lower than base, and is usually higher (turbo) when running other benchmarks, such as CFX.
Other machines are faster than these XEONs in CFX, and the performance is about where I would expect it to be considering clock speed, memory etc. But for ANSYS mechanical, these new XEONs perform much better than they "should" comparing specs and other benchmarks.
The only thing that explains it is AVX512, which i know it is using, and ANSYS even states it accelerates mechanical solving.

[flotus1]

Well, those are some solid indicators that AVX-512 is actually involved.

[evcelica]

Originally Posted by flotus1

Well, those are some solid indicators that AVX-512 is actually involved.

That's what I thought as well. So in the end, I am glad I got XEONs instead of the EPYCs, as most work done by our 4 person group is mechanical. I'm the only real CFD user, but do mechanical as well.
Getting the same machines as them (With XEONs) allows me to distribute to their machines as well for both mechanical and CFX. I know CFX can distribute between different chipsets, but not mechanical.