[Andrew Norfolk]

Hi,

My company is looking to purchase a new HPC workstation to run ANSYS on.

Due to complexities with how our IT works, this will be a standalone Windows based workstation (single node) operated locally and not a Cluster / server.

I am looking for a bit of help to get the best performance for our workloads within this limited format and our available licenses (see below). I've read a lot of the posts on this forum and I feel like a know mostly what we need, but I'd really appreciate a bit of clarity on a couple of points.

Typical Workloads:

• Large CFD models in Fluent (50M Elements+)
• Mostly incompressible flow with conjugate heat transfer and occasionally some radiation modelling
• Some FEA work (ANSYS Mechanical + LS-DYNA)

Our Licenses include 3X HPC Packs which allow us to solve with up to 136 cores when all packs are available. In theory we could purchase 2X 64 core processors (e.g. 9554s) to use all this, but I've read that scaling performance can really drop off with high core counts on a single node as eventually you become memory bandwidth limited.

We have another older workstation running that often takes one of our HPC packs for smaller bits of simulation work, I suspect the majority of the time we'll only have access to 36 cores of licensing (2X HPC packs) so purchasing 2X 9554s seems potentially wasteful and I'd rather focus on a workstation that gives us the best performance under this 36 core scenario.

Back in the day I was told the optimal performance per core with dual EPYC was achieved using a 2X 32 cores (e.g. 9374Fs)... is this still the case or has this increased due to the enhanced memory bandwidth offered by the newer generation EPYCs?

I've also seen that there are now the more expensive "X" versions of the EPYCs with a huge amount of L3 cache (although lower clock speeds) that in theory could offer big performance gains in memory bandwidth limited applications and therefore per core performance for CFD work. I find these parts confusing as in almost all open benchmarks the higher frequency versions (e.g. 2X 9374F) seems to be better performing than the additional cache 'X' versions (e.g. 9384X)?

https://openbenchmarking.org/vs/Proc...+9384X+32-Core

Proposed Workstation:

Processors: 2X AMD EPYC 9374F (32C, 3.85GHz Base, 4.3GHz Turbo, 256MB Cache) OR 2X AMD EPYC 9384X (32C, 3.1GHz Base, 3.9GHz Turbo, 768MB Cache)
Memory: 768GB (24x 32GB) 4800MHz ECC Registered DDR5 Memory (Dual-Rank Modules)
Graphics Card: NVIDIA GeForce RTX 4070 Super - 12GB GDDR6X Graphics Card (7,168x Cores, Top Down) - WE ARE NOT PLANNING TO USE GPU ACCELERATION BUT STILL NEED A GPU
Hard Drive: M.2 Slot 1: 2TB Samsung 990 PRO M.2 PCIe 4.0 x4 NVMe SSD - 7,450MBps Read, 6,900MBps Write, 1.4M/1.55M IOPs

The big decision is whether we specify the 9374F or the 9384X? The latter is more expensive and I see no evidence of if offering performance benefits however it's possible the benchmarks I am looking at our not relevant to our workloads. I was hoping people here could say whether I am missing something? We can afford the 'X' version but we don't want to spend the extra money if it offers worst performance (obviously).

Thanks

[FliegenderZirkus]

The 2x9374F completes the motorbike OpenFOAM benchmark in 11.07 seconds: OpenFOAM benchmarks on various hardware
I don't think we have results for the 9384X yet, but 2x7573X (so one generation older) completes the benchmark in 11.55 seconds: OpenFOAM benchmarks on various hardware

9384X has much better specs than 7573X (for example 12xDDR5 vs 8xDDR4 memory channels), so I think this indicates that the 9384X will be significantly faster than the 9374F.

[Andrew Norfolk]

This makes a lot of sense FilegenderZirkus. It's still a bit of a "should be" rather than 100% certain, I guess I'll be the guinea pig and get the benchmark data so we can confirm the 2X 9384X solution is better than the 2X 9374F.

[flotus1]

How much the additional L3 cache helps exactly, can vary from case to case.
But on average, the Epyc "X" SKUs with more cache are faster than the high frequency "F" SKUs.
So unless you know for a fact that most of the cases you run are outliers, which benefit from slightly higher frequency, just get the 9384X. It is without a doubt the best CPU for CFD with expensive per-core licenses.

The price difference between the CPUs should not matter in the grand scheme of things. For the whole workstation, the difference is maybe 10%. And that's before we even consider the software licenses. I don't know how much you have to pay Ansys annually. But slightly more expensive hardware, to use the expensive software more efficiently, should be a no-brainer.

Edit: I see the link to openbenchmarking in the initial post. Don't be confused by any results you find there. For OpenFOAM, they are questionable to say the least.

[Andrew Norfolk]

Thanks Flotus, I agree with your points about hardware costs not really being important relative to licensing costs. We're going for pure performance within our licensing framework, but I was guided by those open benchmarks which I'm glad to hear you think they're a bit questionable. It never made sense to me when marketing material from AMD for the Genoa-X chips specifically calls out CFD as it's target market.

Most the work we do would be considered "standard" by any CFD professional so I guess we'll give the 9384X a go!

[ameiris]

Hello everyone!
I am also considering buying such a workstation. The only thing I am worried about is the OS. Will this CPU support Windows 11 Pro? I've seen some people using it with Epyc CPUs, but some are having problems as well. Do you have any experience with this? Or it is better to choose Threadripper PRO 7985WX? Just want to check before I buy

[wkernkamp]

Originally Posted by ameiris

Hello everyone!
I am also considering buying such a workstation. The only thing I am worried about is the OS. Will this CPU support Windows 11 Pro? I've seen some people using it with Epyc CPUs, but some are having problems as well. Do you have any experience with this? Or it is better to choose Threadripper PRO 7985WX? Just want to check before I buy

I have never heard of Windows not being supported on amd64 platforms. However, there could be some issue we don't know about. The best way forward for you would be to contact Microsoft Customer Support.

[Andrew Norfolk]

We built a similar workstation in the past with 3rd generation EPYCs (2X 75F3) and it worked fine in Windows.