[cycleback]

Does anyone have any thoughts on the AMD Threadripper Pro for CFD and FEA? It appears that Lenovo has released their P620 Thinkstations which features these processors.

https://www.amd.com/en/processors/ry...readripper-pro
https://www.lenovo.com/us/en/thinkstation-p620

They have both high clockspeeds and 8x memory channels.

My workloads are a combination of software that requires both high CPU clock speeds and memory bandwidth. The specific software includes COMSOL, ANSYS Fluent, ANSYS Maxwell, MATLAB with licenses up to 16 cores.

[cycleback]

While there are 8 memory channels I actually haven't seen documentation about the bandwidth. For example there are Epyc processors with 8 memory channels but reduced memory bandwidths.
https://www.servethehome.com/amd-epy...ory-bandwidth/
For instance the Epyc 7252 has 8 memory channels but only a memory bandwidth of 85.3 GB/s (https://www.amd.com/en/products/cpu/amd-epyc-7252) compared to say an Epyc 7302 (https://www.amd.com/en/products/cpu/amd-epyc-7302) with a memory bandwidth of 204.8 GB/s.

[flotus1]

I had not given that much thought until now, because to me, Threadripper Pro is not a particularly interesting CPU. And AMDs official specifications do not help a lot.

We could try to work out a hypothesis from L3 cache. On all 2nd gen Epyc CPUs we know so far, L3 cache is fully enabled on CCDs that have active cores. Meaning that even if only 4 out of 8 cores are active on a CCD, still the full 32MB of L3 cache are available.
Conversely, you can work out how many CCDs are active on a CPU by looking at the total amount of L3 cache: 64MB->2CCds, 128MB->4CCDs, 192MB->6CCDs, 256MB->8CCDs.
It is worth noting here that all of these SKUs technically have an 8-channel memory controller, which resides in the separate I/O-die. But SKUs with less than 4CCDs active can not leverage the full potential of this memory bandwidth, because the limited bandwidth between CCDs and I/O-die becomes an additional bottleneck.

Extrapolating this knowledge to Threadripper Pro CPUs would imply that those with less than 128MB of L3 cache do not benefit from the 8-channel memory controller. This means that only the 32-core and the 64-core variant effectively have more than 4 channels worth of memory bandwidth.

[cycleback]

Originally Posted by flotus1

I had not given that much thought until now, because to me, Threadripper Pro is not a particularly interesting CPU. And AMDs official specifications do not help a lot.

Why do you not find the Threadripper Pros interesting? I would think the combination of high CPU clock and potentially high memory bandwidth would be attractive for CFD and FEA.

Originally Posted by flotus1

We could try to work out a hypothesis from L3 cache. On all 2nd gen Epyc CPUs we know so far, L3 cache is fully enabled on CCDs that have active cores. Meaning that even if only 4 out of 8 cores are active on a CCD, still the full 32MB of L3 cache are available.
Conversely, you can work out how many CCDs are active on a CPU by looking at the total amount of L3 cache: 64MB->2CCds, 128MB->4CCDs, 192MB->6CCDs, 256MB->8CCDs.
It is worth noting here that all of these SKUs technically have an 8-channel memory controller, which resides in the separate I/O-die. But SKUs with less than 4CCDs active can not leverage the full potential of this memory bandwidth, because the limited bandwidth between CCDs and I/O-die becomes an additional bottleneck.

Extrapolating this knowledge to Threadripper Pro CPUs would imply that those with less than 128MB of L3 cache do not benefit from the 8-channel memory controller. This means that only the 32-core and the 64-core variant effectively have more than 4 channels worth of memory bandwidth.

I think your insight is probably correct. It is likely that only the 3975WX and 3995WX have the full memory bandwidth with 4 CCDs. That is disappointing.

[flotus1]

Why do you not find the Threadripper Pros interesting?

Personal preference. I prefer multi-socket systems over maximum frequency. Not that I would complain if we could have both...
Also, AMD released this much too late for no apparent reason. And its an exclusive part for Lenovo, which I personally consider a dick move.

It is likely that only the 3975WX and 3995WX have the full memory bandwidth with 4 CCDs. That is disappointing.

It is disappointing, but not necessarily a big deal for you. How much extra does Lenovo charge for 32 vs 16 cores? And how much is that compared to the cost of the whole machine?

[cycleback]

Originally Posted by flotus1

Personal preference. I prefer multi-socket systems over maximum frequency. Not that I would complain if we could have both...
Also, AMD released this much too late for no apparent reason. And its an exclusive part for Lenovo, which I personally consider a dick move.

I agree it is pretty late especially with Zen3 Epycs being released sometime soon.

Originally Posted by flotus1

It is disappointing, but not necessarily a big deal for you. How much extra does Lenovo charge for 32 vs 16 cores? And how much is that compared to the cost of the whole machine?

Just to upgrade the processor from 16 to 32 cores adds $4,585 to the system price. That is pretty steep. I was hoping to keep the entire system cost below $5,000. Ideally more like $3,500. Most of my applications are a mix of FEA and CFD including ANSYS Maxwell, Fluent, COMSOL, and others. I need a mix of high CPU single threaded speed for some workloads and multiple cores for running separate optimizations in parallel. Some of my licenses only support up to 16 cores but some have essentially unlimited core count usage.

I need to make a purchase relatively soon. Do you have a guess when the new Zen3 Epyc processors and motherboards will be available? For the Zen 2 Epycs do you have a recommendation? I have also been using Intel 10900X processors because they are relatively cheap and also have high clock speeds. I would like more cores though.

I guess another option would be an Intel W-3235. Though I can't find a prebuilt system and it really has less cores than I want.

[flotus1]

Just to upgrade the processor from 16 to 32 cores adds $4,585 to the system price. That is pretty steep.

That seems a bit excessive, even for a large OEM with exclusive access to a part. This upgrade is more like 2200€ on their German website, including taxes.

I need to make a purchase relatively soon. Do you have a guess when the new Zen3 Epyc processors and motherboards will be available?

Their track record for the first two generations of Epyc CPUs and ecosystem around them is not great. If you have any kind of vague time frame for getting a working machine under your desk, don't wait for Zen3 Epyc. Even if they released the CPUs tomorrow, It could still be months until you can build a workstation with them.

For the Zen 2 Epycs do you have a recommendation?

The closest match would be Epyc 7F52. It's a frequency-optimized 16-core CPU with maximum L3 cache. About 3200€, so probably outside your budget.

I'm afraid what you want doesn't really exist. And parts that come close are pretty much halo products, with a price tag that reflects this.

[CHUIKOV]

Hi,

sorry to hijack the thread, but I got a used 7f52. I was looking for a couple of used 7352s on ebay and the seller got confused and sent this puppy instead (so I even got half a refund lol). If anybody is interested please send a private message, looking to sell with a hefty discout from MSRP,

cheers,

[zx-81]

At first the "Threadripper PRO" specs sound quite exciting.

BUT it is probably less suited as a CFD work station than the standard EPYC Rome server processors, because

For almost the same amount of money you can buy the dual socket system (same number of total cores) offering TWICE the total memory bandwidth ( 2 x 8 memory channels )

CFD software is usually RAM bandwidth bound such that the higher base clock frequency of the Threadripper PRO is not so much of an advantage compared to the doubled bandwidth of a dual EPYC system (provided that the CFD software is NUMA aware)

Furthermore, the very high turbo clock frequency, requires an aggressively noisy cooling solution. I heard a P620 under full load an it is, in my opinion, too loud for under-the-desk work station.