[MFGT]

Originally Posted by flotus1

If you have a budget for hardware worth 7000€, I would not even bother with older generation dual-socket Epyc.
Current-gen Epyc gets you the generational improvements, making them closer to Ryzen Desktop CPU in single-core performance. While still matching the older dual-socket Epyc in terms of memory bandwidth.

Budget is still 5000€, I dont see Gen4 EPYC there, including Rack, RAM etc. necessary Updates.


So I am looking at the DUAL 7F52 (542€ each). Also for Dual CPU, 16 DIMMS are preferred over 8, right?

[flotus1]

No point in getting two CPUs, if you don't populate all their memory channels. So yes, 16 DIMMs for 1st-3rd gen dual-socket Epyc.

I took the 7000€ figure from one of your later posts.
From the history and title of this thread, and the benchmarks posted so far, I concluded that a newer generation Epyc CPU would be the best fit. Much better single-core performance than older generations, similar peak performance as 2nd gen dual-socket setup, and simpler NUMA topology.
5000€ is tight, but you can make it work with maybe a few hundred Euros extra.
Mainboard: Supermicro H13SSL-N retail https://geizhals.de/supermicro-h13ss...loc=at&hloc=de 694€
Memory: 12x 16GB DDR5-4800 reg ECC 1200€
CPU: 2500€
That leaves 600€ for the remaining stuff. I don't know what you need here.

[MFGT]

Thanks for your valuable input Alex.

I have to add that we dont build it by our own. For our company we can only buy complete systems and then its difficult to stay within the budget, when the base price for a server rack with mainboard is already 2k. Adding processors (1k), some further RAM, cooling, power supply and 2year warranty etc quickly adds up to 4.5k€ with the Dual EPYC 7F52.

source: www.serverschmiede.com

For new 4th gen solutions we quickly exceed the budget:
soruce e.g.: https://www.primeline-solutions.com/...u_series=30022

Based on my experience with the CFD software I dont want to go over 2x16 cores.


So one may argue between Dual 16Core Gen2 vs. some Single Core Gen4, but than we cant even use the 128GB DDR4 RAM we already have.

[flotus1]

My apologies, I misinterpreted a few things here.
From "We plan to buy used/refurbished hardware", I concluded that we are mostly talking about budget for the hardware itself. And that it can be bought in parts, from any source.

Let's go back to the start. Title of this thread is "Updating CFD server E5-2697AV4 to something faster (x2/x3 the speed)".
From the benchmarks shown so far, we need a good mix of single- and multi-core performance, because the code doesn't scale too well. For whatever reason that may be.
The ONLY way to achieve the stated goal is with latest-gen server CPUs.
Current desktop CPUs have the single-core performance, but lack multi-core and/or memory bandwidth.
Older generation server CPUs have the memory bandwidth, at least with a dual-socket setup. But they lack the single-core performance. And the complicated NUMA topology may also be an issue.

Long story short, I don't see any other way to achieve the stated goal.
2x 2nd gen Epyc will probably still be an upgrade compared to the current system. I just don't think that's an uplift worth 7000€.
If you don't mind, a more general comment: engineering time is expensive, especially in Germany. So are software licenses. Using both more efficiently, by using state-of-the-art computer hardware, usually pays off quickly.

[MFGT]

Originally Posted by flotus1

If you don't mind, a more general comment: engineering time is expensive, especially in Germany. So are software licenses. Using both more efficiently, by using state-of-the-art computer hardware, usually pays off quickly.

You dont have to tell me, I am just the user but cant argue about the invest limits.
Btw, the 5000€ budget is also considered incl. tax...

So yeah, I think we will go for the Dual EPYC 7F52, as newer Gen Hardware is not affordable and Desktop CPU won't match the old performance (with 8 core and license costs saving).

[wkernkamp]

Originally Posted by MFGT

Thanks a lot you for your input Will.


So the 8 Core 7800X3D would be rougly 70% of performance, right?


Regarding CFD tasks, I do flowbench simulations (20min per case) up to full cycle engine simulations with runtimes between 18-36 hours.

yes, 70% of your dual E5-2697A.

[MFGT]

We will do a test with 7800X3D today, stay tuned.

[MFGT]

We see approx. 65-75% performance with 8 cores 7800X3D compared to 32 cores Dual-E5-2697Av4.


edit: Interesting though, 8 cores with or without hyperhtreadding was nearly the same. 16 cores was only slightly faster.

[MFGT]

So hopefully last questions:

I have the feeling that the CONVERGE code isn't that hungry for Memory and the higher bandwidth or L3 cache may not be so important as for other applications (I am running "fine" with 40MB L3 cache so far). The Dual 7F52 CPUs cost 1084€, while the Dual 7452 CPUs cost 1242€, only 158€ more. In the passmark score its a plus of 18%, twice the cores but lower clock frequenzy and 128MB L3 cache only.

I am curious if that could be a better solution here? Is it worth the risk or should I simply stick to the 2x16 cores with higher single core performance?

[flotus1]

From everything I have seen so far, I have no doubt that the 7F52 is the better option.

[MFGT]

Thanks Alex, which power suppyl would you suggest?

2 x HP 800W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Platinum PLC
or
2 x HP 1000W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Titanium PLC

Is a redundant power supply necessary at all?

[flotus1]

Do you have a choice? If so, is there a price difference?

[MFGT]

Yes, we have the choice and its 192€ vs 564€, as redundant variant
So the difference is 7.4% of the budget

[flotus1]

Redundant power supplies are nice, when you don't have access to the server all the time.
Or when downtime would mean a significant disruption to your operation.
The part itself is certainly not worth 564€, even if it was new. But at this point, in for a penny, in for a pound?

[MFGT]

Originally Posted by flotus1

But at this point, in for a penny, in for a pound?

well, if its not necessary I would rather spend the money for something useful such as RAM or SSD

[flotus1]

Let me take a wild guess, tell me if I'm wrong.

You were the one who suggested buying used hardware, to make the most of the limited budget?
If there is ever something wrong with the server, like downtime caused by a broken PSU, you will never hear the end of it.

[MFGT]

Originally Posted by flotus1

Let me take a wild guess, tell me if I'm wrong.

You were the one who suggested buying used hardware, to make the most of the limited budget?
If there is ever something wrong with the server, like downtime caused by a broken PSU, you will never hear the end of it.

for sure, no way to get something usefull for 5k in new.


btw, available (redundant) power supplies are:

HP 500W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Platinum PLC 178.00 €
HP 800W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Platinum PLC 192.00 €
HP 1000W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Titanium PLC 564.00 €
HP 1400W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Platinum PLC 284.00 €
HP 1600W PSU Netzteil DL360 DL380 ML350 Gen10+ Gen11 80 Plus Platinum PLC 442.00 €



I wonder why the 1000W is so expensive (Titanium?). If the 800W is too small, we could still chose 1400W and save 280€ for new 3200MHz RAM. Currently we use 2133MHz.

[flotus1]

No idea about their price structure. I can't think of any technical reason why the 1000W option is so expensive.

An 800W power supply is enough, as long as you don't plan to add GPUs to the node.
Making enough room in the budget for DDR4-3200 is definitely the right choice.

[MFGT]

So, we packed everthing together and bought the Dual 7F52 with 800W power supply. RAM and HDD will be carry over from the old server.
We received the server yesterday, implemented the server rack into our struckture and will add HDD and RAM of the old server begin of next week (IT department is short-staffed at the moment)


Then we will see what happens.
If everything works we consider upgrading to SSD and 3200MHz RAM.

[MFGT]

So, 3 weeks later we have the server running. We needed to buy some small stuff, install windows and servises, solver software etc and vacaines played a role as well. That were the good news.

However, I think something still isnt right. Based on the benchmark score, it should be 92% faster but it is not. More or less the same speed of the original unaffected old system.

Currently, HT is still activated.
When I use 32 cores for a calculation, the active cores run on 3500MHz, the idle are at 2500MHz.

The old server ran with 2600Mhz clock speed maximum, which is 25% less. So I dont understand its the same performance. We use the old 8x16 GB 2133MHz RAM and according to our IT, it is placed in the correct sockets. Furher carry over is the data HDD.