[flotus1]

Originally Posted by oswald

Results for OpenFOAM 9 on a dual EPYC 9684X with 5600 MHz DDR5 RAM:

Nice. May I ask which motherboard you are using, and how you got the memory to run at DDR5-5600?

[oswald]

Originally Posted by flotus1

Nice. May I ask which motherboard you are using, and how you got the memory to run at DDR5-5600?

Sorry, my bad. The memory is able to run at 5600 MHz, but mainboard (Gigabyte MZ73-LM0) and CPU only allow for 4800. I'll edit the original post.

[Residual56]

CPU: Intel i7 7700K (4C/8T - 4.5 GHz)
RAM: 2x16GB DDR4 3600 MHz
Openfoam9

Cores    Snappy      simpleFoam

1             995.47       616.75

2             677.98        328.12

4             414.04        245.16

[masb]

Hi,

I recently purchased a new PC and am eager to benchmark its performance using OpenFOAM. Could you please guide me on where I can download the standard benchmark files?

I appreciate your assistance and look forward to your response.

Thanks in advance!

[DVSoares]

Originally Posted by masb

Hi,

I recently purchased a new PC and am eager to benchmark its performance using OpenFOAM. Could you please guide me on where I can download the standard benchmark files?

I appreciate your assistance and look forward to your response.

Thanks in advance!

Hello! Unless things have changed, all you need is to go through the first post and use the script indicated in the last paragraph, the "Moderator Note". It should run normally.

There were some tweaks to increase the cell count on some runs in recent messages, but as far as I could understand, those have not become the norm. If I am wrong, someone will jump in and indicate otherwise

[wkernkamp]

Originally Posted by DVSoares

Hello! Unless things have changed, all you need is to go through the first post and use the script indicated in the last paragraph, the "Moderator Note". It should run normally.

There were some tweaks to increase the cell count on some runs in recent messages, but as far as I could understand, those have not become the norm. If I am wrong, someone will jump in and indicate otherwise

The latest openfoam versions have some keyword changes. I made a script for OF 2112 here:

Originally Posted by wkernkamp

Here it is. Run it with run.tst The file has a list of numbers of nodes at the beginning. A little further down you can set prep=0 to avoid recalculating the mesh if you already have a valid mesh. In the loop for running openFOAM itself, I remove the simpleFoam log files, etc to allow a rerun to proceed. On the first try, these files are not there yet, so you see an error message that you can ignore.

Press the blue >> arrow.

[Residual56]

CPU: AMD Ryzen 7 7800X3D (8C/16T - PBO: Auto)
RAM: 2x32GB 6000 MHz CL30

Cores |  Snappy  |  simpleFoam
1  |  689.10  |  413

2  |  460.64  |  199

4  |  271.69  |  122

8  |  179.10  |  93

[wkernkamp]

Originally Posted by Residual56

CPU: AMD Ryzen 7 7800X3D (8C/16T - PBO: Auto)
RAM: 2x32GB 6000 MHz CL30

Cores |  Snappy  |  simpleFoam
1  |  689.10  |  413

2  |  460.64  |  199

4  |  271.69  |  122

8  |  179.10  |  93

Nice result. I think you could get get my numbers (below) if you simply set the multiplier to 62 from 60 to get your mempry to run at 6200.

Meshing Times:
1 566.38
2 386.89
4 231.05
6 200.14
8 155.77
Flow Calculation:
1 301.9
2 174.88
4 107.53
6 93.3
8 88.47

[techtuner]

Into message attachment are two parts of zip archive (due to strict limitations to file size on cfd-online) with aggregated all hardware performance data in OpenFOAM that is presented into thread "OpenFOAM benchmarks on various hardware".
To unzip file rename files to *.zip.001 and *.zip.002.

[wkernkamp]

Originally Posted by techtuner

Into message attachment are two parts of zip archive (due to strict limitations to file size on cfd-online) with aggregated all hardware performance data in OpenFOAM that is presented into thread "OpenFOAM benchmarks on various hardware".
To unzip file rename files to *.zip.001 and *.zip.002.

Nice job!


I "cat" them together and opened the spreadsheet.

[andy_]

My current workstation had some problems yesterday prompting me to start gathering some information for a possible replacement. It consists of 2 x Xeon E5-2620v4, 2.1 GHz, 2x8 cores, 2x4 memory channels, 8x8GB DDR4-2133 memory, SSD, HD, Quadro graphics,... and cost £3k 8 years ago.

Not an OpenFOAM user and so installed the openfoam11 binary package for ubuntu 24.04. Compiling code for the specific processor would likely bring some improvement:

Cores Time     Eff.
 1   1128.43   1.00
 2    599.21   0.94
 4    268.17   1.05
 8    152.54   0.92
16    105.32   0.67

Results are pretty much as expected for a reasonably sized implicit CFD simulation. Some cache effects and parallel efficiency starting to fall when the communications stops increasing with the number crunching. An implicit CFD job of reasonable size taking all cores and running at >50% efficiency was the objective and would be the objective for the replacement workstation.

Given 30% inflation can £4k (including storage and graphics) get me the roughly tenfold improvement history would suggest? Looking for doubling memory channels, doubling memory speed, doubling number of cores,... using the cheaper recently released server processors. Suggestions welcome.

[wkernkamp]

Originally Posted by andy_

My current workstation had some problems yesterday prompting me to start gathering some information for a possible replacement. It consists of 2 x Xeon E5-2620v4, 2.1 GHz, 2x8 cores, 2x4 memory channels, 8x8GB DDR4-2133 memory, SSD, HD, Quadro graphics,... and cost £3k 8 years ago.


Given 30% inflation can £4k (including storage and graphics) get me the roughly tenfold improvement history would suggest? Looking for doubling memory channels, doubling memory speed, doubling number of cores,... using the cheaper recently released server processors. Suggestions welcome.

I don't know what your problems are, but your current system would be about 60% faster if you replaced the cpus with E5-2697A v4 processors (or the cheaper E5-2683 v4 almost as fast) Make sure that you upgrade bios to a version that can handle the v4 cpus.


I think the amount you suggest should be able to buy an older generation dual EPYC system that would reach your goals.

[andy_]

Originally Posted by wkernkamp

[...] but your current system would be about 60% faster if you replaced the cpus with E5-2697A v4 processors (or the cheaper E5-2683 v4 almost as fast) Make sure that you upgrade bios to a version that can handle the v4 cpus.

My processor is a cheap one from the v4 2600 series and if I was to compile openfoam with the appropriiate flags the figures above would improve. Increasing the CPU clockspeed would also improve the figures. My figures are not proper benchmark ones but they are already close to those for more expensive Xeon processors with similar memory configurations. This is because the performance of large implicit CFD solvers is mainly limited by memory performance.

Originally Posted by wkernkamp

I think the amount you suggest should be able to buy an older generation dual EPYC system that would reach your goals.

I can see no evidence of this which is why I posted. If we look at the results from post #780:

Results for OpenFOAM 9 on a dual EPYC 9684X with 4800 MHz DDR5 RAM:

Cores Time   Eff.
  1  546.46  1.00
  4  110.53  1.26
  8   51.49  1.32
 16   27.53  1.24
 32   15.38  1.11
 64    8.67  0.98
128    6.49  0.65
192    6.43  0.44

This is a current generation 96 core EPYC with 12 memory channels. It is around 10 times faster with 48 cores than mine with 16. This is also where the efficiency is starting fall as it does on mine. Note the test problem is a bit too small to be respresentative of what tends be run on that many cores. The efficiency drop is likely to be larger with a larger job.

My workstation provider will supply a 2 x 24 core current generation EPYC for £12k. The previous generation EPYC has 8 memory channels of DDR4-3200 making 2 x 16 cores a better balanced configuration. This will deliver perhaps half the performance for £8k but dates back to 2021. This is why I am posting. The price of a basic CFD workstation equivalent to my current one that keeps up with evolving performance appears to have risen 3-4 fold over the last 8 years. Perhaps I am missing something such as looking at the wrong hardware. I sure hope so. I probably ought to post this as a separate topic.

[FliegenderZirkus]

We have recently built a workstation for ~£4k at my company using 2xEPYC7532. This might give you about sixfold faster solve times: 105s/17s=6. There are several benchmarks of this CPU in this thread. The CPU was bought used from ebay but all other components were new. Final price will depend on how much memory you need. We wanted 1TB, but with e.g. 256 GB your build might even be quite a bit cheaper than £4k

[andy_]

Originally Posted by FliegenderZirkus

We have recently built a workstation for ~£4k at my company using 2xEPYC7532. This might give you about sixfold faster solve times: 105s/17s=6. There are several benchmarks of this CPU in this thread. The CPU was bought used from ebay but all other components were new. Final price will depend on how much memory you need. We wanted 1TB, but with e.g. 256 GB your build might even be quite a bit cheaper than £4k

Second hand isn't a valid comparison given the first hit on ebay was for a slightly younger dual Xeon with more cores than mine, more memory, better graphics card, Windows 11 pro (mine needs daughter card), etc... for £674. Plenty of similar machines around this price. I haven't checked but perhaps twice the CFD performance for 1/5th price looks possible with some effort secondhand. So that is a ten times improvement until the hardware fails early of course (fingers crossed but I only seem to be seeing graphic card errors at the moment).

Buying refurbished seems the rational thing to do at present given the new price for current technology. Unfortunately second hand prices will inevitably rise unless the high new prices correct themselves. Perhaps this will happen if the Chinse flood the market with cheap Risc V processors in a year or two. Then again they may opt to join in and charge high prices as well. Ditto and perhaps earlier for non-Apple ARM processors.

I have been involved in buying computers for CFD for over 40 years and this is the first time I have seen something like this (assuming I am not missing too much after a brief look). I suspect it may be more to do with a change in processor pricing policy rather than engineering costs. Don't know.

[andy_]

After looking some more and getting a better understanding of computer hardware price changes it looks as if a bit over £3k can get a 4-5 times quicker workstation (only new components) using 3 year old technology. This is inline with my expectations and what I have seen pretty much over the last 40 years buyng hardware for CFD. This excludes the tenfold drop in price in the mid 90s when moving from specialised scientific hardware to consumer hardware (and losing support and relibility).

If we look at current technology and a tenfold(ish) increase in performance a competive price seems to be a bit under £7k. Roughly a doubling of price since covid. The wide range in advertised price (my old supplier's price is particularly high) suggests a correction may be underway which would be good.

My conclusion is that if my current workstation fails in the next few weeks I should buy a 6(ish) year old refurbished workstation for £600-700(ish) offering a slight increase in performance but with a shorter life. Look again at buying new in a few years.

[andy_]

Just to add a comparison between ubuntu binary package and compiling source. Used the default optimisation flags in the openfoam scripts. A 6-7% improvement for low numbers of cores reducing to no improvement for the highest number of cores when the bottleneck becomes getting data to and from memory.

Cores Time-1   Time-2
 1   1128.43  1057.54
 2    599.21   563.16
 4    268.17   257.30
 8    152.54   151.72
16    105.32   105.42

[AK238]

Hi all,
Thanks for putting this thread together, it's been a massive help when deciding hardware options. My new (refurbished...) CFD pc, Lenovo Thinkstation P920
2x Xeon Gold 6152 (2x 22 cores), 12*32GB 2666MHz-DDR4's, Ubuntu 22.04, Openfoam v2406.

cores   Wall time (s):
------------------------
1	1030.28
2	471
4 	212.23
8 	110.98
12 	79.89
16 	62.78
20 	53.41
24 	48.22
28 	45.59
32 	43.42
36 	41.62
40 	39.97
44 	39.07

[andy_]

Interesting to see how 6 rather 4 memory channels, more cache, a touch more memory speed, more cores can all add up to comfortably double the speed of an efficiently (>0.9(ish)) running large(ish) implicit CFD simulation compared to mine above. Given the currently very high price for new current hardware I think your secondhand machine likely represents the sweet spot at the moment for a budget CFD workstation so long as one gets one with a reasonable life left. This is always a risk with older equipment and is of course a significant reason why this type of machine is presently on the market in reasonable numbers.

Cores  Time    Eff.
   1  1030.28  1.000
   2   471.00  1.094
   4   212.23  1.214
   8   110.98  1.160
  16    62.78  1.026
  32    43.42  0.742
  44    39.07  0.599

[AK238]

Yeah indeed. I followed the ethos of trying to maximise cpu cache and memory bandwidth as much as my budget allowed. with the higher core count of the 6152, Intel supplies a tiny bit more in the L3 cache. However i imagine a lot of the heavy lifting is done with the memory bandwidth. When researching CPUs, i found conflicting specs of bandwidth figures between internet sources. But it seemed like later generations such as 62** sometimes had slower RAM speeds of DDR4-2400 which translated into lower memory bandwidths. So when looking for a new system keep that in mind!