CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > General Forums > Hardware

OpenFOAM benchmarks on various hardware

Register Blogs Community New Posts Updated Threads Search

Like Tree549Likes

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   August 7, 2024, 12:01
Default
  #781
Super Moderator
 
flotus1's Avatar
 
Alex
Join Date: Jun 2012
Location: Germany
Posts: 3,428
Rep Power: 49
flotus1 has a spectacular aura aboutflotus1 has a spectacular aura about
Quote:
Originally Posted by oswald View Post
Results for OpenFOAM 9 on a dual EPYC 9684X with 5600 MHz DDR5 RAM:
Nice. May I ask which motherboard you are using, and how you got the memory to run at DDR5-5600?
flotus1 is offline   Reply With Quote

Old   August 8, 2024, 04:35
Default
  #782
Member
 
Join Date: Sep 2010
Location: Leipzig, Germany
Posts: 96
Rep Power: 16
oswald is on a distinguished road
Quote:
Originally Posted by flotus1 View Post
Nice. May I ask which motherboard you are using, and how you got the memory to run at DDR5-5600?

Sorry, my bad. The memory is able to run at 5600 MHz, but mainboard (Gigabyte MZ73-LM0) and CPU only allow for 4800. I'll edit the original post.
oswald is offline   Reply With Quote

Old   August 22, 2024, 07:42
Default
  #783
New Member
 
Join Date: Aug 2024
Posts: 3
Rep Power: 2
Residual56 is on a distinguished road
CPU: Intel i7 7700K (4C/8T - 4.5 GHz)
RAM: 2x16GB DDR4 3600 MHz
Openfoam9



Code:
Cores    Snappy      simpleFoam

1             995.47       616.75

2             677.98        328.12

4             414.04        245.16
wkernkamp and Crowdion like this.
Residual56 is offline   Reply With Quote

Old   August 22, 2024, 08:23
Default OpenFOAM Benchmark
  #784
Member
 
Marco Bernardes
Join Date: May 2009
Posts: 59
Rep Power: 17
masb is on a distinguished road
Hi,

I recently purchased a new PC and am eager to benchmark its performance using OpenFOAM. Could you please guide me on where I can download the standard benchmark files?

I appreciate your assistance and look forward to your response.

Thanks in advance!
masb is offline   Reply With Quote

Old   August 22, 2024, 18:48
Lightbulb
  #785
New Member
 
Daniel
Join Date: Jun 2010
Posts: 14
Rep Power: 16
DVSoares is on a distinguished road
Quote:
Originally Posted by masb View Post
Hi,

I recently purchased a new PC and am eager to benchmark its performance using OpenFOAM. Could you please guide me on where I can download the standard benchmark files?

I appreciate your assistance and look forward to your response.

Thanks in advance!

Hello! Unless things have changed, all you need is to go through the first post and use the script indicated in the last paragraph, the "Moderator Note". It should run normally.

There were some tweaks to increase the cell count on some runs in recent messages, but as far as I could understand, those have not become the norm. If I am wrong, someone will jump in and indicate otherwise
masb likes this.
DVSoares is offline   Reply With Quote

Old   August 23, 2024, 04:37
Default
  #786
Senior Member
 
Will Kernkamp
Join Date: Jun 2014
Posts: 372
Rep Power: 14
wkernkamp is on a distinguished road
Quote:
Originally Posted by DVSoares View Post
Hello! Unless things have changed, all you need is to go through the first post and use the script indicated in the last paragraph, the "Moderator Note". It should run normally.

There were some tweaks to increase the cell count on some runs in recent messages, but as far as I could understand, those have not become the norm. If I am wrong, someone will jump in and indicate otherwise

The latest openfoam versions have some keyword changes. I made a script for OF 2112 here:
Quote:
Originally Posted by wkernkamp View Post
Here it is. Run it with run.tst The file has a list of numbers of nodes at the beginning. A little further down you can set prep=0 to avoid recalculating the mesh if you already have a valid mesh. In the loop for running openFOAM itself, I remove the simpleFoam log files, etc to allow a rerun to proceed. On the first try, these files are not there yet, so you see an error message that you can ignore.

Press the blue >> arrow.
masb and DVSoares like this.
wkernkamp is offline   Reply With Quote

Old   August 26, 2024, 08:16
Default
  #787
New Member
 
Join Date: Aug 2024
Posts: 3
Rep Power: 2
Residual56 is on a distinguished road
CPU: AMD Ryzen 7 7800X3D (8C/16T - PBO: Auto)
RAM: 2x32GB 6000 MHz CL30

Code:
Cores |  Snappy  |  simpleFoam
1  |  689.10  |  413

2  |  460.64  |  199

4  |  271.69  |  122

8  |  179.10  |  93
wkernkamp and Crowdion like this.
Residual56 is offline   Reply With Quote

Old   August 27, 2024, 18:48
Default
  #788
Senior Member
 
Will Kernkamp
Join Date: Jun 2014
Posts: 372
Rep Power: 14
wkernkamp is on a distinguished road
Quote:
Originally Posted by Residual56 View Post
CPU: AMD Ryzen 7 7800X3D (8C/16T - PBO: Auto)
RAM: 2x32GB 6000 MHz CL30

Code:
Cores |  Snappy  |  simpleFoam
1  |  689.10  |  413

2  |  460.64  |  199

4  |  271.69  |  122

8  |  179.10  |  93

Nice result. I think you could get get my numbers (below) if you simply set the multiplier to 62 from 60 to get your mempry to run at 6200.


Code:
Meshing Times:
1 566.38
2 386.89
4 231.05
6 200.14
8 155.77
Flow Calculation:
1 301.9
2 174.88
4 107.53
6 93.3
8 88.47
wkernkamp is offline   Reply With Quote

Old   September 4, 2024, 16:22
Default Data aggregation of OpenFOAM benchmarks
  #789
New Member
 
Dmitry
Join Date: Feb 2013
Posts: 29
Rep Power: 13
techtuner is on a distinguished road
Into message attachment are two parts of zip archive (due to strict limitations to file size on cfd-online) with aggregated all hardware performance data in OpenFOAM that is presented into thread "OpenFOAM benchmarks on various hardware".
To unzip file rename files to *.zip.001 and *.zip.002.
Attached Files
File Type: zip OpenFOAM_performance_comparison_v10.001.zip (100.0 KB, 28 views)
File Type: zip OpenFOAM_performance_comparison_v10.002.zip (99.1 KB, 28 views)
andy_ and wkernkamp like this.
techtuner is offline   Reply With Quote

Old   September 5, 2024, 02:12
Default
  #790
Senior Member
 
Will Kernkamp
Join Date: Jun 2014
Posts: 372
Rep Power: 14
wkernkamp is on a distinguished road
Quote:
Originally Posted by techtuner View Post
Into message attachment are two parts of zip archive (due to strict limitations to file size on cfd-online) with aggregated all hardware performance data in OpenFOAM that is presented into thread "OpenFOAM benchmarks on various hardware".
To unzip file rename files to *.zip.001 and *.zip.002.
Nice job!


I "cat" them together and opened the spreadsheet.
wkernkamp is offline   Reply With Quote

Old   September 5, 2024, 14:45
Default
  #791
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
My current workstation had some problems yesterday prompting me to start gathering some information for a possible replacement. It consists of 2 x Xeon E5-2620v4, 2.1 GHz, 2x8 cores, 2x4 memory channels, 8x8GB DDR4-2133 memory, SSD, HD, Quadro graphics,... and cost £3k 8 years ago.

Not an OpenFOAM user and so installed the openfoam11 binary package for ubuntu 24.04. Compiling code for the specific processor would likely bring some improvement:

Code:
Cores Time     Eff.
 1   1128.43   1.00
 2    599.21   0.94
 4    268.17   1.05
 8    152.54   0.92
16    105.32   0.67
Results are pretty much as expected for a reasonably sized implicit CFD simulation. Some cache effects and parallel efficiency starting to fall when the communications stops increasing with the number crunching. An implicit CFD job of reasonable size taking all cores and running at >50% efficiency was the objective and would be the objective for the replacement workstation.

Given 30% inflation can £4k (including storage and graphics) get me the roughly tenfold improvement history would suggest? Looking for doubling memory channels, doubling memory speed, doubling number of cores,... using the cheaper recently released server processors. Suggestions welcome.
andy_ is offline   Reply With Quote

Old   September 5, 2024, 19:54
Default
  #792
Senior Member
 
Will Kernkamp
Join Date: Jun 2014
Posts: 372
Rep Power: 14
wkernkamp is on a distinguished road
Quote:
Originally Posted by andy_ View Post
My current workstation had some problems yesterday prompting me to start gathering some information for a possible replacement. It consists of 2 x Xeon E5-2620v4, 2.1 GHz, 2x8 cores, 2x4 memory channels, 8x8GB DDR4-2133 memory, SSD, HD, Quadro graphics,... and cost £3k 8 years ago.


Given 30% inflation can £4k (including storage and graphics) get me the roughly tenfold improvement history would suggest? Looking for doubling memory channels, doubling memory speed, doubling number of cores,... using the cheaper recently released server processors. Suggestions welcome.
I don't know what your problems are, but your current system would be about 60% faster if you replaced the cpus with E5-2697A v4 processors (or the cheaper E5-2683 v4 almost as fast) Make sure that you upgrade bios to a version that can handle the v4 cpus.


I think the amount you suggest should be able to buy an older generation dual EPYC system that would reach your goals.
wkernkamp is offline   Reply With Quote

Old   September 6, 2024, 08:30
Default
  #793
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
Quote:
Originally Posted by wkernkamp View Post
[...] but your current system would be about 60% faster if you replaced the cpus with E5-2697A v4 processors (or the cheaper E5-2683 v4 almost as fast) Make sure that you upgrade bios to a version that can handle the v4 cpus.
My processor is a cheap one from the v4 2600 series and if I was to compile openfoam with the appropriiate flags the figures above would improve. Increasing the CPU clockspeed would also improve the figures. My figures are not proper benchmark ones but they are already close to those for more expensive Xeon processors with similar memory configurations. This is because the performance of large implicit CFD solvers is mainly limited by memory performance.

Quote:
Originally Posted by wkernkamp View Post
I think the amount you suggest should be able to buy an older generation dual EPYC system that would reach your goals.
I can see no evidence of this which is why I posted. If we look at the results from post #780:

Results for OpenFOAM 9 on a dual EPYC 9684X with 4800 MHz DDR5 RAM:

Code:
Cores Time   Eff.
  1  546.46  1.00
  4  110.53  1.26
  8   51.49  1.32
 16   27.53  1.24
 32   15.38  1.11
 64    8.67  0.98
128    6.49  0.65
192    6.43  0.44
This is a current generation 96 core EPYC with 12 memory channels. It is around 10 times faster with 48 cores than mine with 16. This is also where the efficiency is starting fall as it does on mine. Note the test problem is a bit too small to be respresentative of what tends be run on that many cores. The efficiency drop is likely to be larger with a larger job.

My workstation provider will supply a 2 x 24 core current generation EPYC for £12k. The previous generation EPYC has 8 memory channels of DDR4-3200 making 2 x 16 cores a better balanced configuration. This will deliver perhaps half the performance for £8k but dates back to 2021. This is why I am posting. The price of a basic CFD workstation equivalent to my current one that keeps up with evolving performance appears to have risen 3-4 fold over the last 8 years. Perhaps I am missing something such as looking at the wrong hardware. I sure hope so. I probably ought to post this as a separate topic.
andy_ is offline   Reply With Quote

Old   September 6, 2024, 09:05
Default
  #794
Member
 
Join Date: Nov 2019
Posts: 98
Rep Power: 7
FliegenderZirkus is on a distinguished road
We have recently built a workstation for ~£4k at my company using 2xEPYC7532. This might give you about sixfold faster solve times: 105s/17s=6. There are several benchmarks of this CPU in this thread. The CPU was bought used from ebay but all other components were new. Final price will depend on how much memory you need. We wanted 1TB, but with e.g. 256 GB your build might even be quite a bit cheaper than £4k
andy_ likes this.
FliegenderZirkus is offline   Reply With Quote

Old   September 6, 2024, 13:04
Default
  #795
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
Quote:
Originally Posted by FliegenderZirkus View Post
We have recently built a workstation for ~£4k at my company using 2xEPYC7532. This might give you about sixfold faster solve times: 105s/17s=6. There are several benchmarks of this CPU in this thread. The CPU was bought used from ebay but all other components were new. Final price will depend on how much memory you need. We wanted 1TB, but with e.g. 256 GB your build might even be quite a bit cheaper than £4k
Second hand isn't a valid comparison given the first hit on ebay was for a slightly younger dual Xeon with more cores than mine, more memory, better graphics card, Windows 11 pro (mine needs daughter card), etc... for £674. Plenty of similar machines around this price. I haven't checked but perhaps twice the CFD performance for 1/5th price looks possible with some effort secondhand. So that is a ten times improvement until the hardware fails early of course (fingers crossed but I only seem to be seeing graphic card errors at the moment).

Buying refurbished seems the rational thing to do at present given the new price for current technology. Unfortunately second hand prices will inevitably rise unless the high new prices correct themselves. Perhaps this will happen if the Chinse flood the market with cheap Risc V processors in a year or two. Then again they may opt to join in and charge high prices as well. Ditto and perhaps earlier for non-Apple ARM processors.

I have been involved in buying computers for CFD for over 40 years and this is the first time I have seen something like this (assuming I am not missing too much after a brief look). I suspect it may be more to do with a change in processor pricing policy rather than engineering costs. Don't know.
andy_ is offline   Reply With Quote

Old   September 8, 2024, 08:13
Default
  #796
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
After looking some more and getting a better understanding of computer hardware price changes it looks as if a bit over £3k can get a 4-5 times quicker workstation (only new components) using 3 year old technology. This is inline with my expectations and what I have seen pretty much over the last 40 years buyng hardware for CFD. This excludes the tenfold drop in price in the mid 90s when moving from specialised scientific hardware to consumer hardware (and losing support and relibility).

If we look at current technology and a tenfold(ish) increase in performance a competive price seems to be a bit under £7k. Roughly a doubling of price since covid. The wide range in advertised price (my old supplier's price is particularly high) suggests a correction may be underway which would be good.

My conclusion is that if my current workstation fails in the next few weeks I should buy a 6(ish) year old refurbished workstation for £600-700(ish) offering a slight increase in performance but with a shorter life. Look again at buying new in a few years.
andy_ is offline   Reply With Quote

Old   September 9, 2024, 19:42
Default
  #797
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
Just to add a comparison between ubuntu binary package and compiling source. Used the default optimisation flags in the openfoam scripts. A 6-7% improvement for low numbers of cores reducing to no improvement for the highest number of cores when the bottleneck becomes getting data to and from memory.

Code:
Cores Time-1   Time-2
 1   1128.43  1057.54
 2    599.21   563.16
 4    268.17   257.30
 8    152.54   151.72
16    105.32   105.42
andy_ is offline   Reply With Quote

Old   October 14, 2024, 15:18
Post
  #798
New Member
 
Aaron K
Join Date: Mar 2024
Posts: 2
Rep Power: 0
AK238 is on a distinguished road
Hi all,
Thanks for putting this thread together, it's been a massive help when deciding hardware options. My new (refurbished...) CFD pc, Lenovo Thinkstation P920
2x Xeon Gold 6152 (2x 22 cores), 12*32GB 2666MHz-DDR4's, Ubuntu 22.04, Openfoam v2406.

Code:
cores   Wall time (s):
------------------------
1	1030.28
2	471
4 	212.23
8 	110.98
12 	79.89
16 	62.78
20 	53.41
24 	48.22
28 	45.59
32 	43.42
36 	41.62
40 	39.97
44 	39.07
andy_ and wkernkamp like this.
AK238 is offline   Reply With Quote

Old   October 15, 2024, 05:09
Default
  #799
Senior Member
 
andy
Join Date: May 2009
Posts: 327
Rep Power: 18
andy_ is on a distinguished road
Interesting to see how 6 rather 4 memory channels, more cache, a touch more memory speed, more cores can all add up to comfortably double the speed of an efficiently (>0.9(ish)) running large(ish) implicit CFD simulation compared to mine above. Given the currently very high price for new current hardware I think your secondhand machine likely represents the sweet spot at the moment for a budget CFD workstation so long as one gets one with a reasonable life left. This is always a risk with older equipment and is of course a significant reason why this type of machine is presently on the market in reasonable numbers.

Code:
Cores  Time    Eff.
   1  1030.28  1.000
   2   471.00  1.094
   4   212.23  1.214
   8   110.98  1.160
  16    62.78  1.026
  32    43.42  0.742
  44    39.07  0.599

Last edited by andy_; October 15, 2024 at 13:30.
andy_ is offline   Reply With Quote

Old   October 15, 2024, 06:15
Default
  #800
New Member
 
Aaron K
Join Date: Mar 2024
Posts: 2
Rep Power: 0
AK238 is on a distinguished road
Yeah indeed. I followed the ethos of trying to maximise cpu cache and memory bandwidth as much as my budget allowed. with the higher core count of the 6152, Intel supplies a tiny bit more in the L3 cache. However i imagine a lot of the heavy lifting is done with the memory bandwidth. When researching CPUs, i found conflicting specs of bandwidth figures between internet sources. But it seemed like later generations such as 62** sometimes had slower RAM speeds of DDR4-2400 which translated into lower memory bandwidths. So when looking for a new system keep that in mind!
AK238 is offline   Reply With Quote

Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On


Similar Threads
Thread Thread Starter Forum Replies Last Post
How to contribute to the community of OpenFOAM users and to the OpenFOAM technology wyldckat OpenFOAM 17 November 10, 2017 16:54
UNIGE February 13th-17th - 2107. OpenFOAM advaced training days joegi.geo OpenFOAM Announcements from Other Sources 0 October 1, 2016 20:20
OpenFOAM Training Beijing 22-26 Aug 2016 cfd.direct OpenFOAM Announcements from Other Sources 0 May 3, 2016 05:57
New OpenFOAM Forum Structure jola OpenFOAM 2 October 19, 2011 07:55
Hardware for OpenFOAM LES LijieNPIC Hardware 0 November 8, 2010 10:54


All times are GMT -4. The time now is 19:45.