[hokhay]

I would like to contribute to the chart with my i9-7900X
i9-7900X, 4x8GB DDR4 2133MHz
OpenFOAM 5.X, Ubuntu 16.04, Linux ITHW0115-UB 4.13.0-38-generic

# cores   Wall time (s):
     1         945.35
     2         471.4
     4         272.85
     6         220.38
     8         194.74
    10         185.35

[flotus1]

It is a bit sad to see a 900$ CPU choked by slow memory

[Morlind]

I have a combination of results. The single-core result seems too good to be true, while the others failed in error and I'm not sure why.

* edit - the error was related to the streamlines so I deleted them from the controlDict to get the 2 core. I don't know whether this is an unfair advantage so please don't take these numbers as exactly representative against others.

Intel i7 - 7700 @ 3.6 ghz w/ 32 gb RAM (4x pieces)
Results:

# cores Wall time (s):
------------------------
1 660.52
2 422.81
4

[flotus1]

Given the results for an I7-8700k with fast memory posted earlier, your results kind of make sense.

# cores   Wall time (s):
------------------------
1 531.44
2 312.15
4 249.55
6 247.83

If your CPU is really running at only 3.6GHz (did you deactivate turbo modes? why?) and memory is DDR4-2400, this is well within expectations for your setup.

[Morlind]

Does this case not function well in OpenFoam5? I can't get anything other than the single core to run (this is the second separate system with this issue). It thros errors around the streamlines and some other things. I should add that the single core shows this error but ran, the others didn't run at all. Here is the start of the error string:


--> FOAM Warning :
From function bool Foam::functionObjectList::read()
in file db/functionObjects/functionObjectList/functionObjectList.C at line 675
Caught FatalError
--> FOAM FATAL ERROR:
Attempt to return primitive entry ITstream : /home/rob/Downloads/bench_template/run_1/system/controlDict.functions.streamLines.seedSampleSet, line 45, IOstream: Version 2.0, format ASCII, line 0, OPENED, GOOD
primitiveEntry 'seedSampleSet' comprises
on line 45 the word 'uniform'
as a sub-dictionary

From function virtual const Foam::dictionary& Foam:rimitiveEntry::dict() const
in file db/dictionary/primitiveEntry/primitiveEntry.C at line 189.

nearWallFields near: Sampling 1 fields
--> FOAM Warning :
From function void Foam:article::locate(const vector&, const vector*, Foam::label, bool, Foam::string)
in file particle/particle.C at line 507
"Particle initialised with a location outside of the mesh."

[eric]

Just open the file basecase/system/controlDict and remove the lines

#include streamlines
#include wallBoundedStreamlines

That should take care of it. You should also delete all the run_* folders before rerunning the run.sh script.

By the way, I have updated the results on the benchmarks page with the latest contributions, thank you all for sharing!

[Morlind]

That's exactly what I did. Here are the results:

2x E5-2680 v2
128 gb RAM (1333)
These results were running from the SSD:

# cores Wall time (s):
------------------------
1 845.34
2 487.52
4 237.65
6 171.12
8 147.41
12 126.89
16 116.38
20 118.79

FYI the initial single core was 848.79 so the streamlines were worth a tiny bit.

Here are the results run from the HDD location. Disk interface appears to matter a touch.

# cores Wall time (s):
------------------------
1 846.06
2 469.11
4 235.33
6 172.6
8 147.98
12 128.04
16 119.72
20 127.1

[thanz]

Originally Posted by wyldckat

That and build OpenFOAM from source code, along with dedicated flags for your CPU model-type... don't use pre-built packages.

Which flags do you recommend for the EPYC 7351 for the g++ ?


Thanks in advance.

[wyldckat]

Quick answer:

Originally Posted by thanz

Which flags do you recommend for the EPYC 7351 for the g++ ?

I've never tested it myself, so @flotus1 probably knows it better than I do

According to this page: https://www.phoronix.com/scan.php?pa...lers-nov&num=1

CFLAGS/CXXFLAGS were set to "-O3 -march=znver1"

In OpenFOAM, you should configure in these files:

• wmake/rules/linux64Gcc/c++Opt
• wmake/rules/linux64Gcc/cOpt

and add it on the line for "c++OPT".

This is true as long as "WM_COMPILER" is set to "Gcc", otherwise revise the folder name accordingly.

[kuhngmbh]

2x Intel Xeon X5550, 6x 8GB DDR3 1333 MHz, OpenFOAM 5.0, Host OS: Vmware Esxi 6.0 VM OS: Ubuntu 16.04


I have removed the following lines under basecase/system/controlDict
Code:
#include streamlines
#include wallBoundedStreamlines

# cores Wall time (s):
------------------------
1: 1558.47
2: 907.58
4: 603.29
8: 352.56


2x Intel Xeon Gold 6134, 12x 32GB DDR4 2666 MHz, OpenFOAM 5.0, Host OS: Vmware Esxi 6.5 VM OS: Ubuntu 16.04

# cores Wall time (s):
------------------------
1: 874.24
2: 434.65
4: 222.26
8: 135.39

because there are other machines on this host I have tested only on 8 of the 16 cores

[Morlind]

kuhngmbh, I'm surprised by your results. I would have expected the Gold to benchmark quite a bit higher. I wonder if it's the VM setup effecting the performance? I'm sure it will scale better than my machine but I expected it to be substantially faster across the range.

[kuhngmbh]

I think it's because some other VMs are running on this machine.
If I have time on one of the following weekends, I could test the machine only with the openfoam vm and with 16 cores.

[Morlind]

Ah, of course. If others are utilizing the hardware and you only have partial access then the results make sense.

[spaceprop]

Hi,

I've been going through some homelab cluster upgrades, so I had a chance to test a ton of different processor/RAM combinations. Here are my all-core iteration/s results.

The processor(s) and RAM are called out in the graph. All OF+v1712, OpenMPI 3.1.0, CentOS 7.5. I applied the following BIOS settings to all: HT disabled (if applicable), anything related to maximum performance on. The "slight OC" for the 2x E5-4627v3's comes from the Asus Z10PE-D8's Ai overclock and "xeon turbo" BIOS settings. If RAM faster than the fastest RAM speed supported by the processor was used, I confirmed that it downclocked. An exception to this is the I7-5960X which was perfectly happy with 2400MHz RAM. I confirmed the RAM speed of all other tests was operating at its design frequency (dmidecode -t memory).



I think these pretty much fall in line with what we've been seeing from the rest of the benchmarks.

Cheers

[flotus1]

Did you actually get 2x Xeon E5-2690 running with PC3-14900? Or is the memory running at DDR3-1600? Which motherboard?

[spaceprop]

No, I did not. It downclocked to 1600MHz. Same with the PC4-19200 (2400MHz); it downclocked to 2133MHz with the e5 v3's.

I edited the post to clarify that.

[naffrancois]

Hello, here is my contribution on OpenFoam v1712 linux binaries (with controlDict modified as suggested):

Laptop Intel Core i7-7500U CPU @ 2.70GHz
# cores Wall time (s):
------------------------
1 896.86
2 553.51

2xIntel Xeon(R) CPU E5-2687W v2 @ 3.40GHz
# cores Wall time (s):
------------------------
1 786.29
2 465.3
4 215.12
8 137.39
16 118.12

2xAMD EPYC 7501 32-Core Processor +16x8Go 2666MHz + 2To SSD
# cores Wall time (s):
------------------------
1 1021.56
2 570.31
4 255.25
8 137.12
16 66.39
32 44.13
64 32.21

[gigliagarf]

Originally Posted by naffrancois

Hello, here is my contribution on OpenFoam v1712 linux binaries (with controlDict modified as suggested):

Laptop Intel Core i7-7500U CPU @ 2.70GHz
# cores Wall time (s):
------------------------
1 896.86
2 553.51

2xIntel Xeon(R) CPU E5-2687W v2 @ 3.40GHz
# cores Wall time (s):
------------------------
1 786.29
2 465.3
4 215.12
8 137.39
16 118.12

2xAMD EPYC 7501 32-Core Processor +16x8Go 2666MHz + 2To SSD
# cores Wall time (s):
------------------------
1 1021.56
2 570.31
4 255.25
8 137.12
16 66.39
32 44.13
64 32.21

Jeez thats quick. I'm over here running on a 8 year old laptop, you guys are way better funded than me.

[spaceprop]

Originally Posted by naffrancois

2xAMD EPYC 7501 32-Core Processor +16x8Go 2666MHz + 2To SSD
# cores Wall time (s):
------------------------
1 1021.56
2 570.31
4 255.25
8 137.12
16 66.39
32 44.13
64 32.21

That's faster than the dual 7601's in the main chart (data from havref posted earlier in this thread). Can you give some more info about your setup?

[spaceprop]

Updating my earlier table with more results.

X-axis is iter/s. The processor(s) and RAM are called out in the graph. All OF+v1712, OpenMPI 3.1.0, CentOS 7.5. I applied the following BIOS settings to all: HT disabled (if applicable), anything related to maximum performance on. This is slightly cleaned up from the last one and has the E5-2690v2 results added.



This one includes the results from my homelab 5-node cluster:



19.09s (5.24 ips) . Infiniband <3