[danbence]

An AMD performance brief compares the Turin 9755 (2x128) with the Genoa 9654 (2x96) and shows a 43% uplift on the composite OpenFoam benchmarks they chose.

http://www.amd.com/content/dam/amd/e...b-openfoam.pdf

[gumersindu]

Hi all,

I'm trying to run the benchmark attached on the original post named "bench_template.tar.gz" on my PC: 2 x Intel Xeon E5-2690 v4 (14 cores, 2.6 GHz, 35Mb L3) | 8 x 16GB DDR4 ECC | 1TB HDD | Ubuntu 24.04 LTS | openFOAM-2312

It looks like snappyHexMesh is failing to create the mesh. Is there an updated version maybe?

[andy_]

Originally Posted by gumersindu

It looks like snappyHexMesh is failing to create the mesh. Is there an updated version maybe?

It didn't work for me either (Ubuntu 24.04) but the test case seemed to be one of the tutorials with, if memory serves, an increased grid density so I ran that. I'm not in the office at the moment but will look for the script I used when I am back.

[andy_]

OK some of what I did is coming back but I am not an openfoam user and was hacking to get something working rather than carefully setting up a benchmark.

Version 11 and 12 of openfoam are organised significantly differently and require different scripts. Don't know about earlier versions.

I ran version 12 something like this (change the list of number of processors for the mesh, solver and writing the timing to taste):

#!/bin/bash
# PREPROCS="1 2 4 8 16"
# RUNPROCS="1 2 4 8 16"
PREPROCS=""
RUNPROCS="1"
TIMPROCS="1 2 4 8 16"

# Prepare cases
# This example runs on 1, 2 and 4 cores
for i in $PREPROCS; do
    d=run_$i
    echo "Prepare case ${d}..."
    cp -r basecase $d
    cd $d

    cp $FOAM_TUTORIALS/resources/geometry/motorBike.obj.gz constant/geometry/
    surfaceFeatures > log.surfaceFeatures 2>&1
    blockMesh > log.blockMesh 2>&1

    if [ $i -eq 1 ] 
    then
        snappyHexMesh -overwrite > log.snappyHexMesh 2>&1
    else
        sed -i "s/numberOfSubdomains.*/numberOfSubdomains ${i};/" system/decomposeParDict
        decomposePar -copyZero > log.decomposePar 2>&1
        mpirun -np ${i} snappyHexMesh -overwrite -parallel > log.snappyHexMesh 2>&1
    fi
    cd ..
done

# Run cases
for i in $RUNPROCS; do
    echo "Run for ${i}..."
    cd run_$i
    if [ $i -eq 1 ] 
    then
        potentialFoam > log.potentialFoam 2>&1
        foamRun -solver incompressibleFluid > log.incompressibleFluid 2>&1
    else
        # mpirun -np ${i} patchSummary  -parallel > log.patchSummary 2>&1
        mpirun -np ${i} potentialFoam -parallel > log.potentialFoam 2>&1
        mpirun -np ${i} foamRun -solver incompressibleFluid -parallel > log.incompressibleFluid 2>&1
        reconstructPar -latestTime > log.reconstructPar 2>&1

        # foamRun -solver incompressibleFluid -parallel
        #mpirun -np ${i} foamRun -solver incompressibleFluid -parallel > log.simpleFoam 2>&1
    fi
    cd ..
done

# Extract times
echo "# cores   Wall time (s):"
echo "------------------------"
for i in $TIMPROCS; do
    echo $i `grep Execution run_${i}/log.incompressibleFluid | tail -n 1 | cut -d " " -f 3`
done

and version 11 something like this:

#!/bin/bash

# Prepare cases
# This example runs on 1, 2 and 4 cores
for i in 1 2 4; do
    d=run_$i
    echo "Prepare case ${d}..."
    cp -r basecase $d
    cd $d
    if [ $i -eq 1 ] 
    then
        mv Allmesh_serial Allmesh
    fi
    sed -i "s/method.*/method scotch;/" system/decomposeParDict
    sed -i "s/numberOfSubdomains.*/numberOfSubdomains ${i};/" system/decomposeParDict
    time ./Allmesh
    cd ..
done

# Run cases
for i in 1 2 4; do
    echo "Run for ${i}..."
    cd run_$i
    if [ $i -eq 1 ] 
    then
        simpleFoam > log.simpleFoam 2>&1
    else
        mpirun -np ${i} simpleFoam -parallel > log.simpleFoam 2>&1
    fi
    cd ..
done

# Extract times
echo "# cores   Wall time (s):"
echo "------------------------"
for i in 1 2 4; do
    echo $i `grep Execution run_${i}/log.simpleFoam | tail -n 1 | cut -d " " -f 3`
done

The basecase was the motorBikeSteady tutorial with the following changes:

system/controlDict:
< endTime 500;
> endTime 100;

system/blockMeshDict:
< hex (0 1 2 3 4 5 6 7) (20 8 8) simpleGrading (1 1 1)
> hex (0 1 2 3 4 5 6 7) (40 16 16) simpleGrading (1 1 1)

system/decomposeParDict:
< numberOfSubdomains 16;
> numberOfSubdomains 2;

The first two change the number of iterations and the mesh size hints to match. Not sure about the 3rd but I may have been fiddling. I am not an openfoam user and was making guesses at the likely meaning of parameters. What is really needed is for an openfoam user to diff the earlier benchmark and the current tutorial and keep the parameter changes that are relevant. Whatever, my benchmark results are broadly in line with expectations and so if there are differences they are small.

[wkernkamp]

Originally Posted by andy_

OK some of what I did is coming back but I am not an openfoam user and was hacking to get something working rather than carefully setting up a benchmark.

Version 11 and 12 of openfoam are organised significantly differently and require different scripts. Don't know about earlier versions.

I ran version 12 something like this .....

The first two change the number of iterations and the mesh size hints to match. Not sure about the 3rd but I may have been fiddling. I am not an openfoam user and was making guesses at the likely meaning of parameters. What is really needed is for an openfoam user to diff the earlier benchmark and the current tutorial and keep the parameter changes that are relevant. Whatever, my benchmark results are broadly in line with expectations and so if there are differences they are small.

I did not see your result. From the looks of it you made the correct changes.


I find it extremely annoying that the basic call for the simpleFoam solution has been changed by openfoam.org. They remained the same for OpenFoam.com. (OpenFOAM v2312). I remember loosing all interest in the ruby language because they kept changing the language so that you had to rewrite your code for every version. Developers that don't know that better is the enemy of good, should be shot to save us all a lot of time.

[andy_]

Originally Posted by wkernkamp

I did not see your result. From the looks of it you made the correct changes.

My results are a few posts up on the previous page and thanks for the confirmation.

[wkernkamp]

Originally Posted by andy_

My results are a few posts up on the previous page and thanks for the confirmation.

I looked at your previous posts. I was confused because some are quotes of other people 's results. Is your current system the one doing just over 100 seconds? If it is, you can do ~64 seconds with two 16+ cores. My fastest one does 60 seconds. They have the same total 8 memory channels at 2400 MT/s as you. However, they have much more L3 cache and that is a factor too. Upgrade your bios to the latest before installing the high core count cpus!

[gumersindu]

Hi all,

I finally modified the motorbike tutorial to match the same configuration as in the benchmark from the original post. I've attached the modified code which worked for v2312.

These are the results I got for this PC config: HP Z840 | 2 x Intel Xeon E5-2690 v4 (14 cores, 2,6 GHz, 35Mb L3) | 8 x 16GB DDR4 ECC | 1TB HDD | Ubuntu 24.04 LTS

cores  MeshTime(s)     RunTime(s)     
-----------------------------------
1      1403.79         1098.68        
2      949.89          551.16         
4      495.73          246.11         
6      361.35          163.72         
8      293.58          128.46         
12     244.06          99.28          
16     229.99          84.12          
20     186.59          78.14          
24     183.3           74.44          
28     177.25          72.7