[Kolan]

I recently got a new Mac and wanted to build OpenFOAM natively without the need for virtualisation to make the most of the core count of the M3 Max.

Here is what I needed to do to get it running based on the patches and guide by Alexey Matveichev, so many thanks to his efforts!

https://github.com/mrklein/openfoam-os-x

I installed v2206.

My 16" MacBook Pro is running Sonoma 14.2 (23C64).
I have a 16 core M3 Max.

The main issues I had were related to clearing of environmental variables by SIP when running scripts and linking scotch. This post will follow and tweak this guide:
https://github.com/mrklein/openfoam-...ase-&-Homebrew

First install Xcode and the Xcode command line tools.
You can do this from the app store. Check the command line tools are installed:

$ xcode-select --install

Next install homebrew
https://brew.sh

The guide then asks you to install some dependencies, however the version of scotch failed to install. Instead install the unmodified version of scotch which in my case is 7.0.4.

brew install --cask paraview
$ brew install open-mpi
$ brew install boost
$ brew install cgal
$ brew install metis
$ brew install libomp
$ brew install scotch
$ brew install mrklein/foam/parmgridgen

Following the guide I created a case sensitive sparse bundle in my home directory. Go to your home directory and create and mount the volume to a directory called OpenFOAM:

$ cd $HOME
$ hdiutil create -size 8.3g -type SPARSEBUNDLE -fs HFSX -volname OpenFOAM -fsargs -s OpenFOAM.sparsebundle
$ mkdir -p OpenFOAM
$ hdiutil attach -mountpoint $HOME/OpenFOAM OpenFOAM.sparsebundle

Now we can download the tarballs of the OpenFOAM source. Again I'm downloading v2206 here because there is a patch for this version.

$ curl -L http://dl.openfoam.com/source/v2206/OpenFOAM-v2206.tgz > OpenFOAM-v2206.tgz
$ curl -L http://dl.openfoam.com/source/v2206/ThirdParty-v2206.tgz > ThirdParty-v2206.tgz

There will be downloaded to your home directory, you can delete them later.

Next extract the tarballs into the new OpenFOAM volume.

$ cd OpenFOAM
$ tar xzf $HOME/OpenFOAM-v2206.tgz
$ tar xzf $HOME/ThirdParty-v2206.tgz

This creates two directories in ~/OpenFOAM, OpenFOAM-v2206 and ThirdParty-v2206.

Let's go into OpenFOAM-v2206 and download the patch file and then apply it.

cd OpenFOAM-v2206
$ curl -L https://raw.githubusercontent.com/mrklein/openfoam-os-x/master/OpenFOAM-v2206.patch > OpenFOAM-v2206.patch
$ git apply OpenFOAM-v2206.patch

Before we go and build OpenFOAM we need to fix how scotch is linked.

Looking at our scotch install info:

brew info scotch

we can see where brew put it:

==> scotch: stable 7.0.4 (bottled), HEAD
Package for graph partitioning, graph clustering, and sparse matrix ordering
https://gitlab.inria.fr/scotch/scotch
/opt/homebrew/Cellar/scotch/7.0.4 (227 files, 11.3MB) *
  Poured from bottle using the formulae.brew.sh API on 2023-12-15 at 15:32:38
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/s/scotch.rb
License: CECILL-C
==> Dependencies
Build: bison ✘
Required: open-mpi ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 62 (30 days), 169 (90 days), 435 (365 days)
install-on-request: 60 (30 days), 167 (90 days), 432 (365 days)
build-error: 0 (30 days)

So we need to add this information before building. To do this you need to edit the edit etc/config.sh/scotch file to include:

export SCOTCH_VERSION=scotch-system
export SCOTCH_ARCH_PATH=/opt/homebrew/Cellar/scotch/7.0.4

Note that on my old Intel Mac scotch was in /usr/local/Cellar/scotch/7.0.4

Comment out the existing entries and save the file.

Next run these commands to update some necessary settings:

$ mkdir -p $HOME/.OpenFOAM
$ echo 'WM_COMPILER=Clang' > $HOME/.OpenFOAM/prefs.sh
$ echo 'WM_COMPILE_OPTION=Opt' >> $HOME/.OpenFOAM/prefs.sh
$ echo 'WM_MPLIB=SYSTEMOPENMPI' >> $HOME/.OpenFOAM/prefs.sh
$ echo 'export WM_NCOMPPROCS=$(sysctl -n hw.ncpu)' >> $HOME/.OpenFOAM/prefs.sh
$ echo 'WM_LABEL_SIZE=32' >> $HOME/.OpenFOAM/prefs.sh
$ source etc/bashrc
$ [ "$(ulimit -n)" -lt "4096" ] && ulimit -n 4096

Now we can build

$ cd $HOME/OpenFOAM/OpenFOAM-v2206 
$ ./Allwmake > log.Allwmake 2>&1

Give it some time to build. You can monitor the build log in another terminal window with:

$ tail -f $HOME/OpenFOAM/OpenFOAM-v2206/log.Allwmake

After the build is complete we need to add some files and environmental variables so OpenFOAM is available when we open the Terminal.

mkdir -p $HOME/.OpenFOAM
echo 'v2206' > $HOME/.OpenFOAM/OpenFOAM-release

Finally you need to add the following to your $HOME/.zprofile or where you prefer to put this stuff.

if [ -f "$HOME/.OpenFOAM/OpenFOAM-release" ]; then
    FOAM_DISK_IMAGE=${FOAM_DISK_IMAGE:-"$HOME/OpenFOAM.sparsebundle"}
    FOAM_MOUNT_POINT=${FOAM_MOUNT_POINT:-"$HOME/OpenFOAM"}
    FOAM_VERSION=$(cat $HOME/.OpenFOAM/OpenFOAM-release)
    if [ ! -f "$HOME/OpenFOAM/OpenFOAM-$FOAM_VERSION/etc/bashrc" ]; then
        hdiutil attach -quiet -mountpoint "$FOAM_MOUNT_POINT" "$FOAM_DISK_IMAGE" &&
                . "$HOME/OpenFOAM/OpenFOAM-$FOAM_VERSION/etc/bashrc"
    else
        source "$HOME/OpenFOAM/OpenFOAM-$FOAM_VERSION/etc/bashrc"
    fi
fi

There is still an issue linking Metis but this can be fixed the same way scotch is fixed if you need it.

Finally you can run the OpenFoam Benchmark, but you need to tweak a could have things. Download the OpenFOAM.com specific zip file from this post:
\OpenFOAM benchmarks on various hardware

This runs simpleFOAM on various core counts using a script. As mentioned before System Integrity Protection can clear required environmental variables when running scripts. The suggested fix for this is to include

source $WM_PROJECT_DIR/etc/bashrc

Near the top of any script you run, including this benchmark run.tst script.

The run.tst script needs to be edited. I changed the core counts to:

cores='16 12 10 8 6 4 2 1'

Also the sed command is slightly different on MacOS. Change:

sed -i "s/method.*/method scotch;/" system/decomposeParDict
sed -i "s/numberOfSubdomains.*/numberOfSubdomains ${i};/" system/decomposeParDict

to:

sed -i '' "s/method.*/method scotch;/" system/decomposeParDict
sed -i '' "s/numberOfSubdomains.*/numberOfSubdomains ${i};/" system/decomposeParDict

Next you need to edit basecase/system/snappyHexMeshDict
to replace the reference in line 279 from

minMedianAxisAngle 90;

to

minMedialAxisAngle 90;

I also noticed that there was a warning related to basecase/system/controlDict.
It's not important but you can change:

writeCompression uncompressed;

to

writeCompression off;

Run the benchmark with

./run.tst

gives some very impressive times.

# cores Wall time (s):
------------------------
16 87.19
12 53.91
10 57.39
8 63.66
6 74.18
4 110.1
2 197.9
1 345.8

[olesen]

More changes for handling SIP, rpath installation and xcode will be in OpenFOAM-v2312. Presumably will need less patching (hopefully no patching).
It would be useful if you would also give that a spin in the next days/weeks and see if some of your recipe can be reduced.

[olesen]

Is the homebrew recipe for scotch now fixed to compile without threaded MPI (which is a real performance killer for OpenFOAM)?

[Kolan]

Yes, it seems that scotch is now behaving as expected.

Once I get my exam grading done I'll give v2312 a look!

[gerlero]

Yes, it seems that scotch is now behaving as expected.

This needs to be tested to verify but it would be great news! AFAICT there haven't been any commits at Homebrew that have changed the build options, but I might very well be missing something.

BTW, for those interested in running OpenFOAM natively on macOS, my project at https://github.com/gerlero/openfoam-app automates the builds (for those who want to compile from source) and also provides pre-compiled binaries (for those who just want to get up and running as fast as possible).

[Kolan]

And it is an excellent way to get it running too.

You will know better than me but I’ll report back soon on how I get on.

[Simbelmynë]

Hi,

Did you test v2312? Any improvements with SIP?

I have problems with

The dummy Pstream library cannot be used in parallel mode

which I did not have on Ventura (I think). Not sure if it is a SIP issue though, since adding

source $WM_PROJECT_DIR/etc/bashrc

to the Allrun script has no effect.

Did you have similar issues and how did you solve it?

[Kolan]

I've yet to have time to test v2312 but I've had no issues with SIP. After following the steps in my initial post I've had no further issues with v2306.

[mathslw]

Many thanks for the post. I also have a mac with M3 Max and followed the posted steps to install OpenFOAM 5.0, but got an error "symbol not found in flat namespace '_ompi_mpi_byte'". Then followed this link https://github.com/mrklein/openfoam-os-x/issues/91

Now it seems working well.

[Kolan]

Good stuff, I had posted there also but hadn't followed up.
I also need to install the Foundation version so that is very helpful to know. Thank you.

[Simbelmynë]

Originally Posted by Kolan

Good stuff, I had posted there also but hadn't followed up.
I also need to install the Foundation version so that is very helpful to know. Thank you.

If you manage to get v11 to run please let us know

Edit: I can confirm that v11 works fine with the above method. I have tested both serial and parallel tutorial cases. Perhaps a bit more extensive testing is needed though, but it seems promising

[Counterdoc]

Did someone compare the times with running Docker on macOS? Would be interesting to know the difference in performance. I am currently running it on docker with my M1 Max.

[Simbelmynë]

Originally Posted by Counterdoc

Did someone compare the times with running Docker on macOS? Would be interesting to know the difference in performance. I am currently running it on docker with my M1 Max.

I have posted results in the benchmark thread for M1 Max using native (Gerlero) compilation. You could compare your Docker results to those.

[moh raafat]

hello, I have this error every Time I try to install on Mac Pro M3, Mac suggests to install
command line developer tools and I click install it, however it keeps the error and the installation fails.

xcode-select: Failed to locate 'm4', requesting installation of command line developer tools.
m4 stage failed on expressions/fields/fieldExprLemonParser.lyy-m4

Retaining intermediate: /Users/mohammedshahin/OpenFOAM22/OpenFOAM-v2206/build/darwin64ClangDPInt32Opt/src/OpenFOAM/expressions/fields/fieldExprLemonParser.lyy
make: *** [/Users/mohammedshahin/OpenFOAM22/OpenFOAM-v2206/build/darwin64ClangDPInt32Opt/src/OpenFOAM/expressions/fields/fieldExprLemonParser.o] Error 1
make: *** Waiting for unfinished jobs....

[Simbelmynë]

More information is needed.

This problem can be associated with multiple installations of OF (Gerlero and other). Try to remove config files for OpenFOAM in your user directory

[hyunko]

Hello Kolan,

I have followed your instruction to compile and install OF2206.

After compilation and decomposing a mesh,

I got a following error.
================================================== =
--> FOAM FATAL ERROR: (openfoam-2206)
Attempted to use <scotch> without the scotchDecomp library loaded.
This message is from the dummy scotchDecomp stub library instead.

Please install <scotch> and ensure libscotch.so is in LD_LIBRARY_PATH.
The scotchDecomp library can then be built from src/parallel/decompose/scotchDecomp.
Dynamically loading or linking this library will add <scotch> as a decomposition method.
================================================== =

Do you have any idea for this error?

As per your instruction, I have edited etc/config.sh/scotch file.
================================================== =
#------------------------------------------------------------------------------
# USER EDITABLE PART: Changes made here may be lost with the next upgrade

#SCOTCH_VERSION=scotch_6.1.0
#export SCOTCH_ARCH_PATH=$WM_THIRD_PARTY_DIR/
#platforms/$WM_ARCH$WM_COMPILER$WM_PRECISION_OPTION$WM_LABEL_ OPTION/$SCOTCH_VERSION

export SCOTCH_VERSION=scotch-system
export SCOTCH_ARCH_PATH=/opt/homebrew/Cellar/scotch/7.0.4

# END OF (NORMAL) USER EDITABLE PART
#------------------------------------------------------------------------------
================================================== =

thank you.

[einstein_zee]

Thanks for all the contributions! As they say a picture is worth a thousand words, a video is worth even more . Just in case if someone is still struggling, this video may help : https://youtu.be/_z7K97NE2pg
compilation done on M1 apple silicon with macOS Monterey

[Kolan]

Hi hyunko,
What does

% brew info scotch

tell you?

My system is currently reporting:

==> scotch: stable 7.0.5 (bottled), HEAD
Package for graph partitioning, graph clustering, and sparse matrix ordering
https://gitlab.inria.fr/scotch/scotch
Installed
/opt/homebrew/Cellar/scotch/7.0.5 (231 files, 11.2MB) *
  Poured from bottle using the formulae.brew.sh API on 2024-08-15 at 16:26:11
From: https://github.com/Homebrew/homebrew...la/s/scotch.rb
License: CECILL-C
==> Dependencies
Build: bison ✘
Required: open-mpi ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 86 (30 days), 147 (90 days), 553 (365 days)
install-on-request: 84 (30 days), 143 (90 days), 544 (365 days)
build-error: 0 (30 days)

But I had compiled with 7.0.4. Have you installed a version other than 7.0.4 by any chance?

[hyunko]

Thank you Kolan.

I successfully compiled OF with following instruction.

https://github.com/mrklein/openfoam-...ase-&-Homebrew

[bzollars]

Originally Posted by gerlero

BTW, for those interested in running OpenFOAM natively on macOS, my project at https://github.com/gerlero/openfoam-app automates the builds (for those who want to compile from source) and also provides pre-compiled binaries (for those who just want to get up and running as fast as possible).

Gerlero, is cfMesh part of your pre-compiled OpenFOAM package? I have just installed v2406 and it does not seem to be there.

BTW, You have made it very nice and easy to get up and going with OpenFoam!

Thanks