[7islands]

Hi,

I share some numbers on benchmarking the damBreak/damBreakFine cases of OpenFOAM-2.3.1 on embedded systems. The systems are not meant for real computation, but OpenFOAM was a handy benchmark code when I wanted to get an idea about the performance of the systems.

• Raspberry Pi Model B (ARM11 700MHz single-core, 512MB, Raspbian)
  • damBreak: 402.7s
  • damBreakFine: 1177.85s (serial)
• Intel Edison (Intel Atom dual-core, 500MHz, 1GB, Yocto)
  • damBreak: 130 s
  • damBreakFine: 318.01s (serial), 187.63 s (ThirdParty MPI 2-way)
• Regular PC (Nehalem Xeon 3.46 GHz, Ubuntu 12.04)
  • damBreak: 8.39s
  • damBreakFine: 18.56s (serial), 11.41s (2-way), 7s (4-way)

Notes

• On all platforms, installation went without much trouble after installing required dependencies such as GNU tar, bison, flex, coreutils, etc.
• On Raspbian, I borrowed the settings of wmake/rules from http://rheologic.at/?q=downloads&language=en.
• Compilation took approx. 4-5 days on the Pi, 1 day on the Edison, 1-2 hours on the Xeon.

My overall observation on the Pi and the Edison mostly follows http://www.davidhunt.ie/raspberry-pi...n-benchmarked/. An edison is more than 15x slower than a PC, but is still impressively faster than a Pi especially when two cores are used (3x-6x). But what most impressed me was that the tiny thumb-size circuit (the Edison) runs the full computational fluid dynamics. It is amazing that OpenFOAM scales from an Edison to a supercomputer.

Takuya

[7islands]

I ran the tests this time on a Raspberry Pi 2 and a BeagleBone Black.

• Raspberry Pi 2 Model B (ARM Cortex-A7 900MHz quad-core, 1GBytes, Raspbian, Pi-compatible binary*1)
  • damBreak: 127s
  • damBreakFine: 377s (single), 233s (2-way), 162s (4-way)
• Raspberry Pi 2 Model B (Pi2-specific binary*2)
  • damBreak: 130s
  • damBreakFine: 387s (single), 238s (2-way), 162s (4-way)
• BeagleBone Black (ARM Cortex-A8 1GHz, 512MBytes, Debian, BBB-specific binary*3)
  • damBreak: 260s
  • damBreakFine: 782s (serial)

*1: g++ ARM options are -march=armv6 -mfpu=vfp. The binary is the same one used for testing the Pi.
*2: -march=armv7-a -mfpu=neon-vfpv4
*3: -march=armv7-a -mfpu=vfpv3. The Pi-compatible binary did not run.

Notes

• On the Pi2, compiling with WM_NCOMPPROCS=4 exhausts the memory. It has to be 3 or less. The compilation took approx. 12 hours.
• Similarly, on the BBB the GUI (lightdm) had to be turned off. The compilation took approx. 2 days.

The summary:


Some of the interesting points of the figure are

• Obviously, the Pi2 is significantly faster than the Pi. 3x per core and 7x with all cores. This is even faster than the official claim of 1.5x per thread and 6x for total.
• The Pi2 performs similarly with the Edison per-core.
• The Pi2-specific binary made no performance gain against the Pi-compatible binary.
• The performance of the BBB is behind the Pi2 in spite of the faster 1GHz clock speed and a similar Cortex-A8 architecture. This is because the A8 has only a light version of floating point unit (VFPLite).

Useful links
[1] Learning Cortex-A8 FP unit http://processors.wiki.ti.com/index.php/Cortex-A8
[2] Learning ARM FP unit versions and ARM-Linux ABI https://wiki.debian.org/ArmHardFloatPort/VfpComparison
[3] Summary of FP-related options of GCC http://dench.flatlib.jp/opengl/fpu_vfp (in Japanese but the tables should be self-explanatory)

Takuya

[piprus]

Hello,

I wonder whether you could share the binaries to others to test other cases on these platforms to extend your tests?

Myself, I am interested in the version for RPi2, but encounter some difficulties in compilation on my own.

Best regards,
Piotrek

[7islands]

I'm afraid I can't as I don't have the time to package, support and maintain the binaries. But you can try the binaries built by Rheologic (http://rheologic.at/?q=downloads&language=en). I followed their compilation settings anyway.

T

[leinad]

Originally Posted by piprus

Hello,

I wonder whether you could share the binaries to others to test other cases on these platforms to extend your tests?

Myself, I am interested in the version for RPi2, but encounter some difficulties in compilation on my own.

Best regards,
Piotrek

I have set up a raspberry pi cluster that runs openfoam if you still need help let me know

[preacher]

Greetings

I am trying to install OpenFOAM on Raspberry Pi 2, Model B. I downloaded the OpenFOAM for Pi from RheologicRemix-2.2.1 for Raspberry Pi (Raspbian) ARMv6 (without Paraview) download link.

Hope everybody got it from there too.

I am having an issue installing it into the Pi. Can anyone briefly explain the procedure?

I used tar-xf to extract the OpenFOAM and tried installing it. When I did ./Allwmake in ThirdParty folder. I got an error..

./Allwmake: 34: ./Allwmake: wmakeCheckPwd: not found
Error: Current directory is not $WM_THIRD_PARTY_DIR
The environment variables are inconsistent with the installation.
Check the OpenFOAM entries in your dot-files and source them.

I know there are posts regarding this error, but it would be great if you can list the steps for installation.

Thanks in advance..!!

[wyldckat]

Originally Posted by preacher

Can anyone briefly explain the procedure?

Quick answer: Instructions available at http://www.cfd-online.com/Forums/ope...tml#post556209 - post #13

[steinLeat]

I tried building with Ubuntu 18.04.1 on a Banana Pi M64.


For what I have seen there is some support for ARM64. I get errors of these kinds:

========================================                                                                                                                                                 
Compile OpenFOAM libraries                                                                                                                                                               
                                                                                                                                                                                         
no git description found                                                                                                                                                                 
wmakeLnInclude: linking include files to OpenFOAM/lnInclude                                                                                                                              
wmakeLnInclude: linking include files to OSspecific/POSIX/lnInclude                                                                                                                      
wmake  dummy                                                                                                                                                                             
wmake dummy
wmake  mpi
wmake mpi
wmake libo (POSIX)
wmake OpenFOAM
g++ -std=c++11 -DOPENFOAM=1806 -DlinuxARM64 -DWM_ARCH_OPTION=64 -DWM_DP -DWM_LABEL_SIZE=32 -Wall -Wextra -Wold-style-cast -Wnon-virtual-dtor -Wno-unused-parameter -Wno-invalid-offsetof
-Wno-attributes -Wno-unknown-pragmas -O3 -floop-optimize -falign-loops -falign-labels -falign-functions -falign-jumps  -DNoRepository -ftemplate-depth-100 -I/home/gerstrong/OpenFOAM/Op$
nFOAM-v1806/build/linuxARM64GccDPInt32Opt/src/OpenFOAM -IlnInclude -I. -I/home/gerstrong/OpenFOAM/OpenFOAM-v1806/src/OpenFOAM/lnInclude -I/home/gerstrong/OpenFOAM/OpenFOAM-v1806/src/OS$
pecific/POSIX/lnInclude   -fPIC -c global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C -o /home/gerstrong/OpenFOAM/OpenFOAM-v1806/build/linuxARM64GccDPI$
t32Opt/src/OpenFOAM/global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.o
In file included from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:810:0,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperationTemplates.C: In instantiation of ‘Type Foam::fileOperations::masterUncollatedFileOperation::scatterList$
const Foam::UList<T>&, int, Foam::label) const [with Type = long int; Foam::label = int]’:
global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperationTemplates.C:100:45:   required from ‘Type Foam::fileOperations::masterUncollatedFileOperation::masterOp$
const Foam::fileName&, const fileOp&, int, Foam::label) const [with Type = long int; fileOp = Foam::fileOperations::masterUncollatedFileOperation::fileSizeOp; Foam::label = int]’
global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:1016:5:   required from here
global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperationTemplates.C:46:16: error: ambiguous overload for ‘operator<<’ (operand types are ‘Foam::UOPstream’ and $
const long int’)
             os << masterLst[proci];
             ~~~^~~~~~~~~~~~~
In file included from lnInclude/IOstream.H:45:0,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/char.H:50:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, char)
 Ostream& operator<<(Ostream&, const char);
          ^~~~~~~~
In file included from lnInclude/IOstream.H:46:0,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/bool.H:49:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, bool)
 Ostream& operator<<(Ostream& os, const bool b);
          ^~~~~~~~
In file included from lnInclude/bool.H:59:0,
                 from lnInclude/IOstream.H:46,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/direction.H:53:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, Foam::direction)
 Ostream& operator<<(Ostream& os, const direction d);
          ^~~~~~~~
In file included from lnInclude/int.H:38:0,
                 from lnInclude/label.H:39,
                 from lnInclude/IOstream.H:47,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/int16.H:62:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, int16_t)
 Ostream& operator<<(Ostream& os, const int16_t val);
          ^~~~~~~~
In file included from lnInclude/int.H:39:0,
                 from lnInclude/label.H:39,
                 from lnInclude/IOstream.H:47,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/int32.H:108:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, int32_t)
 Ostream& operator<<(Ostream& os, const int32_t val);
          ^~~~~~~~
In file included from lnInclude/int.H:40:0,
                 from lnInclude/label.H:39,
                 from lnInclude/IOstream.H:47,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/int64.H:109:10: note: candidate: Foam::Ostream& Foam::operator<<(Foam::Ostream&, int64_t)
 Ostream& operator<<(Ostream& os, const int64_t val);
          ^~~~~~~~



....


In file included from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:810:0,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperationTemplates.C:60:12: error: no match for ‘operator>>’ (operand types are ‘Foam::UIPstream’ and ‘long int’)
         is >> myResult;
         ~~~^~~~~~~~~~~
In file included from lnInclude/IOstream.H:45:0,
                 from lnInclude/Istream.H:46,
                 from lnInclude/ISstream.H:39,
                 from lnInclude/fileOperation.H:41,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.H:61,
                 from global/fileOperations/masterUncollatedFileOperation/masterUncollatedFileOperation.C:26:
lnInclude/char.H:49:10: note: candidate: Foam::Istream& Foam::operator>>(Foam::Istream&, char&) <near match>
 Istream& operator>>(Istream&, char&);
          ^~~~~~~~

It seems like some typedefs are missing or the typedef do not match that well with ARM64 Architecture.

[steinLeat]

Well, for now I'm using the OpenFOAM provided by the Ubuntu Repos.

apt install openfoam openfoam-examples libopenfoam

With that version I can simulate just fine.



I might take a second look on the build problem later.