[achim]

After running with the standard gcc and open-mpi configuration I'd like to install and test with Intel's compilers and MPI.

Allwmake tells me:
.../OpenFOAM/OpenFOAM-1.7.1/lib/linux64IccDPOpt/libscotch.so: undefined reference to `pthread_detach'

Where should this pthread_detach come from?

Changes to etc/bashrc:
# Compiler: set to Gcc, Gcc43, Gcc44, or Icc (for Intel's icc)
: ${WM_COMPILER:=Icc}; export WM_COMPILER
and added later in the file appropriate compiler options for the Intel compilers

etc/settings.sh:
INTELMPI)
mpi_version=intelmpi-3.2.2.006
export MPI_ARCH_PATH=$I_MPI_ROOT
# echo $I_MPI_ROOT
# /opt/intel/impi/3.2.2.006


# Tell OpenMPI where to find its install directory
# export OPAL_PREFIX=$MPI_ARCH_PATH

# _foamAddPath $MPI_ARCH_PATH/bin
_foamAddPath $MPI_ARCH_PATH/bin64
# _foamAddLib $MPI_ARCH_PATH/lib
_foamAddLib $MPI_ARCH_PATH/lib64
_foamAddMan $MPI_ARCH_PATH/man

export FOAM_MPI_LIBBIN=$FOAM_LIBBIN/$mpi_version
unset mpi_version
;;

Regards
Achim

[wyldckat]

Greetings Achim,

I haven't tried building OpenFOAM 1.7.1 with Icc, but I can tell you that in OpenFOAM 2.0.x the Scotch libraries are built with Gcc, precisely due to those issues you are getting.

You can see this particular fix in action here: https://github.com/OpenFOAM/ThirdPar.../Allwmake#L221 - it starts in line 221.

Best regards,
Bruno

[achim]

Originally Posted by wyldckat

Greetings Achim,

I haven't tried building OpenFOAM 1.7.1 with Icc, but I can tell you that in OpenFOAM 2.0.x the Scotch libraries are built with Gcc, precisely due to those issues you are getting.

You can see this particular fix in action here: https://github.com/OpenFOAM/ThirdPar.../Allwmake#L221 - it starts in line 221.

Best regards,
Bruno

Thanks Bruno,

so I switched to 2.0.1, but still ran into some error:

+ decompose/Allwmake
+ wmakeLnInclude decompositionMethods
+ '[' -n /work/appl/OpenFOAM/ThirdParty-2.0.1/platforms/linux64Icc/scotch_5.1.11 ']'
+ wmake libso scotchDecomp
Making dependency list for source file scotchDecomp.C
+ '[' -d /work/appl/OpenFOAM/OpenFOAM-2.0.1/platforms/linux64IccDPOpt/lib/ ']'
+ wmakeMpiLib ptscotchDecomp
+ set +x
ptscotchDecomp.C(126): catastrophic error: cannot open source file "ptscotch.h"
#include "ptscotch.h"
^

compilation aborted for ptscotchDecomp.C (code 4)
make: *** [Make/linux64IccDPOptINTELMPI/ptscotchDecomp.o] Error 4

So it seems something in my setup is still wrong ...

Any idea?

Regards
Achim

[wyldckat]

Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:

1. Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:

   ./Allwmake > make.log 2>&1

   This will put every single output generated during the building process into the file "make.log".
2. Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
3. Pack the file into a tarball:

   tar -czf make.log.tar.gz make.log

4. Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.

Best regards,
Bruno

[achim]

Originally Posted by wyldckat

Hi Achim,

OK, let's do some isolate-and-conquer by only building the ThirdParty folder first:

1. Go to the "ThirdParty-2.0.1" folder and run Allwmake like this:

   ./Allwmake > make.log 2>&1

   This will put every single output generated during the building process into the file "make.log".
2. Edit the log file and edit or search-replace any sensitive data you might find (user names and/or paths).
3. Pack the file into a tarball:

   tar -czf make.log.tar.gz make.log

4. Attach said tarball "make.log.tar.gz" to your next post, so I can see what actually occurred.

Best regards,
Bruno

Hi Bruno,

here we go (I have split out and error into 2 files, hope that's OK).

Best regards
Achim

[wyldckat]

Hi Achim,

I wanted the output and error messages to be in the same file, so I could have an historic detailing of where/when the errors occurred. This way I have separate history lines...

Either way, my suspicion is that you edited the following code in "etc/config/settings.sh":

    x86_64)
        case "$WM_ARCH_OPTION" in
        32)
            export WM_COMPILER_ARCH=64
            export WM_CC='gcc'
            export WM_CXX='g++'
            export WM_CFLAGS='-m32 -fPIC'
            export WM_CXXFLAGS='-m32 -fPIC'
            export WM_LDFLAGS='-m32'
            ;;
        64)
            WM_ARCH=linux64
            export WM_COMPILER_LIB_ARCH=64
            export WM_CC='gcc'
            export WM_CXX='g++'
            export WM_CFLAGS='-m64 -fPIC'
            export WM_CXXFLAGS='-m64 -fPIC'
            export WM_LDFLAGS='-m64'
            ;;

If you changed the lines with "WM_CC" and "WM_CXX" to use "icc", then undo those changes! Like I wrote before, Scotch will not build with Icc... at least not without substantial hacking - here's the proof: http://www.openfoam.com/mantisbt/view.php?id=323

By the way, knowing which Gcc version and Linux architecture would make it easier for me to help you. To know the architecture, run:

uname -m

Best regards,
Bruno

[achim]

Hi Bruno,

here it comes:

# cat /etc/SuSE-release
SUSE Linux Enterprise Server 11 (x86_64)
VERSION = 11
PATCHLEVEL = 1

# uname -m
x86_64

# uname -a
Linux machine 2.6.32.12-0.7-default #1 SMP 2010-05-20 11:14:20 +0200 x86_64 x86_64 x86_64 GNU/Linux

# gcc -v
Using built-in specs.
Target: x86_64-suse-linux
Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
Thread model: posix
gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)

Best regards
Achim

[wyldckat]

Hi Achim,

Could you please run the command with the "2>&1" in the end!?

./Allwmake > make.log 2>&1

Or:

./Allwmake 1>> make2in1.log  2>> make2in1.log

Don't forget the new name on this one: "make2in1.log".

Because there seems to have been problems building ptscotch, but those errors aren't in the file you posted!

Best regards,
Bruno

[achim]

Dear Bruno,

I ran as you proposed. So for the ThirdParty stuff it seems to be OK ...
Just to be sure, here it is again (, ThirdParty stuff only, after Allclean):
./Allwmake > make.log 2>&1

Best regards
Achim

[wyldckat]

Then something strange is going on... there is no indication that "ptscotch" built at all You'll probably run into the same problem as before.

[achim]

I checked the settings during excution of ThirdParty.../Allwmake and
found 2 strange things:
#1
SCOTCH_ARCH_PATH in Allwmake is
OpenFOAM/ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11
though normally I'm doing OPT and DP, i.e. I would have expected this to be
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPOpt
This seems wrong to me, right?

#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))

Regards
Achim

[wyldckat]

Hi Achim,

Originally Posted by achim

I checked the settings during excution of ThirdParty.../Allwmake and
found 2 strange things:
#1
SCOTCH_ARCH_PATH in Allwmake is
OpenFOAM/ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11
though normally I'm doing OPT and DP, i.e. I would have expected this to be
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPOpt
This seems wrong to me, right?

Scotch is built by Allwmake with the default of splitting into two main folders:

• "ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
• "ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.
  

Originally Posted by achim

#2
FOAM_MPI is not set, so I don't get into the make of ptscotch. This of course explains why I get the errors.

With the standard OpenMPI config I get libptscotch.so in
OpenFOAM/ThirdParty-2.0.1/platforms/linux64GccDPopt/lib/ openmpi-1.5.3
(this is from my notices, hope I wrote it down right ;-))

That's normal. To use Intel's MPI, add a new entry to "settings.sh" and adjust accordingly, based on what you can see on the other MPI options. I think your initial modification would work, but something was missing.

Right now I can't spend any time looking into this in detail, so all I can say is what I've already written

Best regards,
Bruno

[achim]

Originally Posted by wyldckat

Hi Achim,


Scotch is built by Allwmake with the default of splitting into two main folders:

• "ThirdParty-2.0.1/platforms/linux64Gcc/scotch_5.1.11" - for the applications that Scotch has and some other miscellaneous stuff.
• "ThirdParty-2.0.1/platforms/linux64GccDPOpt/lib/" - where the ThirdParty libraries are kept.
  

That's normal. To use Intel's MPI, add a new entry to "settings.sh" and adjust accordingly, based on what you can see on the other MPI options. I think your initial modification would work, but something was missing.

Right now I can't spend any time looking into this in detail, so all I can say is what I've already written

Best regards,
Bruno

Hi Bruno,

in the meantime I got further: I could link a complete executable and start it, but so far it error exited. (I have another urgent activity, so it may take some time until I can spend cycles on it, but I will keep you updated.)

Many thanks for all your help, Bruno!

Bets regards
Achim