[jans]

I am building OpenFOAM 2.0.0 with icc and SGI MPT. I have made quite a bit of progress but there are the foll two errors that are holding the compilation up. It is quite possible that error 2 is due to error 1, so that its really one error, but I am not sure. Any help will be hugely appreciated.
error 1
========
+ parallel/Allwmake+ decompose/Allwmakeusing SCOTCH_ARCH_PATH=/usr/local/packages/OpenFOAM/2.0.0/ThirdParty-2.0.0/platforms/linux64Icc/scotch_5.1.11+ wmakeLnInclude decompositionMethods+ '[' -n /usr/local/packages/OpenFOAM/2.0.0/ThirdParty-2.0.0/platforms/linux64Icc/scotch_5.1.11 ']'+ wmake libso scotchDecomp'/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib/libscotchDecomp.so' is up to date.+ '[' -d /usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib/mpt-2.03 ']'+ wmakeMpiLib ptscotchDecomp+ set +xwmake libso ptscotchDecompSOURCE=ptscotchDecomp.C ; icpc -std=c++0x -Dlinux64 -DWM_DP -wd327,654,819,1125,1476,1505,1572 -xSSE3 -O1 -no-prec-div -DNoRepository -DOMPI_SKIP_MPICXX -I/opt/sgi/mpt/mpt-2.03/include -I/usr/local/packages/OpenFOAM/2.0.0/ThirdParty-2.0.0/platforms/linux64Icc/scotch_5.1.11/include/mpt-2.03 -I/usr/include/scotch -I../decompositionMethods/lnInclude -IlnInclude -I. -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OpenFOAM/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OSspecific/POSIX/lnInclude -fPIC -c $SOURCE -o Make/linux64IccDPOptOPENMPI/ptscotchDecomp.o/opt/sgi/mpt/mpt-2.03/include/mpi++.h(433): error: more than one instance of overloaded function "PMPI::Init" has "C" linkage Init();

.... and after a few more like that

compilation aborted for ptscotchDecomp.C (code 2)make: *** [Make/linux64IccDPOptOPENMPI/ptscotchDecomp.o] Error 2

error 2
======
make[3]: Entering directory `/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/applications/utilities/mesh/advanced/autoRefineMesh'icpc -std=c++0x -Dlinux64 -DWM_DP -wd327,654,819,1125,1476,1505,1572 -xSSE3 -O1 -no-prec-div -DNoRepository -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/dynamicMesh/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/meshTools/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/triSurface/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/lagrangian/basic/lnInclude -IlnInclude -I. -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OpenFOAM/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OSspecific/POSIX/lnInclude -fPIC Make/linux64IccDPOpt/autoRefineMesh.o -L/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib \ -ldynamicMesh -lmeshTools -ltriSurface -llagrangian -lOpenFOAM -ldl -L/lib -lm -o /usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/bin/autoRefineMesh/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib/libdynamicMesh.so: undefined reference to `Foam::UIPstream::UIPstream(int, Foam::PstreamBuffers&)'make[3]: *** [/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/bin/autoRefineMesh] Error 1make[3]: Leaving directory `/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/applications/utilities/mesh/advanced/autoRefineMesh'make[2]: *** [autoRefineMesh] Error 2

And lots more like that ... the result is that no application gets built.

I think Foam::UIPstream::UIPstream(int, Foam::PstreamBuffers&) should be in /usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib/mpt-2.03/libPstream.so. In fact,

lib/mpt-2.03> nm -s libPstream.so | grep UIPstream
0000000000004510 T _ZN4Foam9UIPstream4readENS_8UPstream10commsTypesEi Pcli
0000000000003a78 T _ZN4Foam9UIPstreamC1ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE
00000000000091f0 r _ZN4Foam9UIPstreamC1ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE$$LSDA
0000000000003f48 T _ZN4Foam9UIPstreamC1EiRNS_14PstreamBuffersE
0000000000009218 r _ZN4Foam9UIPstreamC1EiRNS_14PstreamBuffersE$$LSDA
0000000000004b20 T _ZN4Foam9UIPstreamC2ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE
0000000000004b66 T _ZN4Foam9UIPstreamC2EiRNS_14PstreamBuffersE U _ZTVN4Foam9UIPstreamE

So there are some references to it here. It could be that libPstream.so did not get built properly due to the failure to compile ptscotchDecomp.o ( not sure).Anirban

[jans]

I have found the reason behind error 1. In ptscotchDecomp.C, we have

 
extern C {
   ...
   #include "mpi.h"
   ...
}

But SGI MPT's mpi.h includes in turn mpi++.h, which declares overloaded functions that a pure C compiler (which parses code in extern C {...} ) doesn't understand.

Soln 1
------
Move #include "mpi.h" outside of extern C {...} .

The next question then is why did the OpenFOAM developers choose not to do this?

Soln 2
------
If OpenFOAM code is not to be changed, then add "-DMPI_NO_CPPBIND" option to $WM_CFLAGS:
export WM_CFLAGS="$WM_CFLAGS -DMPI_NO_CPPBIND" (in bash)
This will preempt the including of mpi++.h


Both solns lead to libptscotchDecomp.so being successfully built. I haven't been able to check if they break anything at runtime, because to complete building OF I still need to resolve error 2, which still persists

Any feedback on the above solns and their repurcussions, and/or resolution of error 2, will be great

Thanks
Anirban

[wyldckat]

Greetings Anirban,

I just saw this now: https://github.com/OpenCFD/OpenFOAM-...64Gcc/mplibMPI

The "-DMPI_NO_CPPBIND" is indeed the best option, because it's the same method used with OpenMPI and the file above. If you check the file "wmake/rules/General/mplibOPENMPI" and you'll see "-DOMPI_SKIP_MPICXX" being used as well.
Additionally, using "-DSGIMPI" seems to be a crucial setting as well, since there might be some corner cases that are fixed with this macro definition.

This is indeed a nice thing to know, because I've trying to assist another forum user with a similar operational setting:

• http://www.cfd-online.com/Forums/ope...36-sgimpi.html
• http://www.cfd-online.com/Forums/ope...am1-7-1-a.html

I'm still waiting for feedback from him on the second thread, but it would great if you could help him out with your findings as well!

Best regards,
Bruno

[jans]

Many thanks wyldcat for making this connection

Your suggestions are right as to how to insert -DMPI_NO_CPPBIND cleanly, via wmake/rules/General/mplibSGIMPI. My recipe in the previous post is wrong, $WM_CFLAGS doesn't affect OpenFOAM compilation, but only ThirdParty compilation. I haven't tried -DSGIMPI yet, will do that.

So now the 2nd error in the original post is where I'm stuck. Below are my further observations on it:

Comment 1) No Error msgs (all libraries built) till linking of the 1st exe - autoRefineMesh (this is the 2nd error)

icpc -std=c++0x -Dlinux64 -DWM_DP -wd327,654,819,1125,1476,1505,1572 -xSSE3 -O1 -no-prec-div -DNoRepository -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/dynamicMesh/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/meshTools/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/triSurface/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/lagrangian/basic/lnInclude -IlnInclude -I. -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OpenFOAM/lnInclude -I/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/src/OSspecific/POSIX/lnInclude -fPIC Make/linux64IccDPOpt/autoRefineMesh.o -L/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib \
-ldynamicMesh -lmeshTools -ltriSurface -llagrangian -lOpenFOAM -ldl -L/lib -lm -o /usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/bin/autoRefineMesh
/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/lib/libdynamicMesh.so: undefined reference to `Foam::UIPstream::UIPstream(int, Foam::PstreamBuffers&)'
make[3]: *** [/usr/local/packages/OpenFOAM/2.0.0/OpenFOAM-2.0.0/platforms/linux64IccDPOpt/bin/autoRefineMesh] Error 1

Comment 2) At this point, I checked that libdynamicMesh.so, although earlier reported as up to date, indeed has unresolved references:

nm libdynamicMesh.so | grep UIPstream
000000000026cf96 T _ZN4Foam16fvMeshDistribute11receiveMeshEiRKNS_4Lis tINS_4wordEEES5_S5_RKNS_4TimeERNS1_IiEESA_SA_SA_RN S_9UIPstreamE
0000000000450e68 r _ZN4Foam16fvMeshDistribute11receiveMeshEiRKNS_4Lis tINS_4wordEEES5_S5_RKNS_4TimeERNS1_IiEESA_SA_SA_RN S_9UIPstreamE$$LSDA
U _ZN4Foam9UIPstream4readENS_8UPstream10commsTypesEi Pcli
U _ZN4Foam9UIPstreamC1EiRNS_14PstreamBuffersE
U _ZN4Foam9UIPstreamD1Ev
U _ZN4Foam9UIPstreamD2Ev

ldd libdynamicMesh.so
linux-vdso.so.1 => (0x00007fff2f73c000)
libfiniteVolume.so => not found
libtriSurface.so => not found
libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so (0x00002acb2c3bd000)
libsvml.so => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so (0x00002acb2c751000)
libm.so.6 => /lib64/libm.so.6 (0x00002acb2c968000)
libstdc++.so.6 => /usr/X11R6/lib64/libstdc++.so.6 (0x00002acb2cbbe000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002acb2cec9000)
libintlc.so.5 => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5 (0x00002acb2d0e1000)
libc.so.6 => /lib64/libc.so.6 (0x00002acb2d21f000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002acb2d57d000)
/lib64/ld-linux-x86-64.so.2 (0x00002acb2baf0000)

Comment 3)Note that the link chain is libdynamicMesh.so => libfiniteVolume.so => libOpenFOAM.so => libPstream.so, but for my current build libdynamicMesh.so says libfiniteVolume.so not found, which says libOpenFOAM.so not found, which says libPstream.so not found. This could be the reason for the undefined refs.

Comment 4) I checked that UIPstream, and particularly `Foam::UIPstream::UIPstream(int, Foam::PstreamBuffers&)' does seem to get resolved in $FOAM_MPI_LIBBIN/libPstream.so, so then maybe its the broken link chain thats indeed the culprit

nm libPstream.so | grep UIPstream
0000000000004510 T _ZN4Foam9UIPstream4readENS_8UPstream10commsTypesEi Pcli
0000000000003a78 T _ZN4Foam9UIPstreamC1ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE
00000000000091f0 r _ZN4Foam9UIPstreamC1ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE$$LSDA
0000000000003f48 T _ZN4Foam9UIPstreamC1EiRNS_14PstreamBuffersE
0000000000009218 r _ZN4Foam9UIPstreamC1EiRNS_14PstreamBuffersE$$LSDA
0000000000004b20 T _ZN4Foam9UIPstreamC2ENS_8UPstream10commsTypesEiRNS _11DynamicListIcLj0ELj2ELj1EEERiibNS_8IOstream12st reamFormatENS7_13versionNumberE
0000000000004b66 T _ZN4Foam9UIPstreamC2EiRNS_14PstreamBuffersE
U _ZTVN4Foam9UIPstreamE

Comment 5) But it is not clear though why the "not found" messages occur (ie, why the link chain is broken). The .so files get created in the right order (which they should according to src/Allwmake)
ls -ltrR
...
-rwxr-xr-x 1 anirban staff 57601 2011-07-28 14:43 libPstream.so (in mpt-2.03)
...
-rwxr-xr-x 1 anirban staff 8087244 2011-07-28 14:53 libOpenFOAM.so
...
-rwxr-xr-x 1 anirban staff 27599977 2011-07-28 15:41 libfiniteVolume.so
...
-rwxr-xr-x 1 anirban staff 5532723 2011-07-28 15:53 libdynamicMesh.so
...
　
Any clues?
Thanks a lot
Anirban

[wyldckat]

Hi Anirban,

Check this post: http://www.cfd-online.com/Forums/ope...tml#post318001 - the last part might be the solution, but the rest of the post also has some ideas.

Best regards,
Bruno

[wyldckat]

Hello again Anirban,

I've been thinking a bit more about this and I would like to see the build log. Please run Allwmake again and pack+post the resulting log:

./Allwmake > make.log 2>&1
tar -czf make.log.tar.gz make.log

Well, before packing the file, you might want to replace any sensitive data with pseudonyms...

I'm asking this because I've got a feeling that there is something else missing during build time that you might have overlooked.

In the mean time, I reviewed the link I posted on the other thread and this caught my eyes:

-lmpi -lmpi++abi1002 -lsma -lpthread  for C++.

There might be some more dependencies that are being overlooked during build time, but this will depend on your platform and SGI MPI version... which I'm not familiar with

The other possibility is that there is a problem with the way that library connections are established. In other words, some compilers/linkers are very picky with which libraries should things be built with. Fedora 13 and above have this policy, as well as Windows OS.
OK, I wasn't very clear with "other words"... what I mean is that some linkers require users to explicitly indicate which libraries we wish them to link with, thus reducing margins or error at build time.

Another possibility is the Intel compiler you are using. According to this reply to a bug report:

Originally Posted by http://www.openfoam.com/mantisbt/view.php?id=257#c571

Intel compiler version : 11.1.059 is not supported for compiling OpenFOAM-2.0.x due to the need to support recent gcc and clang releases and keep up with the ever-changing interpretation of template specialisation.

OpenFOAM-2.0.x has been tested with the Intel compiler version 12.0.3.

Either way, gathering more information on how things actually built would be very helpful for me to infer what the problem is.

I advise you to keep an eye on the other thread as well

Best regards,
Bruno

[wyldckat]

Greetings to all!

I hope you guys managed to get things going with SGI's MPT!
But since you didn't give any further feedback about it and since I wasn't sure on how to proceed with the following news, because there are 3 threads that I know of about this subject, here goes...

Recently the build rules for SGIMPI have been updated on OpenFOAM 2.0.x, as you can see here: https://github.com/OpenFOAM/OpenFOAM...3720299a63adea
It assumes that MPI_ROOT is properly defined on the shell environment, prior to sourcing OpenFOAM's bashrc file.

I'm not sure if this will solve every issue you've had so far, but at least this is something new to me that I thought it would be nice to share with you

Best regards,
Bruno