[chiven]

The linux system is "Linux bxfs1 2.6.18-164.6.1.el5 #1 SMP Tue Oct 27 11:28:30 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux".
I am trying to use the system's MPI of "/opt/FJSVplang/bin/mpiexec".

tar -xzf OpenFOAM-1.6.General.gtgz
tar -xzf ThirdParty-1.6.General.gtgz
tar -xzf ThirdParty-1.6.linux64Gcc.gtgz

Edit OpenFOAM/OpenFOAM-1.6/etc/cshrc
# WM_MPLIB = | OPENMPI | MPICH | MPICH-GM | HPMPI | GAMMA | MPI | QSMPI
if ( ! $?WM_MPLIB ) setenv WM_MPLIB FJMPI

Edit OpenFOAM/OpenFOAM-1.6/etc/settings.csh
case FJMPI:
setenv MPI_ARCH_PATH /opt/FJSVplang
setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpi
_foamAddPath $MPI_ARCH_PATH/bin
_foamAddPath /opt/FJSVpnple/bin
_foamAddPath /opt/FJSVmpi2/bin
_foamAddLib /opt/FJSVpnidt/lib
_foamAddLib /opt/FJSVplang/lib64
_foamAddLib /opt/FJSVpnple/lib
_foamAddLib /opt/FJSVmpi2/lib/sparcv9
_foamAddLib /opt/FJSVpnple/lib/sparcv9
breaksw

/OpenFOAM/OpenFOAM-1.6
./Allwmake

However, I meet the errors shown in follow. Any suggestion is appreciated.

make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumCO'
Making dependency list for source file equilibriumCO.C
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumCO'
make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumCO'
SOURCE=equilibriumCO.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/equilibriumCO.o
g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC Make/linux64GccDPOpt/equilibriumCO.o -L/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt \
             -lspecie -lOpenFOAM -liberty -ldl   -lm -o /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/equilibriumCO
/usr/bin/ld: warning: libPstream.so, needed by /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so, not found (try using -rpath or -rpath-link)
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `vtable for Foam::OPstream'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::~OPstream()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::IPstream(Foam::Pstream::commsTypes, int, int, Foam::IOstream::streamFormat, Foam::IOstream::versionNumber)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::exit(int)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::write(Foam::Pstream::commsTypes, int, char const*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::abort()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::read(Foam::Pstream::commsTypes, int, char*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::reduce(double&, Foam::sumOp<double> const&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::init(int&, char**&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::addValidParOptions(Foam::HashTable<Foam::string, Foam::word, Foam::string::hash>&)'
collect2: ld returned 1 exit status
make[2]: *** [/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/equilibriumCO] Error 1
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumCO'
make[1]: *** [equilibriumCO] Error 2
make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumFlameT'
Making dependency list for source file equilibriumFlameT.C
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumFlameT'
make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumFlameT'
SOURCE=equilibriumFlameT.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/equilibriumFlameT.o
g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC Make/linux64GccDPOpt/equilibriumFlameT.o -L/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt \
             -lspecie -lOpenFOAM -liberty -ldl   -lm -o /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/equilibriumFlameT
/usr/bin/ld: warning: libPstream.so, needed by /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so, not found (try using -rpath or -rpath-link)
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `vtable for Foam::OPstream'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::~OPstream()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::IPstream(Foam::Pstream::commsTypes, int, int, Foam::IOstream::streamFormat, Foam::IOstream::versionNumber)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::exit(int)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::write(Foam::Pstream::commsTypes, int, char const*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::abort()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::read(Foam::Pstream::commsTypes, int, char*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::reduce(double&, Foam::sumOp<double> const&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::init(int&, char**&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::addValidParOptions(Foam::HashTable<Foam::string, Foam::word, Foam::string::hash>&)'
collect2: ld returned 1 exit status
make[2]: *** [/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/equilibriumFlameT] Error 1
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/equilibriumFlameT'
make[1]: *** [equilibriumFlameT] Error 2
make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/mixtureAdiabaticFlameT'
Making dependency list for source file mixtureAdiabaticFlameT.C
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/mixtureAdiabaticFlameT'
make[2]: Entering directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/mixtureAdiabaticFlameT'
SOURCE=mixtureAdiabaticFlameT.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/mixtureAdiabaticFlameT.o
g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/thermophysicalModels/specie/lnInclude -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC Make/linux64GccDPOpt/mixtureAdiabaticFlameT.o -L/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt \
             -lspecie -lOpenFOAM -liberty -ldl   -lm -o /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/mixtureAdiabaticFlameT
/usr/bin/ld: warning: libPstream.so, needed by /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so, not found (try using -rpath or -rpath-link)
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `vtable for Foam::OPstream'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::~OPstream()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::IPstream(Foam::Pstream::commsTypes, int, int, Foam::IOstream::streamFormat, Foam::IOstream::versionNumber)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::exit(int)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::OPstream::write(Foam::Pstream::commsTypes, int, char const*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::waitRequests()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::abort()'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::IPstream::read(Foam::Pstream::commsTypes, int, char*, long)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::reduce(double&, Foam::sumOp<double> const&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::init(int&, char**&)'
/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libOpenFOAM.so: undefined reference to `Foam::Pstream::addValidParOptions(Foam::HashTable<Foam::string, Foam::word, Foam::string::hash>&)'
collect2: ld returned 1 exit status
make[2]: *** [/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/bin/linux64GccDPOpt/mixtureAdiabaticFlameT] Error 1
make[2]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical/mixtureAdiabaticFlameT'
make[1]: *** [mixtureAdiabaticFlameT] Error 2
make[1]: Target `application' not remade because of errors.
make[1]: Leaving directory `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/applications/utilities/thermophysical'
make: *** [thermophysical] Error 2
make: Target `application' not remade because of errors.
+ '[' '' = doc ']'
[e090012@bxfs1 OpenFOAM-1.6]$ 

[chiven]

Hi again, I found it is because mpi/libPstream.so is NOT complied successfully. But what a pity I have NO idea to solve it. Any suggestions are appreciated.

++ '[' FJMPI = MPI -o -d /opt/FJSVplang ']'
++ cd ParMetis-3.1
++ WM_OPTIONS=linux64GccDPOptFJMPI
++ wmake libso METISLib
linux64GccDPOptFJMPI/options:5: /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI: No such file or directory
make: *** No rule to make target `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI'. Stop.
wmake error: file 'Make/linux64GccDPOptFJMPI/objectFiles' could not be created
++ wmake libso ParMGridGen-1.0/MGridGen/IMlib
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libIMlib.so' is up to date.
++ wmake libso ParMGridGen-1.0/MGridGen/Lib
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/libMGridGen.so' is up to date.
+ src/Allwmake
+ wmakePrintBuild -check
no git description found
+ /bin/rm -f OpenFOAM/Make/linux64GccDPOpt/global.C OpenFOAM/Make/linux64GccDPOpt/global.o
+ wmakeLnInclude OpenFOAM
+ wmakeLnInclude OSspecific/POSIX
+ Pstream/Allwmake
+ wmake libso dummy
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/dummy/libPstream.so' is up to date.
+ case "$WM_MPLIB" in
+ set +x
Note: ignore spurious warnings about missing mpicxx.h headers
+ WM_OPTIONS=linux64GccDPOptFJMPI
+ wmake libso mpi
linux64GccDPOptFJMPI/options:5: /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI: No such file or directory
make: *** No rule to make target `/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI'. Stop.
wmake error: file 'Make/linux64GccDPOptFJMPI/objectFiles' could not be created

[olesen]

Originally Posted by chiven

Hi again, I found it is because mpi/libPstream.so is NOT complied successfully. But what a pity I have NO idea to solve it. Any suggestions are appreciated.

++ '[' FJMPI = MPI -o -d /opt/FJSVplang ']'
++ cd ParMetis-3.1
++ WM_OPTIONS=linux64GccDPOptFJMPI
++ wmake libso METISLib
linux64GccDPOptFJMPI/options:5: /home/g2/e090012/OpenFOAM/OpenFOAM-1.6/wmake/rules/linux64Gcc/mplibFJMPI: No such file or directory

There is a mplibFJMPI rule here:

wmake/rules/SunOS64Gcc/mplibFJMPI

but apparently nothing equivalent for linux64.

If I look at settings.{csh,sh} it appears that the current FJMPI settings are specific for 'sparcv9'. You'll need appropriate values there, and also an appropriate wmake/rules/General/mplibFJMP

[chiven]

Hi, Mark, thanks a lot for the reply.

I look through the file of wmake/rules/SunOS64Gcc/mplibFJMPI. I am afraid this time it is temporary out of my knowledge in the linux to revise a rule.
The purpose I want to use the "/opt/FJSVplang/bin/mpiexec" embeded in the system is to do a parallel calculation with more than one node.

I can use the ThirdParty/openmpi-1.3.3 well only within one node with less than 8 CPU cores. So I am trying to recompile the OF-1.6. But unfortunately...

Any more suggestions.
with best regards,
Chiven

[chiven]

Hi, Mark, again, you are right, the current FJMPI settings of OpenFOAM-1.6 is for the previous version of Fujitsu MPI. Now we are using a new system with new FJMPI version. The differences between the two versions are shown as follows.

･environment variable PATH(Previous version)
/opt/FJSVmpi2/bin、
/opt/FSUNaprun/bin、
INSTALL/FSUNf90/bin
･environment variable(Previous version)
LD_LIBRARY_PATH_64
/opt/FJSVmpi2/lib/sparcv9、
/opt/FSUNaprun/lib/sparcv9、
INSTALL/FSUNf90/lib/sparcv9

･environment variable PATH(This version)
/opt/FJSVmpi2/bin、
/opt/FJSVpnple/bin、
INSTALL/FSUNf90/bin
･environment variable(This version)
LD_LIBRARY_PATH_64
/opt/FJSVmpi2/lib/sparcv9、
/opt/FJSVpnple/lib/sparcv9、
INSTALL/FSUNf90/lib/sparcv9

[olesen]

Originally Posted by chiven

I can use the ThirdParty/openmpi-1.3.3 well only within one node with less than 8 CPU cores. So I am trying to recompile the OF-1.6. But unfortunately...

Using openmpi is really the path of least resistance. But I don't see why you need to recompile things. AFAIK there should already be openmpi libraries are part of the binary pack. You should just need to adjust your settings to select OPENMPI as the mpi library and re-source your OpenFOAM environment.

If you want to use a newer openmpi (eg, openmpi-1.4.1), you'll obviously need to recompile it and the src/Pstream as well, but shouldn't need to recompile everything.

In general, however, it probably best to be using the 1.6.x version with the current bug fixes and other enhancements.

[chiven]

Originally Posted by olesen

Using openmpi is really the path of least resistance. But I don't see why you need to recompile things.

The computer has many nodes, every node has 8 cores (threads). When the processor number is not larger than 8, one node is enough. In this case, the parallel calculation of openmpi is good.
However, if the more than 8 processors are need, 2 nodes or more should be used. This time, the parallel calculation of openmpi is not good (very slow).
I am sorry for not clearly express. If you are interesting in it about the details, I can send the user manual to you.

This time, I am wondering how to set the openmpi for the computer. And the administrator also only know the Fujitsu Job scheduler.

[chiven]

Hi, I revised the settings.csh as follows.

case FJMPI:
    setenv MPI_ARCH_PATH /opt/FJSVmpi2
    setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpi
    _foamAddPath $MPI_ARCH_PATH/bin
    _foamAddLib  $MPI_ARCH_PATH/lib/sparcv9
    _foamAddLib  /opt/FJSVpnple/lib/sparcv9
    _foamAddLib  /opt/FJSVpnidt/lib
    breaksw

Copy the wmake/rules/SunOS64Gcc/mplibFJMPI to wmake/rules/linux64Gcc/, and revised it as:

PFLAGS     =
PINC       = -I$(MPI_ARCH_PATH)/include/sparcv9/mpi -DMPIPP_H
PLIBS      = -L$(MPI_ARCH_PATH)/lib/sparcv9 -lmpi_f -L/opt/FJSVf90/lib/sparcv9 -lfjgmp64 -L/opt/FJSVpnidt/lib -lfjidt -ljrm -lfj90i -lfj90f -lfj90fmt  -lelf

Then in the directory of OpenFOAM-1.6/, run
./Allwmake

But I met the following errors.

+ Pstream/Allwmake
+ wmake libso dummy
'/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/lib/linux64GccDPOpt/dummy/libPstream.so' is up to date.
+ case "$WM_MPLIB" in
+ set +x
Note: ignore spurious warnings about missing mpicxx.h headers
+ WM_OPTIONS=linux64GccDPOptFJMPI
+ wmake libso mpi
SOURCE=OPwrite.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40  -I/opt/FJSVmpi2/include/sparcv9/mpi -DMPIPP_H -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOptFJMPI/OPwrite.o
OPwrite.C:30:17: error: mpi.h: No such file or directory
OPwrite.C:38: error: 'MPI_Request' was not declared in this scope
OPwrite.C:38: error: template argument 1 is invalid
OPwrite.C:38: error: invalid type in declaration before ';' token
OPwrite.C: In static member function 'static bool Foam::OPstream::write(Foam::Pstream::commsTypes, int, const char*, std::streamsize)':
OPwrite.C:81: error: 'MPI_PACKED' was not declared in this scope
OPwrite.C:84: error: 'MPI_COMM_WORLD' was not declared in this scope
OPwrite.C:85: error: 'MPI_Bsend' was not declared in this scope
OPwrite.C:93: error: 'MPI_PACKED' was not declared in this scope
OPwrite.C:96: error: 'MPI_COMM_WORLD' was not declared in this scope
OPwrite.C:97: error: 'MPI_Send' was not declared in this scope
OPwrite.C:101: error: 'MPI_Request' was not declared in this scope
OPwrite.C:101: error: expected `;' before 'request'
OPwrite.C:107: error: 'MPI_PACKED' was not declared in this scope
OPwrite.C:110: error: 'MPI_COMM_WORLD' was not declared in this scope
OPwrite.C:111: error: 'request' was not declared in this scope
OPwrite.C:112: error: 'MPI_Isend' was not declared in this scope
OPwrite.C:114: error: request for member 'append' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C: In static member function 'static void Foam::OPstream::waitRequests()':
OPwrite.C:132: error: request for member 'size' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C:138: error: request for member 'size' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C:139: error: request for member 'begin' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C:140: error: 'MPI_STATUSES_IGNORE' was not declared in this scope
OPwrite.C:141: error: 'MPI_Waitall' was not declared in this scope
OPwrite.C:150: error: request for member 'clear' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C: In static member function 'static bool Foam::OPstream::finishedRequest(Foam::label)':
OPwrite.C:157: error: request for member 'size' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C:162: error: request for member 'size' in 'OPstream_outstandingRequests_', which is of non-class type 'int'
OPwrite.C:170: error: invalid types 'int[const Foam::label]' for array subscript
OPwrite.C:170: error: 'MPI_STATUS_IGNORE' was not declared in this scope
OPwrite.C:170: error: 'MPI_Test' was not declared in this scope
make: *** [Make/linux64GccDPOptFJMPI/OPwrite.o] Error 1

Any comments? Thanks.

[olesen]

Originally Posted by chiven

Hi, I revised the settings.csh as follows.
[CODE]
+ wmake libso mpi
SOURCE=OPwrite.C ; g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3 -DNoRepository -ftemplate-depth-40 -I/opt/FJSVmpi2/include/sparcv9/mpi -DMPIPP_H -IlnInclude -I. -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/g2/e090012/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude -fPIC -c $SOURCE -o Make/linux64GccDPOptFJMPI/OPwrite.o
OPwrite.C:30:17: error: mpi.h: No such file or directory
.

Is there even an 'mpi.h' somewhere under /opt/FJSVmpi2 ?
For example,

    find  /opt/FJSVmpi2 -name mpi.h

should return something, and this path needs to be found by the include.

[chiven]

Originally Posted by olesen

Is there even an 'mpi.h' somewhere under /opt/FJSVmpi2 ?
For example,

    find  /opt/FJSVmpi2 -name mpi.h

should return something, and this path needs to be found by the include.

mpi.h is in the directory of

/opt/FJSVplang/include64/mpi/fujitsu/

[chiven]

Hi, good morning, Mark, thank you for the comments. With your helps, I just make a big progress.
The settings.csh is revised as follows.

case FJMPI:
    setenv MPI_ARCH_PATH /opt/FJSVplang
    setenv FOAM_MPI_LIBBIN $FOAM_LIBBIN/mpi
    _foamAddPath $MPI_ARCH_PATH/bin
    _foamAddPath /opt/FJSVpnple/bin
    _foamAddLib  $MPI_ARCH_PATH/lib64
    _foamAddLib  /opt/FJSVpnidt/lib
    _foamAddLib  /opt/FJSVpnple/lib
    breaksw

The mplibFJMPI is revised like,

PFLAGS     =
PINC       = -I$(MPI_ARCH_PATH)/include64/mpi/fujitsu -DMPIPP_H 
PLIBS      = -L$(MPI_ARCH_PATH)/lib64 -lmpi_f

The compile is done well, and no errors. I further check it through the command of foamInstallationTest, and it seems OK.
But when I try to run the cavity tutorial using icoFoam solver, I meet a problem:

icoFoam: symbol lookup error: /opt/FJSVplang/lib64/libmpi_f.so.3: undefined symbol: iargc_

I think it is because some parameters are not set. You know, the original rules for FJMPI is like:

PFLAGS     =
PINC       = -I$(MPI_ARCH_PATH)/include/sparcv9/mpi -DMPIPP_H
PLIBS      = -L$(MPI_ARCH_PATH)/lib/sparcv9 -lmpi_f -L/opt/FSUNf90/lib/sparcv9 -lfjgmp64 -L/opt/FJSVpnidt/lib -lfjidt -ljrm -lfj90i -lfj90f -lfj90fmt  -lelf

And I don't know what are the means of -lfjgmp64, -lfjidt, -ljrm, and so on. Thus I have no idea to improve it. Any comments?

[chiven]

I have known the simple means of the parameters, such as -lfjgmp64, -lfjidt, -ljrm, etc. They are the abbreviated forms of the files under the directory. And I use FLMPI rule as,

PFLAGS     =
PINC       = -I$(MPI_ARCH_PATH)/include64/mpi/fujitsu -DMPIPP_H 
PLIBS      = -L$(MPI_ARCH_PATH)/lib64 -lmpi_f -lmpi_f++ -lmpi_f++_mt -lfjgmp64 -lfj90i -lfj90f -lfj90rt -lfjcrt -lelf -L/opt/FJSVpnple/lib -ljrm -lple -lplecf -L/opt/FJSVpnidt/lib -lfjidt

But I still meet the same problem.

icoFoam: symbol lookup error: /opt/FJSVplang/lib64/libmpi_f.so.3: undefined symbol: iargc_