[Argen]

I am trying to link a linear solver based on PETSc with OpenFoam. But I got some errors,

-L/opt/64/intel/fce/9.1.045/lib \ -L/home/s/OpenFOAM/ThirdParty/gcc-4.3.1/platforms/linux64/lib64/ \ ./libP1Module.a -lfiniteVolume
-lOpenFOAM -liberty -ldl \ -lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \ -lm -o /home/s/OpenFOAM/s-1.5/applications
/bin/linux64GccDPOpt/PPPP ./libP.a(mypetsc.o)(.text+0x3c): In function `linearsolvers_petsc_mp_solvelinearsystem_petsc_':
./libP.a(mypetsc.o)(.text+0x57):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:100: undefined reference to `mpi_comm_rank_'
./libP.a(mypetsc.o)(.text+0x72):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:101: undefined reference to `mpi_comm_size_'
./libP.a(mypetsc.o)(.text+0x8d):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:113: undefined reference to `matcreate_'
./libP.a(mypetsc.o)(.text+0xb2):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:114: undefined reference to `matsetsizes_'
./libP.a(mypetsc.o)(.text+0xc3):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:115: undefined reference to `matsetfromoptions_'
./libP.a(mypetsc.o)(.text+0xde):~/OpenFOAM/s/applications/solvers/p/mypetsc.F90:121: undefined reference to `matgetownershiprange_'

Any idea to solve it?

[boger]

Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.

[Argen]

Originally Posted by boger

Have you included the appropriate petsc header files in your code? Of course, you'll also then need to add the include path and library link in your Make/options file.

Hi David,

Yes. I am sure that the appropriate PETSc header files were included. I tested it alone and found no problem. I also added libP.a generated from my entire code with one module based on petsc in the Make/options files. So, can you figure out what I should do next?

Thanks,
Ray

[boger]

Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?

[boger]

Try something like this in your Make/options:

include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
     $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm

This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.

[Argen]

Originally Posted by boger

Can you attach your Make/options file?

How did you "test it alone" if you cannot compile it?

The Make/options file is

EXE_INC = \
-I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
./libP.a \
-lfiniteVolume

where libP.a is generated from my entire code. There is no problem to build libP.a.

I changed the linear-solver module using petsc to a program and then run it.

[Argen]

Originally Posted by boger

Try something like this in your Make/options:

include $(RULES)/mplib$(WM_MPLIB)

EXE_INC = \
     $(PFLAGS) $(PINC) \
    -I$(PETSC_DIR)/include -I$(PETSC_DIR)/$(PETSC_ARCH)/include \
    -I$(LIB_SRC)/finiteVolume/lnInclude

EXE_LIBS = \
    -lfiniteVolume \
    -L$(PETSC_LIBDIR) -lpetsc -lpetscmat -lpetscvec -lpetscksp -lpetscdm

This is based on a slightly old version of PETSc. I think newer versions may have consolidated the libraries into one.

When the above modification is applied, I got a new error like,
-lsvml -lifport -lifcore -limf -lm -lipgo -lirc -lgcc_s -lirc_s -ldl \
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D
/usr/bin/ld: cannot find -lpetscmat
collect2: ld returned 1 exit status
make: *** [/home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/P1M1D] Error 1

Any idea about it?

[boger]

Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.

[Argen]

Originally Posted by boger

Yes, I forgot to mention that you are responsible for PETSC_DIR, PETSC_ARCH, and PETSCLIB_DIR to point to the directories where the include files and compiled libraries are stored. The PETSc documentation (e.g., the pdf file available on-line) discusses the definition of the first two of these and also discusses compiling PETSc as a shared library and linking to it.

I already set the first two. But I have no idea about the third one. Can you let me know that?

[boger]

PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.

[Argen]

Originally Posted by boger

PETSCLIB_DIR should point to the directory where the compiled PETSc libraries (*.so files) are stored, consistent with the usual -L -l syntax for linking libraries. The answer will depend on how you configured and installed PETSc.

First, PETSCLIB_DIR should be PETSC_LIBDIR mentioned before. Is it right?
In my case, two libaries (lib1.so and lib2.so) are stored in /home/s/PETSc/petsc-3.1-p8 which is the same as $PETSC_DIR. Well, after setting PETSC_LIBDIR, I got almost same error. The only difference is that
/usr/bin/ld: cannot find -lpetsc

Before, it's /usr/bin/ld: cannot find -lpetscmat

[boger]

As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.

[Argen]

Originally Posted by boger

As I said, my example was from an older version of petsc, and they have since consolidated the libraries. Just shorten the list of libraries (the -l's) to the ones you have/need.

Do you still remember which version you used?

[boger]

3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.

[Argen]

Originally Posted by boger

3.0.0-p8. But what's important to you is that the change notes for 3.1 mention that the configure was modified to compile a single library file, so just keep the -lpetsc and delete the other -lpet* entries I gave you.

Yes, I think so. But even those -lpet* are deleted, the errors become
-lm -o /home/s/OpenFOAM/s/applications/bin/linux64GccDPOpt/PPP
./libP1Module.a(mypetsc.o)(.text+0xb2): In function `solvelinearsystem_petsc':
/~/OpenFOAM/s/applications/solvers/p/pp/mypetsc.F90:99: undefined reference to `petscinitialize_'
...

I wonder that you called some petsc subroutines inside OpenFOAM or called them in your own module linked with OF?
Anyway, is it due to something wrong with building libP.a from my code?

[boger]

Sorry, my only experience is calling PETSc directly from within OpenFOAM.

[Argen]

Originally Posted by boger

Sorry, my only experience is calling PETSc directly from within OpenFOAM.

That's fine. Thanks a lot.