[fabianpk]

Hi,
I've been trying to optimize my code, by running kcachegrind/valgrind on a version of my code compiled with the debug flags (changed it in that cshrc file). After running 'Allwmake' and then also recompiling lam-7.1.1 to create the libPstream file, I'm at loss what to do.

Down here is what I get when I try to run wmake on my code.

/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/wmake/tcshScripts/mkObjectDir /users/tfd/f98faka/OpenFOAM/f98faka-1.1/applications/bin/linuxDebug/cavDebugFoam
g++ -m32 -Dlinux -Wall -W -Wno-unused-parameter -O0 -fdefault-inline -ggdb3 -DFULLDEBUG -DNoRepository -ftemplate-depth-30 -I/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/src/OpenFOAM/lnInclude -IlnInclude -I. -I/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/src/cfdTools/incompressible -I/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/src/cfdTools/lnInclude -I/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/src/transportModels -DFACE_DECOMP -fPIC -pthread Make/linuxDebug/cavDebugFoam.o -L/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug \
-L/users/tfd/f98faka/OpenFOAM/f98faka-1.1/lib/linuxDebug -ldynamicMesh -lfaceDecompFiniteElement -lincompressibleTransportModels -lcfdTools -lOpenFOAM -ggdb3 -DFULLDEBUG -lm -o /users/tfd/f98faka/OpenFOAM/f98faka-1.1/applications/bin/linuxDebug/cavDebugFoam
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `dlerror'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `MPI::Is_initialized()'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Comm::mpi_comm_map'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `dlclose'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `MPI:p::Init(void (*)(void const*, void*, int, MPI::Datatype const&), bool)'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `MPI:p::Free()'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `dlopen'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `dlsym'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `op_intercept'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Intracomm::current_op'
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `PMPI::Comm::mpi_err_map'
collect2: ld returned 1 exit status
make: *** [/users/tfd/f98faka/OpenFOAM/f98faka-1.1/applications/bin/linuxDebug/cavDebugFoa m] Error 1

What else should I recompile?

/Fabian

[pilot]

Hey Fabian

OK the help disclaimer: I am not an expert but I have had troubles/experience compiling and installing OpenFoam so I know a couple things. So let me give you a couple of my thoughts:

**** Compiling libPstream.so ****

Go into:
OpenFOAM/OpenFOAM-1.1/src/Pstream/mpi

you should see a Pstream.C file there. That is the source for this missing .so file. That is what you compile to get the library. So to have it compiled you type in its directory:

wmake libso ./

That will compile just the Pstream library. The library should be placed in

OpenFOAM/OpenFOAM-1.1/lib/linuxOpt/lam-7.1.1

You may not need to compile Pstream it you might just need to link it properly.

**** Linking libPstream.so ****

So from your output it seems that your compiler is looking for Pstream in:

OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1

Once you are sure that you have libPstream.so compiled you can place a symbolic link in the "linuxDebug/lam-7.1.1" folder. Or you can go into the Make folder of what you are compiling/optimising and add the following to the EXE.LIB section of options file.

-l/<openfoam>/OpenFOAM-1.1/lib/linuxOpt/lam-7.1.1

**** Debug version of libPstream.so ****

There is another option you can re-compile the PStream lib with debugging options as you are trying to do by going into:

OpenFOAM/OpenFOAM-1.1/src/Pstream/mpi/Make

Add the -ggdb3 -DFULLDEBUG flags (Along with any other flags you want) mannually to the options file. Then the libPstream.so will have all your debugging symbols. Then you can place the compiled libPstream.so directly in the "linuxDebug/lam-7.1.1" folder. Then you can recompile libPstream.so without the debug flags and place it in linuxOpt folder. That way you will have a debug version and a optimised version.

There are my thoughts hopefully that will help you out.

Take Care
Mike McW

[fabianpk]

I'm afraid nothing changes when I compile the Pstream library, it just says that they compile fine (I did a wclean first), but my code still won't compile using the debug binary libraries. The error message is the same.

/Fabian

[mattijs]

Seems your linking of cavDebugFoam does not pick up all libraries. It misses:

libdl.so (dlopen etc.)
liblam.so (mpi_init)

Could it be that you do not have the correct settings to use lam for your architecture 'linuxDebug' and it picks up the default system mpi?

Check your $WM_DIR/rules for linuxDebug.

[fabianpk]

I managed to get it working by changing WM_MPLIB to MPICH instead of LAM. Now I can't run mpirun and lamboot, so this means I can't run in parallel?


/Fabian

[hjasak]

MPICH does not use a deamon so if you've done everything right it should just run with mpirun.

Hrv

[mattijs]

Dear All,

1.3 there are three useful environment variable when developing:

FOAM_ABORT : make OF abort and produce a core dump if there is an error in an input. This is useful e.g. when there is an error in a dictionary and you're trying to find out which dictionary.

FOAM_SIGFPE : make OF abort if there is a floating point exception (e.g. division by 0)

FOAM_SETNAN (new): initialize all blocks of dynamically allocated memory to a special floating point value (NaN). It will catch usage of unitialized memory.
- only if the numbers are floating point numbers
- only works on Linux (see sigFpe.C)
- only tested for double precision

[liu]

For the problem due to dlerror, dlopen etc like:
/users/tfd/f98faka/OpenFOAM/OpenFOAM-1.1/lib/linuxDebug/lam-7.1.1/libPstream.so: undefined reference to `dlerror'

Solution: in the corresponding Make/options file
LIB_LIBS = ...\
...\
-dl

The linking flag '-dl' is not generated in my RHEL either. So I added it explicitly and solve the problem.

[liu]

sorry, it should be '-ldl'.