[jgross]

Hi,

I am trying to link my already installed version of OF-extend 4.0 with the CUDA solvers. I have followed the installation guide of CUDA correctly, and it is working just fine for the examples (i.e. deviceQuery, etc) provided in the build. Therefore, it must have something to do with the way OF-extend and CUDA are linked.

I have followed the advice from a forum post of a similar, but slightly different issue:

https://www.cfd-online.com/Forums/openfoam-installation/128834-extend-3-0-cuda-installation.html

However, any time I try to run the Allwmake in $FOAM_SRC/cudaSolvers directory, I consistently receive the following:

No nvcc - CUDA not available.

The Allwmake script is as follows:

#!/bin/bash
cd ${0%/*} || exit 1    # run from this directory

if [ -f $CUDA_BIN_DIR/nvcc ]
then
    if [ -z $CUDA_IGNORE ]
    then
        echo "Found nvcc  --  enabling CUDA support."
        wmake libso
    else
        echo "Found nvcc but \$CUDA_IGNORE set -- not  enabling CUDA support."
    fi
else
    echo "No nvcc - CUDA not available."
fi

This means that the Allwmake is not finding the nvcc file in $CUDA_BIN_DIR. I have exported it correctly in both the prefs.sh and settings.sh located in the .../foam-extend-4.0/etc directory.

export CUDA_DIR=/usr/local/cuda-9.1
export CUDA_BIN_DIR=$CUDA_DIR/bin

Furthermore, when I run

which nvcc

I get the following output:

/usr/local/cuda-9.1/bin/nvcc

I thought I could try to export the directories directly in the Allwmake script because they are clearly not using the prefs.sh or settings.sh to find the file. However, when I try this, I get the following output:

cudaSolver/cudaTypes.H:57:29: fatal error: cusp/coo_matrix.h: No such file or directory
compilation terminated.
In file included from cudaCG/cudaCG.C:26:0:
cudaCG/cudaCG.H:43:24: fatal error: cudaSolver.H: No such file or directory
compilation terminated.
cudaSolver/cudaSolver.dep:268: recipe for target 'Make/linux64GccDPOpt/cudaSolver.o' failed
make: *** [Make/linux64GccDPOpt/cudaSolver.o] Error 1
make: *** Waiting for unfinished jobs....
cudaCG/cudaCG.dep:269: recipe for target 'Make/linux64GccDPOpt/cudaCG.o' failed
make: *** [Make/linux64GccDPOpt/cudaCG.o] Error 1
In file included from cudaBiCGStab/cudaBiCGStab.C:34:0:
cudaBiCGStab/cudaBiCGStab.H:43:24: fatal error: cudaSolver.H: No such file or directory
compilation terminated.
cudaBiCGStab/cudaBiCGStab.dep:269: recipe for target 'Make/linux64GccDPOpt/cudaBiCGStab.o' failed
make: *** [Make/linux64GccDPOpt/cudaBiCGStab.o] Error 1
cudaCG/cgAmg.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgAmg.dep:8: recipe for target 'Make/linux64GccDPOpt/cgAmg.o' failed
make: *** [Make/linux64GccDPOpt/cgAmg.o] Error 1
cudaCG/cgDiag.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaBiCGStab/bicgDiag.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgAinv.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgDiag.dep:8: recipe for target 'Make/linux64GccDPOpt/cgDiag.o' failed
make: *** [Make/linux64GccDPOpt/cgDiag.o] Error 1
cudaBiCGStab/bicgAinv.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaBiCGStab/bicgDiag.dep:7: recipe for target 'Make/linux64GccDPOpt/bicgDiag.o' failed
make: *** [Make/linux64GccDPOpt/bicgDiag.o] Error 1
cudaCG/cgAinv.dep:8: recipe for target 'Make/linux64GccDPOpt/cgAinv.o' failed
make: *** [Make/linux64GccDPOpt/cgAinv.o] Error 1
cudaBiCGStab/bicgAinv.dep:7: recipe for target 'Make/linux64GccDPOpt/bicgAinv.o' failed
make: *** [Make/linux64GccDPOpt/bicgAinv.o] Error 1

So it is not recognising all of the files located in the directory $FOAM_SRC/cudaSolvers, even though they are all there.

Does anyone have any suggestions as to how to get the Allwmake to run correctly?

Thank you in advance.

J

[wyldckat]

Greetings jgross,

Run the following command:

ls -l $FOAM_SRC/cudaSolvers/lnInclude

If it does not give you a list of files symbolically linked to other files, including the file "cudaTypes.H", then that's the reason why it's not working.

Either way, try running the following commands, to try and see if it solves the issue:

cd $FOAM_SRC/cudaSolvers
wclean libso
wmake libso
ls -l $FOAM_SRC/cudaSolvers/lnInclude

Beyond this, I have no idea with which CUDA versions foam-extend 4.0 has been tested with. It's possible that it may only work with CUDA 8.x or 7.x.

Best regards,
Bruno

[jgross]

Hi Bruno,

Thank you for your response. Your assistance is greatly appreciated.

I checked the contents of the lnInlcude directory, and received all the necessary symbolic links:

total 0
lrwxrwxrwx 1 jg847 jg847 28 Mar 18 19:34 buildNormFactor.H -> ../include/buildNormFactor.H
lrwxrwxrwx 1 jg847 jg847 30 Mar 18 19:34 cudaBiCGStab.C -> ../cudaBiCGStab/cudaBiCGStab.C
lrwxrwxrwx 1 jg847 jg847 30 Mar 18 19:34 cudaBiCGStab.H -> ../cudaBiCGStab/cudaBiCGStab.H
lrwxrwxrwx 1 jg847 jg847 18 Mar 18 19:34 cudaCG.C -> ../cudaCG/cudaCG.C
lrwxrwxrwx 1 jg847 jg847 18 Mar 18 19:34 cudaCG.H -> ../cudaCG/cudaCG.H
lrwxrwxrwx 1 jg847 jg847 26 Mar 18 19:34 cudaSolver.C -> ../cudaSolver/cudaSolver.C
lrwxrwxrwx 1 jg847 jg847 26 Mar 18 19:34 cudaSolver.H -> ../cudaSolver/cudaSolver.H
lrwxrwxrwx 1 jg847 jg847 25 Mar 18 19:34 cudaTypes.H -> ../cudaSolver/cudaTypes.H
lrwxrwxrwx 1 jg847 jg847 26 Mar 18 19:34 fillCOOMatrix.H -> ../include/fillCOOMatrix.H
-rw-rw-r-- 1 jg847 jg847  0 Mar 18 19:34 uptodate

Also, I tried your suggestion, but I am still receiving the same error message:

In file included from cudaCG/cudaCG.C:26:0:
cudaCG/cudaCG.H:43:24: fatal error: cudaSolver.H: No such file or directory
compilation terminated.
cudaCG/cudaCG.dep:269: recipe for target 'Make/linux64GccDPOpt/cudaCG.o' failed
make: *** [Make/linux64GccDPOpt/cudaCG.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from cudaSolver/cudaSolver.H:42:0,
                 from cudaSolver/cudaSolver.C:26:
cudaSolver/cudaTypes.H:57:29: fatal error: cusp/coo_matrix.h: No such file or directory
compilation terminated.
In file included from cudaBiCGStab/cudaBiCGStab.C:34:0:
cudaBiCGStab/cudaBiCGStab.H:43:24: fatal error: cudaSolver.H: No such file or directory
compilation terminated.
cudaSolver/cudaSolver.dep:268: recipe for target 'Make/linux64GccDPOpt/cudaSolver.o' failed
make: *** [Make/linux64GccDPOpt/cudaSolver.o] Error 1
cudaBiCGStab/cudaBiCGStab.dep:269: recipe for target 'Make/linux64GccDPOpt/cudaBiCGStab.o' failed
make: *** [Make/linux64GccDPOpt/cudaBiCGStab.o] Error 1
cudaCG/cgDiag.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaBiCGStab/bicgAinv.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgDiag.dep:8: recipe for target 'Make/linux64GccDPOpt/cgDiag.o' failed
make: *** [Make/linux64GccDPOpt/cgDiag.o] Error 1
cudaBiCGStab/bicgAinv.dep:7: recipe for target 'Make/linux64GccDPOpt/bicgAinv.o' failed
make: *** [Make/linux64GccDPOpt/bicgAinv.o] Error 1
cudaCG/cgAmg.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgAmg.dep:8: recipe for target 'Make/linux64GccDPOpt/cgAmg.o' failed
make: *** [Make/linux64GccDPOpt/cgAmg.o] Error 1
cudaCG/cgAinv.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaBiCGStab/bicgDiag.cu:47:23: fatal error: cudaTypes.H: No such file or directory
compilation terminated.
cudaCG/cgAinv.dep:8: recipe for target 'Make/linux64GccDPOpt/cgAinv.o' failed
make: *** [Make/linux64GccDPOpt/cgAinv.o] Error 1
cudaBiCGStab/bicgDiag.dep:7: recipe for target 'Make/linux64GccDPOpt/bicgDiag.o' failed
make: *** [Make/linux64GccDPOpt/bicgDiag.o] Error 1

I will try to install CUDA 8.x later to see if that fixes the problem.

James

[jgross]

Hi Bruno,

It turns out I didn't link the Cusp library correctly as it needed to be in the directory:

$WM_THIRD_PARTY_DIR/LocalDev

I am able to run

wmake libso

without the same errors. I'm not 100% sure it installed correctly, but at least it's a different from before. I would like to test the cudaSolvers, however I cannot link

"libcudaSolvers.so"

in the controlDict as it the .so file does not exist.

My guess is that although it is recognising those files now, it still encountered some fatal errors during installation, even though I didn't find any fatal error messages in the log file.

Any advice on how to proceed further with this?

James

[ThomasLong]

I am facing the same problem now. I have just finished the installation of CUDA-11.1 on Ubuntu 18.04 of WSL2 and it works well for the examples. Then the compilation of Foam-extend 4.0 has no problems. But when I run ./Allwmake in $FOAM_SRC/cudaSolvers directory , I recieve the same message "No nvcc - CUDA not available." as you. I want to know if you have any sollution to this problem.

[tav]

Originally Posted by jgross

Hi Bruno,

It turns out I didn't link the Cusp library correctly as it needed to be in the directory:

$WM_THIRD_PARTY_DIR/LocalDev

I am able to run

wmake libso

without the same errors. I'm not 100% sure it installed correctly, but at least it's a different from before. I would like to test the cudaSolvers, however I cannot link

"libcudaSolvers.so"

in the controlDict as it the .so file does not exist.

My guess is that although it is recognising those files now, it still encountered some fatal errors during installation, even though I didn't find any fatal error messages in the log file.

Any advice on how to proceed further with this?

James

Hi James,

I am getting the same error while compiling foam-extend 5.0. Can you please share how you linked the Cusp library correctly?