[klausb]

I found out, that the AmgX-OpenFOAM integration has been done in another way using a new interface called FOAM2CSR instead of the wrapper.

So it's:

OpenFOAM > PETSc4FOAM > FOAM2CSR > AmgX

See youtube: SU2 Conference 21: An Implementation of AmgX for GPU Acceleration of OpenFOAM

https://www.youtube.com/watch?v=1d1DVURNz4E


FOAM2CSR

https://gitlab.hpc.cineca.it/openfoam/foam2csr

[Nitish25]

Originally Posted by dlahaye

Let us talk very briefly now and find a slot to talk more extensively next week.

Please send me link for Google meet either here or on private cfdonline address.

Dhanyavaad.

Respected Sir,
As per the last conversation you told that to schedule meeting at 24 jan for detailed information
PF Link
https://meet.google.com/ujq-qheo-sdz

Thank you for your valuable time and efforts

Thanks & regards

[Nitish25]

Originally Posted by dlahaye

Let us talk very briefly now and find a slot to talk more extensively next week.

Please send me link for Google meet either here or on private cfdonline address.

Dhanyavaad.

Respected Sir,
As per the last conversation you told that to schedule meeting at 24 jan for detailed information
PF Link
https://meet.google.com/ujq-qheo-sdz

Thank you for your valuable time and efforts

Thanks & regards

[Nitish25]

Originally Posted by Nitish25

Respected Sir,
As per the last conversation you told that to schedule meeting at 24 jan for detailed information
PF Link
https://meet.google.com/ujq-qheo-sdz

Thank you for your valuable time and efforts

Thanks & regards

sir if u are free, plz join the meeting now, I'm eagerly waiting for u.

[dlahaye]

We scheduled our meeting for 16:00 this afternoon, i.e., 5.5 hours from now.

What is progress on compiling and linking AMGX tutorial, independently from OpenFoam?

[dlahaye]

Originally Posted by klausb

I found out, that the AmgX-OpenFOAM integration has been done in another way using a new interface called FOAM2CSR instead of the wrapper.

So it's:

OpenFOAM > PETSc4FOAM > FOAM2CSR > AmgX

See youtube: SU2 Conference 21: An Implementation of AmgX for GPU Acceleration of OpenFOAM

https://www.youtube.com/watch?v=1d1DVURNz4E


FOAM2CSR

https://gitlab.hpc.cineca.it/openfoam/foam2csr

Dear Klaus,

1/ I imagine that FOAM2CSR applies to the OpenFoam LDU matrix format. FOAM2CSR does *not* apply to the PETSc matrix format. Can you confirm?

2/ Does AMGX apply to an decomposed case, or does AMGX enforce its own parallel decomposition?

3/ When using AMGX to an incompressible case (like motorBike), AMGX is to be applied to the pressure field only?

Thanks. Domenico.

[Nitish25]

Originally Posted by dlahaye

We scheduled our meeting for 16:00 this afternoon, i.e., 5.5 hours from now.

What is progress on compiling and linking AMGX tutorial, independently from OpenFoam?

we have successfully build the amgx by following the steps as mention in their website https://github.com/NVIDIA/AMGX i.e. as shown below

mkdir build
cd build
cmake ../
make -j16 all

while, build it automatically compiles all the amgx example and generates their executable in the build directory.

we are able to run the amgx_capi (it is an executable file of AMGX/examples/amgx_capi.c )example on gpu inside the build directory by following command
examples/amgx_capi -m ../examples/matrix.mtx -c ../src/configs/FGMRES_AGGREGATION.json

so manually compilation of amgx/examples is not required.

Thank you for ur valuable time.

[dlahaye]

Yes and no.

Yes, I understand that the manually building the examples of AMGX/examples is *not* required.

No, however, I am not interested in building the examples of AMGX/examples as a final goal. Building the examples of AMGX/examples is intended as a seperate step to *understand* the compile and build process of AMGX/examples.

There are now two ways to proceed

1/ First alternative

Repeat the entire build process, and replace in the last step "make -j16 all" by "make -j16 all >& log.makeAMGX" in such a way that the output of the make process is written to the log-file log.makeAMGX that we can inspect. A search on amgx_capi.c should provide the information we are after.

2/ Second alternative

Assume that all the header files are included in AMGX/include and its subdirectory and that all the library files (files with extension *.a or *.so) are
in some lib directory.

We can proceed with a copy-and-paste of the source of amgx_capi.c into myLaplacianFoam and try to compile, link and run it.

Let me know how it goes.

[Nitish25]

Originally Posted by dlahaye

Yes and no.

Yes, I understand that the manually building the examples of AMGX/examples is *not* required.

No, however, I am not interested in building the examples of AMGX/examples as a final goal. Building the examples of AMGX/examples is intended as a seperate step to *understand* the compile and build process of AMGX/examples.

There are now two ways to proceed

1/ First alternative

Repeat the entire build process, and replace in the last step "make -j16 all" by "make -j16 all >& log.makeAMGX" in such a way that the output of the make process is written to the log-file log.makeAMGX that we can inspect. A search on amgx_capi.c should provide the information we are after.

2/ Second alternative

Assume that all the header files are included in AMGX/include and its subdirectory and that all the library files (files with extension *.a or *.so) are
in some lib directory.

We can proceed with a copy-and-paste of the source of amgx_capi.c into myLaplacianFoam and try to compile, link and run it.

Let me know how it goes.

hello sir
I have repeated the entire built process of AMGX and save in log file .
I hv attached the log file.
I am able to find build of amgx_capi.c at
[ 95%] Building C object examples/CMakeFiles/amgx_capi.dir/amgx_capi.c.o
in log file.

kindly reply next step.

[dlahaye]

We are making a small step in the good direction.

The log file, however, fails to give sufficient information. We need more information.

1/ Compile stage

Here is what we now know

Line 242 in log file
[ 95%] Building C object examples/CMakeFiles/amgx_capi.dir/amgx_capi.c.o

To obtain more information, the compiler (i.e. gcc -c) needs to output more information.

2/ Link stage

Here is what we now know

Line 247 in log file:
[ 97%] Linking C executable amgx_capi

To obtain more information, the linker (i.e. gcc -L ) needs to output more information.

3/ Next Step

To obtain the required additional information, can you please type

cd examples/CMakeFiles/amgx_capi.dir

and subsequently type

make

?

Dhanyavaad.

[Nitish25]

Originally Posted by dlahaye

We are making a small step in the good direction.

The log file, however, fails to give sufficient information. We need more information.

1/ Compile stage

Here is what we now know

Line 242 in log file
[ 95%] Building C object examples/CMakeFiles/amgx_capi.dir/amgx_capi.c.o

To obtain more information, the compiler (i.e. gcc -c) needs to output more information.

2/ Link stage

Here is what we now know

Line 247 in log file:
[ 97%] Linking C executable amgx_capi

To obtain more information, the linker (i.e. gcc -L ) needs to output more information.

3/ Next Step

To obtain the required additional information, can you please type

cd examples/CMakeFiles/amgx_capi.dir

and subsequently type

make

?

Dhanyavaad.

sir
we have entered the examples/CMakeFiles/amgx_capi.dir
and have typed make
but it says
make: *** No targets specified and no makefile found. Stop.

kindly reply

[dlahaye]

I am confused.

The file "makefile" (or variant) seems to be missing.

What is the output of

cd <AMGX-main-directory>

find . -name *makefile* -print

?

[Nitish25]

Originally Posted by dlahaye

I am confused.

The file "makefile" (or variant) seems to be missing.

What is the output of

cd <AMGX-main-directory>

find . -name *makefile* -print

?

after type
find . -name *makefile* -print

in main amgx directory,
it print:
./examples/install_makefiles_mpi
./examples/install_makefiles_nompi


when I type
find . -name *Makefile* -print
it prints:
./build/CMakeFiles/Makefile.cmake
./build/CMakeFiles/Makefile2
./build/examples/Makefile
./build/thrust/Makefile
./build/Makefile
./build/src/Makefile
./examples/Makefile.cray
./examples/amgx_spmv_example/Makefile
./examples/install_makefiles_mpi/Makefile
./examples/install_makefiles_nompi/Makefile
./thrust/Makefile

[Nitish25]

Originally Posted by Nitish25

after type
find . -name *makefile* -print

in main amgx directory,
it print:
./examples/install_makefiles_mpi
./examples/install_makefiles_nompi


when I type
find . -name *Makefile* -print
it prints:
./build/CMakeFiles/Makefile.cmake
./build/CMakeFiles/Makefile2
./build/examples/Makefile
./build/thrust/Makefile
./build/Makefile
./build/src/Makefile
./examples/Makefile.cray
./examples/amgx_spmv_example/Makefile
./examples/install_makefiles_mpi/Makefile
./examples/install_makefiles_nompi/Makefile
./thrust/Makefile

plz find the attached image

[dlahaye]

1/ What is the output of the following:

cd examples

ls

make -f Makefile.cray amgx_capi

2/ I send you a link to join a slack group

Talk soon. D.

[klausb]

Dear Domenico,

regarding your questions:

1/ I imagine that FOAM2CSR applies to the OpenFoam LDU matrix format. FOAM2CSR does *not* apply to the PETSc matrix format. Can you confirm?

See timestamp 2:46 in the youtube video. It explains the role of FOAM2CSR, it handles "the GPU co-ordination, matrix consolidation, data transfers, transformations and dispatching linear systems to AmgX." I haven't looked into the source code of FOAM2CSR to see what's its relationship with PETSc4FOAM. In the video, they talk about converting the LDU format to the CSR format used by AmgX, I assume because it's computationally cheaper to do the conversion.


2/ Does AMGX apply to an decomposed case, or does AMGX enforce its own parallel decomposition?

That's a misunderstanding I'd say. AmgX is the solver library, FOAM2CSR handles the "consolidation" of decomposed cases according to the number of available GPUs if the number of CPU cores and available GPUs differ.


3/ When using AMGX to an incompressible case (like motorBike), AMGX is to be applied to the pressure field only?

No, it's not limited to the pressure field.


Klaus

[Dcn]

Originally Posted by klausb

I found out, that the AmgX-OpenFOAM integration has been done in another way using a new interface called FOAM2CSR instead of the wrapper.

So it's:

OpenFOAM > PETSc4FOAM > FOAM2CSR > AmgX

See youtube: SU2 Conference 21: An Implementation of AmgX for GPU Acceleration of OpenFOAM

https://www.youtube.com/watch?v=1d1DVURNz4E


FOAM2CSR

https://gitlab.hpc.cineca.it/openfoam/foam2csr

Sir i have compile and build the foam2CSR and have got libfoam2csr.so and have also build PETSc4FOAM and have got libpetscfoam.so , similarly i have successfully build the Amgx library , now can you kindly tell me how to use FOAM2CSR with Amgx
Thank you

[dlahaye]

Dear Klaus,

Thank you for your reply. Much appreciated!

Re-1: Thank you for pointing out the link to the video.

Re-2: I fail to understand what the purpose of FOAM2CSR is. Once in PETSc, the matrix is in a format that PETSc understands. Clearly, I am missing something.

Re-3: I agreed that GAMG can solve velocity components. I wonder, however, whether the use of GAMG for the velocity components offers any gains.

Thanks again. Talk soon, Domenico.

[dlahaye]

Dear Dcn,

I fail to understand what the purpose of FOAM2CSR is. Clearly and most obviously, I am missing the point.

I therefore developed myLaplacianFoam at placed it at https://github.com/ziolai/software/t...yLaplacianFoam. This code convert the LDU matrix format to CSR format using code available from https://gist.github.com/BinWang0213/...b8e25ac014cc48.

In a next step I wish to extend myLaplacianFoam with something like

// Call Initialize
AMGX_initialize();

AMGX_resources_create(&rsrc, cfg, &amgx_mpi_comm, 1, &lrank);
AMGX_matrix_create(&A, rsrc, mode);
AMGX_vector_create(&x, rsrc, mode);
AMGX_vector_create(&b, rsrc, mode);
AMGX_solver_create(&solver, rsrc, mode, cfg);

// Call AMGX set-up stage
AMGX_solver_setup(solver, A);

// Call AMGX solve stage
AMGX_solver_solve(solver, b, x);

// Call Finalize
AMGX_finalize();

Does this make sense?

[Walid Badawy]

Hi Nitish, did you managed to do it with Klaus workflow!