[surajsalvip@gmail.com]

Has any one configured AmgX library with OpenFOAM or any one knows how to configure a third party library with OpenFOAM please help

[Nitish25]

Originally Posted by surajsalvip@gmail.com

Has any one configured AmgX library with OpenFOAM or any one knows how to configure a third party library with OpenFOAM please help

hey can u kindly help me to integrate AmgX library with OpenFOAM.

[dlahaye]

What kind of help do you require?

Did you manage to install AMGX on your system? Can you run AMFX on some tutorial example?

[Nitish25]

Originally Posted by dlahaye

What kind of help do you require?

Did you manage to install AMGX on your system? Can you run AMFX on some tutorial example?

yes i have installed amgx and able to run example of amgx on GPU.
I want to Know how to integrate amgx to OpenFOAM to run case(e.g Motorbike ) on GPU.

[dlahaye]

You need to extract the matrix and rhs-vector from OpenFoam and plug it into AMGX.

Have you already looked into the OpenFoam matrix format, see e.g. https://www.linkedin.com/pulse/openf...menico-lahaye/?

What matrix format does AMGX require?

EDIT: AMGX seems to require the compressed row storage (CRS) format for the sparse matrix. It thus amounts to converting the LDU matrix format to the CRS format. This conversion has been implemented before. It is present in e.g. the PARALUTION software, see http://github.com/openparalution/Paralution-Open-Source

Alternative is to write the linear system to file using the ofgdb debugger and to take it from here.

[Nitish25]

Originally Posted by dlahaye

You need to extract the matrix and rhs-vector from OpenFoam and plug it into AMGX.

Have you already looked into the OpenFoam matrix format, see e.g. https://www.linkedin.com/pulse/openf...menico-lahaye/?

What matrix format does AMGX require?

can u help me with steps, how to extract the matrix and rhs-vector from OpenFoam and plug it into AMGX,
basically I want to calculate performance on gpu of hpc motorbike case.
for that we know that we hv to use gpu solver to solve on gpu.
so I hv installed openfoam-1912 and able to run on cpu.
and for running on gpu, I hv installed and build amgx library in same location where openfoam present parallelly(in amgx directory there is examples directory in which there is some cuda program that i am able to execute on gpu).
also cuda/11.2
kindly reply

[dlahaye]

Greetings,

The motorbike tutorial is a tutorial for the simpleFoam solver. I humbly suggest to start from the flange tutorial for the laplacianFoam solver.

I suggest to copy the laplacianFoam solver from src/application to working dir, rename into myLaplacianFoam, compile and run it.

In a next step, I suggest to copy the AMGX tutorial into myLaplacianFoam, compile, link with AMGX and run it.

In a next-next, we can discuss extracting matrix and right-hand vector from the fvScalarMatrix that myLaplacianFoam generates, and plug it into AMGX.

I happy to look into simpleFoam later.

Does this make sense?

[Nitish25]

Originally Posted by dlahaye

Greetings,

The motorbike tutorial is a tutorial for the simpleFoam solver. I humbly suggest to start from the flange tutorial for the laplacianFoam solver.

I suggest to copy the laplacianFoam solver from src/application to working dir, rename into myLaplacianFoam, compile and run it.

In a next step, I suggest to copy the AMGX tutorial into myLaplacianFoam, compile, link with AMGX and run it.

In a next-next, we can discuss extracting matrix and right-hand vector from the fvScalarMatrix that myLaplacianFoam generates, and plug it into AMGX.

I happy to look into simpleFoam later.

Does this make sense?

hello, I hv completed first three statements that u hv suggested, can u kindly tell me the next step(In a next-next, we can discuss extracting matrix and right-hand vector from the fvScalarMatrix that myLaplacianFoam generates, and plug it into AMGX. ), meaning and how can I proceed to run this case on GPU.
kindly reply.

[dlahaye]

Congrats for the progress!

I need to be brief now.

You can, however, copy what the paralution package does.

Download paralution from https://github.com/openparalution/Pa...n-1.1.0.tar.gz, and navigate to src/plug-inns/OPENFOAM to see how paralution resolves the issue that you raise.

Please give it a look. I am happy to provide more elaborate input where required.

[Nitish25]

Originally Posted by dlahaye

Congrats for the progress!

I need to be brief now.

You can, however, copy what the paralution package does.

Download paralution from https://github.com/openparalution/Pa...n-1.1.0.tar.gz, and navigate to src/plug-inns/OPENFOAM to see how paralution resolves the issue that you raise.

Please give it a look. I am happy to provide more elaborate input where required.

hey, i hv download paralution:u can see the attachment.
but I am confused how to integrate Openfoam into Amgx . so that i can run our case on gpu.
kindly reply

[dlahaye]

Well done on the download.

I do not understand your question.

My understanding is that we want to proceed as follows:

1/ extract linear system from laplacianFoam;

2/ plug linear system into AmgX;

3/ solve linear system using AmgX;

Do we have an agreement on these steps? If not, could you please explain your doubts more elaborately.

[Nitish25]

Originally Posted by dlahaye

Well done on the download.

I do not understand your question.

My understanding is that we want to proceed as follows:

1/ extract linear system from laplacianFoam;

2/ plug linear system into AmgX;

3/ solve linear system using AmgX;

Do we have an agreement on these steps? If not, could you please explain your doubts more elaborately.

hello
see what i hv done till now is :
I hv install and build amgx library(we know that Amgx is GPU-accelerated linear solver package.)
and also install openFOAM-v2112(a CFD tool).
i want, when i run case on openFoam, it should not run on CPU but on GPU.

from ur suggestion what i hv done till now is:
I hv copied the laplacianFoam solver from OpenFOAM/OpenFOAM-v2112/applications/solvers
to working dir, rename into myLaplacianFoam, and in Make/option file i hv added the amgx library and compile and run it. Now i able to see new myLaplacianFoam file in bin directory.

and also i hv copied the flange tutorial for the laplacianFoam solver
(OpenFOAM/OpenFOAM-v2112/tutorials/basic/laplacianFoam/flange)
in fvSolution file when i gave solver name to myLaplacianFoam it is running.(but not on GPU).

Kindly reply

for getting better performance I want to run on GPU

[dlahaye]

Thx for update.

Go back to Item #7 in above list and try to copy AMGX tutorial into myLaplacianFoam, compile, link with AMGX library by editing Make/files appropriate and run.

Talk soon.

[Nitish25]

Originally Posted by dlahaye

Thx for update.

Go back to Item #7 in above list and try to copy AMGX tutorial into myLaplacianFoam, compile, link with AMGX library by editing Make/files appropriate and run.

Talk soon.

hlo
I did not get u.
Inside the amgx directory there is no tutorial directory.
can we connect on meet , where I can show my screen.
kindly reply.

[dlahaye]

1/ In Item #4 above you confirm that you ran an example of AMGX. What example is this? Where is the code of this example?

2/ How did you compile this example? Can you show me the output of the compiler?

3/ How do like this link this example with the AMGX library? Can you show me the output of the linker?

4/ Please confirm that you see a subdirectory called Make in the directory called myLaplacianFoam. Please confirm that you see the files called files and options within the Make directory. Please confirm that it is your understand that EXE_INC should be extended with the above output of the compiler and that EXE_LIBS should be extended with the above output of the linker.

5/ Please let me know in what time zone you live in such a way to set-up an appropriate online meeting.

[Nitish25]

Originally Posted by dlahaye

1/ In Item #4 above you confirm that you ran an example of AMGX. What example is this? Where is the code of this example?

2/ How did you compile this example? Can you show me the output of the compiler?

3/ How do like this link this example with the AMGX library? Can you show me the output of the linker?

4/ Please confirm that you see a subdirectory called Make in the directory called myLaplacianFoam. Please confirm that you see the files called files and options within the Make directory. Please confirm that it is your understand that EXE_INC should be extended with the above output of the compiler and that EXE_LIBS should be extended with the above output of the linker.

5/ Please let me know in what time zone you live in such a way to set-up an appropriate online meeting.

Sir Can u meet now (I am from india (GMT +5:30)) if you free now i will send the meeting link
(Google meet)
Thanks & Regards

[dlahaye]

Let us talk very briefly now and find a slot to talk more extensively next week.

Please send me link for Google meet either here or on private cfdonline address.

Dhanyavaad.

[Nitish25]

Originally Posted by dlahaye

Let us talk very briefly now and find a slot to talk more extensively next week.

Please send me link for Google meet either here or on private cfdonline address.

Dhanyavaad.

Sir please join this link(GMT 9:45 )20-01-2023
https://meet.google.com/goe-qnus-mbo

Thanks

[klausb]

Hello,

the external-solver extension which is also called "petsc4Foam" comes with the recent OpenFOAM releases from openfoam.com . With this extension, it's possible to use the Petsc linear algebra solvers as OpenFOAM solvers. You could link this external-solver extension "petsc4Foam" to AmgX using the AmgXWrapper:

OpenFOAM > external-solver extension "petsc4Foam" (linking OpenFOAM with Petsc) > AmgXWrapper (linking Petsc with AmgX) > AmgX

https://github.com/barbagroup/AmgXWrapper

AmgXWrapper library details from the Author:

https://on-demand.gputechconf.com/gt...-cfd-codes.pdf

You can find the external-solver under: OpenFOAM-v2206/modules/external-solver




There are also three other ways to run Petsc solvers on GPUs, see: https://petsc.org/release/overview/gpu_roadmap/




Klaus

[Nitish25]

Originally Posted by klausb

Hello,

the external-solver extension which is also called "petsc4Foam" comes with the recent OpenFOAM releases from openfoam.com . With this extension, it's possible to use the Petsc linear algebra solvers as OpenFOAM solvers. You could link this external-solver extension "petsc4Foam" to AmgX using the AmgXWrapper:

OpenFOAM > external-solver extension "petsc4Foam" (linking OpenFOAM with Petsc) > AmgXWrapper (linking Petsc with AmgX) > AmgX

https://github.com/barbagroup/AmgXWrapper

AmgXWrapper library details from the Author:

https://on-demand.gputechconf.com/gt...-cfd-codes.pdf

You can find the external-solver under: OpenFOAM-v2206/modules/external-solver




There are also three other ways to run Petsc solvers on GPUs, see: https://petsc.org/release/overview/gpu_roadmap/




Klaus

hello Klaus
can you kindly tell how to link the AmgxWrapper with the petsc4Foam , as
i have link petsc4Foam with openfoam and have also successfully build
amgx and amgxwrapper ,