[flotus1]

Point taken about the problem size being too small. And about possible benefits with the licenisng system of Ansys.
So what you are saying is that with larger problem sizes SIMPLE solver also benefits from GPU acceleration? Otherwise I will stick to the conclusions I made so far.

And there is kind of a discrepancy between the need for larger problem sizes and GPU memory being too small. With lets say 4 of the latest and greatest GPU accelerators all you get is 4x16GB of VRAM. Hard to fit models with tens or hundreds of millions of cells into that with the 4GB/million cells you mentioned. Or does the benefit of having larger problem sizes still hold when the case is run on a cluster? How?

2. For Ansys Fluent, the entire model has to fit in the GPU memory. Approx. 1M cells needs about 4 GB of GPU memory.

That point was being made in the first post

3. Ansys Mechanical has published a set of GPUs that are recommended or certified. Someone has already posted a screen shot in this thread.

We figured that out. What is annoying is that this is not a recommended list, but instead a whitelist. And to make matters worse some of the Quadro cards on it are not exactly good compute cards due to their lack of DP performance.

4. Gaming/consumer-grade cards are not benchmarked for the professional apps. Only Quadro & Tesla cards are.

Yes, and because of this and misconceptions about this topic I decided to throw in a GTX card.

5. We don't make any claims when features aren't supported or don't work. We always recommend everyone to run their own tests as a variety of factors play a role in performance (type of CPU core, host memory, I/O, GPU, etc.). Also, not everyone has enough licenses to run on all cores. Each GPU is equivalent to a single CPU core.

Running my own tests is what I did. I explicitly made it clear that Ansys (or Nvidia) never claimed to have working GPU acceleration with SIMPLE solver in the past.
And again, point taken with possible benefits for Ansys licensing.

Sorry for the lengthy statement, but I was a little stumped by your entrance with "All, I'd recommend reviewing the following two URLs to better understand..." which basically implies that there are some wild misunderstandings here.

[juliom]

When you increase the CPU count, do you mean MPI or OpenMP ?
Nice work !

[flotus1]

Fluent uses MPI for CPU parallelization.

[hpvd]

just some new benchmarks for fluent and mechanical with gpu acceleration

of course specially picked by nvidia but nonetheless possibly interesting because latest soft and hardware is used:
- Ansys 19.0
- Tesla/Quadro V100 with 32GB


https://www.nvidia.com/object/tesla-...lerations.html

[LuckyTran]

Originally Posted by hpvd

of course specially picked by nvidia but nonetheless possibly interesting because latest soft and hardware is used:
- Ansys 19.0
- Tesla/Quadro V100 with 32GB

Notice all the benchmarks are performed on single workstations where the problem still neatly fits on one workstation. If you are in this space where you can still do CFD on a workstation and don't yet need a cluster, it's worth a look. But if you're already in the 1000-core category, look away.

Since it's now 2018, I update my outlook for GPU-accelerated CFD. GPU-acclerated computing for non-CFD applications is already superb.

Unfortunately, the market for GPU-computing is not tailored for HPC (even though it is marketed as such, probably because the bigger market is non-CFD computing) but for enthusiasts with workstations (i.e. small businesses). But this is a workable space and gpu's deserve some thought if you are in the 100-core category.

I really have to give Nvidia & Ansys +10 points because the gpu support works pretty much out-of-the-box. This really helps again those users & admins that only have a single PC or workstation (already with a cpu in it of course) and are trying to squeeze out a little more without dumping a ton of cash on a network of workstations or a small cluster. But again this means GPU isn't really for HPC but trying to give the poor man some equity.

Due to digital currency mining, demand for graphics cards have gone thru the roof and there is a shortage in supply that Nvidia for sure cannot meet in the near future. $/card will not improve.

The thermal efficiency of graphics cards are more competitive than I thought previously. In a cluster environment, it's much easier to cool a bunch of graphics cards than a bunch of cpu's due to all the auxiliaries that come with the cpu. If you're upgrading your cluster and don't want to overhaul your A/C unit, gpu's might be the right option from a hardware perspective. It's too bad you can't GPU-cluster.

The V100 series really opened the door again to yet another potential not-executed. It supports matrix/tensor FMA (Fused-Multiply-Add) operations, but only in mixed precision and not full double precision support. I want to say maybe this will be fixed in the next generation but I said the same thing about the Kepler and the FMA in Pascal is also still not in full dp. The unified architecture of Pascal (where CPU and GPU have direct access to RAM) is still not realized. This feature is seriously needed since GPU memory is so limited. There really is an opportunity for a game-changer in computing, but it looks like lots of work need to be done. And GPU OEM's need to be able to ramp up production by about 10x. And since the stock market is all the craze right now, I would give a buy rating to Nvidia for investment for their future portfolio but not because of current capabilities GPU-CFD

[flotus1]

There sure is some potential in GPU acceleration, especially when taking into account how GPUs are treated by the Ansys licensing policy.
When I wrote this article I had one thing in mind in particular: people asking on the forum (or in person) how they can leverage the potential of the GPU they have in their PC or workstation. Or if they should buy a sub-1000$ GPU to get more performance. This is usually a bad idea.
Others willing to shell out 7000$+ for a GPU alone or looking to upgrade their cluster probably don't need my advice.
By the way, I don't think that the GPU shortage caused by crypto-currencies has had a significant impact on this price range. Miners buy GTX 1080TI or below, and the market for GPUs above this price point is less cost-sensitive anyway. Doesn't matter if a Tesla V100 costs 7000$ or 8000$.

[naserian]

Dear Alex
Could you please explain how you activate gtx 1060 for fluent? I have bought a gtx 1060 graphic card for gpu calculations in fluent but I cannot use it. It seems that Ansys does not support GeForce graphic cards for gpu acceleration.
In the Fluent Launcher widow I set GPGPUs per machine=1 and no. Of processors = 2. But the gpu is not used and calculations are done only by cpu. Would you please guide me?

[flotus1]

Which solver type are you using?
Which version of Fluent?
Any error messages in TUI after you start fluent?
What is the output of solve -> parallel -> gpgpu?
Which operating system are you using?
Are GPU drivers installed properly?

[naserian]

Thank you very much for your reply.
- Solver: pressure-based, Scheme: simple (I examine the example in Chapter 3 (fluid flow and heat transfer in a mixing elbow) of ANSYS Fluent Tutorial Guide and also another test with PISO scheme)
- fluent version: 19
- No error is appeared in TUI.
- The output of /parallel/gpgpu/show" is
CUDA visible GPUs on myhostname
CUDA runtime version 6000
Driver version 9010
Number of GPUs 1
0. GeForce GTX 1060 3G
- Operating system: windows 10
- I installel the lastest version for GTX 1060 on windows 10. I think the driver has been installed properly.

Best wishes

[flotus1]

With SIMPLE solver, GPU is deactivated by default because solver times are higher with GPU than without GPU.
You could force Fluent to use the GPU, I would have to look up the commands again - you can do it aswell
But I strongly suggest not using it for SIMPLE. Switch to coupled in order to check if your GPU is working properly.

[naserian]

Dear Alex
I checked your suggestion. It seems that with the coupled scheme the GPU is used. But unfortunately with PISO scheme it is not used

[LuckyTran]

Originally Posted by naserian

Dear Alex
I checked your suggestion. It seems that with the coupled scheme the GPU is used. But unfortunately with PISO scheme it is not used

PISO is a cousin of SIMPLE (PISO does one extra pressure correction step). GPU-acceleration is not as effective for these algorithms so they are disabled.

[naserian]

Dear Alex,
Many thanks for your guidance. It sounds that there are a lot of restrictions with GPU-acceleration in fluent!
Is there any reference that introduce the limitations of Fluent 19 for working with GPU?

[flotus1]

I would rather suggest an inverse approach.
Take the information here highlighting the capabilities of GPU acceleration for various models: https://www.nvidia.com/en-us/data-ce.../ansys-fluent/
The assumption that most solvers and models not covered there do not benefit from GPU acceleration (yet) seems not too far-fetched.

[hpvd]

what should be the most interesting data of a GPU hardware when choosing one for usage within fluent (ok you need the right solver...)
if all have the same amount of memory (e.g. 16GB)?


Is it:

- compute (SP/DP),
- memory bandwidth (like for cfd@cpu) or
- connection to CPU (PCIe 3.0, PCIe 4.0 soon)

?

[hpvd]

sorry for posting in wrong language. Now it's translated to english..

[KEDELLE]

Originally Posted by atomdie

Thank you for very nice review. I use Ansys Mechanical, and yesterday I jusy bought a Gtx titan to test GPU acceleration. However, Ansys only supports Tesla and Quadro K6000/k5000 and program cannot run when I request to use GPU acceleration feature. How you could force ansys run with Geforce card? With DP of titan about 1.2Tflops and 6GB Ram, and price of 230euro, it should be considered

Did u use Titan V or Titan X?

[Echidna]

I am about to swap my FirePro W7100 with a Quadro K6000.

I am using Ansys Mechanical.

FirePro is not supported by Ansys HPC, so the Quadro will help a bit with my cases. I am not debating Nvidia vs AMD here, it's obvious that the K6000 is much better than the W7100 as a GPU alone.

What i would like to know is if the K6000 will be efficient with structural calculations in Ansys. For sure, 12GB GPU memory is not a big deal and most of the times my models are pretty large and requiring more than 60-70GBs of RAM. Do you think the K6000 will do anything for those large models in Ansys HPC?

[Duke711]

Originally Posted by Echidna

I am about to swap my FirePro W7100 with a Quadro K6000.

I am using Ansys Mechanical.

FirePro is not supported by Ansys HPC, so the Quadro will help a bit with my cases. I am not debating Nvidia vs AMD here, it's obvious that the K6000 is much better than the W7100 as a GPU alone.

What i would like to know is if the K6000 will be efficient with structural calculations in Ansys. For sure, 12GB GPU memory is not a big deal and most of the times my models are pretty large and requiring more than 60-70GBs of RAM. Do you think the K6000 will do anything for those large models in Ansys HPC?

Only 6 - 8 Cores per GPU, no more. So you need two K6000.


Epyc 7551 vs 6850K; Ansys Mechanical Bench


GTX 780 = K5200 / Tesla K40 with only 6 GB:
https://www.computerbase.de/forum/sh....php?t=1680741

[Echidna]

I plan to buy one K6000 now and at a later time to invest in a Tesla K80.
Can they work together though?
So, with only one K6000, it is a loss of time?