[JM92]

Hi all,

I got my hands on a high-performance nVidia GPU, which will be used for SPH simulations. As I will be able to build a workstation for roughly 3k€ (excluding GPU) around it, I'm currently thinking about setting this up for a second "use case", which would be simulations with openFoam. For this I'd currently aim at one Epyc 7313P or 7343 with (at least) 64 GB of RAM.

Generally, I think the information given in this forum would be sufficient to build a CPU-only workstation without further help, but I just wanted to ask if anyone has experience with this combination of two use cases (SPH on GPU, openFoam on CPU [of course both simulations will not be run at the same time)] and knows about any pitfalls to avoid.

E.g., I'm unsure about including a motherboard with onboard GPU to minimize the "additional" load on the nVidia card while running SPH simulations.

Here's the checklist (focusing on the non-GPU part)

1. Which software do you intend to use? --> openFoam
2. Are you limited by license constraints? I.e. does your software license only allow you to run on N threads? --> No license constraints
3. What type of simulations do you want to run? And what's the maximum cell count? --> 2D and 3D, mainly two-phase flow (interFoam), max. No. of cells 10-20 million (but a bit unsure what the future may hold in that matter)
4. If there is a budget, how high is it? --> 3k €
5. What kind of setting are you in? Hobbyist? Student? Academic research? Engineer? --> Academic researcher
6. Where can you source your new computer? Buying a complete package from a large OEM? Assemble it yourself from parts? Are used parts an option? --> Planning to assemble it myself
7. Which part of the world are you from? It's cool if you don't want to tell, but since prices and availability vary depending on the region, this can sometimes be relevant. Particularly if it's not North America or Europe. --> Europe
8. Anything else that people should know to help you better?

Kind regards

[flotus1]

Onboard graphics is not something you need to worry about here. As far as I know, all motherboards compatible with Epyc 7000 CPUs come with some basic Aspeed graphics processor on board. The connector is only VGA though, and of course the processing power is severely limited. Displaying some console windows works fine, anything beyond that doesn't.
It might be a good idea to add a small Nvidia graphics card for good measure. Prices are still high, but not as obscene as they used to be. 290€ for a GTX 1660 for example.
That will allow you to use all the available memory of your highend GPU for computations, and generally simplify the setup.

Speaking of pitfalls and unnamed "high-performance nVidia GPU"
If it's PCIe 4.0, make sure your motherboard supports that. And beware of cooling for the GPU. Most of the Nvidia datacenter cards are passively cooled, relying on airflow from the chassis fans for cooling. They can still be used in a workstation, but require some thought and modding effort put into a custom cooling solution.
And if it's just a regular cooling solution with axial fans, beware that all of the heat gets dumped into the case, so plan for that with plenty of airflow.
And obviously, the power supply has to match the highend components.

As for the plan itself, you will probably get better answers in the OpenFOAM sub-forum.

[JM92]

Dear flotus1,

thanks for your reply, I continued the "design" of the workstation and would like to ask for one final opinion on the parts before buying them:

• Mainboard: Supermicro H12SSL-NT
• CPU: AMD Epyc 7313P
• GPU: nVidia RTX A6000 (which is actively cooled)
• RAM: 8 x 16 GB Kingston Server Premier RDIMM 16GB, DDR4-3200, reg ECC, dual rank, x8
• SSDs: Crucial P5 Plus 1 TB + Crucial P5 Plus 2 TB
• CPU Fan: Noctua NH-U14S TR4-SP3
• Case: be quiet! Silent Base 601 schwarz, schallgedämmt
• Power supply: be quiet! Straight Power 11 Platinum 850W ATX 2.51

Especially the dimensioning of the power supply gave me some headaches, but as CPU and GPU won't be running on full throttle at the same time, 850 W should be sufficient in my opinion.
Additionally, I'm thinking about following your advice of adding another small GPU to the setup, but no decision was made on that yet. Maybe a small nVidia T400 or T600 would be sufficient.

Kind regards

[flotus1]

Looks fine to me. The RTX A6000 doesn't have regular PCIe power connectors, but requires an 8-pin EPS connector. The PSU you picked has two of those, one for the motherboard, and one for the GPU. And the GPU should come with an adapter 2x8pin PCIe -> 8pin EPS. 850W are enough for these components.
I guess you are aware that the A6000 has rather limited FP64 performance.