New OpenFOAM Server (Dual Epyc Rome)

fanta · February 2, 2020, 12:09

Hi,
i want to buy a new Server for solving with OpenFOAM. i have a 9 year old cluster of 2 Dual Hex Core Intel CPU, that makes 48 threads. This cluster becomes to slow when we reach more than 30 000 000 cells. It runs all well but 50 000 000 cells take 24h hours, that is too long. Postprocessing is done with ParaView, so i need a good graphicscard. I configured a Dual AMD Epyc Rome configuration, because in benchmarks they are better than Intel and they are cheaper. Please post your suggestions!

Chassis
1 x Fractal Design Define XL R2 - 2x
GPU + 832,86 €
CPU
2 x AMD EPYC 7452 + 4522,44 €
32 Cores pro CPU, 2,35 GHz
RAM
16 x Micron MTA36ASF4G72PZ-3G2 + 2541,76 €
= 512 GB, 16 x 32 GB
GPU
1 x AMD Radeon RX VEGA 56 + 507,02 €
M.2 NVMe
1 x Samsung PM983 MZ1LB960HAJQ + 268,24 €
960 GB, SSD, 1,3 DWPD

I am located in Germany, that configuration costs ~9500 Euro. If anyone can recommend a good company to buy that Server ...
OS: I would like to use Ubuntu 18.04 again, is there anything wrong with that? Regards, fanta

flotus1 · February 2, 2020, 12:40

Seems like you already found the right retailer. Unless you feel confident to assemble it yourself, there is not much room for improvement. So these are mostly tips on how you could save a few € with this retailer.

You could choose their single-GPU platform, which saves a few euros.
And for ~50M cells, 256GB of memory would be enough.
64 cores are cool and all, and I myself would probably not settle for less these days. But you would not loose too much performance going with 48 cores total.
A Vega 56 graphics card is definitely not worth 500€ plus taxes. I usually work with models that are in the 100M cell range, and my RX 570 8GB has never disappointed me.

I see nothing wrong with Ubuntu as an operating system for this workstation.

flotus1 · March 29, 2020, 17:26

Renting hardware can make sense if you don't use it a lot, or to handle brief spikes in demand.
Guess what happens when you rent compute capacity for longer periods of time. After a rather short period, you paid off the hardware for your provider, and then you continue to pay up.

Feel free to post what exactly you would recommend instead. Maybe there are some incredible deals I wasn't aware of?

fanta · April 1, 2020, 15:24

We run OpenFOAM studies 24/7 for years now and we still use this 11 year old cluster for that. If i rent a server, is the server located in my server rack in the company? I mesh at my desktop, i don't wanna up und download those big files to a server not located in our company plus flotus said the important points.

February 2, 2020, 12:09	New OpenFOAM Server (Dual Epyc Rome)	#1
fanta Member Join Date: Oct 2011 Posts: 53 Rep Power: 15	Hi, i want to buy a new Server for solving with OpenFOAM. i have a 9 year old cluster of 2 Dual Hex Core Intel CPU, that makes 48 threads. This cluster becomes to slow when we reach more than 30 000 000 cells. It runs all well but 50 000 000 cells take 24h hours, that is too long. Postprocessing is done with ParaView, so i need a good graphicscard. I configured a Dual AMD Epyc Rome configuration, because in benchmarks they are better than Intel and they are cheaper. Please post your suggestions! Chassis 1 x Fractal Design Define XL R2 - 2x GPU + 832,86 € CPU 2 x AMD EPYC 7452 + 4522,44 € 32 Cores pro CPU, 2,35 GHz RAM 16 x Micron MTA36ASF4G72PZ-3G2 + 2541,76 € = 512 GB, 16 x 32 GB GPU 1 x AMD Radeon RX VEGA 56 + 507,02 € M.2 NVMe 1 x Samsung PM983 MZ1LB960HAJQ + 268,24 € 960 GB, SSD, 1,3 DWPD I am located in Germany, that configuration costs ~9500 Euro. If anyone can recommend a good company to buy that Server ... OS: I would like to use Ubuntu 18.04 again, is there anything wrong with that? Regards, fanta aRazs likes this.

February 2, 2020, 12:40		#2
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,427 Rep Power: 49	Seems like you already found the right retailer. Unless you feel confident to assemble it yourself, there is not much room for improvement. So these are mostly tips on how you could save a few € with this retailer. You could choose their single-GPU platform, which saves a few euros. And for ~50M cells, 256GB of memory would be enough. 64 cores are cool and all, and I myself would probably not settle for less these days. But you would not loose too much performance going with 48 cores total. A Vega 56 graphics card is definitely not worth 500€ plus taxes. I usually work with models that are in the 100M cell range, and my RX 570 8GB has never disappointed me. I see nothing wrong with Ubuntu as an operating system for this workstation. fanta likes this.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[OpenFOAM] Paraview on server for OpenFOAM post-processing	KasperWayne	ParaView	1	November 22, 2019 08:46
OpenFOAM Training Jan-Apr 2017, Virtual, London, Houston, Berlin	cfd.direct	OpenFOAM Announcements from Other Sources	0	September 21, 2016 12:50
OpenFOAM Training, London, Chicago, Munich, Houston 2016-2017	cfd.direct	OpenFOAM Announcements from Other Sources	0	September 14, 2016 04:19
Superlinear speedup in OpenFOAM 13	msrinath80	OpenFOAM Running, Solving & CFD	18	March 3, 2015 06:36
Cross-compiling OpenFOAM 1.7.0 on Linux for Windows 32 and 64bits with Mingw-w64	wyldckat	OpenFOAM Announcements from Other Sources	3	September 8, 2010 07:25

March 29, 2020, 17:26		#3
flotus1 Super Moderator Alex Join Date: Jun 2012 Location: Germany Posts: 3,427 Rep Power: 49	Renting hardware can make sense if you don't use it a lot, or to handle brief spikes in demand. Guess what happens when you rent compute capacity for longer periods of time. After a rather short period, you paid off the hardware for your provider, and then you continue to pay up. Feel free to post what exactly you would recommend instead. Maybe there are some incredible deals I wasn't aware of?

April 1, 2020, 15:24		#4
fanta Member Join Date: Oct 2011 Posts: 53 Rep Power: 15	We run OpenFOAM studies 24/7 for years now and we still use this 11 year old cluster for that. If i rent a server, is the server located in my server rack in the company? I mesh at my desktop, i don't wanna up und download those big files to a server not located in our company plus flotus said the important points.