[HBetta]

Hi guys.

I'm helping my team to configure a server/cluster to run some multiphase simulations with 50M cells, approximately, to OpenFOAM.

After some research, I've decided that 3x a server with: 2x Intel Gold 5218 + 12x 16GB 2667 MHz would be a good choice. The storage could be in another unit with only SATA HDs (7200RPM) and the parts connected with a 10 GbE switch.

I believe this strategy can make things easier if the team wants to increase cluster performance.

However, Dell Brazil and my company support, are recommending only one server with:

2x Intel Gold 6252
12 x 64GB 2667 Hz
12x 2.4TB HDs 10k RPM SAS
3.84 TB SSD.

As I'm not totally comfortable setting up hardware, I don't know what strategy should I take.

What's your guys opinion??

Thanks!!

[flotus1]

However, Dell Brazil and my company support, are recommending only one server

Did they give any particular reason for replacing 3 servers with 1? Other than the fact that it is obviously cheaper.
In terms of performance, 3 nodes will be much faster when solving your rather large models. But you will need an interconnect. 10Gigabit Ethernet might be ok'ish for only 3 nodes, but Infiniband will become mandatory if you ever want to increase the number of nodes.
And of course: AMD Epyc CPUs would be the better choice in terms of price/performance. There is a sticky thread here with lots benchmarks on various hardware. 1st gen Epyc CPUs are available with huge discounts these days, maybe you can negotiate a price for the whole cluster.

As a side-note:

12x 2.4TB HDs 10k RPM SAS

Unless there is a very specific requirement that only this kind of storage setup can deliver, this is a huge waste of rack space and money. I wonder what that could be...maybe 10 users writing huge files simultaneously, so the box runs out of SSD cache?
Otherwise, I would insist on fewer, larger HDDs with 7200rpm or even 5400rpm. Seems like the SSD will be used for caching, so the speed of the HDD array does not really matter.

[HBetta]

Did they give any particular reason for replacing 3 servers with 1?

Unfortunately, it looks like that kind of interconnect switches are not usual around here, so they are really expensive.

Otherwise, I would insist on fewer, larger HDDs with 7200rpm or even 5400rpm. Seems like the SSD will be used for caching, so the speed of the HDD array does not really matter.

Great to hear that. We don't have that request. I havent noticed that HDs were taking most of budget.

Thank you, Alex.

[flotus1]

I can only find two 10k SAS HDDs with 2.4TB capacity. They cost 380€ and 460€ respectively. That's quite a lot of money for 12 of them. You get decent enterprise-grade 12TB HDDs in that price range.

And yes, new Infiniband gear is quite expensive, not only in Brasil. The same can be said for basically any kind of node interconnect as soon as you go beyond 10 Gigabit Ethernet. It might be worth giving it (Ethernet) a try, and see how your simulations scale across nodes. You can still upgrade to a different interconnect later. But in my experience, multiphase simulations tend to put more pressure on the interconnect, compared to single-phase simulations.