[Kerouac]

Hi everyone,
I'm a PhD student, and during my researches I will perform 3D CFD simulations on some 'large' models (10-20 millions of cells, I think). In particular I will model polymer extrusion through FLUENT software.
Now, my department is giving me a fixed research budget in order to search for a workstation that will fulfill these requirements. The total amount for chosing the workstation should be 1500[€].
I was able to perform calculations on my laptop (i5-6200U, 2.30[GHz]; 12 [Gb] (4+8) of DDR3 RAM) for 1 [mln] of cells, but it takes too long to complete (1 week, on average), and the models that we will impelement i the future will require much more cells.
I can't buy the different components from amazon, ebay and then assembly them because of a series of restrictions on how the research founds can be used.
So, I would ask you if there is a workstation which is already assembled which you would suggest me to use for CFD simulations, or alternatively if you can indicate me the setting which you would suggest for CFD simulations with the constaint of the indicated budget.
My Tutor said that I should indicate:
1.) the name for a given workstation, already assembled (it's preferable, as a solution)
2.) alternatively the features of the different components (i.e. the total amount (and type) of RAM, cores, and so on)
I will appreciate your help in taking these technical choises, since I've no idea where to start.

-Kerouac

[flotus1]

Been there, done that. So please don't take my advice the wrong way.
1500€ is not an awful lot of money for what you need. Especially considering that you apparently have to buy pre-built from one of the larger OEMs. You will get a slightly above-average office PC from Dell, HP etc. for 1500€. But not a proper workstation for CFD simulations.

So, since you are in an academic setup, the better angle of approach would be to get external computing resources. Universities usually can get access to compute clusters for little to no charge.

But to answer your second question anyway: what I would realistically recommend for your requirements is least 64GB of RAM, and a dual-CPU setup with CPUs not older than Intel Xeon E5-26xx v3. With a total core count of at least 16. This can be done for less than 1500€, but not if you have to buy pre-built from a large OEM. Of course, the core count is tied to the software licenses you have available. If your license e.g. only allows you to run a single simulation with up to 8 threads, more than 8 cores are not required. So if you decide to buy your own hardware anyway, check your licenses first.

[danbence]

if Ebay is out, there are companies that specialize in refurbished Dell and HP workstations of the vintage that flotus1 recommends.



If the university rules stretch to this sort of company, then you can get a reasonably decent machine for 1500.

[Kerouac]

Dear Flotus1 and danbence, I appreciate your reply to my question.
Now, in these days I thought to make a different choice: instead of using a budget with all the limitations related to where we get the hardware, I am thinking avout buying a PC from my own, for 1500€, also in order to use it for additional post-doctoral researches in CFD, in the future.
Now, I configured through Ankermann a PC with the following specifications:

-intel core i9-10900K 10x3.7 GHz
-64 Gb of Corsair DDR4 RAM (2666 MHz)
-ASUS prime H410M-A
-GT710 2Gb of GPU
-Corsair Hydro series H75-2018 as CPU cooler
-700W be quiet Pure Power 11 as PSU
-1 Tb of SSD

what do you think about dealing with laminar 3D problems, SIMPLE algorithm of the melting of fluids in FLUENT with an averange of 10-20 millions of cells (probably more) with this hardware? I am using FLUENT with Ansys Enterprise license.
Do you think that this is a proper choice or do you have some advices for me?
Another question is: I know that Enterprise license is up to 8 cores... but it means 8 physical cores or 8 threads? This information is very important, clearly, for choosing the right CPU.
Thank you in advance,
Kerouac

[flotus1]

I know that Enterprise license is up to 8 cores... but it means 8 physical cores or 8 threads?

The license is tied to logical threads, i.e. you can't run a simulation with more than 8 threads. Whether you do that on a 4-core CPU with SMT enabled, or on an 8-core CPU without SMT, is up to you. Obviously, running on more physical cores is faster. SMT does not magically double the execution units of a core, it just tries to get higher utilization out of the available hardware.

Do you think that this is a proper choice or do you have some advices for me?

The problem with these mainstream CPUs is: thanks to their dual-channel memory controllers, they lack the memory bandwidth to keep all those fast cores fed with data. That's why scaling usually stops somewhere between 4-6 cores.
Hence why the recommendation for cheap CFD workstations are usually older dual-socket machines with Xeon CPUs. The individual CPU cores are slower, but thanks to 2x quad-channel memory controllers, scaling is much better, resulting in much higher overall performance.

If we absolutely have to limit ourselves to current-gen mainstream CPUs, I can only recommend one thing: save some money by getting a CPU with a maximum of 8 cores. SMT is not required.
The money you saved goes into a "Z"-series motherboard, which allows memory overclocking. And get the fastest memory you can afford, populated as 2x32GB. DDR4-2666 is not an option, DDR4-3600 is around the sweet-spot for price/performance.

[danbence]

Buying new is a natural temptation. New often equates to better. However, there are three problems with new computers

• There is a huge cost premium for a new computer compared to second hand. In the good old days of computers (15 years ago), the cost premium of new was offset by the staggering pace of performance improvements. Sadly those days are gone
• The “per core” Fluent performance has stayed the same for years
• Mainstream computers have generally reduced the number of memory channels over time. CFD has unusual memory access requirement. The CPU requires access to pretty much every allocated byte for each timestep. This is different to most other programs, that concentrate on sections of the memory (that can be usefully cached). CFD is not a common mainstream CPU benchmark, so there is not the pressure to keep lots of memory channels.

The very best performance/euro you can get would be a second hand Dell rack server. These things are noisy, so only worth considering if you have permission to place it in a suitable room away from people. Second hand Dell seems consistently cheaper than HP.

PS. Cell counts are useful for judging RAM requirement, but to get a handle on runtimes, you need to pin down the number of timesteps you need to observe what interest you.

[Kerouac]

Hi,
thank you for your replies.
At the moment, with a budget of 1500€ I haven't yet bought the PC; I've still now some other options, which I'm considering. One of the cheaper is to buy a refurbished PC or server.
I don't know if it's allowed to attach links in the forum, so I will indicate simply the performances of the refurbished server (Supermicro SuperChassis CSE-815 4-Bay Server):

-Motherboard - extended ATX
-RAM Size: 128 GB DDR3 / 1600 MHz
-Processors: 2xAMD Opteron 6272 2.10[GHz]
-Dedicated Video Memory 16 MB DDR2 SDRAM
-Power Supply - 560W Watt PSU
-Storage: No HDD

Another solution is the following:

-CPU: Xeon E7-4870 x 4, 2.40GHz - 2.80GHz, 10 Core CPU
-RAM size: 256 GB PC3
-Storage: NO HDD
-Power: 2 x 1100W Redundant Power Supply

these are the thecnical specifications of the seller. What do you think about similar configurations? Could it be an alternative than to buy a new PC (with less RAM & cores)?
My purpose is to make big calculations with Fluent, and only 64 Gb RAM with 8 cores maybe would be too low.
Best regards,
Kerouac

[danbence]

My opinion is that you have gone too old with those machines.


For 1000 euros you could get a Dell t5600 with dual 2690 (16 cores) and 128GB of 1600Mhz DDR3 ram


For 1500 euros you might be able to get a dell t5610 with dual 2690v2 (20 cores) and 128GB of 1866Mhz DDR3 (if you are lucky)

[flotus1]

I agree, that stuff is definitely too old. Especially the Opterons, you won't have any fun with those. And for the two configurations you posted, my initial guess would be that they come in a package that you don't want sitting under your desk due to noise.

Knowing that your license is limited to 8 threads maximum, you rather want something along those lines: https://www.ebay.de/itm/HP-Z840-2x-X...M/203246853155
Yes, posting links is allowed. New users might have to wait until a moderator manually approves their post, if it has a lot of external links.

This machine has two 6-core Haswell CPUs (that's what the "v3" stands for, and you should not buy older CPUs with a budget of 1500€) with relatively high clock speed. It comes with a reasonably sized SSD, and a pretty beefy graphics card. So it is pretty much ready to go. The only remaining question mark is how those 64GB of RAM are populated. 8x8GB is what you want, so ask the seller before you buy.

If you search longer than I did, you will probably find similar offers with a cheaper graphics card (which is fine) and more RAM.
Then again, you could simply upgrade that machine if it only comes with 4x16GB. Another set of 4x16GB DDR4-2133 reg ECC cost somewhere around 150-200€. Or just ask the seller, maybe they can swap in a lower-end GPU (don't go below 4GB of VRAM) and do the memory upgrade for you.

[Kerouac]

Hi,
my tutor said that the 8 processor limitation is related to the single run, and that we can lunch different runs on the same machine; for this reason, if what I've understood is right, we would use also more processors.
In the last hours I found different refurbished workstations, which seems to satisfy both the high demand for cores and RAM.
I would be greatfull if you can say me what do you think of the following configurations (which are completely in my budget):

https://www.ebay.it/itm/HP-Z840-Work...wAAOSwm2pf~t5I

https://www.ebay.it/itm/12-Core-HP-Z...4AAOSwZrRe1fWC

https://www.ebay.it/itm/16Core-Dell-...MAAOSw8Z9a4G1T

For my research I started by the models you suggested in previous threads, in particular DellT5600 and HP Z840, with the purpose to find cheaper solutions.
Best regards,
Kerouac

[flotus1]

The first one is fine, but don't forget that it is specifically advertised as "barebone". In this case it means that it comes without graphics card and storage. You will have to buy those separately and install them yourself. At which point you are already halfway to building a similar system from scratch for even less money, or a better one for the same money. Or even 2 separate systems for maximum performance when running several simulations simultaneously.

The second has ancient CPUs. To avoid pulling up such offers in the future, just plug the CPU model number into your favorite search engine. One of the first results will be Intels official spec sheet, which also lists the release date. 2010 in this case
https://ark.intel.com/content/www/us...intel-qpi.html
You should really limit your search to Xeon E5-26xx v3 and v4
"v1" and "v2" are still valid as extreme budget options, but at 1500€, you really should not bother.

Option 3...well see option 2

[danbence]

Ok, so don’t go older than the Xeon E5 V1 family. Always go for dual CPU.

https://ark.intel.com/content/www/us...e5-family.html

so it’s a no for the Xeon X5650

Ask the sellers the speed of the ram, it will make a difference. For E5 V1 processors pick 1600Mhz, for E5 V2 pick 1866mhz and for E5 V3 pick 2133Mhz.

The best of the E5 V1 processors are 2690 and 2687W. High rated power consumption is a good indicator of high performance within a family

With the E5 V2, don’t go above 10 cores per CPU, so E5-2690v2 is a good candidate.
With the E5 V3, don’t go above 12 cores per CPU, so E5-2690v3 is a good candidate.

[Kerouac]

Ok, I'm trying to follow your information; I found what you've explained me really clear and it will be really usefull for my final choice .
With:

-Dell Precision T7600
-2 x Intel Xeon E5-2690, 8x 2,9 GHz, 20MB Cache
-128 GB DDR3 RAM (I have to ask for frequency, for what you've explained me)
-1x NVIDIA Quadro K620, 2 GB, GDDR5 (1x DP, 1x DVI)
-256 Gb SSD
-3 Tb HDD
-windows 10 pro

I'm still in the budget.
This configuration has been done through workstation4you.
My fear is that one or more component can behave as bottlenecks for my CFD simulations.
I would ask you if you've some suggestions, also in order to improve this configuration (or if it's totally wrong .. ).

[danbence]

If your purchasing rules allowed you to buy components to build a machine, then your budget could have stretched to newer CPUs. For a pre-build machine, what you have specified is respectable. You will notice a large improvement compared to your laptop


It is worth checking that the 128GB ram is coming in 8 sticks (ie 8 x 16GB)