[Habib-CFD]

What is the best CPU for the low-end users of Flow-3D?

Manual of FLOW-3D:
V11.2: An x86-64 compatible CPU is required. Multiple core CPUs, particularly Intel Core i7 and Intel Xeon CPUs, are strongly recommended. AMD CPUs are not recommended.

V12.0: An x86-64 compatible CPU is required. Multiple core CPUs, particularly Intel Core i7, Intel Core i9, and Intel Xeon, are strongly recommended. AMD Ryzen and Epyc CPUs are compatible

I want to share my personal experience on the simulation as the normal academic user:

1-If you don't plan to use the flow sight, the 16G of ram is really sufficient. I had never seen more than 2G ram usage in my simulation. Higher frequency and quad-channel are recommended.

2-If you don't plan to use the flow sight, you don't need to bye an expensive P4000 or etc. As a cheap option, the GTX1660 or even RX570 can easily handle the GUI for the low-end users.

3-The main question about CPU. I would like to release some benchmark from one of the default examples of FLow-3D itself using my different PCs. Note that the lower time of solving is better and the results are almost similar at both 11.2 and 12 versions. FLow 3D maximum cores limited to 32 cores!

Intel 7700K, 16GRAM, Win10-1903, 08 Cores, 4.30s
Intel 6580K, 64GRAM, CentOS 6.10, 12 Cores, 3.13s
AMD 2970WX, 32GRAM, Win10-1903, 12 Cores, 4.10s
AMD 2970WX, 32GRAM, CentOS 7.7, 12 Cores, 3.14s
AMD 2970WX, 32GRAM, CentOS 7.7, 18 Cores 2.51s
AMD 2970WX, 32GRAM, CentOS 7.7, 24 Cores, 2.59s!
AMD 2970WX, 32GRAM, CentOS 7.7, 32 Cores, 3.12s!!
AMD 2970WX, 32GRAM, Opensuse 15.1, 18 Cores, 2.47s

All the results were repeated three times.
I already installed the last AMD RYZEN AGESA ThreadRipperPI-SP3r2 1.1.0.2.
With respect to other software benchmarks in my AMD PC, I believe the regression or other issues do not relate to AMD and comes from OS or maybe the Solver.

https://www.youtube.com/watch?v=M2LO...L&index=6&t=0s

[Habib-CFD]

After checking many benchmarks and reviews, I found that the core performance issue comes from the memory bandwidth bottleneck.

Updated result:
AMD 2970WX, 4*8GB (O.C. 3200), CentOS 7.7, 20 Cores, 2.25s

The others suggestion did not affect my 2970WX for example:
Disabling the SMT (Hyperthreading)!
Disabling the Numa cores that not connected directly to ram channel!!

Not: The ram frequency overclocking (2133 to 3200) decline the total run time. Increasing the number of DIMM (like using 8 inst. 4) in a quad-channel system just has shown a negative effect due to the decreasing frequency of the memory in 8 DIMMs mode.

Conclusion of the best economical CPU based on AMD platform and considering the memory bandwidth:

Dual-channel: 3600x (Run at 12 threads)
Quad-channel: 2950x (Run at 20 threads)
Octa-channel: AMD Rome (30~36 threads).

Reminding that the performance enhancement is not linear!!

[thunde47]

Originally Posted by Habib-CFD

After checking many benchmarks and reviews, I found that the core performance issue comes from the memory bandwidth bottleneck.

Updated result:
AMD 2970WX, 4*8GB (O.C. 3200), CentOS 7.7, 20 Cores, 2.25s

The others suggestion did not affect my 2970WX for example:
Disabling the SMT (Hyperthreading)!
Disabling the Numa cores that not connected directly to ram channel!!

Not: The ram frequency overclocking (2133 to 3200) decline the total run time. Increasing the number of DIMM (like using 8 inst. 4) in a quad-channel system just has shown a negative effect due to the decreasing frequency of the memory in 8 DIMMs mode.

Conclusion of the best economical CPU based on AMD platform and considering the memory bandwidth:

Dual-channel: 3600x (Run at 12 thread)
Quad-channel: 2950x (Run at 20 thread)
Octa-channel: AMD Rome (30~36 thread).

Reminding that the performance enhancement is not linear!!

Thanks for the in-depth analysis!

[M.Nazrin]

Hi Habib,

I have a question that i want you to clarify. How does ur TR 2970wx have 32 core? I believe 32 core for TR 2000 series is TR 2990wx.

Intel 7700K, 16GRAM, Win10-1903, 08 Cores, 4.30s
Intel 6580K, 64GRAM, CentOS 6.10, 12 Cores, 3.13s
AMD 2970WX, 32GRAM, Win10-1903, 12 Cores, 4.10s
AMD 2970WX, 32GRAM, CentOS 7.7, 12 Cores, 3.14s
AMD 2970WX, 32GRAM, CentOS 7.7, 18 Cores 2.51s
AMD 2970WX, 32GRAM, CentOS 7.7, 24 Cores, 2.59s!
AMD 2970WX, 32GRAM, CentOS 7.7, 32 Cores, 3.12s!!
AMD 2970WX, 32GRAM, Opensuse 15.1, 18 Cores, 2.47s

Cheers!

[Habib-CFD]

Originally Posted by M.Nazrin

Hi Habib,

I have a question that i want you to clarify. How does ur TR 2970wx have 32 core? I believe 32 core for TR 2000 series is TR 2990wx.

Cheers!

Hi, you are right. Due to Flow-3D consider the thread as the core, I used the cores. However, as you mentioned, its better to replace the benchmarks with the following list:


Intel 7700K, 2*8GB (2400MHz), Win10-1903, 08 threads, 4.30s
Intel 6580K, 8*8GB (2133MHz), Cent OS 6.10, 12 threads, 3.13s
AMD 2970WX, 4*8GB (2133MHz), Win10-1903, 12 threads, 4.10s
AMD 2970WX, 4*8GB (2133MHz), CentOS 7.7, 12 threads, 3.14s
AMD 2970WX, 4*8GB (2133MHz), CentOS 7.7, 18 threads, 2.51s
AMD 2970WX, 4*8GB (2133MHz), CentOS 7.7, 24 threads, 2.59s
AMD 2970WX, 4*8GB (2133MHz), CentOS 7.7, 32 threads, 3.12s
AMD 2970WX, 4*8GB (2133MHz), CentOS 7.7, 48 threads, 3.14s

Last update

AMD 2970WX, 4*8GB (3200MHz), CentOS 7.7, 20 threads, 2.25s (Optimal)
Intel 9900K, 4*8GB (2666MHz), Win10-1909, 16 threads, 3.38s (Limited by dual channel memory bandwidth)

AMD 2920X result will be released soon (360$ CPU, support quad channel 3200MHz and also 24 threads!). It will be very interesting vs Intel LGA 2066 (9920X or even 10920X).

[M.Nazrin]

HI Habib,

TQVM for the reply. Really appreciate it! May everything ease for you.

Cheers!

[Habib-CFD]

During checking the solver time, I found a delay time which somehow related to prepossessing and I/O. Considering that this delay does not show the real performance of the processor and making a big difference in a short time benchmark, I tried to remove them from results.

Modified by removing the start delay time:

Intel 7700K, 2*8GB (D-2400MHz), Win10 Pro-1903, 08 threads, 4.02s (Balance)
Intel 9900K, 4*8GB (D-2666MHz), Win10 Pro-1909, 16 threads, 3.10s (RAM bandwith bottleneck)
Intel 6580K, 8*8GB (Q-2133MHz), Cent OS 6.10, 12 threads, 3.11s (Balance)
AMD 2970WX, 4*8GB (Q-2133MHz), Win10 Pro-1903, 12 threads, 3.42s (unstable result in windows, numa problem)
AMD 2970WX, 4*8GB (Q-3200MHz), CentOS 7.7, 20 threads, 2.23s (Almost balance)
AMD 2920X, 4*8GB (Q-3200MHz), Win10-Pro Workstation-1836, 16 threads, 2.19s (The big surprise and ideal for CFD!)

[Habib-CFD]

New result based on AMD EPYC zen2

SMT off, Flow 3D V11.2u6 and 12.0
AMD TR 2920X, 4*8GB (Q-3200MHz), CentOS 7.7,best 10 cores, 2.05s
AMD EPYC 7302P, 8*8GB (O-2666MHz), CentOS 7.7, Full best 16 cores, 1.49s

It seems the Flow 3D has some problem with Numa feature at EPYC. the best performance achieved in NSP0+L3 disable that is not recommended by AMD. The default is NSP1 and highest Memory bandwidth is available at NSP4 if the software and OS support Numa feature.

Testing with 3200 ECC RAM was limited due to considerable price for 8 dimms.

[M.Nazrin]

Originally Posted by Habib-CFD

New result based on AMD EPYC zen2

SMT off, Flow 3D V11.2u6 and 12.0
AMD TR 2920X, 4*8GB (Q-3200MHz), CentOS 7.7,best 10 cores, 2.05s
AMD EPYC 7302P, 8*8GB (O-2666MHz), CentOS 7.7, Full best 16 cores, 1.49s

It seems the Flow 3D has some problem with Numa feature at EPYC. the best performance achieved in NSP0+L3 disable that is not recommended by AMD. The default is NSP1 and highest Memory bandwidth is available at NSP4 if the software and OS support Numa feature.

Testing with 3200 ECC RAM was limited due to considerable price for 8 dimms.

If this is true, looks like Flow3D is not optimize for AMD Epyc processor at all. All that cores gonna waste.

[Habib-CFD]

There are some benefits of EPYC 7302P:

Working at very low temperature, 45 degrees at full stress tests. Don't need to expensive cooler system.
Good potential if you used the new release of FLow 3D with the compatibility of CentOS 8 ver that finally upgraded to Linux Kernal 4.18 (supporting the new CPU architecture).

[M.Nazrin]

what do you think about, EPYC + windows 10 pro for workstation? is it better than any of the linux os? i dont have experience using linux os, so to jump there, i need to think 1000 times.

[Habib-CFD]

Originally Posted by M.Nazrin

what do you think about, EPYC + windows 10 pro for workstation? is it better than any of the Linux os? I don't have experience using Linux os, so to jump there, I need to think 1000 times.

I already performed some tests especially using Win.10 Pro workstation. The condition in the Windows platform is awful!! For Windows 10 you can choose from intel Core i9-10900X or 9900X. The AMD 2920X still shows good performance considering the price difference.

[M.Nazrin]

yes notice i notice that. i believe, comparing epyc and threadripper is like apple to orange. so.......that why i ask. hahaha. sorry for the redundant question tho.

[Habib-CFD]

Scaling analysis of FLow 3D using AMD Epyc 7302P on CentOS 7.7:

Cores..........Default model
1...................575s
2...................354s
4...................269s
8...................183s
12.................144s
16.................140s

[thunde47]

Thanks for all the work. I was thinking that perhaps it may be more meaningful to perform these analyses for longer runtime simulations.

[Habib-CFD]

The ideal condition for EPYC series is running two jobs at the same time (set 8 cores per job) if your license supports it. For simultaneous jobs, there is no chance for a similar price of Intel products due to RAM bandwidth!