|
[Sponsors] |
Fluent run with limit around 10-15MB/s bandwidth |
![]() |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
![]() |
![]() |
#1 |
Member
Nguyen Trong Hiep
Join Date: Aug 2018
Posts: 48
Rep Power: 8 ![]() |
I have 3 computer node with 16 cores each, 1GE interconect.
I run fluent mpitest or fluent report bandwidth give me 100MB/s. But when i run a simple simulation, single phase, no interface, network monitor only show 10 to 15MB in/out. Both intelmpi and openmpi give the same result. How can i use all my network bandwidth in my simulation? |
|
![]() |
![]() |
![]() |
![]() |
#2 |
Senior Member
Join Date: Oct 2011
Posts: 242
Rep Power: 16 ![]() |
Hello,
This probably simply means that your run does not require a high network bandwidth. The amount of data transfered to the network is roughly speaking proportional to the number of cells located at the mesh partitions boundaries. If your mesh is not too big, if your number of partitions is not too big, the amount of data is not so big and 1GbE limit is not reached, which is good ! Note that high latency of 1GbE can also impact the performance, but I would not worry too much with only 3 nodes. If you want to see if your network slows down your calculations, monitor the runtime using 1, then 2 and finally 3 nodes. If it stalls between 1 and 2 or 2 and 3, then you may have a network limiting the performance. |
|
![]() |
![]() |
![]() |
![]() |
#3 | |
Senior Member
Will Kernkamp
Join Date: Jun 2014
Posts: 365
Rep Power: 14 ![]() |
Quote:
In addition to the answer above, there may be another reason: The mpi communication happens between the nodes upon completion of an iteration step. So the majority of the time, the nodes are at work and not sending anything. So if for 15% of the time the nodes are communicating at maximum bandwith, you would see a bandwidth use of 15% * 100 MB/s = 15 MB/s |
||
![]() |
![]() |
![]() |
![]() |
#4 | |
Member
Nguyen Trong Hiep
Join Date: Aug 2018
Posts: 48
Rep Power: 8 ![]() |
Thank you for reply,
In my case, i have 10mil poly mesh and 48 cores. Fluent report that each iteration took 3.328s, data transfer per ite: 300MB, LE wall-clock/iter: 1.346(40%). Quote:
When i increase 3 computer node to 10, speed increase only 50%. That whh i think the network is the bottleneck |
||
![]() |
![]() |
![]() |
![]() |
#5 |
Member
Nguyen Trong Hiep
Join Date: Aug 2018
Posts: 48
Rep Power: 8 ![]() |
||
![]() |
![]() |
![]() |
![]() |
#6 | |
Senior Member
Will Kernkamp
Join Date: Jun 2014
Posts: 365
Rep Power: 14 ![]() |
Quote:
For 10 nodes yes. I have build a network with old ConnectX=3 cards. That should work better for you. If you get dual port cards, you can build a ring config. Still fast and no noisy hub. (Only for people with some vacation time due, haha) |
||
![]() |
![]() |
![]() |
![]() |
#7 | |
Senior Member
Join Date: Jun 2011
Posts: 205
Rep Power: 16 ![]() |
Quote:
|
||
![]() |
![]() |
![]() |
|
|
![]() |
||||
Thread | Thread Starter | Forum | Replies | Last Post |
Stop Fluent from cycling though graphics after end of run | sskp | FLUENT | 0 | August 17, 2022 03:47 |
Preserve meshes in Fluent parametric run | hwerner2014 | FLUENT | 2 | February 28, 2022 16:29 |
Scripts to run fluent in Linux cluster via windows | sanjaykummar2810 | FLUENT | 1 | September 19, 2018 16:23 |
Automatic run Fluent | Phuc_Nguyen | FLUENT | 1 | October 19, 2017 07:31 |
Parallel run in fluent for multiphase flow | apurv | FLUENT | 2 | August 3, 2011 19:44 |