Understanding hardware & parallel processing requirements

yk2359 · May 3, 2018, 18:45

Hello, forum!

I am a grad student. For my thesis, I was given access to below set of Hardware & Software.

Windows Server 2012 R2 Standard
Intel (R) Xeon (R) CPU E5-2650 v3 @2.30GHz (2 Processors)
128GB RAM & 64 bit OS
Sockets-2, Cores-20 & Logical Processors-40

ANSYS 19.0- Workbench, Fluent, Mechanical & System Coupling with following licenses ANSYS Academic Research Mechanical & CFD, ANSYS Academic Research HPC.

In ANSYS license preferences there's statement ***The maximum number of HPC pack licenses that can be requested per user for a solve is 14**

Being naive I have been using these resources without even understanding the complete potential, for instance, by changing processor number for the parallel in the Fluent launcher or Mechanical settings.

In a particular case:

With 10 processors on Mechanical & 24 on Fluent, the transient FSI simulation was working fine. But the use of implicit update in Fluent threw an error "mpirun.exe has stopped working".

However, it was working well if I set 2 or 4 processors on each.

I have not found any solution to this yet.

This error pushed me off bad to get a better understanding of parallel processing settings.

I am requesting this forum to help me understand the capabilities of these resources. How many processors can I possibly set up for Fluent and Mechanical for 10K-1M elements/nodes in each mesh? Do I need an MPI for this server? Is the number of processors related to licenses available?

I would really appreciate if anyone can help me get these basics right.

Thank you.

YK

LuckyTran · May 23, 2018, 10:42

You error so far seems pretty generic. mpirun.exe has stopped running means one of the parallel processes stopped, which doesn't really give much information.

You always need an mpi for parallel simulations.

The HPC pack is a non-linear scaling license which starts at base 8 and scales in powers of 4. 1 HPC Pack lets you run on 8 processes. 2 HPC packs will let you run on 32, 3 Packs on 128, etc. 14 packs is a lot, 2 billion processes simultaneously. So you don't have a licensing problem.

I would go into the bios and disable hyper threading; it causes too many problems without any benefit in a computing job. 1 E5-2650 has 10 physical cores (20 hyperthreaded). So 2x E5-2650's you have 20 physical cores (40 hyperthreaded). When you specify 10+24 processes, that is more than the 20 physical cores so some of these processes end up on the hyper-threaded ones. So you're not getting any speed benefit past a total of 20 processes. But even if you specify a number less than 20, you cannot control and ensure that the simulation is running on only the non-virtual cores. So it's best (especially when you have these problems) to disable hyperthreading entirely. Do this and see if you can run on 20 processes stably.

Micael · May 24, 2018, 14:29

Just for info, the below message has nothing to do with how many HPC license one actually have. I think it just means that, given you could afford 15 HPC packs, you still could only use up to 14 for a solve.

***The maximum number of HPC pack licenses that can be requested per user for a solve is 14

LuckyTran · May 24, 2018, 14:34

ha, ha, ha, I did not think of it.

Well then it would be helpful if the license configuration is posted. Btw, if there aren't enough licenses, the processes will not start and you won't get very far and you would never be able to run anything. Hence, it should be quite obvious... So I assume there's not a licensing problem.

May 3, 2018, 18:45	Understanding hardware & parallel processing requirements	#1
yk2359 New Member YK Join Date: Jul 2017 Posts: 8 Rep Power: 9	Hello, forum! I am a grad student. For my thesis, I was given access to below set of Hardware & Software. Windows Server 2012 R2 Standard Intel (R) Xeon (R) CPU E5-2650 v3 @2.30GHz (2 Processors) 128GB RAM & 64 bit OS Sockets-2, Cores-20 & Logical Processors-40 ANSYS 19.0- Workbench, Fluent, Mechanical & System Coupling with following licenses ANSYS Academic Research Mechanical & CFD, ANSYS Academic Research HPC. In ANSYS license preferences there's statement *The maximum number of HPC pack licenses that can be requested per user for a solve is 14 Being naive I have been using these resources without even understanding the complete potential, for instance, by changing processor number for the parallel in the Fluent launcher or Mechanical settings. In a particular case: With 10 processors on Mechanical & 24 on Fluent, the transient FSI simulation was working fine. But the use of implicit update in Fluent threw an error "mpirun.exe has stopped working". However, it was working well if I set 2 or 4 processors on each. I have not found any solution to this yet. This error pushed me off bad to get a better understanding of parallel processing settings. I am requesting this forum to help me understand the capabilities of these resources. How many processors can I possibly set up for Fluent and Mechanical for 10K-1M elements/nodes in each mesh? Do I need an MPI for this server? Is the number of processors related to licenses available? I would really appreciate if anyone can help me get these basics right. Thank you. YK

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Parallel processing of OpenFOAM cases on multicore processor???	g.akbari	OpenFOAM Running, Solving & CFD	31	November 1, 2017 10:25
foamMonitor in parallel processing	foamingMouth	OpenFOAM Running, Solving & CFD	0	September 15, 2016 10:44
Communication in parallel processing	kcn	OpenFOAM Programming & Development	0	October 7, 2014 07:37
Grid Check Fails in Parallel Processing Mode	askance	Main CFD Forum	0	October 20, 2010 11:11
parallel processing and dynamic mesh	Anand	FLUENT	0	January 9, 2009 16:52

May 23, 2018, 10:42		#2
LuckyTran Senior Member Lucky Join Date: Apr 2011 Location: Orlando, FL USA Posts: 5,761 Rep Power: 66	You error so far seems pretty generic. mpirun.exe has stopped running means one of the parallel processes stopped, which doesn't really give much information. You always need an mpi for parallel simulations. The HPC pack is a non-linear scaling license which starts at base 8 and scales in powers of 4. 1 HPC Pack lets you run on 8 processes. 2 HPC packs will let you run on 32, 3 Packs on 128, etc. 14 packs is a lot, 2 billion processes simultaneously. So you don't have a licensing problem. I would go into the bios and disable hyper threading; it causes too many problems without any benefit in a computing job. 1 E5-2650 has 10 physical cores (20 hyperthreaded). So 2x E5-2650's you have 20 physical cores (40 hyperthreaded). When you specify 10+24 processes, that is more than the 20 physical cores so some of these processes end up on the hyper-threaded ones. So you're not getting any speed benefit past a total of 20 processes. But even if you specify a number less than 20, you cannot control and ensure that the simulation is running on only the non-virtual cores. So it's best (especially when you have these problems) to disable hyperthreading entirely. Do this and see if you can run on 20 processes stably.

May 24, 2018, 14:29		#3
Micael Senior Member Micael Join Date: Mar 2009 Location: Canada Posts: 157 Rep Power: 18	Just for info, the below message has nothing to do with how many HPC license one actually have. I think it just means that, given you could afford 15 HPC packs, you still could only use up to 14 for a solve. ***The maximum number of HPC pack licenses that can be requested per user for a solve is 14

May 24, 2018, 14:34		#4
LuckyTran Senior Member Lucky Join Date: Apr 2011 Location: Orlando, FL USA Posts: 5,761 Rep Power: 66	ha, ha, ha, I did not think of it. Well then it would be helpful if the license configuration is posted. Btw, if there aren't enough licenses, the processes will not start and you won't get very far and you would never be able to run anything. Hence, it should be quite obvious... So I assume there's not a licensing problem.