|
[Sponsors] |
Interesting behavior when using LAM without ib rpi on an infiniband cluster OF 141 |
|
LinkBack | Thread Tools | Search this Thread | Display Modes |
October 2, 2007, 17:31 |
I am running a multphaseInterF
|
#1 |
Senior Member
Srinath Madhavan (a.k.a pUl|)
Join Date: Mar 2009
Location: Edmonton, AB, Canada
Posts: 703
Rep Power: 21 |
I am running a multphaseInterFoam case (approx 1.1 million hex cells) on an infiniband cluster. Thanks to Henry's help I have switched from quartic interfacecompression scheme to quadratic. This seems to keep the volume fractions stable. Anyways, when I tried to use the following procedure to start a parallel run with 8 processors, I get an error message similar to that reported in [1]:
lamboot -v $PBS_NODEFILE mpirun -ssi rpi ib -np 8 multiphaseInterFoam . three_fluid_test_4 -parallel I was just about to give up on getting the inifiniband to work, when I came across this webpage[2]. Following their procedure, I used the following commands to start the run: lamboot -v -b -ssi boot rsh -ssi rsh_agent rsh $PBS_NODEFILE mpirun -ssi rpi tcp -np 8 multiphaseInterFoam . three_fluid_test_4 -parallel and the run started without problems. The lamboot initialization parameters suggest that I am not using the ib module? So I expected degraded performance. However, comparing the time after 7 time steps for a serial run with that of a parallel run, I get a speed up of 8.125818891 which is indeed impressive. I cannot explain why this seems to work. Any comments are appreciated. PS: The cluster is a combination of 256 processors (i.e. 128 dual opteron boxes NOT dual-core) connected with infiniband. [1] http://www.cfd-online.com/OpenFOAM_D...tml?1188946223 [2] http://tensor.tamu.edu/docs/lam.html |
|
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
HPMPI Infiniband problem | carsten | OpenFOAM Bugs | 2 | January 25, 2009 16:36 |
Infiniband | alexander_rudert | OpenFOAM | 20 | January 14, 2009 09:01 |
OpenFOAM and infiniband | mrangitschdowcom | OpenFOAM Installation | 5 | October 30, 2008 08:47 |
infiniband interconnect help | rjh | FLUENT | 0 | February 4, 2008 12:41 |
OpenFOAM 141 parallel results infiniband vs gigabit vs SMP | msrinath80 | OpenFOAM Running, Solving & CFD | 10 | November 30, 2007 19:11 |