[saurabh3737]

Hello everyone,
I have setup clustring environment on ubuntu 12. on 3 machines.
1 server and 2 clients
my cluster setup works well with any 1 server and 1 client combination,
but i was unable to run on all 3 machines for buoyantSimpleFoam in parallel

i m using command
mpirun -hostfile machines -np 3 /nfs/openfoam211/bin/foamExec buoyantSimpleFoam -parallel

when i execute this command for 3 machines than i din get any response not even an error it just hangs,
i check process on other 2 clients machines using Top command, so i can see that server is trying to initiate some process from server IP but it goes off every while and than, and ultimately buyantSimpleFoam is not running,
but it works perfectly with and 2 machines with 1 server and 1 client with different number of cores whatever i assigned.
(i personally feel its interprocess communication problem between clients)

Please Help me, any solution for above, its really very Urgent, Thank u so much.

[wyldckat]

Greetings saurabh3737 and welcome to the forum!

I moved your post to a new thread and the proper sub-forum, because the other thread where you posted was really old...

OK, the problem you're having is very likely due to issues with network configuration. There are two details that come to mind:

• Check if the machines you're working with have the files "/etc/hosts" properly configured.
• What MPI toolbox are you using? Open-MPI? MPICH2?
  I ask this because for Open-MPI you can do something similar to this:
  

  Originally Posted by pkr

  When using MPI_reduce, the OpenMPI was trying to establish TCP through a different interface. The problem is solved if the following command is used:
  mpirun --mca btl_tcp_if_exclude lo,virbr0 -hostfile machines -np 2 /home/rphull/OpenFOAM/OpenFOAM-1.6/bin/foamExec interFoam -parallel
  
  The above command will restrict MPI to use certain networks (lo, vibro in this case).

• Additionally, on how to test if MPI is working: post #4 of "openfoam 1.6 on debian etch", and/or post #19 of "OpenFOAM updates" - Note: As of OpenFOAM 2.0.0, the application "parallelTest" is now called "Test-parallel".

For more information, you can also check my blog post Notes about running OpenFOAM in parallel

Best regards,
Bruno

[saurabh3737]

I tried with this command as below but still same problem persist,

mpirun --mca btl_tcp_if_exclude lo,virbr0 -hostfile machines -np 6 /nfs/openfoam211/bin/foamExec buoyantSimpleFoam -parallel

is there any need to have passwordless SSH between 2 client machines as well?
or any other solution

Also my Test-parallel is not working for 1 server and 2 clients and it hangs after giving execution command, but works perfectly fine when i assigned with 1 server and 1 client.

can you please help me out to identify and resolve the problem.

[wyldckat]

Originally Posted by saurabh3737

is there any need to have passwordless SSH between 2 client machines as well?

Only if one of the client machines is the one acting as master.

Originally Posted by saurabh3737

Also my Test-parallel is not working for 1 server and 2 clients and it hangs after giving execution command, but works perfectly fine when i assigned with 1 server and 1 client.

can you please help me out to identify and resolve the problem.

A few ideas/questions:

1. Check the file "/etc/hosts" in all 3 machines. Do they all have knowledge of the names and IPs of the other machines?
2. Are you able to ping all machines among themselves? Can the server ping each one of the two clients and is each client able to ping the other client and server?

[saurabh3737]

I have checked all the hosts requirement ping will works in all 3 machines to any machine among them. .etc/hosts is properly configured, is there any need to configure wmakeschedular of openfoam or have to set enviornment for schedule or somwhere else in openfoam to give no of machines?
Is it possible to have any direct communcation with you by phone, email or skype, will be grateful to u.
Thanks

[wyldckat]

Originally Posted by saurabh3737

I have checked all the hosts requirement ping will works in all 3 machines to any machine among them. .etc/hosts is properly configured

Testing if you can ping is just one of the most basic tests. The next step is to determine how many network cards are there and if there is visibility among all of them. In other words, when you run on each machine:

/usr/sbin/ifconfig

Are all of the connections visible to the same internal network? Or is there only lo and eth0 on the client machines, as shown here:

eth0      Link encap:Ethernet  HWaddr AA:BB:CC:DD:EE:FF  
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Interrupt:41 Base address:0xc000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Originally Posted by saurabh3737

is there any need to configure wmakeschedular of openfoam or have to set enviornment for schedule or somwhere else in openfoam to give no of machines?

wmakescheduler related information only affects building OpenFOAM itself in parallel, it does not affect running cases. For cases, you should use foamJob and a machines file. Example:

foamJob -s -p Test-parallel

File "machines", placed inside the case folder:

server slots=4
slave1 slots=2
slave2 slots=2

Originally Posted by saurabh3737

Is it possible to have any direct communcation with you by phone, email or skype, will be grateful to u.
Thanks

Via email in my off-duty hours, you can start the conversation by using the "Contact Info -> Send a message via email" from my profile page: http://www.cfd-online.com/Forums/members/wyldckat.html
For Skype or phone, it'll have to be on-duty and you'll have to be willing to pay for the support; you can either send me an email or contact the company I work for, namely blueCAPE: http://www.bluecape.com.pt