[tailele]

Good morning , i try to explain my probem. Now i try for complete my thesis , and i would use a student license of star ccm+ with my two pc . I have a desktop and a laptop. In both the machine i installed windows 10 . I read the help desk but i think that i wrong something. Now the help desk says that i need to start mpi ( for register my credential in a non server version of windows ) , but i don t understand if must to start mpi ibm or interl and if i must restard the mpi everytime that i shoot down the system ; so this is my first problem . The second problem is when i use the powershell ; i don t undertand if i must lunch powershell in both the pc or only in one . this is the procedure :

1. Make sure that your credentials are registered on all machines that you want to use. See Registering Your
Credentials for Distributed Simulations on Windows.
2. Click <Start> and type powershell in the search field.
3. Right-click on Windows PowerShell from the list of search results, and select Run as Administrator. You
must have administrator privileges.
The Administrator: Windows PowerShell command prompt appears.
4. Change to the shared working directory:
> cd \\<MACHINE_NAME>\<SHARED_DIR>
5. Launch Simcenter STAR-CCM+:
◦ If Simcenter STAR-CCM+ has the same installation path on each machine, enter:
> starccm+
◦ If Simcenter STAR-CCM+ has a different installation path on each machine, use the UNC path to the
Simcenter STAR-CCM+ executable:
> \\<MACHINE_NAME>\<STAR-CCM+_INSTALL_DIR>\starccm+
The Simcenter STAR-CCM+ GUI appears.

Thanka a lot

[me3840]

I would advise against using Windows for this, it's much easier to do on Linux.


You don't have to start MPI on both machines provided you performed the registration steps (see step 1 in what you posted). You need only start powershell and star on one machine, then you specify the other before you open the sim file with the parallel options. Alternatively you can run the program from the command line and specify a machine file. Look at the working in parallel section in the user guide, it's quite good.

[tailele]

Originally Posted by me3840

I would advise against using Windows for this, it's much easier to do on Linux.


You don't have to start MPI on both machines provided you performed the registration steps (see step 1 in what you posted). You need only start powershell and star on one machine, then you specify the other before you open the sim file with the parallel options. Alternatively you can run the program from the command line and specify a machine file. Look at the working in parallel section in the user guide, it's quite good.

Hi, thanks for the reply. Now yesterday i installed in both the pc windows server evaluation copy ; so with this os i could see bith the computers , and in particular i can see the installation directory of star in common folder ( one in the node 1 and the other in node 2 ) . But when i lunch the shell command , is doesn t work . So i have lunch a prompt command and write "path" ; actually star is not present in path command. Maybe i ll try to insert a ambient variable in bin directori of star. Is it correct?

[tailele]

ok , now the computers work well. The ethernet's cards work at 35% . In your opinion , is it possible to use another computer? i mean , both the workstations have 2 ethernet ports' , if i use another computer with 2 ports , is it possible to connect the threr workstations without switch?

[me3840]

No, you must use a switch if you want more than 2 computers. If you're only using ethernet switches should be very inexpensive, but don't expect stellar performance.


Use only 1 port per machine. The bottleneck is latency, not bandwidth, so adding an extra port on each machine won't help.

[tailele]

I don t understand, so in tour opinion is not important the utilization of the network...so for you is better have a 10 gb lan?

[me3840]

Unless you can saturate your 1gb connection moving up to 10gb won't help, so, yes, the utilization is not important. If you want better performance you need to move to a low-latency interconnect, i.e. Infiniband.


Though for just 2 nodes it might not be worth it. Depending on your hardware it might be that even 1gb ethernet is fast enough. Two nodes is not a lot of traffic.

[tailele]

Ok now i understand . I try with my desktop and a laptop , now i se that it works and so i buyed another two desktop so i have this hardware:
3 identical nodes with:
2 xeon 2658 v3
64 gb 2133mhz cl13
256 m2 ssd
Asus z10pe d8

So i buyed 3 hp pci card nc522sfp...but now i understand that the latency could be' a problem.......

[tailele]

Originally Posted by me3840

Unless you can saturate your 1gb connection moving up to 10gb won't help, so, yes, the utilization is not important. If you want better performance you need to move to a low-latency interconnect, i.e. Infiniband.


Though for just 2 nodes it might not be worth it. Depending on your hardware it might be that even 1gb ethernet is fast enough. Two nodes is not a lot of traffic.

In your opinion sfp + is paragonable with infiniband? Or is it at least near ?

[tailele]

Originally Posted by tailele

In your opinion sfp + is paragonable with infiniband? Or is it at least near ?

So i readed some articles and i found that gbe standard have 29-100 msec, 10gbe around 13msec, and sfp + with infiniband around 0.1 to 1,5 msec...so i think that are similar... do i wrong?

[cwl]

Originally Posted by me3840

If you want better performance you need to move to a low-latency interconnect, i.e. Infiniband.

Though for just 2 nodes it might not be worth it. Depending on your hardware it might be that even 1gb ethernet is fast enough. Two nodes is not a lot of traffic.

From my experience (and official Star benchmarks) I can tell - that up to 5-6 nodes performance loss (10GbE vs Infiniband) is negligible.
AFAIR, Star benchmark predicts 10GbE to be only 20% slower for 8 nodes, and that is valid for 2 x Xeon ** v2/v3, not to mention common PCs.

[flotus1]

You still have to take these benchmarks with a grain of salt.
In our office, we only have Gigabit Ethernet as an interconnect between the workstations. This is usually enough to get decent scaling across 2 nodes with sufficiently large cases (around 5M cells and up).
However, I recently tried to run a case (~2.4M cells) across 2 nodes that had ~20 different regions with interfaces between them. Scaling was abysmal and I ended up running the case faster on a single node.

[tailele]

Now i use 72 cores with a 6 milion's mesh cells, and i see that there aren t difference between 48 cores and 72 cores. I think that is the ethernet cards. This week i ll mount 3 sfp + cards. And then i ll day if there are some different speed.

[arjun]

Originally Posted by flotus1

You still have to take these benchmarks with a grain of salt.
In our office, we only have Gigabit Ethernet as an interconnect between the workstations. This is usually enough to get decent scaling across 2 nodes with sufficiently large cases (around 5M cells and up).
However, I recently tried to run a case (~2.4M cells) across 2 nodes that had ~20 different regions with interfaces between them. Scaling was abysmal and I ended up running the case faster on a single node.

A good amount of issues come from AMG coarsening too and its degradation upon adding more processors.

[tailele]

Ok, now i m confuse.... i installed the new sfp plus cards and the new router. So all the 3 nodes talks together ... i lunch power shell and then star by the powershell...i lunch the named host with 72 cores...and i see that all 6 xeons work together... bit if i want to see the difference between 72 cores and 24 Cores, ( 24 cores with simple one node turn on ) , i could see that the time for one iteration is always 7 seconds....i don t understand ..
I try my esperiment with a 3 milions mesh's elements...

[tailele]

is it possible that all the nodes do the same operation , and then the software takes the fastest node?