[HHOS]

Hello all!


I have to setup a hpc cluster with 4 nodes with AMD Epyc CPUs and connected by infiniband for making CFD simulations with Ansys Fluent. The head node will also be for calculations.


I just want some tips of how should I do it. The idea is to set it up with linux, maybe CentOS??, but then I don't know how anything would work together.


Everything should be sent to the head node through Ansys RSM, but then... Is it possible to visualize how a simulation is going? or simply not until the end?


Do you know about any guide to set all up? Also, I am not sure whether a GPU is required in the system in order to be able to somehow visualize the simulation status or something.


Thank you very much for your help. Any tip would be helpful.

[vinerm]

It depends whether the installation of Fluent is network installation or each node has its local copy. If each node has local copy, then there is nothing much to be done. You don't even need RSM. Just ensure that passwordless ssh is enabled, i.e., if you use ssh or rsh to log on from node 0 to node 1 it should not ask for password. This is done using public-private key. Once done, you can invoke Fluent from any of the nodes and provide a file containing the names of the nodes. This is done either via -cnf option if using command line to start or on Parallel tab on Fluent launcher. Each node name has to appear as many times in the file as many cores you wish to use from that node.

If the cluster is to be setup for queuing then it depends on queuing system. For a four node cluster, RSM itself may have some capability.

[HHOS]

That was a fast reply! Thanks!


I didn't know about the possibility of having a "network installation", so I thought of installing it everywhere. Does that change the required licenses?



What I am not sure about is how to achieve that the four computers are treated as just one. And cannot picture the workflow when working with a cluster.



And, would be needed a GPU or not even?


Sorry that I am completely lost, but for this big challenge that is this big hardware upgrade I need to learn about too many things, and cannot manage to find a good guide that covers it all (like with everything related to ANSYS that is a bit more advanced...)


Regards!

[vinerm]

License is required by the instances. So, if you have only one Fluent running then you need only once license. However, you will require HPC license for running on more than 4 cores. Do note cores are different from nodes. Nodes imply machines and each node can have multiple cores or processors. If the machines are connected to each and you can login from one to another, then it will work.

[HHOS]

I see, we are fine from that point then, thanks

I still am trying to picture how it would be the workflow working with the cluster. As I understand it, I would setup a case in a workstation, copy it to the cluster and run it there. However, some questions arise:

-Is there a way to monitor the residuals or monitors from the simulation that is running in the cluster? It might be that something goes wrong and that much computing time will be wasted if undetected.

-When running a journal file, I sometimes print intermediate status flow-field pictures every certain amount of timesteps. In order to render those pictures, does the cluster need a GPU, or is it not required?

Thanks

[vinerm]

While running the case, it would be similar to running on single node. You can start Fluent using GUI and monitor the case as it runs. You can save images and animations as well. GPU is helpful but not required provided a graphics card is available on at least one of the nodes.

Another important point is that all the nodes should be able to access the drive that contains case files.

[HHOS]

[QUOTE=vinerm;759618]While running the case, it would be similar to running on single node. You can start Fluent using GUI and monitor the case as it runs./QUOTE]


But that would mean using the cluster as a workstation, or through remote desktop or so.



However, what I would like is to send the simulation to run in the cluster, while being able to monitor it. Something like opening the GUI in my workstation, while every simulating core would be from the cluster.


Thanks

[vinerm]

That requires a queuing system. RSM might be able to help you but I have no experience with that. Otherwise, you can use SGE if you want a free alternative. But it requires knowledge of PERL.