[iilw1314]

hi guys,i'm wondering how to parallelize the code for unstructured grids with adaption and can implicit scheme aply to code for unstructured grids with adpation.any suggestion or any guid ?thank you very much!

[DoHander]

A simple and effective approach will be to parallelize your main loops by using OpenMP or CUDA threads. This works for a multicore computer or for a computer with one or more graphic cards.

If you plan to use more computers for the task, you will need to decompose your grid in pieces and solve each piece on a separate computer using MPI. I have more experience with the first approach so I can't give you more advice for domain decomposition and MPI.

Do

[iilw1314]

your reply is appreciated.thank you.OPENMP seems useless because i can only parallelize 90% of my code.MPI seems difficult with unstructured grids especially with adaption,i have not found any paper that describle how to do it with MPI.
thank you again.

[Martin Hegedus]

I'd suggest starting from the basics and moving to more complex methodologies. Start with pthreads. Write a simple example code with that. It is relatively easy to do for CFD codes as long as you are diligent about threads not clobbering each others memory. Luckily, CFD codes are very systematic and organized. User interfaces are orders of magnitude more difficult. One tip is to ensure that the memory you are using to perform a calculation is not the memory you are writing to, i.e. results_memory_set_b = f(memory_set_a). Then, if you want, move on to OpenMP. I guess I'm not sure why OpenMP is useless for your application. Granted, I use pthreads for my multithreading work since it is straight forward. But OpenMP works too. Then move onto MPI if you want a multi-machine appliction.

[iilw1314]

thank you for your answers,my code actually can only be parallelized up to about 70%~80% of it,it means that you will not get good efficiency for the bottleneck ,do u know any method or information about how to parallelize the code with adaptive grids with MPI?

[Martin Hegedus]

Sorry, I do not know of a detailed "How To" guide for converting an adaptive grid methodology for use with MPI. I assume it is implementation specific. I assume you have taken a look at other software out there in regards to this.

Out of curiosity, what are the 20%-30% parts of the code that you can not parallelize?
Is it the adaptation method? I assume you are using an explicit solver. Or does your reference to implicit solver in your first question mean your code is an implicit solver? Note, I am neglecting issues of memory bandwidth.

Unfortunately, in the end, if you can not parallelize your code for multiple threads then MPI may not help you out either. After all, if you can break up your problem for MPI, you should be able to break up your problem for efficient solution with multiple threads. Again, this is neglecting issues of memory bandwidth.

[iilw1314]

well,the 20%-30% part can not be parallelized because ,you know in OPENMP the main work to parallelize the code is to treat the "do loop",and some varibles in this loop have conection,so you can not parallelize it.maybe i did not make it clear ,if you are interested in why so difficult ,you should look into some books about OPENMP.

And my code is explicit,and i am trying to use explicit-implicity method in my code right now.you know it is very difficult to do it on unstructured grids.i am trying.

thank you for your reply so much.

[Martin Hegedus]

Sorry, just trying to help.

Yes, creating an implicit unstructured solver is a challenging task.

From my own work, parallelizing the creation of the LH and RH matrix for an unstructured solver was straight forward. However, parallelizing the inversion of the LHS matrix for an unstructured solver is implementation dependent. It could be somewhat easy or it could be hard. The devil is in the details. None of my CFD development work explicitly involved grid adaptation, so I can not comment on that aspect of it.

Also, for me, parallelizing an explicit unstructured solver was straight forward.

So I guess you are asking how to parallelize the implicit matrix solving scheme. Sorry, until now, that was not clear for me. Don't get me wrong, I understood going into this that it was an area of possible difficulty. But you could have been new at multithreading, could have been having problems in other areas of your code, or you could have been asking a general question. I had no way of knowing what your level of experience with multi-threading is. And, since you did not provide much detail, I assumed it was on the "get started" level. I will now assume you are very knowledgeable about it. Therefore, your knowledge either exceeds, or is on par with, mine. I probably can not provide much assistance.

I also assume you know that the memory bandwidth bottle neck, resulting from the randomness of the memory storage of an implicit unstructured solver, is a hurdle. In general, the access to main memory is very slow in comparison to the CPU speed and memory is constantly being sent back and forth between the cache and main memory for larger sized unstructured problems. Well, that has been my personal experience. Unfortunately that experience is a few years old and I assume new techniques have been developed to address this. I assume you have much more knowledge of these issues than I.

Once again, I apologize.

[iilw1314]

well,thank you again.

[arjun]

Originally Posted by iilw1314

well,the 20%-30% part can not be parallelized because ,you know in OPENMP the main work to parallelize the code is to treat the "do loop",and some varibles in this loop have conection,so you can not parallelize it.maybe i did not make it clear ,if you are interested in why so difficult ,you should look into some books about OPENMP.

And my code is explicit,and i am trying to use explicit-implicity method in my code right now.you know it is very difficult to do it on unstructured grids.i am trying.

thank you for your reply so much.

It depends on your algorithm, that you will decide how you want do to make it parallel.

You should spent time thinking how you could do or make it parallel, rather than singing that 20-30% of code is impossible to make parallel by openmp.

PS: I have my unstructured grid solver written with openmp and 100% parallel. There is nothing that it does not do in parallel.