Parallel Processisng DecomposePAR - Different Solutions

PeteH · March 23, 2016, 08:22

I am trying to understand the time and solution effects of parallel processing on my computer (i7 mac running CAElinux 2013). I had a case that I've been trying to run on 4 cores (76000 cells on each decomposed mesh part - too many?) and I found that the parallel processing did not speed it up at all. So, experiment:
I ran the Tjunction case in the incompressable tutorial folder as a serial solver and it finished in 15 seconds.
I then decomposed it into (4 1 1) using 4 cores, used the renumberMesh utility and ran it again as a parallel process. This time it took 20 seconds. I can understand that because the geometry would bottleneck the solution at the end of the geometry in addition to the wasted time talking with so many cores for a small simulation.
So I decomposed the geometry into (1 2 1) using 2 cores, renumberMesh, and ran again. This time it finished in 12 seconds. This also made sense to me because now it is splitting the geometry symmetrically along the x axis and for such a small case, it should help.
The only problem I see now, is that the solution when looking in ParaView is very different from the serial solution. any ideas why this would be the case?

Also, I get a random .001 time step folder in the case folder. All other time steps are located in the processor 0 or processor 1 folders except for the IC 0 folder. Any ideas why that is there?

PeteH · March 24, 2016, 08:25

I figured it out.
I was performing the reconstructPar incorrectly. The Tjunction case works correctly, now.
As for my other case -
Split into 4 partitions (76000 cells each) with 4 cores, I am not seeing a decreased solving time.
Could this be a RAM issue?

I will try to decrease the number of cells to about 200k total - about 50k per partition, and see how that affects the speed.

Any thoughts are appreciated.

March 23, 2016, 08:22	Parallel Processisng DecomposePAR - Different Solutions	#1
PeteH New Member Peter Hagan Join Date: Oct 2011 Location: Virginia, USA Posts: 12 Rep Power: 15	I am trying to understand the time and solution effects of parallel processing on my computer (i7 mac running CAElinux 2013). I had a case that I've been trying to run on 4 cores (76000 cells on each decomposed mesh part - too many?) and I found that the parallel processing did not speed it up at all. So, experiment: I ran the Tjunction case in the incompressable tutorial folder as a serial solver and it finished in 15 seconds. I then decomposed it into (4 1 1) using 4 cores, used the renumberMesh utility and ran it again as a parallel process. This time it took 20 seconds. I can understand that because the geometry would bottleneck the solution at the end of the geometry in addition to the wasted time talking with so many cores for a small simulation. So I decomposed the geometry into (1 2 1) using 2 cores, renumberMesh, and ran again. This time it finished in 12 seconds. This also made sense to me because now it is splitting the geometry symmetrically along the x axis and for such a small case, it should help. The only problem I see now, is that the solution when looking in ParaView is very different from the serial solution. any ideas why this would be the case? Also, I get a random .001 time step folder in the case folder. All other time steps are located in the processor 0 or processor 1 folders except for the IC 0 folder. Any ideas why that is there?

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Running decomposePar / reconstructPar as parallel apps?	carcass	OpenFOAM Running, Solving & CFD	3	January 17, 2024 08:19
Explicitly filtered LES	saeedi	Main CFD Forum	16	October 14, 2015 12:58
simpleFoam parallel	AndrewMortimer	OpenFOAM Running, Solving & CFD	12	August 7, 2015 19:45
decomposePar: "Couldn't create directory"	Manm	OpenFOAM Pre-Processing	1	April 19, 2015 12:28
how to make parallel running? decomposePar.	panda60	OpenFOAM	1	October 29, 2009 03:43

March 24, 2016, 08:25		#2
PeteH New Member Peter Hagan Join Date: Oct 2011 Location: Virginia, USA Posts: 12 Rep Power: 15	I figured it out. I was performing the reconstructPar incorrectly. The Tjunction case works correctly, now. As for my other case - Split into 4 partitions (76000 cells each) with 4 cores, I am not seeing a decreased solving time. Could this be a RAM issue? I will try to decrease the number of cells to about 200k total - about 50k per partition, and see how that affects the speed. Any thoughts are appreciated.