[shipman]

Dear All,

I am carrying out a large scale simulation by connecting to a Super Computer using OpenFOAM. Due to huge data, I am using python script via pvbatch option to get the data automatically.

OpenFOAM and separetely paraview is already available in Super Computer. I cant use internet within super computer and I am only allowed to install program into my home location within SUPER PC.

Details of Super computer: Red Hat Enterprise Linux Server release 6.6

When I run the following run.sh file inside Super PC with python script using: sbatch run.sh

run.sh file

SBATCH -p g006m
#SBATCH -N 1
#SBATCH -n 1
#SBATCH -J VOF_100M
#SBATCH -o stdout.%J.log
#SBATCH -e stderr.%J.log

module load PrgEnv-gnu482
module load gnu/openmpi165

 export PATH=/home1/share/Qt/4.8.0/gnu/bin/:/home1/share/cmake/2.8.12/bin/:/home1/share/openmpi/1.6.5/gnu/bin/:/home1/share/gcc/4.8.2/bin/:/home1/share/p$
 export LD_LIBRARY_PATH=/home1/share/Qt/4.8.0/gnu/lib/:/home1/share/openmpi/1.6.5/gnu/lib/:/home1/share/gcc/4.8.2/lib64:/home1/share/python/2.7.6/lib/:$L$

/home1/share/paraview/4.0.1/gnu/openmpi/bin/pvbatch --use-offscreen-rendering PlotImage.py

getting following error:

librdmacm: Fatal: no RDMA devices found
--------------------------------------------------------------------------
[[14715,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: g002

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
/tmp/slurmd/job528788/slurm_script: line 3:  4736 Segmentation fault      /home1/share/paraview/4.0.1/gnu/openmpi/bin/pvbatch --use-offscreen-rendering PlotImage.py

Is there anyone to let me know how can I solve this problem?

Thanks in advance.

Baris

[alexeym]

Hi,

Since your script run in serial mode, you can ignore error message till the second line of minuses (well, you can ignore it even for parallel run in fact).

So the error is:

.../slurm_script: line 3:  4736 Segmentation fault      .../pvbatch --use-offscreen-rendering PlotImage.py

And the question is: why pvbatch segfaults on the node it was run? Since nobody here (except you) neither knows the configuration of the Super Computer, nor the content of the script PlotImage.py, it would be VERY difficult to answer it. Does it work at all (i.e. what will be the result of the slurm script is you just run 'pvbatch --use-offscreen-rendering --version')? Does your PlotImage.py work at all?

Since you are installing everything into home folder, guess the problem could be in conflicting/missing libraries.

[shipman]

Dear Alexey,

First, thank you for reply.

Truly speaking, I just meant that I am only allowed to install into my home directory within SUPER computer but I did not install anything. For OpenFOAM calculation I have just used the already installed OpenFOAM within the share folder of SUPER PC by linking where it is. I just run the parallel using following run.sh file for openfoam case and it worked:

!/bin/bash

#SBATCH -p d072h
#SBATCH -N 5
#SBATCH -n 100
#SBATCH -J VOF_100M
#SBATCH -o stdout.%J.log
#SBATCH -e stderr.%J.log

module load PrgEnv-intel
module load intel/openmpi165

source /home1/share/openfoam/2.1.x/intel/openmpi/OpenFOAM-2.1.x/etc/bashrc
mpirun -np 100 pimpleContamiFoam10_SF1R -parallel

Regarding to paraview, since it was already installed into share folder of Super PC I just wanted to use it by linking like similar OpenFOAM. however, it gave that error message.

Actually, I tried to install of Paraview into my home directory, however, since I can not connect to internet within super PC, it was quite cumbersome, like I have to install all the depended packages one by one and compile within my home folder. Therefore, I gave up this and decided to use already installed paraview within share folder of SUPER PC like openfoam.

The PlotImage.py is only a kind of test case whether we can get data using pvbatch option or not. I already tried it in my desktop pc and it worked. this is the content of PlotImage:

try: paraview.simple except: from paraview.simple import *  #---Connect("hostName", try: paraview.simple 
except: from paraview.simple import * 
 
#---Connect("hostName", portNo) 
#Connect("f0", 11111) 
 
Column_foam = OpenFOAMReader( FileName='../Column/Column.foam' ) 
 
Column_foam.UpdatePipeline(1000) 
 
RenderView1 = GetRenderView() 
 
#----Time 
RenderView1.ViewTime = 1000 
 
#----Background Color (Default) 
RenderView1.Background = [0.32941176470588235, 0.34901960784313724, 0.42745098039215684] 
 
#----Axis on or off 
RenderView1.CenterAxesVisibility = 0 
 
#----OffscreenRendering off 
#---- When you use "pvserver --use-offscreen-rendering" "with osmesa", set both of them 1 
RenderView1.UseOffscreenRendering=0 
RenderView1.UseOffscreenRenderingForScreenshots=0 
 
#----Window size 
RenderView1.ViewSize=[949,706] 
 
#-----Color Table 
r=Column_foam.PointData['p'].GetRange() 
 
#----RGBPoints=[BeginValue, r, g, b, EndValue, r, g, b] 
a1_p_PVLookupTable = GetLookupTableForArray( "p", 1, \ 
RGBPoints=[r[0],0,0,1, r[1],1,0,0], \ 
NumberOfTableValues=30, \ 
) 
 
#-----Color of Body 
DataRepresentation2 = GetDisplayProperties(Column_foam) 
DataRepresentation2.Representation = 'Surface With Edges' 
 
DataRepresentation2.ColorArrayName = 'p' 
DataRepresentation2.LookupTable = a1_p_PVLookupTable 
 
#DataRepresentation2.ScalarOpacityFunction = [] 
#DataRepresentation2.ScalarOpacityUnitDistance = 0.014374810608995429 
 
 
#-----Color of Bar 
#---Position:Position of the bar. Position2:Scale of the bar. 
ScalarBarWidgetRepresentation1 = CreateScalarBar(\ 
                Title='p[Pa]', \ 
                Position2=[0.1, 0.5], \ 
                Enabled=1, \ 
                LabelFontSize=12, \ 
                TitleFontSize=12, \ 
                Position=[0.05, 0.4]\ 
) 
ScalarBarWidgetRepresentation1.LookupTable = a1_p_PVLookupTable 
GetRenderView().Representations.append(ScalarBarWidgetRepresentation1) 
 
 
 
#-----Camera 
RenderView1.CenterOfRotation = [0.00039450079202652,0.000110898166894913, 0.5] 
 
RenderView1.CameraViewUp = [0.21641266854759553, -0.8239812714933662, 0.5236605972576934] 
RenderView1.CameraFocalPoint = [0.00039450079202627014, 0.0001108981668950143, 0.49999999999999983] 
RenderView1.CameraClippingRange = [1.1126808662010967, 3.137745671346546] 
RenderView1.CameraPosition = [1.3484326860003855, -0.5289963646916971, -0.8896539475806535] 
 
 
 
 
Render() 
 
WriteImage("image.png")

Does it work at all (i.e. what will be the result of the slurm script  is you just run 'pvbatch --use-offscreen-rendering --version')? Does  your PlotImage.py work at all?

If I run without giving any link as u said above, it gives this error:

-bash: pvbatch: command not found

Please let me know your idea.

Thank you

[alexeym]

Hi,

Obviously this

-bash: pvbatch: command not found

Means that pvbatch is not in the PATH. I have omitted the path to the executable (/home1/share/paraview/4.0.1/gnu/openmpi/bin/) and written ... instead. Yet you need to put this path into your slurm scripts.

This part of your script conflicts with --use-offscreen-rendering flag you pass to pvbatch.

#----OffscreenRendering off 
#---- When you use "pvserver --use-offscreen-rendering" "with osmesa", set both of them 1 
RenderView1.UseOffscreenRendering=0 
RenderView1.UseOffscreenRenderingForScreenshots=0

The rest of the script... well, did you try it on your desktop? It looks like Paraview trace file, yet path to the file (../Column/Column.foam) and time (1000), are you sure all these are available?

[shipman]

Hi Alexey,

thank you for reply.

The rest of the script... well, did you try it on your desktop? It looks like Paraview trace file, yet path to the file (../Column/Column.foam) and time (1000), are you sure all these are available?

Yes, I tried it on my desktop pc and it is working. Also, all these files are available inside of Column file.

Regarding to your following remarks,

This part of your script conflicts with --use-offscreen-rendering flag you pass to pvbatch.

#----OffscreenRendering off 
#---- When you use "pvserver --use-offscreen-rendering" "with osmesa", set both of them 1 
RenderView1.UseOffscreenRendering=0 
RenderView1.UseOffscreenRenderingForScreenshots=0

I changed the these value to 1

RenderView1.UseOffscreenRendering=1
RenderView1.UseOffscreenRenderingForScreenshots=1

and I got following error which is different than previous one.

librdmacm: Fatal: no RDMA devices found
--------------------------------------------------------------------------
[[34111,1],0]: A high-performance Open MPI point-to-point messaging module
was unable to find any relevant network interfaces:

Module: OpenFabrics (openib)
  Host: g001

Another transport will be used instead, although this may result in
lower performance.
--------------------------------------------------------------------------
ERROR: In /root/paraview/gnu/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 531
vtkXOpenGLRenderWindow (0x21b74d0): bad X server connection. DISPLAY=ERROR: In /root/paraview/gnu/VTK/Rendering/OpenGL/vtkXOpenGLRenderWindow.cxx, line 322
vtkXOpenGLRenderWindow (0x21b74d0): bad X server connection. DISPLAY=/tmp/slurmd/job529292/slurm_script: line 18: 31332 Segmentation fault    /home1/share/paraview/4.0.1/gnu/openmpi/bin/pvbatch --use-offscreen-rendering PlotImage.py

Do you have any idea?

Thank you.

[alexeym]

Hi,

Google is quite useful thing. https://www.google.fr/search?q=pvbat...reen+rendering

And the first link is...

http://www.cfd-online.com/Forums/par...rendering.html