CFD Online Logo CFD Online URL
www.cfd-online.com
[Sponsors]
Home > Forums > Visualization & Post-Processing Software > ParaView

[General] Paraview not reading some fields in dataset

Register Blogs Community New Posts Updated Threads Search

Reply
 
LinkBack Thread Tools Search this Thread Display Modes
Old   January 30, 2018, 15:58
Default Paraview not reading some fields in dataset
  #1
Member
 
Tate
Join Date: Feb 2015
Posts: 31
Rep Power: 11
t8fanning is on a distinguished road
I've output a vtu file from a CFD code. When trying to load the dataset into Paraview I get the following errors:

Code:
Error reading uncompressed binary data header. Read 0 of 4 bytes. 
ERROR: In C:\bbd\7cc78367\source-paraview\VTK\IO\XMLParser\vtkXMLDataParser.cxx, line 609 vtkXMLDataParser (0000009CC7674520): Error reading uncompressed binary data header. Read 0 of 4 bytes.

Cannot read point data array "Cyl_Velocity" from PointData in piece 0. The data array in the element may be too short. ERROR: In C:\bbd\7cc78367\source-paraview\VTK\IO\XML\vtkXMLDataReader.cxx, line 443 vtkXMLUnstructuredGridReader (0000009C837CFCE0): Cannot read point data array "Cyl_Velocity" from PointData in piece 0. The data array in the element may be too short.
If I ask Paraview to not read the offending arrays, it reads the other arrays in fine. From these errors, it seems the data file has some issues with a few specific arrays.

Upon inspection of the first part of the vtu file, the offending arrays (Cyl_Velocity and Rel_Velocity) have negative offsets.

Code:
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="BigEndian">
	<UnstructuredGrid>
    <Piece NumberOfPoints="     9855039" NumberOfCells="     9523296">
      <Points>
        <DataArray type="Float64" Name="coordinates" NumberOfComponents="3" format="appended" offset="           0" />
      </Points>
      <Cells>
        <DataArray type="Int32" Name="connectivity" format="appended" offset="   236520940" />
        <DataArray type="Int32" Name="offsets" format="appended" offset="   541266416" />
        <DataArray type="Int32" Name="types" format="appended" offset="   579359604" />
      </Cells>
      <PointData>
         <DataArray type="Float64" Name="rho" format="appended" offset="   617452792" />
         <DataArray type="Float64" Name="P" format="appended" offset="   696293108" />
         <DataArray type="Float64" Name="Phi" format="appended" offset="   775133424" />
         <DataArray type="Float64" Name="TKE" format="appended" offset="   853973740" />
         <DataArray type="Float64" Name="EPS" format="appended" offset="   932814056" />
         <DataArray type="Float64" Name="mut/mul" format="appended" offset="  1011654372" />
         <DataArray type="Float64" Name="WDist" format="appended" offset="  1090494688" />
         <DataArray type="Float64" Name="TimeStep" format="appended" offset="  1169335004" />
         <DataArray type="Float64" Name="T" format="appended" offset="  1248175320" />
         <DataArray type="Float64" Name="Processor" format="appended" offset="  1327015636" />
         <DataArray type="Float64" Name="Soundspeed" format="appended" offset="  1405855952" />
         <DataArray type="Float64" Name="Lam.Visc" format="appended" offset="  1484696268" />
         <DataArray type="Float64" Name="Yplusw" format="appended" offset="  1563536584" />
         <DataArray type="Float64" Name="Qdotw" format="appended" offset="  1642376900" />
         <DataArray type="Float64" Name="vort-x" format="appended" offset="  1721217216" />
         <DataArray type="Float64" Name="vort-y" format="appended" offset="  1800057532" />
         <DataArray type="Float64" Name="vort-z" format="appended" offset="  1878897848" />
         <DataArray type="Float64" Name="helicity" format="appended" offset="  1957738164" />
         <DataArray type="Float64" Name="Velocity" NumberOfComponents="3" format="appended" offset="  2036578480" />
         <DataArray type="Float64" Name="Cyl_Velocity" NumberOfComponents="3" format="appended" offset=" -2021867876" />
         <DataArray type="Float64" Name="Rel_Velocity" NumberOfComponents="3" format="appended" offset=" -1785346936" />
      </PointData>
      <CellData>  </CellData>
The CFD code also can output data files for a specific surface. When I load one of those into Paraview, it has no issues with any data array in that file, and everything works fine.

Upon inspection of the first part of a single surface vtu file with no issues, no array has a negative offset.

Code:
 	<VTKFile type="UnstructuredGrid" version="0.1" byte_order="BigEndian">
	<UnstructuredGrid>
    <Piece NumberOfPoints="      176025" NumberOfCells="      174528">
      <Points>
        <DataArray type="Float64" Name="coordinates" NumberOfComponents="3" format="appended" offset="           0" />
      </Points>
      <Cells>
        <DataArray type="Int32" Name="connectivity" format="appended" offset="     4224604" />
        <DataArray type="Int32" Name="offsets" format="appended" offset="     7017056" />
        <DataArray type="Int32" Name="types" format="appended" offset="     7715172" />
      </Cells>
      <PointData>
         <DataArray type="Float64" Name="rho" format="appended" offset="     8413288" />
         <DataArray type="Float64" Name="P" format="appended" offset="     9821492" />
         <DataArray type="Float64" Name="Phi" format="appended" offset="    11229696" />
         <DataArray type="Float64" Name="TKE" format="appended" offset="    12637900" />
         <DataArray type="Float64" Name="EPS" format="appended" offset="    14046104" />
         <DataArray type="Float64" Name="mut/mul" format="appended" offset="    15454308" />
         <DataArray type="Float64" Name="WDist" format="appended" offset="    16862512" />
         <DataArray type="Float64" Name="TimeStep" format="appended" offset="    18270716" />
         <DataArray type="Float64" Name="T" format="appended" offset="    19678920" />
         <DataArray type="Float64" Name="Processor" format="appended" offset="    21087124" />
         <DataArray type="Float64" Name="Soundspeed" format="appended" offset="    22495328" />
         <DataArray type="Float64" Name="Lam.Visc" format="appended" offset="    23903532" />
         <DataArray type="Float64" Name="Yplusw" format="appended" offset="    25311736" />
         <DataArray type="Float64" Name="Qdotw" format="appended" offset="    26719940" />
         <DataArray type="Float64" Name="vort-x" format="appended" offset="    28128144" />
         <DataArray type="Float64" Name="vort-y" format="appended" offset="    29536348" />
         <DataArray type="Float64" Name="vort-z" format="appended" offset="    30944552" />
         <DataArray type="Float64" Name="helicity" format="appended" offset="    32352756" />
         <DataArray type="Float64" Name="Velocity" NumberOfComponents="3" format="appended" offset="    33760960" />
         <DataArray type="Float64" Name="Cyl_Velocity" NumberOfComponents="3" format="appended" offset="    37985564" />
         <DataArray type="Float64" Name="Rel_Velocity" NumberOfComponents="3" format="appended" offset="    42210168" />
      </PointData>
      <CellData>  </CellData>
So I think the offset sign is relevant to my problem, but I don't really know what it means or how to fix it. Does anyone know what I can do to fix my data set, or at least know what the underlying problem is?
t8fanning is offline   Reply With Quote

Old   January 30, 2018, 17:07
Default
  #2
New Member
 
Forge Analytics
Join Date: Jan 2018
Posts: 16
Rep Power: 8
forgeanalytics is on a distinguished road
I don't have complete confidence here, but hope this may help:

The data from each data array is appended in the AppendedData section, and the offset value indicates where the start of that data block is, relative to the beginning. If you take the difference between your offsets for the Float64 values, you'll find that they're all 78,840,316 from each other (78840312 + 4). For a 3 component dataset of Float64 values, you'd expect offsets of 236520940 relative to each other (236520936 + 4). Note that 236520936 / 3 = 78840312.

See the top of page 16 here: https://www.vtk.org/wp-content/uploa...le-formats.pdf
And this message board response at the bottom: https://public.kitware.com/pipermail...er/006066.html

The negative values may mean that the writer produced the appended blocks out-of-order (I'm not sure about this). I would guess that your offset values for Velocity, Cyl_Velocity and Rel_Velocity would be:

2036578480
2273099420
2509620360

but maybe not in that respective order.

I would try using those offset values (make a backup, of course) and see if the data loads. Then decide if the values are sensible for those labels, or if what's called Rel_Velocity looks like it may be Velocity or Cyl_Velocity, etc.
__________________
https://forgeanalytics.io/ | Data Management for Scientists and Engineers
forgeanalytics is offline   Reply With Quote

Old   January 30, 2018, 18:38
Default
  #3
Member
 
Tate
Join Date: Feb 2015
Posts: 31
Rep Power: 11
t8fanning is on a distinguished road
Thank you very much. Your solution worked perfectly. I'm happy to know it's essentially an indexing problem and the fix is simple. I was able to read the whole file into Paraview and perform whatever calculation I wanted.

Here's another data set for which I modified the offsets using your method. There are many more negative offsets than in the previous data set.

Code:
<VTKFile type="UnstructuredGrid" version="0.1" byte_order="BigEndian">
  <UnstructuredGrid>
    <Piece NumberOfPoints="    12661959" NumberOfCells="    12155616">
      <Points>
        <DataArray type="Float64" Name="coordinates" NumberOfComponents="3" format="appended" offset="           0" />
      </Points>
      <Cells>
        <DataArray type="Int32" Name="connectivity" format="appended" offset="   303887020" />
        <DataArray type="Int32" Name="offsets" format="appended" offset="   692866736" />
        <DataArray type="Int32" Name="types" format="appended" offset="   741489204" />
      </Cells>
      <PointData> 
         <DataArray type="Float64" Name="rho" format="appended" offset="   790111672" />
         <DataArray type="Float64" Name="P" format="appended" offset="   891407348" />
         <DataArray type="Float64" Name="Phi" format="appended" offset="   992703024" />
         <DataArray type="Float64" Name="TKE" format="appended" offset="  1093998700" />
         <DataArray type="Float64" Name="EPS" format="appended" offset="  1195294376" />
         <DataArray type="Float64" Name="mut/mul" format="appended" offset="  1296590052" />
         <DataArray type="Float64" Name="WDist" format="appended" offset="  1397885728" />
         <DataArray type="Float64" Name="TimeStep" format="appended" offset="  1499181404" />
         <DataArray type="Float64" Name="T" format="appended" offset="  1600477080" />
         <DataArray type="Float64" Name="Processor" format="appended" offset="  1701772756" />
         <DataArray type="Float64" Name="Soundspeed" format="appended" offset="  1803068432" />
         <DataArray type="Float64" Name="Lam.Visc" format="appended" offset="  1904364108" />
         <DataArray type="Float64" Name="Yplusw" format="appended" offset="  2005659784" />
         <DataArray type="Float64" Name="Qdotw" format="appended" offset="  2106955460" />
         <DataArray type="Float64" Name="vort-x" format="appended" offset=" -2086716160" />
         <DataArray type="Float64" Name="vort-y" format="appended" offset=" -1985420484" />
         <DataArray type="Float64" Name="vort-z" format="appended" offset=" -1884124808" />
         <DataArray type="Float64" Name="helicity" format="appended" offset=" -1782829132" />
         <DataArray type="Float64" Name="Velocity" NumberOfComponents="3" format="appended" offset=" -1681533456" />
         <DataArray type="Float64" Name="Cyl_Velocity" NumberOfComponents="3" format="appended" offset=" -1377646436" />
         <DataArray type="Float64" Name="Rel_Velocity" NumberOfComponents="3" format="appended" offset=" -1073759416" />
      </PointData>
      <CellData>  </CellData>
    </Piece>
  </UnstructuredGrid>
In this data, the bottom seven arrays have negative offsets and corresponding issues with Paraview. Using your method, I found the correct offsets to be:

2208251136
2309546812
2410842488
2512138164
2613433840
2917320860
3221207880

After reading the modified offsets file into Paraview, I calculated the average velocity on the inlet of my domain and found that it matched my input boundary condition. Luckily this data set had an inappropriate offset for velocity, and I was able to verify your method works. Thanks again.
t8fanning is offline   Reply With Quote

Reply

« Previous Thread | Next Thread »

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are On

Similar Threads
Thread Thread Starter Forum Replies Last Post
[General] Calling .VTK and .VTI fields in Paraview Xavi7 ParaView 0 April 29, 2015 06:32
[General] Reading Fluent gzip-compressed files to ParaView larsschwarzer ParaView 4 August 8, 2014 05:12
[OpenFOAM] Annoying issue of automatic "Rescale to Data Range " with paraFoam/paraview 3.12 keepfit ParaView 60 September 18, 2013 04:23
[Commercial meshers] fluentMeshToFoam multidomain mesh conversion problem Attesz OpenFOAM Meshing & Mesh Conversion 12 May 2, 2013 11:52
[OpenFOAM] Paraview does not display fields mirko ParaView 1 August 11, 2011 08:19


All times are GMT -4. The time now is 23:33.

Contact Us - CFD Online - Privacy Statement - Top