[the new one]

Hi
I am currently trying to create a plugin for a solver in poitwise . but i am having trouble writing runtimeWrite.c file in order for it to write the grid file in the format i want (netcdf) . Has anybody got any experience in this area and give me guidence or if you have a sample file to write grid in a different file format . Your help will be appreciated .
Thanks

[cnsidero]

In case you were not aware, in the plugin SDK there are two example plugins - unstructured and structured. I've only written unstructured plugins and I used the unstructured example as a good starting point. There is another document that gives a good overview the SDK here http://www.pointwise.com/plugins/CAEPluginSDK-101.pdf but it looks like the link is broken right now.

Some one from Pointwise - hopefully a dev - will have to give you more details.

Originally Posted by the new one

Hi
I am currently trying to create a plugin for a solver in poitwise . but i am having trouble writing runtimeWrite.c file in order for it to write the grid file in the format i want (netcdf) . Has anybody got any experience in this area and give me guidence or if you have a sample file to write grid in a different file format . Your help will be appreciated .
Thanks

[dgarlisch]

New One,

First of all, did you download the .zip version of the SDK? If so, I am sorry to say it is BAD! It has the incorrect contents. If you download the .tar.gz version of the SDK, it is correct. Please use the .tar.gz distro.

Correction:
The SDK download links are now fixed!

I would be more than happy to help. However, I would need more details on the difficulties you are experiencing.

On which platform are you building: win32, win64, macOSX, linux32, linux64?

I assume you have already followed the steps outlined in the Building a CAE Plugin section and verified your SDK installation?

As Chris suggested, a good place to start is with the example XML plugins included with the SDK. They are located in the folders:

• .../PluginSDK/src/plugins/CaeStrXML/
• .../PluginSDK/src/plugins/CaeUnsXML/

Correction: The link to CAEPluginSDK-101.pdf is now fixed!

Understanding the Grid Model used in the CAE Plugin SDK is very important. See the SDK docs Modules page, the CAEPluginSDK-101.pdf file and my posts at the Pointwise blog for details.

enough for now.

[the new one]

Thanks guys for the help .This document is quite helpful and would have saved me alot of time if i had found it before been using the website and user manual till now and i am not a programmer by trade so getting use to handles and other stuff.But till now i have made changes in the header files and am able to write some of my grid data in ASCII file but my problem is writng it in netcdf format . and yeah i am using Linux64 platform. Your suggestions are welcome and will have to ask you guys for help if i run into further road blocks.
Thanks

[dgarlisch]

Glad to hear that the ASCII version of your plugin is functional.

You said you are building for linux_x86_64. I will focus my comments below to building on this platform. However, this can be extrapolated to the other platforms if you have the appropriate netcdf link libs.

You said you were not a programmer by trade. However, I will assume you have some experience with make files and the "advanced" topic of linking in third party libraries like netcdf.

In the text below, I use the following conventions/assumptions:

• …/PluginSDK refers to the full path to your PluginSDK installation
• Your plugin is named CaeXxxPlugin

  $ mkplugin –uns CaeXxxPlugin

• You have set machine to the appropriate value in your shell environment

  $ setenv machine linux_x86_64

First you will need one of the linux_x86_64 libnetcdf.a (static) or libnetcdf.so (dynamic) lib files. To avoid runtime compatibility issues, I suggest linking against the static libnetcdf.a lib. The final plugin file size will be larger with static linking, but will be easier to distribute and install.

You will also need the appropriate netcdf header files. Your plugin will need to include these files.

To make building your plugin on multiple platforms simpler, I suggest you create a platform specific folder structure within your SDK installation.

Copy the linux_x86_64 libnetcdf.a and/or libnetcdf .so files to this folder:

$ cd …/PluginSDK
$ mkdir –p ./external/linux_x86_64/netcdf

Copy the netdcf header files to this folder:

$ cd …/PluginSDK
$ mkdir –p ./external/common/netcdf

Rename your plugin's local makefile from modulelocal-sample.mk to modulelocal.mk:

$ cd …/PluginSDK/src/plugins/CaeXxxPlugin
$ mv modulelocal-sample.mk modulelocal.mk

Edit modulelocal.mk and add the location of the netcdf include folders:

CaeXxxPlugin_INCL_PRIVATE := \
     -I./external/common/netcdf \
     $(NULL)

Add the netcdf link lib information:

CaeXxxPlugin_LIBS_PRIVATE := \
    -L./external/$(machine)/netcdf \
     -lnetcdf \
     $(NULL)

Save these changes.

You can verify the changes using this make command:

$ make print.machine
$ make print.CaeXxxPlugin_INCL_PRIVATE
$ make print.CaeXxxPlugin_LIBS_PRIVATE

You should see output similar to the following:

machine=linux_x86_64
CaeXxxPlugin_INCL_PRIVATE=-I./external/common/netcdf
CaeXxxPlugin_LIBS_PRIVATE=-L./external/linux_x86_64/netcdf -lnetcdf

Your plugin project should now be ready for building.

I will not be able to help you (much) with regard to using the netcdf API. Please refer to their documentation.

If you have more specific questions about the SDK or grid model, don't hesitate to ask.

[adkay]

Hey,

I didn't look in the forums a few days, so I missed this thread...

Maybe you recognized my post a few weeks ago, where I asked how to write binary.

Well, I was also working on a plugin creating netCDF output as input for the TAU solver.

I also tried to use the netCDF libraries but failed to include them and also was not sure how to make them work properly together with Pointwise regarding things like output file creation/handling.
As I found the classic netCDF format not too difficult I used standard c functions to write the binary file.
It works quite fine now, although it is surely not the nicest way to create the output file....

Have you managed to use the netCDF libraries?

[dgarlisch]

adkay and the new one,

As you both have discovered, working with 3rd party libraries in a multi-platform environment can be frustrating/difficult task.

Writing the binary files directly (as adkay did) can be simpler sometimes. However, if the file format changes in the future, your plugin may need significant rework! Third party APIs like netcdf will "hide" these changes internally and make updates as simple as downloading/building new netcdf libs and then recompiling and linking the plugin.

[the new one]

Hey guys,
think should look at this post more .have not got far as was busy with other stuff.

dgarlisch thanks for the help .

adkay looks like you already did what i am supposed to do as i am also trying to make a pointwise plugin for the same solver as you . what we have now decided is to link the solver libraries and header files to the poitnwise plugin (havent accomplished that yet) and use the same functions defined in the solver to write the grid in netcdf format by copying grid data to from plugin data structures to solver data structures.what are your suggestions about going about that way as you probably know both the solver and plugin structures and have experience with both . Or would just save me alot of time to try and write binary files the way you did as it seems to be working for you. thanks

[adkay]

isn't tau using the netCDF libraries for file handling?

I'm also not very deep into that topic. Started working on my thesis only a few weeks ago and had never before to do something with tau or pointwise

I think if your approach to create the plugin is serious you should try to use the official libraries for the reasons dgarlisch mentioned.

When using standard c I had to come around with some workarounds for the big/little endian issue. And there are still some limitations so our plugin is not able to output "2D" grids (not coded). Also I think the performance is not very good, compared to the official cgns plugin.
As the main topic of my thesis is not the plugin, I'm quite fine with that

If you can live with that I can give you some code extracts to give you an idea if you want...

[the new one]

Yes it uses netCDF for file hadling but there are alreay functions written in the solver to write the grid in that format our approach is to utilize those function .As is said i am not that far into it .
And it would be really nice of you if you could give me some code extracts maybe it will help me understand the grid structure better and make my job easier and as i said i am not a programmer still getting use to it so might learn some new things from it .

[dgarlisch]

The information in Post #3 on May 29, 2013, 11:47 is a good place to start. Especially the blog post links. They cover in painful detail how the Pointwise plugin grid model is structured.

Then, looking at the sample XML plugins included in the SDK show how to use the grid model API to access the data.

[adkay]

First I made some definitions for the binary output:
Here you can find how the netCDF format has to look like: http://www.unidata.ucar.edu/software...rmat_spec.html

//++++++++++++++Definitionen fuer Binaerausgabe++++++++++++++++++

bool setASCII=false; //ASCII ausgeben true/false

int64_t offset=0;	//n Byte bis Daten-Block; int stat int64_t fuer netCDF classic
int64_t offsetval=0;
int ndim=0; //n Dimensios
int natt=0; //n Attributes (global)
int nvar=0; //n Variables

struct str_dims {
    char const *dimname;
    int value;
};
struct str_atts{
    char attname[10];
    char value[99];
};
struct str_vars{
    char const *varname; // string name
    int type; //3 short, 4 int, 6 double
    int dimen; //dimensions: 1 list, 2 matrix....
    int dimenids[2]; //of which dims is the variable dependent
    int cvals; //total count of values
};
//end def bin+++++++++++++++++++++++++++++++++++++++++++++++++++++

Then you need to count all the different elements per block (3D) and domain (2D).

For example using something like this:

static int
countTetsPerBlock(CAEP_RTITEM *pRti, PWGM_HBLOCK hBlk)
{
    int TetBlkCnt = 0;	//Tetraederanzahl
    if (pRti && PWGM_HBLOCK_ISVALID(hBlk)) {
        PWGM_ELEMDATA   eData ;
	PWGM_ELEMCOUNTS eCounts ;
        PWP_UINT32      eCnt = PwBlkElementCount(hBlk, &eCounts);
        TetBlkCnt=PWGM_ECNT_Tet(eCounts);
    }
    return TetBlkCnt;
}

static int
countTets(CAEP_RTITEM *pRti, PWGM_HGRIDMODEL model) 
{
    PWP_UINT32 tTotCnt = 0;
    if ( pRti && model ) {
        PWGM_ELEMCOUNTS eCounts ;
        PWP_UINT32      iBlk     = 0;	
        PWGM_HBLOCK    hBlk    = PwModEnumBlocks(model, iBlk);	
        while ( PWGM_HBLOCK_ISVALID(hBlk) ) {			/
            tTotCnt += countTetsPerBlock(pRti,hBlk);   
            hBlk = PwModEnumBlocks(model, ++iBlk);      
        }
    }
    caeuProgressIncr(pRti);
    return tTotCnt;
}

You need all these counts to calculate the record length of the data blocks in the binary file, or for telling netCDF converters if you write in cdl format.

As you have to write the marker info in the header of the file you also need a function to do that, for example something like that:

const void
writeMarkerName(CAEP_RTITEM *pRti, PWGM_CONDDATA *pCondData, int id, int flag, PWP_UINT32 MarkersTotCnt, struct str_atts atts[])
{
    if (pRti && pCondData) {
        if (flag==0){ //Abfrage ob Name (0), CentaurKey (1) oder Boundary-Mapping-File (2) schreiben
            if(setASCII){//ASCII
                fprintf(pRti->fp, "\t\t:marker_%i = \"%s\" ;\n", id+1, pCondData->name);// Schreibt den Namen (nicht typ) der Randbedingung
            }else{//BINARY
                //In Atts array schreiben
                if(id==0){
                    sprintf(atts[id].attname,"type");
                    sprintf(atts[id+1].attname,"marker_%i",id+1);
                    sprintf(atts[id].value,"%s","title");
                    sprintf(atts[id+1].value,"%s",pCondData->name);
                }else{
                    sprintf(atts[id+1].attname,"marker_%i",id+1);
                    sprintf(atts[id+1].value,"%s",pCondData->name);
                }
            }
        }
......

Then you need a function to write you the actual header:

writeHeader(CAEP_RTITEM *pRti, PWGM_HGRIDMODEL model)
//Schreibt den Header des NetCDF-Files ('dimensions' und 'variables')
{
    //Varibeln fuer Binaerausgabe
    int intout;
    char const *cout;
    double dout;
    unsigned char byteout;
    //----
    PWGM_ELEMCOUNTS eCounts ;
    if(setASCII){//ASCII
        fputs("netcdf Plugin {\n", pRti->fp);
        fputs("dimensions:\n", pRti->fp);
        PWP_UINT32 eTotCnt = countElements(pRti,model);
        fprintf( pRti->fp, "\tno_of_elements = %i",eTotCnt );	//Ausgabe Gesamtelementanzahl
        fputs(" ;\n", pRti->fp);
        PWP_UINT32 TetTotCnt = countTets(pRti,model);
        if (TetTotCnt!=0) {	//ueberprueft, ob Tetraeder vorhanden sind
            fprintf( pRti->fp, "\tno_of_tetraeders = %i",TetTotCnt);	//Ausgabe Tetraederanzahl
            fputs(" ;\n", pRti->fp);
            fprintf( pRti->fp, "\tpoints_per_tetraeder = %i",4);	//Ausgabe Knoten pro Tetraeder
            fputs(" ;\n", pRti->fp);
        }
......
}else{//Binary
        //++++++++++++++++++++++DEFINITION DIMS++++++++++++++++++++++++++++++++++++
        ndim=ndim+4;
        PWP_UINT32 TetTotCnt = countTets(pRti,model);
        if (TetTotCnt!=0){ndim=ndim+2;};
        PWP_UINT32 PrismTotCnt = countPrism(pRti,model);
        if (PrismTotCnt!=0){ndim=ndim+2;};
        PWP_UINT32 PyramidTotCnt = countPyramids(pRti,model);
        if (PyramidTotCnt!=0){ndim=ndim+2;};
        PWP_UINT32 HexTotCnt = countHex(pRti,model);
        if (HexTotCnt!=0){ndim=ndim+2;};
        PWP_UINT32 TriTotCnt = countTris(pRti,model);
        if (TriTotCnt!=0){ndim=ndim+2;};
        PWP_UINT32 QuadTotCnt = countQuads(pRti,model);
        if (QuadTotCnt!=0){ndim=ndim+2;};

        int dimindex=0;
        struct str_dims dims[ndim];
        PWP_UINT32 eTotCnt = countElements(pRti,model);
        dims[dimindex].dimname="no_of_elements"; dims[dimindex].value=eTotCnt;	//Ausgabe Gesamtelementanzahl
        dimindex++;
        if (TetTotCnt!=0) {	//ueberprueft, ob Tetraeder vorhanden sind
            dims[dimindex].dimname="no_of_tetraeders"; dims[dimindex].value=TetTotCnt; //Ausgabe Tetraederanzahl
            dimindex++;
            dims[dimindex].dimname="points_per_tetraeder"; dims[dimindex].value=4; //Ausgabe Knoten pro Tetraeder
            dimindex++;
        }
........
//++++++++++++++++++++++++DEFINITION VARS++++++++++++++++++++++++++++++
        nvar=6+(ndim-4)/2;
        int varindex=0; dimindex=0;
        struct str_vars vars[nvar];
        if (TetTotCnt!=0) {  //ueberprueft, ob Tetraeder vorhanden sind
            vars[varindex].varname="points_of_tetraeders"; vars[varindex].type=4; vars[varindex].dimen=2;
            vars[varindex].dimenids[0]=dimindex+1; vars[varindex].dimenids[1]=dimindex+2;
            varindex++; dimindex=dimindex+2;
            //int points_of_tetraeders(no_of_tetraeders, points_per_tetraeder)
        }
......
//++++++++++++++++++++++++++++WRITE BINARY HEADER++++++++++++++++++++++++++++++
        //header magic##########################################################
        cout="CDF";
        fwrite (cout, 3, 1, pRti->fp);
        byteout=2; //NETCDF Version: 1 fuer classsic, 2 fuer 64bit offset format
        fwrite (&byteout, 1, 1, pRti->fp);
        //record
        intout=0;
        fwrite_BEint(intout, sizeof(int), 1, pRti->fp);
        offset=offset+8; // 8byte
        //DIMENSIONS############################################################
        //tag for list of dimensions
        byteout=0;
        fwrite (&byteout, 1, 1, pRti->fp);
        fwrite (&byteout, 1, 1, pRti->fp);
        fwrite (&byteout, 1, 1, pRti->fp);
        byteout=10; //0A
        fwrite (&byteout, 1, 1, pRti->fp);
        offset=offset+4; // 4byte
        //Anzahl dimensions
        fwrite_BEint(ndim, sizeof(int), 1, pRti->fp); //Anzahl
        offset=offset+4;
        //dimension list
        for(int i=0; i<ndim; i++){
            //Laenge des namens i
            intout=strlen(dims[i].dimname);
            fwrite_BEint(intout, sizeof(int), 1, pRti->fp);
            offset=offset+4;
            //Name i und filler->immer volle 4byte blöcke
            fwrite(dims[i].dimname, strlen(dims[i].dimname), 1, pRti->fp);
            offset=offset+strlen(dims[i].dimname);
            int filler=4-strlen(dims[i].dimname)%4;
            if(filler<4){
                byteout=0;
                for(int j=0; j<filler; j++){
                    fwrite(&byteout, 1, 1, pRti->fp);
                    offset=offset+1;
                }
            }
            //wert dimension i
            fwrite_BEint(dims[i].value, sizeof(int), 1, pRti->fp);
            offset=offset+4;
        }
........

To write out Big Endian numbers I use something like that:

void fwrite_BEint64(int64_t uui, int size, int count, FILE * stream){
    if(is_big_endian()==false){
        int64_t a;
        unsigned char *dst = (unsigned char *)&a;
        unsigned char *src = (unsigned char *)&uui;
        dst[0] = src[7];
        dst[1] = src[6];
        dst[2] = src[5];
        dst[3] = src[4];
        dst[4] = src[3];
        dst[5] = src[2];
        dst[6] = src[1];
        dst[7] = src[0];
        fwrite(&a, size, count, stream);
    }else{
    	fwrite(&uui, size, count, stream);
    }
}

Then you need to write the actual data, per block, domain and/or element.

Difference here is, that you have to check when you have to put a ";" instead of a "," when you write ASCII. Not so for binary of course....
Example for a function that writes the nodes per Tet element:

static void
writeTetData(CAEP_RTITEM *pRti, PWGM_ELEMDATA *pElemData, PWP_UINT32 *eTotCnt)
//schreibt Knotennummern des Elements, auf das *pElemData zeigt
{
    if ( pRti && pElemData ) {		//ueberprueft ob gueltige Daten vorhanden
        PWP_UINT32 ii;	//Knotenzahler
	if (pElemData->type == PWGM_ELEMTYPE_TET){	//ueberprueft, ob Element Tetraeder ist
            for (ii=0; ii < pElemData->vertCnt; ++ii) {	//Schleife ueber alle Knoten des Elements
		if(setASCII){//ASCII
                    fprintf( pRti->fp, " %4lu,",
                             (unsigned long)pElemData->index[ii] );  //Ausgabe der Nummer des Knotens mit dem Zahler ii und nachfolgendem Komma
		}else{//BINARY
                    fwrite_BEint((unsigned long)pElemData->index[ii], sizeof(int), 1, pRti->fp);
		}
            }
            if(setASCII){fputs("\n", pRti->fp);}
	}
	caeuProgressIncr(pRti);  
    }
}

For the vertices I think there is a good example in the xml plugin...

All these functions are called (if necessary) from the runtimeWrite function or subfunctions.

.....
int fullsteps=1;
                if(caeuProgressBeginStep(pRti, 7)){
                    writeHeader (pRti, model);	//Schreibt den Header des NetCDF-Files ('dimensions' und 'variables')
                    if(setASCII){
                        fputs("data:", pRti->fp);
                        fputs("\n", pRti->fp);
                        fputs("\n", pRti->fp);
                    }

                    TetTotCnt = countTets(pRti,model);
                    PrismTotCnt = countPrism(pRti,model);
                    PyramidTotCnt = countPyramids(pRti,model);
                    HexTotCnt = countHex(pRti,model);
                    TriTotCnt = countTris(pRti,model);
                    QuadTotCnt = countQuads(pRti,model);
                    caeuProgressEndStep(pRti);
		}
		fullsteps=TetTotCnt+PrismTotCnt+PyramidTotCnt+HexTotCnt+TriTotCnt+QuadTotCnt;
		if(caeuProgressBeginStep(pRti, fullsteps)){
                    if (TetTotCnt!=0) { 	//ueberprueft, ob Tetraeder vorhanden sind
                        writeTets(pRti, model);		//schreibt 'points_of_tetraeders'
                    }
                    if (PrismTotCnt!=0) { 	//ueberprueft, ob Prismen vorhanden sind
                        writePrisms(pRti, model);	//schreibt 'points_of_prisms'
                    }
......

thats it in general I think

[dgarlisch]

I know this is an old post.

However, for anyone who stumbles across this looking for Plugin and TAU information, the full code for the TAU plugin shipped with Pointwise is available on github (and plugins too!).

https://github.com/pointwise/CaeUnsTAU

[dgarlisch]

The full code for the TAU plugin shipped with Pointwise is available on github (and plugins too!).

https://github.com/pointwise/CaeUnsTAU