[David Christopher]

The following Define_On_Demand UDF runs and prints out all the correct data, but then causes the parallel MPI processes to crash, so I have to restart Fluent after every time I use it. Any suggestions?

Dave

[David Christopher]

The UDF is:

/* UDF for printing mass fraction along axis to file */
/************************************************** */
#include "udf.h"

#if RP_NODE
real j=0;
int k;
int itera=0, size;
int nelem=0;
real axisposition[1000];
real sum_mass[1000];
#endif

DEFINE_ADJUST(centerlinehydro, d)
{
#if RP_NODE
cell_t c0;
int i=0;
Thread *t0=Lookup_Thread(d,2); /* refers to the flow region ID in Fluent Inamb in Wedge3*/
Thread *t=Lookup_Thread(d, 5); /* refers to the axis ID in Fluent 1 in Wedge3 and 18 in Wedge4*/

real FC[2]; /* This will hold the position vector */
face_t f;
j+=1;

begin_f_loop(f, t) /* loops over cells in a cell thread */
{
F_CENTROID(FC,f,t);
c0 = F_C0(f,t);
C_CENTROID(FC,c0,t0);
axisposition[i]=FC[0];
sum_mass[i]+=C_YI(c0,t0,0);
i+=1;
}
end_f_loop(f, t)
itera=j;
nelem=i;
Message(" CPU %i, number of elements %i for iteration number %i \n",myid,nelem,itera);
#endif
}

DEFINE_ON_DEMAND(CenterPrint)
{
real *array; /* data passing variables */
int pe;

#if RP_HOST
FILE *fout;
real average_mass[1000];
int nelem, i, size;
char filename[]="LXF_30MPa_center_ave.out";
fout = fopen(filename, "w");
Message("\n Opened %s \n \n",filename);
#endif

#if RP_NODE
int i;
/* Each Node loads up its data passing array */
size=2*nelem;
array = (real *)malloc(size * sizeof(int));
for (k=0;k<nelem;k++)
{
i=2*k;
array[i]= axisposition[k];
array[i+1]= sum_mass[k]/itera;
}
/* Set pe to destination node */
/* If on node_0 send data to host */
/* Else send to node_0 because */
/* compute nodes connect to node_0 & node_0 to host */
pe = (I_AM_NODE_ZERO_P) ? node_host : node_zero; /*this command gives node_nost if true and node_zero if false */
PRF_CSEND_INT(pe, &size, 1, myid);
PRF_CSEND_REAL(pe, array, size, myid);
free(array); /* free array on nodes after data sent */
/* node_0 now collects data sent by the other compute nodes and immediately sends it to the host */
if (I_AM_NODE_ZERO_P)
compute_node_loop_not_zero (pe)
{
PRF_CRECV_INT(pe, &size, 1, pe);
array = (real *)malloc(size * sizeof(real));
PRF_CRECV_REAL(pe, array, size, pe);
PRF_CSEND_INT(node_host, &size, 1, myid);
PRF_CSEND_REAL(node_host, array, size, myid);
free(array);
}
#endif /* RP_NODE */
#if RP_HOST
Message(" in Host \n");
compute_node_loop (pe) /* only acts as a counter in this loop */
{
/* Receive data sent by each node and write it out to the file */
PRF_CRECV_INT(node_zero, &size, 1, node_zero);
array = (real *)malloc(size * sizeof(real));
PRF_CRECV_REAL(node_zero, array, size, node_zero);
if ( size > 0 )
{
for (i=0; i<size/2; i++)
fprintf(fout, "%g %g \n", array[2*i],array[2*i+1]);
free(array);
}
}
fclose(fout);
Message("Closed %s \n",filename);
#endif /* RP_HOST */
Message("End of routine: \n");
}

[`e`]

Which line does Fluent crash at (what is the final message of yours printed to the screen) and are there any error messages printed to the screen?

[David Christopher]

Everything runs fine, all the data is put into the file and the last message prints and then the MPI stops.

[Kokemoor]

If all of your data is making it to that file, this might not matter, but putting PRF_GSYNC(); between your blocks of host and node code could help. As it is, the nodes can get to the end and print "End of routine" before the host has finished writing the data, so that may be misleading.

[David Christopher]

I added the PRF_GSYNC(); at the end and just before the last IF_HOST section, but Fluent still crashed after finishing.

For some processors, my file size is zero since there are no such elements at that processor. What will happen with the file send and receive statements to transfer the file from each node to node zero and then to the host if the size is zero? Will this cause any problems and still end up with the correct results at the host?

[David Christopher]

I finally located my problem. My first allocation uses "int" rather than "real". Now the UDF works well.