[melanie]

Hello,

I would like to understand how works the time selection in foamToEnsight utility. In the beginning I thought that the new timeSelctor features were available, but finally it is not (no way to use the option -time 0.1:0.2 for instance). It seems to use the controlDict keywords startTime and endTime, at least that is what I understood from the source file, but by testing I am not convinced: in a case containing 4 time directories, changing startTime and/or endTime always results in the processing of all available time directories. Does anyone have a hint ?

Thanks!
melanie

[olesen]

Originally Posted by melanie

I would like to understand how works the time selection in foamToEnsight utility. In the beginning I thought that the new timeSelctor features were available, but finally it is not

You are correct that the new timeSelector feature is not used in foamToEnsight. It is used in foamZoneToEnsight, but that utility does not run in parallel (which I gather is what you need).

The main issue with adding the timeSelector to the existing foamToEnsight is that this utility always removes an existing "EnSight" directory at the start, always writes the geometry and writes a *.case file.
Thus if you wanted to convert things incrementally, for example,

    foamToEnsight -time time1:time2
    ... some time later
    foamToEnsight -time time3:time4

The second call to foamToEnsight would remove the EnSight directory and throw away your conversion results. Since the utility has be written like this, it also writes the geometry every time it is called. With some scripting tricks, you could work around that though.

In contrast, the foamZoneToEnsight utility is designed to allow incremental conversion. It not only leaves the ensight directory intact, but will attempt to reduce the number of times the geometry file is written (it uses a file cache with the foam->ensight geometry mapping). But doesn't work in parallel.

[melanie]

Thanks Mark.
Yes, I noticed that each call to foamToEnsight utility removes the existing EnSight directory. I downloaded your foamZonesToEnsight utility from an old thread, and compilation only produced one error: in OF-1.6 there is no addTimeOptionsNoConstant.H file. Did you compile it on OF-1.6?
Anyway, I will try to modify the existing foamToEnsight utility to keep the parallel advantage, and I will see if I am also able to get it writing in an existing directory without removing it.
melanie

[olesen]

Originally Posted by melanie

Thanks Mark.
I downloaded your foamZonesToEnsight utility from an old thread, and compilation only produced one error: in OF-1.6 there is no addTimeOptionsNoConstant.H file. Did you compile it on OF-1.6?

Sorry, that was a mistake on my part. Don't use the foamZonesToEnsight at all. It's now called foamToEnsightParts and is included as part of the standard OpenFOAM-1.6 and should compile fine.

Anyway, I will try to modify the existing foamToEnsight utility to keep the parallel advantage, and I will see if I am also able to get it writing in an existing directory without removing it.

Suppressing the rmDir is easy enough, but adding the timeSelector to foamToEnsight will take a bit of monkeying about, with the main annoyance being the contiguous numbering for the output files. This will make incremental use quite annoying. There should be some bits from foamToEnsightParts that would be useful for when you rewrite foamToEnsight though.

If you had time though, I think it would be better just to add a -reconstruct option to foamToEnsightParts (eg, by stealing code from reconstructPar). This would allow you to convert existing parallel results to ensight as a serial process (eg, on your workstation) without having to startup a separate parallel process.

[melanie]

I did several tries with foamToEnsightParts, and I found some issues. The first point is that this utility is unable to work on time data created with the writeFieldsOften functionObject, since the uniform/time file is not present in this case. Moreover, I need to have Ensight data written in a list, without time directories, of the form

caseName.time1.U
caseName.time1.p
caseName.time2.U
caseName.time2.p
...

This is a naming convention closer to that used by foamToEnsight utility actually. And indeed, as you mentioned, I would need to add a -reconstruct option to post-process my data.

In the end, I don't know what is the easier thing, modify one or the other utility... Regarding the -reconstruct option, are you aware of a utility that performs such a reconstruction as a pre-step ?

[olesen]

Originally Posted by melanie

I did several tries with foamToEnsightParts, and I found some issues. The first point is that this utility is unable to work on time data created with the writeFieldsOften functionObject, since the uniform/time file is not present in this case.

Where did the "writeFieldsOften functionObject" come from? To be compatible with other OpenFOAM components, it should write uniform/time too. You should file a bug report.

... I need to have Ensight data written in a list, without time directories, of the form

caseName.time1.U
...

This is a naming convention closer to that used by foamToEnsight utility actually.

It was a conscious decision on my part to break with the naming convention, but it is a matter of task I guess. For removing many files, I personally prefer something like this,

   rm -rf Ensight/data/00000*

to remove all the iterations below 1000, instead of what foamToEnsight would require for the same operation:

   rm -rf EnSight/rpm-1500.?.*
   rm -rf EnSight/rpm-1500.??.*
   rm -rf EnSight/rpm-1500.???.*

Also, by dropping the caseName, it makes it much easier to write a utility for creating an ensight.case based on the directory contents, and the file names 'U', 'rho', etc are consistent with the field names.

Regarding the -reconstruct option, are you aware of a utility that performs such a reconstruction as a pre-step ?

No, I haven't seen any other places where it is being used. Fortunately, the code in reconstructPar is mostly all packaged into classes anyhow. So it wouldn't take much to add an extra method or two.

I think that adding -reconstructPar to foamToEnsightParts would be the better solution, but adding an '-append' option and the timeSelector to foamToEnsight would probably probably be much faster to do.

[melanie]

The writeFieldsOften functionObject comes from the simpleFunctionObjects library described on the Wiki here: http://openfoamwiki.net/index.php/Co...vailable_types. I am not sure whether I should submit a bug report since this is not included in the release version, but I am sure Bernhard can hear us !

Actually regarding the Ensight file structure, you are right and now I understand your choice, and this is a good choice also for what i'd like to do.

Finally I will have to understand the reconstructPar code and classes!

Thank you very much for all your advices Mark.
melanie

[gschaider]

Originally Posted by melanie

The writeFieldsOften functionObject comes from the simpleFunctionObjects library described on the Wiki here: http://openfoamwiki.net/index.php/Co...vailable_types. I am not sure whether I should submit a bug report since this is not included in the release version, but I am sure Bernhard can hear us !

Actually regarding the Ensight file structure, you are right and now I understand your choice, and this is a good choice also for what i'd like to do.

Finally I will have to understand the reconstructPar code and classes!

Thank you very much for all your advices Mark.
melanie

No I didn't hear you. But you pointed me here in the other thread http://www.cfd-online.com/Forums/ope...tml#post230782

The writeFieldsOften was just intended as a quick hack to .... äh .... write only certain fields often. I therefore never bothered to write the uniform-directory. And AFAIKS this is written in the Time::writeObject-method ... only if Time thinks it is the right time to write it .... something that the functionObject bypasses. So the best guess would be to emulate the uniform-writing stuff in the FO (but ONLY if Time doesn't think that it is write-time)

I think we'll move that discussion to the other thread (it's more appropriate there)

[melanie]

Actually I am answering in this thread because I found a solution in the options of foamToEnsightParts utility. Indeed it is possible to supply a -index option that make the utility ''ignore the time index contained in the time file and use a simple indexing when creating the Ensight files''. This is a very smart feature since it allows to read time directories that do not contain uniform/time file! So I do not need to change the behavior of the functionObject.

I will just have to concentrate on adding a reconstruct option to foamToEnsightParts. Not a tiny thing for me ... !

melanie

[melanie]

I have done some modifications in foamToEnsightParts.C to treat decomposed data sets. By adding a condition linked to the option -reconstruct

if (args.optionFound("reconstruct"))
{
    // Perform reconstruction
}

I can enter the code specific to decomposed cases. This is freely taken from reconstructPar.C code. Before knowing if it is working, I have to tell the code to use some files present in reconstructPar folder (processorMeshes.H and fvFieldReconstructor.H). Adding

#include "processorMeshes.H"
#include "fvFieldReconstructor.H"

in the preamble of foamToEnsightParts.C does not work. Could someone explain me how to include these files properly?

Thanks!
melanie

[olesen]

I'm glad to see you're making progress.

Originally Posted by melanie

#include "processorMeshes.H"
#include "fvFieldReconstructor.H"

in the preamble of foamToEnsightParts.C does not work. Could someone explain me how to include these files properly?

The 'proper' way would be to have these classes in their own library and reference the lnInclude directory, but as a quick (and slightly dirty) workaround just adding the corresponding path to your EXE_INC in Make/options should work:

EXE_INC = \
    -I$(FOAM_UTILITIES)/parallelProcessing/reconstructPar \
...

[melanie]

Ok, thanks; for the moment I keep the 'dirty' solution and the file linking is working. I was able to compile the utility without errors, the compiling log shows the following:

tzntgq@caeldh05:~/OpenFOAM/tzntgq-1.6/applications/utilities/postProcessing/dataConversion/foamToEnsightParts_mine> wclean
tzntgq@caeldh05:~/OpenFOAM/tzntgq-1.6/applications/utilities/postProcessing/dataConversion/foamToEnsightParts_mine> wmake libso
make: Warning: File `linux64GccDPOpt/options' has modification time 0.57 s in the future
make: warning:  Clock skew detected.  Your build may be incomplete.
make: Warning: File `Make/linux64GccDPOpt/dontIncludeDeps' has modification time 0.57 s in the future
make: warning:  Clock skew detected.  Your build may be incomplete.
make: Warning: File `Make/linux64GccDPOpt/dontIncludeDeps' has modification time 0.54 s in the future
Making dependency list for source file foamToEnsightParts_mine.C
make: warning:  Clock skew detected.  Your build may be incomplete.
make: Warning: File `foamToEnsightParts_mine.dep' has modification time 0.53 s in the future
SOURCE=foamToEnsightParts_mine.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/finiteVolume/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/lagrangian/basic/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/conversion/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/applications/utilities/parallelProcessing/reconstructPar -IlnInclude -I. -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/foamToEnsightParts_mine.o
'libNULL.so' is up to date.
make: warning:  Clock skew detected.  Your build may be incomplete.

While this seems to be ok the executable is not created... I guess this is a simple problem...

Another issue for me is the variables declaration. Actually I modified the existing foamToEnsightParts code to include the reconstruct option. This option is handled with several if loops like described in my previous message. So for instance in one loop I will assign the variable timeDirs, differently if the reconstruct option is chosen or not:

extern instantList timeDirs;

 if (args.optionFound("reconstruct"))
{
    // use the times list from the master processor
    // and select a subset based on the command-line options
    instantList timeDirs = timeSelector::select
    (
        databases[0].times(),
        args
    );
    if (timeDirs.empty())
    {
        FatalErrorIn(args.executable())
        << "No times selected"
        << exit(FatalError);
    }
    }
else
{
    instantList timeDirs = timeSelector::select0(runTime, args);
}  

As you can see, in order to have the timeDirs declared outside the if loop, I had to declare it outside the loop as an external variable. Is it right or not? Should I write only one general loop that does all the work with the reconstruct option, and in the else part all the work in serial mode?

[olesen]

Originally Posted by melanie

As you can see, in order to have the timeDirs declared outside the if loop, I had to declare it outside the loop as an external variable. Is it right or not? Should I write only one general loop that does all the work with the reconstruct option, and in the else part all the work in serial mode?

DO NOT use 'extern', that means something completely else from what you want (eg, http://wiki.answers.com/Q/What_is_th...of_extern_in_C)

I would probably do something like this:

 
PtrList<Time> databases(1);
 
const bool doReconstruct = args.optionFound("reconstruct");
 
label nProcs = 0;
 
if (doReconstruct)
{
... get nProcs, etc as per reconstructPar
...
 
    // Create the processor databases
    databases.setSize(nProcs);
    forAll(databases, procI)
    {
       databases.set
       (
           procI,
           new Time
           (
               Time::controlDictName,
               args.rootPath(),
               args.caseName()/fileName(word("processor") + name(procI))
            )
         );
   }
}
else
{
        databases.set
        (
            0,
            new Time
            (
                Time::controlDictName,
                args.rootPath(),
                args.caseName()
            )
        );
    }
 
}
 
 
// get times for normal or from processor0
 
instantList timeDirs = timeSelector::select
(
   databases[0].times(),
   args
);
 
...

You could later use either nProcs or doReconstruct to distinguish between normal and reconstruct, depending on your preference.

[melanie]

Originally Posted by olesen

DO NOT use 'extern', that means something completely else from what you want

Thanks Mark, I promise I will not do that again ...

I have corrected the code with your suggestions. Anyway, I cannot see the result since the executable is still not created... do you have an idea of what could be the problem, since compilation shows no warning? (except clock skew issues but this is not important for us).

[melanie]

Hello,
I am back with a referencing error; actually Mark's suggestion

Originally Posted by olesen

The 'proper' way would be to have these classes in their own library and reference the lnInclude directory, but as a quick (and slightly dirty) workaround just adding the corresponding path to your EXE_INC in Make/options should work:

EXE_INC = \
    -I$(FOAM_UTILITIES)/parallelProcessing/reconstructPar \
...

does not work... Could you please explain me how to build a library with all the classes required? I tried but without success! Thanks.
melanie

[melanie]

Today I have a good news: I managed to compile and have something working! This is a first important step for me !

Now I still have some troubles. The main issue is that for the reconstruct option to work on decomposed cases, I have to define procMeshes like this:

myProcessorMeshes procMeshes(databases, regionName);

for all cases, i.e. decomposed AND non-decomposed cases. But I do not know how to define this for non-decomposed cases (Should I try to write a derived class working for non-decomposed cases? In this case I would have to remove the reading of boundaryProcAddressing etc.)

I tried to follow the advices given by Mark. But it is complicated to handle... If this is unclear let me know and I post the relevant piece of code!

Thanks
melanie

[olesen]

Originally Posted by melanie

Today I have a good news: I managed to compile and have something working! This is a first important step for me !

Good.

The main issue is that for the reconstruct option to work on decomposed cases, I have to define procMeshes like this:

myProcessorMeshes procMeshes(databases, regionName);

for all cases, i.e. decomposed AND non-decomposed cases. But I do not know how to define this for non-decomposed cases

Why don't you handle it simply like this:

autoPtr<procMeshes> myProcMeshes;

if (WANT_DECOMPOSED_TEST) 
{
   myProcMeshes = new procMeshes(databases, regionName);
}

... later

if (myProcMeshes.valid())
{
   // decomposed case
}
else
{
  // standard logic
}

I'll not sure how the rest of your code looks like, but it might be convenient to write a dummy procMeshes class for handling the serial case.
If you use autoPtr (or any pointer) it should work fine.

/mark

[melanie]

Thanks Mark. I tried your proposition but this does not work; when I implement:

autoPtr<myProcessorMeshes> myProcessorMeshes;
    
if (doReconstruct)
{
    // Set all times on processor meshes equal to reconstructed mesh
    ...
        
    // Read all meshes and addressing to reconstructed mesh
    myProcessorMeshes = new procMeshes(databases, regionName);
}
// do other things
forAll(timeDirs, timeI)
{
    if (doReconstruct)
    {
        // use procMeshes to reconstruct mesh and field data
    }
}

it produces

tzntgq@caeldh05:~/OpenFOAM/tzntgq-1.6/applications/utilities/postProcessing/dataConversion/foamToEnsightParts_mine> wmake
make: Warning: File `Make/linux64GccDPOpt/dontIncludeDeps' has modification time 0.65 s in the future
Making dependency list for source file foamToEnsightParts_mine.C
make: warning:  Clock skew detected.  Your build may be incomplete.
make: Warning: File `foamToEnsightParts_mine.dep' has modification time 0.55 s in the future
SOURCE=foamToEnsightParts_mine.C ;  g++ -m64 -Dlinux64 -DWM_DP -Wall -Wno-strict-aliasing -Wextra -Wno-unused-parameter -Wold-style-cast -Wnon-virtual-dtor -O3  -DNoRepository -ftemplate-depth-40 -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/finiteVolume/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/lagrangian/basic/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/conversion/lnInclude -IlnInclude -I. -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude -I/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/OSspecific/POSIX/lnInclude   -fPIC -c $SOURCE -o Make/linux64GccDPOpt/foamToEnsightParts_mine.o
foamToEnsightParts_mine.C: In function ‘int main(int, char**)’:
foamToEnsightParts_mine.C:250: error: expected type-specifier before ‘procMeshes’
foamToEnsightParts_mine.C:250: error: no match for ‘operator=’ in ‘myProcessorMeshes = (int*)operator new(4u)’
/home/tzntgq/OpenFOAM/OpenFOAM-1.6/src/OpenFOAM/lnInclude/autoPtrI.H:187: note: candidates are: void Foam::autoPtr<T>::operator=(const Foam::autoPtr<T>&) [with T = Foam::myProcessorMeshes]
foamToEnsightParts_mine.C:250: error: expected `;' before ‘procMeshes’
foamToEnsightParts_mine.C:300: error: ‘procMeshes’ was not declared in this scope
In file included from foamToEnsightParts_mine.C:361:
doReconstruct.H:7: error: ‘procMeshes’ was not declared in this scope
make: *** [Make/linux64GccDPOpt/foamToEnsightParts_mine.o] Error 1

at compilation. My original implementation was

if (doReconstruct)
{
    // Set all times on processor meshes equal to reconstructed mesh
    ...
}
// Read all meshes and addressing to reconstructed mesh
myProcessorMeshes procMeshes(databases, regionName);

// do other things
...

forAll(timeDirs, timeI)
{
    if (doReconstruct)
    {
        // use procMeshes to reconstruct mesh and field data
    }
}

This last implementation compiles without error; it works fine for parallel cases, but it produces an error for serial cases as the boundaryProcAddressing etc. files are not present in the polyMesh folder. If I move the line of procMeshes definition in the first if (doReconstruct) loop, I have again an error at compilation, since in the following if (doReconstruct) loop, procMeshes is not defined...

The class is named myProcessorMeshes. I studied this class, but I am unable to write a dummy class to treat serial cases, which would probably be the best solution...

[olesen]

Originally Posted by melanie

Thanks Mark. I tried your proposition but this does not work; when I implement:

autoPtr<myProcessorMeshes> myProcessorMeshes;
 
if (doReconstruct)
{
    // Set all times on processor meshes equal to reconstructed mesh
    ...
 
    // Read all meshes and addressing to reconstructed mesh
    myProcessorMeshes = new procMeshes(databases, regionName);
}
// do other things
forAll(timeDirs, timeI)
{
    if (doReconstruct)
    {
        // use procMeshes to reconstruct mesh and field data
    }
}

Do you have a typo, or did you really try to use 'myProcessorMeshes' both for the class name and the variable name?
But in the reconstruct you have new procMeshes().
What is subclassed from what? I can't figure out what you are doing, but it looks like dangerous programming style.

[melanie]

Ok, I should have written

autoPtr<myProcessorMeshes> procMeshes;

The class is named myProcessorMeshes, the variable is procMeshes.
But still I do not understand the details of the implementation you suggested with the autoPtr, I did not find documentation about this. And I don't know either if there is a subclass in all this. And it still does not work ! Please excuse my mistakes that may seem trivial to you, I am just learning C++ and OF programming...
melanie