[jabeken]

Hi all,

I'm running simulations on a cluster in batch-mode and am trying to write the data that is normally shown in the output-window into a text-file.

If a simulation aborts I currently get only an error-file from the cluster with the final error-output from Star. Sometimes this is enough for debugging, sometimes it isn't. From my old job, I'm used to getting the whole output written continuously in a log-file and would like to implement this in my new job as well (also as it is very useful to check into the current proceedings of a running simulation).

Any input or suggestion is appreciated!

Thanks and regards
Jonas

[LuckyTran]

This is more a question to ask your cluster admin.



But at the very least, you can tell us what is your cluster environment? How do you submit jobs to the cluster?

[jabeken]

Originally Posted by LuckyTran

This is more a question to ask your cluster admin.



But at the very least, you can tell us what is your cluster environment? How do you submit jobs to the cluster?

Thanks for the advice, I'm already trying to reach the cluster support, however without any success so far.
It's an IBM cluster, I submit the jobs via the "bsub"-command. I tried the "-o" output option but with that the logfile is only created after the whole job is finished.
I was hoping for an in-Star option to write out the output continuously (which is why I posted here) but am of course open to other solutions.

[jabeken]

So, I found a possible solution through the command "bpeek" (for IBM clusters).

For future references here is my current solution:

bpeek - f job_ID > logfile.log &

writes the output of the job continuously to logfile.log.

[bluebase]

Hi Jonas,


redirecting the standard output into a file is a very basic task in most operating systems.

Assuming you are not running a windows cluster (which would be wasteful for running starccm), your submission script for your cluster should contain a line such as:

starccm+ -batch ..... -np .... -power ...

In UNIX-like systems, such as Linux, you can redirect the output of the program starccm+ (or any other program) by using the ">" operator.
At the end of the command add "> arbitraryOutputfile.txt" so that have a command line, such as:

starccm+ -batch run -np .... -power ... > arbitraryOutputfile.txt

Now, when you also add the string "2>&1", the error messages will also be added to the standard output. "2" is the channel for error messages, which is redirected (">") to be append to the standard output channel ("&1") which is labeled as "1".

starccm+ -batch run -np .... -power ... > arbitraryOutputfile.txt 2>&1

That way all starccm+ related output will be stored in the arbitraryOutputfile.txt.


Such redirection even works in bat scripts of windows systems.


The program bpeek is probably specific program for your cluster software. Redirection to other channels, such as files, works in allmost all cases.



You might do some tutorials on linux environments to understand your cluster system better. Redirection is one of the most basic concepts to be learned.








Best regards,
Sebastian

[jabeken]

Hi Sebastian,

thanks for your input.

I am aware of the general Linux redirection of output via ">".

The issue I faced was simply that I don't execute the starccm+ directly.
So instead of

 starccm+ -batch ...

I run the bsub command of the IBM cluster (which submits a job, reserves ressources etc) and hand over the starccm execution to it:

bsub -n ... starccm+ -power ... -batch batch.java simfile.sim

When I now redirect the output, I simply receive the output of the bsub-command which only states the starting of the job under a certain ID and gives no continuous output during the run.

As mentioned the bpeek command allows me to circumvent that by actually returning the output of starccm+ instead of bsub.

[bluebase]

Although i am not familiar with the LSF job manager, i am sure, that there is a way to tell bsub where to write an output file. You may have look into the manual of bsub.

The first info on google yields:

bsub -o myjob.out -e myjob.err  ./myjob.sh

myjob.out should contain the output of the myjob.sh (starccm in your case)


Edit:

Reading the manual, it revealed, that by design the LSF manager does not continously write out an output; depending on the file system design, other systems do.

So, i guess, using bpeek is the simplest solution for you.

Another solution would be packing the full starccm command into a shell script which would contain the output redirection as described in my first post. I assumed you would do so - because of it's convenience.