[mardar572]

Hi,

For a project with Academic Research License HPC

I'm trying to use Fluent 17.0 which is installed on my universites HPC (CentOS 6.10) with PBS Torque Job Scheduler.

I haven't succeeded yet.

I read in Fluent Documentation that LSF, SGE, and PBS Pro are examples of job scheduler on Linux.

I want to make sure that PBS Torque can be used for Fluent 17.0 on HPC with CentOS 6.10. Could you confirm it?

Also, if you can confirm this, I'd like to ask you why I can't run it with detailed information about my problem.

Best Regards

[alainislas]

Yes, you can run Fluent with PBS torque scheduler. You will need to use a script file, containing the #PBS instructions to request queues, nodes, processors, emailing, etc...


Let me know if you need help with this.

[mardar572]

Hi alainislas,

Actually, I need help and that would be great

When I try to schedule job with "qsub script_filename" from the terminal of HPC. 2 files are created in my home directory in these formats:

JobName.oJobID (this file is empty)
JobName.eJobID (this file has a message and says that:

/var/spool/torque/mom_priv/jobs/23454.poyraz.ae.metu.edu.tr.SC: line 34: ./opt/ansys_inc/v170/fluent/bin/fluent*: No such file or directory


script_filename is in the following format:

# Give your job a name so it can be found readily with qstat.
#PBS -N Your_Job

# Specify time limit of your job. (HH:MM:SS)
#PBS -l walltime=02:00:00

# Specify the number of threads/cores you need (MAX:64)
# number of nodes can only be 1 for shared memory openmp apps
#PBS -l nodes=1pn=64
export OMP_NUM_THREADS=64

# mail alert at the (e)nd of execution
#PBS -m e
#PBS -M your_email_address
# Standard error output merge to the standard output file.
#PBS -j oe
# Specify the file name for the standard output/error
#PBS -o output

# Enter your work directory under /cfd
cd /users/project/aozuzun

# execute your shared memory application
./opt/ansys_inc/v170/fluent/bin/fluent* 3ddp -t np -ssh -pib -g -i /users/project/aozuzun/aa.jou

[alainislas]

The output files you are getting, contain the terminal display before and after your job is executed. This is done by #PBS -j oe. If they are empty or showing an error message, this means fluent is not starting correctly. Please refer to the following link http://docs.adaptivecomputing.com/to...%23mailoptions

to setup your script file.

Another thing, you may need to change your working directory to the PBS directory, this is done by:
cd $PBS_O_WORKDIR


Is fluent installed as a module in the HPC? you can verify this by typing on the terminal:
module avail

if this is the case, load the module and then use the command

fluent -g 3ddp -ssh -tx -i journal.jou

-g (runs in background mode)
3ddp (3D simulation using double precision)
-ssh (Secure Shell turned on)
-tx (specify the number of processors, where x is the number)
-i (run the fluent with a journal file) Need to specify the console commands to run this.

Please check the fluent getting started guide and console commands.

Let me know if this is helpful

[mardar572]

Firstly, I check the fluent is installed as a module and it gives the message below. Is the module necessary?

[aozuzun@poyraz:~]$ module avail
------------------------ /usr/share/Modules/modulefiles ------------------------
dot module-cvs module-info modules null use.own
------------------------------- /etc/modulefiles -------------------------------
compat-openmpi-i386 openmpi-1.4-i386 openmpi-1.5.3-i386
compat-openmpi-x86_64 openmpi-1.4-x86_64 openmpi-1.5.3-x86_64
------------------------ /opt/share/Modules/modulefiles ------------------------

Secondly, I found a mistake in my script file and correct it by deleting the dot in last line of my script.The last version is:

/opt/ansys_inc/v170/fluent/bin/fluent* -g 3ddp -ssh -t64 -i journal.jou


Now, the files created in my home directories says different things. But as I understand at least pbs can start fluent anymore.

May I ask what is these errors?

The file named "Your_Job.e23465"

Error: eval: unbound variable
Error Object: /opt/ansys_inc/v170/fluent/bin/fluent_arch

Error: eval: unbound variable
Error Object: /opt/ansys_inc/v170/fluent/bin/fluent_arch1.0

Error: eval: unbound variable
Error Object: /opt/ansys_inc/v170/fluent/bin/fluentbench.pl

Error: eval: unbound variable
Error Object: /opt/ansys_inc/v170/fluent/bin/fluent-cleanup.pl

Error: eval: unbound variable
Error Object: /opt/ansys_inc/v170/fluent/bin/fluent_sysinfo
Cleanup script file is /users/project/aozuzun/cleanup-fluent-atmaca65-5648.sh



The file named "Your_Job.o23469"

/opt/ansys_inc/v170/fluent/fluent17.0.0/bin/fluent -r17.0.0 /opt/ansys_inc/v170/fluent/bin/fluent_arch /opt/ansys_inc/v170/fluent/bin/fluent_arch1.0 /opt/ansys_inc/v170/fluent/bin/fluentbench.pl /opt/ansys_inc/v170/fluent/bin/fluent-cleanup.pl /opt/ansys_inc/v170/fluent/bin/fluent_sysinfo -g 3ddp -ssh -t32 -i /users/project/aozuzun/journal.jou
/opt/ansys_inc/v170/fluent/fluent17.0.0/cortex/lnamd64/cortex.17.0.0 -f fluent /opt/ansys_inc/v170/fluent/bin/fluent_arch /opt/ansys_inc/v170/fluent/bin/fluent_arch1.0 /opt/ansys_inc/v170/fluent/bin/fluentbench.pl /opt/ansys_inc/v170/fluent/bin/fluent-cleanup.pl /opt/ansys_inc/v170/fluent/bin/fluent_sysinfo -g -i /users/project/aozuzun/journal.jou (fluent "3ddp -pshmem -host -r17.0.0 -t32 -mpi=pcmpi -path/opt/ansys_inc/v170/fluent -ssh")
/opt/ansys_inc/v170/fluent/fluent17.0.0/bin/fluent -r17.0.0 3ddp -pshmem -host -t32 -mpi=pcmpi -path/opt/ansys_inc/v170/fluent -ssh -cx atmaca65:46733:34485
Starting /opt/ansys_inc/v170/fluent/fluent17.0.0/lnamd64/3ddp_host/fluent.17.0.0 host -cx atmaca65:46733:34485 "(list (rpsetvar (QUOTE parallel/function) "fluent 3ddp -flux -node -r17.0.0 -t32 -pshmem -mpi=pcmpi -ssh") (rpsetvar (QUOTE parallel/rhost) "") (rpsetvar (QUOTE parallel/ruser) "") (rpsetvar (QUOTE parallel/nprocs_string) "32") (rpsetvar (QUOTE parallel/auto-spawn?) #t) (rpsetvar (QUOTE parallel/trace-level) 0) (rpsetvar (QUOTE parallel/remote-shell) 1) (rpsetvar (QUOTE parallel/path) "/opt/ansys_inc/v170/fluent") (rpsetvar (QUOTE parallel/hostsfile) "") )"

Welcome to ANSYS Fluent Release 17.0

Copyright 2015 ANSYS, Inc.. All Rights Reserved.
Unauthorized use, distribution or duplication is prohibited.
This product is subject to U.S. laws governing export and re-export.
For full Legal Notice, see documentation.

Build Time: Dec 02 2015 13:21:15 EST Build Id: 10359 Revision: 867598


--------------------------------------------------------------
This is an academic version of ANSYS FLUENT. Usage of this product
license is limited to the terms and conditions specified in your ANSYS
license form, additional terms section.
--------------------------------------------------------------
Host spawning Node 0 on machine "atmaca65" (unix).
/opt/ansys_inc/v170/fluent/fluent17.0.0/bin/fluent -r17.0.0 3ddp -flux -node -t32 -pshmem -mpi=pcmpi -ssh -mport 10.10.10.65:10.10.10.65:42858:0
Starting /opt/ansys_inc/v170/fluent/fluent17.0.0/multiport/mpi/lnamd64/pcmpi/bin/mpirun -e MPI_USE_MALLOPT_MMAP_MAX=0 -np 32 /opt/ansys_inc/v170/fluent/fluent17.0.0/lnamd64/3ddp_node/fluent_mpi.17.0.0 node -mpiw pcmpi -pic shmem -mport 10.10.10.65:10.10.10.65:42858:0

-------------------------------------------------------------------------------
ID Hostname Core O.S. PID Vendor
-------------------------------------------------------------------------------
n0-31 atmaca65 32/64 Linux-64 5819-5850 AMD Opteron(TM) 6276
host atmaca65 Linux-64 5648 AMD Opteron(TM) 6276

MPI Option Selected: pcmpi
Selected system interconnect: shared-memory
-------------------------------------------------------------------------------


Reading journal file /users/project/aozuzun/journal.jou...

> /file/read-case "/users/project/aozuzun/FFF.msh"
Multicore SMT processors detected. Processor affinity set!

Reading "/users/project/aozuzun/FFF.msh"...

Buffering for file scan...

9870 nodes, binary.
2047 nodes, binary.
127893 triangular interior faces, zone 1, binary.
1052 triangular velocity-inlet faces, zone 5, binary.
219 triangular pressure-outlet faces, zone 6, binary.
1694 triangular symmetry faces, zone 7, binary.
1092 triangular wall faces, zone 8, binary.
33 triangular wall faces, zone 9, binary.
64969 tetrahedral cells, zone 2, binary.

Building...
mesh
auto partitioning mesh by Metis (fast),
distributing mesh
parts................................,
faces................................,
nodes................................,
cells................................,
materials,
interface,
domains,
zones,
wing-tip
wing-surface
symmetry
outlet
inlet
fluid
interior-fluid
parallel,
Done.


> /define/boundary-conditions/velocity-inlet (inlet)
inlet no yes yes no 0 yes no 50 no 0 no 0

> /solve/initialize/hyb-initialization
Initialize using the hybrid initialization method.

Checking case topology...
-This case has both inlets & outlets
-Pressure information is not available at the boundaries.
Case will be initialized with constant pressure

iter scalar-0

1 1.000000e+00
2 3.204116e-03
3 6.282962e-04
4 2.183606e-04
5 6.469976e-05
6 2.289022e-05
7 7.921233e-06
8 2.911090e-06
9 1.077119e-06
10 4.187437e-07
hybrid initialization is done.


> /solve/iterate 10
iter continuity x-velocity y-velocity z-velocity time/iter
1 1.0000e+00 9.3978e-03 8.4207e-03 8.6167e-03 0:00:01 9
2 4.5170e-01 3.2794e-03 3.2977e-03 3.2374e-03 0:00:00 8
3 2.8198e-01 1.4109e-03 1.5119e-03 1.5059e-03 0:00:00 7
4 2.1502e-01 7.9329e-04 8.4129e-04 8.4017e-04 0:00:00 6
5 1.7309e-01 5.6978e-04 5.5468e-04 5.6431e-04 0:00:00 5
6 1.3927e-01 4.5844e-04 3.9632e-04 4.0327e-04 0:00:00 4
7 1.0968e-01 3.7456e-04 2.9209e-04 2.9621e-04 0:00:00 3
8 8.6301e-02 3.0567e-04 2.2128e-04 2.2177e-04 0:00:00 2
9 6.8001e-02 2.4458e-04 1.7117e-04 1.7187e-04 0:00:00 1
10 5.3917e-02 1.9585e-04 1.3448e-04 1.3641e-04 0:00:00 0

> /report/forces/wall-forces yes 1 0 0 yes "/users/project/aozuzun/force"
> Halting due to end of file on input.


BEST REGARDS

[alainislas]

By module I meant, to avoid typing the whole ansys installation directory and run it by typing a single word. This doesn't look to be done by your HPC staff. Anyway, Now fluent runs correctly.

This Unbound Variable error may be due to an incomplete setup of your case. Make sure to check all the boundary conditions are well defined. Please note that once Fluent runs in Batch mode, it will depend directly on the journal file set of instructions you are using. This is very sensitive to errors, so it will exit and terminate the job once any error is presented.

I suggest you to do the case setup with the GUI activated, so you can edit your case more comfortably.

If you are exporting reports, monitors, or calculation activities, make sure you are naming and placing them in the proper formats and directories. Otherwise, fluent will detect error and lead you to a job termination.

Greetings