Academic Computing HPC Project Cluster
SGE Scheduler Information

The information below is a brief guide to using the Sun Grid Engine Scheduling System for submitting your jobs. Using a scheduler ensures that your jobs are given the resources that they need and that you are not taking away resources from other users. If you have any questions or comments, please feel free to send email to acis-hpc@mit.edu.

To gain familiarity with our cluster you should first read the ROCKS User Guide, in particular the section entitled Start Computing.

Compiling your application

Our cluster has two compilers available for you to use - gnu and intel. The GNU compiler is free and open to all whereas the Intel compiler is managed with a license server. We currently have 2 floating licenses for the Intel C compiler, Fortran compiler and Math Kernel Libraries. You may have to wait until other users have finished with their compiling before you will be given a license to run the program.

To use a given cluster you will need to set certain environment variables to ensure that your application will compile correctly. Environment variables may be added as follows:

export PATH=$PATH:/opt/mpich/gnu/bin
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/mpich/gnu/lib

The Intel compiler suite can be found in /opt/intel/cc/9.0, and /opt/intel/fc/9.0. If you wish to use them add the path/bin to your PATH and path/lib to your LD_LIBRARY_PATH.

Running your program interactively

NOTE!Programs should only be run interactively to check for run-time problems or compiler errors. Any job that will take longer than a minute or two should be submitted to the scheduler. We reserve the right to kill jobs that are taking away resources from other users.

The following commands will start an interactive job:

/opt/mpich/[gnu][intel]/mpirun -nolocal -np [#nodes] /home/username/programname

Running your program through the scheduler

Our cluster uses the Sun Grid Engine Scheduling System. For more information you should read the Rocks SGE Information or the entire user manual from Sun Microsystems.

Important Notes About The Cluster

It is very important that you move (not copy) your files immediately from the server to your athena account (via sftp) or some other machine. We do not back up any userdata whatsoever at anytime and since the cluster is still under development the loss of data during system changes is possible. Furthermore, our project cluster has very limited diskspace so we ask you to move (not copy) your files off of the server so that fellow users do not run into disk space problems.

last updated: 11.04.05