To use WEXAC, you must register for the service via Internal Services with your lab’s departmental administrator. Please contact us if the lab you are associated with is not a WEXAC member (i.e. does not have WEXAC nodes).

For detailed information on our WEXAC policy, please refer to our WEXAC policy page.

Using the WEXAC Software Stack
with the Environment Modules Package

The Environment Modules package allows you to dynamically modify your WEXAC cluster user environment, so as to run specific software packages.

While Modules implementation is shell dependent, commands are not. The following table lists a number of sample commands.

Environment Modules Package
Command Description
module avail List available Modules software
module list List currently loaded Modules packages
module load matlab/R2011b load Load a particular environment for a package
module switch matlab/R2012a Switch the environment of a package
module unload matlab/R2012a Unload a particular environment for a package

Following is a list of sample installed software packages, which can be loaded using Modules:

  • BEDTools/2.16.2
  • blast/2.2.26
  • blat/3.4
  • clustalw/2.1
  • comsol/4.2a
  • EMBROSS/6.4.0
  • fasta/36.3.5a
  • IGV/2.1
  • jdk/1.7.0
  • matlab/R2007b
  • matlab/R2010b
  • matlab/R2011b
  • matlab/R2012a
  • mpich/1.2.7
  • mysql/5.5.22
  • ncbi-blast+/2.2.25
  • ncbi-blast+/2.2.26
  • perl/5.14.2
  • perl/5.8.8
  • pgi/11.10
  • pgi/12.4
  • php/5.3.10
  • prank/111130
  • pymol/1.5
  • python/3.2.2
  • R/2.12.1
  • R/2.14.2
  • rmblast/1.2
  • rstudio-server/0.95.265
  • samtools/0.1.17
  • tcltk/8.5.11
  • velvet/1.2.03
  • vim-7.3/7.3
  • websvn/2.3.3

To learn more about the Environment Modules project, visit Environment Modules.

LSF (Load Sharing Facility)

WEXAC runs Platform LSF—powerful, enterprise-grade IBM middleware with comprehensive resource management and job scheduling functionality.

LSF’s scheduling policy is based largely on the fairshare model.

Running LSF commands

To run a package from tcsh using LSF, enter the following command:

# bsub -o somepath.o -e somepath.e somescript.csh

Scripts will resemble the following:

#!/bin/bash
module load matlab/R2010b
hostname
matlab -nodisplay -nojvm < /myhome/h.m > /myhome/h.out

bash users can modify the .bashrc file to include the following lines:

if [ -f /storage/apps/Modules/default/etc/profile.modules ]; then
. /apps/Modules/default/etc/profile.modules
module load matlab/R2010b R/2.14.2
fi

The following table lists key LSF commands, along with their SGE equivalents.

key LSF commands
Description LSF Command SGE Equivalent

Queue Commands

   
List all jobs running on the cluster bjobs -u all qstat
List all jobs running on the cluster,
expanded to display nodes
bjobs -u all -X qstat -g t
List your running jobs
(also accepts -X flag to expand to display nodes)
bjobs qstat -u user_ID
Display detailed information on a job
(with JOBID representing the job ID number)
bjobs -l JOBID qstat -j JOBID
List available queues, including the number
of jobs and slots
bqueues qstat -g c

Job Commands

   
Submit job named jobname bsub < jobname.bsub qsub jobname.job
Delete running job
(with JOBID representing the job ID number)
bkill JOBID qdel JOBID

Job Flags

   
Submit job to queue queuename #BSUB -q queuename #$ -q queuename.q
Job name #BSUB -J JobName #$ -N JobName
Array job with 10 elements #BSUB -J JobName[1-10 #$ -t 1-10
Request xx number of slots #BSUB -n xx #$ -pe mvapich xx
Issue e-mail notification upon job initiation #BSUB -B #$ -m b
Issue e-mail notification upon job completion #BSUB -N #$ -m e
Write output to output.log (LSF - I for array JOBINDEX) #BSUB -o output.log #$ -o output.log
Write error to error.log
(LSF - I for array JOBINDEX)
#BSUB -e error.log #$ -e error.log

Job Variables

   
Job ID $LSB_JOBID $JOB_ID
Array job index
(when using -t in SGE and
-J Name[1-X] in LSF)
$LSB_JOBINDEX $SGE_TASK_ID
Number of parallel slots requested $LSB_DJOB_NUMPROC $NSLOTS

 

For further information on LSF, refer to the LSF Quick Reference Guide.

For specifics on SGE to LSF migration, feel free to review the WEXAC SGE-to-LSF Migration user training presentation.