Partners High Performance Computing Cluster
Partners applications via Mac/PC
Clinical & research applications
DFCI bioinformatics computer
PHS Research Computing cluster
Bioinformatics news
Data Storage & Backup
Sharing files & collaboration
HIPAA, ePHI and research (internal)
RPDR
HPCGG
Biomedical Engineering Model Shop
Harvard's GForge Implementation
Institutional research distribution lists

 

pHPC account registration pHPC user guide pHPC services pHPC web protal



How to run Matlab as batch mode jobs on clusters


Intro: The current examples and instructions listed in this page are mainly for the batch mode execution of matlab, and you donot need GUI interface or matlab distributing tool box to control your computation.

However, if you do need matlab GUI, you have two ways to do that

(a) If you are linux user, please allocate one computing node for xwindows, then start your matlab GUI from the computing node, please refer to "how to run interactive jobs" .

(b) If you are windows user, please use windows remote desktop to logon hpcwin.research.partners.org (you need to be inside PHS firewall), then start matlab GUI from there.

Note: if you need to load large image data in the GUI, we suggest you use (b) since hpcwin is networked with the cluster storage system.

For DCT/DCE related (distributed or parallel) jobs, please visit "How to distribute matlab jobs on HPRES cluster via Matlab DCE/DCT"

Step 1 to 4 only are needed if it is the first time for you to use matlab on HPRES cluster.

Step 1. Setup matlab environmnt.

a. Please read and finish "How to setup your working environment"


b Include the sentence of "module load matlab/default" in the module-loading blocks in your .bashrc file (you probably have it already)

c. Check your modules

 [testy@n137 ~] module list
 Currently Loaded Modulefiles:
  1) matlab/default
check your LD_LIBRARY_PATH.
[testy@n137 ]$ echo $LD_LIBRARY_PATH
/source/matlab2007b/extern/lib/glnxa64
This indicates your environment has been setup correctly and you can begin to work and submit your job.

Step 2. Use matlab in batch mode

create new directory
[testy@n137 ~]$ mkdir TestMatlab

Example A. Single Matlab task.

example myplot.m can be downloaded, and put this file in the directory.

Then, create a new file called "testmat.lsf" as your job script, copy/paste the following example into this new file, and make slight modification about your email address. In a real situation, you have to make sure you choose the right queue based on what toolbox you might want to use because of the license issue. If you use multiple toolboxes, choose the one with least number of licenses (the number of license for each toolbox is listed at the bottom of the page)
#!/bin/bash

# enable your environment, which will use .bashrc configuration in your home directory
#BSUB -L /bin/bash

# the name of your job showing on the queue system
#BSUB -J myjob

# the following BSUB line specify the queue that you will use,
# please use bqueues command to check the available queues for each toolbox of matlab
# Bioinformatics toolbox, please use matlabbio queue
# Signal Processing toolbox, please use matlabsig queue
# Image Processing toolbox, please use matlabimg queue
# Wavelet toolbox, please use matlabwav queue
# Matlab DCE, please use matlabdce queue
# Matlab, Optimization and Statistic, please use matlab queue


#BSUB -q matlab


# the system output and error message output, %J will show as your jobID
#BSUB -o %J.out
#BSUB -e %J.err

#the CPU number that you will collect (Attention: each node has 2 CPU)
#BSUB -n 1


#when job finish that you will get email notification
#BSUB -u YourEmail@partners.org
#BSUB -N


#enter your working directory
cd /shr/home/$USER/TestMatlab


# Finally, Start the program
/source/matlab2007b/bin/matlab < myplot.m


Submit your job
[testy@n137 ~]$ bsub < testmat.lsf



When job finish, you will recieve an email, and you will also see a "MyfigFile.jpg" image file created.

Example B. Many identical Matlab tasks with different input parameters, and you need many computing nodes run them simultaneously.

example manyplot.m can be downloaded, and put this file in the directory "TestMatlab" that you just setup in example A.

In this case, we encourage you use Job Array to generate the input parameter and pass it to the matlab command, here is the job script

#!/bin/bash

# enable your environment, which will use .bashrc configuration in your home directory
#BSUB -L /bin/bash

# the name of your job showing on the queue system, here it means you have 4 job elements
#BSUB -J matlabArray[1-4]

# the following BSUB line specify the queue that you will use,
# please use bqueues command to check the available queues for each toolbox of matlab
# Bioinformatics toolbox, please use matlabbio queue
# Signal Processing toolbox, please use matlabsig queue
# Image Processing toolbox, please use matlabimg queue
# Wavelet toolbox, please use matlabwav queue
# Matlab DCE, please use matlabdce queue
# Matlab, Optimization and Statistic, please use matlab queue

#BSUB -q matlab


# the system output and error message output, %J will show as your jobID
#BSUB -o "%J.%I.out"
#BSUB -e "%J.%I.err"

#the computing core number that you will collect for each job element
#BSUB -n 1


#when job finish that you will get email notification
#BSUB -u YourEmail@partners.org
#BSUB -N


#enter your working directory
cd /shr/home/$USER/TestMatlab


# Finally, Start the matlab and pass the paramter to the program, the parameter $LSB_JOBINDEX is system level
# variable from LSF queue. The value is from 1 to 4 (as indicated in job name) and will be distributed to each processor.
# You can associate this variable with your real parameters, your filenames ...etc
# To know more about job array, please check "how to submit massive jobs " webpage in this website.

/source/matlab2007b/bin/matlab -r 'manyplot($LSB_JOBINDEX)'




In the above job scirpt, make sure you choose the right queue based on what toolbox you might want to use. If you use multiple toolboxes, choose the one with least number of licenses (Except if you use DCE, then just choose matlabdce queue). Then Copy/paste the above and create new job script called "matlabarray.lsf" in the same directory, then submit the job.
[testy@n137 TestMatlab]$ bsub < matlabarray.lsf
When job finishs, you will see four figures are generated, each is computed and generated by each job element. The output from matlab is stored in the ".out" files.

Notes:
Here are the TOTAL available licenses in the cluster.

MATLAB:         35 (matlab queue)
Compiler         2  
DCT         6   (matlabdce queue)
DCE         32   (matlabdce queue)
optimization         35   (matlab queue)
Statistics         35   (matlab queue)
Image_Toolbox         8   (matlabimg queue)
Bioinformatics_Toolbox         8   (matlabbio queue)
Signal_Toolbox         16   (matlabsig queue)
Wavelet_Toolbox         8   (matlabwav queue)


For DCT/DCE related distributed or parallel jobs, please visit "How to distribute matlab jobs on HPRES cluster via Matlab DCE/DCT"