Partners Research Computing

Partners High Performance Computing Cluster

Partners applications via Mac/PC

Clinical & research applications

DFCI bioinformatics computer

PHS Research Computing cluster

Bioinformatics news

Data Storage & Backup

Sharing files & collaboration

HIPAA, ePHI and research (internal)

RPDR

HPCGG

Biomedical Engineering Model Shop

Harvard's GForge Implementation

Institutional research distribution lists

pHPC account registration	pHPC user guide	pHPC services	pHPC web protal

How to run parallel Dock6 on the Linux clusters

Step 1: Please read "how to setup your working environment" and do the following:

a. In your .bashrc file, include the sentence of "module load dock6/default" and "module load mpi/mpich2/default" in the module-loading section. You shall have something like this


# modules definitions
if [ -n "$MODULESHOME" ]; then
  module use /shr/modules
  module load mpi/mpich2/default
  module load dock6/default
fi

Note: A non-parallel version of dock6 is also installed in /source/dock6/non-parallel/dock6. For test or study purpose, you can allocate one computing node (interactively) and work with this non-parallel version. To know how to allocate one computing node interactively, please refer to "How to run interactive job".

b. Type ". .bashrc" in your terminal, and then type "module list in your terminal again, you shall be able to see:

[testy@n137 ~]$ . .bashrc
[testy@n137 ~]$ module list
Currently Loaded Modulefiles:
   1) mpi/mpich2/default 1) dock6/default

If you see the above output on terminal, it means your environment has been setup correctly.

c. To confirm your configuration is correct, type the following:

[testy@n137 ~] which mpiexec
/source/mpich2/bin/mpiexec
[testy@n137 ~] which dock6.mpi
/source/dock6/parallel/dock6/bin/dock6.mpi

Step 2: Copy /source/dock6/parallel/dock6/tutorial/ to your own working directory /shr/home/$USER/TestDock6/ , enter "mpi_demo2" folder, then follow the instructions in the 0README file to generate necessary input files. (particularly pay attention mpi.in input file ). When finish the preparation, go to the "4_dock" directory

Step 3: The following script mpidock.lsf is used to submit the mpi docking program into the cluster.

Note: This script will:

1) Setups the job parameters, including allocation of processors

2) Pass the LSF scheduler information to the mpich2 and create mpich2 execution environment

3) Submit the dock6.mpi to the mpich2, runs the program.

4) Clean the job

Here is the mpidock6.lsf job script

#BSUB -L /bin/bash

# the name of your job on the queue system
#BSUB -J mpidock

# the queue that you will use, the example here use the queue called "normal"
# please use bqueus command to check the available queues
#BSUB -q normal


# the system output and error message output, %J will show as your jobID
#BSUB -o %J.out
#BSUB -e %J.err

#the number of processors that you will use, you can modify this number
#BSUB -n 10

#when job finish that you will get email notification
#BSUB -u testy@partners.org
#BSUB -N


############ enter your working directory, change to your own dir ###
work_dir="/shr/home/$USER/TestDock6/tutorials/mpi_demo2/4_dock/"
cd $work_dir

############ this section is to create mpd ring, DO NOT modify unless you understand #####
nproc=0
for proc in $LSB_HOSTS ; do
echo $proc >> mpd.procs
nproc=`expr $nproc + 1`
done
echo $LSB_HOSTS
echo $nproc
`sort -u mpd.procs > mpd.nodes`
nhosts=`less mpd.nodes | wc -l`
mpdboot -n $nhosts -v -f mpd.nodes

############ change the myexe to your own application and ncessary arguments  #####

mpiexec -machinefile mpd.procs -np $nproc dock6.mpi -i mpi.in -o dock.out

############ exit the mpd ring and clean off the nodes ###################

mpdallexit
mpdcleanup
`rm mpd.nodes`
`rm mpd.procs`

Step 4: Submit the job

[testy@n137 ~]$ bsub < mpidock6.lsf
Job <497007> is submitted to queue .
[testy@n137 4_dock]$ bjobs -l

Job <497007>, Job Name , User , Project , Mail , Status , Queue , Command <#
                     BSUB -L /bin/bash; # the name of your job on the queue sys
                     tem;#BSUB -J mpich2_test; # the queue that you will use, t
                     he example here use the queue called "normal";# please use
                     bqueus command to check the available queues;#BSUB -q norm
                     al;  # the system output and error message output, %J will
                     show as your jobID;#BSUB -o %J.out;#BSUB -e %J.err; #the n
                     umber of processors that you will use;#BSUB -n 10; #when j
                     ob finish that you will get email notification;#BSUB -u te
                     sty@partners.org;#BSU>
Wed Apr  8 20:00:45: Submitted from host , CWD <$HOME/TestDock6/tutorials
                     /mpi_demo/4_dock>, Output File <%J.out>, Error File <%J.er
                     r>, Notify when job ends, 10 Processors Requested, Login S
                     hell ;
 PENDING REASONS:
 The schedule of the job is postponed for a while;

 SCHEDULING PARAMETERS:
           r15s   r1m  r15m   ut      pg    io   ls    it    tmp    swp    mem
 loadSched   -     -     -     -       -     -    -     -     -      -      -
 loadStop    -     -     -     -       -     -    -     -     -      -      -

[jxu@n137 4_dock]$

When job finishes, you shall have something like the following output, the dock.out will contain all the results, and there are also list of individual output files generated by each processor.

[testy@n137 4_dock]$ ls -ltr
total 768
-rw-r-----  1 testy testy    734 Apr  6 14:51 0README
-rwxr-x---  1 testy testy    982 Apr  6 14:51 script_demo
-rwxr-x---  1 testy testy     49 Apr  6 14:51 script_clean
-rw-r-----  1 testy testy   4015 Apr  6 14:52 mpi.in
-rw-rw----  1 testy testy   1219 Apr  8 20:00 mpidock.lsf
-rw-rw-r--  1 testy testy   6381 Apr  8 20:01 dock.out.1
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.7
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.6
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.5
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.4
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.3
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.9
-rw-rw-r--  1 testy testy   6382 Apr  8 20:01 dock.out.8
-rw-rw-r--  1 testy testy   6479 Apr  8 20:01 dock.out.2
-rw-rw-r--  1 testy testy   8478 Apr  8 20:01 dock.out
-rw-rw-r--  1 testy testy  23819 Apr  8 20:01 1VRT_scored.mol2
-rw-rw----  1 testy testy    548 Apr  8 20:01 497007.out
-rw-rw----  1 testy testy      0 Apr  8 20:01 497007.err