Please send an e-mail to hpc-admin[at]ceng[dot]metu[dot]edu[dot]tr stating who you are and why you need an account. We will send further information and instructions to you

First, you should choose an MPI library. It can be done using mpi-selector-menu command:

$ mpi-selector-menu
Current system default: <none>
Current user default:   <none>

    "u" and "s" modifiers can be added to numeric and "U"
    commands to specify "user" or "system-wide".

1. mvapich2_gcc-1.2p1
2. mvapich2_intel-1.2p1
3. mvapich_gcc-1.1.0
4. mvapich_intel-1.1.0
5. openmpi_gcc-1.2.8
6. openmpi_intel-1.2.8
U. Unset default
Q. Quit

Selection (1-6[us], U[us], Q):

For example, to select mvapich_gcc-1.1.0 type “3u”. Then simply type “q” to quit.

Alternatively, you can use mpi-selector command:

$ mpi-selector --set mvapich_gcc-1.1.0

Please remember that you should start a new shell to begin using the selected MPI library. You can simply logout and login back to start a new shell.

Then, you should compile your program using mpicc (for C) or mpicxx (for C++):

$ mpicc myprog.c -o myprog

Afterwards, prepare a PBS script for submitting your job to the queue manager:

## select queue for submitting job
#PBS -q cenga

## set the name of the job
#PBS -N myjob

## request resources
#PBS -l nodes=2:ppn=2

## switch to the working directory (by default PBS launches processes from your home directory)
cd $PBS_O_WORKDIR

## run the program on 4 processors
mpirun -np 4 myprog

For different MPI libraries, some modifications in “mpirun” line will be necessary.

• MVAPICH: Number of processors on which the program will run should be specified using -np option:

mpirun -np 4 myprog

• OpenMPI: A hostfile should be provided using -hostfile option. The hostfile provided by PBS can be used for this purpose:

mpirun -hostfile $PBS_NODEFILE myprog

• MVAPICH2: Both number of processors and a hostfile should be specified. Furthermore, mpirun_rsh should be used instead of mpirun:

mpirun_rsh -np 4 -hostfile $PBS_NODEFILE myprog

Finally, you should submit your PBS script to the queue manager:

$ qsub mypbsscript

You can run jobs on METU CENG HPC Cluster with your TR-grid user account from kefal.ulakbim.gov.tr. To do that you should do the following:

1. Create a directory under /trgrid_scratch on kefal.ulakbim.gov.tr with your user name

$ mkdir /trgrid_scratch/user-name

2. Change the permissions of this directory, so that other users can not reach it

$ chmod 700 /trgrid_scratch/user-name

3. Copy the files related to the job you are going to submit to “Nar” cluster to that directory

$ cp -r ~/related-directory /trgrid_scratch/user-name/

4. Prepare the PBS Script for your job. Meanwhile, remember the following:

Specify the name of the queue server with concatenating ”@kavun-ib” to the line where you write the name of the queue you are going to submit your job

<code>#PBS -q trgridb@kavun-ib</code>

After specifying other PBS parameters, add the following lines to your script

<code>. /usr/share/Modules/init/sh
        module load trgrid/scratch</code>
   

For the remaining configurations required for your application, use the “module load” command.
(You can get information about loadable libraries compilers and applications from Available Modules) Ex:

<code>module load espresso/intel/IB</code>

Use TRGRID_SCRATCH environment variable for specifying file paths. Ex:

cd $TRGRID_SCRATCH/user-name/related-directory
mpiexec -comm ib $TRGRID_SCRATCH/user-name/related-directory/executable-name > output.txt 2> error.txt

5. Send your job to the queue manager, using your script

$ qsub your-script

Furthermore, you can check the status of your job or cancel it. Ex:

$ qstat 42099.kavun-ib
$ qstat @kavun-ib
$ qdel 42099.kavun-ib

Microsoft Windows HPC Server 2008 is installed on 3 nodes of our system one being the head node. These nodes are dual-bootable and normally booted with Scientific Linux. To use this product, you should do the following:

1. Ordered List ItemSend an e-mail to hpc-admin@ceng.metu.edu.tr indicating that you want to use MSW HPC Server. We will reboot the nodes and create an account for you.
2. Install MSW HPC Server's Client Side to your computer.
3. Check Microsoft Windows HPC Server 2008 for examples and usage guidelines.

In order to use HOD, firstly the required environment variables should be set. This can be accomplished simply by the following command:

module load application/hod

This line might be appended to ~/.profile file for making it available on every login.

Afterwards, Hadoop jobs might be run in a typical HOD session by using the following sequence of commands:

hod allocate ...
hadoop ...
hadoop ...
...
hod deallocate ...

Furthermore, HOD might also be used for directly running Hadoop scripts. For example, a sample script with the following content

sleep 60
hadoop fs -copyFromLocal hadoop-input.txt /
hadoop jar /home1/software/application/hadoop/hadoop-0.20.1-examples.jar wordcount /hadoop-input.txt /wc-output &> wc-out
hadoop fs -copyToLocal /wc-output .

might be run using the following command:

hod script -d ~/hod-clusters/test -n 5 -s hadoop-script

For more information about using HOD, please refer to http://hadoop.apache.org/common/docs/r0.20.1/hod_user_guide.html