tutorial:torque
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
tutorial:torque [2016/10/18 15:15] sertalpbilal |
tutorial:torque [2016/11/28 10:15] sertalpbilal [Queues] |
||
|---|---|---|---|
| Line 2: | Line 2: | ||
| TORQUE provides control over batch jobs and distributed computing resources. It is an advanced open-source product based on the original PBS project and incorporates the best of both community and professional development. It incorporates significant advances in the areas of scalability, | TORQUE provides control over batch jobs and distributed computing resources. It is an advanced open-source product based on the original PBS project and incorporates the best of both community and professional development. It incorporates significant advances in the areas of scalability, | ||
| + | |||
| + | |||
| + | |||
| + | ===== Prerequisite ===== | ||
| + | In order to extract your output and error results in Torque, you need to have password-less connection between nodes. If you have not set it once, execute the following commands. These commands create a public and private key so that when a node want to transfer a file to your home folder, it does not require the password. | ||
| + | After connecting to polyps enter: | ||
| + | |||
| + | <code bash> | ||
| + | ssh-keygen -N "" | ||
| + | </ | ||
| + | |||
| + | Then just press ENTER for any question. After that type the following commands: | ||
| + | |||
| + | <code bash> | ||
| + | touch ~/ | ||
| + | chmod 600 ~/ | ||
| + | cat ~/ | ||
| + | </ | ||
| + | Now, you will get the error log and output log files for your jobs. | ||
| + | |||
| + | |||
| + | |||
| ===== Hardware ===== | ===== Hardware ===== | ||
| Line 11: | Line 33: | ||
| ===== Submitting Jobs ===== | ===== Submitting Jobs ===== | ||
| - | |||
| - | <note tip> | ||
| Jobs can be submitted either using a submission file or directly from command line. First we explain how it is done and then we will discuss the options. | Jobs can be submitted either using a submission file or directly from command line. First we explain how it is done and then we will discuss the options. | ||
| Line 75: | Line 95: | ||
| ==== Queues ==== | ==== Queues ==== | ||
| - | We have few queues '' | + | We have few queues '' |
| < | < | ||
| Queue Memory CPU Time Walltime Node Run Que Lm State | Queue Memory CPU Time Walltime Node Run Que Lm State | ||
| ---------------- ------ -------- -------- ---- --- --- -- ----- | ---------------- ------ -------- -------- ---- --- --- -- ----- | ||
| + | gpu -- -- | ||
| medium | medium | ||
| short -- -- | short -- -- | ||
| long | long | ||
| batch -- -- | batch -- -- | ||
| - | verylong | + | verylong |
| - | | + | AMPL -- -- |
| - | 0 | + | MOSEK |
| </ | </ | ||
| + | |||
| + | If you want to use AMPL or MOSEK, you have to use queue: AMPL or MOSEK, because we have limited licenses for them. | ||
| + | |||
| + | |||
| You can see limits using this command '' | You can see limits using this command '' | ||
| Line 196: | Line 222: | ||
| c2 | c2 | ||
| </ | </ | ||
| - | |||
| - | ===== Advanced ===== | ||
| - | |||
| - | |||
| - | The qsub command will pass certain environment variables in the Variable_List attribute of the job. These variables will be available to the job. The value for the following variables will be taken from the environment of the qsub command: | ||
| - | * **HOME** (the path to your home directory) | ||
| - | * **LANG** (which language you are using) | ||
| - | * **LOGNAME** (the name that you logged in with) | ||
| - | * **PATH** (standard path to excecutables) | ||
| - | * **MAIL** (location of the users mail file) | ||
| - | * **SHELL** (command shell, i.e bash, | ||
| - | * **TZ** (time zone) | ||
| - | These values will be assigned to a new name which is the current name prefixed with the string " | ||
| - | * **PBS_O_HOST** (the name of the host upon which the qsub command is running) | ||
| - | * **PBS_SERVER** (the hostname of the pbs_server which qsub submits the job to) | ||
| - | * **PBS_O_QUEUE** (the name of the original queue to which the job was submitted) | ||
| - | * **PBS_O_WORKDIR** (the absolute path of the current working directory of the qsub command) | ||
| - | * **PBS_ARRAYID** (each member of a job array is assigned a unique identifier) | ||
| - | * **PBS_ENVIRONMENT** (set to PBS_BATCH to indicate the job is a batch job, or to PBS_INTERACTIVE to indicate the job is a PBS interactive job) | ||
| - | * **PBS_JOBID** (the job identifier assigned to the job by the batch system) | ||
| - | * **PBS_JOBNAME** (the job name supplied by the user) | ||
| - | * **PBS_NODEFILE** (the name of the file contain the list of nodes assigned to the job) | ||
| - | * **PBS_QUEUE** (the name of the queue from which the job was executed from) | ||
| - | * **PBS_WALLTIME** (the walltime requested by the user or default walltime allotted by the scheduler) | ||
| ===== Mass Operations ===== | ===== Mass Operations ===== | ||
| Line 250: | Line 252: | ||
| </ | </ | ||
| to cancel all of your running jobs. | to cancel all of your running jobs. | ||
| - | |||
| - | ===== Prerequisite ===== | ||
| - | In order to extract your output and error results in Torque, you need to have password-less connection between nodes. If you have not set it once, execute the following commands. These commands create a public and private key so that when a node want to transfer a file to your home folder, it does not require the password. | ||
| - | After connecting to polyps enter: | ||
| <code bash> | <code bash> | ||
| - | ssh-keygen -N "" | + | qselect |
| </ | </ | ||
| + | will cancel all jobs (both running jobs and queue). | ||
| - | Then just press ENTER for any question. After that type the following commands: | ||
| - | <code bash> | + | ===== Advanced ===== |
| - | touch ~/ | + | |
| - | chmod 600 ~/ | + | |
| - | cat ~/.ssh/id_rsa.pub >> ~/.ssh/ | + | The qsub command will pass certain environment variables in the Variable_List attribute of the job. These variables will be available to the job. The value for the following variables will be taken from the environment of the qsub command: |
| - | </ | + | * **HOME** (the path to your home directory) |
| - | Now, you will get the error log and output log files for your jobs. | + | * **LANG** (which language you are using) |
| + | * **LOGNAME** (the name that you logged in with) | ||
| + | * **PATH** (standard path to excecutables) | ||
| + | * **MAIL** (location of the users mail file) | ||
| + | * **SHELL** (command shell, i.e bash, | ||
| + | * **TZ** (time zone) | ||
| + | These values will be assigned to a new name which is the current name prefixed with the string " | ||
| + | * **PBS_O_HOST** (the name of the host upon which the qsub command is running) | ||
| + | * **PBS_SERVER** (the hostname of the pbs_server which qsub submits the job to) | ||
| + | * **PBS_O_QUEUE** (the name of the original queue to which the job was submitted) | ||
| + | * **PBS_O_WORKDIR** (the absolute path of the current working directory of the qsub command) | ||
| + | * **PBS_ARRAYID** (each member of a job array is assigned a unique identifier) | ||
| + | * **PBS_ENVIRONMENT** (set to PBS_BATCH to indicate the job is a batch job, or to PBS_INTERACTIVE to indicate the job is a PBS interactive job) | ||
| + | * **PBS_JOBID** (the job identifier assigned to the job by the batch system) | ||
| + | * **PBS_JOBNAME** (the job name supplied by the user) | ||
| + | * **PBS_NODEFILE** (the name of the file contain the list of nodes assigned to the job) | ||
| + | * **PBS_QUEUE** (the name of the queue from which the job was executed from) | ||
| + | * **PBS_WALLTIME** (the walltime requested by the user or default walltime allotted by the scheduler) | ||
tutorial/torque.txt · Last modified: 2024/02/28 13:12 by mjm519
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
