Lehigh ISE / COR@L Lab Wiki

User Tools

Site Tools

Trace:
tutorial:torque

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
Next revision Both sides next revision
tutorial:torque [2017/03/22 14:09]
sey212 [Running Gurobi] 		tutorial:torque [2017/11/06 09:51]
sertalpbilal [Directly submitting job] Typo
Line 61: Line 61:
 </code> </code>
 If you do not want to write the submission script you can do it just by calling If you do not want to write the submission script you can do it just by calling
-<code>qsub -N JobName -q batch -l nodes=1:pnn=2  myscript.sh</code>+<code>qsub -N JobName -q batch -l nodes=1:ppn=2  myscript.sh</code>
 Now, we will run the code but we are setting the job parameters using ''-'' character (e.g. ''-N JobName'') Now, we will run the code but we are setting the job parameters using ''-'' character (e.g. ''-N JobName'')
  
 ===== Options ===== ===== Options =====
  
-  * ''-q <queue>'' set the queue. Often you will use the standard queue, so no need to set this up. +^ Option  ^ Description  ^ 
-  * ''-V'' will pass all environment variables to the job +| ''-q <queue>''  | Set the queue. Often you will use the standard queue, so no need to set this up. |  
-  * ''-v var[=value]'' will specifically pass environment variable 'var' to the job +| ''-V''  | Will pass all environment variables to the job |  
-  * ''-b y'' allow command to be a binary file instead of a script. +| ''-v var[=value]''  | Will specifically pass environment variable 'var' to the job |  
-  * ''-w e'' verify options and abort if there is an error +| ''-b y''  | Allow command to be a binary file instead of a script. |  
-  * ''-N <jobname>'' name of the job. This you will see when you use qstat, to check status of your jobs. +| ''-w e''  | Verify options and abort if there is an error |  
-  * ''-l resource_list'' specify resources +| ''-N <jobname>''  | Name of the job. This you will see when you use qstat, to check status of your jobs. |  
-  * ''-l h_rt=<hh:mm:ss>'' specify the maximum run time (hours, minutes and seconds) +| ''-l resource_list''  | Specify resources |  
-  * ''-l s_rt=hh:mm:ss'' specify the soft run time limit (hours, minutes and seconds) - Remember to set both s_rt and h_rt. +| ''-l h_rt=<hh:mm:ss>''  | Specify the maximum run time (hours, minutes and seconds) |  
-  * ''-cwd'' run in current working directory +| ''-l s_rt=hh:mm:ss''  | Specify the soft run time limit (hours, minutes and seconds) - Remember to set both s_rt and h_rt. |  
-  * ''-wd <dir>'' Set working directory for this job as <dir> +| ''-cwd''  | Run in current working directory |  
-  * ''-o <output_logfile>'' name of the output log file +| ''-wd <dir>''  | Set working directory for this job as <dir> |  
-  * ''-e <error_logfile>'' name of the error log file +| ''-o <output_logfile>''  | Name of the output log file |  
-  * ''-m ea'' Will send email when job ends or aborts +| ''-e <error_logfile>''  | Name of the error log file |  
-  * ''-P <projectName>'' set the job's project +| ''-m ea''  | Will send email when job ends or aborts |  
-  * ''-M <emailaddress>'' Email address to send email to +| ''-P <projectName>''  | Set the job's project |  
-  * ''-t <start>-<end>:<incr>'' submit a job array with start index , stop index in increments using+| ''-M <emailaddress>''  | Email address to send email to |  
 +| ''-t <start>-<end>:<incr>''  | Submit a job array with start index , stop index in increments using |
  
-See [[http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html|THIS]] for more details+You can find detailed information [[http://gridscheduler.sourceforge.net/htmlman/htmlman1/qsub.html|here]].
  
 +<note tip>You need to use option ''-V'' to pass environment variables, which is needed to run solvers such as (Cplex, Gurobi, MOSEK, etc..). [[tutorial:torque#running_solvers|See here]].</note>
 ===== Monitoring and Removing jobs ===== ===== Monitoring and Removing jobs =====
  
Line 157: Line 159:
 <note tip>Use **-singleCompThread** [[https://www.mathworks.com/help/matlab/ref/maxnumcompthreads.html|option]] for Matlab to use a single thread. A similar option may be needed for the program/solver you're using.</note> <note tip>Use **-singleCompThread** [[https://www.mathworks.com/help/matlab/ref/maxnumcompthreads.html|option]] for Matlab to use a single thread. A similar option may be needed for the program/solver you're using.</note>
  
-==== Running Solvers (Gurobi/CPLEX/Mosek/AMPL/...) ====+==== Running Solvers ====
  
-In order to run solvers, you need to use "-V" (it is Upper case) option. i.e.:+In order to run solvers (such as Gurobi/CPLEX/Mosek/AMPL/...), you need to use "-V" (it is Upper case) option. i.e.:
  
 <code>qsub -V submitFile.pbs </code> <code>qsub -V submitFile.pbs </code>
Line 181: Line 183:
  
 However, first you have to have a permission to use GPU (given by Prof. Takac) -- this is just formality to allow to certain users to use video driver on polyp30 However, first you have to have a permission to use GPU (given by Prof. Takac) -- this is just formality to allow to certain users to use video driver on polyp30
 +
 +If you are using TensorFlow in Python, you can set the limit on amount of GPU memory using:
 +<code>config_tf = tf.ConfigProto()
 +config_tf.gpu_options.per_process_gpu_memory_fraction = p</code>
 +in which **//p//** is the percent of GPU memory (a number between zero and one). 
  
 ==== Running MPI and Parallel Jobs ==== ==== Running MPI and Parallel Jobs ====
Line 306: Line 313:
  
  
 +==== Tensorflow with GPU ==== 
 +To use tensorflow with a specific GPU, say GPU 1, you can simply set 
 +<code bash> 
 +export CUDA_VISIBLE_DEVICES=1 
 +</code> 
 +and then schedule your jobs with Torque to perform experiments on GPU 1.
tutorial/torque.txt · Last modified: 2024/02/28 13:12 by mjm519

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki