You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: resources/computecan_runningjobs.md
+8-7
Original file line number
Diff line number
Diff line change
@@ -4,22 +4,23 @@ title: Running jobs on Compute Canada
4
4
---
5
5
# Running jobs on Compute Canada
6
6
7
-
## Batch scripts and schedulers
7
+
A good overview of the way we most often use Compute Canada is in the [Parallel Computing in R with WestGrid](https://www.sfu.ca/content/dam/sfu/stat/documents/Statgen/ParallelComputing_inR_CC.pdf)
8
+
presentation by Bhagya Karunarathna. (Note: WestGrid is the western-Canada division of Compute Canada.)
9
+
Bhagya mentions, "batch scripts", "schedulers" and "R scripts". Links to more information on these topics is given below.
8
10
9
-
To run a job you must create what's called a "batch script" and submit it to a "scheduler".
11
+
## Batch scripts and schedulers
10
12
11
13
* See the [introduction to schedulers](https://docs.computecanada.ca/wiki/What_is_a_scheduler%3F) for background on schedulers and compute clusters. The name of the scheduler
12
14
used by Compute Canada is slurm.
13
15
* See the [running jobs](https://docs.computecanada.ca/wiki/Running_jobs) documentation for basic information on running jobs on Compute Canada.
14
16
* See the [array jobs](https://docs.computecanada.ca/wiki/Job_arrays) documentation to see how to run the same job multiple times, possibly with different inputs, random seeds, etc. We often use array jobs to run Monte Carlo simulations; e.g., for importance sampling or to get permutation/bootstrap distributions.
15
-
* See the [META](https://docs.computecanada.ca/wiki/META_package_for_serial_farming) documentation for an alternative to array jobs that requires less work for the scheduler. META is supposed to have several convenience features for users, such as the ability to automatically re-submit jobs from an initial submission that crashed or ran out of time. No one in our group has used META yet. You can watch a [webinar](https://www.youtube.com/watch?v=GcYbaPClwGE).
17
+
<!--* See the [META](https://docs.computecanada.ca/wiki/META_package_for_serial_farming) documentation for an alternative to array jobs that requires less work for the scheduler. META is supposed to have several convenience features for users, such as the ability to automatically re-submit jobs from an initial submission that crashed or ran out of time. No one in our group has used META yet. You can watch a [webinar](https://www.youtube.com/watch?v=GcYbaPClwGE). -->
16
18
17
19
## Running R in batch mode
18
20
19
21
* For jobs that run R, the last line of the slurm batch script will
20
-
be of the form `R CMD BATCH <myscript.R>` where `<myscript.R>` is
22
+
be of the form `Rscript <myscript.R>` or `R CMD BATCH <myscript.R>` where `<myscript.R>` is
21
23
your R script.
22
-
* Say something about setting seeds for reproducibility.
23
-
* See the [Parallel Computing in R with WestGrid](https://www.sfu.ca/content/dam/sfu/stat/documents/Statgen/ParallelComputing_inR_CC.pdf)
24
-
presentation by Bhagya Karunarathna. (Note: WestGrid is the western-Canada division of Compute Canada.)
24
+
* For reproducibility, **please** get in the habit of setting the seed and printing the session information in each R script. That is, **all** of your R scripts should start with the line `set.seed(N)` for some number `N`, and should end with `sessionInfo()`.
0 commit comments