Home

RSpec Autograder for Prairielearn (PL)

This repository contains the source code for the RSpec Autograder developed at UC Berkeley.

Usage

info.json

Add the following fields to info.json in the question you would like to autograde

"gradingMethod": "External",
"externalGradingOptions": {
        "enabled": true,
        "image" : "saasbook/pl-rspec-autograder",
        "entrypoint": "/grader/run.py",
        "timeout" : 60
    }

tests/ directory

A question writer must create the following in their Prairielearn question:

• the folder tests/ in the root directory of their question
  • subfolder tests/common/
    • which will contain the code that is shared between variants*
  • json file tests/meta.json
    • which will contain
      • the mapping of _submission_file as a value for the key "submission_file"
      • (optional) the location to which additional submitted files would be copied
  • subfolder tests/solution/
    • which must contain the file _submission_file that has the instructor solution
  • a number of subfolders tests/suite<number>
    • each of which contains the difference from the files in tests/common/ expressed by including a modified copy of the file that should be replaced/added to tests/common/

* This should include packaged gems (i.e. the result of bundle package --all && bundle install --development or whichever package environment you wish to use)

The tests/ folder in your question should look something like this:

tests/
+-- common/
|   +-- .bundle/
|   +-- spec/
|   +-- vendor/
|   +-- Gemfile
|   +-- Gemfile.lock
|   `-- replaced.rb
|
+-- solution/
|   +-- _submission_file  # this will hold the instructor's "submission"
|   |
|   ...                # any other submitted files can be included*
|
+-- var_description_of_this_variant/
|   `-- replaced.rb    # this file will replace the one in common
|
`-- meta.json          # this will contain the aforementioned mappings

*The student would need to submit additional files in the file submission box on the right of each question, but this feature has not yet been tested

Grading non-FPP questions

The default search location for a student submission is in data['submitted_answers']['student-parsons-solution']. If you are not collecting code from a student in an FPP, you may provide the file tests/submission_processing.py with the function prepSubmission that will prepare the files needed in step 7 of the grading process. Below is an explanation of the function's requirements.

prepSubmission(data: Dict, ROOT_DIR: str, SUBMISSION_DIR: str) -> None

• data is the data.json file provided by prairielearn parsed into a dictionary. The student's submission data will be contained under the key 'submitted_answers'
• ROOT_DIR is the root directory of the question as it was loaded into the autograder (i.e. /grade/)
• SUBMISSION_DIR is the directory into which the files the function must produce must be written as per step 7 of the grading process