Skip to content

feat: add support for private requirements #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
andrey-canon wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add support for private requirements #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 42 additions & 14 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,19 @@ The path to the fixtures file of your plugin to load initial data for the tests.
The path to the Python file in your plugin that contains the test function for validating Open edX imports. This path is relative to your plugin directory.
* *Example*: `"eox_test/edxapp_wrapper/test_backends.py"`

### `ssh_private_key`

**Optional**
A private SSH key (usually stored as a GitHub Secret) that allows cloning of private repositories containing additional Open edX requirements.
* *Example*: `${{ secrets.SERVICE_USER_SSH_KEY }}`
Copy link
Contributor

@magajh magajh Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remember to update this with the correct secret name


### `openedx_private_requirements`

**Optional**
A space-separated list of private Git repositories to be cloned and installed inside LMS and CMS. Each entry must specify the repository URL and optionally a branch/tag.
* *Example*: `"[email protected]:org/repo1.git@main [email protected]:org/[email protected]"`


## Overview

This GitHub Action automates the process of setting up a Tutor Open edX environment to test your plugin. It performs the following steps:
Expand Down Expand Up @@ -127,29 +140,44 @@ specified in `tutor_plugins` input.

10. **Launch Tutor**: Launches the Open edX platform using Tutor with the specified configuration.

11. **Add Mount for Plugin**: Mounts your plugin into the LMS and CMS containers.
11. **Setup SSH Agent for Private Repositories** *(Optional)*:
If `ssh_private_key` is provided, configures an SSH agent (`webfactory/ssh-agent`) to authenticate against private repositories.

12. **Add GitHub to Known Hosts** *(Optional)*:
Ensures that `github.com` is trusted for SSH connections, preventing host authenticity errors during cloning.

13. **Clone Private Requirements** *(Optional)*:
If `openedx_private_requirements` is provided, clones each specified repository into a `private_requirements` directory with correct permissions.

14. **Add Mount for Private Requirements** *(Optional)*:
Mounts the `private_requirements` directory into both LMS and CMS containers.

15. **Add Mount for Plugin**: Mounts your plugin into the LMS and CMS containers.

16. **Recreate Containers**: Recreates the LMS and CMS containers to apply the changes.

12. **Recreate Containers**: Recreates the LMS and CMS containers to apply the changes.
17. **Install Private Requirements inside LMS and CMS** *(Optional)*:
Runs `pip install` for each cloned repository inside both LMS and CMS environments.

13. **Install Open edX Plugin as an Editable Package**: Installs your plugin in editable mode inside both LMS and CMS containers.
18. **Install Open edX Plugin as an Editable Package**: Installs your plugin in editable mode inside both LMS and CMS containers.

14. **Copy Inline Tutor Plugins to the Plugins Folder**: Copies the inline Tutor plugins of your plugin to the plugins folder.
19. **Copy Inline Tutor Plugins to the Plugins Folder**: Copies the inline Tutor plugins of your plugin to the plugins folder.

15. **Enable Inline Tutor Plugins**: Installs the inline Tutor plugins inside the plugins folder.
20. **Enable Inline Tutor Plugins**: Installs the inline Tutor plugins inside the plugins folder.

16. **Install Extra Requirements**: Installs any additional Python packages specified in `openedx_extra_pip_requirements`.
21. **Install Extra Requirements**: Installs any additional Python packages specified in `openedx_extra_pip_requirements`.

17. **Run Migrations and Restart Services**: Applies database migrations and restarts Tutor services.
22. **Run Migrations and Restart Services**: Applies database migrations and restarts Tutor services.

18. **Import Demo Course**: Imports the Open edX demo course for testing purposes.
23. **Import Demo Course**: Imports the Open edX demo course for testing purposes.

19. **Test Open edX Imports in Plugin** *(Optional)*: Runs pytest to validate Open edX imports in your plugin if `openedx_imports_test_file_path` is provided. The only two dependencies installed to run these tests are `pytest` and `pytest-django`, so ensure that your import tests do not require any extra packages that are not in the plugin's base requirements.
24. **Test Open edX Imports in Plugin** *(Optional)*: Runs pytest to validate Open edX imports in your plugin if `openedx_imports_test_file_path` is provided. The only two dependencies installed to run these tests are `pytest` and `pytest-django`, so ensure that your import tests do not require any extra packages that are not in the plugin's base requirements.

20. **Load Initial Data for the Tests** *(Optional)*: Loads initial data from a fixtures file into the LMS if `fixtures_file` is provided.
25. **Load Initial Data for the Tests** *(Optional)*: Loads initial data from a fixtures file into the LMS if `fixtures_file` is provided.

21. **Check LMS Heartbeat**: Verifies that the LMS is running by hitting the heartbeat endpoint.
26. **Check LMS Heartbeat**: Verifies that the LMS is running by hitting the heartbeat endpoint.

22. **Set `DEMO_COURSE_ID` Environment Variable**:
27. **Set `DEMO_COURSE_ID` Environment Variable**:
Sets the `DEMO_COURSE_ID` environment variable based on the Tutor version. This variable allows you to refer to the demo course in your tests, which can be helpful when you need to interact with course content during testing.

**Usage in Your Tests**:
Expand All @@ -162,9 +190,9 @@ specified in `tutor_plugins` input.
# Use DEMO_COURSE_ID in your tests
```

23. **Run Extra Tutor Commands** *(Optional)*: Executes the shell script specified in `tutor_extra_commands_path` to run additional Tutor commands after installing Tutor.
28. **Run Extra Tutor Commands** *(Optional)*: Executes the shell script specified in `tutor_extra_commands_path` to run additional Tutor commands after installing Tutor.

24. **Run Integration Tests**: Activates the test virtual environment and runs your integration tests using the specified shell script.
29. **Run Integration Tests**: Activates the test virtual environment and runs your integration tests using the specified shell script.

## Notes

Expand Down
68 changes: 68 additions & 0 deletions action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ inputs:
openedx_imports_test_file_path:
description: 'Path to the file that contains the test function for validating Open edX imports. This should be a Python file within your project.'
required: false
ssh_private_key:
Copy link
Contributor

@magajh magajh Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's please make this a secret instead of a regular input

description: 'Service user SSH key for repository checkout'
required: false
openedx_private_requirements:
description: "Optional list of SSH Git repos to install inside Open edX. E.g: '[email protected]:org/repo1.git@main [email protected]:org/[email protected]'"
required: false

runs:
using: 'composite'
Expand Down Expand Up @@ -115,6 +121,49 @@ runs:
tutor local launch -I
shell: bash

- name: Setup SSH agent for private repositories cloning
if: ${{ inputs.ssh_private_key }}
Copy link
Contributor

@magajh magajh Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be a secret

Copy link
Contributor Author

@andrey-canon andrey-canon Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's not possible

https://stackoverflow.com/questions/70098241/using-secrets-in-composite-actions-github/70111134?utm_source=chatgpt.com

https://github.com/actions/runner/blob/main/docs/adrs/1144-composite-actions.md?plain=1#L20

uses: webfactory/[email protected]
with:
ssh-private-key: ${{ inputs.ssh_private_key }}

- name: Add GitHub to known hosts
run: |
ssh-keyscan github.com >> ~/.ssh/known_hosts
shell: bash

- name: Clone Open edX private pip requirements
if: ${{ inputs.openedx_private_requirements }}
run: |
mkdir -m 777 -p private_requirements
cd private_requirements

for entry in ${{ inputs.openedx_private_requirements }}; do
repo_url="${entry%@*}"
branch="${entry##*@}"

echo "Repo URL: $repo_url"
echo "Branch/Tag: $branch"

if [ "$repo_url" = "$entry" ]; then
git clone "$repo_url"
else
git clone --branch "$branch" "$repo_url"
fi
done

chmod -R a+rwX .
shell: bash

- name: Add mount for private pip requirements
if: ${{ inputs.openedx_private_requirements }}
run: |
source .tutor_venv/bin/activate

tutor mounts add "lms:$GITHUB_WORKSPACE/private_requirements:/openedx/private_requirements"
tutor mounts add "cms:$GITHUB_WORKSPACE/private_requirements:/openedx/private_requirements"
shell: bash

- name: Add mount for Open edX plugin
run: |
source .tutor_venv/bin/activate
Expand All @@ -130,6 +179,25 @@ runs:
tutor local start -d lms cms
shell: bash

- name: Pip install private requirements inside LMS and CMS
Copy link
Contributor

@magajh magajh Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please move this step to before or after the step where we install the extra requirements

integration-test-in-tutor/action.yml

Line 227 in a5cb1cc

- name: Install extra requirements

Copy link
Contributor Author

@andrey-canon andrey-canon Nov 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? anyway, that will make that some implementations fail, there are two options

  1. you need a private requirement that is not an app requirement, this will work with your suggestion
  2. you need a private requirement that is an app requirement, when the app(plugin) is installed in the next step the installation will fail since that won't be able to install the private dependency

if: ${{ inputs.openedx_private_requirements }}
run: |
source .tutor_venv/bin/activate

tutor local exec lms bash -c '
for d in /openedx/private_requirements/*; do
echo "Installing $d..."
pip install "$d"
done
'
tutor local exec cms bash -c '
for d in /openedx/private_requirements/*; do
echo "Installing $d..."
pip install "$d"
done
'
shell: bash

- name: Install Open edX plugin as an editable package
run: |
source .tutor_venv/bin/activate
Expand Down