Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression Testing of REST API #106

Open
IanMayo opened this issue Jul 17, 2023 · 5 comments
Open

Regression Testing of REST API #106

IanMayo opened this issue Jul 17, 2023 · 5 comments
Labels
enhancement New feature or request

Comments

@IanMayo
Copy link
Collaborator

IanMayo commented Jul 17, 2023

Is your feature request related to a problem? Please describe.

At this early point in its life Soul is going through lots of changes. It would be quite easy for the addition of a new feature, or implementation of a bug-fix to break some existing part of the API.

Soul developers could work with more confidence if there was an automated test-suite that verified the full API worked.

Describe the solution you'd like

Introduction of a GitHub action that runs a test-site across the full Soul REST API. This would run in each Pull Request, and give the maintainer(s) confidence that a new feature (or bug-fix) doesn't break any existing functionality.

Since Soul is a js project, a library such as Frisby may be an obvious choice.

I guess we'd build up a chinook.db instance that could be used for the unit testing. This would grow as required. So, when we were developing/reviewing the export views feature, we would have added a SQL VIEW to the database, to check it supported views.

Any other candidates for a library to run unit tests on the Soul REST API:?

Describe alternatives you've considered

  • Manual testing - that's the only current option.

Additional context
Add any other context or screenshots about the feature request here.

@IanMayo IanMayo added the enhancement New feature or request label Jul 17, 2023
@IanMayo
Copy link
Collaborator Author

IanMayo commented Jul 18, 2023

Ok, I've found this existing test content:
https://github.com/thevahidal/soul/blob/main/core/src/controllers/rows.test.js

This may have fallen behind the growth of new features, but we should be able to update/extend this test suite, and include it as a Pull Request action.

@thevahidal
Copy link
Owner

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind.
But to run tests in the pipeline is a must-have and I'll try to do it asap.

@thevahidal
Copy link
Owner

We should have an strict acceptance criteria to enforce tests on newly added features, and also should write tests for the parts that lack any.

@IanMayo
Copy link
Collaborator Author

IanMayo commented Jul 20, 2023

Yeah, you're completely right, Soul has tests right now but as you mentioned they might fall behind. But to run tests in the pipeline is a must-have and I'll try to do it asap.

Here is how to do it in a GitHub action:
https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

You'd then go into repo settings / branches, create a new config for develop, and require that status checks are complete.

@thevahidal
Copy link
Owner

Thanks for the link, I'll make it a priority.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants