Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add automated tests #24

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

akloeckner
Copy link
Contributor

I thought, I'd try and contribute even if it's just a triviality. So, I configured and tested a GitHub action to run tests on changes in main and weekly. See #20 .

The action was proposed by GitHub. It might not be appropriate. So, please just close, if that is the case.

I'm especially unsure about the tox environments. They seem to have a matrix on dependencies which I don't know how to reproduce in GitHub actions. Also, tox seems to test against py36, which is no longer supported by the GitHub python setup action. (It's also outdated. So, maybe drop it?)

I just chose one of the proposed GH actions and changed from pytest to tox. Let's see if that works. ;-)
More current versions
Add weekly trigger
Copy link
Contributor

@1Maxnet1 1Maxnet1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script looks good to me. However some of the tests do not pass reliably. As this is probably not due to the code but rather sometimes server issues may occur, I wouldn't add pytest yet or change the execution to only execute local tests. But only in my humble opinion.

.github/workflows/python-package.yml Outdated Show resolved Hide resolved
@1Maxnet1
Copy link
Contributor

Additionally to get them into sync, I would propose to update the python versions in tox.ini accordingly.

@akloeckner
Copy link
Contributor Author

only execute local tests

Does that mean, exclude the tests in the request folders?

@1Maxnet1
Copy link
Contributor

only execute local tests

Does that mean, exclude the tests in the request folders?

Yep. These should definitely pass at any time.

checking once working local tests weekly doesn't make sense to me.
=> remove weeekly tests

removing actual api tests doesn't make sense to me either.
=> if a test fails in a PR, it should just be retriggered.
@akloeckner
Copy link
Contributor Author

akloeckner commented Jun 30, 2023

I updated the Python versions to 3.8 trough 3.11.

I also removed the weekly trigger, but kept the remote API access tests, because:

  1. I think,those remote API things should be tested for each PR. If those tests fail occasionally, they can be retriggered.
  2. Since the API accesses are not tested each week anymore, they will not bother us unexpectedly.
  3. It is important to test the actual API access for each PR.
  4. I dont see the use to re-test the local tests weekly. The only reason for them to fail could be faulty, new Python core code. And this should not really be happening much.

What do you think?

@1Maxnet1
Copy link
Contributor

1Maxnet1 commented Jul 3, 2023

I updated the Python versions to 3.8 trough 3.11.

I also removed the weekly trigger, but kept the remote API access tests, because:

1. I think,those remote API things should be tested for each PR. If those tests fail occasionally, they can be retriggered.

2. Since the API accesses are not tested each week anymore, they will not bother us unexpectedly.

3. It is important to test the actual API access for each PR.

4. I dont see the use to re-test the local tests weekly. The only reason for them to fail could be faulty, new Python core code. And this should not really be happening much.

What do you think?

In my opinion this is a good solution for the problem 👍

@akloeckner
Copy link
Contributor Author

Great! One more thing, I'd like to check is whether we can use tox itself for the automated tests. I tried at first but had some difficulty related to Python versions. May be it will work now.

@akloeckner
Copy link
Contributor Author

akloeckner commented Jul 6, 2023

I think, I got the GitHub actions working with tox. The issues were actually with Python 3.10 compatibility in the dependencies. After all, spotting those things, that's what continuous tests are for...

So, I updated the dependencies to requests>=2.28 and pytz>=2022.1 .

See also psf/requests@8bce583 and stub42/pytz@2ed682a .

Sorry for the messy commit log.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants