Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

migrate pgk_resources to importlib_resources #1968

Merged
merged 10 commits into from
Jul 31, 2023

Conversation

zormit
Copy link
Contributor

@zormit zormit commented Jul 21, 2023

Motivation

While playing around with the codebase, I saw a warning pop up (weirdly I'm having trouble to reproduce it right now) that pgk_resources is deprecated. It's not an important thing, but I took it as a small first issue to get a bit used to the codebase and the contribution mechanism again :)

Approach

When following the link in the warning, you'll be able to find a migration guide, which I partly followed.

Side-Effects

This probably closes #1911, because the dependency on pkg_resources is gone.

Open Questions

  • I'm not sure whether I should have called importlib.resources.as_file in some of the places.
  • It's a bit tricky because the state of things differs in the different python versions. I still have to find the right fix for that.

@CLAassistant
Copy link

CLAassistant commented Jul 21, 2023

CLA assistant check
All committers have signed the CLA.

@zormit zormit changed the title [WIP] migrate pgk_resources to importlib [WIP] migrate pgk_resources to importlib_resources Jul 21, 2023
@zormit zormit changed the title [WIP] migrate pgk_resources to importlib_resources migrate pgk_resources to importlib_resources Jul 21, 2023
@zormit zormit changed the title migrate pgk_resources to importlib_resources [WIP] migrate pgk_resources to importlib_resources Jul 21, 2023
@saulpw saulpw requested a review from anjakefala July 21, 2023 20:29
@zormit
Copy link
Contributor Author

zormit commented Jul 21, 2023

@saulpw @anjakefala before I try to fix the version problems... what do you think of this in general?

regarding the version problems...
do you know why import_resources would not exist in 3.10, while it seems to work again in 3.11 (not sure though, the test was cancelled... 🤔 )? It is probably somehow an implicit dependency that got dropped from the providing package(s)?

@zormit
Copy link
Contributor Author

zormit commented Jul 24, 2023

weirdly I'm having trouble to reproduce it right now

Ok I realised my mistake. The warning only pops up when launching via the installed bin/vd:

% vd
/Users/mn/.venv/visidata/bin/vd:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  __import__('pkg_resources').require('visidata==2.12.dev0')
saul.pw/VisiData v2.12dev

I'm not even sure how this file is generated, but I'd be curious to find out & maybe also remove the pkg_resources dependency from there. Probably through updating the underlying deployment dependencies, so I opened some kind of rabbit hole here...

@anjakefala
Copy link
Collaborator

Ahhh, thank you for tackling this @zormit! And researching what needed to change. I think bin/vd is generated by some installers.

I learned that importlib_resources is actually referred to as importlib.resources wrt the stdlib: https://docs.python.org/3/library/importlib.resources.html. I made the change, and tested locally. Let's see how the build looks!

@anjakefala
Copy link
Collaborator

Failing now because importlib.resources.files is new in Python 3.9! Hmmmm.

@anjakefala
Copy link
Collaborator

anjakefala commented Jul 27, 2023

Seems like we need a try/except to use importlib.resources.files > Python 3.9, and importlib_resources.files in 3.8, 3.7: https://stackoverflow.com/questions/6028000/how-to-read-a-static-file-from-inside-a-python-package. Which is a bit gross. We would also need to then install importlib_resources in 3.7, 3.8.

@anjakefala
Copy link
Collaborator

Upon further research, it seems like pkgutil can possibly give us what we want without the version challenges with importlib.resources: https://docs.python.org/3/library/pkgutil.html

@anjakefala anjakefala force-pushed the migrate-pgk-resources-to-importlib branch from f118bfa to 38d4d5f Compare July 31, 2023 03:42
@anjakefala
Copy link
Collaborator

I decided to go with importlib.resources and importlib_resources. It seemed like the change that was closest to pkg_resources, and when we drop support for 3.7 and 3.8, we will be in a good position.

@anjakefala anjakefala force-pushed the migrate-pgk-resources-to-importlib branch from 38d4d5f to 451f52c Compare July 31, 2023 03:46
@anjakefala anjakefala marked this pull request as ready for review July 31, 2023 03:50
@anjakefala anjakefala changed the title [WIP] migrate pgk_resources to importlib_resources migrate pgk_resources to importlib_resources Jul 31, 2023
Copy link
Owner

@saulpw saulpw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @zormit and @anjakefala ! If everything seems to work, this is good.

@anjakefala anjakefala merged commit 7d327dc into saulpw:develop Jul 31, 2023
13 checks passed
@zormit zormit deleted the migrate-pgk-resources-to-importlib branch July 31, 2023 08:34
@zormit
Copy link
Contributor Author

zormit commented Jul 31, 2023

Thanks for the review @anjakefala and pulling it through! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(Unnecessary) hidden dependency on setuptools; debian package broken
4 participants