feat: include inspector package urls as part of the malicious metadata facts for pypi packages

[art1f1c3R]

To allow for retention of packages when they are taken off PyPI, this new feature includes the inspector.pypi.io URL for the distribution files as part of the MaliciousMetadataFacts detail_information field. This has been done by modifying the wheel_absence.py heuristic to instead of returning filenames, return the python hosted URL and corresponding pypi inspector URL. The unit test for this, test_wheel_absence.py has been updated to reflect these changes accordingly.

[art1f1c3R]

One change I have made that I am unsure about is, in wheel_absence.py:91 if _valid_url returns False, then currently I add an empty string. I am not sure what action should be taken if the constructed inspector link is not accessible.

The reason the check for a 200 OK response exists is due to the way I have constructed the pypi inspector link. I observe that the format of the pypi inspector link was the same as files.pythonhosted.org except replacing this prefix with the inspector URL, followed by the package name and then version. Everything else after was the same (using blake2b_256 and the filename), only this was only an observation and I haven't managed to find this documented anywhere, so I added in that check.

[behnazh-w]

One change I have made that I am unsure about is, in wheel_absence.py:91 if _valid_url returns False, then currently I add an empty string. I am not sure what action should be taken if the constructed inspector link is not accessible.

The reason the check for a 200 OK response exists is due to the way I have constructed the pypi inspector link. I observe that the format of the pypi inspector link was the same as files.pythonhosted.org except replacing this prefix with the inspector URL, followed by the package name and then version. Everything else after was the same (using blake2b_256 and the filename), only this was only an observation and I haven't managed to find this documented anywhere, so I added in that check.

I have added a suggestion in this comment to improve the return value. If the inspector HEAD request fails, the value corresponding to the inspector could be None.

[art1f1c3R]

Took a look at the source code for inspector, and I have confirmed the way it organises its URLs is what is used in this new feature. At https://github.com/pypi/inspector/blob/main/inspector/main.py line 125:

@app.route(
    "/project/<project_name>/<version>/packages/<first>/<second>/<rest>/<distname>/"
)

[tromai]

LGTM. Thanks.