refactor(api): refactor purl API query code #2900
Conversation
There was a problem hiding this comment.
very nice! Is there a way we can do some kind of testing to make sure we're not introducing any regressions on matching behaviour?
i.e. some kind of comprehensive comparison between test (with this deployed) and prod (without this deployed) for specific ecosystems?
We have integration tests for the API (LONG_TESTS), which I have manually checked. These tests do not produce any 500 errors. Also, after checking in the test instance, it will be tested by performance testing, and I will monitor for any new 500 errors. I think this change should be safe. |
| '200:{"commit": "d374094d8c49b6b7d288f307e11217ec5a502391", "package": {"purl": "pkg:pypi/[email protected]", "version": "0.8.5"}}\n', | ||
| '200:{"commit": "d374094d8c49b6b7d288f307e11217ec5a502391", "package": {"purl": "pkg:pypi/[email protected]"}}\n', | ||
| '200:{"commit": "d374094d8c49b6b7d288f307e11217ec5a502391", "package": {"version": "0.8.5"}}\n', | ||
| [ '400:{"commit": "d374094d8c49b6b7d288f307e11217ec5a502391", "package": ' |
There was a problem hiding this comment.
This long test has been skipped and hasn't been updated in a long time (it is also failing for the master branch). The script is generated using the command TESTS_GENERATE=1. Most of the 400 failures are caused by querying both the PURL and package name at the same time, which is invalid.
We could also re-enable long tests on the pipeline (run after merge), it doesn't run too long (maybe 1 min)
Converted PURL queries to package queries in
do_query()to remove unnecessary code. Querying the Datastore directly with PURLs gives the same results as querying by package name, so we can simplify things by treating PURLs like regular package queries after extracting thepackage/ecosystem/version.This will also resolve issue #2842 by rewriting the PURL-to-ecosystem logic. For most ecosystems, we can get the name from
purl.type, but for Linux distributions, we need to usepurl.namespace.