Verify and implement latest and all functionality for all endpoints

[jordanpadams]

💡 Description

Need to update this for consistency across the API

⚔️ Sub-tasks

• As a user, I want to only return the latest version of a product, by default, in all search results #426
• As a user, I want to search for past versions of a product where the LID changed during the product history #427
• As a user, I want to get the latest version of a product given a lid #435

Engineering Discussion

I believe we narrowed the all versions functionality to only show up when someone specifies /products/{identifier}/all, since this is a very specialized use case for trying to track down provenance of a product.

From a user perspective, rarely do end users actually want to search for and/or find old versions of products. They are irrelevant to current science use cases, and we do not want to present that to the users. Newer versions indicate better metadata and better data. This is why the default functionality for all endpoints (except this specialized one) should be only return the latest versions (those without a superseded_by specified).

This is one of the conversations @tloubrieu-jpl and I were having about functionality in the API that we could potentially abstract away using an OpenSearch query template (or something else)? Since we want this built into all queries of the API by default (except for the very specialized /products/{identifier}/all, we should figure out a way to ensure it is always there. For instance, in our legacy solr schema, we have these RequestHandler endpoints that response with pre-filtered results: https://github.com/NASA-PDS/registry-mgr-legacy/blob/main/src/main/resources/collections/data/solrconfig.xml#L817

It looks like OpenSearch/ElasticSearch may use query templates for this? https://sematext.com/blog/parametrizing-queries-in-solr-and-elasticsearch/

[jordanpadams]

📆 05/2024 status: In work. Delayed 1 sprint due to delays in NASA-PDS/registry#185. No impact on build expected

[jordanpadams]

@tloubrieu-jpl has this and/or will this be implemented in the improvements you are performing now?

[jordanpadams]

📆 06/2024 status: delayed another sprint per NASA-PDS/registry#185