Invalid OPTIMADE API responses

[ml-evs]

Describe the bug
The responses from the JARVIS-DFT OPTIMADE API have several outstanding issues, as reported by the dashboard and pasted into this gist (https://gist.github.com/ml-evs/89f7bacac1c8a266074b82c89b78a11a).

I can't find a repo for the actual API (the code is not in this repo or https://github.com/usnistgov/JARVIS-DFT). Is there anywhere I can help fix this?

[knc6]

Hi @ml-evs
Thanks for checking this.
The JARVIS web-app is maintained using a private repo and is not part of the jarvis-tools package necessarily.
I have kept a copy of jarvis_optimade in the private repo: https://github.com/knc6/jarvis_optimade
and added you as a collaborator. Please feel free to check out.

Currently, after a few bug fixes (not pushed to production yet), I see the following output

Testing entire implementation at http://localhost:8000/optimade/jarvisdft/v1
.....✖......✖✖✖.✖.✖✖✖✖✖✖✖✖✖✖✖✖✖......

FAILURES
========

http://localhost:8000/optimade/jarvisdft/v1 - _test_page_limit - failed with error
        ResponseError: Field `meta->more_data_available` was missing.
http://localhost:8000/optimade/jarvisdft/v1/structures?page_limit=5&response_fields=assemblies,chemical_formula_hill,last_modified,chemical_formula_descriptive,elements,dimension_types,structure_features,species_at_sites,nsites,nperiodic_dimensions,species,chemical_formula_anonymous,nelements,chemical_formula_reduced,cartesian_site_positions,elements_ratios,immutable_id,lattice_vectors - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta
          field required (type=value_error.missing)
http://localhost:8000/optimade/jarvisdft/v1/info/structures - _test_must_properties - failed with error
        ResponseError: Some 'MUST' properties were missing from info/structures: {'structure_features'}
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=id STARTS "JVASP-13526" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=id STARTS "JVASP-13526".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=last_modified >= "2022-02-03T17:14:39.696262+00:00" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=last_modified >= "2022-02-03T17:14:39.696262+00:00".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=chemical_formula_reduced STARTS "Br3Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=chemical_formula_reduced STARTS "Br3Y".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=chemical_formula_anonymous STARTS "A3B" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=chemical_formula_anonymous STARTS "A3B".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=nelements = 16 - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=nelements = 16.
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=elements HAS ANY "Br","Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=elements HAS ANY "Br","Y".
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=nsites = 16 - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=nsites = 16.
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_optb88vdw_bandgap - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_optb88vdw_bandgap' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_mbj_bandgap - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_mbj_bandgap' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_formation_energy_peratom - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_formation_energy_peratom' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_stability - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_stability' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/; testing queries for structures->_jarvisdft_spacegroup - _construct_queries_for_property - failed with error
        ResponseError: Found unknown field '_jarvisdft_spacegroup' in `/info/structures` and no provider prefix was provided in `/info`
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=_crazyprovider_field=2 - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta
          field required (type=value_error.missing)


OPTIONAL TEST FAILURES
======================

http://localhost:8000/optimade/jarvisdft/versions - _test_versions_headers - failed with error
        ResponseError: Incorrect 'Content-Type' header 'text/plain'.
        Missing at least one expected parameter(s): ['header=present']
http://localhost:8000/optimade/jarvisdft/v1/structures?filter=species_at_sites HAS ANY "Br","Y" - _construct_queries_for_property - failed with error
        ResponseError: Required field `meta->more_data_available` missing from response for structures?filter=species_at_sites HAS ANY "Br","Y".


Passed 17 out of 33 tests.
Additionally passed 2 out of 4 optional tests.

[ml-evs]

Hi @knc6, this is causing some issues again for us as we try to integrate the NIST Materials Resource registry with OPTIMADE. Have the changes above been pushed to production yet? The main thing we are currently running into is the lack of meta and links fields in the structure responses, which means you cannot paginate through the results at all.

[knc6]

Hi @ml-evs , we just pushed the recent updates to the server.

[ml-evs]

Hi @knc6, looks like some of the problems I mentioned have been fixed (around the static metadata) but pagination does not work (and filtering is not implemented?). Perhaps you could take a look at the PR I made at https://github.com/knc6/jarvis_optimade/pull/1/?

[ml-evs]

Here's what I see with the latest validator:

$ optimade-validator https://jarvis.nist.gov/optimade/jarvisdft/v1
Testing entire implementation at https://jarvis.nist.gov/optimade/jarvisdft/v1
............✖✖✖✖.✖..✖✖✖.✖.✖......

FAILURES
========

https://jarvis.nist.gov/optimade/jarvisdft/v1 - _test_page_limit - failed with error
        ResponseError: Endpoint did not obey page limit: 5 entries vs 4 limit
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?page_limit=4&response_fields=structure_features,dimension_types,elements_ratios,nelements,assemblies,immutable_id,chemical_formula_descriptive,nsites,chemical_formula_hill,nperiodic_dimensions,last_modified,species,lattice_vectors,species_at_sites,chemical_formula_anonymous,elements,chemical_formula_reduced,cartesian_site_positions - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta -> api_version
          Unable to validate the version string '1.1' as a semantic version (expected <major>.<minor>.<patch>).See https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string for more information. (type=value_error)
https://jarvis.nist.gov/optimade/jarvisdft/v1/info/structures - _test_must_properties - failed with error
        ResponseError: Some 'MUST' properties were missing from info/structures: {'structure_features'}
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?response_fields=species,chemical_formula_anonymous,_jarvis_reference&page_limit=1 - _check_response_fields - failed with error
        ResponseError: Response fields not obeyed by 'structures':
        Expected: {'species', 'chemical_formula_anonymous', '_jarvis_reference'}
        Returned: {'_jarvis_source', 'nperiodic_dimensions', 'structure_features', 'dimension_types', 'last_modified', 'species', 'elements_ratios', '_jarvis_reference', 'chemical_formula_descriptive', 'lattice_vectors', 'nelements', 'species_at_sites', 'chemical_formula_anonymous', 'elements', 'chemical_formula_reduced', 'cartesian_site_positions', 'nsites'}
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=id < "JVASP-6838" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'id': JVASP-6838 was not excluded by 'id < "JVASP-6838"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=chemical_formula_reduced < "GaTe" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'chemical_formula_reduced': GaTe was not excluded by 'chemical_formula_reduced < "GaTe"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=chemical_formula_anonymous < "AB" - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'chemical_formula_anonymous': AB was not excluded by 'chemical_formula_anonymous < "AB"'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=nelements < 4 - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'nelements': 4 was not excluded by 'nelements < 4'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=nsites < 4 - _construct_queries_for_property - failed with error
        ResponseError: Entry JVASP-6838 with value 'nsites': 4 was not excluded by 'nsites < 4'
https://jarvis.nist.gov/optimade/jarvisdft/v1/structures?filter=_crazyprovider_field=2 - _deserialize_response - failed with error
        ValidationError: 1 validation error for ValidatorStructureResponseMany
        meta -> api_version
          Unable to validate the version string '1.1' as a semantic version (expected <major>.<minor>.<patch>).See https://semver.org/#is-there-a-suggested-regular-expression-regex-to-check-a-semver-string for more information. (type=value_error)