Provide more metadata for HDF5 types #85
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
axelboc
commented
Jan 24, 2024
| @@ -123,7 +123,74 @@ def parse_slice_member(slice_member: str) -> Union[slice, int]: | |||
|
|
|||
|
|
|||
| def sorted_dict(*args: Tuple[str, Any]): | |||
| return dict(sorted(args)) | |||
| return dict(sorted(args, key=lambda entry: entry[0])) | |||
There was a problem hiding this comment.
Sorting on the first item of each tuple only (i.e. the future dictionary keys) to avoid errors with nested dictionaries.
t20100
approved these changes
Jan 24, 2024
|
I've added a test that covers all the advanced types (vlen, array, ref, enum, etc.) and I've updated the API schema. |
axelboc
commented
Jan 25, 2024
axelboc
commented
Jan 25, 2024
axelboc
force-pushed
the
dtype
branch
3 times, most recently
from
47c65de to
69ac664
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new snapshot test of
H5GroveApiin H5Web reveals that a lot of advanced HDF5 types are incorrectly parsed or simply marked as unknown.This is due to the fact that h5grove's
/meta/endpoint currently provides only a very basicdtypestring (or dict ofdtypestrings for compound datasets), which is not sufficient and sometimes inaccurate (e.g.|Ofor variable-length ASCII strings instead of|S).Following h5py's low-level API and taking inspiration from h5wasm, I modify the dataset/attribute metadata response to include a
typedictionary with all the interesting information about the dataset/attribute's HDF5 type.Unfortunately, the
typeproperty was already used for the entity kind ("dataset", "group", etc.). I've renamed the existing property tokindto match what we do in H5Web. I've also moved the previousdtypeproperty inside the newtypedictionary (I think it's still worth keeping, as some applications may prefer to avoid parsing the full type metadata). Of course, these are both breaking changes for h5grove, which is not ideal. I'm open to suggestions if that's a no-go.