Skip to content

Commit

Permalink
Fix hdf5 read with broken links (#904)
Browse files Browse the repository at this point in the history
* Update hdf5.py

Resolves issue with loading Dectris-compressed data (lzf compression if I'm not mistaken).

* Update pyproject.toml

Added import of hdf5plugin

* Update hdf5.py

ignore flake8 here.

* Update hdf5.py

Fighting flake8

* Update CHANGELOG.md

Fixed the years of previous changelog entries, and added mine.

* Update CHANGELOG.md

Fixing a trailing whitespace for flake8's pleasure.

* fix to pass broken external HDF5 links less destructively.

* Update CHANGELOG.md

message on the graceful handling of broken external links in HDF5 files.

* getting flake8 to pass in vs code first.

* flake8 is more strict on the CI/CD end.

* Todo note added with reference to the PR detailing the issue
  • Loading branch information
toqduj authored Mar 11, 2025
1 parent 4bc8b6b commit ed90338
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
Write the date in place of the "Unreleased" in the case a new version is released. -->
# Changelog


## 0.1.0-b20 (2025-03-07)

### Added
Expand All @@ -14,6 +15,7 @@ Write the date in place of the "Unreleased" in the case a new version is release
- Added an hdf5plugin import to handle reading lzf-compressed data from Dectris Eiger HDF5 files.
- Removed no-op `?include_data_sources=false` (which is the default) from some
requests issued by the Python client.
- Added a try-except statement to gracefully skip over broken external links in HDF5 files.

### Maintenance

Expand Down
10 changes: 9 additions & 1 deletion tiled/adapters/hdf5.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,15 @@ def __iter__(self) -> Iterator[Any]:
yield from self._file

def __getitem__(self, key: str) -> Union["HDF5Adapter", ArrayAdapter]:
value = self._file[key]
# TODO: Handle broken external links better in the future. See tiled PR #904.
try:
value = self._file[key]
except KeyError as e:
warnings.warn(
f"KeyError: {e}, probably broken external link. Returning warning as value:"
)
return from_dataset(numpy.array([f"KeyError: {e}"]))

if isinstance(value, h5py.Group):
return HDF5Adapter(value)
else:
Expand Down

0 comments on commit ed90338

Please sign in to comment.