-
Notifications
You must be signed in to change notification settings - Fork 18
improve behavior in case of contradictory data #130
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
- Added `sub_block_directory_info_policy_` to `CCZIReader` to control sub-block directory information prioritization. - Updated `OpenOptions` with a new enumeration for sub-block directory policies: `SubBlockDirectoryPrecedence`, `SubBlockHeaderPrecedence`, and `IgnoreDiscrepancy`. - Implemented discrepancy checks in `ReadSubBlock` to enforce consistency between sub-block directory and header information, throwing exceptions when necessary. - Introduced a new error code, `SubBlockDirectoryToSubBlockHeaderMismatch`, in `LibCZICZIParseException` for discrepancy detection. - Added a test case, `CheckThatSubBlockInfoFromSubBlockDirectoryIsAuthorative`, to validate the new functionality. - Made minor improvements to `CMemOutputStream`, including removing `virtual` from the destructor and adding an override specifier to the `Write` method.
- Removed commented-out code in `CCZIReader::ReadSubBlock` to streamline property assignments from `subBlkData`. - Added `CreateCziDocumentOneSubblock4x4Gray8` function in `test_reader.cpp` to facilitate the creation of a CZI document with a 4x4 grayscale subblock. - Updated the test `CheckThatSubBlockInfoFromSubBlockDirectoryIsAuthorative` to utilize the new function for generating CZI documents, ensuring it verifies the expected content correctly.
Renamed the test case `CheckThatSubBlockInfoFromSubBlockDirectoryIsAuthorative` to `CheckThatSubBlockInfoFromSubBlockDirectoryIsAuthorativeNoException` to reflect its updated focus. Added a new test case `CheckThatExceptionIsThrownWhenEnabledIfSubBlockDirectoryAndSubblockHeaderDiffer` to verify exception handling for mismatches between sub-block directory and header. Updated the setting of `subBlockDirectoryInfoPolicy` to use `static_cast` for improved type safety. Additional assertions were included to validate the expected exception behavior.
Modified the `ReadSubBlock` method to improve discrepancy checks between sub-block directory and header information, making the directory information authoritative by default. Added a new enumeration `SubBlockDirectoryInfoPolicy` in `libCZI.h` to define handling policies for discrepancies, along with operator overloads for flag manipulation. Updated test cases in `test_reader.cpp` to reflect these changes, ensuring accurate retrieval of information and added a test to verify exception handling for discrepancies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
Src/libCZI/CZIReader.cpp:318
- Consider using the overloaded operator& for SubBlockDirectoryInfoPolicy instead of casting to uint8_t. For example, rewrite the condition as 'if (!(this->sub_block_directory_info_policy_ & OpenOptions::SubBlockDirectoryInfoPolicy::IgnoreDiscrepancy))' for improved readability and consistency.
if (!((uint8_t)this->sub_block_directory_info_policy_ & (uint8_t)OpenOptions::SubBlockDirectoryInfoPolicy::IgnoreDiscrepancy))
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #130 +/- ##
==========================================
+ Coverage 65.88% 66.57% +0.68%
==========================================
Files 86 89 +3
Lines 11004 11227 +223
==========================================
+ Hits 7250 7474 +224
+ Misses 3754 3753 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
- Modified `CreateBitmapFromSubBlock_Uncompressed` to accept a new parameter for handling uncompressed data size mismatches. - Updated `CreateBitmapFromSubBlock` to utilize the new `CreateBitmapOptions` structure, allowing for better control over bitmap creation options. - Introduced `CreateBitmapOptions` struct in the `libCZI` namespace to manage various bitmap creation options. - Refactored `CreateBitmap` method in `CCziSubBlock` to accept the new options parameter. - Added test cases to ensure correct behavior when sub-block data is too short, verifying that bitmaps are filled with zeroes as expected. - Improved code clarity and maintainability by removing commented-out code and streamlining logic.
Modified Decode method signatures to accept pointers for pixel type, width, and height, improving flexibility. Introduced a new parameter in CreateBitmapFromSubBlock_JpgXr to manage bitmap size mismatches. Updated comments and documentation for clarity. Adjusted test cases to align with new signatures and added tests for handling subblock size mismatches. Enhanced error handling and logging for better robustness.
- Removed explicit pixel type check in `CreateBitmapFromSubBlock_JpgXr`, allowing for more flexible bitmap decoding. - Updated `CBitmapOperations::CopyWithOffsetInfo` to use the pixel type from the decoded bitmap. - Cleaned up error handling in `CJxrLibDecoder::Decode` by removing commented-out code. - Standardized variable names in `test_reader.cpp` for consistency. - Replaced direct `Lock()` calls with `ScopedBitmapLockerSP` in tests for better resource management. - Added new test cases for handling subblocks with mismatched pixel types. - Made minor formatting adjustments for improved code readability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 22 out of 22 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
Src/libCZI/CZIReader.cpp:334
- [nitpick] The use of the magic constant '1' for checking the flag value reduces clarity; consider replacing it with a well-named constant or an enum method to improve readability.
if (((uint8_t)this->sub_block_directory_info_policy_ & 1) == (uint8_t)OpenOptions::SubBlockDirectoryInfoPolicy::SubBlockDirectoryPrecedence)
Updated logic to manage discrepancies between sub-block-directory and sub-block-header using configuration options. Enhanced checks to throw exceptions on mismatches in pixel type, compression, coordinates, and dimensions. Simplified precedence condition for sub-block-directory information using a bitmask for clarity.
- Introduced `decoder_zstd.h` for Zstd compressed data handling. - Enhanced `CreateBitmapFromSubBlock_JpgXr` and `CreateBitmapFromSubBlock_ZStd0` with new parameters for better mismatch handling. - Updated `Decode` methods in relevant decoders to accept additional arguments for customization. - Added utility functions for token matching in `decoder_zstd.cpp`. - Created `CreateBitmapOptions` structure to manage bitmap size discrepancy options. - Updated tests in `test_reader.cpp` to validate new Zstd functionality. - Made minor formatting and documentation improvements.
Updated `libCZI_Site.h` to add detailed documentation for the `Decode` method, clarifying parameter usage and introducing a convenience overload. In `subblock_cache.h`, replaced existing atomic types with `std::atomic<std::uint64_t>` and `std::atomic<std::uint32_t>` for improved type safety and clarity.
Updated `CreateBitmapFromSubBlock_ZStd0` to use correct physical sizes. Introduced new functions in `decoder_zstd.cpp` for better handling of Zstd data size mismatches. Refactored `IsTokenMatch` to utilize the new decoding functions. Updated tests in `test_reader.cpp` to reflect changes in exception handling and added new tests for size mismatch scenarios.
- Updated `CreateBitmapFromSubBlock_ZStd0` and `ZStd1` to use references for `SubBlockInfo`, improving performance. - Added a parameter for handling data size mismatches in `CreateBitmapFromSubBlock_ZStd1`. - Modified `libCZI::CreateBitmapFromSubBlock` to accommodate the new parameter. - Introduced new functions in `decoder_zstd.cpp` for size mismatch handling and HiLo byte unpacking with error checks. - Added new test cases in `test_reader.cpp` for Zstd1 compressed data size scenarios, including HiLo byte packing. - Renamed existing tests to better reflect changes in resolution protocol handling.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 23 out of 23 changed files in this pull request and generated no comments.
Comments suppressed due to low confidence (3)
Src/libCZI/decoder_zstd.cpp:75
- [nitpick] The function 'DecodeAndHiLoBytePackRequireCorrectSize' uses 'BytePack' in its name, while a related function uses 'ByteUnpack'. Consider renaming it for consistency if the operation performed is actually unpacking.
shared_ptr<libCZI::IBitmapData> DecodeAndHiLoBytePackRequireCorrectSize(const void* ptrData, size_t size, libCZI::PixelType pixelType, uint32_t width, uint32_t height)
Src/libCZI/CZIReader.cpp:317
- The strict mismatch check between sub-block directory and header information may be too rigid. Verify that this condition fully reflects the intended resolution protocol and consider adding more granular logging to aid debugging in case of expected discrepancies.
if (entry.PixelType != subBlkData.pixelType || entry.Compression != subBlkData.compression || ... )
Src/libCZI/CreateBitmap.cpp:130
- Ensure that the logic for handling insufficient uncompressed data correctly calculates the remaining size and zero-fills incomplete rows. A detailed review of the loop logic and the 'remaining_size' updates is recommended to avoid potential data corruption.
if (expected_size <= size) { ... } else { /* zero-fill logic */ }
Renamed `DecodeAndHiLoByteUnpackAndHandleSizeMismatch` to `DecodeAndHiLoBytePackAndHandleSizeMismatch` to reflect its new functionality. Commented out several unused processing functions in `decoder_zstd.cpp`. Updated pointer casting in `test_reader.cpp` for improved type safety and added new test cases to validate exception handling for Zstd1 compression edge cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 24 out of 24 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
Src/libCZI/CZIReader.cpp:317
- [nitpick] Consider refactoring the bitwise flag comparison for clarity, such as by introducing a helper function or a boolean variable to improve readability.
if (static_cast<std::underlying_type<OpenOptions::SubBlockDirectoryInfoPolicy>::type>(this->sub_block_directory_info_policy_ & OpenOptions::SubBlockDirectoryInfoPolicy::IgnoreDiscrepancy) == 0)
Updated the `libCZI` project version from `0.63.2` to `0.64.0` in `CMakeLists.txt` and added a corresponding entry in `version-history.markdown` for the new "Resolution Protocol for Ambiguous or Contradictory Information". Introduced new overloads for the `Decode` method in `CJxrLibDecoder` and `CZstd1Decoder`, simplifying the interface by removing pointer requirements for pixel type, width, and height. Updated tests in `test_JxrlibCodec.cpp`, `test_ZstdDecode.cpp`, and `test_writer.cpp` to utilize the new method signatures, ensuring they validate expected behavior while improving code readability and consistency across the decoding interface.
Updated variable names for consistency, changing `ptrData` and `pixelType` to `ptr_data` and `pixel_type`. Enhanced error messages for better clarity regarding zstd-compressed data size issues. Refactored code for improved readability and maintainability, utilizing `GetZstdContentSizeOrThrow` for size checks. Standardized handling of decompressed sizes to ensure consistent references, enhancing the overall robustness of the decoding functions.
Included `./libCZI/Doc/resolution_protocol.markdown` in the list of source files for Doxygen processing to enhance documentation coverage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 22 out of 24 changed files in this pull request and generated 1 comment.
Files not reviewed (2)
- CMakeLists.txt: Language not supported
- Src/Doxyfile: Language not supported
Comments suppressed due to low confidence (2)
Src/libCZI/CreateBitmap.cpp:140
- [nitpick] Consider renaming 'remaining_size' to 'data_remaining' to more clearly indicate its purpose.
size_t remaining_size = size;
Src/libCZI/CZIReader.cpp:317
- [nitpick] Consider extracting the bitmask check into a named variable or helper function to improve readability.
if (static_cast<std::underlying_type<OpenOptions::SubBlockDirectoryInfoPolicy>::type>(this->sub_block_directory_info_policy_ & OpenOptions::SubBlockDirectoryInfoPolicy::IgnoreDiscrepancy) == 0)
Co-authored-by: Copilot <[email protected]>
Introduced a new function `DecompressAndThrowIfError` to encapsulate decompression and error handling. This change replaces inline decompression logic in multiple locations, enhancing code readability and maintainability. Additionally, updated comments and parameter descriptions to clarify expected behavior and requirements.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Approved w/ recommendations/questions
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Co-authored-by: DaveyJonesBitPail <[email protected]>
Description
This PR aims to define "expected, correct and canonical behavior" for cases where contradictory information is found in a CZI-document. The goal is to have defined and consistent behavior even in cases where a malformed CZI is processed.
According to this definition, libCZI is modified to conform to this specification.
Type of change
How Has This Been Tested?
locally
Checklist: