[test](be) Add storage index lifecycle test fixture#64348
Draft
eldenmoon wants to merge 2 commits into
Draft
Conversation
### What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: This adds a focused BE unit-test fixture for storage index lifecycle coverage around inverted index probe events, patched tablet schemas, variant sparse/nested metadata, and COUNT_ON_INDEX no-need-read-data behavior. The tests cover index add/drop and build/drop on text and variant path indexes, missing required variant columns, V1/V2 index storage files, rowset map updates after schema patching, and red/green validation that COUNT_ON_INDEX reads only rows filtered by an applied inverted index. The implementation records index probe events for applied, fallback, and not-attempted column predicates, preserves existing SegmentIterator debug points, skips missing physical columns only when nullable/default columns make that valid, and resolves field_pattern variant index build/drop against materialized paths.
### Release note
None
### Check List (For Author)
- Test: Unit Test
- `PATH=/tmp/codex-clang-format16:$PATH build-support/clang-format.sh`
- `PATH=/tmp/codex-clang-format16:$PATH build-support/check-format.sh`
- `git diff --check && git diff --cached --check`
- `env DORIS_CLANG_HOME=/mnt/disk1/claude-max/ldb_toolchain20 DORIS_THIRDPARTY=/mnt/disk1/claude-max/tmp/doris/thirdparty ./run-be-ut.sh --run --filter='IndexStorageLifecycleTest.*:IndexStorageSchemaPatchTest.*'`
- Attempted `env CLANG_TIDY_BINARY=/mnt/disk1/claude-max/ldb_toolchain20/bin/clang-tidy build-support/run-clang-tidy.sh --build-dir be/ut_build_ASAN`, but local clang-tidy could not analyze Doris headers due toolchain/PCH diagnostics (`stddef.h` not found from `slice.h` and existing `core/types.h` NOLINT parsing). A clang-tidy-16 probe also failed with incompatible PCH AST.
- Behavior changed: No
- Does this need documentation: No
Member
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29590 ms |
Contributor
TPC-DS: Total hot run time: 169390 ms |
### What problem does this PR solve? Issue Number: close apache#64205 Related PR: apache#64205 Problem Summary: Add a focused BE unit test that reproduces the variant sparse stats limit reader bug. The test writes a variant column with one materialized path, an exact sparse path v.b, and a hidden child path v.b.c that is omitted from sparse path statistics after the configured limit is reached. Reading v.b through an explicit path-column schema currently returns NULL for the row whose value only exists under b.c, exposing the missing hierarchical read behavior without applying the production fix. ### Release note None ### Check List (For Author) - Test: Unit Test - Ran Target system: Linux; Target arch: x86_64 Python 3.11.13 Check JAVA_HOME version Apache Maven 3.9.12 (848fbb4bf2d427b72bdb2471c22fced7ebd9a7a1) Maven home: /usr/local Java version: 17.0.16, vendor: Oracle Corporation, runtime: /mnt/disk6/common/jdk-17.0.16 Default locale: en_US, platform encoding: UTF-8 OS name: "linux", version: "5.14.0-694.el9.x86_64", arch: "amd64", family: "unix" cmake version 3.31.8 CMake suite maintained and supported by Kitware (kitware.com/cmake). ccache version 4.5.1 Get params: PARALLEL -- 39 CLEAN -- 0 ENABLE_PCH -- ON EXTRA_BE_MODULES -- Build Backend UT install datasketches-cpp to thirdparty path before build backend ut Update datasketches-cpp submodule ... -- Configuring done (0.0s) -- Generating done (0.0s) -- Build files have been written to: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/contrib/datasketches-cpp/build/Release Install the project... -- Install configuration: "Release" -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/version.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/binomial_bounds.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bounds_binomial_proportions.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/ceiling_power_of_2.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/common_defs.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/conditional_back_inserter.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/conditional_forward.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/count_zeros.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/inv_pow2_table.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kolmogorov_smirnov_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kolmogorov_smirnov.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/memory_operations.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/MurmurHash3.h -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/optional.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/quantiles_sorted_view_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/quantiles_sorted_view.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/serde.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/xxhash64.h -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/hll.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/AuxHashMap.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CompositeInterpolationXTable.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/hll.private.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllSketchImplFactory.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CouponHashSet.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CouponList.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CubicInterpolation.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HarmonicNumbers.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll4Array.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll6Array.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll8Array.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllArray.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllSketchImpl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllUtil.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/coupon_iterator.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/RelativeErrorTables.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/AuxHashMap-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CompositeInterpolationXTable-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CouponHashSet-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CouponList-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/CubicInterpolation-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HarmonicNumbers-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll4Array-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll6Array-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/Hll8Array-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllArray-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllSketch-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllSketchImpl-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/HllUnion-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/coupon_iterator-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/RelativeErrorTables-internal.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/compression_data.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_common.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_compressor.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_compressor_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_confidence.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_union.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_union_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/cpc_util.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/icon_estimator.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kxp_byte_lookup.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/u32_table.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/u32_table_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kll_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kll_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kll_helper.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/kll_helper_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/frequent_items_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/frequent_items_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/reverse_purge_hash_map.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/reverse_purge_hash_map_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_union.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_union_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_intersection.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_intersection_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_a_not_b.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_a_not_b_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_jaccard_similarity.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_comparators.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_constants.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_helpers.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_update_sketch_base.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_update_sketch_base_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_union_base.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_union_base_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_intersection_base.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_intersection_base_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_set_difference_base.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_set_difference_base_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/theta_jaccard_similarity_base.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bounds_on_ratios_in_sampled_sets.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bounds_on_ratios_in_theta_sketched_sets.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/compact_theta_sketch_parser.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/compact_theta_sketch_parser_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bit_packing.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/var_opt_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/var_opt_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/var_opt_union.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/var_opt_union_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/ebpps_sample.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/ebpps_sample_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/ebpps_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/ebpps_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_union.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_union_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_intersection.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_intersection_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_a_not_b.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_a_not_b_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tuple_jaccard_similarity.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_of_doubles_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_union.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_union_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_intersection.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_intersection_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_a_not_b.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/array_tuple_a_not_b_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/req_common.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/req_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/req_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/req_compactor.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/req_compactor_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/quantiles_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/quantiles_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/count_min.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/count_min_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/density_sketch.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/density_sketch_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tdigest.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/tdigest_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bloom_filter.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bloom_filter_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bloom_filter_builder_impl.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/include/DataSketches/bit_array_ops.hpp -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/lib/DataSketches/cmake/DataSketches.cmake -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/lib/DataSketches/cmake/DataSketchesConfigVersion.cmake -- Installing: /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/lib/DataSketches/cmake/DataSketchesConfig.cmake Update apache-orc submodule ... Update clucene submodule ... -- Make program: /usr/bin/make -- Use ccache: -DCMAKE_CXX_COMPILER_LAUNCHER=ccache and -DCMAKE_C_COMPILER_LAUNCHER=ccache -- Extra cxx flags: -- GLIBC_COMPATIBILITY is ON -- USE_LIBCPP is OFF -- USE_JEMALLOC is OFF -- USE_UNWIND is ON -- ENABLE_PCH is ON -- USE_AVX2 is ON -- Build type is ASAN_UT -- Build target arch is x86_64 -- DORIS_HOME is /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework -- THIRDPARTY_DIR is /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed -- make test: ON -- make benchmark: OFF -- build fs benchmark tool: OFF -- build task executor simulator: OFF -- build file cache lru tool: OFF -- build gensrc if necessary make: Entering directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc' make -C script make[1]: Entering directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/script' /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/script/gen_build_version.sh make[1]: Leaving directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/script' make -C proto make[1]: Entering directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/proto' make[1]: Nothing to be done for 'all'. make[1]: Leaving directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/proto' make -C thrift make[1]: Entering directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift' /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/thirdparty/installed/bin/thrift -I /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift -I /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift/../build//thrift/ --gen cpp:moveable_types,no_skeleton -out /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift/../build//gen_cpp --allow-64bit-consts -strict /mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift/Normalization.thrift make[1]: Leaving directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc/thrift' make: Leaving directory '/mnt/disk1/claude-max/tmp/doris-variant-gtest-framework/gensrc' -- Configuring incomplete, errors occurred!; expected failure before the apache#64205 fix: v.b values are scalar-0, scalar-1, scalar-2, scalar-3, NULL and do not include child-0. - Behavior changed: No - Does this need documentation: No
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: N/A
Related PR: N/A
Problem Summary: This adds a focused BE unit-test fixture for storage index lifecycle coverage around inverted index probe events, patched tablet schemas, variant sparse/nested metadata, and COUNT_ON_INDEX no-need-read-data behavior. The tests cover index add/drop and build/drop on text and variant path indexes, missing required variant columns, V1/V2 index storage files, rowset map updates after schema patching, and red/green validation that COUNT_ON_INDEX reads only rows filtered by an applied inverted index. The implementation records index probe events for applied, fallback, and not-attempted column predicates, preserves existing SegmentIterator debug points, skips missing physical columns only when nullable/default columns make that valid, and resolves field_pattern variant index build/drop against materialized paths.
Release note
None
Check List (For Author)
PATH=/tmp/codex-clang-format16:$PATH build-support/clang-format.shPATH=/tmp/codex-clang-format16:$PATH build-support/check-format.shgit diff --check && git diff --cached --checkenv DORIS_CLANG_HOME=/mnt/disk1/claude-max/ldb_toolchain20 DORIS_THIRDPARTY=/mnt/disk1/claude-max/tmp/doris/thirdparty ./run-be-ut.sh --run --filter='IndexStorageLifecycleTest.*:IndexStorageSchemaPatchTest.*'env CLANG_TIDY_BINARY=/mnt/disk1/claude-max/ldb_toolchain20/bin/clang-tidy build-support/run-clang-tidy.sh --build-dir be/ut_build_ASAN, but local clang-tidy could not analyze Doris headers due toolchain/PCH diagnostics (stddef.hnot found fromslice.hand existingcore/types.hNOLINT parsing). A clang-tidy-16 probe also failed with incompatible PCH AST.