branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)#62961
Merged
yiguolei merged 1 commit intobranch-4.1from May 1, 2026
Merged
branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)#62961yiguolei merged 1 commit intobranch-4.1from
yiguolei merged 1 commit intobranch-4.1from
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
There was a problem hiding this comment.
Pull request overview
Refactors Doris BE casting/serde/conversion code to remove the legacy util/io_helper indirection and route parsing/formatting through the canonical CastTo* implementations.
Changes:
- Delete
be/src/util/io_helper.{h,cpp}and replace call sites withCastToInt/CastToFloat/CastToDecimal/CastToStringand datelike cast implementations. - Extend cast helpers (notably
CastToString128-bit overloads) and update converters/serdes to use the new entry points. - Add/extend BE unit tests covering 128-bit string formatting, DecimalV2 cast behavior, datelike conversions, Arrow parsing, and sequence-match time conditions.
Reviewed changes
Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| be/test/exprs/function/cast/cast_to_string_api_test.cpp | Adds tests for new CastToString 128-bit overload behavior. |
| be/test/exprs/function/cast/cast_to_float_double.cpp | Adds regression test ensuring DecimalV2 preserves scale when casting to float/double. |
| be/test/exprs/aggregate/vec_sequence_match_test.cpp | Adds tests for time-condition parsing + serialize/deserialize and malformed numeric conditions. |
| be/test/exec/column_type_convert_test.cpp | Updates largeint formatting to CastToString and adds string→datelike conversion tests. |
| be/test/core/field_test.cpp | Replaces io_helper include with CastToString. |
| be/test/core/data_type_serde/data_type_serde_number_test.cpp | Adds Arrow string→(DateV2/DateTimeV2) numeric parsing coverage. |
| be/test/core/data_type_serde/data_type_serde_decimal_test.cpp | Adds JSON deserialize test to preserve underflow compatibility. |
| be/test/common/check_overflow.cpp | Switches integer parsing from io_helper helpers to CastToInt. |
| be/src/util/jsonb_document_cast.h | Routes decimal→decimal casting via CastToDecimal::from_decimal. |
| be/src/util/io_helper.h | Removes legacy helper header. |
| be/src/util/io_helper.cpp | Removes legacy helper implementation. |
| be/src/util/counts.h | Drops io_helper include. |
| be/src/storage/olap_scan_common.h | Drops io_helper include. |
| be/src/format/column_type_convert.h | Replaces io_helper parsing with CastTo* APIs and threads CastParameters through string-cast helpers. |
| be/src/exprs/function/function_json.cpp | Drops io_helper include. |
| be/src/exprs/function/cast/cast_to_timestamptz.h | Drops io_helper include. |
| be/src/exprs/function/cast/cast_to_string.h | Adds explicit from_int128/from_uint128 helpers and wires them into from_number. |
| be/src/exprs/function/cast/cast_to_jsonb.h | Drops io_helper include. |
| be/src/exprs/function/cast/cast_to_int.h | Uses CastToInt::from_decimal and simplifies decimal overflow/nullability handling. |
| be/src/exprs/function/cast/cast_to_float.h | Uses CastToFloat::from_decimal for decimal→float/double conversions. |
| be/src/exprs/function/cast/cast_to_decimal.h | Replaces io_helper decimal parsing and introduces from_decimal wrapper for decimal→decimal casts. |
| be/src/exprs/function/cast/cast_to_boolean.h | Replaces bool parsing with StringParser::string_to_bool. |
| be/src/exprs/function/cast/cast_to_basic_number_common.h | Centralizes numeric parsing via StringParser and provides shared int/float cast helpers. |
| be/src/exprs/function/cast/cast_base.h | Drops io_helper include from cast base utilities. |
| be/src/exprs/aggregate/moments.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_uniq.h | Drops io_helper include; adds needed column/string headers. |
| be/src/exprs/aggregate/aggregate_function_topn.h | Drops io_helper include; adds needed column/string headers. |
| be/src/exprs/aggregate/aggregate_function_sequence_match.h | Replaces integer parsing with StringParser, improves parse-failure reset behavior. |
| be/src/exprs/aggregate/aggregate_function_rpc.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_product.h | Adjusts debug logging to format int128_t via CastToString. |
| be/src/exprs/aggregate/aggregate_function_min_max.h | Drops io_helper include; adds decimal column header. |
| be/src/exprs/aggregate/aggregate_function_java_udaf.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_histogram.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_corr.h | Drops io_helper include; adds standard headers required by implementation. |
| be/src/exprs/aggregate/aggregate_function_collect.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_bit.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_avg_weighted.h | Drops io_helper include. |
| be/src/exprs/aggregate/aggregate_function_array_agg.h | Drops io_helper include. |
| be/src/exec/common/histogram_helpers.hpp | Drops io_helper include. |
| be/src/exec/common/hash_table/hash_table.h | Drops io_helper include. |
| be/src/core/field.cpp | Drops io_helper include. |
| be/src/core/data_type_serde/data_type_serde.h | Updates comment to reference CastToDecimal::from_string. |
| be/src/core/data_type_serde/data_type_number_serde.cpp | Switches parsing/formatting to CastTo* APIs (incl. IPv6 formatting). |
| be/src/core/data_type_serde/data_type_ipv6_serde.cpp | Uses CastToIPv6::from_string instead of io_helper reader. |
| be/src/core/data_type_serde/data_type_ipv4_serde.cpp | Uses CastToIPv4::from_string instead of io_helper reader. |
| be/src/core/data_type_serde/data_type_decimal_serde.cpp | Replaces decimal JSON parsing with StringParser-based paths. |
| be/src/core/data_type_serde/data_type_datev2_serde.cpp | Uses CastToDateV2 parsing instead of io_helper reader. |
| be/src/core/data_type_serde/data_type_datetimev2_serde.cpp | Uses CastToDatetimeV2 parsing instead of io_helper reader. |
| be/src/core/data_type_serde/data_type_date_or_datetime_serde.cpp | Uses CastToDateOrDatetime parsing and explicit date/datetime post-processing. |
| be/src/core/data_type/data_type_quantilestate.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_number_base.cpp | Uses CastToString for 128-bit number formatting. |
| be/src/core/data_type/data_type_ipv6.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_ipv4.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_hll.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_decimal.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_date_time.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_date_or_datetime_v2.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_date.cpp | Drops io_helper include. |
| be/src/core/data_type/data_type_bitmap.cpp | Drops io_helper include. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+211
to
+218
| StringParser::ParseResult result = StringParser::PARSE_SUCCESS; | ||
| if constexpr (T == TYPE_DECIMALV2) { | ||
| val = DecimalV2Value(StringParser::string_to_decimal<TYPE_DECIMALV2>( | ||
| str_ref.data, cast_set<Int32>(str_ref.size), DecimalV2Value::PRECISION, | ||
| DecimalV2Value::SCALE, &result)); | ||
| } else { | ||
| val.value = StringParser::string_to_decimal<T>(str_ref.data, cast_set<Int32>(str_ref.size), | ||
| precision, scale, &result); |
Comment on lines
41
to
44
| #include "exprs/function/cast/cast_parameters.h" | ||
| #include "exprs/function/function.h" | ||
| #include "exprs/function/function_helpers.h" | ||
| #include "util/io_helper.h" | ||
|
|
Contributor
Author
|
run buildall |
…#62179) Problem Summary: Remove the legacy io_helper wrapper layer and move its parsing and formatting entry points onto the existing cast implementations so serde and conversion code depends on the real owners of that behavior.
Contributor
Author
|
run buildall |
yiguolei
approved these changes
May 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
pick #62179