Skip to content

branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)#62961

Merged
yiguolei merged 1 commit intobranch-4.1from
io_helper_41
May 1, 2026
Merged

branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179)#62961
yiguolei merged 1 commit intobranch-4.1from
io_helper_41

Conversation

@zclllyybb
Copy link
Copy Markdown
Contributor

pick #62179

Copilot AI review requested due to automatic review settings April 30, 2026 03:07
@zclllyybb zclllyybb requested a review from yiguolei as a code owner April 30, 2026 03:07
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zclllyybb zclllyybb changed the title [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179) branch-4.1: [refactor](be) Remove io_helper helper indirection from BE cast paths (#62179) Apr 30, 2026
@zclllyybb
Copy link
Copy Markdown
Contributor Author

run buildall

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors Doris BE casting/serde/conversion code to remove the legacy util/io_helper indirection and route parsing/formatting through the canonical CastTo* implementations.

Changes:

  • Delete be/src/util/io_helper.{h,cpp} and replace call sites with CastToInt/CastToFloat/CastToDecimal/CastToString and datelike cast implementations.
  • Extend cast helpers (notably CastToString 128-bit overloads) and update converters/serdes to use the new entry points.
  • Add/extend BE unit tests covering 128-bit string formatting, DecimalV2 cast behavior, datelike conversions, Arrow parsing, and sequence-match time conditions.

Reviewed changes

Copilot reviewed 59 out of 59 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
be/test/exprs/function/cast/cast_to_string_api_test.cpp Adds tests for new CastToString 128-bit overload behavior.
be/test/exprs/function/cast/cast_to_float_double.cpp Adds regression test ensuring DecimalV2 preserves scale when casting to float/double.
be/test/exprs/aggregate/vec_sequence_match_test.cpp Adds tests for time-condition parsing + serialize/deserialize and malformed numeric conditions.
be/test/exec/column_type_convert_test.cpp Updates largeint formatting to CastToString and adds string→datelike conversion tests.
be/test/core/field_test.cpp Replaces io_helper include with CastToString.
be/test/core/data_type_serde/data_type_serde_number_test.cpp Adds Arrow string→(DateV2/DateTimeV2) numeric parsing coverage.
be/test/core/data_type_serde/data_type_serde_decimal_test.cpp Adds JSON deserialize test to preserve underflow compatibility.
be/test/common/check_overflow.cpp Switches integer parsing from io_helper helpers to CastToInt.
be/src/util/jsonb_document_cast.h Routes decimal→decimal casting via CastToDecimal::from_decimal.
be/src/util/io_helper.h Removes legacy helper header.
be/src/util/io_helper.cpp Removes legacy helper implementation.
be/src/util/counts.h Drops io_helper include.
be/src/storage/olap_scan_common.h Drops io_helper include.
be/src/format/column_type_convert.h Replaces io_helper parsing with CastTo* APIs and threads CastParameters through string-cast helpers.
be/src/exprs/function/function_json.cpp Drops io_helper include.
be/src/exprs/function/cast/cast_to_timestamptz.h Drops io_helper include.
be/src/exprs/function/cast/cast_to_string.h Adds explicit from_int128/from_uint128 helpers and wires them into from_number.
be/src/exprs/function/cast/cast_to_jsonb.h Drops io_helper include.
be/src/exprs/function/cast/cast_to_int.h Uses CastToInt::from_decimal and simplifies decimal overflow/nullability handling.
be/src/exprs/function/cast/cast_to_float.h Uses CastToFloat::from_decimal for decimal→float/double conversions.
be/src/exprs/function/cast/cast_to_decimal.h Replaces io_helper decimal parsing and introduces from_decimal wrapper for decimal→decimal casts.
be/src/exprs/function/cast/cast_to_boolean.h Replaces bool parsing with StringParser::string_to_bool.
be/src/exprs/function/cast/cast_to_basic_number_common.h Centralizes numeric parsing via StringParser and provides shared int/float cast helpers.
be/src/exprs/function/cast/cast_base.h Drops io_helper include from cast base utilities.
be/src/exprs/aggregate/moments.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_uniq.h Drops io_helper include; adds needed column/string headers.
be/src/exprs/aggregate/aggregate_function_topn.h Drops io_helper include; adds needed column/string headers.
be/src/exprs/aggregate/aggregate_function_sequence_match.h Replaces integer parsing with StringParser, improves parse-failure reset behavior.
be/src/exprs/aggregate/aggregate_function_rpc.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_product.h Adjusts debug logging to format int128_t via CastToString.
be/src/exprs/aggregate/aggregate_function_min_max.h Drops io_helper include; adds decimal column header.
be/src/exprs/aggregate/aggregate_function_java_udaf.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_histogram.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_corr.h Drops io_helper include; adds standard headers required by implementation.
be/src/exprs/aggregate/aggregate_function_collect.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_bit.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_avg_weighted.h Drops io_helper include.
be/src/exprs/aggregate/aggregate_function_array_agg.h Drops io_helper include.
be/src/exec/common/histogram_helpers.hpp Drops io_helper include.
be/src/exec/common/hash_table/hash_table.h Drops io_helper include.
be/src/core/field.cpp Drops io_helper include.
be/src/core/data_type_serde/data_type_serde.h Updates comment to reference CastToDecimal::from_string.
be/src/core/data_type_serde/data_type_number_serde.cpp Switches parsing/formatting to CastTo* APIs (incl. IPv6 formatting).
be/src/core/data_type_serde/data_type_ipv6_serde.cpp Uses CastToIPv6::from_string instead of io_helper reader.
be/src/core/data_type_serde/data_type_ipv4_serde.cpp Uses CastToIPv4::from_string instead of io_helper reader.
be/src/core/data_type_serde/data_type_decimal_serde.cpp Replaces decimal JSON parsing with StringParser-based paths.
be/src/core/data_type_serde/data_type_datev2_serde.cpp Uses CastToDateV2 parsing instead of io_helper reader.
be/src/core/data_type_serde/data_type_datetimev2_serde.cpp Uses CastToDatetimeV2 parsing instead of io_helper reader.
be/src/core/data_type_serde/data_type_date_or_datetime_serde.cpp Uses CastToDateOrDatetime parsing and explicit date/datetime post-processing.
be/src/core/data_type/data_type_quantilestate.cpp Drops io_helper include.
be/src/core/data_type/data_type_number_base.cpp Uses CastToString for 128-bit number formatting.
be/src/core/data_type/data_type_ipv6.cpp Drops io_helper include.
be/src/core/data_type/data_type_ipv4.cpp Drops io_helper include.
be/src/core/data_type/data_type_hll.cpp Drops io_helper include.
be/src/core/data_type/data_type_decimal.cpp Drops io_helper include.
be/src/core/data_type/data_type_date_time.cpp Drops io_helper include.
be/src/core/data_type/data_type_date_or_datetime_v2.cpp Drops io_helper include.
be/src/core/data_type/data_type_date.cpp Drops io_helper include.
be/src/core/data_type/data_type_bitmap.cpp Drops io_helper include.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +211 to +218
StringParser::ParseResult result = StringParser::PARSE_SUCCESS;
if constexpr (T == TYPE_DECIMALV2) {
val = DecimalV2Value(StringParser::string_to_decimal<TYPE_DECIMALV2>(
str_ref.data, cast_set<Int32>(str_ref.size), DecimalV2Value::PRECISION,
DecimalV2Value::SCALE, &result));
} else {
val.value = StringParser::string_to_decimal<T>(str_ref.data, cast_set<Int32>(str_ref.size),
precision, scale, &result);
Comment on lines 41 to 44
#include "exprs/function/cast/cast_parameters.h"
#include "exprs/function/function.h"
#include "exprs/function/function_helpers.h"
#include "util/io_helper.h"

@zclllyybb
Copy link
Copy Markdown
Contributor Author

run buildall

…#62179)

Problem Summary: Remove the legacy io_helper wrapper layer and move its
parsing and formatting entry points onto the existing cast
implementations so serde and conversion code depends on the real owners
of that behavior.
@zclllyybb
Copy link
Copy Markdown
Contributor Author

run buildall

@yiguolei yiguolei merged commit f632c93 into branch-4.1 May 1, 2026
28 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants