Skip to content

Preserve string slice function return types#23330

Draft
xudong963 wants to merge 1 commit into
apache:mainfrom
xudong963:fix/string-slice-return-types
Draft

Preserve string slice function return types#23330
xudong963 wants to merge 1 commit into
apache:mainfrom
xudong963:fix/string-slice-return-types

Conversation

@xudong963

@xudong963 xudong963 commented Jul 6, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Rationale for this change

left, right, and substr currently always return Utf8View, even when their first argument is Utf8 or LargeUtf8. This makes them inconsistent with nearby string-producing functions such as lower, upper, reverse, translate, and substr_index, which preserve the first argument's string type.

The unconditional Utf8View return type also means callers that intentionally avoid view types cannot keep Utf8 flowing through these functions without explicit casts back to Utf8, even when they explicitly opt out of view types with datafusion.sql_parser.map_string_types_to_utf8view=false and datafusion.execution.parquet.schema_force_view_types=false.

This PR revisits the performance tradeoff from #21441 / #21442 and makes these functions follow the same input-driven return-type contract as the other string functions.

What changes are included in this PR?

  • Change left, right, and substr return type inference to use the first argument's string type.
  • Return StringArray / LargeStringArray for Utf8 / LargeUtf8 inputs.
  • Keep the optimized StringViewArray path for Utf8View inputs.
  • Update unit tests and sqllogictests to cover Utf8, LargeUtf8, and Utf8View return types.

Are these changes tested?

Yes. I ran:

cargo fmt --all
cargo test -p datafusion-functions unicode::left::tests::test_functions
cargo test -p datafusion-functions unicode::right::tests::test_functions
cargo test -p datafusion-functions unicode::substr::tests::test_functions
cargo test -p datafusion-functions unicode::substr::tests::test_sliced_string_array_array_args
cargo test -p datafusion-sqllogictest --test sqllogictests -- string/string.slt --test-threads 1
cargo test -p datafusion-sqllogictest --test sqllogictests -- string/string_view.slt string/large_string.slt string/dictionary_utf8.slt --test-threads 1
cargo clippy -p datafusion-functions --lib --tests -- -D warnings

Are there any user-facing changes?

Yes. left, right, and substr now preserve the first argument's string type:

  • Utf8 -> Utf8
  • LargeUtf8 -> LargeUtf8
  • Utf8View -> Utf8View

This may remove implicit downstream casts for users that expect Utf8, but it also means Utf8 / LargeUtf8 inputs no longer use the previous zero-copy Utf8View result representation.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Jul 6, 2026
@xudong963 xudong963 marked this pull request as draft July 6, 2026 03:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

left, right, and substr always return Utf8View for Utf8 input

1 participant