[SPARK-57932][SQL][4.0] Fix regexp_instr and regexp_replace returning wrong results for supplementary characters

[sarutak]

What changes were proposed in this pull request?

This PR backports SPARK-57932 (#56998) for branch-4.0 with conflict resolution.

regexp_instr returned matcher.start() + 1 (a UTF-16 code-unit index), and regexp_replace's position argument was used as a UTF-16 offset in matcher.region(pos-1, ...) and its out-of-range guard. Both now use code-point positions via codePointCount/offsetByCodePoints.

Why are the changes needed?

Spark string positions are 1-based code points (instr, locate, position, substring, length). For strings containing supplementary characters (code points > U+FFFF, e.g. emojis), these two functions gave wrong results:

• regexp_instr('😀ab', 'ab') returned 3 instead of 2.
• regexp_replace('😀aXa', 'a', 'Z', 3) returned 😀ZXZ instead of 😀aXZ (replaced a match before position); regexp_replace('😀😀', '😀', 'Z', 3) returned 😀Z instead of the input unchanged. This is the sibling of the LIKE fix in SPARK-55453.

Does this PR introduce any user-facing change?

Yes. regexp_instr and regexp_replace now return code-point-consistent results for strings with supplementary characters.

How was this patch tested?

Added UT.

Was this patch authored or co-authored using generative AI tooling?

Yes.