gh-156122: Allow lone surrogates in module filenames for crossinterp

[AdityaM06]

• Issue: Segfault/debug abort because _PyModule_GetFilenameUTF8 asserts module filenames must be utf-8 (called by crossinterp) #156122

The Problem

When cross-interpreter calls pickle arguments via _PyPickle_GetXIData(), _set_pickle_xid_context() invokes _Py_GetMainfile() -> _PyModule_GetFilenameUTF8() to record __main__.__file__ in case the module needs to be re-executed upon unpickling.

_PyModule_GetFilenameUTF8() previously used PyUnicode_AsUTF8AndSize(), which strictly rejects surrogate code points. If __main__.__file__ contained a lone surrogate (e.g. from an undecodable filesystem path like \udcff or \ud800), PyUnicode_AsUTF8AndSize() failed with UnicodeEncodeError and left size = -1, resulting in an assertion failure (assert(size >= 0)) in debug builds or undefined behavior in release builds.

The Solution

• Updated _PyModule_GetFilenameUTF8() in Objects/moduleobject.c to encode filenames using PyUnicode_EncodeFSDefault() instead of PyUnicode_AsUTF8AndSize(), matching standard filesystem encoding handling with surrogateescape.
• Added defensive NULL checking on the resulting bytes object: if encoding fails, size remains -1 and the caller _set_pickle_xid_context() safely clears the exception with PyErr_Clear().
• Ensured balanced reference counts and strict diff minimization.

Adding Tests & Verification

• Added test_surrogate_filename_in___main__ in Lib/test/test_interpreters/test_api.py covering both:
  • \udcff (low surrogate / PEP 383 surrogateescape range).
  • \ud800 (high surrogate / unencodable error-handling path).
• Verified with CPython's reference leak detector (./python.exe -m test -v -R 3:3 test_interpreters -m test_surrogate_filename_in___main__ -> 0 leaks).
• Added corresponding NEWS entry under Misc/NEWS.d/next/Core_and_Builtins/.

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3721552b8e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Preserve a decodable path for re-executing main

On POSIX, a real filename containing an undecodable byte is exposed as (for example) \udcff; PyUnicode_EncodeFSDefault() converts that surrogate back to raw 0xff, not UTF-8. When a __main__ function that uses globals is unpickled in the target interpreter, runpy_run_path() later passes this buffer through Py_BuildValue("sOs") (Python/crossinterp.c:44), which strictly decodes it as UTF-8 and raises UnicodeDecodeError. Thus cross-interpreter calls from scripts with surrogate-escaped filenames still fail when they need the existing re-execution path; retain a Unicode/UTF-8-surrogatepass representation through that handoff (or pass a Unicode filename object) instead of filesystem bytes.

Useful? React with 👍 / 👎.