Skip to content

Fix subprocess.Popen.communicate type conflicts via defensive input normalizer (_BinDataBox) - #130

Merged
dmitry-lipetsk merged 1 commit into
postgrespro:masterfrom
dmitry-lipetsk:D20260916_001--fix-issue-of-input
Sep 16, 2026
Merged

dmitry-lipetsk merged 1 commit into
postgrespro:masterfrom
dmitry-lipetsk:D20260916_001--fix-issue-of-input

Conversation

@dmitry-lipetsk

Copy link
Copy Markdown
Collaborator

Description

This PR resolves a fundamental architecture limitation in Python's standard library subprocess.Popen inside both OsOperations::run and OsProcessController::communicate layers. The issue manifests as a type conflict between textual and binary spaces when handling distributed pipelines that require a mixed execution model (e.g., streaming raw database dumps into standard input while capturing readable, decoded text logs via stdout/stderr).

The Problem

Python's subprocess assumes an all-or-nothing approach to I/O modes, causing heavy runtime crashes under two distinct scenarios:

  1. Binary Input in Text Mode (text=True):
    When streaming raw binary data (e.g., database file chunks) into a process configured in text mode to capture human-readable errors, standard library's internal _save_input() method blindly triggers .encode() on the target input:

    if input is not None and self.text_mode:
        self._input = self._input.encode(self.stdin.encoding, self.stdin.errors)

    If the incoming input consists of native bytes, this call crashes immediately with AttributeError: 'bytes' object has no attribute 'encode', disrupting binary integrity before reaching the network socket/pipe.

  2. Text Input in Binary Mode (text=False):
    Conversely, passing a string object into a binary execution pipeline triggers a delayed crash down the stack during the low-level selector chunking stage, where memoryview tries to map raw buffer regions:

    if self._input:
        input_view = memoryview(self._input)
    # TypeError: memoryview: a bytes-like object is required, not 'str'

The Solution: Industrial-Grade Duck Typing

To maintain total data integrity without corrupting surrogate binary structures (such as blobs or heavy ICU collations) through unnecessary round-trip string conversions, we introduced a highly resilient duck-typed dispatcher wrapper via Helpers::prepare_process_input:

  • _BinDataBox(bytes): A lightweight subclass of native bytes that retains identical low-level memory layout performance (no duplication overhead) but exposes a safe, self-returning .encode() shim. This seamlessly cheats Python's internal _save_input() verification logic in text mode.
  • Symmetric Type Normalization: String payloads intended for binary nodes are transparently compiled down to byte allocations via the active encoding profile before reaching the C-API boundary of memoryview, completely neutralising the threat of an implicit TypeError.

Code Architecture Shift

class _BinDataBox(bytes):
    def encode(self, *args, **kwargs) -> _BinDataBox:
        return self

class Helpers:
    @staticmethod
    def prepare_process_input(
        input: typing.Optional[typing.Union[str, bytes]],
        encoding: typing.Optional[str],
    ) -> typing.Optional[bytes]:
        assert encoding is None or type(encoding) is str

        if input is None:
            return None

        if type(input) is str:
            b = input.encode(encoding or __class__.get_default_encoding())
            return _BinDataBox(b)

        assert type(input) is bytes
        return _BinDataBox(input)

Impact & Verification

  • Covered edge cases spanning dual text/binary configurations without altering target asset structures or encoding mappings.
  • Passed full distributed validation checks (over 1200+ multi-node CI regressions).

@dmitry-lipetsk
dmitry-lipetsk merged commit f0621f0 into postgrespro:master Sep 16, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant