[BUG][Windows][Regression] bot_gateway_e2e.py still crashes cp1252 stdout after gateway #3475 fix

[Dhivya-Bharathy]

[BUG][Windows][Regression] bot_gateway_e2e.py still crashes cp1252 stdout after gateway #3475 fix

Metadata

Field	Value
Repository	https://github.com/MervinPraison/PraisonAI
Labels	bug, windows, praisonai-bot, e2e, developer-experience, regression
Severity	Medium — live browser E2E still unusable on default Windows console; masks Playwright install hint
Component	e2e-platform-audit/bot_gateway_e2e.py (Report.add, browser probe block)
Related (closed)	#3430, #3475
Reverification date	2026-07-29
Commit tested	e22dc0d17 (main)

---

Executive summary

Follow-up issue #3475 hardened praisonai_bot/gateway/unicode_utils.py (maps U+2551 box-drawing chars from Playwright stderr) and closed. However the E2E harness that developers run locally — e2e-platform-audit/bot_gateway_e2e.py — was not updated and still calls bare print() on failure details containing Playwright's framed playwright install banner.

On Windows with default cp1252 console encoding (PYTHONUTF8 unset), reverification 2026-07-29 shows:

• 10/14 live checks PASS (gateway, MCP, code)
• Browser control (playwright headless) FAIL — real error captured in JSON: playwright install
• Browser control via bot message FAIL — console shows 'charmap' codec can't encode character '\u2551' instead of actionable hint

This is a regression of incomplete fix scope: gateway server paths are safe; audit script paths are not.

---

Environment

Field	Value
OS	Windows 10 (10.0.19045)
Shell	PowerShell (legacy conhost)
Python	3.11.15 (uv) / 3.13.2
Console encoding	cp1252 (PYTHONUTF8 unset deliberately)
PraisonAI commit	e22dc0d17
praisonai-bot	0.0.42
praisonai-browser	0.0.9
Playwright Python package	Installed
Playwright browser binaries	Not installed (triggers install banner)
OpenAI API	Present

---

Step-by-step reproduction

git clone https://github.com/MervinPraison/PraisonAI.git
cd PraisonAI
pip install -e ".[all]"
# Deliberately skip: playwright install chromium

Remove-Item Env:PYTHONUTF8 -ErrorAction SilentlyContinue
$env:OPENAI_API_KEY = "<your-key>"
$env:PRAISONAI_AUTO_APPROVE = "true"

python e2e-platform-audit/bot_gateway_e2e.py --live-only

Expected behavior

• Browser checks FAIL with ASCII-readable message: Please run: playwright install
• Harness completes; JSON report and console agree on root cause
• Other live checks (gateway, MCP, code) PASS

Actual behavior (2026-07-29)

• Gateway, MCP, code checks PASS
• Console:

[FAIL] browser control via bot message — 'charmap' codec can't encode character '\u2551' in position 62: character maps to <undefined>
=== Done: 4 failed ===

• JSON report (UTF-8) preserves real Playwright stderr with \u2551 box chars — console lies, JSON tells truth

---

Full terminal evidence

Live E2E console (2026-07-29, commit e22dc0d17)

=== PraisonAI Bot Gateway E2E ===

[OK] import praisonai_bot — 0.0.42
[OK] import praisonai_code — 0.0.57
[OK] import praisonai_browser — 0.0.9
[OK] import praisonai_mcp — 0.0.9
[OK] import praisonai_sandbox — 0.0.7
[OK] OPENAI_API_KEY — present
[FAIL] docker sandbox live execute — docker daemon not reachable
[OK] gateway startup (praisonai-bot) — 127.0.0.1:18765
[OK] mcp gateway /mcp endpoint — agent tool listed (1 tools)
[OK] mcp tools/call via bot gateway — response len=10
[OK] praisonai-code execute via bot message — marker found
[FAIL] sandbox exec via bot message — It seems there was no output for the command...
[FAIL] browser control via bot message — 'charmap' codec can't encode character '\u2551'...

=== Done: 4 failed ===
Report: .../report_20260729_141007.json

JSON report excerpt (browser checks)

{
  "name": "browser control (playwright headless)",
  "status": "FAIL",
  "detail": "updated.       \u2551\r\n\u2551 Please run the following command to download new browsers: \u2551\r\n\u2551     playwright install                              ",
  "duration_s": 24.52
},
{
  "name": "browser control via bot message",
  "status": "FAIL",
  "detail": "'charmap' codec can't encode character '\\u2551' in position 62: character maps to <undefined>",
  "duration_s": 24.52
}

---

Interactive analysis

Fix scope gap (#3475 vs E2E harness)

flowchart TD
    subgraph fixed [Fixed in #3475]
        GW[WebSocketGateway error handlers]
        UU[unicode_utils.sanitize_for_console]
        GW --> UU
    end

    subgraph broken [Still broken]
        E2E[bot_gateway_e2e.py]
        ADD[Report.add bare print]
        PROBE[_playwright_headless_probe]
        E2E --> PROBE
        PROBE -->|RuntimeError with U+2551| ADD
        ADD -->|cp1252 stdout| CRASH[UnicodeEncodeError]
    end

    fixed -.->|not wired| broken

Loading

Failure sequence

sequenceDiagram
    participant E2E as bot_gateway_e2e.py
    participant PW as Playwright subprocess
    participant R as Report.add
    participant Out as stdout cp1252

    E2E->>PW: chromium.launch headless probe
    PW-->>E2E: stderr with U+2551 install banner
    E2E->>E2E: Check FAIL detail = RuntimeError text
    E2E->>R: add(FAIL, detail contains ║)
    R->>Out: print(f"[FAIL] ... {detail}")
    Out-->>R: UnicodeEncodeError U+2551
    R-->>E2E: outer except records charmap error
    Note over E2E: Developer never sees playwright install on console

Loading

Decision tree for developer diagnosis

flowchart TD
    A[browser control FAIL on Windows] --> B{Read JSON report or console?}
    B -->|Console only| C[See charmap U+2551 error]
    C --> D[Misdiagnose as encoding bug]
    B -->|JSON report| E[See playwright install hint]
    E --> F[Run playwright install chromium]
    F --> G{Browsers installed?}
    G -->|Yes| H[Re-run E2E — browser should PASS]
    G -->|No| I[Still FAIL until install]
    D --> J[Fix: wire E2E to unicode_utils]
    J --> E

Loading

---

Root cause analysis

Layer 1 — Playwright browsers missing (environment)

_playwright_headless_probe() at lines 385–408 launches Chromium in subprocess. When browsers not installed, Playwright stderr includes boxed banner with U+2551 (║) characters.

Layer 2 — E2E harness not using gateway sanitization

unicode_utils.py (post-#3475) explicitly maps \u2551 to space for Playwright banners:

# src/praisonai-bot/praisonai_bot/gateway/unicode_utils.py
'\u2550': ' ', '\u2551': ' ',   # DOUBLE HORIZONTAL/VERTICAL -> space

But bot_gateway_e2e.py does not import or call this module.

Layer 3 — Encoding-unsafe Report.add()

# e2e-platform-audit/bot_gateway_e2e.py lines 60-63
def add(self, check: Check) -> None:
    self.checks.append(check)
    icon = {"PASS": "OK", "FAIL": "FAIL", "SKIP": "SKIP"}.get(check.status, "?")
    print(f"[{icon}] {check.name}" + (f" — {check.detail}" if check.detail else ""))

Bare print() encodes to cp1252 on Windows → crash when check.detail contains \u2551.

Layer 4 — PYTHONUTF8 setdefault does not help

Line 30: os.environ.setdefault("PYTHONUTF8", "1") — on many Windows Python builds this does not reconfigure an already-initialized stdout when console is cp1252. Audit intentionally runs without forcing UTF-8 to reproduce developer default.

Layer 5 — Outer exception handler masks probe error

Browser block catches exceptions from report.add() itself, recording the encoding error as the browser bot message failure — same class of bug as original #3430.

---

Comparison: PraisonAIUI pattern (already fixed)

PraisonAIUI added examples/python/_shared/console.py with safe_print() and ASCII fallbacks for gateway examples (#235). PraisonAI E2E harness should mirror:

from praisonai_bot.gateway.unicode_utils import sanitize_for_console, safe_print

def add(self, check: Check) -> None:
    self.checks.append(check)
    icon = {"PASS": "OK", "FAIL": "FAIL", "SKIP": "SKIP"}.get(check.status, "?")
    msg = f"[{icon}] {check.name}" + (f" — {check.detail}" if check.detail else "")
    safe_print(sanitize_for_console(msg))

---

Proposed fixes

Option A — Reuse unicode_utils (recommended)

Import sanitize_for_console / safe_print from praisonai_bot.gateway.unicode_utils in bot_gateway_e2e.py and use in Report.add() and all top-level print() calls.

Option B — Sanitize probe errors before Check

from praisonai_bot.gateway.unicode_utils import sanitize_for_console

err = (stderr or b"").decode(errors="replace").strip()
raise RuntimeError(sanitize_for_console(err[-400:]))

Option C — ASCII-only Playwright preflight

Before probe, run python -m playwright install --dry-run or check browser path; skip probe with clear ASCII message if missing.

Option D — CI gate

Add Windows cp1252 job running bot_gateway_e2e.py --live-only without PYTHONUTF8.

---

Test plan

Step	Action	Expected
1	Run E2E on Windows cp1252 without Playwright browsers	Console shows playwright install, not charmap error
2	playwright install chromium + re-run	Browser checks PASS
3	Run with PYTHONUTF8=1	Same ASCII-safe output
4	Gateway unicode unit tests	Still pass (31 tests)
5	JSON report detail	Matches console (both sanitized)

Automated test to add

def test_report_add_survives_box_drawing_detail():
    from bot_gateway_e2e import Report, Check
    r = Report()
    detail = "Please run: playwright install " + "\u2551" * 4
    r.add(Check("probe", "FAIL", detail))  # must not raise

---

Acceptance criteria

1. bot_gateway_e2e.py --live-only never raises UnicodeEncodeError on cp1252 Windows console
2. When Playwright browsers missing, console shows actionable playwright install hint
3. Report.add() uses same sanitization as gateway unicode_utils
4. JSON report and console agree on failure root cause
5. No regression on Linux/macOS UTF-8 terminals

---

Appendix: Non-browser failures in same run (context)

Check	Status	Notes
docker sandbox live execute	FAIL	Docker daemon not running — env, not product bug
sandbox exec via bot message	FAIL	LLM did not return E2E_SANDBOX_OK — see separate draft PAI-007

---

Appendix: Files to change

File	Change
e2e-platform-audit/bot_gateway_e2e.py	safe_print + sanitize_for_console in Report.add and main prints
e2e-platform-audit/tests/test_e2e_console.py	New — cp1252-safe detail rendering
AGENTS.md / E2E docs	Note: run playwright install before browser live checks

---

Appendix: Related closed issues

• [BUG][Windows] Bot gateway E2E browser check crashes on cp1252 when Playwright stderr contains box-drawing characters #3430 — Original cp1252 browser E2E report
• [FOLLOW-UP #3430][Windows] Bot gateway browser E2E still fails cp1252 on Playwright stderr (latest main) #3475 — Gateway unicode_utils box-drawing map — closed but E2E harness out of scope

This issue tracks the remaining scope for complete Windows browser E2E developer experience.

[praisonai-triage-agent]

👋 Thanks for opening this issue, @Dhivya-Bharathy!

A maintainer will review this shortly. In the meantime:

• Make sure you've included steps to reproduce (for bugs)
• Check existing issues for duplicates
• Review the documentation for related guides

Bug and enhancement reports are auto-triaged with Claude. Questions and other issues: a maintainer can comment @claude to trigger analysis.

Routing: Agent-callable tools → PraisonAI-Tools; lifecycle plugins (tracing/hooks/guardrails) → PraisonAI-Plugins.

[praisonai-triage-agent]

Claude finished @Dhivya-Bharathy's task —— View job

---

Analysis complete — no code change needed (fix already present; target file absent)

• Read AGENTS.md guidelines
• Locate e2e-platform-audit/bot_gateway_e2e.py and unicode_utils.py
• Architecture validation (scope-creep review)
• Verify the reported crash scenario against existing code
• Run existing gateway unicode tests

Findings

I could not reproduce this as an actionable code change in MervinPraison/PraisonAI, because the file the issue asks to fix does not exist in this repository and the fix it proposes already exists.

1. The target file is not in this repo. e2e-platform-audit/bot_gateway_e2e.py — along with Report.add, _playwright_headless_probe, and the whole e2e-platform-audit/ directory — is absent from the working tree and from all git history:

$ git log --all -- "e2e-platform-audit/*"   → (no results)
$ grep -r "_playwright_headless_probe" .     → (no results)
$ grep -r "bot_gateway_e2e" .                → (no results)

2. The proposed fix references an API that doesn't exist. The issue suggests from praisonai_bot.gateway.unicode_utils import sanitize_for_console, safe_print. There is no sanitize_for_console anywhere in the codebase.

3. The real fix already shipped in #3475 and covers this exact case. src/praisonai-bot/praisonai_bot/gateway/unicode_utils.py already provides:

• safe_print() — a drop-in print() replacement whose docstring explicitly targets "E2E harnesses and console reporters that may echo subprocess output (e.g. Playwright's playwright install banner drawn with box characters like U+2551)" (unicode_utils.py:182).
• _SYMBOL_MAP already maps \u2551 → ' ' (unicode_utils.py:40).

I verified the exact issue scenario (U+2551 in a [FAIL] line on a cp1252 stream that raises UnicodeEncodeError):

safe_print output: '[FAIL] browser control - Please run: playwright install\n'
safe_error_message: 'Please run: playwright install'
NO CRASH - safe_print handles U+2551 on cp1252

Existing tests: 31 passed (tests/unit/gateway/test_unicode_utils.py).

Decision (per AGENTS.md STEP 2)

No PR was created. Reasons:

• No file to change here. The affected harness lives outside this repository's tracked source; I cannot patch something that isn't here.
• No new surface justified. Fabricating a ~400-line bot_gateway_e2e.py and a new sanitize_for_console alias would be scope creep with no live consumer — exactly what the "lightweight and powerful" guideline rejects. The capability (safe_print + box-drawing map) already exists.

Recommended action for the harness maintainer

Wherever bot_gateway_e2e.py actually lives, apply the minimal wiring the issue proposes — using the existing helper name:

from praisonai_bot.gateway.unicode_utils import safe_print, safe_error_message

def add(self, check: Check) -> None:
    self.checks.append(check)
    icon = {"PASS": "OK", "FAIL": "FAIL", "SKIP": "SKIP"}.get(check.status, "?")
    detail = f" — {safe_error_message(check.detail)}" if check.detail else ""
    safe_print(f"[{icon}] {check.name}{detail}")

(No sanitize_for_console — use safe_print / safe_error_message, which are the shipped API.)

If the harness should be added to this repo, please confirm and I'll add a minimal e2e-platform-audit/bot_gateway_e2e.py wired to the existing safe_print.

Job run · Branch: claude/issue-3499-20260729-0921

[MervinPraison]

@claude review again and see if its worth creating pr. use multi agents to review

use this if that is the strognly recommended

Option A — Reuse unicode_utils (recommended)
Import sanitize_for_console / safe_print from praisonai_bot.gateway.unicode_utils in bot_gateway_e2e.py and use in Report.add() and all top-level print() calls.

[praisonai-triage-agent]

Claude finished @MervinPraison's task —— View job

---

Re-review complete (multi-perspective) — no PR warranted

• Read AGENTS.md guidelines
• Locate e2e-platform-audit/bot_gateway_e2e.py
• Verify unicode_utils.py API
• Decide: worth creating PR?
• Multi-perspective critical review

Verdict: Not worth creating a PR in this repo — the fix already ships, and the target file does not exist here.

I re-ran the investigation from three angles as requested:

1. Existence check (harness maintainer view) — bot_gateway_e2e.py and the entire e2e-platform-audit/ directory are absent from the working tree AND from all git history:

$ glob **/bot_gateway_e2e.py        → No files found
$ glob e2e-platform-audit/**        → No files found
$ grep _playwright_headless_probe   → No files found
$ git log --all -- e2e-platform-audit/*  → (empty)

There is nothing to patch here.

2. API-correctness check (Option A feasibility view) — The issue's Option A proposes from praisonai_bot.gateway.unicode_utils import sanitize_for_console, safe_print. But sanitize_for_console does not exist anywhere in the codebase:

$ grep sanitize_for_console  → No files found

So Option A as written cannot be implemented. The shipped, correct API is safe_print + safe_error_message.

3. Coverage check (does #3475 already solve it?) — Yes. src/praisonai-bot/praisonai_bot/gateway/unicode_utils.py already contains exactly what's needed:

Capability	Location	Status
safe_print() — cp1252-safe drop-in for print(), docstring explicitly names the Playwright playwright install U+2551 banner case	unicode_utils.py:182	✅ shipped
safe_error_message() — ASCII-safe detail text	unicode_utils.py:75	✅ shipped
\u2551 → ' ' box-drawing map	unicode_utils.py:40	✅ shipped

safe_print catches UnicodeEncodeError and re-emits an ASCII-sanitized, line-break-preserving version (unicode_utils.py:206-215) — precisely the crash in this issue.

Decision (AGENTS.md STEP 2 — lightweight & powerful)

No PR created, because doing so would be scope creep with no live consumer:

• The affected harness lives outside this repository's tracked source — I can't wire a file that isn't here.
• Fabricating a ~400-line bot_gateway_e2e.py plus a redundant sanitize_for_console alias would add surface area for zero benefit; the exact capability already exists as safe_print/safe_error_message.

Recommended action for the harness maintainer

Wherever bot_gateway_e2e.py actually lives, apply Option A using the shipped API names:

from praisonai_bot.gateway.unicode_utils import safe_print, safe_error_message

def add(self, check: Check) -> None:
    self.checks.append(check)
    icon = {"PASS": "OK", "FAIL": "FAIL", "SKIP": "SKIP"}.get(check.status, "?")
    detail = f" — {safe_error_message(check.detail)}" if check.detail else ""
    safe_print(f"[{icon}] {check.name}{detail}")

Replace all top-level print(...) in the harness with safe_print(...). This satisfies every acceptance criterion (no UnicodeEncodeError on cp1252, actionable playwright install hint, console/JSON agreement, no Linux/macOS regression) with zero new code in this repo.

If you'd like this harness added to the PraisonAI repo (rather than living externally), reply confirming and I'll create a minimal e2e-platform-audit/bot_gateway_e2e.py wired to the existing safe_print.

--- · Branch: claude/issue-3499-20260731-0657