Split ExceptionInfo and the Repr objects into capture, data model and renderers

[RonnyPfannschmidt]

Written by Claude Fable 5.1 via Claude Code for the pytest maintainers; I prompted it, it did the work, I read it.

Summary

ExceptionInfo and the Repr* objects in _pytest/_code/code.py date back to py.code (2004 era). #3399 asked for a strategy to refactor them and never got one. This issue is that strategy: split the current objects into three layers, capture, a frozen data model, and renderers, so that the same recorded exception can be matched in pytest.raises, shipped through xdist, and rendered by the terminal, junitxml, pytest-html or a JSON reporter without each of them re-deriving structure from pre-rendered text. rich is a hard dependency by the time this lands, so the renderers are rich renderables; the model stays plain data.

Supersedes #3399. Directly resolves #14720, #12175, the remaining #9159 fallback, and gives #10224 a place to land. Migrating the rest of the terminal reporter to rich is a separate effort and not covered here.

What is wrong today

Verified against main (9.2.0.dev):

• Style is baked in at build time. ExceptionInfo.getrepr() takes style, showlocals, abspath, truncate_*, chain and emits ReprEntry.lines as already-rendered text with > and E markers. ReprEntry._write_entry_lines then re-parses those markers to syntax-highlight. --tb=line/--tb=no drop reprfileloc entirely, which is --tb affects longrepr construction, so plugins cannot suppress traceback display without losing traceback data #14720. nodes.py:444 has carried # XXX should excinfo.getrepr record all data and toterminal() process it? for years; the answer is yes.
• Serialization is hand-written and lossy. reports.py:574-735 dispatches on hasattr(longrepr, "reprtraceback"), rebuilds dataclasses by hand, and turns any other TerminalRepr (ReprFailDoctest, FixtureLookupErrorRepr, CollectErrorRepr) into a plain string. xdist cannot ship pytest_internalerror data at all (dsession.py:226). The wire format carries no function name, module, column, raw source or structured locals.
• longrepr is a four-way union (str | tuple[str,int,str] | TerminalRepr | ExceptionInfo | None). The skip 3-tuple is load-bearing in terminal.py, junitxml.py, runner.py and xdist tests. Duck-typed hasattr/getattr probes for reprcrash/toterminal exist in five places. None of the Repr* classes are exported or documented, yet reprcrash.message, reprtraceback.reprentries[-1].reprfileloc (pastebin) and chain/sections are read by plugins.
• Exception groups still fall back to native text; the chain stops at a group (code.py:1213, the Pytest does not show inner exceptions in PEP-654 ExceptionGroups #9159 workaround).
• Live-object coupling. Traceback/TracebackEntry wrap real frames; for_later() loses the assertion strip text (ExceptionInfo.for_later() wont add assertion strip text #12175); _getreprcrash recomputes from live frames; statement ranges come from pytest's own AST search (getstatementrange_ast) while co_positions has carried line, end line and columns since 3.11.
• Raisers duplicate matching logic. ExceptionInfo.match vs AbstractRaises._check_match, errisinstance+group_contains vs RaisesExc.matches (raises.py:358, TODO at :477), and fail_reason is a string that gets stitched together.

What everyone else converged on

The agent surveyed rich, structlog, the stdlib traceback module, Sentry's event protocol, OpenTelemetry, stackprinter, tblib and the exceptiongroup backport. The tools that model exceptions as data agree on the shape:

	exception node	frame fields	locals	source	chain	groups	positions	serializable
stdlib TracebackException	type module/qualname, str, notes, SyntaxError fields, exceptions	filename, lineno, end_lineno, colno, end_colno, name, line, locals	{name: repr} opt-in	lazy linecache	nested cause/context	exceptions list	3.11+	pickle only on 3.11/3.12 (_code slot breaks 3.13+)
rich Trace/Stack/Frame	type str, value str, notes, syntax_error, is_group	filename, lineno, name, last_instruction	pretty Node trees, capped	lazy at render	flat stacks with is_cause	nested Trace per member	yes	plain dataclasses, no export
structlog ExceptionDictTransformer	same as rich	filename, lineno, name	{name: str} capped, hide dunder, path suppress	none	flat with is_cause	nested	no	JSON via asdict, 50-frame cut
Sentry	type, value(+notes), module, mechanism{source, exception_id, parent_id, is_exception_group}	filename, abs_path, function, module, lineno, colno, context_line, pre/post_context, in_app, vars	serializer-limited, per-frame filter	inline 5+5 lines	flat list, oldest first, ids	ids + source="exceptions[i]"	lineno/colno	JSON with truncation metadata
OpenTelemetry	type, message, stacktrace	none	none	inline in string	none	none	none	string

Common ground: capture is eager and produces plain data (strings for values, ints for positions); style, suppression, frame elision and colour are renderer options; groups are children of a node, chaining is a link between nodes. Where they differ (tree vs flat-with-ids, inline vs lazy source) the choice is driven by whether the data crosses a process boundary. pytest's does.

Why not rich's Trace as the model

rich is the right rendering substrate and the wrong wire model. Its Trace/Stack/Frame dataclasses are undocumented (no __all__, not in the reference docs), Frame.line is never filled, locals are rich.pretty.Node trees rather than strings, __notes__ of the outer exception are copied onto every chained stack, there is no hidden-frame flag, no module or qualname per frame, no source capture (linecache at render time) and no serialization API. structlog's port of the same model to JSON shows the shape transfers; pytest's model below is a superset of it, so a lossless adapter to rich.traceback.Trace is cheap and pytest keeps control of what crosses the xdist boundary.

Proposed design

Three layers. Every name below is a placeholder and open to bikeshedding; the shape is the proposal, not the spelling.

1. Capture: ExceptionInfo stays the live handle. It keeps type/value/tb, traceback navigation, match, group_contains, and is what pytest.raises returns. It gains one method, record(...), that produces layer 2. The striptext for assertions is computed at record time, which removes the for_later() gap (#12175).

2. Model: frozen dataclasses, plain data, versioned, no rich import.

@dataclass(frozen=True)
class Location:            # replaces ReprFileLocation and the skip 3-tuple
    path: str
    lineno: int
    message: str

@dataclass(frozen=True)
class FrameRecord:
    path: str              # as recorded (relative or absolute is a render choice; both kept)
    abspath: str | None
    lineno: int
    end_lineno: int | None       # from co_positions; None only on 3.10
    colno: int | None
    end_colno: int | None
    function: str          # co_qualname where available
    module: str | None
    statement: tuple[str, ...]   # source lines of the failing statement, captured eagerly
    statement_lineno: int        # first line of `statement`
    locals: Mapping[str, str] | None     # saferepr'd, only when requested
    args: tuple[tuple[str, str], ...] | None
    hidden: bool           # __tracebackhide__ / internal; rendering decides whether to show

@dataclass(frozen=True)
class ExceptionRecord:
    type_name: str
    type_module: str
    message: str           # exconly without the type prefix
    notes: tuple[str, ...]
    syntax_error: SyntaxErrorDetails | None
    frames: tuple[FrameRecord, ...]
    cause: ExceptionRecord | None
    context: ExceptionRecord | None
    suppress_context: bool
    children: tuple[ExceptionRecord, ...]   # exception groups
    sections: tuple[tuple[str, str, str], ...]
    recursion_index: int | None

    @property
    def crash(self) -> Location | None: ...   # last non-hidden frame + message
    def walk(self) -> Iterator[ExceptionRecord]: ...
    def to_dict(self) -> dict: ...            # generated, schema-versioned
    @classmethod
    def from_dict(cls, data) -> ExceptionRecord: ...
    def to_rich_trace(self) -> rich.traceback.Trace: ...   # lossless

Everything is JSON-able and picklable. The full traceback is always recorded; --tb no longer decides what is stored (#14720). Source lines are captured eagerly because the current wire format already inlines rendered text, and because xdist workers and file edits after the fact mean linecache cannot be trusted on the reading side.

3. Renderers are rich renderables over the model.

• PytestTraceback(record, style, showlocals, abspath, chain, truncate_locals, truncate_args, funcargs) implements __rich_console__ and reproduces today's long/short/line/no/native/value layouts, including the per-entry style override that Collector uses. Source goes through rich.syntax.Syntax, locals through rich.pretty, the >/E markers and separators become Text with styles instead of re-parsed strings.
• --tb=rich is a new style: rich.traceback.Traceback(trace=record.to_rich_trace(), ...), rich's own layout for people who already use it.
• str(record) renders the default style to plain text; record.toterminal(tw) stays as a shim for one deprecation cycle.
• Console.export_html()/export_svg() give pytest-html and docs a faithful rendering without a second renderer.
• junitxml, pastebin and the short summary read record.crash, record.frames, record.message instead of probing attributes. JSON reporters get to_dict().
• Exception groups render in pytest style from children; the TracebackException.format() fallback goes away.
• Positions come from co_positions, not from the AST statement search: statement is the lineno..end_lineno range of the failing instruction, and PEP 657 carets (PEP-657 enhanced error locations in pytest tracebacks #10224) are a renderer feature over colno/end_colno. On 3.10 the record carries line numbers only and the renderer degrades to today's single-line behaviour.

Raisers. AbstractRaises.matches() produces a structured failure instead of a string, rendered through the same path (a rich.tree.Tree for RaisesGroup mismatches is the obvious win); ExceptionInfo.match and _check_match share one implementation. pytest_exception_interact keeps the live CallInfo.excinfo; the report carries the record. pytest_internalerror receives the record, so xdist can forward it.

Compatibility

• TestReport.longrepr becomes ExceptionRecord | FailureRecord | None. For one cycle the record exposes reprcrash (returns crash), reprtraceback.reprentries[-1].reprfileloc, chain and sections as deprecated properties, and Location unpacks as a 3-tuple.
• JSON: _report_to_json emits the record's to_dict(); _report_kwargs_from_json accepts both shapes for one cycle. xdist already requires matching pytest versions on both ends.
• ReprFailDoctest, FixtureLookupErrorRepr, CollectErrorRepr migrate to a small FailureRecord protocol (crash, sections, to_dict, __rich_console__) so they stop degrading to strings over xdist.
• ExceptionInfo.getrepr() keeps working by recording then rendering into the old-shaped objects, deprecated.
• Terminal output is purpose-matching, not byte-identical: each --tb style keeps conveying the same information with the same visual roles (source context, > flow marker, E failure lines, location line, locals, chain descriptions), and tests that pin exact bytes are updated where the difference is cosmetic. A deliberate restyle of the default look is still a separate issue.

Increments

Each step ships on its own and keeps the suite green.

1. Add the model and ExceptionInfo.record() alongside the existing code, with to_dict/from_dict round-trip tests and to_rich_trace(). No consumer changes.
2. Add the PytestTraceback renderable and run it against testing/code/test_excinfo.py and testing/test_terminal.py, adjusting assertions that pin cosmetic bytes. Ship --tb=rich.
3. Switch getrepr(), reports and JSON to record-then-render. Fixes --tb affects longrepr construction, so plugins cannot suppress traceback display without losing traceback data #14720. xdist suite green.
4. Migrate doctest, fixture-lookup and collect-error reprs; replace the skip 3-tuple with Location.
5. Native exception-group rendering; remove the Pytest does not show inner exceptions in PEP-654 ExceptionGroups #9159 fallback.
6. Fold raiser matching onto the model; fix ExceptionInfo.for_later() wont add assertion strip text #12175.
7. Render PEP 657 carets (PEP-657 enhanced error locations in pytest tracebacks #10224). Delete getstatementrange_ast and the AST cache once 3.10 is dropped (EOL October 2026).
8. Document the model as public API, deprecate the Repr* classes and getrepr() parameters.

Open points

• Tree (cause/context/children, stdlib-like) or flat list with ids (Sentry-like) for the chain. To be discovered in step 1.
• Whether statement capture is unconditional or bounded (line count, byte size) for pathological frames.