Skip to content

openapi: the lowering scans raw YAML for each $ref into it and discards what it finds #778

Description

@OmarAlJarrah

Problem

resolve.Scope.ModelAt asks jsonpointer.GetTarget for the schema the model holds at a pointer. Where the pointer passes into raw YAML, such as an extension's value, the library scans each raw mapping on the way, and ModelAt then answers nil, since a raw node is never a schema. schema.settledDecl asks it for every sub-schema $ref the lowering hoists, so n $refs into one wide extension cost n scans of it, all discarded.

Reproduction

Verified at 25bdace, with references off. gen.py writes n component schemas, each a $ref to its own entry of one extension:

import sys
n = int(sys.argv[1])
doc = ["openapi: 3.1.0", "info: {title: T, version: '1'}", "paths: {}", "components:", "  schemas:"]
doc += [f"    S{i}: {{$ref: '#/x-lib/D{i}'}}" for i in range(n)]
doc += ["x-lib:"] + [f"  D{i}: {{type: object}}" for i in range(n)]
open("root.yaml", "w").write("\n".join(doc) + "\n")

/usr/bin/time -l morphic validate root.yaml takes 1.28 s at 10,000, 3.13 s at 20,000 and 9.76 s at 40,000. Nothing is reported.

Cause

A single-threaded CPU profile of the 40,000 run puts 0.66 s under ModelAt. The same profile has 1.84 s in checkUniqueKeys (#776), and 1.63 s in the library resolving each $ref, which is its own scan and is not addressed here.

Proposal

Walk the model a token at a time with resolve.Step, and answer nil as soon as a step lands in raw YAML. That is exact only where no token reads as an index: the library retries such a token as an index when the key path fails further down. So fall back to the whole read for those.

Acceptance

  • ModelAt reads no raw mapping below the point where the pointer leaves the model, and answers as it did for every pointer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions