Skip to content

feat(load): add a document loader that keeps source positions - #574

Open
GenerikUser wants to merge 1 commit into
crossplane:mainfrom
GenerikUser:erik.miller--pkg-load-document-loader
Open

GenerikUser wants to merge 1 commit into
crossplane:mainfrom
GenerikUser:erik.miller--pkg-load-document-loader

Conversation

@GenerikUser

@GenerikUser GenerikUser commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Description of your changes

Adds a document-level loader to pkg/load, alongside the existing Load().

pkg/load today decodes each YAML document straight into an unstructured object. That loses what some callers need: which file a document came from, where in that file it starts, and its raw bytes with comments. A caller that reports positions in the source cannot get them from the existing loaders. The first such caller will be crossplane xrd lint (#481), which will be rebased onto this PR so that it reads its inputs through pkg/load rather than through a loader of its own.

What it adds:

  • Document{Source, StartLine, Raw}: one YAML document as it appears in its source. Source is the file path, or - for stdin.
  • NewDocumentLoader(input, ...DocumentLoaderOption): accepts the same sources as NewLoader: a comma-separated list of files, directories searched recursively for .yaml and .yml files, and - for stdin. It returns the documents without decoding them, so empty documents and documents that are not Kubernetes objects are returned too.
  • SplitDocuments(source, reader): splits a stream into documents exactly as YamlStream does, recording the line each document starts on.
  • WithStdin(reader): sets the reader - reads, defaulting to os.Stdin, so callers and tests can supply stdin without replacing os.Stdin.

Existing loaders (NewLoader, NewCompositeLoader, NewExactCompositeLoader, YamlStream) and all their callers are unchanged.

Tests. SplitDocuments is checked against YamlStream on every case, so the two always split a stream into the same documents. Start lines are checked for:

  • separators, and separators with trailing comments;
  • leading comments before the first document;
  • empty and separator-only documents;
  • CRLF line endings and a missing final newline;
  • an invalid separator, which is an error for both.

The loader is also tested on comma-separated lists, recursive directories and a missing source, and WithStdin with - alone and in a list.

For reviewers. SplitDocuments reimplements the separator rules of k8s.io/apimachinery's YAMLReader, because that reader does not report line numbers. The equivalence check in the tests is what keeps the two in step.

Refs #481

I have:

  • Read and followed Crossplane's contribution process.
  • Read and followed Crossplane's AI policy and confirmed a human stands behind this PR.
  • Run ./nix.sh flake check to ensure this PR is ready for review.
  • Added or updated unit tests.
  • Linked a PR or a docs tracking issue to document this change. Not applicable: this adds a library API in pkg/load and changes no command or user-facing behaviour.
  • Added backport release-x.y labels to auto-backport this PR. Not applicable: this is a new feature, not a fix.

Need help with this checklist? See the cheat sheet.

Add NewDocumentLoader, SplitDocuments and Document to pkg/load. The new
loader reads the same sources as NewLoader (a comma-separated list of
files, directories searched recursively for .yaml and .yml files, and
"-" for stdin) and splits each stream into documents exactly as
YamlStream does. Instead of decoding the documents, it returns each
one's raw bytes, comments included, with the file it came from and the
line it starts on, so callers can report positions in the source.

A WithStdin option sets the reader "-" reads, defaulting to os.Stdin,
so callers and tests can supply stdin without replacing os.Stdin.

Existing loaders and their callers are unchanged.

Tests check that SplitDocuments returns the same documents as
YamlStream and records exact start lines across separators, separators
with trailing comments, leading comments, empty documents, CRLF line
endings and a missing final newline, and that WithStdin is honoured.

Signed-off-by: GenerikUser <generikuser@users.noreply.github.com>
@GenerikUser
GenerikUser requested review from a team, jcogilvie and tampakrap as code owners November 20, 2026 17:30
@GenerikUser
GenerikUser requested review from phisco and removed request for a team November 20, 2026 17:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant