What happened
On PR #4667, human reviewers identified four specification-design issues that depend on understanding Backstage's catalog entity processing model. The review agent ran 12 successful reviews but missed all four because it lacks domain context about entity lifecycle concepts:
- locationKey vs managed-by-location conflation: gabemontero identified that
design.md conflated Backstage's locationKey (the processing-ownership key set on mutations) with backstage.io/managed-by-location (a provenance annotation). The author fixed this with a targeted commit.
- Multi-registry entity identity collisions: johnmcollier raised how multiple MCP registries emitting the same
(kind, namespace, name) tuple would cause catalog identity collisions. The agent touched collision at a surface level (pure-function algorithm finding) but never asked the fundamental design question. This drove the addition of baseName/prefix registry scoping.
- Full mutation pruning semantics: gabemontero questioned whether silently dropping unmappable entities from full mutations -- thus pruning them from the catalog on the next sync -- was intentional. The author chose a last-good retention strategy with a
sync-status annotation.
- Annotation round-trip data loss: gabemontero identified that URL normalization would destroy the original
repository.url, violating the annotation-projection contract. The agent flagged the URL algorithm as unspecified but missed the data-loss implication. The author added a dedicated annotation to preserve the original URL.
What could go better
The ai-integrations workspace AGENTS.md documents implementation patterns (service-to-service auth, config visibility, test conventions) but not the conceptual model that entity provider specifications describe. When reviewing OpenSpec specifications for entity providers, the review agent needs to understand how entity identity, ownership, mutations, and data contracts work at the Backstage catalog level.
Existing issue #4781 proposes adding entity provider implementation-level conventions (source-location format, mutation error handling, timeouts, pagination). There is partial overlap on mutation semantics, but the remaining three concepts -- entity identity and collision handling, locationKey vs managed-by-location distinction, and annotation projection contracts -- are not covered by #4781 or any other open issue. This proposal provides the specification-review-level concepts that complement #4781's code-level patterns.
Confidence is moderate-to-high. The specific gaps are well-evidenced from this PR (4 distinct findings, all driving design changes). The ai-integrations workspace has multiple entity provider specifications in development, so the pattern is likely to recur.
Proposed change
In workspaces/ai-integrations/AGENTS.md, add an "Entity provider design concepts" subsection after the existing "Backstage Backend Conventions" section (after line 170):
## Entity Provider Design Concepts
Reference for reviewing and authoring entity provider specifications
(OpenSpec files under `openspec/changes/`).
### Entity identity
Backstage catalog entities are uniquely identified by `(kind, namespace,
name)`. When multiple providers can emit the same identity tuple, the
specification must state how the provider constructs each component and
what happens when a collision with another provider is possible. If the
provider uses caller-supplied prefixes for scoping, the specification
should define the default and document the collision behavior.
### locationKey vs managed-by-location
These are distinct mechanisms -- do not conflate them:
- **`locationKey`** is set on the entity mutation and determines which
provider owns the entity for processing. The catalog uses it to scope
updates and deletions.
- **`backstage.io/managed-by-location`** is an annotation on the entity
showing provenance (which location ingested it). It is informational.
### Full mutation semantics
`type: 'full'` mutations replace the entire set of entities owned by
the provider's locationKey. An entity absent from the next full mutation
is pruned from the catalog. Specifications must explicitly state what
happens when a previously valid entity becomes unmappable (transient
failure, schema drift): is it pruned or retained from the last
successful sync? If retained, how is the stale state communicated?
### Annotation data contracts
When a specification promises that upstream fields are projected into
catalog annotations, it creates a round-trip contract: the original
value should be recoverable. URL normalization, truncation, or lossy
transforms can violate this. Specifications should state which fields
are preserved exactly and which are normalized, and provide a separate
annotation for the original value when normalization is lossy.
Validation criteria
On the next 3 entity provider specification PRs in the ai-integrations workspace, the review agent should: (1) correctly distinguish locationKey from managed-by-location when both appear in a specification, (2) question unspecified full-mutation pruning behavior when the spec describes entity lifecycle transitions, and (3) identify lossy transforms in annotation projection contracts that violate round-trip guarantees. Compare agent findings against human review findings on those PRs.
Generated by retro agent from #4667
What happened
On PR #4667, human reviewers identified four specification-design issues that depend on understanding Backstage's catalog entity processing model. The review agent ran 12 successful reviews but missed all four because it lacks domain context about entity lifecycle concepts:
design.mdconflated Backstage'slocationKey(the processing-ownership key set on mutations) withbackstage.io/managed-by-location(a provenance annotation). The author fixed this with a targeted commit.(kind, namespace, name)tuple would cause catalog identity collisions. The agent touched collision at a surface level (pure-function algorithm finding) but never asked the fundamental design question. This drove the addition ofbaseName/prefixregistry scoping.sync-statusannotation.repository.url, violating the annotation-projection contract. The agent flagged the URL algorithm as unspecified but missed the data-loss implication. The author added a dedicated annotation to preserve the original URL.What could go better
The ai-integrations workspace AGENTS.md documents implementation patterns (service-to-service auth, config visibility, test conventions) but not the conceptual model that entity provider specifications describe. When reviewing OpenSpec specifications for entity providers, the review agent needs to understand how entity identity, ownership, mutations, and data contracts work at the Backstage catalog level.
Existing issue #4781 proposes adding entity provider implementation-level conventions (source-location format, mutation error handling, timeouts, pagination). There is partial overlap on mutation semantics, but the remaining three concepts -- entity identity and collision handling, locationKey vs managed-by-location distinction, and annotation projection contracts -- are not covered by #4781 or any other open issue. This proposal provides the specification-review-level concepts that complement #4781's code-level patterns.
Confidence is moderate-to-high. The specific gaps are well-evidenced from this PR (4 distinct findings, all driving design changes). The ai-integrations workspace has multiple entity provider specifications in development, so the pattern is likely to recur.
Proposed change
In
workspaces/ai-integrations/AGENTS.md, add an "Entity provider design concepts" subsection after the existing "Backstage Backend Conventions" section (after line 170):Validation criteria
On the next 3 entity provider specification PRs in the ai-integrations workspace, the review agent should: (1) correctly distinguish locationKey from managed-by-location when both appear in a specification, (2) question unspecified full-mutation pruning behavior when the spec describes entity lifecycle transitions, and (3) identify lossy transforms in annotation projection contracts that violate round-trip guarantees. Compare agent findings against human review findings on those PRs.
Generated by retro agent from #4667