You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 6bbbb33
Browse filesBrowse the repository at this point in the historyBrowse files
refactor(submitqueue): remove duplicate List projection
Summary:
Intent:
- Complete the receipt-based List migration from #800: keep one authoritative request summary and immutable lookup keys.
Changes:
- Remove queue-summary writes, the duplicate entity/store, and obsolete tests and mocks.
- Idempotently ensure URI mappings before the receipt key, including partial-write retries and stale logs.
- Update rollout docs; retain the legacy SQL definition for deployment compatibility.
Test Plan:
- Deploy after #800 is stable. Rollback must retain the receipt-based reader once duplicate writes stop.
- Retire the table in a separate internal schema change after old binaries and the rollback window are gone. UQL DEPRECATED can drop immediately in Vitess; no physical retirement is enabled here.
---
<sub>Generated by the 🪄 [pr-create](https://sg.uberinternal.com/code.uber.internal/uber-code/devexp-agent-marketplace/-/blob/claude-code/plugins/dev/uber-dev/skills/pr-create/SKILL.md) skill in devexp-agent-marketplace</sub>
Copy file name to clipboardExpand all lines: doc/rfc/submitqueue/status-list-api.md
+16-14Lines changed: 16 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -67,7 +67,7 @@ Page size is optional and subject to a server default and maximum.
67
67
68
68
New requests may appear before the first page while a caller is paging. The immutable tuple cursor prevents duplicates within the caller's traversal.
69
69
70
-
The first version does not filter by current status. Current status is mutable, while the queue receipt projection is ordered by immutable receipt time. Efficient status filtering requires an additional application-maintained membership projection keyed by queue, status, receipt time, and sqid. That projection is deferred until a concrete server-side filtering use case justifies its write and reconciliation cost. Clients may filter a returned page for presentation, but client-side filtering is not equivalent to a server-side filtered query.
70
+
The first version does not filter by current status. Current status is mutable, while the receipt lookup is ordered by immutable receipt time. Efficient status filtering requires an additional application-maintained membership projection keyed by queue, status, receipt time, and sqid. That projection is deferred until a concrete server-side filtering use case justifies its write and reconciliation cost. Clients may filter a returned page for presentation, but client-side filtering is not equivalent to a server-side filtered query.
71
71
72
72
## Errors
73
73
@@ -77,7 +77,7 @@ The transport representation of these errors follows the gateway-wide RPC error
77
77
78
78
## Gateway-Owned Read Model
79
79
80
-
The gateway owns the append-only request log and three new logical read models. The orchestrator's request and change stores are pipeline working state with different retention semantics, so neither API reads them.
80
+
The gateway owns the append-only request log, materialized request summaries, and lookup stores. The orchestrator's request and change stores are pipeline working state with different retention semantics, so neither API reads them.
81
81
82
82
### Request Summary by Sqid
83
83
@@ -87,17 +87,17 @@ The immutable context is queue, change URIs, and receipt time. The mutable respo
87
87
88
88
The key supports authoritative lookup and conditional status updates for one request without a secondary index, and leads with the queue so the table is shardable by queue.
89
89
90
-
### Request Summaries by Queue
90
+
### Requests by Receipt Time
91
91
92
-
The queue projection is logically keyed by `(queue, received_at_ms, sqid)`and must support a bounded descending scan over `(received_at_ms, sqid)`. A backend may satisfy that contract with a reverse range scan or by descending-encoding the ordered key components; cursors always carry the original receipt time and sqid values.
92
+
The immutable `request_receipt` lookup contains only `(queue, received_at_ms, request_id)`as its primary key and supports a bounded descending scan over `(received_at_ms, request_id)`. Request IDs break timestamp ties in string order, not numeric order. A backend may use a reverse range scan or descending-encode the ordered key components; cursors always carry the original receipt time and request ID.
93
93
94
-
The row duplicates the complete `List` response deliberately. A page is served by one bounded range scan rather than one follow-up authoritative-summary read per result. Status updates propagate from the authoritative sqid summary to this projection.
94
+
List resolves these keys through `request_summary`; the lookup does not duplicate response fields.
95
95
96
96
The logical key covers the `List` queue predicate, receipt-time range, newest-first ordering, and complete keyset cursor in one bounded scan.
97
97
98
98
### Requests by Change URI
99
99
100
-
The URI reverse mapping is logically keyed by `(queue, change_uri, received_at_ms, sqid)` and must support a bounded descending scan over `(received_at_ms, sqid)` within one queue. As with the queue projection, a backend may use a reverse range scan or descending-encoded key components while exposing cursors and results in the original values. The mapping contains immutable lookup data and does not duplicate mutable status fields.
100
+
The URI reverse mapping is logically keyed by `(queue, change_uri, received_at_ms, sqid)` and must support a bounded descending scan over `(received_at_ms, sqid)` within one queue. As with the receipt lookup, a backend may use a reverse range scan or descending-encoded key components while exposing cursors and results in the original values. The mapping contains immutable lookup data and does not duplicate mutable status fields.
101
101
102
102
The mapping repeats `received_at_ms` because receipt time is part of the promised newest-first ordering. This allows the gateway to perform a bounded ordered scan before resolving the matching authoritative summaries. Without receipt time in the mapping, the gateway would have to fetch and sort every request associated with a URI before enforcing the result maximum.
103
103
@@ -109,25 +109,25 @@ The URI is stored in the canonical form received from the validated Land request
109
109
110
110
### Land Receipt
111
111
112
-
After synchronous validation, Land generates the sqid and one receipt timestamp. The gateway persists the authoritative summary in the internal `accepting` state, then publishes the request to the orchestrator. It does not create the URI or queue projections while the request remains `accepting`.
112
+
After synchronous validation, Land generates the sqid and one receipt timestamp. The gateway persists the authoritative summary in the internal `accepting` state, then publishes the request to the orchestrator. It does not create the URI or receipt mappings while the request remains `accepting`.
113
113
114
-
After publication succeeds, Land appends the initial `accepted` request log. Materializing `accepted`, `started`, or any later event promotes the authoritative summary out of `accepting` and creates the URI and queue projections. This handles the race where the orchestrator emits `started` before Land finishes persisting `accepted`.
114
+
After publication succeeds, Land appends the initial `accepted` request log. Materializing `accepted`, `started`, or any later lifecycle state promotes the authoritative summary out of `accepting`, ensures each immutable URI mapping, then ensures the receipt lookup key exists. This handles the race where the orchestrator emits `started` before Land finishes persisting `accepted`.
115
115
116
116
An `accepted` event is the lowest public lifecycle state. A late `accepted` event is retained in request history but must not replace `started` or any later materialized status.
117
117
118
118
Pipeline publication is the Land success boundary. If publication fails, Land returns an error and leaves the hidden `accepting` receipt for operational cleanup. If publication succeeds but appending or materializing `accepted` fails, Land still returns the sqid because retrying the RPC would submit a duplicate request that is already in the pipeline. A later pipeline event activates and repairs the public projections.
119
119
120
120
### Request-Log Materialization
121
121
122
-
Every gateway request-log persistence path uses the same materialization component. It appends the audit log, compares the incoming entry with the authoritative summary, conditionally advances the winner, and propagates the authoritative value to the queue projection.
122
+
Every gateway request-log persistence path uses the same materialization component. It appends the audit log, compares the incoming entry with the authoritative summary, and conditionally advances the winner. For public summaries, it idempotently ensures URI mappings before the receipt key, including on unchanged or stale-log retries. Lookup-write failures are returned for retry.
123
123
124
124
The winner comparison preserves the existing current-status reconciliation behavior:
125
125
126
126
1. A terminal request-state entry with a positive request version beats every non-terminal or unversioned winner.
127
127
2. Between versioned terminal entries, the greater request version wins.
128
128
3. Equal terminal versions use the greater log timestamp as a tie-breaker.
129
129
4. When no versioned terminal winner exists, the greater log timestamp wins.
130
-
5.Any retained lifecycle event promotes an `accepting` summary into the public projections.
130
+
5.A retained lifecycle status entry promotes an `accepting` summary into the public projections; audit-only events do not.
131
131
6.`accepted` cannot replace `started` or any later status, even when the accepted log arrives later.
132
132
133
133
Materialization uses optimistic concurrency so stale or out-of-order consumers cannot replace a newer winner. Version arithmetic and reconciliation decisions belong to the materialization component; stores perform only mechanical creates, reads, conditional updates, and bounded page queries.
@@ -144,15 +144,17 @@ The gateway performs a bounded newest-first scan of the URI reverse mapping and
144
144
145
145
### List by Queue and Receipt Time
146
146
147
-
The gateway performs one bounded range scan of the queue projection using queue, receipt-time bounds, and an optional keyset cursor. Queue projections are created only when the request reaches `accepted` or a later state, so `accepting` receipts are absent by construction. The ordering key is immutable, so later status updates cannot move an item across an issued cursor.
147
+
The gateway scans `request_receipt` using the queue, receipt-time bounds, and optional keyset cursor, then point-reads `request_summary` for each returned request. Results preserve receipt-key order; the next token uses the last returned receipt. Later status updates cannot move an item across the immutable cursor.
148
+
149
+
Receipt keys are created only for public summaries. A missing or inconsistent summary fails the page as an internal consistency error rather than silently omitting the request.
148
150
149
151
## Consistency
150
152
151
-
The request log remains the append-only audit record. The authoritative summary and queue projection are eventually consistent views of its winning current state.
153
+
The request log remains the append-only audit record. The authoritative summary is an eventually consistent view of its winning current state.
152
154
153
-
Because the authoritative summary and queue projection are separate writes, a short interval can exist where request-summary retrieval and `List` show different statuses. Retried materialization repairs the queue projection from the authoritative sqid summary until both converge. Neither API reconciles logs during reads to hide this interval.
155
+
List and request-summary retrieval read the same authoritative projection. List membership may lag until the receipt key is written; retries repair partial writes. Separate calls and per-item reads are not a shared snapshot and may observe different lifecycle versions. Neither API reconciles logs during reads.
154
156
155
-
Request context has a stronger guarantee than status convergence: the authoritative receipt is persisted before the request is published to the orchestrator. Public URI and queue projections are activated only by `accepted` or a later event. Once a queue projection is visible, its queue, sqid, change URIs, and receipt time are complete.
157
+
Request context has a stronger guarantee than status convergence: the authoritative receipt is persisted before the request is published to the orchestrator. Public lookup keys are activated only by `accepted` or a later lifecycle state. Once a receipt key is visible, its authoritative summary has complete queue, sqid, change URIs, and receipt time.
0 commit comments