uber
diff --git a/‎doc/rfc/submitqueue/speculation-generator.md‎
Lines changed: 151 additions & 0 deletions b/‎doc/rfc/submitqueue/speculation-generator.md‎
Lines changed: 151 additions & 0 deletions
@@ -0,0 +1,151 @@
+# Speculation Path Generation
+
+## Summary
+
+The merge queue can build a batch early by guessing whether each batch it conflicts with will succeed or fail. These guesses create many possible build paths, but CI can run only a few. The default generator returns the most likely paths first and stops when the build budget is full.
+
+It does that by starting from each batch's single most likely path and generating every other path as a deviation from it, most probable next, across all batches at once. Paths come out in score order, and only the paths actually requested are ever created. In the architecture of [speculation.md](speculation.md), this is the Generator inside the default Speculator. This RFC records that design — already implemented — and the alternatives considered and rejected.
+
+## The problem
+
+A batch that conflicts with n undecided batches has one possible path per combination of outcomes — 2ⁿ in all. A batch overlapping ten others has 1,024 possible futures, and CI can afford perhaps two or three of them. The rest of the system asks the generator for candidates one at a time, best first, and stops asking once the build budget is full.
+
+So the generator must return paths in order of how likely they are to be useful, never return the same path twice, never return a path that contradicts a known outcome, and do work proportional to the number of paths requested — not to 2ⁿ.
+
+## The queue in this doc
+
+One worked example runs through the whole document. Four batches are in flight — A, B, and C all conflict with each other, and D stands alone:
+
+```
+main ◀── A ◀── B        B conflicts with A
+         ▲     ▲
+         └─ C ─┘        C conflicts with both A and B
+    D                   D conflicts with nothing
+```
+
+This is the **DAG**: the queue's batches and the dependency **edges** between them — an edge is a direct conflict making one batch depend on another. Each batch stores exactly the batches it directly conflicts with, fixed when the batch is created: A and D store nothing, B stores `[A]`, C stores `[A, B]`. A batch guesses only about its own direct conflicts — a batch that conflicted only with B would carry no guess about A; A could still delay its merge, but only through B, and that is the merge stage's business ([speculation.md](speculation.md)), not the generator's.
+
+A pluggable scorer estimates how likely each batch's build is to succeed. Only batches that appear in someone's conflict list are ever asked about — A and B here; nothing conflicts with C or D, so their probabilities never matter:
+
+| conflict | P(succeeds) |
+| --- | --- |
+| A | 0.9 |
+| B | 0.8 |
+
+## Every possible path, drawn as a tree
+
+A **path** is one guess — *succeeds* or *fails* — per direct conflict of the batch being evaluated. (speculation.md calls that batch the path's *head*.) Its **score** is the probability that every guess comes true, and it is computed by plain multiplication: start at 1, and for each guess multiply by p if the guess is *succeeds* or by 1−p if it is *fails*.
+
+That multiplication is easiest to see as a tree — one branching level per conflict, in queue order. Start at the top with score 1: nothing guessed yet, so nothing can be wrong. Every branch taken multiplies the score. Every leaf is one complete path, and its score is the product collected on the way down:
+
+```
+ batch A — no conflicts        batch B — conflicts [A]          batch C — conflicts [A, B]
+
+      start 1.0                     start 1.0                           start 1.0
+                                ┌───────┴───────┐                 ┌──────────┴──────────┐
+   already a leaf:           A✓ ×0.9         A✗ ×0.1          A✓ ×0.9               A✗ ×0.1
+   one path, guess              │               │                 │                     │
+   nothing, score 1            0.9             0.1               0.9                   0.1
+                              leaf            leaf         ┌──────┴──────┐       ┌──────┴──────┐
+ batch D — same as A:                                   B✓ ×0.8      B✗ ×0.2  B✓ ×0.8      B✗ ×0.2
+ one path, score 1                                          │            │        │            │
+                                                          0.72          0.18     0.08         0.02
+```
+
+Read A's tree first, because it answers the question the notation invites. A conflicts with nothing, so there is nothing to guess: the start is already a leaf, and A has exactly one path — build A directly, guess nothing, score 1. D is the same. B has two paths: build on top of A (`[A✓]`, 0.9) or build without A (`[A✗]`, 0.1). C has four, from `[A✓ B✓]` at 0.9 × 0.8 = 0.72 down to `[A✗ B✗]` at 0.1 × 0.2 = 0.02.
+
+A path is self-contained, and its identity is the batch plus its guesses. Because the stored conflict list is fixed when the batch is created, a path's identity never changes, however many runs revisit it.
+
+Nobody builds these trees up front — they are the picture of the space. The next section is about how the generator hands out the leaves in score order without ever drawing the whole picture.
+
+## The decision: start at the most likely leaf, flip outward
+
+Each tree has a **most likely leaf**: take the likelier branch at every level (at exactly 0.5, succeeds counts as likelier). Note that is not always "everything succeeds" — a conflict that will probably fail is guessed to fail. In the example every conflict leans succeeds, so the most likely leaves are `A []` (1.0), `D []` (1.0), `B [A✓]` (0.9), and `C [A✓ B✓]` (0.72).
+
+Every other leaf is the most likely leaf with some guesses **flipped** to the other side. A flip always costs the same ratio no matter what else is flipped: flipping B swaps the ×0.8 for ×0.2, multiplying the whole product by 0.2/0.8. Check it on C's tree: 0.72 × (0.2/0.8) = 0.18, and 0.18 × (0.1/0.9) = 0.02. A flip's ratio is at most 1 — flipping to the unlikelier side can only lower a score — so the most likely leaf really is the best, one flip is next, and so on outward.
+
+## How generation works, step by step
+
+The generator keeps one list of paths across all batches, ordered by score — a priority queue in the implementation. It starts with just the four most likely leaves on the list, then repeats two plain steps each time a candidate is requested:
+
+1. Take the highest-scoring path off the list and return it.
+2. Before returning it, put its follow-up variants on the list: the same path with the next-cheapest flip added, and — if it already has flips — the same path with its newest flip traded for the next-cheapest one. Between those two moves, every combination of flips reaches the list exactly once.
+
+Walking the example one request at a time (the two 1.0 paths order by batch ID, per the ties rule):
+
+| request | returned (score) | follow-ups put on the list | the list afterwards |
+| --- | --- | --- | --- |
+| 1 | **A: `[]`** (1.0) | — (nothing to flip) | `D []` 1.0 · `B [A✓]` 0.9 · `C [A✓B✓]` 0.72 |
+| 2 | **D: `[]`** (1.0) | — | `B [A✓]` 0.9 · `C [A✓B✓]` 0.72 |
+| 3 | **B: `[A✓]`** (0.9) | `B [A✗]` (0.1) | `C [A✓B✓]` 0.72 · `B [A✗]` 0.1 |
+| 4 | **C: `[A✓ B✓]`** (0.72) | `C [A✓B✗]` (0.18) | `C [A✓B✗]` 0.18 · `B [A✗]` 0.1 |
+
+Four candidates out — the unconditional builds of A and D, B on top of A, C on top of both — in exactly the order a human would fund them. If the caller kept asking: request 5 returns `C [A✓B✗]` (0.18) and queues its two variants `C [A✗B✓]` (0.08) and `C [A✗B✗]` (0.02); then 0.1, 0.08, 0.02 follow, descending all the way.
+
+The point is what happens when the caller *stops*. After four requests, exactly four paths have been returned and only two more exist anywhere — the two on the list. On C's tree, the leaves 0.08 and 0.02 were never created; a batch nobody pulls from never even works out its flip order. Generating k paths creates k paths; the 2ⁿ space is never enumerated.
+
+The guarantees, each with its one-line reason:
+
+- **Candidates come out in non-increasing score.** A variant only ever adds or trades up to costlier flips, so it never outscores the path it came from; taking the highest entry first means no later path can beat an earlier one.
+- **Every leaf exactly once.** Each variant is created by exactly one of the two moves from exactly one path, so continuing until the list is empty returns every batch's every path, none twice.
+- **Work is proportional to what is requested.** Each request returns one path and puts at most two on the list.
+- **Impossible guesses are not silently dropped.** A side with probability 0 makes its flip ratio 0 — the path scores 0 and sorts after every possible one, but is still returned if the caller keeps asking. Nothing is discarded.
+
+## One implementation note: scores are stored as logarithms
+
+Everything above multiplies probabilities, and that ranking is exactly what the implementation preserves — but it cannot multiply literally. A batch with 7,100 conflicts at 0.9 each has a best score of 0.9⁷¹⁰⁰ ≈ 10⁻³²⁵, smaller than the smallest number float64 can represent: the product rounds to exactly 0, every path of that batch ties at zero, and the order is lost.
+
+So the implementation stores every score as its logarithm, which turns multiplication into addition:
+
+```
+log(p₁ × p₂ × ⋯ × pₙ) = log p₁ + log p₂ + ⋯ + log pₙ
+```
+
+Because log is strictly increasing, comparing the logs orders paths exactly as comparing the products would — and the sums stay comfortably finite: that wide batch's best value is 7,100 × log 0.9 ≈ −748, an ordinary float64. On C: 0.72 becomes log 0.9 + log 0.8 = −0.328, and a flip's ratio becomes a subtraction (B's flip costs log 0.2 − log 0.8 = −1.386, taking −0.328 to −1.715, and e^−1.715 ≈ 0.18 — the score we started with). The log is what a candidate actually carries out of the generator as its ranking value; it orders candidates within one run and means nothing across runs — the next run rescores from scratch.
+
+## Example across two runs
+
+The generator keeps no state between runs: each run starts from the queue's current state, and what changed since last time shows up as fewer guesses to make. (What the rest of the system does with the candidates — funding, cancelling, merging — is covered in [speculation.md](speculation.md).)
+
+**Run 1.** The queue is as above; the four most probable paths are the ones the trace returned:
+
+```
+A: []  1.0   ·   D: []  1.0   ·   B: [A✓]  0.9   ·   C: [A✓ B✓]  0.72
+```
+
+**Between runs.** A's build fails. A had one unconditional path, so no other future exists in which it passes: A's outcome is now known.
+
+**Run 2.** A is no longer a batch being evaluated, so it proposes nothing. As a *conflict* it is now a fact, not a guess: every path fixes A to the known outcome (*fails*), and A's level vanishes from the trees — the paths still record A✗, but nothing branches on it anymore:
+
+```
+ batch B — nothing left to guess        batch C — only B still worth guessing about
+
+     [A✗]  1.0                                    start 1.0
+                                            ┌────────┴────────┐
+  A's failure is a fact;                 B✓ ×0.8           B✗ ×0.2
+  the tree is a single leaf                 │                 │
+                                       [A✗ B✓] 0.8       [A✗ B✗] 0.2
+```
+
+The first candidates returned are `B: [A✗]` and `D: []`, both at score 1.0, then `C: [A✗ B✓]` at 0.8. Look at B's closely: it is the *same path* that scored 0.1 in run 1 — the guesses are identical, so its identity is identical — but what was a long shot is now a certainty, because A's failure stopped being a probability and became a fact. Scores mean nothing across runs; a path's identity is what survives, which is how the rest of the system recognizes paths it already acted on. And no path returned in run 2 contradicts the known outcome: every path guessing A✓ is simply no longer generated.
+
+## Edge cases and ties
+
+Paths guess only about what is genuinely undecided; everything else is settled before generation starts.
+
+- **A conflict whose outcome is known** is fixed to it — Succeeded fixes *succeeds*; Failed or Cancelled fixes *fails* — and drops out of the search, exactly as A did in run 2. The fixed guess is still recorded on every path, so paths stay self-contained. A conflict that is still being cancelled is not yet known and stays a live guess.
+- **A conflict absent from the run's snapshot** cannot be scored. Each run hands the generator the batches the controller read, and a stored conflict ID with no batch among them has nothing to score. It stays a live guess with a fixed default probability of succeeding, 0.95, a constant in the generator chosen on the observation that nearly every batch a queue accepts does build successfully; it only affects ranking, never which paths exist.
+- **Each unique conflict is scored once per run**, however many conflict lists it appears in. In the example, A appears in both B's and C's lists but costs one scorer call; the whole queue costs two (A and B — nothing conflicts with C or D, so they are never scored). Returning candidates triggers no further scoring: by the first request, every probability the run needs is known.
+- **Exact ties need a fixed order**, or two runs over the same queue could propose different paths. Ties break, in order: higher score, then fewer flips, then which flips were taken, then the batch's ID. Fewer-flips comes before batch ID so that every batch's most likely path is offered before any batch's equally-scored deviation: a conflict at exactly 0.5 flips for free, and ordering on the batch instead would let one batch's tied deviations absorb the whole budget before another batch's most likely path was offered at all. Two batches each with one conflict at 0.5 therefore return as *first batch's likelier path, second batch's likelier path, first's flip, second's flip* — the batches interleave. Because every combination reaches the list exactly once and the order is total, every step has a unique winner and repeated runs agree.
+
+## Alternatives considered
+
+**Enumerate and sort.** Compute every path of every batch, sort by score, return from the top. Simplest possible model, but the work is 2ⁿ per batch per run whether or not anyone asks for more than two paths. Rejected for cost.
+
+**A forward decision tree per batch.** Walk the trees above directly: keep *partial* paths on the shared list, and split the highest-scoring one at its next level (one copy guessing succeeds, one guessing fails) until complete paths fall out the bottom. It returns the same paths in the same order, and its state is arguably more intuitive — an entry on the list *is* a partial path with a real score. But it does more list work (reaching a leaf costs one operation per conflict instead of one per path), and it replaces a correct, tested implementation without changing the contract at all. Rejected as churn: same output, no new capability.
+
+**Trees over derived dependency chains (the DAG walk).** Extend the tree idea so a batch guesses about its whole *chain* of ancestors — walk the DAG from its roots down, and give a batch that conflicts only with B a guess about A as well, so that one path pins the entire stack beneath it. Rejected on the dependency-closure problem. Batches store direct conflicts only, so the chain would have to be derived fresh every run by walking the DAG — and the derived set changes shape as links finalize, are cancelled, or drop out of the snapshot. A path's identity is the hash of its full guess list, so the same logical guess could change identity from one run to the next, breaking the matching that lets an already-funded path keep its CI slot instead of being rebuilt. It also spreads the derivation beyond the generator — the controller's validation of returned paths and its snapshot reads would all have to walk the same chains the same way. The chosen design needs none of that: a path covers exactly the batch's stored conflict list, fixed at creation.
+
+## Implementation impact
+
+None — this RFC records the design the generator package already implements; no code changes follow from it. Interfaces, entities, scoring, and behavior are all as described. One unrelated correction rides behind it: the entity comment on the stored dependency list claims it holds the transitive closure of all dependencies, while it holds direct conflicts only; that comment will be fixed separately.