feat(speculation): generator contract and bestfirst impl

Add submitqueue/extension/speculation/generator, the candidate-stream composition point (Generator/PathIterator) the standard Speculator pulls from, plus the bestfirst implementation and mocks.

Open takes only the queue's batches. The generator offers every coherent path in the space, including paths whose builds already ran — suppressing finished paths belongs to the Allocator, the piece that already reconciles candidates against the stored path sets by ID. Keeping the generator ignorant of path sets leaves it a pure enumerator and removes the expand-before-skip subtlety from its iterator: Next pops, expands, returns, with no filtering loop.

bestfirst ranks each path by the product of its dependencies' landing chances (scored through an injected scorer) and builds them lazily. A single heap holds every path built but not yet handed out, across all heads; handing one out builds only the one or two paths that come after it within that same head. Pulling k candidates builds O(k) paths however large the space behind them, so a head with twelve unresolved dependencies costs no more at the front of the queue than one with two.

Within a head, the best path bets the preferred (likelier) way on every unresolved dependency, and its score is precomputed once; every other path applies some subset of the head's flips — a flip is the option to bet one unresolved dependency the unlikely way — and scores bestScore times the applied flips' penalties (alternative over preferred). The heap stores only paths: which flips a path applies is read back off its bets, so there is no parallel flip-set representation to keep in sync. Subsets are walked by a canonical expansion under which every subset has exactly one parent, so each path is built exactly once with no visited set, and a child never outranks its parent — which is what lets the heap hand paths out in true descending order. Equal scores break on path ID among candidates resident in the heap together, which makes a run repeatable rather than globally ID-sorted.

There is no depth bound. It existed to cap 2^n enumeration, and lazy generation removed that cost, so a head may now have as many unresolved dependencies as it likes.

A dependency absent from the live batches cannot be scored, and is assumed very likely to land rather than treated as a coin flip. The even-odds threshold used to pick which way to bet is a separate constant from that default, so the two move independently — sharing one constant would have silently inverted the preferred bet for every dependency below the default.

Tests pin the exact emitted sequence across heads, the preferred bet on a below-even dependency, settled dependencies dropping out of the search, a head table over all eight batch states, the built-path count behind the laziness claim, deterministic tie-breaking, and a randomized cross-check of the whole walk against brute-force enumeration. The generator README carries a worked walkthrough tracing the heap pop by pop.

Also documents on scorer.Scorer that a speculation run scores each batch at most once but does not carry results across runs, so anything expensive belongs behind the implementation's own cache.

Author: behinddwalls

submitqueue/extension/scorer/scorer.go

@@ -27,6 +27,11 @@ type Scorer interface {
 	// Score returns a probability between 0.0 and 1.0 indicating the likelihood
 	// of a successful land for the given batch. It is handed the batch identity
 	// and resolves the batch's changes itself through an injected changeset.Resolver.
+	//
+	// Callers may score every batch a queue is waiting on, so implementations
+	// should be cheap: a speculation run scores each batch at most once, but it
+	// does not carry results over to the next run, so anything expensive to
+	// compute belongs behind the implementation's own cache.
 	Score(ctx context.Context, batch entity.Batch) (float64, error)
 }
 

submitqueue/extension/speculation/generator/BUILD.bazel

@@ -0,0 +1,9 @@
+load("@rules_go//go:def.bzl", "go_library")
+
+go_library(
+    name = "go_default_library",
+    srcs = ["generator.go"],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/generator",
+    visibility = ["//visibility:public"],
+    deps = ["//submitqueue/entity:go_default_library"],
+)

submitqueue/extension/speculation/generator/README.md

@@ -0,0 +1,51 @@
+# generator
+
+The `generator` package is a piece the `standard` `Speculator` is built from: a `Generator` produces the queue's candidate paths as one stream, best first, across all heads. It is **not** controller-facing — the speculate controller only knows the `Speculator` contract, and a different `Speculator` need not split its work this way. So there is no `Config` or `Factory` here; a `Generator` is chosen when the `standard` `Speculator` is constructed.
+
+`Open` starts the stream over the queue's live batches and returns a `PathIterator`. The caller pulls one candidate at a time and the generator does only the work that answer needs. Candidates descend in score, never repeat, and never contradict a known fact. The generator offers every path in the space, including paths whose builds already ran — suppressing finished paths is the `Allocator`'s job, since it is the piece that reconciles candidates against the stored path sets. A candidate's score means something only for the current run; scores are never stored.
+
+## `bestfirst`
+
+A head waiting on unfinished dependencies has one path per combination of outcomes — 2ⁿ of them — while callers want the best few. `bestfirst` hands them out in score order without building the rest.
+
+Dependencies that already finished carry a forced bet: landed means included, failed or cancelled means excluded. They stay in the path but drop out of the search. Each unresolved one is a two-way bet whose chance of landing comes from an injected `scorer.Scorer`, remembered per dependency so one shared by several heads is scored once. A dependency that is not a live batch cannot be scored at all, and is assumed very likely to land — nearly everything a queue is asked to land does. A path's score is its bets' chances multiplied together.
+
+**The best path is not "bet everything lands."** Each bet takes the *likelier* outcome, so a dependency that will probably fail is excluded. Every other path flips some of those bets, and each flip costs a known factor. One consequence worth knowing: applying two cheap flips can beat applying one expensive flip, and the ordering handles that correctly.
+
+**Laziness.** One heap holds every path built but not yet handed out, across all heads. Each path handed out is replaced by the one or two that come after it within the same head, so the heap always holds each head's current best and the top of it is the best path in the queue. Pulling *k* candidates builds about *k* paths, whatever the size of the set behind them.
+
+`bestfirst` discards nothing: pull long enough and every path for every head comes out. It does no conflict relaxation (`dropped` bets) — that is the `Speculator`'s call, not the generator's.
+
+### Worked example
+
+Say a head waits on three dependencies with landing chances d0 0.9, d1 0.8, d2 0.6. All three are better than even, so its best path includes all three and scores 0.9 × 0.8 × 0.6 = 0.432. Sorted cheapest-first the flips come out f0=d2, f1=d1, f2=d0.
+
+Every other path is the best path with some flips applied, written as the set of applied flips: `{f0}` applies only the cheapest, `{f0,f1}` the two cheapest, and so on. `expand` in `iterator.go` grows this space one step at a time — each path handed out queues the one or two that come just after it:
+
+```
+{}      -> {0}
+{0}     -> {0,1}, {1}
+{1}     -> {1,2}, {2}
+{0,1}   -> {0,1,2}, {0,2}
+{2}, {0,2}, {1,2}, {0,1,2} have nothing left to flip above position 2
+```
+
+Pulling all eight paths then goes:
+
+```
+handed out         still waiting afterwards
+{}         .432    {f0}·.288
+{f0}       .288    {f1}·.108  {f0,f1}·.072
+{f1}       .108    {f0,f1}·.072  {f2}·.048  {f1,f2}·.012
+{f0,f1}    .072    {f2}·.048  {f0,f2}·.032  {f1,f2}·.012  {f0,f1,f2}·.008
+{f2}       .048    {f0,f2}·.032  {f1,f2}·.012  {f0,f1,f2}·.008
+{f0,f2}    .032    {f1,f2}·.012  {f0,f1,f2}·.008
+{f1,f2}    .012    {f0,f1,f2}·.008
+{f0,f1,f2} .008    —
+```
+
+The scores come out in order even though nothing is ever sorted, and the waiting list never holds more than four paths for a set of eight. Stop after two and only three paths were ever built.
+
+Look at the fourth row: `{f0,f1}` applies two flips yet still beats `{f2}`, which applies only one. That is correct — the two cheap flips together cost less than the expensive one.
+
+The proof that this walk reaches every path exactly once, in score order, is on `expand` in `iterator.go`. The behavior is pinned by tests in `bestfirst_test.go`.

submitqueue/extension/speculation/generator/bestfirst/BUILD.bazel

@@ -0,0 +1,29 @@
+load("@rules_go//go:def.bzl", "go_library", "go_test")
+
+go_library(
+    name = "go_default_library",
+    srcs = [
+        "bestfirst.go",
+        "iterator.go",
+    ],
+    importpath = "github.com/uber/submitqueue/submitqueue/extension/speculation/generator/bestfirst",
+    visibility = ["//visibility:public"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/scorer:go_default_library",
+        "//submitqueue/extension/speculation/generator:go_default_library",
+    ],
+)
+
+go_test(
+    name = "go_default_test",
+    srcs = ["bestfirst_test.go"],
+    embed = [":go_default_library"],
+    deps = [
+        "//submitqueue/entity:go_default_library",
+        "//submitqueue/extension/scorer:go_default_library",
+        "//submitqueue/extension/speculation/generator:go_default_library",
+        "@com_github_stretchr_testify//assert:go_default_library",
+        "@com_github_stretchr_testify//require:go_default_library",
+    ],
+)

submitqueue/extension/speculation/generator/bestfirst/bestfirst.go

@@ -0,0 +1,278 @@
+// Copyright (c) 2025 Uber Technologies, Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+// Package bestfirst hands out speculation paths for a queue, best ones first.
+//
+// A batch we want to build (a "head") often depends on other batches that have
+// not finished yet. Each unfinished dependency could go either way, so there is
+// more than one sensible thing to build: one path per combination of outcomes.
+// A path's score is how likely its combination is to come true, which is just
+// the individual chances multiplied together.
+//
+// Three words carry the package: a bet is a path's answer on one dependency —
+// included or excluded, forced when the dependency already finished. A flip is
+// the option to answer one unresolved dependency the unlikely way; the head's
+// best path applies no flips, and every other path applies some subset of
+// them. A candidate is one complete set of answers — a path — waiting in the
+// heap with its score.
+//
+// The catch is the arithmetic. A head with n unresolved dependencies has 2^n
+// paths, and the caller normally wants only the best handful. So this package
+// never builds them all. It builds one path per head up front, and builds
+// another only when the caller takes one. Asking for three paths costs about
+// three paths of work, even when the head technically has 4096.
+package bestfirst
+
+import (
+	"context"
+	"math"
+	"sort"
+
+	"github.com/uber/submitqueue/submitqueue/entity"
+	"github.com/uber/submitqueue/submitqueue/extension/scorer"
+	"github.com/uber/submitqueue/submitqueue/extension/speculation/generator"
+)
+
+const (
+	// evenOdds is the midpoint between the two outcomes. A dependency at least
+	// this likely to land is bet to land; below it, the bet goes the other
+	// way. It is only ever a comparison threshold, never a chance in its own
+	// right.
+	evenOdds = 0.5
+
+	// defaultLandingChance is assumed for a dependency that cannot be scored
+	// because it is not among the live batches. Nearly everything a queue is
+	// asked to land does land, so assuming the dependency almost certainly holds
+	// is closer to the truth than a coin flip, and it keeps an unscoreable
+	// dependency from dragging down every path that includes it.
+	defaultLandingChance = 0.95
+)
+
+// gen builds best-first iterators. scorer says how likely a dependency is to
+// land.
+type gen struct {
+	scorer scorer.Scorer
+}
+
+// New returns a best-first generator.Generator. scorer scores each unresolved
+// dependency to get its chance of landing.
+func New(sc scorer.Scorer) generator.Generator {
+	return gen{scorer: sc}
+}
+
+// Open works out each head's flips and queues that head's best path, ready to
+// hand out. All the scoring happens here, which is why Next never calls the
+// scorer and never fails.
+func (g gen) Open(ctx context.Context, batches []entity.Batch) (generator.PathIterator, error) {
+	batchByID := make(map[string]entity.Batch, len(batches))
+	for _, b := range batches {
+		batchByID[b.ID] = b
+	}
+	chances := newChanceCache(g.scorer, batchByID)
+
+	it := &iterator{}
+	for _, batch := range batches {
+		// Only Speculating heads get paths. Batches in any other state (created,
+		// merging, cancelling, finished) still tell us how their dependencies
+		// turned out, but we never propose work on them.
+		if batch.State != entity.BatchStateSpeculating {
+			continue
+		}
+		h, err := newHead(ctx, batch, batchByID, chances)
+		if err != nil {
+			return nil, err
+		}
+		// Queue the head's best path — no flips applied, every dependency bet
+		// the likely way. It is the only path built up front; the rest grow out
+		// of it as the caller pulls.
+		it.push(h, nil)
+	}
+	return it, nil
+}
+
+// chanceCache answers "how likely is this dependency to land", scoring each one
+// at most once. Several heads usually wait on the same dependency, and there is
+// no reason to score it once per head.
+type chanceCache struct {
+	scorer  scorer.Scorer
+	batches map[string]entity.Batch
+	chances map[string]float64
+}
+
+func newChanceCache(sc scorer.Scorer, batches map[string]entity.Batch) *chanceCache {
+	return &chanceCache{scorer: sc, batches: batches, chances: make(map[string]float64)}
+}
+
+// of returns the chance the given dependency lands.
+func (c *chanceCache) of(ctx context.Context, batchID string) (float64, error) {
+	if chance, ok := c.chances[batchID]; ok {
+		return chance, nil
+	}
+	batch, ok := c.batches[batchID]
+	if !ok {
+		// Not a live batch, so there is nothing to score. Remember the default
+		// too, so we don't look it up again.
+		c.chances[batchID] = defaultLandingChance
+		return defaultLandingChance, nil
+	}
+	chance, err := c.scorer.Score(ctx, batch)
+	if err != nil {
+		return 0, err
+	}
+	c.chances[batchID] = chance
+	return chance, nil
+}
+
+// flip is the option to bet one unresolved dependency the unlikely way.
+//
+// Each unresolved dependency has a likelier outcome (the "preferred" bet, what
+// the head's best path uses) and an unlikelier one (the "alternative"). The
+// best path applies no flips; every other path applies some subset of them,
+// each multiplying the path's score by that flip's penalty. A penalty near 1
+// means the flip barely costs anything (the dependency was close to a coin
+// flip); a penalty near 0 means it is very expensive.
+//
+// A flip holds no applied/unapplied state of its own: each path applies its
+// own subset, readable off the path's bets.
+type flip struct {
+	// betIndex is this dependency's position in the head's bets (queue order).
+	// The flips slice gets sorted by penalty, which scrambles that order, so
+	// each flip carries the index it came from — that is how an applied flip's
+	// bet is written back in queue order.
+	betIndex int
+	// alternativeBet is the bet a path that applies this flip carries.
+	alternativeBet entity.DependencyBetType
+	// penalty is alternative/preferred: the score multiplier for applying this
+	// flip, always in [0, 1].
+	penalty float64
+}
+
+// head is one batch we might build, and everything needed to enumerate its
+// paths. It holds no heap of its own — every head's paths compete in the
+// iterator's single heap.
+type head struct {
+	// id is the batch these paths are for.
+	id string
+	// bets is the best path's bets, in queue order: the known bet where the
+	// dependency has already finished, the preferred bet everywhere else.
+	bets []entity.DependencyBet
+	// flips is the option to flip each unresolved dependency, cheapest first.
+	flips []flip
+	// bestScore is the best path's score: the preferred chances multiplied
+	// together. Every other path's score is bestScore times its applied flips'
+	// penalties.
+	bestScore float64
+}
+
+// newHead works out a head's search space. Dependencies that already finished
+// contribute a known bet and drop out of the search; each of the rest becomes
+// a flip, ordered so the cheapest one comes first.
+func newHead(
+	ctx context.Context,
+	batch entity.Batch,
+	batchByID map[string]entity.Batch,
+	chances *chanceCache,
+) (*head, error) {
+	bets := make([]entity.DependencyBet, len(batch.Dependencies))
+	flips := make([]flip, 0, len(batch.Dependencies))
+	bestScore := 1.0
+	for i, dep := range batch.Dependencies {
+		if bet, settled := settledBet(batchByID[dep].State); settled {
+			bets[i] = entity.DependencyBet{Batch: dep, Bet: bet}
+			continue
+		}
+		chance, err := chances.of(ctx, dep)
+		if err != nil {
+			return nil, err
+		}
+		// Bet the likelier outcome. On an exact coin flip, bet it lands.
+		preferredBet, alternativeBet := entity.BetIncluded, entity.BetExcluded
+		if chance < evenOdds {
+			preferredBet, alternativeBet = entity.BetExcluded, entity.BetIncluded
+		}
+		// chance is the likelihood the dependency lands, so whichever way we bet,
+		// preferred is the bigger of chance and 1-chance.
+		preferred, alternative := math.Max(chance, 1-chance), math.Min(chance, 1-chance)
+		bets[i] = entity.DependencyBet{Batch: dep, Bet: preferredBet}
+		bestScore *= preferred
+		flips = append(flips, flip{
+			betIndex:       i,
+			alternativeBet: alternativeBet,
+			// preferred is at least 0.5, so this never divides by zero.
+			penalty: alternative / preferred,
+		})
+	}
+	// Cheapest flip first — the bet we'd mind changing least. expand depends on
+	// this order: it only ever moves an applied flip to the next position along,
+	// so having them sorted is what guarantees a path never scores above the one
+	// it came from.
+	sort.Slice(flips, func(i, j int) bool {
+		if flips[i].penalty != flips[j].penalty {
+			return flips[i].penalty > flips[j].penalty
+		}
+		return flips[i].betIndex < flips[j].betIndex
+	})
+
+	return &head{id: batch.ID, bets: bets, flips: flips, bestScore: bestScore}, nil
+}
+
+// settledBet reports the bet a dependency's state forces, if that state is
+// final. An unresolved dependency has no forced bet and stays in the search.
+func settledBet(state entity.BatchState) (entity.DependencyBetType, bool) {
+	switch state {
+	case entity.BatchStateSucceeded:
+		// Already landed, so including it is a fact, not a bet at risk.
+		return entity.BetIncluded, true
+	case entity.BatchStateFailed, entity.BatchStateCancelled:
+		// Never landing, so leaving it out is a fact too.
+		return entity.BetExcluded, true
+	default:
+		return entity.BetUnknown, false
+	}
+}
+
+// pathFor builds the path that applies the given flips, and its score. applied
+// holds ascending positions in h.flips. It starts from the head's best path,
+// switches each applied flip's dependency to its alternative bet, and pays that
+// flip's penalty. Dependencies that already finished are certainties, so they
+// have no flip and never change the score.
+func (h *head) pathFor(applied []int) (entity.SpeculationPath, float64) {
+	// Work on a copy. h.bets is the template every one of this head's paths
+	// starts from, so writing a flip straight into it would corrupt every path
+	// built after this one.
+	bets := make([]entity.DependencyBet, len(h.bets))
+	copy(bets, h.bets)
+
+	score := h.bestScore
+	for _, i := range applied {
+		f := h.flips[i]
+		bets[f.betIndex].Bet = f.alternativeBet
+		score *= f.penalty
+	}
+	return entity.SpeculationPath{Head: h.id, Bets: bets}, score
+}
+
+// appliedFlips reads back which of h's flips a path applies, as ascending
+// positions in h.flips: exactly the ones whose dependency the path bets the
+// alternative way. It inverts pathFor, so the iterator needs no flip
+// bookkeeping on the paths it holds.
+func (h *head) appliedFlips(path entity.SpeculationPath) []int {
+	var applied []int
+	for i, f := range h.flips {
+		if path.Bets[f.betIndex].Bet == f.alternativeBet {
+			applied = append(applied, i)
+		}
+	}
+	return applied
+}