diff --git a/README.md b/README.md index 48f4a6d..5d0061c 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ These are load-bearing. They show up in every prompt and every config default. - **Grounding over generation.** Research directives can't invent URLs or tool names. Every citation must come from an allowlist built from the source register entry; every tool call must exact-match an allowlist. A verifier scans the output for fabrication; anything dubious ships with a `⚠ FLAGGED ISSUES` block. - **Cross-domain is a first-class move.** On surprising findings, an analog probe asks the engine which *distant* fields have structural analogs. On unsurprising-but-confirmed findings, an assumption probe surfaces premises the field takes for granted. A negative-space scan maps which `(method, problem)` combinations *aren't* in the journal yet. - **Audit trails, not overwrites.** Re-verification appends to a log instead of mutating the original verdict — both old and new are preserved so you can see what changed under updated rules. +- **Lessons carry forward as edges, not beliefs.** Every few cycles the engine clusters related investigations (connected components of the knowledge graph) and distills each cluster into a *frontier edge* — the unexplored synthesis or contradiction at its boundary — which is injected into the next introspection as something to push *past*, never to confirm. The distillation runs on the cross-family verifier model so the generator never authors and then consumes its own prior. (Direction backpropagation — see [Self-evolution](#self-evolution-phases-6-9-borrowed-from-comparable-public-systems).) - **Domain-agnostic by construction.** Prompts use shape, not content. The same engine works identically for any structured field. --- @@ -266,6 +267,7 @@ Once the verifier side stabilized, we audited public research-agent systems for - **Phase 7** (persona-conditioned introspection, planned) — borrows multi-perspective question generation from **[Stanford STORM / Co-STORM](https://github.com/stanford-oval/storm)**. Each persona (skeptic / outsider / historian / contrarian / practitioner) surfaces blind spots the single-voice introspection misses. - **Phase 8** (idea evolution from downgraded extensions, planned) — borrows the mutation loop from **[Sakana AI's "AI Scientist"](https://github.com/SakanaAI/AI-Scientist)** and the Evolution agent from Co-Scientist. Internally aligns with `r-3c792e21` ("typed supervision from false positives via retrospective unification") — we treat verifier downgrades as typed supervision signal for generator-side mutation. - **Phase 9** (hypothesis variants in investigation, planned) — borrows branching exploration from **[Tree of Thoughts](https://arxiv.org/abs/2305.10601)**. The explorer persona generates N divergent priors; the most-distant-from-majority-literature variant drives the investigation. +- **Direction insight backpropagation** — borrows the *carry-lessons-across-time* mechanism from **Arbor / Hypothesis Tree Refinement ([arXiv:2606.11926](https://arxiv.org/abs/2606.11926))**, whose ablation showed that propagating distilled insights matters more than the tree structure that holds them (removing propagation hurt more than removing the tree). CE adapts it to a *novelty* objective rather than Arbor's scalar one: the carried signal is a frontier edge to **exceed**, not a belief to confirm, because CE's value is divergence (OOD synthesis) where Arbor's is convergence to a measured optimum. Deliberately omits Arbor's frontier *pruning* — for a novelty engine there is no clean falsification signal, and pruning on a weak proxy (repeated low surprise) would kill exactly the directions that pay off late. ### General agentic patterns CE builds on @@ -279,6 +281,7 @@ Once the verifier side stabilized, we audited public research-agent systems for - **Pareto-dominance admission** (Phase 4) is multi-objective optimization theory applied to a register-admission gate. - **Negative-space mapping** (the `(method × problem)` matrix) is a long-standing literature-review discipline; CE just instruments it. - **Falsifiable predictions with target dates** is descended from prediction-market and forecasting-literature practice (Tetlock, Good Judgement Project, Metaculus). +- **The adversarial verifier is CE's held-out gate.** Arbor ([arXiv:2606.11926](https://arxiv.org/abs/2606.11926)) admits an improvement only if it beats the current best on a *held-out* eval — separating "looked good on the exploration signal" from verified progress, which is what keeps it from overfitting. CE's adversarial prior-art search plays the same *role*: apparent novelty measured against the journal (the dev signal) is admitted only after surviving search against the literature (the held-out set). Local novelty that collapses under prior-art search is CE's "overfit." This is an analogy of role, not identity of mechanism — CE's gate is structural prior-art search, not a scalar metric comparison. CE deliberately does **not** adopt a numeric dev/held-out gate: a scalar admission target would push the engine toward optimizing a novelty *proxy*, violating "novelty is structural, not vibes." ### Where CE is genuinely novel @@ -497,6 +500,12 @@ python curiosity_engine.py --backfill-canonical-forms --backfill-force # re-ca python curiosity_engine.py --synth-orphaned-xrefs # recover from mid-run crashes python curiosity_engine.py --scan-gaps # negative-space scan +# Direction insight backpropagation (Arbor / HTR) — runs automatically every +# [engine].direction_abstract_every_n_cycles cycles; these force/inspect it +python curiosity_engine.py --abstract-directions # force a distillation run now +python curiosity_engine.py --show-directions # show the injected frontier edges +python curiosity_engine.py --suppress-direction dir-abc12345 # kill a bad prior (stops injection) + # Research directives python curiosity_engine.py --export-directive r-cd730b6d # per-record python curiosity_engine.py --export-directives-bundle # all qualifying diff --git a/config.py b/config.py index 1b42c41..484f45e 100644 --- a/config.py +++ b/config.py @@ -247,6 +247,28 @@ class EngineSettings: # Set to a non-zero value (e.g. 0.70) on mature journals where you # specifically want to prune low-priority noise. question_priority_floor: float = 0.0 + # ── Direction insight backpropagation (Arbor / HTR, arXiv:2606.11926) ── + # Abstract clusters of related investigations into direction-level priors + # and inject them into introspection as FRONTIER EDGES to push past — the + # carry-forward layer CE was missing. The framing is deliberately + # divergent ("exceed these edges", not "confirm these beliefs"); abstraction + # runs on the cross-family verifier model, never the primary, to avoid the + # self-flattery loop where the generator authors and then consumes its own + # prior. See docs/superpowers/specs/2026-06-18-direction-backprop-design.md. + direction_backprop_enabled: bool = True + # In-loop trigger cadence: abstract directions every N cycles, before + # introspection, so fresh priors are available to the same cycle. + direction_abstract_every_n_cycles: int = 5 + # Journal-size floor — abstraction stays silent below this many entries + # (small journals have no stable directions yet, like negative_space). + direction_min_entries: int = 8 + # Minimum connected-component size (entries) to qualify as a direction. + direction_min_cluster_size: int = 4 + # Max directions abstracted per run (largest components first) — bounds cost. + direction_max_count: int = 5 + # Max frontier edges injected into introspection — the convergence dial. + # Lower = safer (priors can't dominate the prompt / homogenize personas). + direction_max_injected: int = 4 # Parallel fan-out — how many investigations / xref-synth+verify pipelines # run concurrently per cycle. Default 1 = fully serial (zero behavior change). # Rate limiters are shared across threads so raising these does not burst @@ -254,6 +276,33 @@ class EngineSettings: # pipelines; beyond that most time is spent waiting on rate limiters anyway. parallel_investigations: int = 1 parallel_xref_pipeline: int = 1 + # ── Selective verification pipeline (A2 plumbing for Phases B/C/D) ── + # All defaults are NO-OPS — A2 alone changes no behavior. Each knob + # turns on when the corresponding phase ships. + # + # Phase B (alias-gap routing). Candidates with alias_gap below the reject + # threshold are short-circuited to the rejection_log without an LLM + # verification call. Candidates above the fast-track threshold skip + # deep prior-art search and take the cheap path. The middle band runs + # the existing full pipeline. 0.0 / 1.0 = no-op (no gating, every + # candidate runs full pipeline). + alias_gap_reject_threshold: float = 0.0 + alias_gap_fasttrack_threshold: float = 1.0 + # Phase B audit-back: fraction of below-reject-threshold candidates that + # still get full LLM verification, tagged in the rejection_log as audit + # samples. Detects discriminator drift — if audit-back items keep + # validating, the reject threshold is too aggressive. 0.0 = never + # sample-back (no-op). + rejection_audit_back_rate: float = 0.0 + # Phase C (paraphrase-perturbation verifier stability). Run verification + # against N paraphrase variants of the prompt, compute verdict variance + # as a paraphrase_inconsistency_score. 1 = single pass (no-op, + # pre-Phase-C behavior). 3 = recommended once C ships. + paraphrase_variant_count: int = 1 + # Phase D (committee escalation). When paraphrase_inconsistency_score + # exceeds this threshold (or committee verdicts disagree), escalate + # to a second cross-family verifier. 1.0 = never escalate (no-op). + committee_dissent_threshold: float = 1.0 CONFIG_DIR = Path.home() / ".CuriosityEngine" CONFIG_PATH = CONFIG_DIR / "engine.toml" @@ -403,6 +452,24 @@ def load(cls, path: Path = CONFIG_PATH) -> CuriosityEngineConfig: question_priority_floor=float( eng_section.get("question_priority_floor", 0.70) ), + direction_backprop_enabled=bool( + eng_section.get("direction_backprop_enabled", True) + ), + direction_abstract_every_n_cycles=max( + 1, int(eng_section.get("direction_abstract_every_n_cycles", 5)) + ), + direction_min_entries=max( + 1, int(eng_section.get("direction_min_entries", 8)) + ), + direction_min_cluster_size=max( + 2, int(eng_section.get("direction_min_cluster_size", 4)) + ), + direction_max_count=max( + 1, int(eng_section.get("direction_max_count", 5)) + ), + direction_max_injected=max( + 0, int(eng_section.get("direction_max_injected", 4)) + ), register_admission_mode=str( eng_section.get("register_admission_mode", "scalar") ).strip().lower() or "scalar", @@ -440,6 +507,21 @@ def load(cls, path: Path = CONFIG_PATH) -> CuriosityEngineConfig: investigation_assessor_role=str(eng_section.get("investigation_assessor_role", "")).strip(), parallel_investigations=int(eng_section.get("parallel_investigations", 1)), parallel_xref_pipeline=int(eng_section.get("parallel_xref_pipeline", 1)), + alias_gap_reject_threshold=float( + eng_section.get("alias_gap_reject_threshold", 0.0) + ), + alias_gap_fasttrack_threshold=float( + eng_section.get("alias_gap_fasttrack_threshold", 1.0) + ), + rejection_audit_back_rate=float( + eng_section.get("rejection_audit_back_rate", 0.0) + ), + paraphrase_variant_count=max( + 1, int(eng_section.get("paraphrase_variant_count", 1)), + ), + committee_dissent_threshold=float( + eng_section.get("committee_dissent_threshold", 1.0) + ), ) # Resolve cross_ref profile: @@ -791,6 +873,19 @@ def _build_toml( # before the journal could build context). Set to a non-zero value only on # mature journals where you specifically want to prune low-priority noise. question_priority_floor = {eng.question_priority_floor} +# Direction insight backpropagation (Arbor / HTR, arXiv:2606.11926). Every N +# cycles, cluster related investigations and distill each into a FRONTIER EDGE +# injected into introspection as something to push PAST (never confirm). +# Abstraction runs on the verifier model to avoid a self-flattery loop. +# Set direction_backprop_enabled = false to disable entirely. +direction_backprop_enabled = {str(eng.direction_backprop_enabled).lower()} +direction_abstract_every_n_cycles = {eng.direction_abstract_every_n_cycles} +direction_min_entries = {eng.direction_min_entries} +direction_min_cluster_size = {eng.direction_min_cluster_size} +direction_max_count = {eng.direction_max_count} +# Max frontier edges injected per introspection — the convergence dial; lower +# is safer (priors can't dominate the prompt / homogenize the personas). +direction_max_injected = {eng.direction_max_injected} # Register admission mode. "scalar" = single confidence floor + status checks # (default; backward compatible). "pareto" = ALSO require the new entry to be # non-dominated by any existing active entry across the 4-axis Pareto set @@ -889,6 +984,24 @@ def _build_toml( # 2–3 xref pipelines before rate-limit waits dominate anyway. parallel_investigations = {eng.parallel_investigations} parallel_xref_pipeline = {eng.parallel_xref_pipeline} +# ── Selective verification pipeline (A2 plumbing for Phases B/C/D) ── +# All defaults below are NO-OPS until the corresponding phase ships. +# Phase B — alias-gap routing. Candidates below reject_threshold short- +# circuit to rejection_log without LLM verification; above +# fasttrack_threshold skip deep prior-art search. Middle band runs the +# full pipeline. 0.0 / 1.0 = no gating (current behavior). +alias_gap_reject_threshold = {eng.alias_gap_reject_threshold} +alias_gap_fasttrack_threshold = {eng.alias_gap_fasttrack_threshold} +# Phase B audit-back — fraction of below-reject-threshold candidates that +# still get full LLM verification, tagged as audit samples. Detects +# discriminator drift. 0.0 = no audit-back (current behavior). +rejection_audit_back_rate = {eng.rejection_audit_back_rate} +# Phase C — paraphrase-perturbation. N variants per verification, verdict +# variance scored as paraphrase_inconsistency_score. 1 = single pass. +paraphrase_variant_count = {eng.paraphrase_variant_count} +# Phase D — committee escalation when paraphrase_inconsistency_score +# exceeds threshold (or verdicts disagree). 1.0 = never escalate. +committee_dissent_threshold = {eng.committee_dissent_threshold} """ ) return header + "\n".join(sections) diff --git a/curiosity_engine.py b/curiosity_engine.py index c42ceee..3b94827 100644 --- a/curiosity_engine.py +++ b/curiosity_engine.py @@ -119,6 +119,12 @@ def main(): help="Synthesize + verify every cross-reference that doesn't yet have a matching insight (e.g. after a mid-run failure between cross-ref and synthesis).") parser.add_argument("--scan-gaps", action="store_true", help="Run a negative-space scan: build (method × problem) matrix from journal entries, classify empty cells, verify underexplored gaps via academic_search, enqueue questions for verified gaps. Gated by [engine].negative_space_min_entries.") + parser.add_argument("--abstract-directions", action="store_true", + help="Direction insight backpropagation: cluster related investigations (connected components of the entry graph), abstract each into a frontier-edge prior on the verifier model, and persist it for injection into introspection. Runs automatically every [engine].direction_abstract_every_n_cycles cycles; this forces a run now. Gated by [engine].direction_min_entries.") + parser.add_argument("--show-directions", action="store_true", + help="Print the current (latest, non-suppressed) direction insights — the frontier edges injected into introspection.") + parser.add_argument("--suppress-direction", type=str, default=None, metavar="DIR_ID", + help="Suppress a direction insight by id (e.g. dir-abc12345) so its lineage stops being injected into introspection. The anti-pollution lever for a bad prior.") parser.add_argument("--backfill-canonical-forms", action="store_true", help="Populate canonical_form on existing register entries that lack one. One-shot maintenance pass — safe to interrupt and re-run.") parser.add_argument("--backfill-force", action="store_true", @@ -174,6 +180,9 @@ def main(): or args.reverify_insight is not None or args.synth_orphaned_xrefs or args.scan_gaps + or args.abstract_directions + or args.show_directions + or args.suppress_direction is not None ) if args.domain is None: if read_only: @@ -389,6 +398,33 @@ def _override(cli_val, toml_val): engine.test_pareto_admission(args.pareto_admission_test) elif args.scan_gaps: engine.scan_gaps() + elif args.abstract_directions: + engine.abstract_directions() + elif args.show_directions: + directions = engine.journal.latest_direction_insights() + if not directions: + print("No direction insights yet. Run --abstract-directions or let the cycle loop produce them.") + else: + ranked = sorted( + directions, + key=lambda r: (float(r.get("confidence") or 0.0), r.get("generated_at", "")), + reverse=True, + ) + print(f"{len(ranked)} direction insight(s) (latest, non-suppressed):\n") + for r in ranked: + print(f" {r.get('id')} [conf={float(r.get('confidence') or 0.0):.2f}] {r.get('label','')}") + settled = r.get("settled") or [] + if settled: + print(f" settled: {settled[0]}") + print(f" OPEN EDGE: {r.get('open_edge','')}") + print(f" members: {len(r.get('member_entry_ids') or [])} entries\n") + elif args.suppress_direction is not None: + ok = engine.journal.suppress_direction_insight(args.suppress_direction.strip()) + print( + f"Suppressed direction {args.suppress_direction.strip()}." + if ok else + f"No direction insight with id {args.suppress_direction.strip()!r}." + ) elif args.export_directive: engine.export_directive_for(args.export_directive.strip()) elif args.export_directives_bundle: diff --git a/docs/superpowers/specs/2026-06-18-direction-backprop-design.md b/docs/superpowers/specs/2026-06-18-direction-backprop-design.md new file mode 100644 index 0000000..e0ffab6 --- /dev/null +++ b/docs/superpowers/specs/2026-06-18-direction-backprop-design.md @@ -0,0 +1,238 @@ +# Direction Insight Backpropagation + Verifier-as-Held-Out-Gate Note + +**Date:** 2026-06-18 +**Status:** Design approved, pending implementation +**Influence:** Arbor / Hypothesis Tree Refinement (arXiv:2606.11926, Jin et al.) + +## Motivation + +Arbor's ablation shows that *carrying distilled lessons forward across the +research loop* is the load-bearing component of long-horizon autonomous +research — removing insight propagation hurt more (81.82% → 54.54% Any-Medal) +than removing the tree structure itself (→ 63.64%). "The tree is useful only +when evidence can accumulate over it." + +CE today regenerates uncertainties from scratch every cycle. `_build_journal_context()` +(`engine/introspect.py:89`) shows only the raw last-5 takeaways, the question +queue, and the domain-tag list. There is no *compressed, direction-level memory* +of what a line of inquiry has settled or where its open edge is. The engine can +re-ask resolved questions and fail to push past its own frontier. + +This spec adds that missing layer — **adapted to CE's objective**, which is +structural novelty, not a scalar metric. The adaptation is the whole point: +Arbor's propagation is a *convergent* force toward a measurable optimum; CE's +value is *divergence* toward out-of-distribution synthesis. So the carried-forward +signal is framed as a **frontier edge to exceed**, never a belief to confirm. + +## Scope + +**In scope** +- **Item 1** — Direction Insight Backpropagation: abstract clusters of related + investigations into direction-level priors and inject them into introspection. +- **Item 3** — Documentation note: CE's adversarial prior-art verifier is the + role-analog of Arbor's held-out merge gate; CE deliberately rejects a scalar + dev/held-out gate. + +**Explicitly out of scope** +- **Item 2** — frontier pruning / question downweighting. For a novelty objective + there is no clean falsification signal; pruning on a weak proxy (repeated low + surprise / verifier rejection) antagonizes CE's premise that verifier-failure ≠ + capability-failure. The CE-safe positive half (push toward under-explored cells) + already exists as the negative-space scan (`engine/negative_space.py`). Not built. + +## Item 1 — Direction Insight Backpropagation + +### Data flow + +``` +every N cycles, before introspect: + build_graph(journal) # engine/graph.py (existing) + → connected components of the entry subgraph + → keep components with ≥ direction_min_cluster_size entries + → cap to top direction_max_count by size + for each candidate direction: + _call_verifier(ABSTRACT_DIRECTION_PROMPT) # cross-family model, NOT primary + → {label, settled, open_edge, confidence} + → journal.add_direction_insight(...) # append-only, supersedes prior + +introspect (every cycle): + _build_journal_context() + → inject top-K latest non-suppressed open_edges as "FRONTIER EDGES" +``` + +### 1. Clustering — reuse `engine/graph.py` + +Connected components of the entry subgraph (already computed in +`graph_summary`, `engine/graph.py:384-391`). Components are formed from the +engineered edges (shares-tag, cites-source, semantic-similarity, cross-referenced) +— CE's curated relatedness structure, the analog of an Arbor subtree. + +- A component with ≥ `direction_min_cluster_size` entries (default **4**) is a + candidate direction. Singletons/tiny components are skipped — nothing to abstract. +- Cap to the top `direction_max_count` components by size (default **5**) to bound + token cost and storage volume. +- A direction's identity for supersession is its `member_signature`: a stable hash + of its sorted member entry ids. + +### 2. Abstraction call — verifier (cross-family) model + +One `_call_verifier` (`engine/core.py:229`) per candidate direction. Routing to +the verifier model — not `_call_primary` — is deliberate: abstracting over the +engine's own outputs is an *evaluative* act, and CE's design principle is +"different model for evaluation than generation." This breaks the self-flattery +loop where the primary model would author a prior and then consume it. + +**Input** (members only): for each member entry — `question`, `key_takeaways`, +`surprise_delta`; plus descriptions of any cross-references whose `source_entries` +fall within the cluster. + +**Prompt** (`ABSTRACT_DIRECTION_PROMPT`, new in `prompts.py`): domain-agnostic +(shape, not content, per CE convention). It explicitly invokes CE's novelty +signature — "the premises exist in the literature, the synthesis does not" — and +instructs the model to find the cluster's **negative-space edge**, not to +summarize content. Returns JSON: + +| field | meaning | +|---|---| +| `label` | short direction name | +| `settled` | ≤2 bullets, deliberately terse — what this cluster has established | +| `open_edge` | **load-bearing**: the unexplored synthesis / contradiction / missing `(method, problem)` combination at the cluster's frontier, framed as an investigable edge | +| `confidence` | model self-rating [0,1] | + +### 3. Storage — append-only, auditable + +New journal field `direction_insights: list[dict]`. Each record: + +```json +{ + "id": "dir-<8hex>", + "label": "...", + "member_entry_ids": ["j-...", "..."], + "member_signature": "", + "settled": ["...", "..."], + "open_edge": "...", + "confidence": 0.0, + "generated_at": "", + "model": "", + "journal_size_at_gen": 0, + "suppressed": false, + "supersedes": "dir-... | null" +} +``` + +Re-abstracting a cluster whose membership changed appends a **new** record and +sets `supersedes` to the prior record's id for that `member_signature` lineage. +Original records are never mutated (audit-trail consistent with +`reverification_log`, `known_prior_art`, `rejection_log`). + +**Journal methods** (in `journal.py`, persisted in `save()`/`_load()`): +- `add_direction_insight(record: dict)` — append + save. +- `latest_direction_insights() -> list[dict]` — for each `member_signature` + lineage, the newest record by `generated_at`, **but only if that newest record + is not suppressed**. If the lineage's newest record is suppressed, the lineage + is excluded entirely — a suppressed latest never falls back to a stale older + record. Used by injection. +- `suppress_direction_insight(insight_id: str) -> bool` — set `suppressed=true` + on the record so its lineage drops out of `latest_direction_insights` (and thus + out of injection). + +### 4. Injection — frontier edges, not beliefs + +`_build_journal_context()` (`engine/introspect.py:89`) gains a block appended +after the existing context, gated on `direction_backprop_enabled` and on there +being ≥1 latest non-suppressed insight. Top-K = `direction_max_injected` +(default **4**), ordered by `confidence` desc then recency: + +``` +FRONTIER EDGES (distilled from prior investigation clusters — do NOT +re-confirm what is settled; your job is to push PAST these open edges): + - [