diff --git a/.claude/CLAUDE.md b/.claude/CLAUDE.md
index 758edc1..a8394b4 100644
--- a/.claude/CLAUDE.md
+++ b/.claude/CLAUDE.md
@@ -26,6 +26,20 @@ inference-time descent** into distribution-free reliability certificates. Read
 
 ## Phase map (see docs/EXPERIMENTS.md)
 
-- Phase 0 skeleton ✅ / Phase 1 base reasoner ✅ (this pass)
-- Phase 2 geometry features · Phase 3 conformal + halting · Phase 4 experiments/OOD/Modal
-  · Phase 5 paper — stubbed, `NotImplementedError` with a phase note until built.
+- Phase 0 skeleton ✅ / Phase 1 base reasoner ✅ / Phase 2 geometry features ✅
+- Phase 3 conformal falsification + LTT abstention ✅ (E1: geometry beats energy, ΔAURC CI
+  excludes 0; F2/F3 regenerate from the ledger).
+- Phase 4a robustness + K-ablation ✅ (`run_sweep` + `aggregate` + `make_tables` T1/T2/T3 + S1
+  K-lift figure; S1: geometry lift grows monotonically with restarts, K=1≈EBT barely separates).
+- Phase 4b adaptive halting ✅ (opt-in per-step decode + `halting.adaptive` CRC + F4; H1: ~58%
+  compute saved at halting risk ≤ α, no accuracy loss). Both guarantees now live.
+- Phase 4c arithmetic figure set complete ✅ (F5 mechanism + F6 OOD stress; F6: selective risk
+  0.075 ID vs 0.762 OOD — guarantee breaks under shift, motivating abstention).
+- Phase 4d generalization ✅ (graph shortest-path task; E2: geometry beats energy 5/5 seeds,
+  ΔAURC +0.111 — discovery replicates on a 2nd structurally distinct family; per-task T1).
+- Phase 4e baseline stress test ✅ ⚠️ (MSP/temp/entropy; geometry beats scalar energy 5/5 but
+  does NOT beat softmax confidence — ties arith, loses graph). Key caveat; see EXPERIMENTS.md.
+- Phase 4f feature-group ablation ✅ (A1/T2b; basin agreement is the dominant driver, drop_energy≈full
+  so the win is non-energy features — rebuts "it's just energy").
+- Next: IRED landscape training (the priority lever from 4e) · E3/E4 · Modal · Phase 5 paper —
+  stubbed, `NotImplementedError` / Phase-1 placeholder until built.
diff --git a/README.md b/README.md
index e232db5..f389aa1 100644
--- a/README.md
+++ b/README.md
@@ -58,7 +58,7 @@ PYTHONPATH=src python -m edc.cli smoke --config configs/smoke.toml
 src/edc/
   energy/       encoder → E_θ(h_x, z) → decoder            (JAX/Flax)
   inference/    K-restart Langevin descent + trajectory logging
-  geometry/     basin · curvature (HVP) · energy stats · dynamics   [Phase 2]
+  geometry/     basin · curvature (HVP) · energy stats · dynamics   ✓
   conformal/    Learn-then-Test (abstention) · Conformal Risk Control (halting)  [Phase 3]
   halting/      adaptive compute policy                    [Phase 3]
   tasks/        synthetic reasoning families (+ OOD splits)
@@ -67,12 +67,32 @@ src/edc/
 configs/  experiments/  analysis/  results/ledger.jsonl  tests/  docs/  paper/
 ```
 
-## Status
-
-Phase 0 (skeleton) and Phase 1 (runnable base reasoner) are in. Geometry features,
-the conformal layer, full experiments, and the paper are scaffolded and tracked in
-[`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md). Current state:
-[`docs/SESSION_HANDOFF.md`](docs/SESSION_HANDOFF.md).
+## Status (v0.0.1)
+
+The full method and its distribution-free certificate machinery are implemented and tested
+(offline, CPU): base reasoner (Phase 1), 14-feature landscape geometry (Phase 2), the conformal
+layer — split-conformal, Learn-then-Test **selective prediction/abstention**, and Conformal Risk
+Control **adaptive halting** (Phase 3, 4b) — a multi-seed experiment/sweep/table harness, and the
+figure set F2–F6 + S1, all regenerable from `results/ledger.jsonl`.
+
+**Findings so far (honest).**
+- **Geometry beats the EBT scalar energy** as a nonconformity score on two structurally distinct
+  tasks — arithmetic (E1) and graph shortest-path (E2) — with the paired-bootstrap ΔAURC CI
+  excluding 0 on 5/5 seeds each. This is the project's specific falsification target (invariant 8).
+- The restart geometry is the mechanism: the lift **grows monotonically with K** (S1 ablation);
+  K=1 (≈ EBT, no basin geometry) barely separates.
+- Both guarantees hold in-distribution (F2/F3 abstention validity, F4 halting saves ~58% compute
+  at a guaranteed error budget) and **break under distribution shift** (F6) — the argument for
+  abstention in critical systems.
+- **Key caveat (Phase 4e):** geometry does **not** beat a standard *softmax-confidence* baseline
+  (MSP / temperature-scaled / entropy) — it ties on arithmetic and loses on graph. So the current
+  contribution is the certificate machinery + beating the EBT scalar-energy signal specifically,
+  **not** a universal win over all confidence signals. The likely lever — a fully-learned
+  **IRED-style landscape** (vs the current supervised basin-center reasoner) — is the priority next
+  experiment.
+
+Experiments and figures/tables: [`docs/EXPERIMENTS.md`](docs/EXPERIMENTS.md). Rolling state and next
+steps: [`docs/SESSION_HANDOFF.md`](docs/SESSION_HANDOFF.md). Decisions: [`docs/DECISIONS.md`](docs/DECISIONS.md).
 
 ## License
 
diff --git a/analysis/aggregate.py b/analysis/aggregate.py
new file mode 100644
index 0000000..c671269
--- /dev/null
+++ b/analysis/aggregate.py
@@ -0,0 +1,118 @@
+"""Aggregate ``split='selective'`` ledger rows across seeds for the tables/figures. [Phase 4]
+
+Pure NumPy over already-computed per-seed metrics (invariant 6: everything derives from the
+ledger; no re-training, no re-bootstrapping). The multi-seed ΔAURC is summarised from each row's
+own point estimate + 95% CI — we report the mean±std over seeds and how many seeds' CIs exclude 0,
+rather than pooling raw scores across seeds (which would break the per-run exchangeability).
+"""
+
+from __future__ import annotations
+
+import numpy as np
+
+from edc.ledger import read_all
+
+
+def selective_rows(rows: list[dict] | None = None, task: str | None = None) -> list[dict]:
+    """Latest ``split=='selective'`` row per ``(config_hash, seed)`` (cf. latest_per_config).
+
+    ``task`` filters to one reasoning family (``row["task"]``) so multi-task ledgers do not
+    cross-contaminate per-K aggregates.
+    """
+    if rows is None:
+        rows = read_all()
+    latest: dict[tuple[str, int], dict] = {}
+    for r in rows:
+        if r.get("split") == "selective" and (task is None or r.get("task") == task):
+            latest[(r["config_hash"], r["seed"])] = r  # later rows win
+    return list(latest.values())
+
+
+def tasks_present(rows: list[dict] | None = None) -> list[str]:
+    """Sorted distinct task names among selective rows."""
+    return sorted({r.get("task") for r in selective_rows(rows)} - {None})
+
+
+def _k(row: dict) -> int:
+    m = row["metrics"]
+    return int(m.get("k_restarts", row["config"]["inference"]["k_restarts"]))
+
+
+def by_k(rows: list[dict] | None = None, task: str | None = None) -> dict[int, list[dict]]:
+    """Group selective rows by ``k_restarts`` (ascending), optionally filtered to one ``task``."""
+    out: dict[int, list[dict]] = {}
+    for r in selective_rows(rows, task=task):
+        out.setdefault(_k(r), []).append(r)
+    return dict(sorted(out.items()))
+
+
+def _ms(vals) -> tuple[float, float]:
+    a = np.asarray(vals, dtype=float)
+    return float(a.mean()), float(a.std(ddof=0))
+
+
+def aggregate_cell(rows_for_k: list[dict]) -> dict:
+    """Aggregate one K's per-seed rows into mean/std stats + the multi-seed verdict."""
+    ms = [r["metrics"] for r in rows_for_k]
+    dvb = [m["delta_aurc_vs_best_energy"] for m in ms]  # each [delta, lo, hi]
+    n = len(ms)
+    delta_mean, delta_std = _ms([d[0] for d in dvb])
+    ci_excludes_0 = sum(1 for d in dvb if d[1] > 0.0)  # lo > 0 => positive & CI clears 0
+    geo_mean, geo_std = _ms([m["aurc"]["geometry"] for m in ms])
+    best_mean, best_std = _ms([m["aurc"][m["best_energy_baseline"]] for m in ms])
+    # vs the strongest of ALL baselines (energy + softmax); older rows fall back to energy.
+    dvbl = [m.get("delta_aurc_vs_best_baseline", m["delta_aurc_vs_best_energy"]) for m in ms]
+    dbl_mean, dbl_std = _ms([d[0] for d in dvbl])
+    bl_ci = sum(1 for d in dvbl if d[1] > 0.0)
+    return {
+        "task": rows_for_k[0].get("task"),
+        "k_restarts": _k(rows_for_k[0]),
+        "n_seeds": n,
+        "seeds": sorted(r["seed"] for r in rows_for_k),
+        "run_ids": [r["run_id"] for r in rows_for_k],
+        "accuracy_id": _ms([m["accuracy_id"] for m in ms]),
+        "aurc_geometry": (geo_mean, geo_std),
+        "aurc_best_energy": (best_mean, best_std),
+        "best_energy_names": sorted({m["best_energy_baseline"] for m in ms}),
+        "best_baseline_names": sorted(
+            {m.get("best_baseline", m["best_energy_baseline"]) for m in ms}),
+        "delta_aurc": (delta_mean, delta_std),
+        "seeds_ci_excludes_0": ci_excludes_0,
+        "geometry_wins_all": bool(ci_excludes_0 == n and delta_mean > 0),
+        "delta_aurc_baseline": (dbl_mean, dbl_std),
+        "seeds_ci_excludes_0_baseline": bl_ci,
+        # True only when every row genuinely carried the Phase-4e baseline field (not a fallback).
+        "has_baseline": all("delta_aurc_vs_best_baseline" in m for m in ms),
+        # Feature-group leave-one-out ablation (Phase 4f), mean/std per subset over seeds.
+        "feature_ablation": (
+            {k: _ms([m["feature_ablation"][k] for m in ms])
+             for k in ms[0]["feature_ablation"]}
+            if all("feature_ablation" in m for m in ms) else {}),
+        "ltt_coverage": _ms([m["ltt"]["coverage"] for m in ms]),
+        "ltt_selective_risk": _ms([m["ltt"]["selective_risk"] for m in ms]),
+        "ltt_abstain_rate": _ms([m["ltt"]["abstain_rate"] for m in ms]),
+    }
+
+
+def aggregate_by_k(rows: list[dict] | None = None, task: str | None = None) -> dict[int, dict]:
+    """``{K: aggregate_cell(...)}`` for every K present (optionally filtered to one ``task``)."""
+    return {k: aggregate_cell(v) for k, v in by_k(rows, task=task).items()}
+
+
+def headline_cell(rows: list[dict] | None = None, task: str | None = None) -> dict:
+    """Aggregate the headline seed-sweep for a task: the largest single-config seed group.
+
+    Prefers rows carrying the Phase-4e baseline field, then picks the ``config_hash`` group with the
+    most seeds — the multi-seed sweep — so T1 is not polluted by mixing fold sizes / configs that
+    happen to share a K value. Falls back to all rows for the task if none carry the field.
+    """
+    sel = selective_rows(rows, task=task)
+    with_bl = [r for r in sel if "delta_aurc_vs_best_baseline" in r["metrics"]]
+    pool = with_bl or sel
+    # Group by the experiment config *excluding seed* (config_hash bakes in the seed). (K, n_test)
+    # separates a seed-sweep (many seeds, one fold size) from one-off full-fold runs at the same K.
+    groups: dict[tuple, list[dict]] = {}
+    for r in pool:
+        m = r["metrics"]
+        groups.setdefault((_k(r), m.get("n_test")), []).append(r)
+    return aggregate_cell(max(groups.values(), key=len))
diff --git a/analysis/figures/F2_risk_coverage.png b/analysis/figures/F2_risk_coverage.png
new file mode 100644
index 0000000..f98c466
Binary files /dev/null and b/analysis/figures/F2_risk_coverage.png differ
diff --git a/analysis/figures/F3_coverage_validity.png b/analysis/figures/F3_coverage_validity.png
new file mode 100644
index 0000000..5884f20
Binary files /dev/null and b/analysis/figures/F3_coverage_validity.png differ
diff --git a/analysis/figures/F4_halting_pareto.png b/analysis/figures/F4_halting_pareto.png
new file mode 100644
index 0000000..68613ea
Binary files /dev/null and b/analysis/figures/F4_halting_pareto.png differ
diff --git a/analysis/figures/F5_feature_diagnostics.png b/analysis/figures/F5_feature_diagnostics.png
new file mode 100644
index 0000000..6e66445
Binary files /dev/null and b/analysis/figures/F5_feature_diagnostics.png differ
diff --git a/analysis/figures/F6_ood_stress.png b/analysis/figures/F6_ood_stress.png
new file mode 100644
index 0000000..1e36b70
Binary files /dev/null and b/analysis/figures/F6_ood_stress.png differ
diff --git a/analysis/figures/S1_k_restart_lift.png b/analysis/figures/S1_k_restart_lift.png
new file mode 100644
index 0000000..f89d6a6
Binary files /dev/null and b/analysis/figures/S1_k_restart_lift.png differ
diff --git a/analysis/make_figures.py b/analysis/make_figures.py
index e7dc6e5..0bdde16 100644
--- a/analysis/make_figures.py
+++ b/analysis/make_figures.py
@@ -7,16 +7,41 @@
 
 from __future__ import annotations
 
+from pathlib import Path
+
+from edc import plotting
 from edc.ledger import read_all
 
+FIG_DIR = Path(__file__).resolve().parent / "figures"
+
+# (filename, plotting fn). F1 (schematic) is TikZ; F4/F6 (halting, OOD stress) are Phase 4/5.
+_FIGURES = [
+    ("F2_risk_coverage.png", plotting.risk_coverage_curve),
+    ("F3_coverage_validity.png", plotting.coverage_validity),
+    ("F4_halting_pareto.png", plotting.halting_pareto),
+    ("F5_feature_diagnostics.png", plotting.feature_diagnostics),
+    ("F6_ood_stress.png", plotting.ood_stress),
+    ("S1_k_restart_lift.png", plotting.k_restart_lift),
+]
+
 
 def main() -> int:
     rows = read_all()
     print(f"[figures] {len(rows)} ledger rows available.")
-    raise NotImplementedError(
-        "Phase 3/5: emit F1..F6 via edc.plotting from the ledger. "
-        "Run `make smoke` first to populate results/ledger.jsonl."
-    )
+    FIG_DIR.mkdir(parents=True, exist_ok=True)
+    n_ok = 0
+    for fname, fn in _FIGURES:
+        out = FIG_DIR / fname
+        try:
+            fn(rows, str(out))
+        except (ValueError, NotImplementedError, ImportError) as e:
+            print(f"[figures] skip {fname}: {e}")
+            continue
+        n_sel = sum(1 for r in rows if r.get("split") == "selective")
+        print(f"[figures] wrote {out}  (from {n_sel} selective rows)")
+        n_ok += 1
+    print(f"[figures] {n_ok}/{len(_FIGURES)} figures generated -> {FIG_DIR}")
+    return 0
 
 
 if __name__ == "__main__":
diff --git a/analysis/make_tables.py b/analysis/make_tables.py
index 03b727b..9e9686c 100644
--- a/analysis/make_tables.py
+++ b/analysis/make_tables.py
@@ -1,18 +1,155 @@
-"""Regenerate LaTeX tables (T1..T3) from results/ledger.jsonl. [Phase 3/5]
+"""Regenerate LaTeX tables (T1..T3) from results/ledger.jsonl. [Phase 4]
 
-Main-results, ablation, and reproducibility-appendix tables, emitted to paper/. Reads only the
-ledger. See paper/README.md for the table->claim map.
+T1 main results (primary K, aggregated over seeds), T2 K-ablation, T3 reproducibility appendix.
+Reads only the ledger (invariant 6) and writes ``\\input``-able ``.tex`` to ``paper/tables/``, each
+with a provenance comment naming the run_ids it consumed. See paper/README.md for the table->claim
+map.
 """
 
 from __future__ import annotations
 
+from edc.config import REPO_ROOT
 from edc.ledger import read_all
 
+try:  # sibling import when run as a script (analysis/ on sys.path)
+    from aggregate import aggregate_by_k, headline_cell, selective_rows, tasks_present
+except ImportError:  # pragma: no cover
+    from analysis.aggregate import aggregate_by_k, headline_cell, selective_rows, tasks_present
+
+TABLE_DIR = REPO_ROOT / "paper" / "tables"
+
+
+def _esc(s: str) -> str:
+    return str(s).replace("_", r"\_")
+
+
+def _pm(mean_std: tuple[float, float], nd: int = 3) -> str:
+    m, s = mean_std
+    return f"${m:.{nd}f} \\pm {s:.{nd}f}$"
+
+
+def _tabular(header: list[str], rows: list[list[str]], colspec: str, caption: str,
+             label: str, provenance: str) -> str:
+    lines = [
+        f"% auto-generated by analysis/make_tables.py — do not edit. {provenance}",
+        r"\begin{table}[t]", r"  \centering", f"  \\caption{{{caption}}}", f"  \\label{{{label}}}",
+        f"  \\begin{{tabular}}{{{colspec}}}", r"    \toprule",
+        "    " + " & ".join(header) + r" \\", r"    \midrule",
+    ]
+    lines += ["    " + " & ".join(r) + r" \\" for r in rows]
+    lines += [r"    \bottomrule", r"  \end{tabular}", r"\end{table}", ""]
+    return "\n".join(lines)
+
+
+def _t1(rows: list[dict]) -> str:
+    """Main results, one row per task (each at its own primary K, aggregated over seeds).
+
+    Reports geometry's ΔAURC against **both** the best raw-energy baseline (invariant-8 sacred test)
+    and the best of *all* baselines (energy + MSP + temperature + entropy), with the seeds-win count
+    for the stronger, all-baseline comparison.
+    """
+    body, all_ids = [], []
+    for task in tasks_present(rows):
+        a = headline_cell(rows, task=task)             # largest same-config baseline-carrying sweep
+        win = f"{a['seeds_ci_excludes_0_baseline']}/{a['n_seeds']}"
+        body.append([
+            _esc(task), str(a["k_restarts"]), _pm(a["accuracy_id"], 2), _pm(a["aurc_geometry"]),
+            _pm(a["delta_aurc"]), _pm(a["delta_aurc_baseline"]), win, _pm(a["ltt_coverage"], 2),
+        ])
+        all_ids += a["run_ids"]
+    header = ["task", "$K$", "acc", "AURC geom", r"$\Delta$AURC vs energy",
+              r"$\Delta$AURC vs best base", "seeds win", "LTT cov"]
+    return _tabular(
+        header, body, "lrcccccc",
+        "Main selective-prediction results across reasoning families (mean$\\pm$std over seeds): "
+        "geometry's $\\Delta$AURC over the best raw-energy baseline and over the best of all "
+        "baselines (energy, MSP, temperature-scaled MSP, predictive entropy). $\\Delta$AURC $>0$ "
+        "with the CI clearing 0 means geometry wins; ``seeds win`` counts seeds beating the best "
+        "overall baseline.",
+        "tab:main", f"run_ids={all_ids}")
+
+
+def _t2(agg: dict[int, dict]) -> str:
+    rows = []
+    all_ids: list[str] = []
+    for k, a in agg.items():
+        rows.append([
+            str(k), _pm(a["aurc_geometry"]), _pm(a["aurc_best_energy"]), _pm(a["delta_aurc"]),
+            f"{a['seeds_ci_excludes_0']}/{a['n_seeds']}", _pm(a["ltt_coverage"], 2),
+        ])
+        all_ids += a["run_ids"]
+    header = ["$K$", "AURC geom", "AURC energy", r"$\Delta$AURC", "seeds win", "LTT cov"]
+    return _tabular(
+        header, rows, "rccccc",
+        "Restart ablation: does landscape geometry across $K$ restarts add signal over a single "
+        "descent ($K{=}1\\approx$ EBT)? Lift is $\\Delta$AURC $>0$ with the CI clearing 0.",
+        "tab:kablation", f"run_ids={all_ids}")
+
+
+def _t2b(rows: list[dict]) -> str:
+    """Feature-group leave-one-out ablation (A1): AURC per subset, one column per task.
+
+    ``full`` is the whole geometry vector; ``drop_energy`` is geometry *without* energy columns —
+    if it stays near ``full`` (and below raw energy), the basin/curvature/dynamics features are
+    genuinely carrying the win, not merely re-using energy.
+    """
+    tasks = tasks_present(rows)
+    cells = {t: headline_cell(rows, task=t).get("feature_ablation", {}) for t in tasks}
+    variants = ["full", "drop_basin", "drop_energy", "drop_curv", "drop_dynamics"]
+    body = []
+    for v in variants:
+        row = [_esc(v)]
+        for t in tasks:
+            fa = cells[t].get(v)
+            row.append(_pm(fa) if fa else "--")
+        body.append(row)
+    header = ["subset (AURC, lower=better)", *[_esc(t) for t in tasks]]
+    colspec = "l" + "c" * len(tasks)
+    return _tabular(
+        header, body, colspec,
+        "Feature-group leave-one-out ablation: test AURC of the geometry mapper refit on subsets "
+        "of the 14 features (mean$\\pm$std over seeds). ``drop\\_X`` removes that group; comparing "
+        "``drop\\_energy`` to ``full`` isolates the non-energy geometry contribution.",
+        "tab:ablation", "feature-group leave-one-out")
+
+
+def _t3(rows: list[dict]) -> str:
+    sel = selective_rows(rows)
+    env = sel[-1]["env"] if sel else {}
+    body = [
+        [_esc(r["run_id"]), str(r["seed"]),
+         str(r["metrics"].get("k_restarts", "?")), _esc(r["config_hash"]), _esc(r["git_sha"])]
+        for r in sorted(sel, key=lambda r: (r["metrics"].get("k_restarts", 0), r["seed"]))
+    ]
+    caption = (f"Reproducibility appendix. Env: python {env.get('python', '?')}, "
+               f"jax {env.get('jax', '?')} ({env.get('jax_backend', '?')}). "
+               f"Every row regenerates from seed + config via \\texttt{{run\\_sweep}}.")
+    return _tabular(["run\\_id", "seed", "$K$", "config hash", "git sha"], body, "lrrll",
+                    caption, "tab:repro", f"n_rows={len(body)}")
+
 
 def main() -> int:
     rows = read_all()
-    print(f"[tables] {len(rows)} ledger rows available.")
-    raise NotImplementedError("Phase 3/5: emit T1..T3 LaTeX from the ledger.")
+    tasks = tasks_present(rows)
+    print(f"[tables] {len(rows)} ledger rows; tasks = {tasks}")
+    if not tasks:
+        print("[tables] no split='selective' rows — run experiments/run_sweep.py first.")
+        return 0
+    TABLE_DIR.mkdir(parents=True, exist_ok=True)
+
+    # T2 (K-ablation) uses whichever task has a K-sweep (>1 distinct K); default the first.
+    kablation_task = next((t for t in tasks if len(aggregate_by_k(rows, task=t)) > 1), tasks[0])
+    outputs = {
+        "T1_main.tex": _t1(rows),
+        "T2_kablation.tex": _t2(aggregate_by_k(rows, task=kablation_task)),
+        "T2b_feature_ablation.tex": _t2b(rows),
+        "T3_repro.tex": _t3(rows),
+    }
+    for name, tex in outputs.items():
+        (TABLE_DIR / name).write_text(tex)
+        print(f"[tables] wrote {TABLE_DIR / name}")
+    print(f"[tables] {len(outputs)} tables -> {TABLE_DIR}  (K-ablation task={kablation_task})")
+    return 0
 
 
 if __name__ == "__main__":
diff --git a/configs/experiments/arithmetic_halting.toml b/configs/experiments/arithmetic_halting.toml
new file mode 100644
index 0000000..64b4106
--- /dev/null
+++ b/configs/experiments/arithmetic_halting.toml
@@ -0,0 +1,37 @@
+# Adaptive-halting experiment (Phase 4b, F4). Same task/inference as the selective E1 config, but
+# per-step decoding is recorded (restarts.solve(record_steps=True)), so folds are kept modest to
+# bound the extra decode memory on CPU. CRC calibrates the basin-agreement halting threshold at the
+# alpha error budget; F4 is the compute-vs-accuracy Pareto.
+[run]
+task = "arithmetic"
+seed = 0
+notes = "F4 adaptive-halting run"
+
+[task.arithmetic]
+modular = false
+n_operands = 6
+ood_n_operands = 6
+max_operand = 7
+ood_max_operand = 9
+
+[model]
+latent_dim = 32
+hidden_dim = 128
+context_dim = 64
+
+[inference]
+k_restarts = 12
+steps = 50
+
+[train]
+epochs = 7
+n_train = 6000
+n_neg = 4
+
+[eval]
+n_eval = 800        # test fold (per-step decode kept small)
+
+[conformal]
+alpha = 0.1         # halting error budget: E[early answer != full answer] <= alpha
+delta = 0.05
+n_calib = 800       # calibration fold
diff --git a/configs/experiments/arithmetic_selective.toml b/configs/experiments/arithmetic_selective.toml
index 3c66bdb..7f4a27c 100644
--- a/configs/experiments/arithmetic_selective.toml
+++ b/configs/experiments/arithmetic_selective.toml
@@ -1,15 +1,42 @@
-# Main selective-prediction experiment on arithmetic (Phase 3/4).
-# Compares geometry nonconformity scores vs the raw-energy / MSP / MC-dropout / ensemble
-# baselines under identical conformal calibration. Produces F2 (risk-coverage) + T1.
+# E1 — main selective-prediction experiment on arithmetic (Phase 3).
+# Compares the geometry nonconformity score vs the raw-energy / rho-basin baselines under
+# identical conformal calibration, on three disjoint folds (fit / calib / test). Produces the
+# ΔAURC falsification test (invariant 8), the LTT abstention guarantee, and F2/F3.
+#
+# Operating regime (docs/EXPERIMENTS.md): the task is tuned to ~75-82% ID base accuracy so AURC
+# has error to remove (a saturated task ties every method). Base error there exceeds alpha=0.1,
+# so the guarantee must genuinely abstain. OOD is a graceful magnitude shift (operands 0-7 -> 0-9,
+# same operand count) rather than the degenerate operand-count extrapolation.
 [run]
 task = "arithmetic"
-notes = "selective-prediction main run"
+seed = 0
+notes = "E1 selective-prediction main run"
+
+[task.arithmetic]
+modular = false
+n_operands = 6
+ood_n_operands = 6        # OOD shift is magnitude, not operand count
+max_operand = 7           # ID operands in [0, 7]
+ood_max_operand = 9       # OOD operands in [0, 9] -> larger sums, low-sum region shared
+
+[model]
+latent_dim = 32
+hidden_dim = 128
+context_dim = 64
 
 [inference]
-k_restarts = 16      # geometry needs several restarts; K=1 ~ EBT baseline
+k_restarts = 12           # geometry needs several restarts; K=1 ~ the EBT baseline
 steps = 50
 
+[train]
+epochs = 7                # tuned for the ~77% ID regime (base error > alpha, abstention exercised)
+n_train = 6000
+n_neg = 4
+
+[eval]
+n_eval = 1500             # fit and test fold sizes
+
 [conformal]
-alpha = 0.1
-delta = 0.05
-n_calib = 2000
+alpha = 0.1               # target error rate among answered inputs
+delta = 0.05              # LTT confidence: P(R_sel <= alpha) >= 1 - delta
+n_calib = 1500            # calibration fold size (LTT power)
diff --git a/configs/experiments/graph_selective.toml b/configs/experiments/graph_selective.toml
new file mode 100644
index 0000000..bf3c062
--- /dev/null
+++ b/configs/experiments/graph_selective.toml
@@ -0,0 +1,36 @@
+# E2 — generalization: selective prediction on graph shortest-path (a relational reasoning family
+# distinct from additive arithmetic). Tests whether geometry beats scalar energy beyond arithmetic.
+# Tuned to ~72% ID base accuracy (majority-class baseline ~0.41, so this is real learning); OOD is a
+# size shift (larger graphs) degrading to ~0.36. Inference/train mirror the arithmetic E1 config.
+[run]
+task = "graph_planning"
+seed = 0
+notes = "E2 graph selective-prediction"
+
+[task.graph_planning]
+n_nodes = 7
+ood_n_nodes = 10        # OOD = larger graphs (IRED-style size generalization)
+edge_prob = 0.4
+max_len = 4             # classes: 0 = unreachable/farther, 1..4 = exact shortest-path length
+
+[model]
+latent_dim = 32
+hidden_dim = 128
+context_dim = 64
+
+[inference]
+k_restarts = 12
+steps = 50
+
+[train]
+epochs = 25             # graph shortest-path needs more epochs than arithmetic to reach the regime
+n_train = 8000
+n_neg = 4
+
+[eval]
+n_eval = 1500
+
+[conformal]
+alpha = 0.1
+delta = 0.05
+n_calib = 1500
diff --git a/configs/sweeps/arith_seeds.toml b/configs/sweeps/arith_seeds.toml
new file mode 100644
index 0000000..c2da52d
--- /dev/null
+++ b/configs/sweeps/arith_seeds.toml
@@ -0,0 +1,13 @@
+# Arithmetic multi-seed at K=12 with the Phase-4e baselines, matching graph_seeds.toml so T1's
+# cross-task vs-best-baseline verdict is 5-seed on both tasks (the arithmetic K-sweep rows predate
+# the softmax-confidence baselines). Reduced folds keep it fast; the full-fold E1 stays the F2-F6
+# headline (plotting._selective_row picks max n_test).
+[sweep]
+base = "configs/experiments/arithmetic_selective.toml"
+
+[sweep.override]
+"eval.n_eval" = 600
+"conformal.n_calib" = 800
+
+[sweep.grid]
+"run.seed" = [0, 1, 2, 3, 4]
diff --git a/configs/sweeps/graph_seeds.toml b/configs/sweeps/graph_seeds.toml
new file mode 100644
index 0000000..90fe076
--- /dev/null
+++ b/configs/sweeps/graph_seeds.toml
@@ -0,0 +1,12 @@
+# E2 multi-seed: run the graph selective-prediction experiment across 5 seeds at K=12 for a
+# credible ΔAURC generalization verdict (does geometry beat energy on graphs?). Reduced folds keep
+# each cell fast on CPU and keep arithmetic's full-fold E1 (n_test=1500) as the F2-F6 headline.
+[sweep]
+base = "configs/experiments/graph_selective.toml"
+
+[sweep.override]
+"eval.n_eval" = 600
+"conformal.n_calib" = 800
+
+[sweep.grid]
+"run.seed" = [0, 1, 2, 3, 4]
diff --git a/configs/sweeps/k_restarts.toml b/configs/sweeps/k_restarts.toml
index 7710dcf..4cb7f64 100644
--- a/configs/sweeps/k_restarts.toml
+++ b/configs/sweeps/k_restarts.toml
@@ -1,8 +1,16 @@
-# Sweep the number of restarts: K=1 (~EBT, no basin geometry) .. K=32.
-# Tests the ablation "does restart geometry add signal over a single descent?" (invariant 8).
+# S1 — sweep the number of restarts: K=1 (~EBT, no basin geometry) .. K=16, across 5 seeds.
+# Tests the secondary falsifier "does restart geometry add signal over a single descent?"
+# (invariant 8): K=1 has degenerate basin features (one restart), so the K>1 lift over K=1 is the
+# ablation. Produces T2 (K-ablation) + the K-lift figure. K=32 dropped to bound curvature cost on
+# CPU (add it back for the paper run). Folds are reduced (n_eval=600, n_calib=800) to keep each of
+# the 25 cells ~1 min; AURC/ΔAURC (the ablation metric) is threshold-free and fine at n_test=600.
 [sweep]
 base = "configs/experiments/arithmetic_selective.toml"
 
+[sweep.override]
+"eval.n_eval" = 600
+"conformal.n_calib" = 800
+
 [sweep.grid]
-"inference.k_restarts" = [1, 2, 4, 8, 16, 32]
+"inference.k_restarts" = [1, 2, 4, 8, 16]
 "run.seed" = [0, 1, 2, 3, 4]
diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md
index 5d876ca..46c96da 100644
--- a/docs/DECISIONS.md
+++ b/docs/DECISIONS.md
@@ -2,6 +2,149 @@
 
 Short, dated, append-only. Newest first.
 
+## 2026-07-27 — Phase 4f: feature-group leave-one-out ablation (which geometry features drive it)
+**Decision:** `evaluate` computes a `feature_ablation` block — refit the logistic geometry mapper on
+feature subsets (leave-one-group-out and group-only across basin/energy/curvature/dynamics) on the
+fit fold and score the test fold (`_feature_ablation`). Aggregated over seeds and emitted as
+`T2b_feature_ablation.tex`. **Why:** the geometry vector *includes* the 3 energy statistics, so
+"geometry beats energy" could be partly re-using energy; `drop_energy` (geometry without its energy
+columns) vs `full` isolates the genuine basin/curvature/dynamics contribution — directly probing the
+Phase-4e caveat. Cheap (logistic on ≤14 features), no training-recipe risk. **Reversible?** Yes —
+additive metric block; older rows aggregate to an empty ablation.
+**Note:** full IRED annealed-landscape / score-matching training (the priority lever from 4e) is a
+substantial research effort scoped as the next phase, not folded into the v0.0.1 release; the
+release ships the complete certificate machinery + honest findings under the Phase-1 reasoner.
+
+## 2026-07-27 — Phase 4e: softmax-confidence baselines + a best-of-all-baselines falsification
+**Decision:** add three standard softmax-confidence nonconformity baselines (`eval/baselines.py`) —
+**MSP**, **temperature-scaled MSP**, and **predictive entropy** — computed from the mean decoder
+logits over the K restarts. Temperature is fit on the **fit** fold only (invariant 7). `evaluate`
+now reports AURC for these alongside the energy baselines, and adds
+`delta_aurc_vs_best_baseline` (geometry vs the strongest of *all* baselines) with a bootstrap CI,
+**in addition to** the sacred `delta_aurc_vs_best_energy` (invariant 8). T1 shows both ΔAURCs per
+task. **Why:** "geometry beats energy" is vulnerable to "energy is a weak baseline"; a paper needs
+geometry to beat a standard softmax confidence too. Mean-logit read-off gives one temperature-
+scalable confidence vector per input; these are cheap (decoder logits already exist). **Deferred:**
+MC-dropout and deep ensembles (need model/dropout changes or multiple trainings; the K-restart
+best-of-N already supplies ensemble-like diversity). **Honesty:** the run records the verdict vs the
+best baseline whatever it is; a tie with MSP would be reported, not hidden. **Reversible?** Yes —
+additive scores + metric fields; aggregation falls back to the energy ΔAURC for pre-4e rows.
+
+## 2026-07-26 — Phase 4d: second task (graph shortest-path) + task-aware aggregation
+**Decision:** the generalization task is **graph shortest-path length** (`tasks/graph_planning.py`):
+random Erdős–Rényi graph, random `(source, target)`, label = BFS shortest-path length capped at
+`max_len` (class 0 = unreachable/farther). Fixed `feature_dim` via `max_nodes` padding (flattened
+adjacency ⊕ source/target one-hots) so one model spans the ID split (`n_nodes`) and the
+size-generalization OOD (`ood_n_nodes` > it). Tuned to ID ≈ 0.72 (majority-class baseline ≈ 0.41 →
+real learning) via `n_nodes=7, edge_prob=0.4, max_len=4, epochs=25`. `analysis/aggregate.py` gains a
+`task` filter (and `tasks_present`) so multi-task ledgers do not cross-contaminate per-K aggregates;
+`make_tables` T1 is now one row per task. **Why:** a relational/combinatorial family maximally
+distinct from additive arithmetic is the strongest generalization test; edge density is the cleanest
+difficulty knob (denser → shorter, more learnable paths); larger graphs are a natural covariate
+shift. The whole E1 pipeline is task-agnostic, so the task rides `evaluate`/`run_sweep` unchanged.
+**Reversible?** Yes — additive task + optional aggregation arg; arithmetic results untouched.
+
+## 2026-07-25 — Phase 4c: F6 exhibits the guarantee breaking OOD; F5 stores compact diagnostics
+**Decision:** F6 (OOD stress) is produced by calibrating the LTT selective-risk threshold on the
+**ID** calibration fold and then evaluating the achieved risk on the **OOD** test fold across the
+α-sweep (`evaluate` gains `ood_validity` + an `ood_ltt` block, guarded by `include_ood`). F5 stores
+`feature_diagnostics` = per-feature `single_feature_auroc` + 20-bin correct/incorrect histograms
+(~850 numbers), **not** raw per-input features. **Why:** the exchangeability that the LTT/CRC
+guarantees assume is exactly what a distribution shift violates, so applying the *ID-calibrated*
+threshold to OOD is the honest way to show the guarantee failing (the argument for abstention in
+critical systems); storing histograms keeps the figure regenerable from the ledger (invariant 6)
+without bloating it. **Honesty:** F6 is a negative result by construction — the row records whatever
+happened (`risk_within_budget`), it is never forced. **Reversible?** Yes — both are additive metric
+blocks; `include_ood=False` (the sweep path) omits F6.
+
+## 2026-07-25 — Phase 4b: adaptive halting — opt-in per-step decodes, basin-agreement signal, λ=1−τ
+**Decision:** Adaptive halting stops the descent when **basin agreement** (plurality fraction of
+decoded answers across restarts) at a step crosses a threshold τ; the halting loss
+`L = 1[early best-of-N ≠ full-budget best-of-N]` is CRC-calibrated via the existing
+`conformal.crc.calibrate`. Three implementation choices: (1) per-step decoding is **opt-in**
+(`langevin_descent(record_z=…)` → `restarts.solve(record_steps=…)` → optional
+`TrajectoryRecord.step_pred`), so E1/sweep pay nothing and the descent is byte-identical with it off;
+(2) the halting signal is **basin agreement** — the method's own geometry signal, not an ad-hoc
+grad-norm proxy; (3) `halting.adaptive.calibrate` reparametrises **`lambda = 1 − tau`** so ascending
+λ = lower agreement bar = stop earlier = weakly more errors + more compute saved, matching
+`crc.calibrate`'s "largest admissible λ = most compute saved", and returns `tau_hat = 1 − lambda_hat`.
+**Why:** confines JAX to the core (invariant 1), keeps the common paths cheap, and reuses the
+tested CRC calibrator unchanged. **Monotonicity caveat:** CRC assumes risk monotone in the
+threshold; `L(τ)` is monotone only in the limit (answers can flip), so `evaluate_halting` reports
+the risk-vs-τ curve + a `risk_monotone_in_tau` flag, and `crc.calibrate` scans conservatively (no
+cherry-picking dips). If materially non-monotone, LTT (already built) is the rigorous fallback —
+we do not silently claim CRC validity. **Reversible?** Yes — `record_steps` defaults off; the
+signal/threshold live behind `halting.adaptive`.
+
+## 2026-07-25 — Phase 4a: sweep harness (override+grid), reduced folds, multi-seed ΔAURC aggregation
+**Decision:** `experiments/run_sweep.py` expands a `[sweep.grid]` of dotted keys (cartesian) over a
+`base` config, with optional constant `[sweep.override]`; every cell reuses `run_experiment.run_and_append`
+(the single train→evaluate→ledger path). The S1 K-sweep runs with **reduced folds**
+(`eval.n_eval=600`, `conformal.n_calib=800`) and **`evaluate(include_ood=False)`** (3 folds not 4) to
+keep 25 cells CPU-tractable. Multi-seed ΔAURC is aggregated in `analysis/aggregate.py` from each
+row's **own point estimate + 95% CI** (mean±std over seeds; count of seeds whose CI excludes 0) —
+**not** by pooling raw scores across seeds, which would break per-run exchangeability. **Why:** the
+falsification metric (ΔAURC) is threshold-free and stable at n_test=600, so smaller folds are fine
+for the ablation; each seed already carries a valid within-run bootstrap CI, so the honest
+cross-seed summary is "how many seeds independently clear 0," not a re-bootstrap. **Reversible?**
+Yes — folds/grid are config; `include_ood` defaults to True (E1 unchanged); K=32 can be re-added.
+
+## 2026-07-24 — Phase 3: LTT uses Bonferroni over a bounded grid, not ascending fixed-sequence
+**Decision:** `conformal.ltt.calibrate` controls FWER with **Bonferroni** (`p(lambda) <= delta/m`)
+over a bounded quantile grid of `m` candidate thresholds, and picks the max-coverage admissible
+`lambda_hat`. **Why:** the selective risk `R_sel(lambda)` is non-monotone, and a fixed-sequence on
+*ascending* lambda (the initial design) stalls immediately — the smallest thresholds answer ~1
+point, so Hoeffding-Bentkus has no power and the very first hypothesis fails. Bonferroni tests all
+thresholds, so it finds the mid-range one that controls risk even when base error > alpha (the
+regime where selective prediction is useful). Validity is confirmed by
+`tests/test_conformal_ltt.py::test_ltt_coverage_guarantee` (0 violations over 250 draws).
+**Reversible?** Yes — pass explicit `lambdas` or swap the correction behind the same signature.
+
+## 2026-07-24 — AURC is tie-robust (group-mean errors)
+**Decision:** `eval.metrics` sorts by score and replaces each point's error with its **tie-group
+mean** (expected risk under random tie-breaking) before sweeping coverage. **Why:** scores with few
+distinct values (`rho_basin` has only K+1 levels) otherwise give an input-order-dependent AURC,
+biasing the geometry-vs-energy comparison. Group-mean errors make AURC order-independent and make
+"constant score => AURC = base error rate" exact. Standard split-conformal thresholds admit whole
+tie-groups anyway, so the calibration layer is unaffected.
+
+## 2026-07-24 — 70-85% regime via plain-sum operand count; graceful OOD via a magnitude shift
+**Decision:** E1 (`configs/experiments/arithmetic_selective.toml`) runs plain-sum arithmetic with
+`n_operands=6, max_operand=7` and `epochs=7`, landing ID base accuracy ~77-81% (base error > alpha,
+so abstention is genuinely exercised). OOD is a **magnitude covariate shift** (`ood_max_operand=9`,
+same operand count) — a new `ArithmeticTask.ood_max_operand` knob. **Why:** the modular "grokking"
+variant is bimodal (saturates or collapses to chance) and its operand-count OOD is always at chance;
+plain-sum accuracy is smoothly tunable via epochs, and a magnitude shift keeps the low-sum region
+in-distribution so OOD degrades *gracefully* (~20% acc, 15x chance) instead of the old degenerate
+0.8% (5-operand sums hit label classes never trained). Feature/class dims cover both splits so one
+model handles ID+OOD. **Reversible?** Yes — `ood_max_operand` defaults to `max_operand` (no shift).
+
+## 2026-07-24 — Phase 3 scope: falsification + abstention now; adaptive halting deferred
+**Decision:** this pass built the selective-prediction falsification (AURC/ΔAURC + geometry mapper
++ LTT abstention + `evaluate`/`run_experiment` + F2/F3) and the pure `crc.calibrate`. The **halting
+policy** (`halting.adaptive`) and figure **F4** are deferred. **Why:** halting needs per-step decoded
+answers on `TrajectoryRecord` (a schema change), and the discovery claim — geometry beats scalar
+energy — does not depend on it. `crc.calibrate` is implemented and tested so the conformal layer is
+complete; only the trajectory-coupled policy waits. **Reversible?** Yes — additive when built.
+
+## 2026-07-24 — Geometry features are plain NumPy; curvature batching stays in the JAX core
+**Decision:** `geometry/{basin,energy_stats,dynamics,features}.py` are pure NumPy over
+`TrajectoryRecord` arrays. All JAX for the features — the per-particle HVPs and their `vmap`
+over restarts — lives in a new `curvature.batched_curvature`, whose output `features.py`
+converts straight back to NumPy.
+**Why:** invariant 1 confines JAX to `energy/*`, `inference/*`, and `geometry/curvature.py`.
+The stub docstrings' "computable under vmap over restarts" is met by fixed-shape vectorized
+NumPy over the K axis (and real `jax.vmap` inside `curvature.py`).
+**Reversible?** Yes — the feature functions take arrays, so a JAX rewrite behind the same
+signatures is local if ever needed.
+
+## 2026-07-24 — `h_x` is stored on `TrajectoryRecord`
+**Decision:** add `h_x (B, context_dim)` to `TrajectoryRecord`, populated in
+`inference.restarts.solve`. **Why:** curvature at `z*` needs the per-input context, but
+`geometry_features(traj, fns, params, key)` receives no `x`; storing the already-computed
+`h_x` keeps that signature and avoids re-encoding. It is genuine raw material, consistent with
+the record keeping the full descent rather than just the endpoint.
+
 ## 2026-07-24 — JAX as the numeric core
 **Decision:** JAX/Flax for `energy/`, `inference/`, `geometry/curvature.py`; everything else
 plain Python behind `edc.energy.base.ReasonerFns`.
diff --git a/docs/EXPERIMENTS.md b/docs/EXPERIMENTS.md
index e351b39..4972541 100644
--- a/docs/EXPERIMENTS.md
+++ b/docs/EXPERIMENTS.md
@@ -6,21 +6,23 @@ Every experiment: config file → purpose → expected figure/table → status.
 | ID | Config | Purpose | Output | Status |
 |----|--------|---------|--------|--------|
 | E0 | `configs/smoke.toml` | Liveness: train→K-restart infer→decode on CPU <60s | ledger row | ✅ Phase 1 |
-| E1 | `configs/experiments/arithmetic_selective.toml` | Geometry vs baselines, selective prediction on arithmetic | F2, T1 | ⏳ Phase 3 |
-| E2 | (graph) | Same on graph planning | F2, T1 | ⏳ Phase 4 |
+| D1 | `configs/smoke.toml` (`edc.cli geometry`) | Per-feature correct-vs-incorrect AUROC, geometry vs raw energy — F5 precursor | ledger row | ✅ Phase 2 |
+| E1 | `configs/experiments/arithmetic_selective.toml` | Geometry vs baselines, selective prediction on arithmetic | F2, F3, T1 | ✅ Phase 3 |
+| E2 | `configs/experiments/graph_selective.toml` (+`sweeps/graph_seeds.toml`) | Generalization: geometry vs energy on graph shortest-path | T1 | ✅ Phase 4d |
 | E3 | (logic) | Same on logic | F2, T1 | ⏳ Phase 4 |
 | E4 | (hard sudoku) | Stronger-result task, OOD split | F2, F6, T1 | ⏳ Phase 4 |
-| S1 | `configs/sweeps/k_restarts.toml` | K=1..32: does restart geometry add signal? | F5, T2 | ⏳ Phase 4 |
+| S1 | `configs/sweeps/k_restarts.toml` | K=1..16: does restart geometry add signal? | S1 fig, T2 | ✅ Phase 4a |
+| H1 | `configs/experiments/arithmetic_halting.toml` | Adaptive halting: CRC compute-vs-accuracy | F4 | ✅ Phase 4b |
 | A1 | (ablation grid) | Feature-group leave-one-out; τ, curvature fidelity | T2 | ⏳ Phase 4 |
 
 ## Figures → claims (mirrored in `paper/README.md`)
 
 - **F1** method schematic (encode → K-restart descent → geometry → certificate) — TikZ, not data.
-- **F2** risk–coverage curves per nonconformity score — **core selective-prediction result**.
-- **F3** empirical coverage vs nominal 1−α — calibration validity (must sit on the diagonal).
-- **F4** compute–accuracy tradeoff under adaptive halting.
-- **F5** geometry-feature diagnostics: basin/curvature distributions, correct vs incorrect.
-- **F6** distribution-shift stress: coverage/abstention under OOD difficulty.
+- **F2** risk–coverage curves per nonconformity score — **core selective-prediction result**. ✅
+- **F3** empirical coverage vs nominal 1−α — calibration validity (must sit on the diagonal). ✅
+- **F4** compute–accuracy tradeoff under adaptive halting. ✅
+- **F5** geometry-feature diagnostics: basin/curvature distributions, correct vs incorrect. ✅
+- **F6** distribution-shift stress: selective risk holds ID, breaks under OOD → motivates abstention. ✅
 - **T1** main results (accuracy, coverage, abstention rate, avg compute).
 - **T2** ablations (feature leave-one-out, K sweep).
 - **T3** reproducibility appendix (seeds, configs, env) — auto-generated from ledger.
@@ -30,6 +32,110 @@ Every experiment: config file → purpose → expected figure/table → status.
 Paired-bootstrap 95% CI on `ΔAURC(raw terminal energy − geometry)`. If it includes 0, the core
 claim fails. Keep the raw-energy baseline in every selective experiment (invariant 8).
 
+**E1 result (2026-07-24, run_id `c1557364f51d`, seed 0):** ID base acc 0.81 (regime ✓). The learned
+geometry mapper **beats the best raw-energy baseline**: AURC 0.071 vs 0.145, `ΔAURC = +0.074`, 95%
+CI `[+0.054, +0.093]` — **excludes 0, geometry wins.** LTT abstention at α=0.1, δ=0.05 answers 69%
+of inputs at selective risk 0.075 ≤ 0.1 (base error 0.19 → the guarantee genuinely abstains). This
+*reverses* the Phase-2 single-feature picture where `energy/std` out-separated any lone geometry
+feature — the signal is in the **combined** geometry vector, exactly the paper's claim. Regenerate
+F2/F3 with `make figures`. Single-seed so far; the ≥5-seed paired bootstrap is Phase 4.
+
+**S1 K-ablation (2026-07-25, 5 seeds/K, reduced folds n_test=600):** the geometry lift over the best
+raw-energy baseline **grows monotonically with restarts** — exactly the mechanism the method claims:
+
+| K | AURC(geom) | ΔAURC (energy−geom) | seeds with CI excl. 0 |
+|---|-----------|---------------------|-----------------------|
+| 1 (≈EBT) | 0.123 | +0.026 ± 0.016 | **1/5** |
+| 2 | 0.104 | +0.043 ± 0.024 | 3/5 |
+| 4 | 0.082 | +0.062 ± 0.035 | 4/5 |
+| 8 | 0.076 | +0.070 ± 0.033 | **5/5** |
+| 16 | 0.051 | +0.083 ± 0.035 | **5/5** |
+
+K=1 — a single descent with no basin geometry (≈ the EBT baseline) — barely separates (1/5 seeds);
+the **restart** geometry is what carries the signal. This confirms the secondary falsifier does *not*
+fire: K>1 gives clear lift over K=1. (T2 also lists a K=12 row from the two full-fold E1 runs, which
+sits on the same trend.) Regenerate T1/T2/T3 with `make tables`, the figure with `make figures`.
+
+## Adaptive halting — the second guarantee (H1 → F4)
+
+`L(τ) = 1[early-stopped best-of-N answer ≠ full-budget best-of-N answer]`, stopping when basin
+agreement crosses τ; CRC picks the most compute-saving τ with `E[L] ≤ α`.
+
+**H1 result (2026-07-25, run_id `685d46f13e58`, seed 0):** CRC chose τ̂=0.917 at α=0.1 and the
+empirical risk was **monotone in τ** (CRC's assumption held). At that operating point the descent
+uses **42% of the step budget (57.8% compute saved)** with a halting risk of **0.9% ≤ α=10%** and
+**no accuracy loss** (full 0.804 → halted 0.807). F4 is the compute-vs-accuracy Pareto; the CRC
+point sits at the elbow. Both guarantees are now live: LTT abstention (E1) and CRC halting (H1).
+
+## OOD stress — the guarantee breaks under shift (F6, run_id `1acd65c19072`)
+
+The LTT selective-risk threshold is calibrated on the **ID** calib fold and applied to the **OOD**
+test fold (magnitude shift `ood_max_operand`, OOD acc 0.21). At α=0.1 the achieved selective risk is
+**0.075 ID (valid) vs 0.762 OOD** — the ID-calibrated threshold answers 66% of OOD inputs at 76%
+error, so the guarantee is void once exchangeability breaks. F6 plots achieved-vs-target risk: ID on
+the diagonal, OOD far above. This is the concrete argument for abstention/shift-detection in
+critical systems (the guarantees are *marginal under exchangeability*, by design). **F5** shows the
+mechanism: basin-agreement features (entropy, ρ) separate correct from incorrect best on arithmetic;
+curvature separates weakly here — reported honestly.
+
+## E2 — generalization to a second reasoning family (graph shortest-path)
+
+Does the discovery replicate beyond additive arithmetic? Graph shortest-path length (BFS label,
+larger-graph OOD) is a relational/combinatorial family; tuned to ID ≈ 0.70 (majority baseline ≈ 0.41).
+**Result (2026-07-26, 5 seeds, `sweeps/graph_seeds.toml`):** geometry beats the best raw-energy
+baseline on **5/5 seeds** — AURC 0.159 vs 0.270, `ΔAURC = +0.111 ± 0.054`, every seed's 95% CI
+excludes 0. Alongside arithmetic (ΔAURC +0.083 ± 0.034, 5/5), the T1 table now shows the discovery
+holds across **two structurally distinct tasks** — it is not an arithmetic artifact. (Caveat: on
+graph the LTT abstention at α=0.1 with reduced calib folds certifies zero coverage — conservative,
+not a failure; ΔAURC, the threshold-free ranking metric, is the generalization evidence.)
+
+## Baseline stress test — geometry vs standard softmax confidence (Phase 4e) ⚠️ **key caveat**
+
+Until now geometry was only compared against scalar **energy**. Adding the standard
+softmax-confidence baselines the plan calls for — **MSP, temperature-scaled MSP, predictive
+entropy** (from the decoder logits) — sharply bounds the claim. Over 5 seeds per task
+(`arith_seeds.toml` / `graph_seeds.toml`):
+
+| task | ΔAURC vs best **energy** (5 seeds) | ΔAURC vs best **overall baseline** (5 seeds) |
+|------|-----------------------------------|----------------------------------------------|
+| arithmetic | **+0.085 ± 0.036** — geometry wins 5/5 | **−0.011 ± 0.006** — geometry wins **0/5** |
+| graph | **+0.111 ± 0.054** — geometry wins 5/5 | **−0.034 ± 0.008** — geometry wins **0/5** |
+
+**Verdict.** The invariant-8 sacred test holds: geometry beats the EBT **scalar energy** on both
+tasks, all seeds. But the *broader* claim does **not** survive — a plain softmax-confidence baseline
+(MSP / entropy / temperature) **ties geometry on arithmetic and beats it on graph** (F2 shows
+geometry, MSP, and entropy overlapping at the bottom, all far below energy). So: **landscape geometry
+beats scalar energy but is not a better nonconformity score than softmax confidence** under the
+current reasoner. This is a real, reported limitation, not a bug.
+
+**Which geometry features drive it? (A1 feature ablation → T2b, 5 seeds/task).** The geometry
+vector *includes* the 3 energy statistics, so does the win over raw energy just re-use energy?
+Leave-one-group-out AURC (lower = better) says no:
+
+| subset | arithmetic AURC | graph AURC |
+|--------|-----------------|-----------|
+| full (14 feats) | 0.059 | 0.159 |
+| **drop energy** | **0.062** | **0.162** |
+| drop curvature | 0.063 | 0.160 |
+| drop dynamics | 0.064 | 0.187 |
+| drop basin | **0.110** | **0.175** |
+| *(best raw energy)* | *0.144* | *0.270* |
+
+`drop_energy ≈ full` and both far below raw energy → the win is **genuinely from the non-energy
+features**, not re-using energy. **Basin agreement is the dominant driver** on both tasks (removing
+it nearly doubles arithmetic AURC toward the energy level); descent **dynamics** helps too (most on
+graph); curvature adds little here. This is the mechanism behind the invariant-8 win — and it is
+consistent with F5. (It does not change the 4e caveat: basin/dynamics beat *energy* but not softmax.)
+
+**Likely cause of the softmax tie + the lever.** The reasoner is the Phase-1 **supervised
+basin-center** model with a
+shallow decoder — its softmax is already informative, leaving little for geometry to add beyond
+energy. The research plan's **Phase-4 IRED annealed-landscape / score-matching training** is the
+intended fix: a genuinely learned, multi-basin landscape should make restart geometry carry signal
+the softmax does not. Re-running this baseline stress test under IRED training is the priority next
+experiment (see SESSION_HANDOFF). MC-dropout / deep ensembles remain deferred (the K-restart
+best-of-N already supplies ensemble-like diversity).
+
 ## Operating regime
 
 Tune each task's difficulty so base accuracy is **70–85%**. Fully-solved tasks saturate AURC and
diff --git a/docs/SESSION_HANDOFF.md b/docs/SESSION_HANDOFF.md
index d354f5d..0b73f6a 100644
--- a/docs/SESSION_HANDOFF.md
+++ b/docs/SESSION_HANDOFF.md
@@ -2,7 +2,7 @@
 
 **Start here.** Rolling state of the project between sessions.
 
-## Current state (2026-07-24)
+## Current state (2026-07-25)
 
 - **Phase 0 (skeleton): ✅** repo tree, `uv`/`pyproject.toml`, `Makefile`, CI, config system
   (`edc.config`), deterministic seeding (`edc.seeding`), append-only ledger (`edc.ledger`),
@@ -12,24 +12,93 @@
   arithmetic task (`edc.tasks.arithmetic`). `make smoke` runs end-to-end on CPU and appends a
   ledger row. HVP curvature primitive (`edc.geometry.curvature`) implemented + tested early
   because it is the method's crux.
-- **Phases 2–5: stubbed** with `NotImplementedError` + phase notes.
+- **Phase 2 (geometry features): ✅** `edc.geometry.{basin,energy_stats,dynamics}` (plain
+  NumPy, invariant 1) + `features.geometry_features` assembling a 14-feature per-input vector
+  (basin 3 · energy 3 · curvature 4 · dynamics 4). Curvature is batched over particles by the
+  new `curvature.batched_curvature` (the only JAX in the feature path). `TrajectoryRecord` now
+  carries `h_x` so curvature can be recomputed at `z*`. `edc.cli geometry` prints per-feature
+  correct-vs-incorrect AUROC (geometry vs the raw-energy baseline) and appends a ledger row.
+- **Phase 3 (conformal + falsification + abstention): ✅** `eval.metrics` (`risk_coverage_curve`,
+  tie-robust `aurc`, `paired_bootstrap_delta_aurc`, `selective_accuracy_at_coverage`, `ece`);
+  `conformal.nonconformity` (sklearn logistic mapper + `1−ρ_basin` fallback), `conformal.selective`,
+  `conformal.ltt` (Hoeffding-Bentkus p-value + Bonferroni LTT), `conformal.crc.calibrate`;
+  `eval.evaluate` (disjoint fit/calib/test folds) + `experiments/run_experiment.py`; F2/F3 in
+  `plotting` + `analysis/make_figures.py`. **E1 ran: geometry beats raw energy** (ΔAURC +0.074, 95%
+  CI [+0.054, +0.093]); LTT holds (69% coverage @ selective risk 0.075 ≤ α=0.1). See EXPERIMENTS.md.
+- **Phase 4a (robustness + K-ablation + reporting): ✅** `experiments/run_sweep.py` (grid+override,
+  reuses `run_experiment.run_and_append`; `evaluate(include_ood=False)` for cheaper cells),
+  `analysis/aggregate.py` (multi-seed ledger aggregation), `analysis/make_tables.py` (T1/T2/T3 →
+  `paper/tables/*.tex`), and the S1 K-lift figure. **S1 ran (5 seeds × K∈{1,2,4,8,16}):** the ΔAURC
+  lift grows monotonically with K — K=1 (≈EBT) separates on only 1/5 seeds, K≥8 on 5/5. The
+  restart geometry is the mechanism; the "K>1 gives no lift" falsifier does not fire. See EXPERIMENTS.md.
+- **Phase 4b (adaptive halting — the second guarantee): ✅** opt-in per-step decoding
+  (`optimizer.record_z` → `restarts.solve(record_steps=)` → `TrajectoryRecord.step_pred`),
+  `halting.adaptive` (basin-agreement policy + CRC calibration via the built `crc.calibrate`,
+  `λ=1−τ`), `eval.evaluate_halting`, `experiments/run_halting.py`, F4 in plotting/make_figures.
+  **H1 ran:** CRC τ̂=0.917 saves **57.8% compute** at halting risk **0.9% ≤ α=0.1**, no accuracy
+  loss, risk monotone in τ. Both guarantees now live (LTT abstention + CRC halting).
+- **Phase 4c (arithmetic figure set complete — F5 + F6): ✅** `evaluate` gained `feature_diagnostics`
+  (F5) and `ood_validity`/`ood_ltt` (F6, ID-calibrated λ applied to OOD); `plotting.feature_diagnostics`
+  + `plotting.ood_stress`; F5/F6 in make_figures. **F6:** at α=0.1 selective risk is 0.075 ID (valid)
+  vs **0.762 OOD** — the guarantee breaks under shift, motivating abstention. **F5:** basin-agreement
+  features separate correct/incorrect best; curvature weak on arithmetic (reported honestly).
+- **Phase 4d (generalization — 2nd task): ✅** `GraphPlanningTask` (BFS shortest-path, larger-graph
+  OOD; `tasks/graph_planning.py`), task-aware `analysis.aggregate` + per-task `make_tables` T1,
+  `configs/experiments/graph_selective.toml` + `sweeps/graph_seeds.toml`. **E2 (5 seeds):** geometry
+  beats energy on **5/5 seeds**, ΔAURC +0.111±0.054 (vs arithmetic +0.083±0.034) — the discovery
+  replicates on a structurally distinct family. `plotting.k_restart_lift` made task-aware so graph's
+  fixed-K rows don't contaminate the arithmetic S1 figure.
+- **Phase 4e (baseline stress test — softmax confidence): ✅ ⚠️ key caveat.** Added MSP,
+  temperature-scaled MSP, predictive-entropy baselines (`eval/baselines.py`), a
+  `delta_aurc_vs_best_baseline` falsification, task-aware `headline_cell`, per-task T1 with both
+  ΔAURCs. **Result (5 seeds/task):** geometry beats scalar **energy** (invariant 8 ✓: +0.085
+  arithmetic, +0.111 graph, 5/5 each) but does **not** beat softmax confidence — ΔAURC vs best
+  baseline is −0.011 (arith, 0/5) and −0.034 (graph, 0/5). MSP/entropy tie-or-beat geometry (F2).
+  **The "geometry is the best nonconformity score" claim fails against softmax confidence** under
+  the Phase-1 basin-center reasoner. Likely cause: shallow decoder + simple landscape → softmax
+  already informative. **Lever: Phase-4 IRED landscape training** (below).
+- **Phase 4f (feature-group ablation — A1/T2): ✅** `evaluate` computes `feature_ablation`
+  (leave-one-group-out + group-only mappers); aggregated → `T2b_feature_ablation.tex`. **Result
+  (5 seeds/task):** `drop_energy` ≈ `full` and both far below raw energy → geometry's win over
+  energy is from the **non-energy** features; **basin agreement is the dominant driver** (dynamics
+  second, esp. graph; curvature/energy minor). Rebuts "it's just energy"; consistent with F5.
+- **Deferred / Phase 5: stubbed.** IRED landscape training (priority lever, below), Modal, paper.
 
 ## What is real vs stub
 
-- Real: config/seeding/ledger, arithmetic task, energy/encoder/decoder, Langevin
-  optimizer+restarts, training loop, curvature HVP/λ_max/trace, split-conformal threshold.
-- Stub: geometry `features/basin/energy_stats/dynamics`, conformal `ltt/crc/selective/
-  nonconformity`, halting, eval, graph/logic/hard tasks, experiment/sweep/modal runners,
-  figure/table generators.
+- Real: config/seeding/ledger, arithmetic task (+`ood_max_operand` shift) + **graph shortest-path
+  task** (size-shift OOD), energy/encoder/decoder, Langevin optimizer+restarts (opt-in per-step
+  decode), training loop,
+  curvature HVP/λ_max/trace + batched wrapper, **all geometry features + assembly**,
+  `single_feature_auroc`, `edc.cli geometry`; **the full selective-prediction stack**: AURC/ΔAURC,
+  nonconformity mapper, split-conformal/LTT/CRC, `eval.evaluate`, `run_experiment.py`, F2/F3;
+  **the sweep/aggregation/tables stack**: `run_sweep.py`, `analysis.aggregate`, `make_tables`
+  (T1–T3), S1 K-lift figure; **adaptive halting**: `halting.adaptive`, `eval.evaluate_halting`,
+  `run_halting.py`, F4; **F5 mechanism + F6 OOD stress** (`feature_diagnostics`, `ood_validity`);
+  **softmax-confidence baselines** (`eval/baselines.py`: MSP/temp/entropy).
+- Stub: **IRED landscape training** (`train/losses.py` is still the Phase-1 basin-center scheme),
+  logic/hard tasks (E3/E4), Modal runner, paper write-up.
 
-## Next steps (Phase 2)
+## Next steps (Phase 4f) — reoriented by the 4e caveat
 
-1. Implement `edc.geometry.{basin,energy_stats,dynamics}` from `TrajectoryRecord`; assemble in
-   `features.geometry_features` (must run under `vmap` over restarts).
-2. Add `tests/test_geometry_features.py` (shapes, invariances) + an F5-style diagnostic showing
-   geometry separates correct vs incorrect on arithmetic.
-3. Then Phase 3: nonconformity mapper → LTT/CRC → selective/halting, with the coverage-validity
-   test and the ΔAURC falsification harness.
+1. **IRED landscape training (the priority).** Replace the supervised basin-center loss
+   (`train/losses.py`) with an IRED-style annealed-landscape / denoising-score-matching objective so
+   the energy is a genuinely learned multi-basin surface. Hypothesis: restart geometry then carries
+   signal the (now less-informative) softmax does not. **Re-run the 4e baseline stress test** — this
+   is the make-or-break experiment for the discovery's practical claim.
+2. **If geometry still ties softmax:** reframe honestly — the contribution is the distribution-free
+   *certificate* machinery (LTT/CRC + F2–F6) over an EBM reasoner, with geometry beating the EBT
+   scalar-energy signal specifically (invariant 8), not a universal win over all confidences.
+
+3. **Lower priority (after the IRED verdict):** a 3rd task (logic/sudoku, E3/E4) for more
+   generalization; full-fold ≥5-seed E1 for a headline T1; paper write-up (figure set F2–F6/S1 and
+   tables T1–T3 are complete for arithmetic + graph).
+
+## Operating regime note
+
+E1 is tuned to ~77–81% ID base accuracy via `n_operands=6, max_operand=7, epochs=7` (base error >
+α, so abstention is genuinely exercised). The smoke config saturates (~97%) and its `n_calib=128` is
+too small for LTT to certify — that is expected, not a bug; use the E1 config for real numbers.
 
 ## Invariants to keep green
 
diff --git a/experiments/run_experiment.py b/experiments/run_experiment.py
index a4f76b9..8e8bffc 100644
--- a/experiments/run_experiment.py
+++ b/experiments/run_experiment.py
@@ -1,21 +1,85 @@
-"""Run a single named experiment from a config and append a ledger row. [Phase 4]
+"""Run a single named experiment from a config and append a ledger row. [Phase 3]
 
     PYTHONPATH=src python experiments/run_experiment.py configs/experiments/<name>.toml
 
-Phase 1 uses ``edc.cli smoke`` for the train->infer->accuracy path. This runner adds the full
-geometry + conformal evaluation (``edc.eval.evaluate``) once Phase 3 lands.
+Trains the reasoner, runs the disjoint-fold geometry-vs-energy selective-prediction evaluation
+(``edc.eval.evaluate``), and appends one ``split="selective"`` row to results/ledger.jsonl. The
+falsification verdict (does geometry beat raw energy in ΔAURC?) and the LTT abstention guarantee
+live in that row's ``metrics``; figures/tables regenerate from it (invariant 6).
+
+``run_and_append`` is the shared train->evaluate->ledger step; ``experiments/run_sweep.py`` reuses
+it for every grid cell.
 """
 
 from __future__ import annotations
 
+import datetime as _dt
 import sys
+import uuid
+
+from edc.config import load_config
+from edc.eval.evaluate import evaluate
+from edc.ledger import RunRecord, append
+from edc.registry import build_task
+
+
+def run_and_append(cfg, task, *, include_ood: bool = True, quiet: bool = False) -> dict:
+    """Train + evaluate ``cfg`` and append one ``split="selective"`` ledger row; return the row.
+
+    The single source of truth for turning a resolved config into a recorded result — used by both
+    the single-experiment CLI and the sweep runner so they cannot drift. Stamps wall-clock time
+    here (keeping ``edc.ledger`` pure).
+    """
+    m = evaluate(cfg, task, include_ood=include_ood)
+    if not quiet:
+        _print_summary(cfg, m)
+    row = append(RunRecord(
+        run_id=uuid.uuid4().hex[:12],
+        timestamp=_dt.datetime.now(_dt.UTC).isoformat(),
+        resolved_config=cfg.to_dict(),
+        task=task.name,
+        split="selective",
+        seed=cfg.run.seed,
+        metrics=m,
+    ))
+    return row
+
+
+def _print_summary(cfg, m: dict) -> None:
+    d, lo, hi = m["delta_aurc_vs_best_energy"]
+    verdict = "GEOMETRY WINS" if m["geometry_wins"] else "not separated (CI includes 0)"
+    ood = f"  OOD acc={m['accuracy_ood']:.3f}" if "accuracy_ood" in m else ""
+    print(f"[edc] K={m['k_restarts']} seed={cfg.run.seed}  ID acc={m['accuracy_id']:.3f}{ood}  "
+          f"base error={m['base_error']:.3f}")
+    print(f"[edc] AURC geometry={m['aurc']['geometry']:.4f}  "
+          f"best energy ({m['best_energy_baseline']})={m['aurc'][m['best_energy_baseline']]:.4f}")
+    print(f"[edc] ΔAURC(best energy − geometry)={d:+.4f}  95% CI=[{lo:+.4f}, {hi:+.4f}]")
+    print(f"[edc] falsification: {verdict}")
+    if "delta_aurc_vs_best_baseline" in m:
+        db, lob, hib = m["delta_aurc_vs_best_baseline"]
+        vb = "GEOMETRY WINS" if m["geometry_wins_vs_baseline"] else "not separated (CI includes 0)"
+        print(f"[edc] best baseline overall: {m['best_baseline']} "
+              f"(AURC={m['aurc'][m['best_baseline']]:.4f}, T={m['temperature']:.2f})")
+        print(f"[edc] ΔAURC(best baseline − geometry)={db:+.4f}  95% CI=[{lob:+.4f}, {hib:+.4f}]"
+              f"  -> {vb}")
+    ltt_b = m["ltt"]
+    print(f"[edc] LTT @ alpha={ltt_b['alpha']}, delta={ltt_b['delta']}: "
+          f"coverage={ltt_b['coverage']:.3f}  selective_risk={ltt_b['selective_risk']:.3f}  "
+          f"within_budget={ltt_b['risk_within_budget']}")
 
 
 def main(argv: list[str]) -> int:
-    raise NotImplementedError(
-        "Phase 4: load config -> edc.eval.evaluate -> edc.ledger.append. "
-        "For now use: PYTHONPATH=src python -m edc.cli smoke --config configs/smoke.toml"
-    )
+    if not argv:
+        print("usage: run_experiment.py <config.toml>", file=sys.stderr)
+        return 2
+    cfg = load_config(argv[0])
+    task_kwargs = cfg.to_dict().get("task", {}).get(cfg.run.task, {})
+    task = build_task(cfg.run.task, **task_kwargs)
+
+    print(f"[edc] experiment '{cfg.run.task}' (seed={cfg.run.seed}) — training + evaluating ...")
+    row = run_and_append(cfg, task)
+    print(f"[edc] appended ledger row run_id={row['run_id']} -> results/ledger.jsonl")
+    return 0
 
 
 if __name__ == "__main__":
diff --git a/experiments/run_halting.py b/experiments/run_halting.py
new file mode 100644
index 0000000..0614f99
--- /dev/null
+++ b/experiments/run_halting.py
@@ -0,0 +1,56 @@
+"""Run the adaptive-halting experiment and append a ledger row. [Phase 4b]
+
+    PYTHONPATH=src python experiments/run_halting.py configs/experiments/arithmetic_halting.toml
+
+Trains the reasoner, CRC-calibrates the basin-agreement halting threshold, and appends one
+``split="halting"`` row with the compute-vs-accuracy metrics + tau-sweep. F4 regenerates from it
+(invariant 6).
+"""
+
+from __future__ import annotations
+
+import datetime as _dt
+import sys
+import uuid
+
+from edc.config import load_config
+from edc.eval.evaluate_halting import evaluate_halting
+from edc.ledger import RunRecord, append
+from edc.registry import build_task
+
+
+def main(argv: list[str]) -> int:
+    if not argv:
+        print("usage: run_halting.py <config.toml>", file=sys.stderr)
+        return 2
+    cfg = load_config(argv[0])
+    task_kwargs = cfg.to_dict().get("task", {}).get(cfg.run.task, {})
+    task = build_task(cfg.run.task, **task_kwargs)
+
+    print(f"[edc] halting '{cfg.run.task}' (seed={cfg.run.seed}) — training + calibrating ...")
+    m = evaluate_halting(cfg, task)
+
+    tau = m["tau_hat"]
+    tau_s = f"{tau:.3f}" if tau is not None else "None (budget infeasible)"
+    print(f"[edc] CRC tau_hat={tau_s}  @ alpha={m['alpha']}  (risk monotone in tau: "
+          f"{m['risk_monotone_in_tau']})")
+    print(f"[edc] compute used={m['compute_used']:.3f} (saved {m['compute_saved']:.3f})  "
+          f"halting risk={m['halting_risk']:.3f}  within_budget={m['risk_within_budget']}")
+    print(f"[edc] accuracy: full={m['full_accuracy']:.3f}  halted={m['halted_accuracy']:.3f}  "
+          f"drop={m['accuracy_drop']:+.3f}")
+
+    row = append(RunRecord(
+        run_id=uuid.uuid4().hex[:12],
+        timestamp=_dt.datetime.now(_dt.UTC).isoformat(),
+        resolved_config=cfg.to_dict(),
+        task=task.name,
+        split="halting",
+        seed=cfg.run.seed,
+        metrics=m,
+    ))
+    print(f"[edc] appended ledger row run_id={row['run_id']} -> results/ledger.jsonl")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main(sys.argv[1:]))
diff --git a/experiments/run_sweep.py b/experiments/run_sweep.py
index 7c0cdb3..a0562f7 100644
--- a/experiments/run_sweep.py
+++ b/experiments/run_sweep.py
@@ -2,17 +2,93 @@
 
     PYTHONPATH=src python experiments/run_sweep.py configs/sweeps/<grid>.toml
 
-Grids sweep K restarts, step size, Langevin temperature, and conformal alpha (see
-configs/sweeps/). Each cell calls the same path as run_experiment and appends its own row.
+A sweep file names a ``base`` config, optional constant ``[sweep.override]`` keys, and a
+``[sweep.grid]`` of dotted-key -> list. The grid is cartesian-expanded; every cell deep-merges its
+overrides onto the base, then takes the same train->evaluate->ledger path as ``run_experiment`` (one
+appended ``split="selective"`` row per cell). OOD is skipped in the sweep to save compute.
+
+    [sweep]
+    base = "configs/experiments/arithmetic_selective.toml"
+    [sweep.override]                       # applied to every cell
+    "eval.n_eval" = 600
+    [sweep.grid]                           # cartesian-expanded
+    "inference.k_restarts" = [1, 2, 4, 8, 16]
+    "run.seed" = [0, 1, 2, 3, 4]
 """
 
 from __future__ import annotations
 
+import copy
+import itertools
 import sys
+import tomllib
+from pathlib import Path
+
+from edc.config import REPO_ROOT, load_config
+from edc.registry import build_task
+
+try:  # sys.path[0] is experiments/ when run as a script; fall back to package form.
+    from run_experiment import run_and_append
+except ImportError:  # pragma: no cover
+    from experiments.run_experiment import run_and_append
+
+
+def _set_dotted(d: dict, dotted: str, value) -> None:
+    """Set ``d["a"]["b"] = value`` from the key ``"a.b"``, creating intermediate dicts."""
+    keys = dotted.split(".")
+    node = d
+    for k in keys[:-1]:
+        node = node.setdefault(k, {})
+    node[keys[-1]] = value
+
+
+def expand_grid(grid: dict[str, list]) -> list[dict]:
+    """Cartesian product of a dotted-key -> list grid into ``{dotted_key: value}`` cells."""
+    if not grid:
+        return [{}]
+    keys = list(grid)
+    combos = itertools.product(*(grid[k] for k in keys))
+    return [dict(zip(keys, combo, strict=True)) for combo in combos]
+
+
+def load_sweep(path: str | Path) -> tuple[dict, dict, list[dict]]:
+    """Return ``(base_resolved_dict, override, cells)`` from a sweep TOML."""
+    with open(path, "rb") as f:
+        spec = tomllib.load(f)["sweep"]
+    base = load_config(spec["base"]).to_dict()
+    override = spec.get("override", {})
+    cells = expand_grid(spec.get("grid", {}))
+    return base, override, cells
+
+
+def cell_config_dict(base: dict, override: dict, cell: dict) -> dict:
+    """Deep-copy ``base`` and apply the constant overrides then this cell's assignments."""
+    d = copy.deepcopy(base)
+    for dotted, value in {**override, **cell}.items():
+        _set_dotted(d, dotted, value)
+    return d
 
 
 def main(argv: list[str]) -> int:
-    raise NotImplementedError("Phase 4: cartesian-expand the grid and dispatch run_experiment.")
+    if not argv:
+        print("usage: run_sweep.py <sweep.toml>", file=sys.stderr)
+        return 2
+    base, override, cells = load_sweep(argv[0])
+    from edc.config import load_from_dict  # local: keep module import surface small
+
+    print(f"[edc] sweep '{Path(argv[0]).name}': {len(cells)} cells "
+          f"(override={override or '{}'}) -> appending to results/ledger.jsonl")
+    for i, cell in enumerate(cells, 1):
+        d = cell_config_dict(base, override, cell)
+        cfg = load_from_dict(d)
+        task_kwargs = d.get("task", {}).get(cfg.run.task, {})
+        task = build_task(cfg.run.task, **task_kwargs)
+        print(f"\n[edc] cell {i}/{len(cells)}: {cell}")
+        row = run_and_append(cfg, task, include_ood=False)
+        print(f"[edc] -> run_id={row['run_id']}")
+    ledger = REPO_ROOT / "results/ledger.jsonl"
+    print(f"\n[edc] sweep complete: {len(cells)} rows appended -> {ledger}")
+    return 0
 
 
 if __name__ == "__main__":
diff --git a/paper/tables/T1_main.tex b/paper/tables/T1_main.tex
new file mode 100644
index 0000000..d9a664a
--- /dev/null
+++ b/paper/tables/T1_main.tex
@@ -0,0 +1,14 @@
+% auto-generated by analysis/make_tables.py — do not edit. run_ids=['46752558c5df', 'b8dc1a068df5', 'aa60425c10e1', 'c948f8130a3d', '4a5576e4c244', 'd1ae14a590d9', '452500308f3b', 'b1e83969abe5', '70d97e02bed0', '46bb116f8c7c']
+\begin{table}[t]
+  \centering
+  \caption{Main selective-prediction results across reasoning families (mean$\pm$std over seeds): geometry's $\Delta$AURC over the best raw-energy baseline and over the best of all baselines (energy, MSP, temperature-scaled MSP, predictive entropy). $\Delta$AURC $>0$ with the CI clearing 0 means geometry wins; ``seeds win`` counts seeds beating the best overall baseline.}
+  \label{tab:main}
+  \begin{tabular}{lrcccccc}
+    \toprule
+    task & $K$ & acc & AURC geom & $\Delta$AURC vs energy & $\Delta$AURC vs best base & seeds win & LTT cov \\
+    \midrule
+    arithmetic & 12 & $0.83 \pm 0.02$ & $0.059 \pm 0.018$ & $0.085 \pm 0.036$ & $-0.011 \pm 0.006$ & 0/5 & $0.55 \pm 0.30$ \\
+    graph\_planning & 12 & $0.70 \pm 0.02$ & $0.159 \pm 0.012$ & $0.111 \pm 0.054$ & $-0.034 \pm 0.008$ & 0/5 & $0.00 \pm 0.00$ \\
+    \bottomrule
+  \end{tabular}
+\end{table}
diff --git a/paper/tables/T2_kablation.tex b/paper/tables/T2_kablation.tex
new file mode 100644
index 0000000..7aef7e2
--- /dev/null
+++ b/paper/tables/T2_kablation.tex
@@ -0,0 +1,18 @@
+% auto-generated by analysis/make_tables.py — do not edit. run_ids=['acd437459961', '852e712d4157', '8b6b1348cf28', '56ea729e5ea5', '8b8e1e4a7381', 'a4170c27c22b', 'af69f3e49ad2', '25320822fdfd', '9519934cb397', '978eda331152', 'b73a5b1750f2', 'f14b18cfaee4', '04b6026d0afc', '3dc77bab6f6f', '8c4a180e7c9e', 'db2094e35b89', '7a85d96687d2', '7f822ee20bdb', 'e7cfb7fd624f', '539e9c3f0431', '1bfe6b304e0f', 'f63036690d31', '46752558c5df', 'b8dc1a068df5', 'aa60425c10e1', 'c948f8130a3d', '4a5576e4c244', '8d6b3f5b24a2', 'b5b8cfd55dee', '1a2ccd4ef091', 'ff953d4a0e15', 'e14af222ab26']
+\begin{table}[t]
+  \centering
+  \caption{Restart ablation: does landscape geometry across $K$ restarts add signal over a single descent ($K{=}1\approx$ EBT)? Lift is $\Delta$AURC $>0$ with the CI clearing 0.}
+  \label{tab:kablation}
+  \begin{tabular}{rccccc}
+    \toprule
+    $K$ & AURC geom & AURC energy & $\Delta$AURC & seeds win & LTT cov \\
+    \midrule
+    1 & $0.123 \pm 0.040$ & $0.149 \pm 0.053$ & $0.026 \pm 0.016$ & 1/5 & $0.16 \pm 0.32$ \\
+    2 & $0.103 \pm 0.036$ & $0.146 \pm 0.048$ & $0.043 \pm 0.024$ & 3/5 & $0.18 \pm 0.36$ \\
+    4 & $0.082 \pm 0.038$ & $0.144 \pm 0.046$ & $0.062 \pm 0.035$ & 4/5 & $0.29 \pm 0.36$ \\
+    8 & $0.076 \pm 0.029$ & $0.145 \pm 0.053$ & $0.070 \pm 0.033$ & 5/5 & $0.54 \pm 0.29$ \\
+    12 & $0.057 \pm 0.019$ & $0.137 \pm 0.047$ & $0.080 \pm 0.031$ & 7/7 & $0.62 \pm 0.28$ \\
+    16 & $0.051 \pm 0.021$ & $0.134 \pm 0.050$ & $0.083 \pm 0.034$ & 5/5 & $0.68 \pm 0.11$ \\
+    \bottomrule
+  \end{tabular}
+\end{table}
diff --git a/paper/tables/T2b_feature_ablation.tex b/paper/tables/T2b_feature_ablation.tex
new file mode 100644
index 0000000..2a05474
--- /dev/null
+++ b/paper/tables/T2b_feature_ablation.tex
@@ -0,0 +1,17 @@
+% auto-generated by analysis/make_tables.py — do not edit. feature-group leave-one-out
+\begin{table}[t]
+  \centering
+  \caption{Feature-group leave-one-out ablation: test AURC of the geometry mapper refit on subsets of the 14 features (mean$\pm$std over seeds). ``drop\_X`` removes that group; comparing ``drop\_energy`` to ``full`` isolates the non-energy geometry contribution.}
+  \label{tab:ablation}
+  \begin{tabular}{lcc}
+    \toprule
+    subset (AURC, lower=better) & arithmetic & graph\_planning \\
+    \midrule
+    full & $0.059 \pm 0.018$ & $0.159 \pm 0.012$ \\
+    drop\_basin & $0.110 \pm 0.030$ & $0.175 \pm 0.017$ \\
+    drop\_energy & $0.062 \pm 0.018$ & $0.162 \pm 0.014$ \\
+    drop\_curv & $0.063 \pm 0.020$ & $0.160 \pm 0.012$ \\
+    drop\_dynamics & $0.064 \pm 0.024$ & $0.187 \pm 0.024$ \\
+    \bottomrule
+  \end{tabular}
+\end{table}
diff --git a/paper/tables/T3_repro.tex b/paper/tables/T3_repro.tex
new file mode 100644
index 0000000..baa6f63
--- /dev/null
+++ b/paper/tables/T3_repro.tex
@@ -0,0 +1,49 @@
+% auto-generated by analysis/make_tables.py — do not edit. n_rows=37
+\begin{table}[t]
+  \centering
+  \caption{Reproducibility appendix. Env: python 3.12.13, jax 0.11.0 (cpu). Every row regenerates from seed + config via \texttt{run\_sweep}.}
+  \label{tab:repro}
+  \begin{tabular}{lrrll}
+    \toprule
+    run\_id & seed & $K$ & config hash & git sha \\
+    \midrule
+    1bfe6b304e0f & 0 & ? & 53bd1f04ae90 & 7369abb \\
+    acd437459961 & 0 & 1 & f314930a27aa & 758d5f2 \\
+    852e712d4157 & 1 & 1 & 610a639808b6 & 758d5f2 \\
+    8b6b1348cf28 & 2 & 1 & cf02c245cbac & 758d5f2 \\
+    56ea729e5ea5 & 3 & 1 & a56243cda69f & 758d5f2 \\
+    8b8e1e4a7381 & 4 & 1 & e923fb1d771e & 758d5f2 \\
+    a4170c27c22b & 0 & 2 & b1a0a6873eb1 & 758d5f2 \\
+    af69f3e49ad2 & 1 & 2 & 1632b77f4493 & 758d5f2 \\
+    25320822fdfd & 2 & 2 & 17b96f1b0f17 & 758d5f2 \\
+    9519934cb397 & 3 & 2 & 059694ff7fbf & 758d5f2 \\
+    978eda331152 & 4 & 2 & ac1d6e63a53d & 758d5f2 \\
+    b73a5b1750f2 & 0 & 4 & 419a01656461 & 758d5f2 \\
+    f14b18cfaee4 & 1 & 4 & 45792209eed5 & 758d5f2 \\
+    04b6026d0afc & 2 & 4 & 825c1996aff6 & 758d5f2 \\
+    3dc77bab6f6f & 3 & 4 & 0347ef3f7e0e & 758d5f2 \\
+    8c4a180e7c9e & 4 & 4 & d5bb5763735c & 758d5f2 \\
+    db2094e35b89 & 0 & 8 & bbd87d16beee & 758d5f2 \\
+    7a85d96687d2 & 1 & 8 & 537005d71045 & 758d5f2 \\
+    7f822ee20bdb & 2 & 8 & 9237080427a6 & 758d5f2 \\
+    e7cfb7fd624f & 3 & 8 & 6cacd4aa98fa & 758d5f2 \\
+    539e9c3f0431 & 4 & 8 & 1be43f267351 & 758d5f2 \\
+    f63036690d31 & 0 & 12 & cecbb6158c46 & 3d7ce49 \\
+    d1ae14a590d9 & 0 & 12 & 3643423663c7 & 5207145 \\
+    46752558c5df & 0 & 12 & c6246ddaef7e & 5207145 \\
+    452500308f3b & 1 & 12 & 60da27bc4d66 & 5207145 \\
+    b8dc1a068df5 & 1 & 12 & e65ef9e88f92 & 5207145 \\
+    b1e83969abe5 & 2 & 12 & ff22f3433ff5 & 5207145 \\
+    aa60425c10e1 & 2 & 12 & cacbb6ece843 & 5207145 \\
+    70d97e02bed0 & 3 & 12 & fbed14eeb756 & 5207145 \\
+    c948f8130a3d & 3 & 12 & 054f01c6c637 & 5207145 \\
+    46bb116f8c7c & 4 & 12 & 609a4152cf1e & 5207145 \\
+    4a5576e4c244 & 4 & 12 & 0b564d7f9e23 & 5207145 \\
+    8d6b3f5b24a2 & 0 & 16 & 3c20300a5e78 & 758d5f2 \\
+    b5b8cfd55dee & 1 & 16 & 22b25365d745 & 758d5f2 \\
+    1a2ccd4ef091 & 2 & 16 & b6cea5e21b36 & 758d5f2 \\
+    ff953d4a0e15 & 3 & 16 & ef0196b5978e & 758d5f2 \\
+    e14af222ab26 & 4 & 16 & 9c69bc8a7ce8 & 758d5f2 \\
+    \bottomrule
+  \end{tabular}
+\end{table}
diff --git a/results/ledger.jsonl b/results/ledger.jsonl
index e69de29..a4fe703 100644
--- a/results/ledger.jsonl
+++ b/results/ledger.jsonl
@@ -0,0 +1,58 @@
+{"run_id": "96eb9d2e007d", "timestamp": "2026-07-24T14:26:12.273421+00:00", "git_sha": "7369abb", "config_hash": "40bc3df0b42a", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "smoke"}, "model": {"latent_dim": 24, "hidden_dim": 64, "context_dim": 32}, "inference": {"k_restarts": 4, "steps": 25, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 12, "batch_size": 128, "lr": 0.002, "n_train": 3000, "n_neg": 2, "neg_noise": 0.5}, "eval": {"n_eval": 128}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 128}}, "task": "arithmetic", "split": "id+ood", "seed": 0, "metrics": {"chance": 0.0136986301369863, "id": {"best_of_n_acc": 0.9765625, "majority_acc": 0.984375, "mean_terminal_energy": 1.8337225914001465}, "ood": {"best_of_n_acc": 0.0, "majority_acc": 0.0078125, "mean_terminal_energy": 1.7481226921081543}, "k_restarts": 4, "steps": 25, "final_train_loss": 0.6574077606201172}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "0bfced43260d", "timestamp": "2026-07-24T16:17:25.791375+00:00", "git_sha": "7369abb", "config_hash": "40bc3df0b42a", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "smoke"}, "model": {"latent_dim": 24, "hidden_dim": 64, "context_dim": 32}, "inference": {"k_restarts": 4, "steps": 25, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 12, "batch_size": 128, "lr": 0.002, "n_train": 3000, "n_neg": 2, "neg_noise": 0.5}, "eval": {"n_eval": 128}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 128}}, "task": "arithmetic", "split": "geometry-diagnostic", "seed": 0, "metrics": {"id": {"accuracy": 0.984375, "feature_auroc": {"basin/rho": 0.7261904761904762, "basin/entropy": 0.27380952380952384, "basin/dispersion": 0.2976190476190476, "energy/mean": 0.5515873015873016, "energy/min": 0.4603174603174603, "energy/std": 0.8095238095238095, "curv/lmax_mean": 0.3611111111111111, "curv/lmax_best": 0.5, "curv/trace_mean": 0.3253968253968254, "curv/trace_best": 0.32936507936507936, "dynamics/steps": 0.5, "dynamics/monotonic": 0.5694444444444444, "dynamics/drop": 0.4365079365079365, "dynamics/residual": 0.46825396825396826}, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"]}, "ood": {"accuracy": 0.0078125, "feature_auroc": {"basin/rho": 0.6259842519685039, "basin/entropy": 0.37401574803149606, "basin/dispersion": 0.952755905511811, "energy/mean": 0.33070866141732286, "energy/min": 0.33070866141732286, "energy/std": 0.3700787401574803, "curv/lmax_mean": 0.08661417322834646, "curv/lmax_best": 0.03937007874015748, "curv/trace_mean": 0.16535433070866143, "curv/trace_best": 0.031496062992125984, "dynamics/steps": 0.5, "dynamics/monotonic": 0.2283464566929134, "dynamics/drop": 0.023622047244094488, "dynamics/residual": 0.03937007874015748}, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"]}, "best_geometry_feature": "basin/rho", "best_geometry_sep": 0.22619047619047616, "best_energy_feature": "energy/std", "best_energy_sep": 0.30952380952380953, "k_restarts": 4, "steps": 25}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "f28af2a10ebe", "timestamp": "2026-07-24T16:17:45.486983+00:00", "git_sha": "7369abb", "config_hash": "40bc3df0b42a", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "smoke"}, "model": {"latent_dim": 24, "hidden_dim": 64, "context_dim": 32}, "inference": {"k_restarts": 4, "steps": 25, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 12, "batch_size": 128, "lr": 0.002, "n_train": 3000, "n_neg": 2, "neg_noise": 0.5}, "eval": {"n_eval": 128}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 128}}, "task": "arithmetic", "split": "id+ood", "seed": 0, "metrics": {"chance": 0.0136986301369863, "id": {"best_of_n_acc": 0.9765625, "majority_acc": 0.984375, "mean_terminal_energy": 1.8337225914001465}, "ood": {"best_of_n_acc": 0.0, "majority_acc": 0.0078125, "mean_terminal_energy": 1.7481226921081543}, "k_restarts": 4, "steps": 25, "final_train_loss": 0.6574077606201172}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "1bfe6b304e0f", "timestamp": "2026-07-24T18:52:49.992964+00:00", "git_sha": "7369abb", "config_hash": "53bd1f04ae90", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 9, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 1500}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 1500}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 1500, "n_calib": 1500, "n_test": 1500, "accuracy_id": 0.8953333333333333, "accuracy_ood": 0.21933333333333332, "base_error": 0.10466666666666667, "final_train_loss": 0.8188491463661194, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.029349183011227364, "rho_basin": 0.04703923516800908, "energy_min": 0.09284518792755422, "energy_mean": 0.09258513305715307, "energy_std": 0.09811602442052769}, "aurc_ood": {"geometry": 0.7904529574770697, "rho_basin": 0.734783754853945, "energy_min": 0.7454266622850384, "energy_mean": 0.7396153486239366, "energy_std": 0.7984967327940918}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.06349600491632686, 0.045868762847212426, 0.0820132697887756], "delta_aurc_vs_best_energy": [0.0632359500459257, 0.04732536012023457, 0.0812207368388719], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.005555555555555555, 0.009523809523809526, 0.008333333333333333, 0.011111111111111112, 0.010000000000000002, 0.00909090909090909, 0.013888888888888888, 0.015384615384615385, 0.016666666666666666, 0.017777777777777774, 0.016666666666666666, 0.01568627450980392, 0.014814814814814814, 0.015789473684210527, 0.018333333333333333, 0.019047619047619046, 0.01818181818181818, 0.01884057971014493, 0.01805555555555555, 0.02, 0.020512820512820513, 0.020987654320987655, 0.023809523809523808, 0.0264367816091954, 0.025555555555555557, 0.026881720430107527, 0.026041666666666668, 0.025252525252525252, 0.026470588235294117, 0.027619047619047616, 0.029629629629629627, 0.032432432432432434, 0.03684210526315789, 0.042735042735042736, 0.051666666666666666, 0.05934959349593495, 0.06269841269841282, 0.06666666666666667, 0.07121212121212121, 0.07481481481481482, 0.08115942028985507, 0.08581560283687942, 0.08958333333333332, 0.09319727891156462, 0.10466666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.038427167113494164, 0.03842716711349416, 0.03842716711349415, 0.03842716711349415, 0.03842716711349415, 0.03842716711349415, 0.03842716711349415, 0.03842716711349426, 0.038427167113494344, 0.03842716711349441, 0.03842716711349447, 0.03842716711349452, 0.03842716711349456, 0.03842716711349457, 0.03842716711349448, 0.038427167113494407, 0.038427167113494344, 0.03842716711349428, 0.038427167113494226, 0.03842716711349418, 0.038427167113494136, 0.038427167113494094, 0.03842716711349406, 0.038427167113494025, 0.03842716711349399, 0.03842716711349396, 0.03842716711349394, 0.03842716711349395, 0.038427167113494046, 0.038427167113494136, 0.038427167113494226, 0.0384271671134943, 0.03842716711349438, 0.038427167113494455, 0.03842716711349452, 0.03842716711349459, 0.03842716711349464, 0.04261239035087765, 0.048330662393162846, 0.05376302083333377, 0.058930386178862246, 0.06388888888888938, 0.06866428145497969, 0.07322261072261137, 0.07856630824372833, 0.08387096774193631, 0.08961960025789893, 0.09539930555555645, 0.1011054421768716, 0.10466666666666735]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.06666666666666667, 0.08333333333333333, 0.13333333333333333, 0.125, 0.10666666666666667, 0.09999999999999999, 0.08571428571428573, 0.07916666666666666, 0.08148148148148149, 0.07333333333333335, 0.08787878787878788, 0.08611111111111111, 0.08717948717948718, 0.09285714285714286, 0.0911111111111111, 0.0875, 0.08627450980392157, 0.08518518518518532, 0.08596491228070176, 0.085, 0.0857142857142857, 0.08484848484848485, 0.09130434782608696, 0.09444444444444443, 0.09066666666666667, 0.09615384615384616, 0.09876543209876543, 0.0988095238095238, 0.10229885057471264, 0.10333333333333333, 0.1043010752688172, 0.10104166666666667, 0.1, 0.09705882352941177, 0.09428571428571428, 0.0935185185185185, 0.0918918918918919, 0.0912280701754386, 0.09145299145299145, 0.08916666666666667, 0.09024390243902437, 0.09285714285714285, 0.09612403100775194, 0.09621212121212121, 0.09703703703703703, 0.0963768115942029, 0.09999999999999999, 0.09999999999999999, 0.10204081632653061, 0.10466666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.13333333333333333, 0.1, 0.08333333333333333, 0.09333333333333334, 0.09444444444444443, 0.09047619047619049, 0.09583333333333334, 0.09259259259259259, 0.09333333333333335, 0.09393939393939393, 0.08611111111111111, 0.08461538461538462, 0.08571428571428572, 0.08666666666666666, 0.08333333333333333, 0.08627450980392157, 0.08518518518518517, 0.08421052631578947, 0.09166666666666666, 0.09206349206349206, 0.1, 0.09855072463768116, 0.09999999999999999, 0.10266666666666667, 0.10512820512820513, 0.1037037037037037, 0.10595238095238095, 0.1057471264367816, 0.10333333333333333, 0.1053763440860215, 0.10208333333333333, 0.1, 0.09705882352941177, 0.09428571428571428, 0.09166666666666665, 0.09009009009009009, 0.09035087719298246, 0.08974358974358974, 0.09083333333333334, 0.09349593495934957, 0.09365079365079364, 0.09302325581395349, 0.09621212121212121, 0.09777777777777778, 0.09710144927536232, 0.09999999999999999, 0.09999999999999999, 0.10136054421768707, 0.10466666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.06666666666666667, 0.06666666666666667, 0.05555555555555555, 0.075, 0.08, 0.08888888888888888, 0.08571428571428573, 0.0875, 0.08888888888888889, 0.08666666666666668, 0.09696969696969697, 0.10555555555555556, 0.1, 0.10238095238095238, 0.09999999999999999, 0.10416666666666667, 0.1, 0.10185185185185183, 0.10175438596491228, 0.1, 0.09999999999999999, 0.10606060606060606, 0.10434782608695652, 0.10416666666666666, 0.10133333333333333, 0.10128205128205128, 0.1, 0.09761904761904762, 0.09885057471264384, 0.09888888888888889, 0.1043010752688172, 0.10625, 0.10606060606060606, 0.10490196078431373, 0.10476190476190475, 0.10370370370370369, 0.1045045045045045, 0.10701754385964912, 0.1076923076923077, 0.1075, 0.1065040650406504, 0.10714285714285712, 0.10775193798449613, 0.1053030303030303, 0.10666666666666667, 0.10434782608695652, 0.10425531914893615, 0.1048611111111111, 0.10408163265306122, 0.10466666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.36636651217454386, "coverage": 0.9066666666666666, "abstain_rate": 0.09333333333333334, "selective_risk": 0.07426470588235294, "selective_accuracy": 0.9257352941176471, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.03102189781021898, 0.07426470588235294, 0.10127431254191818, 0.10466666666666667, 0.10466666666666667], "coverage": [0.0, 0.7306666666666667, 0.9066666666666666, 0.994, 1.0, 1.0]}, "ece_geometry": 0.03426030734520334}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "c1557364f51d", "timestamp": "2026-07-24T18:54:27.142632+00:00", "git_sha": "7369abb", "config_hash": "cecbb6158c46", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 1500}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 1500}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 1500, "n_calib": 1500, "n_test": 1500, "accuracy_id": 0.81, "accuracy_ood": 0.20533333333333334, "base_error": 0.19, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0711432604851103, "rho_basin": 0.1192048662021191, "energy_min": 0.1447332231633481, "energy_mean": 0.14862598663924864, "energy_std": 0.18501378708598182}, "aurc_ood": {"geometry": 0.8035703588171388, "rho_basin": 0.7533412924431383, "energy_min": 0.7509755512657853, "energy_mean": 0.740760170602411, "energy_std": 0.7992759846101417}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "delta_aurc_vs_best_energy": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.05, 0.03333333333333333, 0.025, 0.02666666666666667, 0.033333333333333326, 0.052380952380952396, 0.05, 0.04814814814814815, 0.046666666666666676, 0.04242424242424243, 0.041666666666666664, 0.04358974358974359, 0.04047619047619048, 0.04444444444444444, 0.04375, 0.043137254901960784, 0.04444444444444444, 0.043859649122807015, 0.045, 0.04285714285714285, 0.04242424242424243, 0.04057971014492753, 0.04166666666666666, 0.044, 0.04358974358974359, 0.04938271604938271, 0.05238095238095238, 0.0586206896551724, 0.06444444444444444, 0.06451612903225806, 0.06458333333333334, 0.06666666666666667, 0.07058823529411765, 0.07714285714285712, 0.08148148148148163, 0.08558558558558559, 0.08947368421052632, 0.10170940170940171, 0.11, 0.11788617886178875, 0.12380952380952392, 0.12945736434108526, 0.13636363636363635, 0.14444444444444443, 0.1492753623188406, 0.15673758865248238, 0.16805555555555565, 0.1782312925170068, 0.19]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.10957642725598533, 0.10957642725598518, 0.10957642725598526, 0.1095764272559854, 0.10957642725598545, 0.10957642725598521, 0.10957642725598504, 0.10957642725598492, 0.10957642725598482, 0.10957642725598482, 0.10957642725598507, 0.10957642725598529, 0.10957642725598546, 0.10957642725598561, 0.10957642725598575, 0.10957642725598586, 0.10957642725598597, 0.10957642725598606, 0.10957642725598614, 0.1095764272559862, 0.10957642725598626, 0.10957642725598632, 0.10957642725598638, 0.10957642725598643, 0.10957642725598647, 0.10957642725598653, 0.10957642725598656, 0.1095764272559866, 0.10957642725598664, 0.10957642725598667, 0.1095764272559867, 0.10957642725598672, 0.10957642725598675, 0.10957642725598678, 0.10957642725598679, 0.10957642725598682, 0.11300813008130228, 0.11709456568250055, 0.12097144048363749, 0.12465447154471758, 0.12967479674796975, 0.1350907029478481, 0.14025470653377864, 0.14618844696969926, 0.15196759259259482, 0.15961352657005046, 0.1675650118203331, 0.17561111111111322, 0.18333333333333537, 0.19000000000000197]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.16666666666666666, 0.16666666666666666, 0.15833333333333333, 0.16, 0.15555555555555553, 0.14285714285714268, 0.1375, 0.14074074074074075, 0.1399999999999999, 0.13636363636363635, 0.1361111111111111, 0.13333333333333333, 0.13095238095238096, 0.12888888888888886, 0.13541666666666666, 0.13725490196078433, 0.13518518518518516, 0.13859649122807016, 0.14333333333333334, 0.14126984126984124, 0.1409090909090909, 0.14202898550724638, 0.1444444444444444, 0.14533333333333334, 0.14743589743589744, 0.14938271604938272, 0.15, 0.14712643678160917, 0.15, 0.14623655913978495, 0.14375, 0.1404040404040404, 0.13725490196078433, 0.13619047619047617, 0.13425925925925924, 0.13423423423423422, 0.13508771929824562, 0.13931623931623932, 0.1425, 0.14552845528455283, 0.14841269841269839, 0.15426356589147286, 0.15757575757575756, 0.16296296296296298, 0.16666666666666666, 0.17163120567375897, 0.17638888888888887, 0.18503401360544217, 0.19]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1, 0.15, 0.17777777777777778, 0.14166666666666666, 0.15333333333333332, 0.14999999999999997, 0.14761904761904765, 0.14166666666666666, 0.14814814814814814, 0.1466666666666667, 0.14545454545454545, 0.1388888888888889, 0.13333333333333333, 0.1357142857142857, 0.13999999999999999, 0.14375, 0.1411764705882353, 0.1444444444444444, 0.14385964912280702, 0.14333333333333334, 0.13968253968253966, 0.15, 0.15072463768115943, 0.15138888888888885, 0.14933333333333335, 0.15512820512820513, 0.15432098765432098, 0.15119047619047618, 0.14942528735632182, 0.14777777777777779, 0.14731182795698924, 0.14270833333333333, 0.1404040404040404, 0.13823529411764707, 0.1352380952380952, 0.13425925925925924, 0.13603603603603603, 0.13771929824561405, 0.14017094017094017, 0.14083333333333334, 0.1422764227642276, 0.146031746031746, 0.15193798449612403, 0.1590909090909091, 0.16296296296296298, 0.17028985507246377, 0.1730496453900709, 0.17638888888888898, 0.18435374149659864, 0.19]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.23333333333333334, 0.18333333333333332, 0.16666666666666666, 0.19166666666666668, 0.16666666666666666, 0.1611111111111111, 0.16666666666666669, 0.15833333333333333, 0.15925925925925927, 0.16666666666666669, 0.1696969696969697, 0.18611111111111112, 0.18461538461538463, 0.19523809523809524, 0.1911111111111111, 0.19166666666666668, 0.18627450980392157, 0.18888888888888886, 0.18947368421052632, 0.18833333333333332, 0.18888888888888886, 0.19242424242424241, 0.19130434782608696, 0.18888888888888886, 0.192, 0.18974358974358974, 0.18518518518518517, 0.18214285714285713, 0.18160919540229883, 0.17888888888888888, 0.18494623655913978, 0.18645833333333334, 0.18686868686868688, 0.18627450980392157, 0.1876190476190476, 0.18703703703703717, 0.1873873873873874, 0.18859649122807018, 0.18803418803418803, 0.1925, 0.19430894308943086, 0.19444444444444442, 0.1937984496124031, 0.19242424242424241, 0.18962962962962962, 0.18840579710144928, 0.18936170212765954, 0.19097222222222218, 0.19115646258503402, 0.19]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.19374220945381396, "coverage": 0.6933333333333334, "abstain_rate": 0.30666666666666664, "selective_risk": 0.075, "selective_accuracy": 0.925, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.075, 0.1295577967416602, 0.17525773195876287, 0.1879598662207358], "coverage": [0.0, 0.0, 0.6933333333333334, 0.8593333333333333, 0.97, 0.9966666666666667]}, "ece_geometry": 0.03153101157848922}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "acd437459961", "timestamp": "2026-07-25T10:21:25.796386+00:00", "git_sha": "758d5f2", "config_hash": "f314930a27aa", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 1, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 1, "accuracy_id": 0.7966666666666666, "base_error": 0.20333333333333334, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1567482825912684, "rho_basin": 0.2033333333333332, "energy_min": 0.1961369674772423, "energy_mean": 0.1961369674772423, "energy_std": 0.2033333333333332}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.039388684885973885, -0.005756146444602916, 0.08216092372404789], "delta_aurc_vs_best_energy": [0.039388684885973885, -0.005756146444602916, 0.08216092372404789], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.20833333333333334, 0.25, 0.22916666666666666, 0.2, 0.19444444444444442, 0.2023809523809524, 0.17708333333333334, 0.16666666666666666, 0.15000000000000002, 0.14393939393939395, 0.1388888888888889, 0.15384615384615385, 0.1488095238095238, 0.14999999999999997, 0.14583333333333334, 0.15196078431372548, 0.1481481481481481, 0.14912280701754385, 0.14583333333333334, 0.14285714285714282, 0.13636363636363635, 0.13768115942028986, 0.13888888888888887, 0.14, 0.13782051282051283, 0.1419753086419753, 0.13690476190476192, 0.1408045977011494, 0.14722222222222223, 0.14516129032258066, 0.15104166666666666, 0.14646464646464646, 0.14460784313725492, 0.1452380952380954, 0.1481481481481481, 0.1554054054054054, 0.15570175438596492, 0.1517094017094017, 0.15, 0.15447154471544713, 0.1587301587301587, 0.15891472868217055, 0.1590909090909091, 0.16666666666666666, 0.17572463768115942, 0.17907801418439726, 0.18229166666666663, 0.18877551020408162, 0.20333333333333334]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.20333333333333328, 0.20333333333333323, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.375, 0.2777777777777778, 0.20833333333333334, 0.18333333333333332, 0.16666666666666663, 0.1785714285714286, 0.16666666666666666, 0.2037037037037037, 0.20833333333333337, 0.19696969696969696, 0.19444444444444445, 0.1987179487179487, 0.19642857142857142, 0.19444444444444442, 0.1875, 0.18627450980392157, 0.19907407407407404, 0.19298245614035087, 0.18333333333333332, 0.1785714285714287, 0.17424242424242425, 0.17028985507246377, 0.17013888888888887, 0.16666666666666666, 0.16666666666666666, 0.1697530864197531, 0.16666666666666666, 0.16954022988505743, 0.17222222222222222, 0.1693548387096774, 0.16927083333333334, 0.17424242424242425, 0.17401960784313725, 0.1714285714285714, 0.17361111111111108, 0.17342342342342343, 0.17763157894736842, 0.1752136752136752, 0.17291666666666666, 0.17073170731707316, 0.1765873015873017, 0.17635658914728683, 0.18371212121212122, 0.18333333333333332, 0.18659420289855072, 0.19148936170212763, 0.1909722222222223, 0.1989795918367347, 0.20333333333333334]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.375, 0.2777777777777778, 0.20833333333333334, 0.18333333333333332, 0.16666666666666663, 0.1785714285714286, 0.16666666666666666, 0.2037037037037037, 0.20833333333333337, 0.19696969696969696, 0.19444444444444445, 0.1987179487179487, 0.19642857142857142, 0.19444444444444442, 0.1875, 0.18627450980392157, 0.19907407407407404, 0.19298245614035087, 0.18333333333333332, 0.1785714285714287, 0.17424242424242425, 0.17028985507246377, 0.17013888888888887, 0.16666666666666666, 0.16666666666666666, 0.1697530864197531, 0.16666666666666666, 0.16954022988505743, 0.17222222222222222, 0.1693548387096774, 0.16927083333333334, 0.17424242424242425, 0.17401960784313725, 0.1714285714285714, 0.17361111111111108, 0.17342342342342343, 0.17763157894736842, 0.1752136752136752, 0.17291666666666666, 0.17073170731707316, 0.1765873015873017, 0.17635658914728683, 0.18371212121212122, 0.18333333333333332, 0.18659420289855072, 0.19148936170212763, 0.1909722222222223, 0.1989795918367347, 0.20333333333333334]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.20333333333333328, 0.20333333333333323, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.2033333333333332, 0.2033333333333332, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317, 0.20333333333333317]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.1827768014059754, 0.20333333333333334], "coverage": [0.0, 0.0, 0.0, 0.0, 0.9483333333333334, 1.0]}, "ece_geometry": 0.03480073649618563}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "852e712d4157", "timestamp": "2026-07-25T10:21:28.092239+00:00", "git_sha": "758d5f2", "config_hash": "610a639808b6", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 1, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 1, "accuracy_id": 0.82, "base_error": 0.18, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.13241758466403533, "rho_basin": 0.18000000000000047, "energy_min": 0.17112847454878022, "energy_mean": 0.17112847454878022, "energy_std": 0.18000000000000047}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.03871088988474489, -0.0010208298806007605, 0.07896053842479216], "delta_aurc_vs_best_energy": [0.03871088988474489, -0.0010208298806007605, 0.07896053842479216], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.08333333333333333, 0.1, 0.11111111111111109, 0.10714285714285716, 0.10416666666666667, 0.12037037037037036, 0.15000000000000002, 0.1590909090909091, 0.1527777777777778, 0.14102564102564102, 0.1488095238095238, 0.14999999999999997, 0.140625, 0.13725490196078433, 0.13888888888888887, 0.13596491228070176, 0.13333333333333333, 0.13095238095238093, 0.12878787878787878, 0.13043478260869565, 0.12847222222222218, 0.13333333333333333, 0.13141025641025642, 0.12654320987654322, 0.12797619047619047, 0.132183908045977, 0.1361111111111111, 0.13709677419354838, 0.13541666666666666, 0.13383838383838384, 0.12990196078431374, 0.13095238095238093, 0.13425925925925924, 0.13288288288288289, 0.1337719298245614, 0.13247863247863248, 0.1375, 0.14430894308943087, 0.14087301587301598, 0.14534883720930233, 0.14583333333333334, 0.14629629629629629, 0.14855072463768115, 0.15070921985815613, 0.15624999999999997, 0.16666666666666666, 0.18]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.17999999999999997, 0.18000000000000005, 0.17999999999999994, 0.1799999999999999, 0.17999999999999985, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999974, 0.17999999999999983, 0.18000000000000005, 0.18000000000000027, 0.18000000000000047, 0.18000000000000066, 0.18000000000000083, 0.180000000000001, 0.18000000000000116, 0.1800000000000013, 0.18000000000000144, 0.18000000000000158, 0.18000000000000171, 0.18000000000000183, 0.18000000000000194, 0.18000000000000205, 0.18000000000000216, 0.18000000000000227, 0.18000000000000235, 0.18000000000000246, 0.18000000000000255, 0.18000000000000263]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.19444444444444445, 0.16666666666666666, 0.15, 0.13888888888888887, 0.130952380952381, 0.13541666666666666, 0.14814814814814814, 0.16666666666666669, 0.1590909090909091, 0.18055555555555555, 0.1858974358974359, 0.17261904761904762, 0.1722222222222222, 0.171875, 0.16666666666666666, 0.17129629629629628, 0.17543859649122806, 0.175, 0.1746031746031747, 0.17424242424242425, 0.17028985507246377, 0.17013888888888887, 0.17, 0.1762820512820513, 0.17592592592592593, 0.17261904761904762, 0.17241379310344826, 0.17222222222222222, 0.17473118279569894, 0.17447916666666666, 0.17676767676767677, 0.17401960784313725, 0.17142857142857157, 0.1759259259259259, 0.17567567567567569, 0.17763157894736842, 0.17735042735042736, 0.17708333333333334, 0.17886178861788615, 0.17857142857142855, 0.17635658914728683, 0.17803030303030304, 0.17777777777777778, 0.17753623188405798, 0.17907801418439714, 0.17881944444444453, 0.18197278911564627, 0.18]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.19444444444444445, 0.16666666666666666, 0.15, 0.13888888888888887, 0.130952380952381, 0.13541666666666666, 0.14814814814814814, 0.16666666666666669, 0.1590909090909091, 0.18055555555555555, 0.1858974358974359, 0.17261904761904762, 0.1722222222222222, 0.171875, 0.16666666666666666, 0.17129629629629628, 0.17543859649122806, 0.175, 0.1746031746031747, 0.17424242424242425, 0.17028985507246377, 0.17013888888888887, 0.17, 0.1762820512820513, 0.17592592592592593, 0.17261904761904762, 0.17241379310344826, 0.17222222222222222, 0.17473118279569894, 0.17447916666666666, 0.17676767676767677, 0.17401960784313725, 0.17142857142857157, 0.1759259259259259, 0.17567567567567569, 0.17763157894736842, 0.17735042735042736, 0.17708333333333334, 0.17886178861788615, 0.17857142857142855, 0.17635658914728683, 0.17803030303030304, 0.17777777777777778, 0.17753623188405798, 0.17907801418439714, 0.17881944444444453, 0.18197278911564627, 0.18]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.17999999999999997, 0.18000000000000005, 0.17999999999999994, 0.1799999999999999, 0.17999999999999985, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999974, 0.17999999999999983, 0.18000000000000005, 0.18000000000000027, 0.18000000000000047, 0.18000000000000066, 0.18000000000000083, 0.180000000000001, 0.18000000000000116, 0.1800000000000013, 0.18000000000000144, 0.18000000000000158, 0.18000000000000171, 0.18000000000000183, 0.18000000000000194, 0.18000000000000205, 0.18000000000000216, 0.18000000000000227, 0.18000000000000235, 0.18000000000000246, 0.18000000000000255, 0.18000000000000263]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.13043478260869565, 0.18], "coverage": [0.0, 0.0, 0.0, 0.0, 0.69, 1.0]}, "ece_geometry": 0.034397943552624266}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "8b6b1348cf28", "timestamp": "2026-07-25T10:21:30.244910+00:00", "git_sha": "758d5f2", "config_hash": "cf02c245cbac", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 1, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 1, "accuracy_id": 0.875, "base_error": 0.125, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.044287672839589405, "rho_basin": 0.125, "energy_min": 0.04758207732870357, "energy_mean": 0.04758207732870357, "energy_std": 0.125}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.0032944044891141666, -0.006697394449172673, 0.012367838068729414], "delta_aurc_vs_best_energy": [0.0032944044891141666, -0.006697394449172673, 0.012367838068729414], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.013888888888888886, 0.011904761904761906, 0.010416666666666666, 0.027777777777777776, 0.025000000000000005, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.02976190476190476, 0.03888888888888888, 0.036458333333333336, 0.03431372549019608, 0.0324074074074074, 0.03070175438596491, 0.0375, 0.035714285714285705, 0.041666666666666664, 0.03985507246376811, 0.03819444444444444, 0.043333333333333335, 0.041666666666666664, 0.04938271604938271, 0.047619047619047616, 0.048850574712643674, 0.04722222222222222, 0.0456989247311828, 0.046875, 0.047979797979797977, 0.051470588235294115, 0.054761904761904755, 0.053240740740740734, 0.05630630630630631, 0.05701754385964912, 0.057692307692307696, 0.0625, 0.0630081300813008, 0.06944444444444443, 0.07170542635658915, 0.07954545454545454, 0.09074074074074075, 0.09782608695652174, 0.09929078014184396, 0.10590277777777787, 0.11734693877551021, 0.125]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.01666666666666667, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.017857142857142856, 0.027777777777777773, 0.026041666666666668, 0.029411764705882353, 0.027777777777777773, 0.02631578947368421, 0.029166666666666667, 0.03174603174603174, 0.03787878787878788, 0.03985507246376811, 0.04513888888888888, 0.05, 0.05128205128205128, 0.05246913580246913, 0.05654761904761905, 0.05747126436781608, 0.058333333333333334, 0.06182795698924731, 0.0625, 0.06313131313131314, 0.06372549019607843, 0.06666666666666665, 0.06944444444444443, 0.06981981981981981, 0.07017543859649122, 0.07051282051282051, 0.07291666666666667, 0.07520325203252046, 0.07738095238095251, 0.0872093023255814, 0.0946969696969697, 0.0962962962962963, 0.10144927536231885, 0.10283687943262422, 0.10937499999999999, 0.12074829931972789, 0.125]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.01666666666666667, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.017857142857142856, 0.027777777777777773, 0.026041666666666668, 0.029411764705882353, 0.027777777777777773, 0.02631578947368421, 0.029166666666666667, 0.03174603174603174, 0.03787878787878788, 0.03985507246376811, 0.04513888888888888, 0.05, 0.05128205128205128, 0.05246913580246913, 0.05654761904761905, 0.05747126436781608, 0.058333333333333334, 0.06182795698924731, 0.0625, 0.06313131313131314, 0.06372549019607843, 0.06666666666666665, 0.06944444444444443, 0.06981981981981981, 0.07017543859649122, 0.07051282051282051, 0.07291666666666667, 0.07520325203252046, 0.07738095238095251, 0.0872093023255814, 0.0946969696969697, 0.0962962962962963, 0.10144927536231885, 0.10283687943262422, 0.10937499999999999, 0.12074829931972789, 0.125]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125, 0.125]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.21075083755725627, "coverage": 0.79, "abstain_rate": 0.21, "selective_risk": 0.05907172995780591, "selective_accuracy": 0.9409282700421941, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.05907172995780591, 0.10314685314685315, 0.12520868113522537, 0.12520868113522537], "coverage": [0.0, 0.0, 0.79, 0.9533333333333334, 0.9983333333333333, 0.9983333333333333]}, "ece_geometry": 0.04247528944153519}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "56ea729e5ea5", "timestamp": "2026-07-25T10:21:32.359580+00:00", "git_sha": "758d5f2", "config_hash": "a56243cda69f", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 1, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 1, "accuracy_id": 0.82, "base_error": 0.18, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1415301620625212, "rho_basin": 0.18000000000000047, "energy_min": 0.20633979176428102, "energy_mean": 0.20633979176428102, "energy_std": 0.18000000000000047}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.06480962970175982, 0.021114517500458844, 0.11090394539211393], "delta_aurc_vs_best_energy": [0.03846983793747927, 0.008910879097292781, 0.06699235273946363], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1388888888888889, 0.10416666666666667, 0.1, 0.11111111111111109, 0.1309523809523808, 0.11458333333333333, 0.1111111111111111, 0.13333333333333336, 0.14393939393939395, 0.14583333333333334, 0.14102564102564102, 0.15476190476190477, 0.1444444444444444, 0.15104166666666666, 0.15196078431372548, 0.1481481481481481, 0.14912280701754385, 0.14166666666666666, 0.13888888888888887, 0.14393939393939395, 0.13768115942028986, 0.13541666666666663, 0.14, 0.13782051282051283, 0.13271604938271606, 0.13095238095238096, 0.13218390804597718, 0.1388888888888889, 0.13709677419354838, 0.13541666666666666, 0.13383838383838384, 0.13480392156862744, 0.1333333333333333, 0.13425925925925924, 0.14189189189189189, 0.14035087719298245, 0.14316239316239315, 0.14166666666666666, 0.14024390243902438, 0.14087301587301584, 0.1434108527131783, 0.14393939393939395, 0.14629629629629629, 0.1503623188405797, 0.15780141843971643, 0.16319444444444453, 0.17006802721088435, 0.18]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.17999999999999997, 0.18000000000000005, 0.17999999999999994, 0.1799999999999999, 0.17999999999999985, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999974, 0.17999999999999983, 0.18000000000000005, 0.18000000000000027, 0.18000000000000047, 0.18000000000000066, 0.18000000000000083, 0.180000000000001, 0.18000000000000116, 0.1800000000000013, 0.18000000000000144, 0.18000000000000158, 0.18000000000000171, 0.18000000000000183, 0.18000000000000194, 0.18000000000000205, 0.18000000000000216, 0.18000000000000227, 0.18000000000000235, 0.18000000000000246, 0.18000000000000255, 0.18000000000000263]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4583333333333333, 0.3888888888888889, 0.375, 0.31666666666666665, 0.34722222222222215, 0.3095238095238096, 0.2708333333333333, 0.25, 0.25000000000000006, 0.23484848484848486, 0.2222222222222222, 0.21794871794871795, 0.21428571428571427, 0.21111111111111108, 0.19791666666666666, 0.19607843137254902, 0.19907407407407404, 0.19298245614035087, 0.18333333333333332, 0.17857142857142855, 0.18181818181818182, 0.18115942028985507, 0.18402777777777776, 0.18, 0.1762820512820513, 0.17592592592592593, 0.17261904761904762, 0.17528735632183906, 0.175, 0.17204301075268819, 0.16666666666666666, 0.16161616161616163, 0.15931372549019607, 0.15952380952380948, 0.15509259259259256, 0.15315315315315314, 0.1513157894736842, 0.15384615384615385, 0.15833333333333333, 0.15447154471544713, 0.1587301587301587, 0.15891472868217055, 0.1553030303030303, 0.15555555555555556, 0.15760869565217392, 0.1631205673758865, 0.16840277777777787, 0.17687074829931973, 0.18]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4583333333333333, 0.3888888888888889, 0.375, 0.31666666666666665, 0.34722222222222215, 0.3095238095238096, 0.2708333333333333, 0.25, 0.25000000000000006, 0.23484848484848486, 0.2222222222222222, 0.21794871794871795, 0.21428571428571427, 0.21111111111111108, 0.19791666666666666, 0.19607843137254902, 0.19907407407407404, 0.19298245614035087, 0.18333333333333332, 0.17857142857142855, 0.18181818181818182, 0.18115942028985507, 0.18402777777777776, 0.18, 0.1762820512820513, 0.17592592592592593, 0.17261904761904762, 0.17528735632183906, 0.175, 0.17204301075268819, 0.16666666666666666, 0.16161616161616163, 0.15931372549019607, 0.15952380952380948, 0.15509259259259256, 0.15315315315315314, 0.1513157894736842, 0.15384615384615385, 0.15833333333333333, 0.15447154471544713, 0.1587301587301587, 0.15891472868217055, 0.1553030303030303, 0.15555555555555556, 0.15760869565217392, 0.1631205673758865, 0.16840277777777787, 0.17687074829931973, 0.18]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.17999999999999997, 0.18000000000000005, 0.17999999999999994, 0.1799999999999999, 0.17999999999999985, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.17999999999999983, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.1799999999999998, 0.1799999999999998, 0.1799999999999998, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999977, 0.17999999999999974, 0.17999999999999983, 0.18000000000000005, 0.18000000000000027, 0.18000000000000047, 0.18000000000000066, 0.18000000000000083, 0.180000000000001, 0.18000000000000116, 0.1800000000000013, 0.18000000000000144, 0.18000000000000158, 0.18000000000000171, 0.18000000000000183, 0.18000000000000194, 0.18000000000000205, 0.18000000000000216, 0.18000000000000227, 0.18000000000000235, 0.18000000000000246, 0.18000000000000255, 0.18000000000000263]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.16551724137931034, 0.17863105175292154], "coverage": [0.0, 0.0, 0.0, 0.0, 0.9666666666666667, 0.9983333333333333]}, "ece_geometry": 0.023752087800975603}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "8b8e1e4a7381", "timestamp": "2026-07-25T10:21:34.506069+00:00", "git_sha": "758d5f2", "config_hash": "e923fb1d771e", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 1, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 1, "accuracy_id": 0.8166666666666667, "base_error": 0.18333333333333332, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.13905694499810536, "rho_basin": 0.18333333333333302, "energy_min": 0.14999526980020578, "energy_mean": 0.14999526980020578, "energy_std": 0.18333333333333302}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.010938324802100413, -0.03025416556025576, 0.05112144758249504], "delta_aurc_vs_best_energy": [0.010938324802100413, -0.03025416556025576, 0.05112144758249504], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.1111111111111111, 0.10416666666666667, 0.11666666666666667, 0.13888888888888887, 0.1666666666666665, 0.14583333333333334, 0.1574074074074074, 0.15000000000000002, 0.15151515151515152, 0.14583333333333334, 0.14102564102564102, 0.14285714285714285, 0.13888888888888887, 0.13541666666666666, 0.13725490196078433, 0.1296296296296296, 0.12719298245614036, 0.12916666666666668, 0.12698412698412695, 0.125, 0.12681159420289856, 0.13194444444444442, 0.13, 0.1282051282051282, 0.12654320987654322, 0.12797619047619047, 0.1293103448275862, 0.13055555555555556, 0.13172043010752688, 0.1328125, 0.1388888888888889, 0.13970588235294118, 0.14047619047619062, 0.14120370370370383, 0.14414414414414414, 0.14035087719298245, 0.1388888888888889, 0.13958333333333334, 0.1463414634146341, 0.14285714285714282, 0.1434108527131783, 0.14962121212121213, 0.15, 0.1539855072463768, 0.1578014184397163, 0.16319444444444442, 0.17006802721088435, 0.18333333333333332]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.18333333333333332, 0.18333333333333326, 0.18333333333333338, 0.18333333333333343, 0.18333333333333346, 0.18333333333333346, 0.1833333333333335, 0.1833333333333335, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.1833333333333334, 0.1833333333333332, 0.18333333333333302, 0.18333333333333285, 0.1833333333333327, 0.18333333333333257, 0.18333333333333243, 0.18333333333333232, 0.18333333333333224, 0.18333333333333213, 0.18333333333333204, 0.183333333333332, 0.1833333333333319, 0.18333333333333185, 0.18333333333333177, 0.1833333333333319, 0.18333333333333207, 0.18333333333333224, 0.18333333333333238, 0.18333333333333252, 0.18333333333333265, 0.1833333333333328, 0.1833333333333329, 0.18333333333333302, 0.18333333333333313, 0.1833333333333332, 0.18333333333333332, 0.1833333333333334, 0.1833333333333335, 0.18333333333333357, 0.18333333333333365, 0.18333333333333374, 0.1833333333333338, 0.18333333333333388, 0.18333333333333393, 0.183333333333334]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.1388888888888889, 0.125, 0.1, 0.08333333333333331, 0.08333333333333334, 0.10416666666666667, 0.1111111111111111, 0.13333333333333336, 0.14393939393939395, 0.1736111111111111, 0.16025641025641027, 0.15476190476190477, 0.1611111111111111, 0.15104166666666666, 0.15196078431372548, 0.15740740740740738, 0.16228070175438597, 0.15416666666666667, 0.15476190476190474, 0.15151515151515152, 0.15217391304347827, 0.1527777777777779, 0.15333333333333332, 0.15705128205128205, 0.1574074074074074, 0.15773809523809523, 0.15517241379310362, 0.15833333333333333, 0.15591397849462366, 0.15885416666666666, 0.1590909090909091, 0.1568627450980392, 0.15952380952380948, 0.16435185185185183, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16869918699187003, 0.1706349206349206, 0.17248062015503876, 0.17424242424242425, 0.17592592592592593, 0.17572463768115942, 0.17730496453900707, 0.17708333333333331, 0.17857142857142858, 0.18333333333333332]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.1388888888888889, 0.125, 0.1, 0.08333333333333331, 0.08333333333333334, 0.10416666666666667, 0.1111111111111111, 0.13333333333333336, 0.14393939393939395, 0.1736111111111111, 0.16025641025641027, 0.15476190476190477, 0.1611111111111111, 0.15104166666666666, 0.15196078431372548, 0.15740740740740738, 0.16228070175438597, 0.15416666666666667, 0.15476190476190474, 0.15151515151515152, 0.15217391304347827, 0.1527777777777779, 0.15333333333333332, 0.15705128205128205, 0.1574074074074074, 0.15773809523809523, 0.15517241379310362, 0.15833333333333333, 0.15591397849462366, 0.15885416666666666, 0.1590909090909091, 0.1568627450980392, 0.15952380952380948, 0.16435185185185183, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16869918699187003, 0.1706349206349206, 0.17248062015503876, 0.17424242424242425, 0.17592592592592593, 0.17572463768115942, 0.17730496453900707, 0.17708333333333331, 0.17857142857142858, 0.18333333333333332]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.18333333333333332, 0.18333333333333326, 0.18333333333333338, 0.18333333333333343, 0.18333333333333346, 0.18333333333333346, 0.1833333333333335, 0.1833333333333335, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.18333333333333351, 0.1833333333333334, 0.1833333333333332, 0.18333333333333302, 0.18333333333333285, 0.1833333333333327, 0.18333333333333257, 0.18333333333333243, 0.18333333333333232, 0.18333333333333224, 0.18333333333333213, 0.18333333333333204, 0.183333333333332, 0.1833333333333319, 0.18333333333333185, 0.18333333333333177, 0.1833333333333319, 0.18333333333333207, 0.18333333333333224, 0.18333333333333238, 0.18333333333333252, 0.18333333333333265, 0.1833333333333328, 0.1833333333333329, 0.18333333333333302, 0.18333333333333313, 0.1833333333333332, 0.18333333333333332, 0.1833333333333334, 0.1833333333333335, 0.18333333333333357, 0.18333333333333365, 0.18333333333333374, 0.1833333333333338, 0.18333333333333388, 0.18333333333333393, 0.183333333333334]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.15714285714285714, 0.17922948073701842], "coverage": [0.0, 0.0, 0.0, 0.0, 0.9333333333333333, 0.995]}, "ece_geometry": 0.02159338122747919}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "a4170c27c22b", "timestamp": "2026-07-25T10:21:41.099131+00:00", "git_sha": "758d5f2", "config_hash": "b1a0a6873eb1", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 2, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 2, "accuracy_id": 0.8133333333333334, "base_error": 0.18666666666666668, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.11981182321914574, "rho_basin": 0.15834255057834776, "energy_min": 0.1806686674801733, "energy_mean": 0.1706909394127385, "energy_std": 0.18378335158682801}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.060856844261027554, 0.01808505456984964, 0.10668108310296104], "delta_aurc_vs_best_energy": [0.050879116193592755, 0.01021438574480109, 0.09586020861775758], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.08333333333333333, 0.10416666666666667, 0.11666666666666667, 0.11111111111111109, 0.10714285714285716, 0.10416666666666667, 0.1111111111111111, 0.11666666666666668, 0.11363636363636363, 0.1111111111111111, 0.10897435897435898, 0.10714285714285714, 0.11666666666666664, 0.109375, 0.10294117647058823, 0.09722222222222221, 0.10087719298245613, 0.09583333333333334, 0.09920634920634919, 0.10606060606060606, 0.10507246376811594, 0.10069444444444443, 0.10333333333333333, 0.10256410256410256, 0.10185185185185185, 0.09821428571428571, 0.10057471264367815, 0.10277777777777777, 0.10215053763440861, 0.10416666666666667, 0.10858585858585859, 0.11029411764705882, 0.11428571428571427, 0.11342592592592592, 0.11036036036036036, 0.1118421052631579, 0.11538461538461539, 0.11875, 0.128048780487805, 0.13095238095238107, 0.1434108527131783, 0.15151515151515152, 0.15555555555555556, 0.15579710144927536, 0.16666666666666663, 0.17013888888888898, 0.17857142857142858, 0.18666666666666668]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.15682656826568267, 0.15682656826568267, 0.15682656826568278, 0.15682656826568286, 0.15682656826568278, 0.1568265682656827, 0.15682656826568261, 0.15682656826568256, 0.15682656826568253, 0.1568265682656825, 0.15682656826568248, 0.15682656826568245, 0.15682656826568242, 0.15682656826568242, 0.1568265682656824, 0.1568265682656824, 0.15682656826568236, 0.15682656826568256, 0.15682656826568273, 0.1568265682656829, 0.156826568265683, 0.15682656826568314, 0.15682656826568325, 0.15682656826568336, 0.15682656826568345, 0.15682656826568353, 0.1568265682656836, 0.1568265682656837, 0.15682656826568378, 0.15682656826568384, 0.1568265682656839, 0.15682656826568395, 0.156826568265684, 0.15682656826568406, 0.15682656826568409, 0.15682656826568414, 0.15682656826568417, 0.15682656826568422, 0.15682656826568425, 0.1568265682656843, 0.15682656826568434, 0.15682656826568436, 0.1568265682656844, 0.15682656826568442, 0.15682656826568447, 0.16241879060469955, 0.16886769381267014, 0.17504789272030863, 0.18097583861130875, 0.18666666666666892]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.2916666666666667, 0.25, 0.25, 0.21666666666666667, 0.19444444444444442, 0.21428571428571433, 0.19791666666666666, 0.19444444444444445, 0.1916666666666667, 0.17424242424242425, 0.1597222222222222, 0.17307692307692307, 0.17857142857142858, 0.1722222222222222, 0.171875, 0.16176470588235295, 0.17129629629629628, 0.16228070175438597, 0.1625, 0.16666666666666663, 0.17803030303030304, 0.17391304347826086, 0.17361111111111108, 0.17666666666666667, 0.17307692307692307, 0.1697530864197531, 0.16666666666666666, 0.16666666666666663, 0.16111111111111112, 0.16129032258064516, 0.1640625, 0.16161616161616163, 0.16176470588235295, 0.16428571428571426, 0.16666666666666663, 0.16666666666666666, 0.17105263157894737, 0.16666666666666666, 0.16666666666666666, 0.1646341463414634, 0.1726190476190476, 0.17054263565891473, 0.16856060606060605, 0.17037037037037037, 0.1721014492753623, 0.175531914893617, 0.17534722222222218, 0.17857142857142858, 0.18666666666666668]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.2222222222222222, 0.1875, 0.18333333333333332, 0.20833333333333331, 0.2023809523809524, 0.17708333333333334, 0.17592592592592593, 0.17500000000000002, 0.17424242424242425, 0.1875, 0.1794871794871795, 0.18452380952380953, 0.18888888888888886, 0.18229166666666666, 0.17647058823529413, 0.1759259259259259, 0.16666666666666666, 0.16666666666666666, 0.16269841269841268, 0.1590909090909091, 0.16666666666666666, 0.16666666666666663, 0.16333333333333333, 0.16346153846153846, 0.16666666666666666, 0.16071428571428573, 0.16091954022988503, 0.16666666666666666, 0.16666666666666666, 0.16145833333333334, 0.1590909090909091, 0.1642156862745098, 0.16428571428571426, 0.16435185185185183, 0.16441441441441443, 0.16666666666666666, 0.16452991452991453, 0.1625, 0.16463414634146353, 0.16865079365079363, 0.17248062015503876, 0.17234848484848486, 0.17037037037037037, 0.1721014492753623, 0.17375886524822692, 0.17534722222222218, 0.17857142857142858, 0.18666666666666668]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1388888888888889, 0.16666666666666666, 0.23333333333333334, 0.2222222222222223, 0.22619047619047603, 0.20833333333333334, 0.2037037037037037, 0.2166666666666667, 0.21212121212121213, 0.20833333333333334, 0.21153846153846154, 0.20238095238095238, 0.19999999999999998, 0.19270833333333334, 0.18627450980392157, 0.1898148148148148, 0.18421052631578946, 0.1875, 0.18650793650793648, 0.1856060606060606, 0.17753623188405798, 0.17361111111111108, 0.17333333333333334, 0.17307692307692307, 0.16666666666666666, 0.16964285714285715, 0.16954022988505743, 0.175, 0.1693548387096774, 0.16666666666666666, 0.1691919191919192, 0.16666666666666666, 0.16904761904761903, 0.17129629629629628, 0.17342342342342343, 0.17324561403508773, 0.1752136752136752, 0.17708333333333334, 0.17886178861788615, 0.17658730158730157, 0.17829457364341086, 0.18371212121212122, 0.1814814814814815, 0.18297101449275363, 0.18439716312056745, 0.18576388888888887, 0.18877551020408162, 0.18666666666666668]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.13214990138067062, 0.16784452296819788, 0.18697829716193656], "coverage": [0.0, 0.0, 0.0, 0.845, 0.9433333333333334, 0.9983333333333333]}, "ece_geometry": 0.049253000064196834}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "af69f3e49ad2", "timestamp": "2026-07-25T10:21:43.324500+00:00", "git_sha": "758d5f2", "config_hash": "1632b77f4493", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 2, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 2, "accuracy_id": 0.8033333333333333, "base_error": 0.19666666666666666, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1517691987779376, "rho_basin": 0.1575100576244524, "energy_min": 0.21197157825580276, "energy_mean": 0.20754691788381563, "energy_std": 0.17277649227597028}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.060202379477865164, 0.011637042166519484, 0.1065981913736233], "delta_aurc_vs_best_energy": [0.02100729349803268, -0.021513153201701787, 0.06282693863399035], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1111111111111111, 0.10416666666666667, 0.16666666666666666, 0.16666666666666674, 0.1785714285714284, 0.15625, 0.1388888888888889, 0.15833333333333335, 0.15151515151515152, 0.1527777777777778, 0.15384615384615385, 0.1488095238095238, 0.14999999999999997, 0.14583333333333334, 0.1568627450980392, 0.162037037037037, 0.16228070175438597, 0.15833333333333333, 0.15476190476190474, 0.1590909090909091, 0.16304347826086957, 0.15972222222222218, 0.15666666666666668, 0.15384615384615385, 0.15123456790123457, 0.15178571428571427, 0.15229885057471262, 0.14722222222222223, 0.1478494623655914, 0.14322916666666666, 0.14393939393939395, 0.14460784313725492, 0.14523809523809522, 0.1435185185185185, 0.13963963963963963, 0.13596491228070176, 0.1346153846153846, 0.13333333333333333, 0.13414634146341461, 0.13492063492063489, 0.13565891472868216, 0.14015151515151514, 0.1537037037037037, 0.15942028985507245, 0.17198581560283685, 0.17708333333333343, 0.18537414965986396, 0.19666666666666666]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.15514018691588788, 0.1551401869158879, 0.1551401869158878, 0.1551401869158877, 0.1551401869158878, 0.15514018691588788, 0.15514018691588796, 0.15514018691588802, 0.155140186915888, 0.15514018691588785, 0.15514018691588774, 0.15514018691588763, 0.15514018691588755, 0.1551401869158875, 0.15514018691588743, 0.15514018691588738, 0.15514018691588732, 0.1551401869158873, 0.15514018691588727, 0.1551401869158872, 0.15514018691588718, 0.15514018691588716, 0.15514018691588713, 0.15514018691588713, 0.1551401869158871, 0.15514018691588707, 0.15514018691588707, 0.15514018691588705, 0.15514018691588702, 0.15514018691588702, 0.155140186915887, 0.155140186915887, 0.155140186915887, 0.15514018691588696, 0.15514018691588696, 0.15514018691588696, 0.15514018691588696, 0.15514018691588694, 0.15514018691588694, 0.15514018691588694, 0.1551401869158869, 0.1551401869158869, 0.1551401869158869, 0.1551401869158869, 0.1586894586894577, 0.1669453734671115, 0.174849972722312, 0.18242521367521242, 0.18969126111983117, 0.1966666666666652]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.20833333333333334, 0.2777777777777778, 0.25, 0.21666666666666667, 0.20833333333333331, 0.22619047619047625, 0.21875, 0.24074074074074073, 0.24166666666666656, 0.25, 0.24305555555555555, 0.24358974358974358, 0.23809523809523808, 0.22777777777777772, 0.21875, 0.22549019607843138, 0.22685185185185183, 0.22807017543859648, 0.22083333333333333, 0.21825396825396823, 0.22348484848484848, 0.213768115942029, 0.21874999999999997, 0.21666666666666667, 0.20833333333333334, 0.20679012345679013, 0.20238095238095238, 0.1982758620689655, 0.19444444444444445, 0.19086021505376344, 0.19010416666666666, 0.19444444444444445, 0.20098039215686275, 0.20714285714285713, 0.2060185185185185, 0.20270270270270271, 0.19956140350877194, 0.1987179487179487, 0.2, 0.1971544715447154, 0.2023809523809525, 0.20155038759689922, 0.20265151515151514, 0.20185185185185187, 0.20108695652173914, 0.19858156028368792, 0.19965277777777787, 0.1989795918367347, 0.19666666666666666]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.20833333333333334, 0.25, 0.22916666666666666, 0.23333333333333334, 0.23611111111111108, 0.23809523809523814, 0.22916666666666666, 0.24074074074074073, 0.25000000000000006, 0.24242424242424243, 0.2361111111111111, 0.23076923076923078, 0.23214285714285715, 0.21666666666666665, 0.203125, 0.19607843137254902, 0.19907407407407404, 0.19298245614035087, 0.1875, 0.19047619047619044, 0.19696969696969696, 0.19927536231884058, 0.19791666666666663, 0.19666666666666666, 0.20192307692307693, 0.2006172839506173, 0.19642857142857142, 0.2011494252873563, 0.2, 0.1989247311827957, 0.19791666666666666, 0.19444444444444445, 0.19852941176470587, 0.19761904761904758, 0.19444444444444442, 0.19369369369369369, 0.19078947368421054, 0.19017094017094016, 0.19375, 0.1930894308943089, 0.19047619047619044, 0.187984496124031, 0.19318181818181818, 0.19444444444444445, 0.1956521739130435, 0.19680851063829785, 0.19444444444444442, 0.19727891156462585, 0.19666666666666666]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.16666666666666666, 0.1875, 0.15, 0.13888888888888887, 0.1547619047619046, 0.14583333333333334, 0.17592592592592593, 0.18333333333333335, 0.17424242424242425, 0.1597222222222222, 0.14743589743589744, 0.1488095238095238, 0.1444444444444444, 0.14583333333333334, 0.15196078431372548, 0.15740740740740738, 0.16228070175438597, 0.1625, 0.15476190476190474, 0.16287878787878787, 0.16304347826086957, 0.16319444444444442, 0.16333333333333333, 0.17307692307692307, 0.1697530864197531, 0.16964285714285715, 0.16379310344827583, 0.1638888888888889, 0.16666666666666666, 0.171875, 0.17424242424242425, 0.17892156862745098, 0.1809523809523811, 0.18055555555555552, 0.18018018018018017, 0.17982456140350878, 0.18376068376068377, 0.18333333333333332, 0.18699186991869915, 0.19246031746031758, 0.19573643410852712, 0.19128787878787878, 0.1962962962962963, 0.1956521739130435, 0.19503546099290778, 0.19965277777777776, 0.1989795918367347, 0.19666666666666666]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.13714285714285715, 0.19532554257095158], "coverage": [0.0, 0.0, 0.0, 0.0, 0.875, 0.9983333333333333]}, "ece_geometry": 0.06150903179591116}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "25320822fdfd", "timestamp": "2026-07-25T10:21:45.541118+00:00", "git_sha": "758d5f2", "config_hash": "17b96f1b0f17", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 2, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 2, "accuracy_id": 0.8683333333333333, "base_error": 0.13166666666666665, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.041234383694194514, "rho_basin": 0.09152749865095022, "energy_min": 0.05154744798322308, "energy_mean": 0.0502452392511024, "energy_std": 0.13454142425492768}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.010313064289028569, -0.0013514248754594867, 0.02325728838136145], "delta_aurc_vs_best_energy": [0.009010855556907886, -0.0022455955790842757, 0.021281381689459384], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.027777777777777773, 0.02380952380952381, 0.020833333333333332, 0.018518518518518517, 0.025000000000000005, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.023809523809523808, 0.02222222222222222, 0.020833333333333332, 0.0196078431372549, 0.018518518518518514, 0.017543859649122806, 0.025, 0.027777777777777773, 0.026515151515151516, 0.028985507246376812, 0.027777777777777773, 0.03, 0.028846153846153848, 0.033950617283950615, 0.03273809523809524, 0.03448275862068965, 0.03333333333333333, 0.03225806451612903, 0.03125, 0.03535353535353535, 0.03676470588235294, 0.04047619047619047, 0.043981481481481476, 0.04279279279279279, 0.04824561403508772, 0.05128205128205128, 0.05625, 0.060975609756097685, 0.07142857142857155, 0.07751937984496124, 0.08143939393939394, 0.08888888888888889, 0.09963768115942029, 0.10460992907801417, 0.1111111111111112, 0.11904761904761904, 0.13166666666666665]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08921933085501856, 0.0892193308550186, 0.08921933085501854, 0.08921933085501854, 0.08921933085501861, 0.08921933085501865, 0.08921933085501868, 0.08921933085501865, 0.08921933085501858, 0.08921933085501851, 0.08921933085501847, 0.08921933085501843, 0.0892193308550184, 0.08921933085501836, 0.08921933085501835, 0.08921933085501843, 0.08921933085501851, 0.08921933085501858, 0.08921933085501864, 0.0892193308550187, 0.08921933085501875, 0.08921933085501879, 0.08921933085501883, 0.08921933085501887, 0.08921933085501892, 0.08921933085501894, 0.08921933085501897, 0.089219330855019, 0.08921933085501903, 0.08921933085501903, 0.08921933085501894, 0.08921933085501886, 0.08921933085501878, 0.0892193308550187, 0.08921933085501862, 0.08921933085501856, 0.0892193308550185, 0.08921933085501844, 0.08921933085501838, 0.08921933085501833, 0.08921933085501828, 0.08921933085501822, 0.08921933085501818, 0.08921933085501814, 0.09074074074074025, 0.09963768115941982, 0.1081560283687939, 0.11631944444444405, 0.12414965986394515, 0.13166666666666624]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.037037037037037035, 0.03333333333333334, 0.03787878787878788, 0.034722222222222224, 0.03205128205128205, 0.03571428571428571, 0.03888888888888888, 0.036458333333333336, 0.03431372549019608, 0.03703703703703703, 0.039473684210526314, 0.04583333333333333, 0.04365079365079364, 0.045454545454545456, 0.04710144927536232, 0.048611111111111105, 0.05, 0.04807692307692308, 0.04938271604938271, 0.050595238095238096, 0.051724137931034475, 0.05, 0.04838709677419355, 0.052083333333333336, 0.05555555555555555, 0.05392156862745098, 0.054761904761904755, 0.055555555555555546, 0.060810810810810814, 0.06798245614035088, 0.06623931623931624, 0.06666666666666667, 0.06910569105691056, 0.07341269841269854, 0.08333333333333333, 0.08901515151515152, 0.09444444444444444, 0.10507246376811594, 0.11170212765957445, 0.11979166666666666, 0.12244897959183673, 0.13166666666666665]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.027777777777777773, 0.02380952380952381, 0.020833333333333332, 0.018518518518518517, 0.025000000000000005, 0.03787878787878788, 0.034722222222222224, 0.03205128205128205, 0.03571428571428571, 0.033333333333333326, 0.03125, 0.029411764705882353, 0.0324074074074074, 0.03508771929824561, 0.041666666666666664, 0.04761904761904761, 0.045454545454545456, 0.050724637681159424, 0.048611111111111105, 0.05333333333333334, 0.05128205128205128, 0.05246913580246913, 0.05357142857142857, 0.054597701149425276, 0.05277777777777778, 0.051075268817204304, 0.052083333333333336, 0.05555555555555555, 0.058823529411764705, 0.059523809523809514, 0.057870370370370364, 0.05855855855855856, 0.06359649122807018, 0.0641025641025641, 0.07291666666666667, 0.07723577235772357, 0.08134920634920634, 0.08527131782945736, 0.08712121212121213, 0.09444444444444444, 0.10326086956521739, 0.10815602836879444, 0.11458333333333343, 0.12244897959183673, 0.13166666666666665]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.1388888888888889, 0.1875, 0.16666666666666666, 0.16666666666666663, 0.1547619047619048, 0.13541666666666666, 0.1574074074074074, 0.1416666666666667, 0.15151515151515152, 0.1597222222222222, 0.15384615384615385, 0.15476190476190477, 0.15555555555555553, 0.15104166666666666, 0.14705882352941177, 0.1435185185185185, 0.13596491228070176, 0.13333333333333333, 0.13888888888888887, 0.13636363636363635, 0.14130434782608695, 0.13541666666666677, 0.13666666666666666, 0.1346153846153846, 0.12962962962962962, 0.13690476190476192, 0.1408045977011494, 0.14166666666666666, 0.13978494623655913, 0.13541666666666666, 0.13131313131313133, 0.12990196078431374, 0.13095238095238093, 0.13425925925925924, 0.13288288288288289, 0.1337719298245614, 0.1346153846153846, 0.13333333333333333, 0.13414634146341461, 0.13492063492063489, 0.13372093023255813, 0.13068181818181818, 0.12962962962962962, 0.1286231884057971, 0.1276595744680852, 0.12847222222222232, 0.12755102040816327, 0.13166666666666665]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.3405526585749826, "coverage": 0.8916666666666667, "abstain_rate": 0.10833333333333334, "selective_risk": 0.08971962616822429, "selective_accuracy": 0.9102803738317757, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.08971962616822429, 0.1157167530224525, 0.13166666666666665, 0.13166666666666665], "coverage": [0.0, 0.0, 0.8916666666666667, 0.965, 1.0, 1.0]}, "ece_geometry": 0.030994720351470172}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "9519934cb397", "timestamp": "2026-07-25T10:21:47.809747+00:00", "git_sha": "758d5f2", "config_hash": "059694ff7fbf", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 2, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 2, "accuracy_id": 0.835, "base_error": 0.165, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.09650922320068285, "rho_basin": 0.12603993510354664, "energy_min": 0.2214910233272095, "energy_mean": 0.22731667150886053, "energy_std": 0.16836492310982842}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.12498180012652664, 0.07895245447131151, 0.1708645351434382], "delta_aurc_vs_best_energy": [0.07185569990914557, 0.02751019561579705, 0.11687465888677867], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.08333333333333333, 0.11666666666666667, 0.11111111111111109, 0.09523809523809525, 0.09375, 0.09259259259259259, 0.09166666666666667, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333331, 0.08333333333333333, 0.0784313725490196, 0.07407407407407406, 0.08333333333333333, 0.08333333333333333, 0.08333333333333331, 0.07954545454545454, 0.08333333333333333, 0.07986111111111109, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08908045977011493, 0.09166666666666666, 0.0913978494623656, 0.08854166666666667, 0.09090909090909091, 0.08823529411764706, 0.09285714285714285, 0.09027777777777776, 0.09234234234234234, 0.08991228070175439, 0.09401709401709402, 0.10208333333333333, 0.10772357723577235, 0.10912698412698411, 0.12015503875968993, 0.1268939393939394, 0.13333333333333333, 0.13949275362318841, 0.147163120567376, 0.15624999999999997, 0.16156462585034015, 0.165]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.12307692307692308, 0.12307692307692308, 0.12307692307692303, 0.12307692307692293, 0.12307692307692288, 0.12307692307692283, 0.1230769230769228, 0.12307692307692279, 0.12307692307692276, 0.12307692307692276, 0.12307692307692275, 0.12307692307692274, 0.12307692307692274, 0.12307692307692272, 0.12307692307692272, 0.12307692307692271, 0.12307692307692271, 0.12307692307692271, 0.1230769230769227, 0.1230769230769227, 0.1230769230769227, 0.1230769230769227, 0.1230769230769227, 0.1230769230769227, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692268, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.12307692307692267, 0.1278409090909087, 0.13472222222222183, 0.14130434782608656, 0.14760638297872306, 0.15364583333333298, 0.15943877551020372, 0.16499999999999965]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.4166666666666667, 0.4166666666666667, 0.3958333333333333, 0.35, 0.3194444444444445, 0.33333333333333337, 0.3333333333333333, 0.2962962962962963, 0.29166666666666674, 0.2878787878787879, 0.2638888888888889, 0.24358974358974358, 0.23809523809523808, 0.22777777777777772, 0.21875, 0.21568627450980393, 0.21759259259259256, 0.2149122807017544, 0.20833333333333334, 0.19841269841269837, 0.19318181818181818, 0.1956521739130435, 0.19097222222222218, 0.18333333333333332, 0.1762820512820513, 0.1728395061728395, 0.17261904761904762, 0.17241379310344826, 0.16944444444444445, 0.1639784946236559, 0.16145833333333334, 0.1590909090909091, 0.1568627450980392, 0.15476190476190474, 0.15740740740740738, 0.1554054054054054, 0.15570175438596492, 0.1517094017094017, 0.14791666666666667, 0.14837398373983737, 0.1468253968253968, 0.14534883720930233, 0.14583333333333334, 0.14444444444444443, 0.14673913043478262, 0.147163120567376, 0.15451388888888887, 0.1598639455782313, 0.165]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.5833333333333334, 0.4722222222222222, 0.4375, 0.4, 0.36111111111111105, 0.3452380952380953, 0.3229166666666667, 0.2962962962962963, 0.2750000000000001, 0.2878787878787879, 0.2638888888888889, 0.25, 0.23214285714285715, 0.21666666666666665, 0.20833333333333334, 0.2107843137254902, 0.20833333333333331, 0.20175438596491227, 0.2, 0.19444444444444442, 0.1893939393939394, 0.18840579710144928, 0.18749999999999997, 0.18, 0.1762820512820513, 0.1697530864197531, 0.16964285714285715, 0.17241379310344826, 0.16944444444444445, 0.16666666666666666, 0.16666666666666666, 0.16414141414141414, 0.15931372549019607, 0.15952380952380948, 0.15740740740740738, 0.1554054054054054, 0.15350877192982457, 0.1517094017094017, 0.14791666666666667, 0.15040650406504064, 0.1468253968253968, 0.14728682170542637, 0.14772727272727273, 0.14629629629629629, 0.14492753623188406, 0.147163120567376, 0.14930555555555552, 0.15816326530612246, 0.165]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.1388888888888889, 0.16666666666666666, 0.16666666666666666, 0.15277777777777776, 0.1547619047619046, 0.16666666666666666, 0.16666666666666666, 0.19166666666666657, 0.17424242424242425, 0.16666666666666666, 0.1794871794871795, 0.19047619047619047, 0.17777777777777776, 0.17708333333333334, 0.1715686274509804, 0.162037037037037, 0.15350877192982457, 0.15416666666666667, 0.1468253968253968, 0.14772727272727273, 0.14492753623188406, 0.15624999999999997, 0.15333333333333332, 0.15384615384615385, 0.15123456790123457, 0.15773809523809523, 0.15804597701149423, 0.15555555555555556, 0.15591397849462366, 0.15885416666666666, 0.15656565656565657, 0.15931372549019607, 0.16190476190476188, 0.16435185185185183, 0.16441441441441443, 0.16666666666666666, 0.16666666666666666, 0.17083333333333334, 0.1686991869918699, 0.1706349206349206, 0.17054263565891473, 0.16856060606060605, 0.17037037037037037, 0.17028985507246377, 0.17021276595744678, 0.17013888888888887, 0.1683673469387755, 0.165]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.102880658436214, 0.15114235500878734, 0.16387959866220736], "coverage": [0.0, 0.0, 0.0, 0.81, 0.9483333333333334, 0.9966666666666667]}, "ece_geometry": 0.04089846776929624}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "978eda331152", "timestamp": "2026-07-25T10:21:50.014667+00:00", "git_sha": "758d5f2", "config_hash": "ac1d6e63a53d", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 2, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 2, "accuracy_id": 0.8166666666666667, "base_error": 0.18333333333333332, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1079878468454813, "rho_basin": 0.14408482719572901, "energy_min": 0.1707430136442979, "energy_mean": 0.16883477585185394, "energy_std": 0.19222202260353177}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.0627551667988166, 0.03425426664674045, 0.09327759618436239], "delta_aurc_vs_best_energy": [0.060846929006372646, 0.03187151271616071, 0.09050543894246903], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.06666666666666667, 0.06944444444444443, 0.07142857142857144, 0.07291666666666667, 0.08333333333333333, 0.07500000000000001, 0.08333333333333333, 0.09027777777777778, 0.08974358974358974, 0.08928571428571429, 0.09999999999999999, 0.09895833333333333, 0.09803921568627451, 0.10185185185185183, 0.10964912280701754, 0.10833333333333334, 0.11111111111111109, 0.10984848484848485, 0.10869565217391304, 0.11111111111111109, 0.11, 0.10897435897435898, 0.1111111111111111, 0.11011904761904762, 0.10919540229885055, 0.11666666666666667, 0.11290322580645161, 0.109375, 0.1111111111111111, 0.11029411764705882, 0.11428571428571427, 0.11342592592592592, 0.11261261261261261, 0.1118421052631579, 0.11965811965811966, 0.12083333333333333, 0.12601626016260173, 0.1329365079365079, 0.13372093023255813, 0.14772727272727273, 0.15555555555555556, 0.15942028985507245, 0.16312056737588665, 0.16493055555555552, 0.1717687074829932, 0.18333333333333332]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.14150943396226418, 0.14150943396226412, 0.14150943396226418, 0.14150943396226426, 0.14150943396226429, 0.1415094339622643, 0.1415094339622643, 0.14150943396226434, 0.14150943396226434, 0.14150943396226429, 0.14150943396226412, 0.141509433962264, 0.1415094339622639, 0.14150943396226381, 0.14150943396226373, 0.14150943396226368, 0.14150943396226362, 0.14150943396226356, 0.1415094339622635, 0.14150943396226348, 0.14150943396226343, 0.1415094339622634, 0.14150943396226337, 0.14150943396226334, 0.14150943396226331, 0.1415094339622633, 0.14150943396226326, 0.14150943396226326, 0.14150943396226323, 0.1415094339622632, 0.1415094339622632, 0.14150943396226318, 0.14150943396226315, 0.14150943396226315, 0.14150943396226312, 0.14150943396226312, 0.14150943396226312, 0.14150943396226315, 0.14150943396226331, 0.14150943396226348, 0.14150943396226362, 0.1415094339622638, 0.14150943396226393, 0.14150943396226404, 0.14814814814814806, 0.15579710144927528, 0.16312056737588648, 0.17013888888888884, 0.17687074829931965, 0.18333333333333326]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.20833333333333334, 0.2222222222222222, 0.20833333333333334, 0.18333333333333332, 0.18055555555555564, 0.1785714285714286, 0.15625, 0.14814814814814814, 0.15833333333333335, 0.16666666666666666, 0.1597222222222222, 0.15384615384615385, 0.17261904761904762, 0.18888888888888886, 0.18229166666666666, 0.17647058823529413, 0.17129629629629628, 0.17543859649122806, 0.17083333333333334, 0.16666666666666663, 0.16287878787878787, 0.15942028985507245, 0.16319444444444442, 0.16333333333333333, 0.16346153846153846, 0.1574074074074074, 0.15476190476190477, 0.15804597701149423, 0.15833333333333333, 0.1586021505376344, 0.15885416666666666, 0.16161616161616163, 0.1642156862745098, 0.16666666666666663, 0.1666666666666668, 0.16666666666666666, 0.16666666666666666, 0.17307692307692307, 0.16875, 0.16666666666666677, 0.16666666666666663, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16847826086956522, 0.16666666666666663, 0.17013888888888898, 0.17857142857142858, 0.18333333333333332]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.16666666666666666, 0.16666666666666666, 0.1875, 0.16666666666666666, 0.13888888888888887, 0.1547619047619048, 0.15625, 0.14814814814814814, 0.15000000000000002, 0.17424242424242425, 0.18055555555555555, 0.16666666666666666, 0.17261904761904762, 0.16666666666666663, 0.16145833333333334, 0.1568627450980392, 0.15277777777777793, 0.16228070175438597, 0.15833333333333333, 0.16666666666666663, 0.16666666666666666, 0.16666666666666666, 0.16666666666666663, 0.17, 0.17307692307692307, 0.1697530864197531, 0.16964285714285715, 0.16954022988505763, 0.16944444444444445, 0.1693548387096774, 0.16666666666666666, 0.16666666666666666, 0.16911764705882354, 0.17619047619047634, 0.1782407407407407, 0.17792792792792791, 0.17543859649122806, 0.1752136752136752, 0.17708333333333334, 0.17886178861788615, 0.17460317460317457, 0.17248062015503876, 0.17045454545454544, 0.1685185185185185, 0.17028985507246377, 0.1684397163120567, 0.1736111111111112, 0.18027210884353742, 0.18333333333333332]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.25, 0.2777777777777778, 0.22916666666666666, 0.23333333333333334, 0.22222222222222218, 0.2023809523809524, 0.1875, 0.18518518518518517, 0.17500000000000002, 0.17424242424242425, 0.16666666666666666, 0.16666666666666666, 0.17261904761904762, 0.17777777777777792, 0.19270833333333334, 0.20098039215686275, 0.20833333333333331, 0.21052631578947367, 0.2, 0.19841269841269837, 0.19318181818181818, 0.18840579710144928, 0.18402777777777776, 0.18333333333333332, 0.18269230769230768, 0.18209876543209877, 0.17857142857142858, 0.17816091954022986, 0.175, 0.17473118279569894, 0.17447916666666666, 0.17929292929292928, 0.18872549019607843, 0.1857142857142857, 0.1898148148148148, 0.1891891891891892, 0.19078947368421054, 0.18803418803418803, 0.18958333333333333, 0.19105691056910568, 0.19246031746031744, 0.187984496124031, 0.1875, 0.18518518518518517, 0.18659420289855072, 0.18971631205673767, 0.18749999999999997, 0.18537414965986396, 0.18333333333333332]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.11839323467230443, 0.18196994991652754, 0.18196994991652754], "coverage": [0.0, 0.0, 0.0, 0.7883333333333333, 0.9983333333333333, 0.9983333333333333]}, "ece_geometry": 0.05161956752782487}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "b73a5b1750f2", "timestamp": "2026-07-25T10:21:57.192336+00:00", "git_sha": "758d5f2", "config_hash": "419a01656461", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 4, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 4, "accuracy_id": 0.7916666666666666, "base_error": 0.20833333333333334, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.12658178782217444, "rho_basin": 0.15202030139254138, "energy_min": 0.16322338754539742, "energy_mean": 0.1728504471727919, "energy_std": 0.2204744623041527}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.03664159972322298, -0.006923467518887684, 0.08247438933094382], "delta_aurc_vs_best_energy": [0.03664159972322298, -0.006923467518887684, 0.08247438933094382], "geometry_wins": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.15, 0.13888888888888887, 0.11904761904761907, 0.10416666666666667, 0.10185185185185185, 0.09166666666666667, 0.11363636363636363, 0.125, 0.12179487179487179, 0.13095238095238096, 0.12222222222222219, 0.11458333333333333, 0.10784313725490197, 0.11111111111111109, 0.10526315789473684, 0.1, 0.09920634920634919, 0.09848484848484848, 0.09420289855072464, 0.09374999999999999, 0.09, 0.08974358974358974, 0.08950617283950617, 0.08928571428571429, 0.09195402298850573, 0.08888888888888889, 0.08870967741935484, 0.09114583333333333, 0.09595959595959595, 0.1053921568627451, 0.1119047619047619, 0.11574074074074073, 0.11711711711711711, 0.1206140350877193, 0.12606837606837606, 0.13125, 0.14024390243902438, 0.1468253968253968, 0.1569767441860465, 0.16477272727272727, 0.17222222222222222, 0.17572463768115942, 0.18617021276595752, 0.19444444444444453, 0.20068027210884354, 0.20833333333333334]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.14658634538152612, 0.14658634538152612, 0.14658634538152612, 0.14658634538152612, 0.14658634538152607, 0.14658634538152593, 0.14658634538152585, 0.14658634538152576, 0.1465863453815257, 0.14658634538152568, 0.14658634538152565, 0.1465863453815256, 0.14658634538152557, 0.14658634538152557, 0.14658634538152554, 0.1465863453815255, 0.1465863453815255, 0.14658634538152548, 0.14658634538152562, 0.1465863453815258, 0.14658634538152593, 0.14658634538152607, 0.14658634538152618, 0.1465863453815263, 0.1465863453815264, 0.14658634538152648, 0.14658634538152657, 0.14658634538152665, 0.14658634538152673, 0.1465863453815268, 0.14658634538152687, 0.14658634538152693, 0.14658634538152698, 0.14658634538152704, 0.1465863453815271, 0.14658634538152715, 0.14658634538152707, 0.14658634538152693, 0.1465863453815268, 0.14658634538152665, 0.14658634538152654, 0.1507163471449189, 0.15868821101379255, 0.16629771743408106, 0.17356902356902343, 0.1805241859589683, 0.18718338399189424, 0.1936805555555551, 0.20115646258503356, 0.20833333333333293]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1388888888888889, 0.16666666666666666, 0.16666666666666666, 0.13888888888888887, 0.14285714285714288, 0.13541666666666666, 0.12037037037037036, 0.13333333333333336, 0.12878787878787878, 0.13194444444444445, 0.14743589743589744, 0.14285714285714285, 0.1333333333333333, 0.140625, 0.13725490196078433, 0.1342592592592594, 0.14473684210526316, 0.15, 0.1507936507936509, 0.17045454545454544, 0.18115942028985507, 0.18055555555555552, 0.18333333333333332, 0.1858974358974359, 0.18518518518518517, 0.18154761904761904, 0.18390804597701146, 0.17777777777777778, 0.17473118279569894, 0.17708333333333334, 0.17929292929292928, 0.17647058823529413, 0.17857142857142855, 0.1759259259259259, 0.18018018018018017, 0.18421052631578946, 0.18376068376068377, 0.18333333333333332, 0.18699186991869915, 0.18650793650793648, 0.18992248062015504, 0.19128787878787878, 0.19074074074074074, 0.19202898550724637, 0.18971631205673756, 0.19444444444444453, 0.20408163265306123, 0.20833333333333334]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.1388888888888889, 0.16666666666666666, 0.16666666666666666, 0.13888888888888887, 0.14285714285714288, 0.15625, 0.16666666666666666, 0.17500000000000002, 0.16666666666666666, 0.18055555555555555, 0.17307692307692307, 0.17857142857142858, 0.16666666666666663, 0.16145833333333334, 0.16176470588235295, 0.16666666666666663, 0.17105263157894737, 0.17083333333333334, 0.1785714285714287, 0.17424242424242425, 0.17753623188405798, 0.17708333333333331, 0.18, 0.1794871794871795, 0.18209876543209877, 0.18154761904761904, 0.18965517241379307, 0.18611111111111112, 0.18548387096774194, 0.1796875, 0.18181818181818182, 0.17892156862745098, 0.18095238095238092, 0.18518518518518515, 0.18018018018018017, 0.18201754385964913, 0.1794871794871795, 0.18541666666666667, 0.18699186991869915, 0.1845238095238095, 0.1821705426356589, 0.18371212121212122, 0.18888888888888888, 0.1956521739130435, 0.20212765957446815, 0.20659722222222218, 0.20238095238095238, 0.20833333333333334]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.19444444444444445, 0.22916666666666666, 0.25, 0.24999999999999997, 0.25000000000000006, 0.2604166666666667, 0.25, 0.2583333333333334, 0.25757575757575757, 0.2638888888888889, 0.25, 0.24404761904761904, 0.24444444444444438, 0.22916666666666666, 0.22058823529411764, 0.22685185185185183, 0.2236842105263158, 0.22083333333333333, 0.23015873015873026, 0.22727272727272727, 0.2210144927536232, 0.21874999999999997, 0.21666666666666667, 0.21794871794871795, 0.22530864197530864, 0.2261904761904762, 0.2183908045977011, 0.21666666666666667, 0.22043010752688172, 0.21614583333333334, 0.21212121212121213, 0.21323529411764705, 0.2095238095238095, 0.20833333333333345, 0.20945945945945946, 0.21271929824561403, 0.21153846153846154, 0.20833333333333334, 0.21341463414634143, 0.21428571428571425, 0.2189922480620155, 0.2196969696969697, 0.21851851851851853, 0.21739130434782608, 0.21985815602836878, 0.21527777777777776, 0.21258503401360543, 0.20833333333333334]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.11338339409684943, "coverage": 0.5833333333333334, "abstain_rate": 0.4166666666666667, "selective_risk": 0.09142857142857143, "selective_accuracy": 0.9085714285714286, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.09142857142857143, 0.13250517598343686, 0.19478260869565217, 0.20833333333333334], "coverage": [0.0, 0.0, 0.5833333333333334, 0.805, 0.9583333333333334, 1.0]}, "ece_geometry": 0.056508899370470926}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "f14b18cfaee4", "timestamp": "2026-07-25T10:21:59.558386+00:00", "git_sha": "758d5f2", "config_hash": "45792209eed5", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 4, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 4, "accuracy_id": 0.8083333333333333, "base_error": 0.19166666666666668, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.12053282876022095, "rho_basin": 0.1302793637385637, "energy_min": 0.17421604752784897, "energy_mean": 0.17973212632171362, "energy_std": 0.19617010941107094}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.05368321876762802, 0.007047980279657124, 0.10108678811590206], "delta_aurc_vs_best_energy": [0.05368321876762802, 0.007047980279657124, 0.10108678811590206], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.20833333333333334, 0.19444444444444445, 0.16666666666666666, 0.13333333333333333, 0.11111111111111109, 0.09523809523809525, 0.08333333333333333, 0.07407407407407407, 0.08333333333333334, 0.08333333333333333, 0.08333333333333333, 0.08974358974358974, 0.10714285714285714, 0.10555555555555554, 0.109375, 0.10784313725490197, 0.10185185185185183, 0.10087719298245613, 0.1, 0.09920634920634919, 0.10227272727272728, 0.10144927536231885, 0.10069444444444443, 0.09666666666666666, 0.09935897435897435, 0.09567901234567901, 0.09226190476190477, 0.08908045977011493, 0.08611111111111111, 0.08602150537634409, 0.09114583333333333, 0.09090909090909091, 0.09558823529411764, 0.09285714285714285, 0.0949074074074074, 0.0945945945945946, 0.09868421052631579, 0.10683760683760683, 0.10833333333333334, 0.11788617886178861, 0.12499999999999999, 0.13372093023255813, 0.14204545454545456, 0.15, 0.16304347826086957, 0.17375886524822706, 0.17708333333333331, 0.18197278911564627, 0.19166666666666668]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.12396694214876035, 0.12396694214876029, 0.12396694214876024, 0.12396694214876021, 0.1239669421487602, 0.1239669421487602, 0.12396694214876018, 0.12396694214876018, 0.12396694214876017, 0.12396694214876017, 0.1239669421487602, 0.12396694214876033, 0.12396694214876046, 0.12396694214876056, 0.12396694214876065, 0.12396694214876074, 0.1239669421487608, 0.12396694214876086, 0.12396694214876092, 0.12396694214876097, 0.12396694214876101, 0.12396694214876106, 0.1239669421487611, 0.12396694214876113, 0.12396694214876115, 0.12396694214876118, 0.12396694214876121, 0.12396694214876124, 0.12396694214876126, 0.12396694214876128, 0.1239669421487613, 0.12396694214876132, 0.12396694214876135, 0.12396694214876136, 0.12396694214876137, 0.12396694214876139, 0.1239669421487614, 0.12396694214876142, 0.12396694214876143, 0.12396694214876144, 0.12883787661406135, 0.13585434173669572, 0.1425444596443238, 0.14893048128342334, 0.15503267973856294, 0.1608695652173921, 0.1664580725907392, 0.17411514336917636, 0.18307000219442676, 0.19166666666666718]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.16666666666666666, 0.16666666666666666, 0.18055555555555564, 0.1785714285714286, 0.15625, 0.1574074074074074, 0.15000000000000002, 0.1590909090909091, 0.1527777777777778, 0.15384615384615385, 0.15476190476190477, 0.1611111111111111, 0.17708333333333334, 0.18627450980392157, 0.18518518518518515, 0.17982456140350878, 0.17916666666666667, 0.17460317460317457, 0.17424242424242425, 0.17753623188405798, 0.17708333333333331, 0.17333333333333334, 0.16987179487179488, 0.17592592592592593, 0.17261904761904762, 0.17816091954022986, 0.18333333333333332, 0.1881720430107527, 0.18229166666666666, 0.18181818181818182, 0.17647058823529413, 0.1809523809523811, 0.18287037037037035, 0.17792792792792791, 0.17982456140350878, 0.1794871794871795, 0.17708333333333334, 0.1768292682926829, 0.17857142857142855, 0.18023255813953487, 0.1875, 0.18888888888888888, 0.18478260869565216, 0.18971631205673767, 0.19097222222222218, 0.18877551020408162, 0.19166666666666668]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.1388888888888889, 0.20833333333333334, 0.2, 0.19444444444444442, 0.2023809523809524, 0.21875, 0.2037037037037037, 0.20833333333333323, 0.1893939393939394, 0.1875, 0.20512820512820512, 0.19642857142857142, 0.19444444444444442, 0.1875, 0.18627450980392157, 0.18518518518518515, 0.17982456140350878, 0.18333333333333332, 0.1825396825396825, 0.17424242424242425, 0.17753623188405798, 0.17708333333333331, 0.18, 0.1794871794871795, 0.18209876543209877, 0.18154761904761904, 0.17816091954023006, 0.175, 0.17473118279569894, 0.17447916666666666, 0.17676767676767677, 0.17647058823529413, 0.17857142857142855, 0.18055555555555552, 0.17792792792792791, 0.17763157894736842, 0.1752136752136752, 0.17916666666666667, 0.17886178861788615, 0.1765873015873017, 0.18023255813953487, 0.17992424242424243, 0.18333333333333332, 0.18297101449275363, 0.18439716312056734, 0.18576388888888887, 0.18877551020408162, 0.19166666666666668]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.19444444444444445, 0.20833333333333334, 0.25, 0.23611111111111108, 0.2023809523809524, 0.19791666666666666, 0.19444444444444445, 0.20000000000000004, 0.2196969696969697, 0.20833333333333334, 0.1987179487179487, 0.20833333333333334, 0.21111111111111108, 0.20833333333333334, 0.20588235294117646, 0.20833333333333331, 0.20175438596491227, 0.20416666666666666, 0.19444444444444442, 0.19696969696969696, 0.18840579710144928, 0.19791666666666674, 0.20333333333333334, 0.20833333333333334, 0.20987654320987653, 0.20535714285714285, 0.20114942528735646, 0.20277777777777778, 0.20430107526881722, 0.203125, 0.20454545454545456, 0.20098039215686275, 0.19761904761904758, 0.1921296296296296, 0.1891891891891892, 0.19078947368421054, 0.19230769230769232, 0.19166666666666668, 0.18699186991869915, 0.1884920634920636, 0.19186046511627908, 0.1893939393939394, 0.18518518518518517, 0.18297101449275363, 0.1826241134751774, 0.18402777777777776, 0.19047619047619047, 0.19166666666666668]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0951276102088167, 0.124, 0.19166666666666668], "coverage": [0.0, 0.0, 0.0, 0.7183333333333334, 0.8333333333333334, 1.0]}, "ece_geometry": 0.0466896147946997}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "04b6026d0afc", "timestamp": "2026-07-25T10:22:01.949658+00:00", "git_sha": "758d5f2", "config_hash": "825c1996aff6", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 4, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 4, "accuracy_id": 0.87, "base_error": 0.13, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.02761391710128735, "rho_basin": 0.06699643572970755, "energy_min": 0.0572185525362447, "energy_mean": 0.05836000424349217, "energy_std": 0.1250598714201054}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.029604635434957352, 0.014804941268848, 0.04736534452828845], "delta_aurc_vs_best_energy": [0.029604635434957352, 0.014804941268848, 0.04736534452828845], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.008333333333333335, 0.007575757575757576, 0.006944444444444444, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.009259259259259257, 0.008771929824561403, 0.008333333333333333, 0.011904761904761902, 0.011363636363636364, 0.010869565217391304, 0.01041666666666678, 0.013333333333333334, 0.01282051282051282, 0.015432098765432098, 0.01488095238095238, 0.017241379310344824, 0.019444444444444445, 0.01881720430107527, 0.018229166666666668, 0.017676767676767676, 0.01715686274509804, 0.021428571428571425, 0.023148148148148143, 0.02702702702702703, 0.03289473684210526, 0.038461538461538464, 0.04791666666666667, 0.05691056910569118, 0.0734126984126984, 0.07945736434108527, 0.08712121212121213, 0.08888888888888889, 0.10144927536231885, 0.11170212765957457, 0.11631944444444443, 0.12244897959183673, 0.13]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.06048387096774196, 0.06048387096774197, 0.06048387096774198, 0.060483870967741986, 0.060483870967741986, 0.06048387096774198, 0.06048387096774198, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.06048387096774198, 0.06048387096774198, 0.06048387096774198, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741986, 0.060483870967741916, 0.06048387096774185, 0.06048387096774178, 0.06048387096774172, 0.06048387096774167, 0.06048387096774161, 0.06048387096774156, 0.06048387096774152, 0.06048387096774148, 0.06048387096774144, 0.0604838709677414, 0.06048387096774137, 0.060483870967741334, 0.0604838709677413, 0.06048387096774127, 0.06048387096774124, 0.060483870967741216, 0.060483870967741195, 0.06048387096774117, 0.06691672102630938, 0.07619199320377969, 0.08504566210045596, 0.09350583460172437, 0.10159817351598113, 0.10934615758282276, 0.11659946236559092, 0.12343647136273815, 0.12999999999999956]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.027777777777777776, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.038461538461538464, 0.041666666666666664, 0.04444444444444444, 0.041666666666666664, 0.049019607843137254, 0.04629629629629644, 0.05263157894736842, 0.05416666666666667, 0.05158730158730158, 0.04924242424242424, 0.057971014492753624, 0.05902777777777777, 0.056666666666666664, 0.05448717948717949, 0.05246913580246913, 0.05654761904761905, 0.054597701149425276, 0.05277777777777778, 0.051075268817204304, 0.0546875, 0.05808080808080808, 0.06372549019607843, 0.06666666666666682, 0.06712962962962962, 0.06756756756756757, 0.07017543859649122, 0.07264957264957266, 0.07291666666666667, 0.07520325203252032, 0.08134920634920648, 0.08914728682170543, 0.09090909090909091, 0.0962962962962963, 0.09963768115942029, 0.10638297872340437, 0.11805555555555554, 0.12414965986394558, 0.13]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.045454545454545456, 0.04861111111111111, 0.05128205128205128, 0.05357142857142857, 0.049999999999999996, 0.046875, 0.04411764705882353, 0.050925925925925916, 0.05701754385964912, 0.05416666666666667, 0.05158730158730158, 0.05303030303030303, 0.05434782608695652, 0.05208333333333333, 0.05333333333333334, 0.057692307692307696, 0.05555555555555555, 0.05357142857142857, 0.051724137931034475, 0.05555555555555555, 0.056451612903225805, 0.0546875, 0.06060606060606061, 0.061274509803921566, 0.06428571428571427, 0.06712962962962962, 0.06531531531531531, 0.06578947368421052, 0.06623931623931624, 0.07083333333333333, 0.07317073170731705, 0.08134920634920634, 0.08527131782945736, 0.09090909090909091, 0.0962962962962963, 0.09963768115942029, 0.10992907801418439, 0.11284722222222232, 0.12244897959183673, 0.13]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1111111111111111, 0.125, 0.11666666666666667, 0.12499999999999999, 0.10714285714285716, 0.125, 0.12037037037037036, 0.13333333333333322, 0.12121212121212122, 0.11805555555555555, 0.11538461538461539, 0.10714285714285714, 0.10555555555555554, 0.10416666666666667, 0.11274509803921569, 0.11574074074074073, 0.11842105263157894, 0.11666666666666667, 0.11507936507936506, 0.11742424242424243, 0.11956521739130435, 0.12152777777777787, 0.12333333333333334, 0.11858974358974358, 0.12037037037037036, 0.13095238095238096, 0.13218390804597718, 0.1361111111111111, 0.13709677419354838, 0.13541666666666666, 0.1388888888888889, 0.13725490196078433, 0.13571428571428568, 0.13194444444444442, 0.13513513513513514, 0.1337719298245614, 0.13247863247863248, 0.13333333333333333, 0.13008130081300825, 0.1289682539682541, 0.12790697674418605, 0.13257575757575757, 0.12962962962962962, 0.1286231884057971, 0.12943262411347514, 0.12847222222222218, 0.13095238095238096, 0.13]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.34533349095960586, "coverage": 0.8583333333333333, "abstain_rate": 0.14166666666666666, "selective_risk": 0.07766990291262135, "selective_accuracy": 0.9223300970873787, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.017587939698492462, 0.07766990291262135, 0.1073345259391771, 0.13, 0.13], "coverage": [0.0, 0.6633333333333333, 0.8583333333333333, 0.9316666666666666, 1.0, 1.0]}, "ece_geometry": 0.06572641460207279}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "3dc77bab6f6f", "timestamp": "2026-07-25T10:22:04.275929+00:00", "git_sha": "758d5f2", "config_hash": "0347ef3f7e0e", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 4, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 4, "accuracy_id": 0.8316666666666667, "base_error": 0.16833333333333333, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.05743518107609235, "rho_basin": 0.09939148305207537, "energy_min": 0.19778637148843448, "energy_mean": 0.20031402982035093, "energy_std": 0.18513998134992563}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.14035119041234212, 0.09732104801385821, 0.18551141273176394], "delta_aurc_vs_best_energy": [0.12770480027383327, 0.0867155327766748, 0.17125023618432983], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.027777777777777773, 0.02380952380952381, 0.03125, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.03571428571428571, 0.033333333333333326, 0.03125, 0.029411764705882353, 0.027777777777777773, 0.02631578947368421, 0.025, 0.023809523809523805, 0.026515151515151516, 0.025362318840579712, 0.024305555555555552, 0.023333333333333334, 0.02564102564102564, 0.024691358024691357, 0.02976190476190476, 0.02873563218390804, 0.030555555555555555, 0.03494623655913978, 0.036458333333333336, 0.04292929292929293, 0.04411764705882353, 0.054761904761904755, 0.0648148148148148, 0.07207207207207207, 0.07675438596491228, 0.08974358974358974, 0.1, 0.10975609756097573, 0.11111111111111122, 0.11434108527131782, 0.11931818181818182, 0.12407407407407407, 0.1358695652173913, 0.14539007092198578, 0.15104166666666677, 0.1598639455782313, 0.16833333333333333]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.09071274298056155, 0.09071274298056159, 0.09071274298056153, 0.09071274298056152, 0.09071274298056149, 0.09071274298056148, 0.09071274298056148, 0.09071274298056153, 0.09071274298056162, 0.09071274298056169, 0.09071274298056174, 0.0907127429805618, 0.09071274298056184, 0.09071274298056187, 0.09071274298056185, 0.09071274298056177, 0.0907127429805617, 0.09071274298056163, 0.09071274298056158, 0.09071274298056152, 0.09071274298056148, 0.09071274298056144, 0.0907127429805614, 0.09071274298056137, 0.09071274298056133, 0.0907127429805613, 0.09071274298056127, 0.09071274298056124, 0.09071274298056121, 0.09071274298056127, 0.09071274298056135, 0.09071274298056144, 0.09071274298056152, 0.0907127429805616, 0.09071274298056167, 0.09071274298056174, 0.0907127429805618, 0.09071274298056185, 0.0937796771130108, 0.10087962962963003, 0.10763324299909714, 0.11406525573192293, 0.12019810508182655, 0.1260521885521891, 0.13164609053498003, 0.13699677938808438, 0.14512599969820514, 0.15318410165484728, 0.1609133014908101, 0.16833333333333445]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.3333333333333333, 0.3055555555555556, 0.3541666666666667, 0.31666666666666665, 0.27777777777777773, 0.27380952380952367, 0.23958333333333334, 0.26851851851851855, 0.2499999999999999, 0.24242424242424243, 0.2222222222222222, 0.22435897435897437, 0.22023809523809523, 0.21666666666666665, 0.203125, 0.19607843137254902, 0.1851851851851853, 0.18421052631578946, 0.1875, 0.1825396825396825, 0.17803030303030304, 0.17391304347826086, 0.17361111111111108, 0.17, 0.16987179487179488, 0.16358024691358025, 0.16071428571428573, 0.15804597701149423, 0.1527777777777778, 0.15053763440860216, 0.15104166666666666, 0.15404040404040403, 0.14950980392156862, 0.1452380952380954, 0.1435185185185185, 0.14189189189189189, 0.1425438596491228, 0.14102564102564102, 0.13958333333333334, 0.1463414634146341, 0.14484126984126983, 0.14147286821705427, 0.14204545454545456, 0.14444444444444443, 0.14673913043478262, 0.15602836879432622, 0.15972222222222218, 0.1649659863945578, 0.16833333333333333]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3611111111111111, 0.2916666666666667, 0.26666666666666666, 0.24999999999999997, 0.28571428571428575, 0.2916666666666667, 0.28703703703703703, 0.26666666666666655, 0.26515151515151514, 0.24305555555555555, 0.22435897435897437, 0.20833333333333334, 0.20555555555555552, 0.19791666666666666, 0.18627450980392157, 0.18055555555555552, 0.17543859649122806, 0.175, 0.1706349206349206, 0.16666666666666666, 0.16666666666666666, 0.16319444444444442, 0.17, 0.16666666666666666, 0.1697530864197531, 0.1636904761904762, 0.16091954022988503, 0.1638888888888889, 0.16129032258064516, 0.15625, 0.1590909090909091, 0.1568627450980392, 0.15238095238095237, 0.15046296296296294, 0.14864864864864866, 0.14473684210526316, 0.14316239316239315, 0.14375, 0.1463414634146341, 0.1468253968253968, 0.14534883720930233, 0.14772727272727273, 0.15185185185185185, 0.15217391304347827, 0.15425531914893614, 0.15972222222222218, 0.16326530612244897, 0.16833333333333333]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.19444444444444445, 0.1875, 0.2, 0.18055555555555552, 0.1904761904761905, 0.20833333333333334, 0.2222222222222222, 0.2166666666666667, 0.21212121212121213, 0.20833333333333334, 0.23076923076923078, 0.23214285714285715, 0.22777777777777772, 0.21354166666666666, 0.20098039215686275, 0.19907407407407404, 0.19298245614035087, 0.19583333333333333, 0.19841269841269837, 0.1893939393939394, 0.18840579710144928, 0.18055555555555552, 0.17333333333333334, 0.1794871794871795, 0.17901234567901234, 0.17559523809523808, 0.17528735632183906, 0.16944444444444445, 0.17473118279569894, 0.17447916666666666, 0.1717171717171717, 0.17401960784313725, 0.17380952380952377, 0.17592592592592607, 0.17792792792792791, 0.17543859649122806, 0.17307692307692307, 0.17291666666666666, 0.17479674796747965, 0.1706349206349206, 0.1686046511627907, 0.17234848484848486, 0.17037037037037037, 0.16847826086956522, 0.1684397163120567, 0.16840277777777787, 0.1683673469387755, 0.16833333333333333]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.07692307692307693, 0.14028776978417265, 0.16833333333333333], "coverage": [0.0, 0.0, 0.0, 0.7583333333333333, 0.9266666666666666, 1.0]}, "ece_geometry": 0.0676574005201399}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "8c4a180e7c9e", "timestamp": "2026-07-25T10:22:06.589628+00:00", "git_sha": "758d5f2", "config_hash": "d5bb5763735c", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 4, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 4, "accuracy_id": 0.8283333333333334, "base_error": 0.17166666666666666, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.07892820656559074, "rho_basin": 0.11324946962739008, "energy_min": 0.1450436399153879, "energy_mean": 0.1413901715335405, "energy_std": 0.1728798293160865}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.06611543334979716, 0.03371258701353665, 0.10053648274043608], "delta_aurc_vs_best_energy": [0.06246196496794976, 0.032243977469080747, 0.0969300406381104], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.08333333333333333, 0.0625, 0.06666666666666667, 0.055555555555555546, 0.07142857142857124, 0.07291666666666667, 0.06481481481481481, 0.05833333333333334, 0.05303030303030303, 0.04861111111111111, 0.05128205128205128, 0.05357142857142857, 0.055555555555555546, 0.052083333333333336, 0.05392156862745098, 0.050925925925925916, 0.05701754385964912, 0.05416666666666667, 0.059523809523809514, 0.06818181818181818, 0.07608695652173914, 0.07291666666666666, 0.07, 0.07051282051282051, 0.06790123456790123, 0.06845238095238096, 0.06896551724137949, 0.075, 0.07526881720430108, 0.07291666666666667, 0.07323232323232323, 0.07598039215686274, 0.07619047619047618, 0.07870370370370369, 0.08108108108108109, 0.08552631578947369, 0.0876068376068376, 0.09791666666666667, 0.10162601626016272, 0.10714285714285712, 0.12209302325581395, 0.12310606060606061, 0.13148148148148148, 0.1358695652173913, 0.14716312056737585, 0.15972222222222218, 0.16326530612244897, 0.17166666666666666]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.10794297352342162, 0.10794297352342158, 0.10794297352342153, 0.10794297352342151, 0.10794297352342148, 0.10794297352342147, 0.10794297352342148, 0.10794297352342147, 0.10794297352342147, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342146, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342144, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.10794297352342143, 0.1085167558992662, 0.11522454510259378, 0.12162034411041771, 0.1277254249815224, 0.13355916892502245, 0.13913927182750077, 0.1444819235426396, 0.15104166666666666, 0.16156462585034015, 0.17166666666666677]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333331, 0.08333333333333334, 0.10416666666666667, 0.12037037037037036, 0.11666666666666668, 0.14393939393939395, 0.1388888888888889, 0.14743589743589744, 0.1488095238095238, 0.16666666666666663, 0.16145833333333334, 0.16176470588235295, 0.1666666666666668, 0.17105263157894737, 0.16666666666666666, 0.16666666666666663, 0.16666666666666666, 0.17028985507246377, 0.16319444444444442, 0.16, 0.16987179487179488, 0.16666666666666666, 0.16666666666666666, 0.16091954022988503, 0.16111111111111112, 0.16129032258064516, 0.15625, 0.1590909090909091, 0.1568627450980392, 0.15476190476190474, 0.15509259259259256, 0.15765765765765766, 0.15789473684210525, 0.15384615384615385, 0.15, 0.15040650406504064, 0.1468253968253968, 0.14534883720930233, 0.14772727272727273, 0.14814814814814814, 0.15217391304347827, 0.15602836879432622, 0.16145833333333343, 0.1649659863945578, 0.17166666666666666]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.11666666666666667, 0.12499999999999999, 0.10714285714285716, 0.13541666666666666, 0.1388888888888889, 0.13333333333333336, 0.12121212121212122, 0.11805555555555555, 0.11538461538461539, 0.125, 0.1333333333333333, 0.15104166666666666, 0.15196078431372548, 0.15277777777777793, 0.15789473684210525, 0.15833333333333333, 0.15873015873015883, 0.16287878787878787, 0.16304347826086957, 0.15972222222222218, 0.16, 0.16346153846153846, 0.1574074074074074, 0.15476190476190477, 0.15229885057471262, 0.1527777777777778, 0.1478494623655914, 0.15364583333333334, 0.15404040404040403, 0.15196078431372548, 0.15000000000000013, 0.14814814814814828, 0.14864864864864866, 0.14912280701754385, 0.14957264957264957, 0.15, 0.14837398373983737, 0.1468253968253968, 0.14728682170542637, 0.14772727272727273, 0.15185185185185185, 0.1539855072463768, 0.15957446808510636, 0.15798611111111108, 0.1649659863945578, 0.17166666666666666]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.16666666666666666, 0.14583333333333334, 0.15, 0.13888888888888887, 0.14285714285714288, 0.17708333333333334, 0.18518518518518517, 0.16666666666666669, 0.17424242424242425, 0.1875, 0.1858974358974359, 0.17261904761904762, 0.17777777777777776, 0.17708333333333334, 0.17647058823529413, 0.1759259259259259, 0.17543859649122806, 0.175, 0.17460317460317457, 0.17045454545454544, 0.17028985507246377, 0.17361111111111108, 0.17666666666666667, 0.1762820512820513, 0.17592592592592593, 0.17857142857142858, 0.17528735632183906, 0.17777777777777778, 0.17473118279569894, 0.17447916666666666, 0.1717171717171717, 0.17647058823529413, 0.17380952380952377, 0.17129629629629628, 0.17117117117117117, 0.16666666666666666, 0.1623931623931624, 0.16041666666666668, 0.16666666666666663, 0.1646825396825398, 0.16472868217054262, 0.16477272727272727, 0.16666666666666666, 0.1721014492753623, 0.17021276595744678, 0.17013888888888898, 0.1717687074829932, 0.17166666666666666]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.10420841683366733, 0.16298811544991512, 0.17028380634390652], "coverage": [0.0, 0.0, 0.0, 0.8316666666666667, 0.9816666666666667, 0.9983333333333333]}, "ece_geometry": 0.04422131777655634}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "db2094e35b89", "timestamp": "2026-07-25T10:22:13.844050+00:00", "git_sha": "758d5f2", "config_hash": "bbd87d16beee", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 8, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 8, "accuracy_id": 0.8016666666666666, "base_error": 0.19833333333333333, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.09964969410606941, "rho_basin": 0.136911158807512, "energy_min": 0.1582496348019472, "energy_mean": 0.15223701900438943, "energy_std": 0.20988674828910214}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.05859994069587779, 0.024844196234793395, 0.09348680881996146], "delta_aurc_vs_best_energy": [0.05258732489832002, 0.021098904075024343, 0.0860474855092495], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1111111111111111, 0.08333333333333333, 0.1, 0.08333333333333344, 0.09523809523809525, 0.08333333333333333, 0.08333333333333333, 0.08333333333333334, 0.08333333333333333, 0.0763888888888889, 0.07692307692307693, 0.08333333333333333, 0.08333333333333331, 0.078125, 0.07352941176470588, 0.07870370370370369, 0.08333333333333333, 0.07916666666666666, 0.07539682539682538, 0.07196969696969698, 0.06884057971014493, 0.06597222222222221, 0.06333333333333334, 0.0641025641025641, 0.06481481481481481, 0.06547619047619048, 0.07471264367816091, 0.07222222222222222, 0.07795698924731183, 0.08072916666666667, 0.08333333333333333, 0.08823529411764706, 0.09523809523809537, 0.10185185185185183, 0.10135135135135136, 0.1074561403508772, 0.11965811965811966, 0.12708333333333333, 0.13008130081300812, 0.13095238095238093, 0.13953488372093023, 0.14772727272727273, 0.15925925925925927, 0.17028985507246377, 0.17198581560283685, 0.18402777777777776, 0.19047619047619047, 0.19833333333333333]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1290322580645161, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451604, 0.12903225806451607, 0.12903225806451624, 0.12903225806451638, 0.12903225806451651, 0.12903225806451663, 0.12903225806451674, 0.12903225806451685, 0.12903225806451696, 0.12903225806451704, 0.12903225806451712, 0.1290322580645172, 0.12903225806451726, 0.12903225806451735, 0.1290322580645174, 0.12903225806451746, 0.12903225806451754, 0.1290322580645176, 0.12903225806451762, 0.13046757164404377, 0.1360294117647074, 0.14131994261119246, 0.1463585434173687, 0.15116279069767624, 0.1596320346320363, 0.16772486772486922, 0.17546583850931813, 0.18282111899133305, 0.1898148148148161, 0.1953352769679313, 0.1983333333333347]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.1388888888888889, 0.10416666666666667, 0.11666666666666667, 0.13888888888888887, 0.130952380952381, 0.125, 0.14814814814814814, 0.14999999999999988, 0.15151515151515152, 0.1527777777777778, 0.14743589743589744, 0.15476190476190477, 0.15555555555555553, 0.15104166666666666, 0.16666666666666666, 0.18055555555555552, 0.17543859649122806, 0.17083333333333334, 0.1746031746031747, 0.17424242424242425, 0.17391304347826086, 0.17708333333333331, 0.17666666666666667, 0.17307692307692307, 0.1728395061728395, 0.16666666666666666, 0.16954022988505763, 0.17222222222222222, 0.1693548387096774, 0.171875, 0.1717171717171717, 0.16911764705882354, 0.16428571428571442, 0.162037037037037, 0.15765765765765766, 0.16228070175438597, 0.16666666666666666, 0.16666666666666666, 0.16869918699187003, 0.1726190476190476, 0.17635658914728683, 0.18181818181818182, 0.18703703703703703, 0.18659420289855072, 0.19148936170212763, 0.19270833333333331, 0.195578231292517, 0.19833333333333333]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.08333333333333333, 0.06666666666666667, 0.09722222222222232, 0.130952380952381, 0.11458333333333333, 0.12037037037037036, 0.12499999999999988, 0.14393939393939395, 0.1597222222222222, 0.16666666666666666, 0.17857142857142858, 0.1833333333333333, 0.17708333333333334, 0.1715686274509804, 0.17592592592592607, 0.17105263157894737, 0.16666666666666666, 0.1706349206349206, 0.16287878787878787, 0.16666666666666666, 0.16319444444444442, 0.16333333333333333, 0.16987179487179488, 0.1697530864197531, 0.16964285714285715, 0.17528735632183906, 0.16944444444444445, 0.16666666666666666, 0.16927083333333334, 0.16414141414141414, 0.16176470588235295, 0.15952380952380948, 0.15972222222222218, 0.16216216216216217, 0.16447368421052633, 0.16666666666666666, 0.17083333333333334, 0.1727642276422764, 0.1706349206349206, 0.17054263565891473, 0.17424242424242425, 0.17777777777777778, 0.18115942028985507, 0.18794326241134748, 0.19270833333333331, 0.19727891156462585, 0.19833333333333333]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.25, 0.19444444444444445, 0.16666666666666666, 0.18333333333333332, 0.20833333333333343, 0.21428571428571433, 0.19791666666666666, 0.18518518518518517, 0.1916666666666667, 0.19696969696969696, 0.19444444444444445, 0.1987179487179487, 0.20833333333333334, 0.20000000000000015, 0.20833333333333334, 0.2107843137254902, 0.21296296296296294, 0.2149122807017544, 0.21666666666666667, 0.21428571428571425, 0.2159090909090909, 0.2246376811594203, 0.22569444444444442, 0.22, 0.22115384615384615, 0.2191358024691358, 0.21726190476190477, 0.2183908045977011, 0.2111111111111111, 0.20967741935483872, 0.20572916666666666, 0.20202020202020202, 0.2034313725490196, 0.19999999999999998, 0.19444444444444442, 0.19369369369369369, 0.19517543859649122, 0.19658119658119658, 0.19583333333333333, 0.19512195121951217, 0.19444444444444442, 0.19767441860465115, 0.19507575757575757, 0.1925925925925926, 0.18840579710144928, 0.19148936170212763, 0.1909722222222223, 0.19387755102040816, 0.19833333333333333]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.08858404036026933, "coverage": 0.5666666666666667, "abstain_rate": 0.43333333333333335, "selective_risk": 0.06764705882352941, "selective_accuracy": 0.9323529411764706, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.06764705882352941, 0.12101910828025478, 0.1895093062605753, 0.19699499165275458], "coverage": [0.0, 0.0, 0.5666666666666667, 0.785, 0.985, 0.9983333333333333]}, "ece_geometry": 0.04730039025188318}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "7a85d96687d2", "timestamp": "2026-07-25T10:22:16.344468+00:00", "git_sha": "758d5f2", "config_hash": "537005d71045", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 8, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 8, "accuracy_id": 0.8083333333333333, "base_error": 0.19166666666666668, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.09480381514717583, "rho_basin": 0.11311553298788574, "energy_min": 0.17291036853935912, "energy_mean": 0.17759597664832433, "energy_std": 0.21521046312171294}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.0781065533921833, 0.040663124950666624, 0.11906619658077004], "delta_aurc_vs_best_energy": [0.0781065533921833, 0.040663124950666624, 0.11906619658077004], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.1388888888888889, 0.10416666666666667, 0.08333333333333333, 0.08333333333333331, 0.08333333333333334, 0.07291666666666667, 0.06481481481481481, 0.05833333333333334, 0.06060606060606061, 0.06944444444444445, 0.07051282051282051, 0.07738095238095238, 0.0722222222222222, 0.078125, 0.07352941176470588, 0.06944444444444443, 0.07017543859649122, 0.07083333333333333, 0.07142857142857141, 0.06818181818181818, 0.07608695652173914, 0.07291666666666666, 0.07333333333333333, 0.07051282051282051, 0.07098765432098765, 0.07142857142857142, 0.0689655172413793, 0.06944444444444445, 0.06989247311827956, 0.07291666666666667, 0.07323232323232323, 0.0857843137254902, 0.08809523809523809, 0.0949074074074074, 0.10135135135135136, 0.10526315789473684, 0.10897435897435898, 0.11458333333333333, 0.11991869918699186, 0.123015873015873, 0.12790697674418605, 0.12878787878787878, 0.1388888888888889, 0.14855072463768115, 0.1595744680851065, 0.17187499999999997, 0.18197278911564627, 0.19166666666666668]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1004566210045662, 0.10045662100456626, 0.10045662100456627, 0.10045662100456627, 0.10045662100456629, 0.10045662100456629, 0.10045662100456629, 0.1004566210045663, 0.1004566210045663, 0.1004566210045663, 0.1004566210045663, 0.1004566210045663, 0.1004566210045663, 0.1004566210045662, 0.1004566210045661, 0.100456621004566, 0.10045662100456591, 0.10045662100456584, 0.10045662100456577, 0.1004566210045657, 0.10045662100456565, 0.10045662100456561, 0.10045662100456555, 0.10045662100456551, 0.10045662100456547, 0.10045662100456544, 0.10045662100456546, 0.10045662100456555, 0.10045662100456564, 0.10045662100456572, 0.1004566210045658, 0.10045662100456587, 0.10045662100456594, 0.100456621004566, 0.10045662100456607, 0.10045662100456612, 0.10460460460460451, 0.1125730994152045, 0.12013295346628657, 0.12731481481481452, 0.1341463414634143, 0.1406525573192236, 0.146856158484065, 0.15248521803399806, 0.15776874435410979, 0.16282255213856445, 0.16886524822695, 0.1770138888888886, 0.18488824101068968, 0.19166666666666635]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.13333333333333333, 0.13888888888888887, 0.1666666666666665, 0.16666666666666666, 0.14814814814814814, 0.15000000000000002, 0.16666666666666666, 0.16666666666666666, 0.1794871794871795, 0.18452380952380953, 0.18888888888888886, 0.17708333333333334, 0.1715686274509804, 0.16666666666666663, 0.15789473684210525, 0.15, 0.1507936507936509, 0.16666666666666666, 0.16304347826086957, 0.16666666666666663, 0.16333333333333333, 0.16666666666666666, 0.16666666666666666, 0.16964285714285715, 0.16954022988505743, 0.16944444444444445, 0.17473118279569894, 0.18229166666666666, 0.18686868686868688, 0.18382352941176472, 0.1857142857142857, 0.18287037037037035, 0.18693693693693694, 0.18421052631578946, 0.18803418803418803, 0.18541666666666667, 0.19105691056910568, 0.1884920634920636, 0.18604651162790697, 0.18371212121212122, 0.18333333333333332, 0.18297101449275363, 0.18794326241134748, 0.18923611111111108, 0.19047619047619047, 0.19166666666666668]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.16666666666666666, 0.1875, 0.15, 0.15277777777777787, 0.1547619047619048, 0.14583333333333334, 0.16666666666666666, 0.17500000000000002, 0.1893939393939394, 0.19444444444444445, 0.1858974358974359, 0.18452380952380953, 0.17777777777777776, 0.17708333333333334, 0.17647058823529413, 0.18055555555555552, 0.17543859649122806, 0.17083333333333334, 0.1706349206349206, 0.17803030303030304, 0.17391304347826086, 0.18402777777777776, 0.19, 0.19230769230769232, 0.19753086419753085, 0.19642857142857142, 0.19252873563218387, 0.19444444444444445, 0.19086021505376344, 0.19010416666666666, 0.1893939393939394, 0.18627450980392157, 0.1833333333333333, 0.1851851851851853, 0.18693693693693694, 0.18640350877192982, 0.1858974358974359, 0.18333333333333332, 0.18089430894308955, 0.18055555555555552, 0.18023255813953487, 0.18181818181818182, 0.18333333333333332, 0.18659420289855072, 0.18971631205673756, 0.18923611111111108, 0.1870748299319728, 0.19166666666666668]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.20833333333333334, 0.1388888888888889, 0.20833333333333334, 0.26666666666666666, 0.24999999999999997, 0.23809523809523814, 0.23958333333333334, 0.2222222222222222, 0.2416666666666667, 0.23484848484848486, 0.2222222222222222, 0.23076923076923078, 0.23214285714285715, 0.23333333333333328, 0.22395833333333334, 0.22549019607843138, 0.2314814814814816, 0.23684210526315788, 0.25, 0.2380952380952382, 0.24242424242424243, 0.2391304347826087, 0.23263888888888887, 0.23, 0.22756410256410256, 0.2191358024691358, 0.21428571428571427, 0.2097701149425287, 0.2111111111111111, 0.20698924731182797, 0.2109375, 0.20707070707070707, 0.2034313725490196, 0.20238095238095236, 0.19907407407407404, 0.20270270270270271, 0.20394736842105263, 0.202991452991453, 0.20208333333333334, 0.20121951219512194, 0.20039682539682552, 0.1996124031007752, 0.20265151515151514, 0.2074074074074074, 0.20471014492753623, 0.20390070921985812, 0.19965277777777776, 0.195578231292517, 0.19166666666666668]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.13266301279136802, "coverage": 0.62, "abstain_rate": 0.38, "selective_risk": 0.06989247311827956, "selective_accuracy": 0.9301075268817205, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.06989247311827956, 0.10480349344978165, 0.12927756653992395, 0.19166666666666668], "coverage": [0.0, 0.0, 0.62, 0.7633333333333333, 0.8766666666666667, 1.0]}, "ece_geometry": 0.023784750979134787}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "7f822ee20bdb", "timestamp": "2026-07-25T10:22:18.866530+00:00", "git_sha": "758d5f2", "config_hash": "9237080427a6", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 8, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 8, "accuracy_id": 0.875, "base_error": 0.125, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.01989117381283462, "rho_basin": 0.050917071654170226, "energy_min": 0.05005538716211986, "energy_mean": 0.054835843985386316, "energy_std": 0.1257274374034565}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.03016421334928524, 0.016179264983221718, 0.04545631751718017], "delta_aurc_vs_best_energy": [0.03016421334928524, 0.016179264983221718, 0.04545631751718017], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0025252525252525255, 0.007352941176470588, 0.0071428571428573, 0.016203703703703855, 0.02702702702702703, 0.03070175438596491, 0.038461538461538464, 0.04583333333333333, 0.048780487804878044, 0.05555555555555568, 0.06782945736434108, 0.07954545454545454, 0.08703703703703704, 0.09239130434782608, 0.09574468085106394, 0.10416666666666677, 0.11394557823129252, 0.125]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.04185022026431718, 0.041850220264317235, 0.041850220264317284, 0.04185022026431733, 0.04185022026431738, 0.04185022026431742, 0.042694627192982726, 0.047609508547008815, 0.052278645833333595, 0.05672002032520353, 0.06094990079365106, 0.06498304263565917, 0.07409460458240977, 0.08383017163505005, 0.0931424531636625, 0.10248226950354662, 0.11201388888888945, 0.119897959183674, 0.12500000000000053]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.025000000000000005, 0.03787878787878788, 0.04861111111111111, 0.04487179487179487, 0.05357142857142857, 0.049999999999999996, 0.046875, 0.049019607843137254, 0.04629629629629629, 0.043859649122807015, 0.041666666666666664, 0.05158730158730158, 0.04924242424242424, 0.04710144927536232, 0.04513888888888888, 0.04666666666666667, 0.04487179487179487, 0.04938271604938271, 0.050595238095238096, 0.054597701149425276, 0.05555555555555555, 0.053763440860215055, 0.0546875, 0.05303030303030303, 0.05392156862745098, 0.054761904761904755, 0.055555555555555546, 0.05405405405405406, 0.05701754385964912, 0.05555555555555555, 0.058333333333333334, 0.0630081300813008, 0.06944444444444443, 0.07170542635658915, 0.08143939393939394, 0.08703703703703704, 0.09601449275362318, 0.10460992907801417, 0.11458333333333331, 0.12414965986394558, 0.125]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.027777777777777887, 0.03571428571428572, 0.041666666666666664, 0.05555555555555555, 0.05833333333333334, 0.05303030303030303, 0.04861111111111111, 0.04487179487179487, 0.041666666666666664, 0.03888888888888888, 0.041666666666666664, 0.04411764705882353, 0.04166666666666681, 0.043859649122807015, 0.041666666666666664, 0.04365079365079364, 0.045454545454545456, 0.04710144927536232, 0.04513888888888888, 0.04666666666666667, 0.04807692307692308, 0.04938271604938271, 0.050595238095238096, 0.048850574712643674, 0.05, 0.04838709677419355, 0.049479166666666664, 0.050505050505050504, 0.051470588235294115, 0.049999999999999996, 0.053240740740740734, 0.0518018018018018, 0.05482456140350877, 0.05555555555555555, 0.058333333333333334, 0.058943089430894303, 0.06746031746031744, 0.07364341085271318, 0.08522727272727272, 0.08888888888888889, 0.09601449275362318, 0.10460992907801417, 0.11111111111111109, 0.11734693877551021, 0.125]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.16666666666666666, 0.1388888888888889, 0.14583333333333334, 0.13333333333333333, 0.12499999999999999, 0.11904761904761907, 0.11458333333333333, 0.10185185185185185, 0.09166666666666667, 0.09848484848484848, 0.09027777777777778, 0.10256410256410256, 0.10119047619047619, 0.09999999999999999, 0.09895833333333333, 0.10784313725490197, 0.10185185185185183, 0.10087719298245613, 0.10416666666666667, 0.10714285714285725, 0.11742424242424243, 0.12318840579710146, 0.12499999999999999, 0.13, 0.1282051282051282, 0.12962962962962962, 0.12797619047619047, 0.1264367816091954, 0.125, 0.12365591397849462, 0.125, 0.12626262626262627, 0.12745098039215685, 0.12857142857142853, 0.1296296296296296, 0.12612612612612611, 0.125, 0.12179487179487179, 0.12291666666666666, 0.12804878048780485, 0.12698412698412695, 0.12403100775193798, 0.12310606060606061, 0.1259259259259259, 0.12318840579710146, 0.12411347517730495, 0.12326388888888888, 0.12244897959183673, 0.125]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.3845914746549172, "coverage": 0.8766666666666667, "abstain_rate": 0.12333333333333334, "selective_risk": 0.07984790874524715, "selective_accuracy": 0.9201520912547528, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.009501187648456057, 0.07984790874524715, 0.12227805695142378, 0.125, 0.125], "coverage": [0.0, 0.7016666666666667, 0.8766666666666667, 0.995, 1.0, 1.0]}, "ece_geometry": 0.06916965670563589}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "e7cfb7fd624f", "timestamp": "2026-07-25T10:22:21.340582+00:00", "git_sha": "758d5f2", "config_hash": "6cacd4aa98fa", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 8, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 8, "accuracy_id": 0.8116666666666666, "base_error": 0.18833333333333332, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.08034697964353028, "rho_basin": 0.0951891189791025, "energy_min": 0.24649230968809713, "energy_mean": 0.2506627437960276, "energy_std": 0.20867825407201335}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.16614533004456683, 0.12121799646397864, 0.21014859172134256], "delta_aurc_vs_best_energy": [0.12833127442848308, 0.08363487013398609, 0.170404467917797], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.06666666666666667, 0.06944444444444443, 0.059523809523809534, 0.0625, 0.05555555555555555, 0.05000000000000001, 0.05303030303030303, 0.05555555555555555, 0.05128205128205128, 0.05357142857142857, 0.049999999999999996, 0.046875, 0.04411764705882353, 0.04166666666666666, 0.043859649122807015, 0.041666666666666664, 0.03968253968253967, 0.041666666666666664, 0.043478260869565216, 0.048611111111111105, 0.05, 0.04807692307692308, 0.04938271604938271, 0.050595238095238096, 0.051724137931034475, 0.05555555555555555, 0.053763440860215055, 0.052083333333333336, 0.05555555555555555, 0.0661764705882353, 0.07142857142857141, 0.07870370370370385, 0.08108108108108109, 0.08991228070175439, 0.10042735042735043, 0.11041666666666666, 0.1239837398373985, 0.1329365079365079, 0.1375968992248062, 0.14583333333333334, 0.1574074074074074, 0.16847826086956522, 0.17198581560283685, 0.17708333333333331, 0.17857142857142858, 0.18833333333333332]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08048780487804877, 0.08048780487804881, 0.08048780487804881, 0.08048780487804882, 0.08048780487804875, 0.08048780487804869, 0.08048780487804866, 0.08048780487804862, 0.0804878048780486, 0.08048780487804859, 0.08048780487804857, 0.08048780487804856, 0.08048780487804855, 0.08048780487804853, 0.08048780487804853, 0.08048780487804852, 0.08048780487804856, 0.08048780487804866, 0.08048780487804874, 0.08048780487804881, 0.08048780487804888, 0.08048780487804894, 0.08048780487804899, 0.08048780487804903, 0.08048780487804909, 0.08048780487804913, 0.08048780487804917, 0.0804878048780492, 0.08048780487804924, 0.08048780487804927, 0.0804878048780493, 0.08048780487804932, 0.08048780487804935, 0.08048780487804928, 0.08475572047000673, 0.08961640211640272, 0.09421434421434484, 0.09857028935976371, 0.10270285270285341, 0.10662878787878863, 0.11234294161123509, 0.12049062049062123, 0.12825933756166383, 0.1356749311294772, 0.1446054750402582, 0.15374921235034703, 0.16250385445575127, 0.17089371980676363, 0.17975451049985228, 0.18833333333333335]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.75, 0.5833333333333334, 0.4722222222222222, 0.4166666666666667, 0.4, 0.37499999999999994, 0.3452380952380953, 0.3541666666666667, 0.37037037037037035, 0.34166666666666673, 0.3181818181818182, 0.2916666666666667, 0.27564102564102566, 0.2619047619047619, 0.2555555555555555, 0.23958333333333334, 0.23039215686274508, 0.21759259259259256, 0.2149122807017544, 0.2125, 0.21825396825396823, 0.20833333333333334, 0.20652173913043478, 0.19791666666666663, 0.19666666666666666, 0.1891025641025641, 0.18209876543209877, 0.17857142857142858, 0.17816091954022986, 0.175, 0.1774193548387097, 0.1796875, 0.17676767676767677, 0.17401960784313725, 0.1714285714285714, 0.16898148148148162, 0.16666666666666666, 0.16228070175438597, 0.1623931623931624, 0.1625, 0.16260162601626013, 0.1607142857142857, 0.15891472868217055, 0.16098484848484848, 0.16111111111111112, 0.16304347826086957, 0.16843971631205684, 0.17881944444444442, 0.1836734693877551, 0.18833333333333332]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5833333333333334, 0.4583333333333333, 0.4444444444444444, 0.4791666666666667, 0.43333333333333335, 0.41666666666666663, 0.3928571428571429, 0.375, 0.3611111111111111, 0.33333333333333337, 0.3106060606060606, 0.2986111111111111, 0.27564102564102566, 0.26785714285714285, 0.2666666666666668, 0.2552083333333333, 0.25, 0.24074074074074084, 0.2324561403508772, 0.225, 0.2261904761904763, 0.22727272727272727, 0.2210144927536232, 0.21527777777777787, 0.21, 0.20512820512820512, 0.19753086419753085, 0.19345238095238096, 0.18965517241379307, 0.18611111111111112, 0.1881720430107527, 0.18489583333333334, 0.1893939393939394, 0.18382352941176472, 0.17857142857142855, 0.1782407407407407, 0.17342342342342343, 0.16885964912280702, 0.16452991452991453, 0.1625, 0.16260162601626013, 0.1587301587301587, 0.16085271317829458, 0.1571969696969697, 0.1574074074074074, 0.16485507246376813, 0.17021276595744692, 0.1753472222222223, 0.18537414965986396, 0.18833333333333332]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.2777777777777778, 0.20833333333333334, 0.2, 0.16666666666666674, 0.1785714285714286, 0.17708333333333334, 0.19444444444444445, 0.2166666666666667, 0.2196969696969697, 0.2222222222222222, 0.20512820512820512, 0.21428571428571427, 0.2277777777777779, 0.22916666666666666, 0.22058823529411764, 0.2175925925925927, 0.21929824561403508, 0.2125, 0.21031746031746043, 0.20833333333333334, 0.21739130434782608, 0.20833333333333331, 0.21333333333333335, 0.21153846153846154, 0.21296296296296297, 0.20833333333333334, 0.2126436781609195, 0.20833333333333334, 0.20161290322580644, 0.203125, 0.20454545454545456, 0.20098039215686275, 0.19761904761904775, 0.2037037037037038, 0.20045045045045046, 0.19736842105263158, 0.1987179487179487, 0.19375, 0.19512195121951217, 0.1964285714285714, 0.1996124031007752, 0.19696969696969696, 0.1925925925925926, 0.19384057971014493, 0.19148936170212763, 0.18749999999999997, 0.1870748299319728, 0.18833333333333332]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.12764229887170703, "coverage": 0.6583333333333333, "abstain_rate": 0.3416666666666667, "selective_risk": 0.05569620253164557, "selective_accuracy": 0.9443037974683545, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.05569620253164557, 0.1285140562248996, 0.18833333333333332, 0.18833333333333332], "coverage": [0.0, 0.0, 0.6583333333333333, 0.83, 1.0, 1.0]}, "ece_geometry": 0.0645687294351687}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "539e9c3f0431", "timestamp": "2026-07-25T10:22:23.797847+00:00", "git_sha": "758d5f2", "config_hash": "1be43f267351", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 8, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 8, "accuracy_id": 0.82, "base_error": 0.18, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.08347985060453421, "rho_basin": 0.10270248698021288, "energy_min": 0.14577277460843363, "energy_mean": 0.14269046297025517, "energy_std": 0.20344554572241264}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.06229292400389942, 0.027442911103842102, 0.09493409199597845], "delta_aurc_vs_best_energy": [0.05921061236572096, 0.024789655786357095, 0.09176410606591125], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.05555555555555555, 0.08333333333333333, 0.06666666666666667, 0.06944444444444443, 0.07142857142857144, 0.0625, 0.05555555555555555, 0.06666666666666668, 0.06818181818181818, 0.0763888888888889, 0.07692307692307693, 0.07142857142857142, 0.07777777777777777, 0.078125, 0.07352941176470588, 0.08333333333333347, 0.08333333333333333, 0.07916666666666666, 0.07936507936507935, 0.07575757575757576, 0.07246376811594203, 0.06944444444444443, 0.06666666666666667, 0.0641025641025641, 0.06481481481481481, 0.06547619047619048, 0.0632183908045977, 0.06111111111111111, 0.05913978494623656, 0.059895833333333336, 0.06313131313131314, 0.0661764705882353, 0.06666666666666665, 0.06944444444444459, 0.07657657657657657, 0.07894736842105263, 0.07905982905982906, 0.0875, 0.09756097560975623, 0.11111111111111122, 0.12209302325581395, 0.13257575757575757, 0.14444444444444443, 0.15217391304347827, 0.15780141843971643, 0.16666666666666674, 0.17517006802721088, 0.18]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0921052631578947, 0.0921052631578947, 0.0921052631578947, 0.0921052631578947, 0.0921052631578947, 0.0921052631578947, 0.0921052631578947, 0.09210526315789479, 0.09210526315789487, 0.09210526315789494, 0.092105263157895, 0.09210526315789505, 0.09210526315789509, 0.09210526315789512, 0.09210526315789508, 0.09210526315789501, 0.09210526315789494, 0.09210526315789487, 0.09210526315789481, 0.09210526315789476, 0.09210526315789472, 0.09210526315789468, 0.09210526315789463, 0.0921052631578946, 0.09210526315789457, 0.09210526315789454, 0.09210526315789451, 0.0921052631578945, 0.09210526315789448, 0.09210526315789457, 0.09210526315789466, 0.09210526315789475, 0.09210526315789483, 0.0921052631578949, 0.09210526315789497, 0.09210526315789504, 0.09210526315789509, 0.09210526315789516, 0.09935897435897477, 0.1062500000000004, 0.11280487804878092, 0.11904761904761949, 0.12764857881136987, 0.13585858585858634, 0.1437037037037043, 0.1513504611330705, 0.1590909090909098, 0.16650883838383906, 0.1734693877551027, 0.18000000000000063]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.0625, 0.1, 0.138888888888889, 0.14285714285714288, 0.13541666666666666, 0.12962962962962962, 0.12500000000000003, 0.12121212121212122, 0.1388888888888889, 0.16025641025641027, 0.15476190476190477, 0.14999999999999997, 0.15104166666666666, 0.14705882352941177, 0.15277777777777776, 0.15350877192982457, 0.1625, 0.16666666666666677, 0.16666666666666666, 0.17028985507246377, 0.16319444444444442, 0.15666666666666668, 0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.16379310344827583, 0.16111111111111112, 0.1639784946236559, 0.1640625, 0.16414141414141414, 0.1642156862745098, 0.15952380952380965, 0.15740740740740738, 0.1554054054054054, 0.15570175438596492, 0.15384615384615385, 0.15208333333333332, 0.15243902439024387, 0.15277777777777776, 0.15310077519379844, 0.1553030303030303, 0.16111111111111112, 0.16666666666666666, 0.17021276595744678, 0.16666666666666674, 0.17687074829931973, 0.18]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.11111111111111109, 0.10714285714285716, 0.14583333333333334, 0.1388888888888889, 0.13333333333333336, 0.12878787878787878, 0.125, 0.1346153846153846, 0.13690476190476192, 0.1444444444444444, 0.15104166666666666, 0.15196078431372548, 0.16203703703703717, 0.17105263157894737, 0.17083333333333334, 0.17460317460317457, 0.16666666666666666, 0.17028985507246377, 0.16666666666666663, 0.16333333333333333, 0.16346153846153846, 0.16358024691358025, 0.1636904761904762, 0.16091954022988503, 0.15833333333333333, 0.16129032258064516, 0.1640625, 0.1590909090909091, 0.1568627450980392, 0.1571428571428571, 0.15277777777777776, 0.14864864864864866, 0.14692982456140352, 0.14743589743589744, 0.14583333333333334, 0.1463414634146341, 0.1468253968253968, 0.15310077519379844, 0.1553030303030303, 0.1574074074074074, 0.16304347826086957, 0.1648936170212767, 0.17013888888888887, 0.1717687074829932, 0.18]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.25, 0.2222222222222222, 0.20833333333333334, 0.21666666666666667, 0.19444444444444442, 0.1785714285714286, 0.1875, 0.17592592592592593, 0.16666666666666669, 0.18181818181818182, 0.1875, 0.1794871794871795, 0.18452380952380953, 0.1833333333333333, 0.1875, 0.19117647058823528, 0.19444444444444442, 0.19298245614035087, 0.2, 0.20238095238095236, 0.19696969696969696, 0.19927536231884058, 0.20138888888888887, 0.19666666666666666, 0.1955128205128205, 0.19753086419753085, 0.19345238095238096, 0.1982758620689655, 0.19722222222222222, 0.1935483870967742, 0.1953125, 0.1919191919191919, 0.19117647058823528, 0.19285714285714284, 0.1921296296296296, 0.1891891891891892, 0.18640350877192982, 0.18162393162393162, 0.18125, 0.18495934959349605, 0.18849206349206346, 0.187984496124031, 0.18371212121212122, 0.1814814814814815, 0.18840579710144928, 0.18794326241134748, 0.18576388888888887, 0.1836734693877551, 0.18]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.09670781893004116, 0.17676767676767677, 0.18], "coverage": [0.0, 0.0, 0.0, 0.81, 0.99, 1.0]}, "ece_geometry": 0.045387404661788915}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "8d6b3f5b24a2", "timestamp": "2026-07-25T10:22:31.600594+00:00", "git_sha": "758d5f2", "config_hash": "3c20300a5e78", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 16, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 16, "accuracy_id": 0.81, "base_error": 0.19, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.08336533162010568, "rho_basin": 0.12910368375196765, "energy_min": 0.16182448173609607, "energy_mean": 0.1604931265797101, "energy_std": 0.2103839703983656}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.07845915011599039, 0.04419008664232815, 0.11481453166442503], "delta_aurc_vs_best_energy": [0.07712779495960442, 0.043276310244538756, 0.11459298263204881], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.041666666666666664, 0.08333333333333333, 0.08333333333333331, 0.07142857142857144, 0.0625, 0.05555555555555555, 0.05833333333333334, 0.06060606060606061, 0.06944444444444445, 0.0641025641025641, 0.05952380952380952, 0.061111111111111095, 0.0625, 0.058823529411764705, 0.0555555555555557, 0.05701754385964912, 0.05416666666666667, 0.05158730158730158, 0.06439393939393939, 0.06521739130434782, 0.06944444444444443, 0.07, 0.0673076923076923, 0.06481481481481481, 0.06547619047619048, 0.0718390804597701, 0.07222222222222222, 0.07526881720430108, 0.07552083333333333, 0.08585858585858586, 0.09313725490196079, 0.09761904761904776, 0.10185185185185183, 0.10135135135135136, 0.10087719298245613, 0.1047008547008547, 0.11458333333333333, 0.1219512195121951, 0.13492063492063489, 0.13953488372093023, 0.14583333333333334, 0.15, 0.15942028985507245, 0.16666666666666677, 0.1753472222222223, 0.18537414965986396, 0.19]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1201923076923077, 0.12019230769230764, 0.12019230769230758, 0.12019230769230756, 0.12019230769230754, 0.12019230769230761, 0.12019230769230772, 0.1201923076923078, 0.12019230769230786, 0.12019230769230792, 0.12019230769230796, 0.12019230769230786, 0.12019230769230776, 0.12019230769230768, 0.12019230769230761, 0.12019230769230754, 0.12019230769230749, 0.12019230769230743, 0.12019230769230739, 0.12019230769230735, 0.12019230769230732, 0.12019230769230728, 0.12019230769230727, 0.12019230769230724, 0.12019230769230721, 0.1201923076923072, 0.12019230769230717, 0.12019230769230715, 0.12019230769230713, 0.12019230769230711, 0.1201923076923071, 0.12019230769230708, 0.12019230769230707, 0.12019230769230706, 0.1211822660098516, 0.12404214559386917, 0.12674743709226416, 0.12931034482758572, 0.13174182139699336, 0.13428030303030264, 0.13691796008869161, 0.13943001443001438, 0.14437984496124026, 0.1509387351778655, 0.16014492753623166, 0.16938405797101425, 0.17907801418439692, 0.18129960317460306, 0.1832096474953616, 0.1899999999999998]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.20833333333333334, 0.19444444444444445, 0.16666666666666666, 0.15, 0.15277777777777776, 0.14285714285714288, 0.13541666666666666, 0.12962962962962962, 0.11666666666666668, 0.13636363636363635, 0.14583333333333334, 0.14102564102564102, 0.14285714285714285, 0.14999999999999997, 0.14583333333333334, 0.1568627450980392, 0.16203703703703717, 0.16228070175438597, 0.16666666666666666, 0.16269841269841268, 0.1590909090909091, 0.16304347826086957, 0.16319444444444442, 0.16666666666666666, 0.16025641025641027, 0.15432098765432098, 0.15476190476190477, 0.15804597701149442, 0.16111111111111112, 0.1586021505376344, 0.16145833333333334, 0.16161616161616163, 0.16176470588235295, 0.15952380952380948, 0.15972222222222218, 0.15990990990990991, 0.15789473684210525, 0.16025641025641027, 0.16458333333333333, 0.1686991869918699, 0.16666666666666663, 0.16666666666666666, 0.16856060606060605, 0.17407407407407408, 0.17753623188405798, 0.18085106382978722, 0.18229166666666663, 0.18537414965986396, 0.19]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.08333333333333333, 0.11666666666666667, 0.1111111111111112, 0.1547619047619048, 0.15625, 0.16666666666666666, 0.15833333333333335, 0.1590909090909091, 0.1597222222222222, 0.16025641025641027, 0.16071428571428573, 0.16666666666666663, 0.17708333333333334, 0.18137254901960784, 0.1898148148148148, 0.18859649122807018, 0.1875, 0.17857142857142855, 0.1856060606060606, 0.18115942028985507, 0.17708333333333331, 0.17, 0.16987179487179488, 0.1697530864197531, 0.16666666666666666, 0.16954022988505743, 0.16666666666666666, 0.1693548387096774, 0.16666666666666666, 0.16414141414141414, 0.15931372549019607, 0.15952380952380948, 0.15972222222222218, 0.15765765765765766, 0.15789473684210525, 0.16025641025641027, 0.15833333333333333, 0.16056910569105703, 0.1646825396825398, 0.17054263565891473, 0.17045454545454544, 0.17037037037037037, 0.1793478260869565, 0.18085106382978733, 0.18402777777777787, 0.1870748299319728, 0.19]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.3333333333333333, 0.2777777777777778, 0.2916666666666667, 0.26666666666666666, 0.27777777777777773, 0.261904761904762, 0.2604166666666667, 0.23148148148148148, 0.23333333333333336, 0.22727272727272727, 0.22916666666666666, 0.23076923076923078, 0.22023809523809523, 0.20555555555555552, 0.19791666666666666, 0.20098039215686275, 0.19907407407407404, 0.20175438596491227, 0.19583333333333333, 0.20238095238095236, 0.19696969696969696, 0.1956521739130435, 0.19791666666666663, 0.19666666666666666, 0.1955128205128205, 0.19753086419753085, 0.19345238095238096, 0.1982758620689655, 0.19444444444444445, 0.19086021505376344, 0.1875, 0.1893939393939394, 0.18872549019607843, 0.1857142857142857, 0.18055555555555552, 0.18243243243243243, 0.18201754385964913, 0.18376068376068377, 0.17916666666666667, 0.17682926829268303, 0.17658730158730157, 0.1821705426356589, 0.18371212121212122, 0.18518518518518517, 0.18478260869565216, 0.1861702127659574, 0.18749999999999997, 0.18877551020408162, 0.19]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.1150395352560259, "coverage": 0.605, "abstain_rate": 0.395, "selective_risk": 0.0743801652892562, "selective_accuracy": 0.9256198347107438, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0743801652892562, 0.12348178137651822, 0.1793103448275862, 0.19], "coverage": [0.0, 0.0, 0.605, 0.8233333333333334, 0.9666666666666667, 1.0]}, "ece_geometry": 0.044446663263318587}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "b5b8cfd55dee", "timestamp": "2026-07-25T10:22:34.276837+00:00", "git_sha": "758d5f2", "config_hash": "22b25365d745", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 16, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 16, "accuracy_id": 0.8233333333333334, "base_error": 0.17666666666666667, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.05865890740035407, "rho_basin": 0.09299699257461479, "energy_min": 0.1752275799430285, "energy_mean": 0.17761219400545514, "energy_std": 0.18469530607274298}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.11656867254267445, 0.07769454277240975, 0.15816209355513938], "delta_aurc_vs_best_energy": [0.11656867254267445, 0.07769454277240975, 0.15816209355513938], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.010416666666666666, 0.009259259259259259, 0.01666666666666667, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.02976190476190476, 0.027777777777777773, 0.03125, 0.03431372549019608, 0.03703703703703703, 0.039473684210526314, 0.0375, 0.03968253968253967, 0.041666666666666664, 0.03985507246376811, 0.03819444444444444, 0.043333333333333335, 0.04487179487179487, 0.04938271604938271, 0.047619047619047616, 0.054597701149425276, 0.058333333333333334, 0.05913978494623656, 0.0625, 0.06565656565656566, 0.07107843137254902, 0.07619047619047618, 0.07638888888888888, 0.08108108108108109, 0.08552631578947369, 0.09188034188034189, 0.08958333333333333, 0.09146341463414633, 0.09523809523809536, 0.10852713178294573, 0.125, 0.13333333333333333, 0.14130434782608695, 0.14716312056737585, 0.154513888888889, 0.1649659863945578, 0.17666666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.07932692307692309, 0.07932692307692311, 0.07932692307692306, 0.07932692307692302, 0.07932692307692307, 0.07932692307692311, 0.07932692307692316, 0.07932692307692317, 0.07932692307692318, 0.0793269230769232, 0.07932692307692321, 0.07932692307692323, 0.07932692307692324, 0.07932692307692324, 0.07932692307692325, 0.07932692307692325, 0.07932692307692327, 0.07932692307692327, 0.07932692307692327, 0.07932692307692328, 0.07932692307692328, 0.07932692307692328, 0.07932692307692328, 0.0793269230769233, 0.0793269230769233, 0.0793269230769233, 0.0793269230769233, 0.0793269230769233, 0.0793269230769233, 0.0793269230769233, 0.07932692307692331, 0.07932692307692331, 0.07932692307692331, 0.07932692307692327, 0.08121693121693137, 0.08667695473251046, 0.091841841841842, 0.09673489278752455, 0.1013770180436849, 0.10604838709677436, 0.11054287962234477, 0.11507936507936524, 0.12015503875969011, 0.1300837320574165, 0.14064327485380146, 0.14788329519450827, 0.15602836879432652, 0.16468253968253993, 0.1725417439703156, 0.17666666666666678]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.16666666666666666, 0.19444444444444445, 0.16666666666666666, 0.18333333333333332, 0.20833333333333343, 0.2023809523809524, 0.1875, 0.18518518518518517, 0.18333333333333335, 0.17424242424242425, 0.16666666666666666, 0.15384615384615385, 0.16666666666666666, 0.1611111111111111, 0.16666666666666666, 0.1715686274509804, 0.1759259259259259, 0.17543859649122806, 0.18333333333333332, 0.1825396825396825, 0.18181818181818182, 0.18478260869565216, 0.18055555555555552, 0.18666666666666668, 0.1891025641025641, 0.18518518518518517, 0.19047619047619047, 0.18390804597701146, 0.18611111111111112, 0.1827956989247312, 0.18229166666666666, 0.18181818181818182, 0.18137254901960784, 0.17857142857142855, 0.18287037037037035, 0.18018018018018017, 0.17982456140350878, 0.17735042735042736, 0.17291666666666666, 0.17479674796747965, 0.17857142857142855, 0.17635658914728683, 0.17424242424242425, 0.17222222222222222, 0.16847826086956522, 0.16666666666666677, 0.17013888888888887, 0.17517006802721088, 0.17666666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.16666666666666666, 0.1875, 0.15, 0.18055555555555564, 0.2023809523809524, 0.19791666666666666, 0.18518518518518517, 0.17500000000000002, 0.16666666666666666, 0.1736111111111111, 0.1794871794871795, 0.17261904761904762, 0.16666666666666663, 0.171875, 0.18627450980392157, 0.1898148148148148, 0.19298245614035087, 0.19166666666666668, 0.19444444444444442, 0.19696969696969696, 0.18840579710144928, 0.18055555555555552, 0.19, 0.1891025641025641, 0.1882716049382716, 0.18154761904761904, 0.18103448275862086, 0.17777777777777778, 0.1774193548387097, 0.18229166666666666, 0.17929292929292928, 0.17892156862745098, 0.17619047619047618, 0.17361111111111108, 0.16891891891891891, 0.16885964912280702, 0.16452991452991453, 0.16666666666666666, 0.1686991869918699, 0.17460317460317457, 0.1744186046511628, 0.17613636363636365, 0.17222222222222222, 0.1721014492753623, 0.17375886524822692, 0.1736111111111112, 0.17517006802721088, 0.17666666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.16666666666666666, 0.20833333333333334, 0.21666666666666667, 0.2222222222222223, 0.22619047619047625, 0.23958333333333334, 0.23148148148148148, 0.2166666666666667, 0.2196969696969697, 0.20833333333333334, 0.20512820512820512, 0.19642857142857142, 0.18888888888888886, 0.1875, 0.18627450980392157, 0.1898148148148148, 0.19298245614035087, 0.19166666666666668, 0.19841269841269837, 0.19318181818181818, 0.18840579710144928, 0.18749999999999997, 0.18666666666666668, 0.1858974358974359, 0.18209876543209877, 0.18154761904761904, 0.18103448275862066, 0.17777777777777778, 0.1774193548387097, 0.171875, 0.1717171717171717, 0.16911764705882354, 0.1714285714285714, 0.1759259259259259, 0.17342342342342343, 0.17324561403508773, 0.17094017094017094, 0.175, 0.17479674796747965, 0.17658730158730157, 0.1744186046511628, 0.17424242424242425, 0.17592592592592593, 0.17572463768115942, 0.17375886524822706, 0.17708333333333331, 0.17857142857142858, 0.17666666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.12092417903770743, "coverage": 0.5616666666666666, "abstain_rate": 0.43833333333333335, "selective_risk": 0.04747774480712166, "selective_accuracy": 0.9525222551928784, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.04747774480712166, 0.08149779735682819, 0.14, 0.17666666666666667], "coverage": [0.0, 0.0, 0.5616666666666666, 0.7566666666666667, 0.9166666666666666, 1.0]}, "ece_geometry": 0.04551762209694324}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "1a2ccd4ef091", "timestamp": "2026-07-25T10:22:36.950705+00:00", "git_sha": "758d5f2", "config_hash": "b6cea5e21b36", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 16, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 16, "accuracy_id": 0.8933333333333333, "base_error": 0.10666666666666667, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0182813747536626, "rho_basin": 0.03499570395922966, "energy_min": 0.04018393233543402, "energy_mean": 0.03806428706561064, "energy_std": 0.10758910107766798}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.02190255758177142, 0.00887645534331844, 0.039644103744765444], "delta_aurc_vs_best_energy": [0.019782912311948037, 0.007700839628327744, 0.035539667058161535], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.002976190476190476, 0.002873563218390804, 0.005555555555555556, 0.005376344086021506, 0.005208333333333333, 0.010101010101010102, 0.012254901960784314, 0.014285714285714284, 0.023148148148148143, 0.02702702702702703, 0.03289473684210526, 0.03632478632478633, 0.03958333333333333, 0.046747967479674926, 0.055555555555555546, 0.060077519379844964, 0.06439393939393939, 0.07037037037037037, 0.07608695652173914, 0.08333333333333331, 0.08680555555555554, 0.0935374149659864, 0.10666666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.023584905660377357, 0.023584905660377364, 0.023584905660377378, 0.023584905660377384, 0.02358490566037739, 0.023584905660377395, 0.023584905660377395, 0.023584905660377395, 0.023584905660377395, 0.0235849056603774, 0.0235849056603774, 0.0235849056603774, 0.023584905660377402, 0.023584905660377402, 0.0235849056603774, 0.0235849056603774, 0.0235849056603774, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377402, 0.023584905660377384, 0.023584905660377353, 0.02358490566037733, 0.0235849056603773, 0.02358490566037728, 0.023584905660377256, 0.023584905660377235, 0.02787628053585491, 0.034023385087214746, 0.039846957820082, 0.04537188579741759, 0.04969362745098029, 0.05350310856049732, 0.05713118580765637, 0.06104651162790692, 0.06506238859180029, 0.07145969498910666, 0.07789855072463758, 0.0842198581560283, 0.09062499999999993, 0.0986394557823128, 0.10666666666666655]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.03333333333333333, 0.027777777777777773, 0.02380952380952381, 0.03125, 0.027777777777777776, 0.025000000000000005, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.017857142857142856, 0.02222222222222222, 0.020833333333333332, 0.024509803921568627, 0.027777777777777773, 0.03508771929824561, 0.03333333333333333, 0.03174603174603174, 0.03787878787878788, 0.036231884057971016, 0.03472222222222222, 0.03666666666666667, 0.035256410256410256, 0.033950617283950615, 0.03273809523809524, 0.037356321839080456, 0.03888888888888889, 0.04032258064516129, 0.041666666666666664, 0.04040404040404041, 0.0392156862745098, 0.04047619047619047, 0.039351851851852, 0.04054054054054054, 0.043859649122807015, 0.04487179487179487, 0.04375, 0.05081300813008129, 0.055555555555555546, 0.0562015503875969, 0.06439393939393939, 0.07037037037037037, 0.07608695652173914, 0.08156028368794326, 0.08506944444444456, 0.09863945578231292, 0.10666666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.027777777777777773, 0.02380952380952381, 0.020833333333333332, 0.027777777777777776, 0.025000000000000005, 0.030303030303030304, 0.027777777777777776, 0.03205128205128205, 0.03571428571428571, 0.033333333333333326, 0.03125, 0.029411764705882353, 0.027777777777777773, 0.02631578947368421, 0.029166666666666667, 0.027777777777777773, 0.026515151515151516, 0.028985507246376812, 0.027777777777777773, 0.02666666666666667, 0.03205128205128205, 0.030864197530864196, 0.03273809523809524, 0.03160919540229885, 0.030555555555555555, 0.03225806451612903, 0.036458333333333336, 0.03535353535353535, 0.03676470588235294, 0.03809523809523809, 0.03703703703703703, 0.038288288288288286, 0.041666666666666664, 0.042735042735042736, 0.04375, 0.05081300813008129, 0.055555555555555546, 0.05813953488372093, 0.0625, 0.06296296296296296, 0.07608695652173914, 0.08333333333333344, 0.09201388888888888, 0.09863945578231292, 0.10666666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.06944444444444456, 0.09523809523809525, 0.10416666666666667, 0.10185185185185185, 0.10833333333333335, 0.09848484848484848, 0.10416666666666667, 0.10256410256410256, 0.10714285714285714, 0.10555555555555554, 0.125, 0.11764705882352941, 0.11111111111111109, 0.11842105263157894, 0.1125, 0.10714285714285712, 0.10606060606060606, 0.11231884057971014, 0.10763888888888888, 0.10666666666666667, 0.10897435897435898, 0.1111111111111111, 0.11011904761904762, 0.11206896551724135, 0.10833333333333334, 0.10752688172043011, 0.109375, 0.11616161616161616, 0.11764705882352941, 0.11666666666666665, 0.11574074074074088, 0.11711711711711711, 0.11403508771929824, 0.11324786324786325, 0.11666666666666667, 0.11585365853658536, 0.11507936507936506, 0.11434108527131782, 0.11553030303030302, 0.11296296296296296, 0.11231884057971014, 0.10992907801418439, 0.10763888888888888, 0.10714285714285714, 0.10666666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.3658091453005846, "coverage": 0.86, "abstain_rate": 0.14, "selective_risk": 0.060077519379844964, "selective_accuracy": 0.939922480620155, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.02320185614849188, 0.060077519379844964, 0.09661016949152543, 0.10666666666666667, 0.10666666666666667], "coverage": [0.0, 0.7183333333333334, 0.86, 0.9833333333333333, 1.0, 1.0]}, "ece_geometry": 0.0566481063377724}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "ff953d4a0e15", "timestamp": "2026-07-25T10:22:39.626580+00:00", "git_sha": "758d5f2", "config_hash": "ef0196b5978e", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 16, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 16, "accuracy_id": 0.84, "base_error": 0.16, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.05432083427879411, "rho_basin": 0.08079406537936447, "energy_min": 0.20385264551720791, "energy_mean": 0.2261103934273237, "energy_std": 0.16354343706354418}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.1495318112384138, 0.10489969702271222, 0.19505166406248708], "delta_aurc_vs_best_energy": [0.10922260278475007, 0.07259535618924547, 0.1475034380697952], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.025000000000000005, 0.022727272727272728, 0.020833333333333332, 0.019230769230769232, 0.017857142857142856, 0.02222222222222222, 0.020833333333333332, 0.0196078431372549, 0.018518518518518514, 0.017543859649122806, 0.025, 0.027777777777777773, 0.026515151515151516, 0.028985507246376812, 0.031249999999999997, 0.03333333333333333, 0.038461538461538464, 0.040123456790123455, 0.041666666666666664, 0.04022988505747126, 0.04722222222222222, 0.04838709677419355, 0.052083333333333336, 0.05555555555555555, 0.056372549019607844, 0.06428571428571443, 0.06944444444444443, 0.07432432432432433, 0.08333333333333333, 0.08547008547008547, 0.08958333333333333, 0.09349593495934957, 0.09722222222222221, 0.10465116279069768, 0.10984848484848485, 0.12037037037037036, 0.12681159420289856, 0.13475177304964536, 0.14583333333333345, 0.15476190476190477, 0.16]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.06486486486486487, 0.06486486486486487, 0.0648648648648649, 0.06486486486486494, 0.06486486486486497, 0.064864864864865, 0.06486486486486501, 0.06486486486486502, 0.06486486486486502, 0.06486486486486503, 0.06486486486486498, 0.06486486486486492, 0.06486486486486487, 0.06486486486486481, 0.06486486486486477, 0.06486486486486474, 0.0648648648648647, 0.06486486486486467, 0.06486486486486466, 0.06486486486486463, 0.06486486486486466, 0.06486486486486472, 0.06486486486486477, 0.06486486486486481, 0.06486486486486487, 0.06486486486486491, 0.06486486486486495, 0.06486486486486498, 0.06486486486486502, 0.06486486486486505, 0.06554019457245285, 0.06944444444444463, 0.07311207311207327, 0.07656395891690022, 0.07981859410430851, 0.08289241622574965, 0.08804898648648657, 0.0931332236842106, 0.09714003944773182, 0.09855769230769236, 0.10054623983739848, 0.10931299603174612, 0.11767199612403105, 0.12436868686868688, 0.12901234567901224, 0.13420893719806753, 0.14021867612293146, 0.1460813492063493, 0.15184645286686113, 0.16000000000000017]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.4166666666666667, 0.3611111111111111, 0.375, 0.4, 0.36111111111111116, 0.3571428571428572, 0.34375, 0.3148148148148148, 0.29166666666666674, 0.2727272727272727, 0.2638888888888889, 0.25, 0.23809523809523808, 0.22222222222222235, 0.21354166666666666, 0.20098039215686275, 0.19444444444444442, 0.19298245614035087, 0.18333333333333332, 0.17857142857142855, 0.17045454545454544, 0.17028985507246377, 0.16666666666666663, 0.16, 0.15705128205128205, 0.1574074074074074, 0.15476190476190477, 0.15229885057471262, 0.15555555555555556, 0.1532258064516129, 0.15104166666666666, 0.14898989898989898, 0.14705882352941177, 0.14999999999999997, 0.1481481481481481, 0.14414414414414414, 0.1425438596491228, 0.1388888888888889, 0.13541666666666666, 0.13211382113821135, 0.1329365079365079, 0.13178294573643412, 0.13257575757575757, 0.13518518518518519, 0.13949275362318841, 0.14361702127659573, 0.14756944444444442, 0.15306122448979592, 0.16]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.5416666666666666, 0.5277777777777778, 0.4583333333333333, 0.4166666666666667, 0.4305555555555555, 0.380952380952381, 0.3541666666666667, 0.3333333333333333, 0.3083333333333334, 0.2878787878787879, 0.2777777777777778, 0.2692307692307692, 0.26785714285714285, 0.2555555555555555, 0.24479166666666666, 0.23039215686274508, 0.21759259259259256, 0.20614035087719298, 0.19583333333333333, 0.19047619047619044, 0.1856060606060606, 0.17753623188405798, 0.17361111111111108, 0.16666666666666666, 0.16346153846153846, 0.16049382716049382, 0.16071428571428573, 0.15804597701149423, 0.15555555555555556, 0.15053763440860216, 0.1484375, 0.14393939393939395, 0.14705882352941177, 0.14285714285714282, 0.14120370370370366, 0.13963963963963963, 0.1425438596491228, 0.1388888888888889, 0.13541666666666666, 0.13414634146341461, 0.1369047619047619, 0.13565891472868216, 0.13257575757575757, 0.13518518518518519, 0.13768115942028986, 0.14184397163120566, 0.14583333333333345, 0.15306122448979592, 0.16]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.2222222222222222, 0.20833333333333334, 0.18333333333333332, 0.18055555555555564, 0.2023809523809524, 0.1875, 0.16666666666666666, 0.15833333333333335, 0.14393939393939395, 0.13194444444444445, 0.16025641025641027, 0.16071428571428573, 0.15555555555555553, 0.15104166666666666, 0.1568627450980392, 0.16666666666666663, 0.16666666666666666, 0.15833333333333333, 0.15476190476190474, 0.1553030303030303, 0.15942028985507245, 0.1597222222222223, 0.17, 0.16346153846153846, 0.16358024691358025, 0.16666666666666666, 0.16379310344827583, 0.15833333333333333, 0.15591397849462366, 0.15625, 0.15404040404040403, 0.15441176470588236, 0.15476190476190474, 0.15046296296296294, 0.15090090090090091, 0.14692982456140352, 0.1452991452991453, 0.14791666666666667, 0.14837398373983737, 0.1468253968253968, 0.14922480620155038, 0.1534090909090909, 0.1537037037037037, 0.15579710144927536, 0.15425531914893614, 0.15451388888888887, 0.15816326530612246, 0.16]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.1946581415068475, "coverage": 0.7366666666666667, "abstain_rate": 0.2633333333333333, "selective_risk": 0.07239819004524888, "selective_accuracy": 0.9276018099547512, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.07239819004524888, 0.10877862595419847, 0.15719063545150502, 0.15719063545150502], "coverage": [0.0, 0.0, 0.7366666666666667, 0.8733333333333333, 0.9966666666666667, 0.9966666666666667]}, "ece_geometry": 0.037936460222854664}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "e14af222ab26", "timestamp": "2026-07-25T10:22:42.386303+00:00", "git_sha": "758d5f2", "config_hash": "9c69bc8a7ce8", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 16, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 16, "accuracy_id": 0.8283333333333334, "base_error": 0.17166666666666666, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.040671709886151576, "rho_basin": 0.07434120685783897, "energy_min": 0.13672077821526255, "energy_mean": 0.13355301436237477, "energy_std": 0.15435131518238096}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.09604906832911098, 0.06634013622922631, 0.1287278562906846], "delta_aurc_vs_best_energy": [0.0928813044762232, 0.06328851651900529, 0.1260992591179057], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.009259259259259259, 0.008333333333333335, 0.007575757575757576, 0.006944444444444444, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.009259259259259257, 0.008771929824561403, 0.008333333333333333, 0.011904761904761902, 0.015151515151515152, 0.018115942028985508, 0.02083333333333333, 0.02, 0.019230769230769232, 0.024691358024691357, 0.023809523809523808, 0.022988505747126433, 0.022222222222222223, 0.024193548387096774, 0.026041666666666668, 0.027777777777777776, 0.041666666666666664, 0.050000000000000155, 0.055555555555555546, 0.06306306306306306, 0.07236842105263158, 0.07478632478632478, 0.08125, 0.08943089430894322, 0.10119047619047632, 0.1124031007751938, 0.11742424242424243, 0.12777777777777777, 0.1322463768115942, 0.1418439716312058, 0.15451388888888887, 0.16156462585034015, 0.17166666666666666]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.055961070559610714, 0.05596107055961073, 0.055961070559610734, 0.055961070559610686, 0.05596107055961065, 0.05596107055961064, 0.05596107055961062, 0.05596107055961061, 0.0559610705596106, 0.055961070559610596, 0.05596107055961059, 0.05596107055961059, 0.05596107055961065, 0.05596107055961071, 0.055961070559610755, 0.0559610705596108, 0.05596107055961083, 0.055961070559610866, 0.055961070559610894, 0.05596107055961092, 0.05596107055961094, 0.05596107055961096, 0.055961070559610984, 0.055961070559611, 0.05596107055961101, 0.055961070559611026, 0.05596107055961104, 0.055961070559611054, 0.05596107055961107, 0.05596107055961108, 0.05596107055961109, 0.0559610705596111, 0.05596107055961111, 0.055961070559611116, 0.06056547619047661, 0.06640625000000036, 0.0719313063063066, 0.07716557017543894, 0.08505917159763342, 0.09350961538461558, 0.10219766260162623, 0.11092509920634941, 0.11924660852713194, 0.1265388257575759, 0.13728395061728413, 0.14895330112721436, 0.15760441292356214, 0.16571969696969727, 0.17346938775510232, 0.17166666666666683]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.125, 0.1111111111111111, 0.08333333333333333, 0.08333333333333333, 0.06944444444444443, 0.059523809523809534, 0.07291666666666667, 0.08333333333333333, 0.08333333333333334, 0.10606060606060606, 0.09722222222222222, 0.14102564102564102, 0.14285714285714285, 0.1444444444444446, 0.15104166666666666, 0.16176470588235295, 0.15277777777777776, 0.14912280701754385, 0.15833333333333333, 0.16666666666666663, 0.17045454545454544, 0.16304347826086957, 0.15624999999999997, 0.15333333333333332, 0.15384615384615385, 0.15123456790123457, 0.1488095238095238, 0.1436781609195402, 0.14444444444444443, 0.1424731182795699, 0.1484375, 0.14393939393939395, 0.14215686274509803, 0.13809523809523824, 0.13657407407407404, 0.13288288288288289, 0.1337719298245614, 0.13675213675213677, 0.13958333333333334, 0.144308943089431, 0.14880952380952378, 0.1511627906976744, 0.14962121212121213, 0.15185185185185185, 0.15942028985507245, 0.16134751773049658, 0.16666666666666674, 0.1683673469387755, 0.17166666666666666]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.08333333333333333, 0.0625, 0.08333333333333333, 0.06944444444444443, 0.059523809523809534, 0.052083333333333336, 0.07407407407407407, 0.08333333333333334, 0.11363636363636363, 0.10416666666666667, 0.11538461538461539, 0.1130952380952381, 0.12222222222222219, 0.13541666666666666, 0.13725490196078433, 0.15277777777777776, 0.15789473684210525, 0.15833333333333333, 0.1587301587301587, 0.1553030303030303, 0.15579710144927536, 0.15624999999999997, 0.16, 0.16346153846153846, 0.1574074074074074, 0.15773809523809523, 0.15229885057471262, 0.1527777777777778, 0.15053763440860216, 0.1484375, 0.14646464646464646, 0.14460784313725492, 0.14285714285714282, 0.13888888888888887, 0.14414414414414414, 0.14473684210526316, 0.14316239316239315, 0.14166666666666666, 0.1422764227642276, 0.14087301587301584, 0.14534883720930233, 0.14204545454545456, 0.14629629629629629, 0.14855072463768115, 0.15602836879432622, 0.15972222222222218, 0.1649659863945578, 0.17166666666666666]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.1111111111111111, 0.125, 0.15, 0.138888888888889, 0.14285714285714288, 0.13541666666666666, 0.12962962962962962, 0.11666666666666668, 0.12121212121212122, 0.11805555555555555, 0.14102564102564102, 0.13690476190476192, 0.13888888888888887, 0.13020833333333334, 0.15196078431372548, 0.15277777777777776, 0.17105263157894737, 0.16666666666666666, 0.17063492063492075, 0.17045454545454544, 0.17391304347826086, 0.17361111111111108, 0.17, 0.1762820512820513, 0.17592592592592593, 0.17559523809523808, 0.17241379310344845, 0.17222222222222222, 0.17473118279569894, 0.1796875, 0.17929292929292928, 0.17647058823529413, 0.17857142857142855, 0.1759259259259259, 0.17567567567567569, 0.17543859649122806, 0.17094017094017094, 0.17083333333333334, 0.1727642276422764, 0.16865079365079363, 0.1686046511627907, 0.16856060606060605, 0.1648148148148148, 0.17028985507246377, 0.17021276595744678, 0.17187499999999997, 0.1717687074829932, 0.17166666666666666]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.1345544247227158, "coverage": 0.6416666666666667, "abstain_rate": 0.35833333333333334, "selective_risk": 0.02857142857142857, "selective_accuracy": 0.9714285714285714, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.02857142857142857, 0.09292929292929293, 0.13799283154121864, 0.17166666666666666], "coverage": [0.0, 0.0, 0.6416666666666667, 0.825, 0.93, 1.0]}, "ece_geometry": 0.054308796799181706}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "685d46f13e58", "timestamp": "2026-07-25T15:07:09.652803+00:00", "git_sha": "b5e47cc", "config_hash": "358641cb8496", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "F4 adaptive-halting run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 800}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "halting", "seed": 0, "metrics": {"k_restarts": 12, "steps": 50, "alpha": 0.1, "n_calib": 800, "n_test": 800, "final_train_loss": 1.010847568511963, "tau_hat": 0.9167, "compute_used": 0.42152500000000004, "compute_saved": 0.578475, "halting_risk": 0.00875, "risk_within_budget": true, "full_accuracy": 0.80375, "halted_accuracy": 0.8075, "accuracy_drop": -0.003750000000000031, "risk_monotone_in_tau": true, "calib_risk_by_tau": [0.9025, 0.845, 0.845, 0.71, 0.48375, 0.48375, 0.37625, 0.19, 0.19, 0.13875, 0.00875, 0.00875], "calib_compute_by_tau": [0.0, 0.013049999999999886, 0.013049999999999886, 0.03759999999999957, 0.09074999999999989, 0.09074999999999989, 0.1236749999999998, 0.20574999999999993, 0.20574999999999993, 0.2681749999999995, 0.4182, 0.4182], "tau_sweep": {"taus": [0.0833, 0.1667, 0.25, 0.3333, 0.4167, 0.5, 0.5833, 0.6667, 0.75, 0.8333, 0.9167, 1.0], "compute_used": [0.0, 0.013375000000000001, 0.013375000000000001, 0.0383, 0.09365000000000002, 0.09365000000000002, 0.12260000000000001, 0.20592500000000002, 0.20592500000000002, 0.25880000000000003, 0.42152500000000004, 0.42152500000000004], "accuracy": [0.0875, 0.14375, 0.14375, 0.2375, 0.455, 0.455, 0.56, 0.69125, 0.69125, 0.7425, 0.8075, 0.8075], "disagreement": [0.91625, 0.85625, 0.85625, 0.7425, 0.4975, 0.4975, 0.37375, 0.20875, 0.20875, 0.14875, 0.00875, 0.00875]}}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "1acd65c19072", "timestamp": "2026-07-25T20:20:34.663369+00:00", "git_sha": "b1d1ecc", "config_hash": "cecbb6158c46", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 1500}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 1500}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 1500, "n_calib": 1500, "n_test": 1500, "k_restarts": 12, "accuracy_id": 0.81, "base_error": 0.19, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0711432604851103, "rho_basin": 0.1192048662021191, "energy_min": 0.1447332231633481, "energy_mean": 0.14862598663924864, "energy_std": 0.18501378708598182}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "delta_aurc_vs_best_energy": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.05, 0.03333333333333333, 0.025, 0.02666666666666667, 0.033333333333333326, 0.052380952380952396, 0.05, 0.04814814814814815, 0.046666666666666676, 0.04242424242424243, 0.041666666666666664, 0.04358974358974359, 0.04047619047619048, 0.04444444444444444, 0.04375, 0.043137254901960784, 0.04444444444444444, 0.043859649122807015, 0.045, 0.04285714285714285, 0.04242424242424243, 0.04057971014492753, 0.04166666666666666, 0.044, 0.04358974358974359, 0.04938271604938271, 0.05238095238095238, 0.0586206896551724, 0.06444444444444444, 0.06451612903225806, 0.06458333333333334, 0.06666666666666667, 0.07058823529411765, 0.07714285714285712, 0.08148148148148163, 0.08558558558558559, 0.08947368421052632, 0.10170940170940171, 0.11, 0.11788617886178875, 0.12380952380952392, 0.12945736434108526, 0.13636363636363635, 0.14444444444444443, 0.1492753623188406, 0.15673758865248238, 0.16805555555555565, 0.1782312925170068, 0.19]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.10957642725598533, 0.10957642725598518, 0.10957642725598526, 0.1095764272559854, 0.10957642725598545, 0.10957642725598521, 0.10957642725598504, 0.10957642725598492, 0.10957642725598482, 0.10957642725598482, 0.10957642725598507, 0.10957642725598529, 0.10957642725598546, 0.10957642725598561, 0.10957642725598575, 0.10957642725598586, 0.10957642725598597, 0.10957642725598606, 0.10957642725598614, 0.1095764272559862, 0.10957642725598626, 0.10957642725598632, 0.10957642725598638, 0.10957642725598643, 0.10957642725598647, 0.10957642725598653, 0.10957642725598656, 0.1095764272559866, 0.10957642725598664, 0.10957642725598667, 0.1095764272559867, 0.10957642725598672, 0.10957642725598675, 0.10957642725598678, 0.10957642725598679, 0.10957642725598682, 0.11300813008130228, 0.11709456568250055, 0.12097144048363749, 0.12465447154471758, 0.12967479674796975, 0.1350907029478481, 0.14025470653377864, 0.14618844696969926, 0.15196759259259482, 0.15961352657005046, 0.1675650118203331, 0.17561111111111322, 0.18333333333333537, 0.19000000000000197]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.16666666666666666, 0.16666666666666666, 0.15833333333333333, 0.16, 0.15555555555555553, 0.14285714285714268, 0.1375, 0.14074074074074075, 0.1399999999999999, 0.13636363636363635, 0.1361111111111111, 0.13333333333333333, 0.13095238095238096, 0.12888888888888886, 0.13541666666666666, 0.13725490196078433, 0.13518518518518516, 0.13859649122807016, 0.14333333333333334, 0.14126984126984124, 0.1409090909090909, 0.14202898550724638, 0.1444444444444444, 0.14533333333333334, 0.14743589743589744, 0.14938271604938272, 0.15, 0.14712643678160917, 0.15, 0.14623655913978495, 0.14375, 0.1404040404040404, 0.13725490196078433, 0.13619047619047617, 0.13425925925925924, 0.13423423423423422, 0.13508771929824562, 0.13931623931623932, 0.1425, 0.14552845528455283, 0.14841269841269839, 0.15426356589147286, 0.15757575757575756, 0.16296296296296298, 0.16666666666666666, 0.17163120567375897, 0.17638888888888887, 0.18503401360544217, 0.19]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1, 0.15, 0.17777777777777778, 0.14166666666666666, 0.15333333333333332, 0.14999999999999997, 0.14761904761904765, 0.14166666666666666, 0.14814814814814814, 0.1466666666666667, 0.14545454545454545, 0.1388888888888889, 0.13333333333333333, 0.1357142857142857, 0.13999999999999999, 0.14375, 0.1411764705882353, 0.1444444444444444, 0.14385964912280702, 0.14333333333333334, 0.13968253968253966, 0.15, 0.15072463768115943, 0.15138888888888885, 0.14933333333333335, 0.15512820512820513, 0.15432098765432098, 0.15119047619047618, 0.14942528735632182, 0.14777777777777779, 0.14731182795698924, 0.14270833333333333, 0.1404040404040404, 0.13823529411764707, 0.1352380952380952, 0.13425925925925924, 0.13603603603603603, 0.13771929824561405, 0.14017094017094017, 0.14083333333333334, 0.1422764227642276, 0.146031746031746, 0.15193798449612403, 0.1590909090909091, 0.16296296296296298, 0.17028985507246377, 0.1730496453900709, 0.17638888888888898, 0.18435374149659864, 0.19]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.23333333333333334, 0.18333333333333332, 0.16666666666666666, 0.19166666666666668, 0.16666666666666666, 0.1611111111111111, 0.16666666666666669, 0.15833333333333333, 0.15925925925925927, 0.16666666666666669, 0.1696969696969697, 0.18611111111111112, 0.18461538461538463, 0.19523809523809524, 0.1911111111111111, 0.19166666666666668, 0.18627450980392157, 0.18888888888888886, 0.18947368421052632, 0.18833333333333332, 0.18888888888888886, 0.19242424242424241, 0.19130434782608696, 0.18888888888888886, 0.192, 0.18974358974358974, 0.18518518518518517, 0.18214285714285713, 0.18160919540229883, 0.17888888888888888, 0.18494623655913978, 0.18645833333333334, 0.18686868686868688, 0.18627450980392157, 0.1876190476190476, 0.18703703703703717, 0.1873873873873874, 0.18859649122807018, 0.18803418803418803, 0.1925, 0.19430894308943086, 0.19444444444444442, 0.1937984496124031, 0.19242424242424241, 0.18962962962962962, 0.18840579710144928, 0.18936170212765954, 0.19097222222222218, 0.19115646258503402, 0.19]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.19374220945381396, "coverage": 0.6933333333333334, "abstain_rate": 0.30666666666666664, "selective_risk": 0.075, "selective_accuracy": 0.925, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.075, 0.1295577967416602, 0.17525773195876287, 0.1879598662207358], "coverage": [0.0, 0.0, 0.6933333333333334, 0.8593333333333333, 0.97, 0.9966666666666667]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7041700960219479, "basin/entropy": 0.2946790845426323, "basin/dispersion": 0.5298130098909826, "energy/mean": 0.3736336726590138, "energy/min": 0.36821890116237094, "energy/std": 0.49345462421485814, "curv/lmax_mean": 0.4698808750270739, "curv/lmax_best": 0.4712165186629124, "curv/trace_mean": 0.4452761533463288, "curv/trace_best": 0.4445397444227854, "dynamics/steps": 0.5, "dynamics/monotonic": 0.341111833080644, "dynamics/drop": 0.3597834091401343, "dynamics/residual": 0.5355483358602267}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [2, 0, 0, 0, 0, 14, 0, 22, 0, 28, 0, 0, 38, 0, 54, 0, 0, 90, 0, 967], "incorrect_counts": [1, 0, 2, 0, 0, 14, 0, 28, 0, 32, 0, 0, 26, 0, 30, 0, 0, 33, 0, 119]}, "basin/entropy": {"edges": [0.0, 0.06789889274936621, 0.13579778549873242, 0.20369667824809862, 0.27159557099746484, 0.33949446374683107, 0.40739335649619723, 0.47529224924556346, 0.5431911419949297, 0.6110900347442959, 0.6789889274936621, 0.7468878202430284, 0.8147867129923945, 0.8826856057417607, 0.9505844984911269, 1.0184833912404931, 1.0863822839898594, 1.1542811767392256, 1.2221800694885918, 1.290078962237958, 1.3579778549873243], "correct_counts": [967, 0, 0, 0, 90, 0, 53, 0, 37, 25, 35, 0, 3, 1, 1, 1, 1, 0, 0, 1], "incorrect_counts": [119, 0, 0, 0, 33, 0, 26, 0, 27, 29, 35, 0, 3, 2, 6, 3, 2, 0, 0, 0]}, "basin/dispersion": {"edges": [1.592531476672375, 1.614116358729778, 1.6357012407871807, 1.6572861228445837, 1.6788710049019866, 1.7004558869593893, 1.7220407690167923, 1.7436256510741952, 1.7652105331315981, 1.7867954151890009, 1.8083802972464038, 1.8299651793038068, 1.8515500613612095, 1.8731349434186124, 1.8947198254760154, 1.9163047075334183, 1.937889589590821, 1.959474471648224, 1.981059353705627, 2.0026442357630296, 2.0242291178204326], "correct_counts": [3, 2, 8, 9, 23, 47, 59, 82, 96, 177, 167, 139, 126, 93, 70, 52, 31, 17, 10, 4], "incorrect_counts": [0, 0, 2, 1, 5, 11, 19, 21, 37, 36, 38, 29, 30, 20, 17, 12, 5, 1, 1, 0]}, "energy/mean": {"edges": [-0.30860641847054165, -0.24559052648643653, -0.18257463450233144, -0.11955874251822635, -0.056542850534121225, 0.006473041449983896, 0.06948893343408896, 0.13250482541819408, 0.1955207174022992, 0.25853660938640427, 0.32155250137050945, 0.3845683933546145, 0.4475842853387196, 0.5106001773228248, 0.5736160693069299, 0.6366319612910349, 0.69964785327514, 0.7626637452592451, 0.8256796372433501, 0.8886955292274554, 0.9517114212115606], "correct_counts": [10, 53, 216, 277, 172, 82, 46, 41, 37, 37, 39, 36, 43, 41, 43, 21, 18, 2, 0, 1], "incorrect_counts": [2, 11, 35, 49, 32, 8, 2, 2, 4, 7, 14, 6, 21, 33, 19, 24, 12, 3, 1, 0]}, "energy/min": {"edges": [-0.6984671354293823, -0.628644996881485, -0.5588228583335877, -0.48900071978569026, -0.41917858123779295, -0.34935644268989563, -0.27953430414199826, -0.20971216559410089, -0.13989002704620357, -0.07006788849830625, -0.00024574995040893555, 0.06957638859748849, 0.1393985271453858, 0.20922066569328313, 0.27904280424118055, 0.34886494278907776, 0.4186870813369752, 0.4885092198848726, 0.5583313584327698, 0.6281534969806672, 0.6979756355285645], "correct_counts": [5, 25, 85, 195, 264, 157, 92, 53, 48, 43, 42, 45, 44, 51, 33, 14, 13, 6, 0, 0], "incorrect_counts": [0, 3, 21, 25, 47, 32, 10, 2, 5, 6, 12, 17, 21, 28, 22, 19, 11, 3, 0, 1]}, "energy/std": {"edges": [0.058812152065186904, 0.07638791223625852, 0.09396367240733014, 0.11153943257840175, 0.12911519274947336, 0.14669095292054496, 0.1642667130916166, 0.1818424732626882, 0.1994182334337598, 0.21699399360483146, 0.23456975377590306, 0.25214551394697465, 0.2697212741180463, 0.2872970342891179, 0.3048727944601895, 0.32244855463126115, 0.34002431480233275, 0.35760007497340435, 0.375175835144476, 0.3927515953155476, 0.4103273554866192], "correct_counts": [3, 6, 20, 52, 77, 135, 180, 211, 168, 133, 90, 68, 39, 16, 9, 5, 1, 0, 1, 1], "incorrect_counts": [0, 3, 4, 12, 10, 38, 42, 44, 44, 39, 19, 10, 16, 1, 1, 1, 0, 1, 0, 0]}, "curv/lmax_mean": {"edges": [1.0000369946161907, 1.0007940803964934, 1.001551166176796, 1.0023082519570987, 1.0030653377374015, 1.003822423517704, 1.0045795092980068, 1.0053365950783095, 1.006093680858612, 1.0068507666389148, 1.0076078524192176, 1.0083649381995201, 1.0091220239798229, 1.0098791097601254, 1.0106361955404282, 1.011393281320731, 1.0121503671010335, 1.0129074528813362, 1.013664538661639, 1.0144216244419415, 1.0151787102222443], "correct_counts": [617, 173, 79, 69, 52, 45, 45, 33, 29, 19, 20, 8, 6, 5, 6, 2, 6, 0, 0, 1], "incorrect_counts": [119, 74, 29, 22, 10, 10, 4, 5, 3, 4, 3, 1, 0, 0, 0, 1, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0017322301864624, 1.0034644603729248, 1.0051966905593872, 1.0069289207458496, 1.008661150932312, 1.0103933811187744, 1.0121256113052368, 1.0138578414916992, 1.0155900716781616, 1.017322301864624, 1.0190545320510864, 1.0207867622375488, 1.0225189924240112, 1.0242512226104736, 1.025983452796936, 1.0277156829833984, 1.0294479131698608, 1.0311801433563232, 1.0329123735427856, 1.034644603729248], "correct_counts": [934, 93, 63, 35, 27, 11, 9, 7, 9, 7, 3, 3, 2, 5, 3, 2, 1, 0, 0, 1], "incorrect_counts": [213, 31, 16, 6, 6, 3, 3, 3, 2, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00269635518392, 32.00753688812256, 32.0123774210612, 32.01721795399984, 32.02205848693848, 32.02689901987712, 32.03173955281576, 32.036580085754395, 32.04142061869303, 32.046261151631676, 32.05110168457031, 32.05594221750895, 32.06078275044759, 32.06562328338623, 32.07046381632487, 32.075304349263504, 32.08014488220215, 32.084985415140785, 32.08982594807942, 32.094666481018066, 32.0995070139567], "correct_counts": [175, 275, 151, 82, 84, 56, 49, 58, 42, 42, 57, 38, 42, 21, 20, 15, 4, 2, 1, 1], "incorrect_counts": [9, 50, 53, 31, 37, 20, 19, 17, 9, 8, 7, 13, 4, 5, 2, 0, 1, 0, 0, 0]}, "curv/trace_best": {"edges": [32.0009880065918, 32.010338592529294, 32.0196891784668, 32.029039764404295, 32.0383903503418, 32.0477409362793, 32.057091522216794, 32.0664421081543, 32.075792694091795, 32.0851432800293, 32.0944938659668, 32.103844451904294, 32.1131950378418, 32.122545623779295, 32.1318962097168, 32.1412467956543, 32.150597381591794, 32.1599479675293, 32.169298553466795, 32.1786491394043, 32.1879997253418], "correct_counts": [402, 281, 150, 106, 83, 59, 46, 28, 17, 9, 10, 6, 3, 5, 2, 2, 2, 3, 0, 1], "incorrect_counts": [66, 67, 55, 31, 17, 13, 8, 8, 7, 6, 3, 2, 0, 1, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [1215, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [285, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7133333333333333, 0.7193333333333333, 0.7253333333333333, 0.7313333333333333, 0.7373333333333333, 0.7433333333333333, 0.7493333333333333, 0.7553333333333333, 0.7613333333333333, 0.7673333333333333, 0.7733333333333334, 0.7793333333333334, 0.7853333333333334, 0.7913333333333334, 0.7973333333333334, 0.8033333333333335, 0.8093333333333335, 0.8153333333333335, 0.8213333333333335, 0.8273333333333335, 0.8333333333333335], "correct_counts": [1, 0, 2, 23, 32, 66, 131, 145, 240, 187, 167, 125, 55, 25, 12, 3, 1, 0, 0, 0], "incorrect_counts": [1, 0, 0, 3, 8, 14, 10, 17, 33, 38, 46, 39, 19, 20, 14, 11, 6, 3, 2, 1]}, "dynamics/drop": {"edges": [63.47809874266386, 76.39078911232451, 89.30347948198516, 102.21616985164583, 115.12886022130648, 128.04155059096712, 140.9542409606278, 153.86693133028845, 166.7796216999491, 179.69231206960976, 192.6050024392704, 205.51769280893106, 218.43038317859174, 231.3430735482524, 244.25576391791304, 257.1684542875737, 270.08114465723435, 282.993835026895, 295.90652539655565, 308.8192157662163, 321.73190613587695], "correct_counts": [10, 31, 108, 203, 266, 281, 168, 89, 28, 12, 10, 3, 1, 2, 2, 0, 0, 1, 0, 0], "incorrect_counts": [1, 7, 23, 29, 45, 36, 31, 27, 17, 10, 15, 12, 4, 11, 5, 4, 0, 4, 1, 3]}, "dynamics/residual": {"edges": [1.1409447093804677, 1.1559197435776392, 1.1708947777748107, 1.1858698119719822, 1.2008448461691539, 1.2158198803663254, 1.2307949145634969, 1.2457699487606684, 1.2607449829578399, 1.2757200171550114, 1.2906950513521829, 1.3056700855493546, 1.320645119746526, 1.3356201539436976, 1.350595188140869, 1.3655702223380406, 1.380545256535212, 1.3955202907323838, 1.4104953249295553, 1.4254703591267268, 1.4404453933238983], "correct_counts": [2, 4, 6, 19, 33, 43, 85, 102, 112, 156, 148, 145, 122, 97, 61, 43, 24, 7, 4, 2], "incorrect_counts": [0, 2, 2, 1, 11, 8, 21, 34, 36, 27, 34, 40, 24, 26, 10, 6, 3, 0, 0, 0]}}}, "ece_geometry": 0.03153101157848922, "accuracy_ood": 0.20533333333333334, "aurc_ood": {"geometry": 0.8035703588171388, "rho_basin": 0.7533412924431383, "energy_min": 0.7509755512657853, "energy_mean": 0.740760170602411, "energy_std": 0.7992759846101417}, "ood_ltt": {"alpha": 0.1, "lambda_hat": 0.19374220945381396, "selective_risk": 0.7619532044760936, "coverage": 0.6553333333333333, "risk_within_budget": false}, "ood_validity": {"target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "id_risk": [0.0, 0.0, 0.075, 0.1295577967416602, 0.17525773195876287, 0.1879598662207358], "ood_risk": [0.0, 0.0, 0.7619532044760936, 0.7915726109857035, 0.7924914675767918, 0.7949231796927188], "id_coverage": [0.0, 0.0, 0.6933333333333334, 0.8593333333333333, 0.97, 0.9966666666666667], "ood_coverage": [0.0, 0.0, 0.6553333333333333, 0.886, 0.9766666666666667, 0.998]}}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "7727dfb1423e", "timestamp": "2026-07-26T13:07:14.638654+00:00", "git_sha": "1e6727a", "config_hash": "3643423663c7", "config": {"run": {"seed": 0, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.227946937084198, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1527490825609553, "rho_basin": 0.22247851187975562, "energy_min": 0.37245598613988906, "energy_mean": 0.3894178842534342, "energy_std": 0.28068693518661764}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.21970690357893377, 0.16353680784617466, 0.27315268065902204], "delta_aurc_vs_best_energy": [0.12793785262566235, 0.08274255159121499, 0.17357149948028552], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.0625, 0.06666666666666667, 0.08333333333333331, 0.10714285714285696, 0.09375, 0.08333333333333333, 0.09166666666666667, 0.09090909090909091, 0.09722222222222222, 0.11538461538461539, 0.125, 0.1333333333333333, 0.13020833333333334, 0.12745098039215685, 0.12499999999999999, 0.12280701754385964, 0.12916666666666668, 0.13095238095238093, 0.13257575757575757, 0.13043478260869565, 0.13888888888888887, 0.13333333333333333, 0.14102564102564102, 0.1419753086419753, 0.1488095238095238, 0.14942528735632182, 0.15, 0.16129032258064516, 0.1640625, 0.1691919191919192, 0.17892156862745098, 0.1833333333333333, 0.18287037037037035, 0.18468468468468469, 0.18421052631578946, 0.19017094017094016, 0.20416666666666666, 0.20934959349593493, 0.21428571428571425, 0.2248062015503876, 0.23484848484848486, 0.2462962962962963, 0.25181159420289856, 0.25886524822695045, 0.26736111111111105, 0.272108843537415, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2165991902834008, 0.2165991902834009, 0.21659919028340094, 0.21659919028340077, 0.21659919028340063, 0.21659919028340055, 0.21659919028340052, 0.21659919028340047, 0.21659919028340044, 0.2165991902834004, 0.21659919028340038, 0.21659919028340036, 0.21659919028340055, 0.21659919028340077, 0.21659919028340097, 0.21659919028340113, 0.21659919028340127, 0.2165991902834014, 0.21659919028340155, 0.21659919028340163, 0.21659919028340174, 0.21659919028340183, 0.2165991902834019, 0.216599190283402, 0.21659919028340202, 0.2165991902834021, 0.21659919028340216, 0.21659919028340222, 0.21659919028340227, 0.2165991902834023, 0.21659919028340235, 0.21659919028340238, 0.21659919028340244, 0.21659919028340247, 0.2165991902834025, 0.21659919028340255, 0.21659919028340258, 0.2165991902834026, 0.21659919028340263, 0.21659919028340266, 0.2165991902834027, 0.22372534872535066, 0.23191214470284427, 0.23948863636363826, 0.24416666666666834, 0.250762776506485, 0.259985069055619, 0.2673611111111122, 0.27494331065759725, 0.2816666666666674]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4583333333333333, 0.4666666666666667, 0.45833333333333337, 0.4523809523809525, 0.4479166666666667, 0.46296296296296297, 0.42500000000000004, 0.42424242424242425, 0.4027777777777778, 0.3974358974358974, 0.4107142857142857, 0.40000000000000013, 0.4010416666666667, 0.4019607843137255, 0.40740740740740733, 0.39473684210526316, 0.4, 0.39682539682539675, 0.38636363636363635, 0.391304347826087, 0.39236111111111105, 0.38333333333333336, 0.3717948717948718, 0.36419753086419754, 0.36607142857142855, 0.3706896551724139, 0.3638888888888889, 0.3521505376344086, 0.3515625, 0.3434343434343434, 0.3382352941176471, 0.33571428571428563, 0.32870370370370366, 0.32657657657657657, 0.3223684210526316, 0.3247863247863248, 0.3229166666666667, 0.3191056910569105, 0.3115079365079366, 0.3081395348837209, 0.30303030303030304, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.28819444444444436, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.5, 0.5, 0.4375, 0.5, 0.49999999999999994, 0.4999999999999999, 0.46875, 0.4722222222222222, 0.45833333333333326, 0.44696969696969696, 0.4236111111111111, 0.4166666666666667, 0.42857142857142855, 0.4277777777777777, 0.421875, 0.4215686274509804, 0.412037037037037, 0.41228070175438597, 0.3958333333333333, 0.39682539682539686, 0.4015151515151515, 0.39492753623188404, 0.39583333333333326, 0.3933333333333333, 0.38782051282051283, 0.37962962962962965, 0.37202380952380953, 0.36781609195402293, 0.3611111111111111, 0.3548387096774194, 0.3463541666666667, 0.3383838383838384, 0.33578431372549017, 0.3309523809523809, 0.33101851851851866, 0.32882882882882886, 0.3267543859649123, 0.32264957264957267, 0.32083333333333336, 0.31707317073170727, 0.31349206349206343, 0.3081395348837209, 0.30303030303030304, 0.30185185185185187, 0.2971014492753623, 0.29078014184397155, 0.28645833333333326, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.33333333333333326, 0.3214285714285715, 0.3020833333333333, 0.3055555555555556, 0.2833333333333334, 0.2727272727272727, 0.2847222222222222, 0.28846153846153844, 0.27976190476190477, 0.28333333333333327, 0.2760416666666667, 0.27450980392156865, 0.26388888888888884, 0.2675438596491228, 0.2791666666666667, 0.26984126984126994, 0.2803030303030303, 0.2753623188405797, 0.26736111111111105, 0.26, 0.266025641025641, 0.2623456790123457, 0.2648809523809524, 0.26724137931034475, 0.26944444444444443, 0.27419354838709675, 0.2734375, 0.2727272727272727, 0.27941176470588236, 0.2738095238095238, 0.2708333333333334, 0.2725225225225225, 0.2741228070175439, 0.27136752136752135, 0.27708333333333335, 0.2804878048780489, 0.28174603174603186, 0.2810077519379845, 0.2821969696969697, 0.2814814814814815, 0.28442028985507245, 0.2836879432624113, 0.27951388888888895, 0.2789115646258503, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.14285714285714285, 0.2527075812274368], "coverage": [0.0, 0.0, 0.0, 0.0, 0.525, 0.9233333333333333]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6337264377599913, "basin/entropy": 0.36597839069729127, "basin/dispersion": 0.5371984788368869, "energy/mean": 0.6647400431087741, "energy/min": 0.6541275964799077, "energy/std": 0.4885020387429811, "curv/lmax_mean": 0.3431952662721893, "curv/lmax_best": 0.3816018890978734, "curv/trace_mean": 0.3301253449388377, "curv/trace_best": 0.3412526256538393, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6490067134364832, "dynamics/drop": 0.6809401556858277, "dynamics/residual": 0.5138044179629044}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [4, 0, 0, 3, 0, 0, 6, 0, 0, 0, 6, 0, 0, 11, 0, 0, 14, 0, 0, 387], "incorrect_counts": [6, 0, 0, 6, 0, 0, 9, 0, 0, 0, 13, 0, 0, 9, 0, 0, 19, 0, 0, 107]}, "basin/entropy": {"edges": [0.0, 0.04592138924965487, 0.09184277849930975, 0.1377641677489646, 0.1836855569986195, 0.22960694624827438, 0.2755283354979292, 0.3214497247475841, 0.367371113997239, 0.41329250324689387, 0.45921389249654876, 0.5051352817462036, 0.5510566709958584, 0.5969780602455134, 0.6428994494951682, 0.6888208387448231, 0.734742227994478, 0.7806636172441328, 0.8265850064937877, 0.8725063957434426, 0.9184277849930975], "correct_counts": [387, 0, 0, 0, 0, 0, 14, 0, 0, 10, 0, 0, 7, 6, 3, 4, 0, 0, 0, 0], "incorrect_counts": [107, 0, 0, 0, 0, 0, 19, 0, 0, 9, 0, 0, 12, 7, 6, 6, 0, 2, 0, 1]}, "basin/dispersion": {"edges": [1.620605021245252, 1.641571274378292, 1.662537527511332, 1.683503780644372, 1.7044700337774121, 1.725436286910452, 1.746402540043492, 1.767368793176532, 1.788335046309572, 1.809301299442612, 1.830267552575652, 1.851233805708692, 1.872200058841732, 1.893166311974772, 1.914132565107812, 1.935098818240852, 1.9560650713738919, 1.977031324506932, 1.9979975776399719, 2.018963830773012, 2.039930083906052], "correct_counts": [0, 2, 2, 8, 13, 22, 28, 29, 46, 69, 50, 43, 35, 31, 20, 17, 7, 7, 1, 1], "incorrect_counts": [1, 3, 0, 5, 2, 6, 17, 14, 22, 23, 22, 15, 14, 8, 13, 2, 1, 1, 0, 0]}, "energy/mean": {"edges": [0.6107773731152216, 0.6431646155814329, 0.6755518580476443, 0.7079391005138556, 0.7403263429800669, 0.7727135854462782, 0.8051008279124896, 0.8374880703787009, 0.8698753128449122, 0.9022625553111235, 0.9346497977773349, 0.9670370402435462, 0.9994242827097575, 1.0318115251759687, 1.0641987676421802, 1.0965860101083915, 1.1289732525746028, 1.161360495040814, 1.1937477375070253, 1.2261349799732368, 1.2585222224394481], "correct_counts": [1, 3, 8, 10, 11, 27, 35, 32, 45, 36, 55, 40, 46, 27, 21, 12, 15, 3, 3, 1], "incorrect_counts": [4, 1, 7, 6, 18, 17, 16, 20, 24, 11, 12, 11, 11, 5, 1, 1, 2, 1, 0, 1]}, "energy/min": {"edges": [0.20125961303710938, 0.2415154755115509, 0.2817713379859924, 0.322027200460434, 0.3622830629348755, 0.402538925409317, 0.4427947878837586, 0.4830506503582001, 0.5233065128326416, 0.5635623753070831, 0.6038182377815247, 0.6440741002559662, 0.6843299627304078, 0.7245858252048493, 0.7648416876792908, 0.8050975501537323, 0.8453534126281739, 0.8856092751026154, 0.9258651375770569, 0.9661210000514985, 1.00637686252594], "correct_counts": [1, 1, 6, 2, 13, 14, 29, 37, 39, 39, 58, 46, 39, 44, 24, 21, 10, 1, 3, 4], "incorrect_counts": [1, 0, 2, 4, 12, 13, 19, 20, 22, 21, 17, 10, 14, 5, 4, 2, 1, 2, 0, 0]}, "energy/std": {"edges": [0.06508645292670491, 0.08179583921728364, 0.09850522550786239, 0.11521461179844114, 0.1319239980890199, 0.1486333843795986, 0.16534277067017736, 0.1820521569607561, 0.19876154325133483, 0.2154709295419136, 0.23218031583249232, 0.24888970212307104, 0.2655990884136498, 0.28230847470422854, 0.29901786099480726, 0.31572724728538604, 0.33243663357596476, 0.3491460198665435, 0.36585540615712225, 0.382564792447701, 0.3992741787382797], "correct_counts": [1, 3, 7, 18, 27, 54, 60, 71, 59, 43, 31, 26, 20, 6, 1, 2, 2, 0, 0, 0], "incorrect_counts": [1, 0, 3, 10, 13, 16, 22, 21, 29, 13, 18, 11, 6, 2, 1, 0, 1, 0, 1, 1]}, "curv/lmax_mean": {"edges": [0.9988154868284861, 0.9990137209494908, 0.9992119550704955, 0.9994101891915003, 0.9996084233125051, 0.9998066574335098, 1.0000048915545146, 1.0002031256755193, 1.000401359796524, 1.0005995939175287, 1.0007978280385335, 1.0009960621595382, 1.001194296280543, 1.0013925304015476, 1.0015907645225526, 1.0017889986435573, 1.001987232764562, 1.0021854668855668, 1.0023837010065715, 1.0025819351275762, 1.002780169248581], "correct_counts": [2, 2, 10, 38, 119, 132, 72, 24, 11, 6, 10, 2, 1, 0, 0, 1, 0, 0, 0, 1], "incorrect_counts": [1, 0, 3, 9, 21, 43, 36, 30, 11, 5, 3, 1, 3, 2, 1, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.9958722591400146, 0.9962057709693909, 0.9965392827987671, 0.9968727946281433, 0.9972063064575195, 0.9975398182868958, 0.997873330116272, 0.9982068419456482, 0.9985403537750244, 0.9988738656044006, 0.9992073774337769, 0.9995408892631531, 0.9998744010925293, 1.0002079129219055, 1.0005414247512818, 1.000874936580658, 1.001208448410034, 1.0015419602394104, 1.0018754720687866, 1.002208983898163, 1.002542495727539], "correct_counts": [0, 0, 0, 0, 0, 1, 3, 4, 8, 10, 25, 92, 246, 23, 5, 1, 6, 4, 2, 1], "incorrect_counts": [1, 0, 0, 0, 0, 0, 1, 0, 1, 6, 6, 26, 85, 24, 6, 6, 3, 2, 2, 0]}, "curv/trace_mean": {"edges": [31.95863151550293, 31.962152846654256, 31.96567417780558, 31.969195508956908, 31.972716840108234, 31.97623817125956, 31.97975950241089, 31.983280833562215, 31.98680216471354, 31.990323495864867, 31.993844827016193, 31.99736615816752, 32.00088748931885, 32.004408820470175, 32.0079301516215, 32.01145148277283, 32.01497281392415, 32.01849414507548, 32.022015476226805, 32.02553680737813, 32.02905813852946], "correct_counts": [1, 0, 2, 7, 14, 14, 24, 46, 62, 37, 62, 49, 47, 21, 19, 14, 3, 2, 5, 2], "incorrect_counts": [0, 0, 0, 0, 0, 3, 4, 7, 10, 20, 17, 21, 31, 18, 22, 9, 3, 4, 0, 0]}, "curv/trace_best": {"edges": [31.937339782714844, 31.943468475341795, 31.94959716796875, 31.955725860595702, 31.961854553222658, 31.96798324584961, 31.97411193847656, 31.980240631103516, 31.986369323730468, 31.992498016357423, 31.998626708984375, 32.00475540161133, 32.01088409423828, 32.01701278686524, 32.02314147949219, 32.02927017211914, 32.03539886474609, 32.041527557373044, 32.04765625, 32.053784942626955, 32.059913635253906], "correct_counts": [2, 1, 3, 7, 6, 15, 18, 43, 86, 101, 89, 32, 11, 11, 4, 1, 0, 0, 1, 0], "incorrect_counts": [0, 0, 0, 0, 1, 3, 6, 6, 17, 43, 37, 26, 15, 10, 4, 0, 0, 0, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6466666666666666, 0.6523333333333333, 0.6579999999999999, 0.6636666666666666, 0.6693333333333332, 0.6749999999999999, 0.6806666666666666, 0.6863333333333332, 0.692, 0.6976666666666665, 0.7033333333333333, 0.709, 0.7146666666666666, 0.7203333333333333, 0.7259999999999999, 0.7316666666666666, 0.7373333333333333, 0.7429999999999999, 0.7486666666666666, 0.7543333333333332, 0.7599999999999999], "correct_counts": [0, 5, 4, 8, 19, 36, 33, 63, 55, 44, 53, 28, 27, 22, 10, 12, 6, 2, 1, 3], "incorrect_counts": [2, 3, 7, 9, 14, 22, 20, 25, 18, 11, 18, 8, 5, 6, 0, 0, 1, 0, 0, 0]}, "dynamics/drop": {"edges": [13.273132185141245, 17.449723252654074, 21.626314320166905, 25.802905387679733, 29.97949645519256, 34.15608752270539, 38.33267859021822, 42.509269657731046, 46.68586072524388, 50.86245179275671, 55.03904286026954, 59.21563392778236, 63.392224995295194, 67.56881606280803, 71.74540713032086, 75.92199819783369, 80.09858926534652, 84.27518033285935, 88.45177140037218, 92.62836246788501, 96.80495353539784], "correct_counts": [12, 54, 92, 87, 56, 46, 29, 19, 11, 9, 4, 2, 5, 0, 2, 2, 0, 0, 0, 1], "incorrect_counts": [24, 44, 39, 23, 17, 11, 7, 0, 1, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471606890360515, 1.1608674337466558, 1.1745741784572603, 1.1882809231678646, 1.201987667878469, 1.2156944125890732, 1.2294011572996775, 1.243107902010282, 1.2568146467208863, 1.2705213914314906, 1.284228136142095, 1.2979348808526994, 1.3116416255633037, 1.325348370273908, 1.3390551149845122, 1.3527618596951168, 1.366468604405721, 1.3801753491163253, 1.3938820938269298, 1.4075888385375341, 1.4212955832481384], "correct_counts": [1, 1, 3, 4, 7, 18, 32, 34, 40, 41, 45, 44, 52, 38, 27, 26, 10, 4, 2, 2], "incorrect_counts": [1, 0, 0, 2, 4, 10, 10, 14, 15, 14, 24, 17, 18, 12, 10, 8, 5, 5, 0, 0]}}}, "ece_geometry": 0.04034562456459658}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "cc153278742e", "timestamp": "2026-07-26T13:07:19.898124+00:00", "git_sha": "1e6727a", "config_hash": "60da27bc4d66", "config": {"run": {"seed": 1, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6933333333333334, "base_error": 0.30666666666666664, "final_train_loss": 0.2080264687538147, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1675196272459423, "rho_basin": 0.25457099646739273, "energy_min": 0.19884075586381236, "energy_mean": 0.19759334670590525, "energy_std": 0.29045958157255325}, "best_energy_baseline": "energy_mean", "delta_aurc_vs_energy_min": [0.031321128617870064, 0.011535510095268008, 0.05241587205968214], "delta_aurc_vs_best_energy": [0.030073719459962955, 0.008993436060296952, 0.05227588726640192], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.05555555555555555, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.0625, 0.05555555555555555, 0.08333333333333334, 0.09090909090909091, 0.10416666666666667, 0.09615384615384616, 0.10714285714285714, 0.11666666666666664, 0.11979166666666667, 0.12254901960784313, 0.13888888888888887, 0.13596491228070176, 0.14166666666666666, 0.14682539682539694, 0.15151515151515152, 0.15942028985507245, 0.16666666666666663, 0.16666666666666666, 0.1762820512820513, 0.17901234567901234, 0.1875, 0.1954022988505747, 0.2, 0.20430107526881722, 0.20572916666666666, 0.21212121212121213, 0.21568627450980393, 0.22142857142857153, 0.22916666666666663, 0.23198198198198197, 0.2412280701754386, 0.24572649572649571, 0.25833333333333336, 0.26219512195121947, 0.2658730158730158, 0.2693798449612403, 0.2784090909090909, 0.2796296296296296, 0.2807971014492754, 0.2854609929078015, 0.29166666666666674, 0.3010204081632653, 0.30666666666666664]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2489878542510122, 0.24898785425101208, 0.24898785425101208, 0.24898785425101222, 0.2489878542510123, 0.24898785425101236, 0.2489878542510124, 0.24898785425101244, 0.24898785425101247, 0.2489878542510125, 0.2489878542510125, 0.24898785425101252, 0.24898785425101252, 0.24898785425101252, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.25492831541218686, 0.26175293823455936, 0.2689393939393948, 0.2777777777777784, 0.28623188405797145, 0.29078014184397205, 0.29714209401709435, 0.3030990173847319, 0.3066666666666668]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.1111111111111111, 0.125, 0.11666666666666667, 0.11111111111111109, 0.09523809523809525, 0.11458333333333333, 0.1388888888888889, 0.1416666666666667, 0.1590909090909091, 0.1597222222222222, 0.16025641025641027, 0.16666666666666666, 0.17222222222222236, 0.18229166666666666, 0.19117647058823528, 0.19444444444444442, 0.18859649122807018, 0.19166666666666668, 0.18650793650793662, 0.19318181818181818, 0.1956521739130435, 0.20833333333333331, 0.2, 0.19230769230769232, 0.20679012345679013, 0.20833333333333334, 0.2155172413793103, 0.2222222222222222, 0.22311827956989247, 0.22395833333333334, 0.22474747474747475, 0.22794117647058823, 0.23571428571428582, 0.2384259259259259, 0.24549549549549549, 0.2543859649122807, 0.2606837606837607, 0.26666666666666666, 0.27032520325203246, 0.26785714285714296, 0.27325581395348836, 0.2840909090909091, 0.28888888888888886, 0.2898550724637681, 0.2943262411347517, 0.29513888888888895, 0.3010204081632653, 0.30666666666666664]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.05555555555555555, 0.0625, 0.08333333333333333, 0.11111111111111109, 0.10714285714285716, 0.11458333333333333, 0.12962962962962962, 0.12500000000000003, 0.12121212121212122, 0.11805555555555555, 0.14743589743589744, 0.14285714285714285, 0.1611111111111113, 0.17708333333333334, 0.18137254901960784, 0.18518518518518515, 0.17982456140350878, 0.17916666666666667, 0.17857142857142855, 0.17045454545454544, 0.18478260869565216, 0.18402777777777776, 0.19666666666666666, 0.20833333333333334, 0.2006172839506173, 0.2113095238095238, 0.2097701149425287, 0.21666666666666667, 0.22580645161290322, 0.22916666666666666, 0.2297979797979798, 0.2426470588235294, 0.24047619047619045, 0.24074074074074084, 0.24774774774774774, 0.2565789473684211, 0.26282051282051283, 0.26458333333333334, 0.26626016260162594, 0.2678571428571428, 0.2655038759689923, 0.2708333333333333, 0.2740740740740741, 0.2807971014492754, 0.28900709219858167, 0.29687499999999994, 0.30272108843537415, 0.30666666666666664]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.16666666666666666, 0.19444444444444445, 0.22916666666666666, 0.25, 0.26388888888888884, 0.261904761904762, 0.2708333333333333, 0.26851851851851855, 0.2833333333333334, 0.30303030303030304, 0.2986111111111111, 0.28846153846153844, 0.30357142857142855, 0.29999999999999993, 0.3177083333333333, 0.3235294117647059, 0.3101851851851851, 0.2982456140350877, 0.3, 0.2896825396825398, 0.29924242424242425, 0.30434782608695654, 0.30208333333333326, 0.31, 0.3108974358974359, 0.3148148148148148, 0.31845238095238093, 0.3189655172413793, 0.32222222222222224, 0.31451612903225806, 0.3229166666666667, 0.3181818181818182, 0.31862745098039214, 0.31904761904761897, 0.31481481481481477, 0.31756756756756754, 0.3157894736842105, 0.3141025641025641, 0.3125, 0.306910569105691, 0.3055555555555557, 0.3062015503875969, 0.3068181818181818, 0.3037037037037037, 0.3061594202898551, 0.30673758865248224, 0.3055555555555555, 0.304421768707483, 0.30666666666666664]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.15985130111524162, 0.24301075268817204], "coverage": [0.0, 0.0, 0.0, 0.0, 0.4483333333333333, 0.775]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6111648202341137, "basin/entropy": 0.3888286475752508, "basin/dispersion": 0.505382525083612, "energy/mean": 0.32454535953177255, "energy/min": 0.3349576714046823, "energy/std": 0.5020641722408027, "curv/lmax_mean": 0.46143394648829433, "curv/lmax_best": 0.5073879076086957, "curv/trace_mean": 0.4757133152173913, "curv/trace_best": 0.4546535326086957, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6186833716555183, "dynamics/drop": 0.6482545986622074, "dynamics/residual": 0.4986151755852843}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [0, 0, 0, 0, 0, 4, 0, 5, 0, 4, 0, 0, 9, 0, 9, 0, 0, 14, 0, 371], "incorrect_counts": [1, 0, 0, 0, 0, 3, 0, 4, 0, 9, 0, 0, 9, 0, 18, 0, 0, 17, 0, 123]}, "basin/entropy": {"edges": [0.0, 0.05493061443340548, 0.10986122886681096, 0.16479184330021646, 0.21972245773362192, 0.2746530721670274, 0.3295836866004329, 0.3845143010338384, 0.43944491546724385, 0.4943755299006493, 0.5493061443340548, 0.6042367587674603, 0.6591673732008658, 0.7140979876342712, 0.7690286020676768, 0.8239592165010822, 0.8788898309344877, 0.9338204453678932, 0.9887510598012986, 1.0436816742347041, 1.0986122886681096], "correct_counts": [371, 0, 0, 0, 0, 14, 0, 0, 9, 0, 8, 3, 8, 1, 0, 1, 1, 0, 0, 0], "incorrect_counts": [123, 0, 0, 0, 0, 17, 0, 0, 18, 0, 8, 9, 4, 1, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.6302329015759547, 1.6519526525410289, 1.6736724035061028, 1.6953921544711767, 1.717111905436251, 1.738831656401325, 1.760551407366399, 1.782271158331473, 1.8039909092965472, 1.8257106602616213, 1.8474304112266953, 1.8691501621917692, 1.8908699131568434, 1.9125896641219176, 1.9343094150869915, 1.9560291660520654, 1.9777489170171396, 1.9994686679822138, 2.021188418947288, 2.0429081699123617, 2.064627920877436], "correct_counts": [0, 4, 8, 11, 19, 21, 38, 49, 52, 38, 51, 40, 27, 21, 15, 10, 5, 5, 1, 1], "incorrect_counts": [2, 1, 2, 3, 11, 7, 21, 24, 18, 17, 23, 18, 20, 8, 3, 3, 1, 1, 1, 0]}, "energy/mean": {"edges": [-0.2394584914048513, -0.1896760197977225, -0.13989354819059374, -0.09011107658346496, -0.04032860497633617, 0.009453866630792618, 0.05923633823792138, 0.10901880984505016, 0.15880128145217895, 0.20858375305930774, 0.25836622466643655, 0.3081486962735652, 0.357931167880694, 0.4077136394878228, 0.4574961110949516, 0.5072785827020804, 0.5570610543092092, 0.606843525916338, 0.6566259975234667, 0.7064084691305955, 0.7561909407377243], "correct_counts": [1, 1, 6, 20, 16, 22, 43, 36, 38, 42, 43, 28, 32, 32, 31, 9, 9, 4, 1, 2], "incorrect_counts": [0, 0, 2, 0, 0, 6, 8, 12, 12, 12, 20, 17, 19, 22, 14, 19, 14, 4, 1, 2]}, "energy/min": {"edges": [-0.5504263639450073, -0.4960724890232086, -0.4417186141014099, -0.38736473917961123, -0.3330108642578125, -0.2786569893360138, -0.22430311441421513, -0.16994923949241642, -0.1155953645706177, -0.06124148964881898, -0.006887614727020264, 0.0474662601947784, 0.10182013511657706, 0.15617401003837583, 0.2105278849601745, 0.26488175988197327, 0.31923563480377193, 0.3735895097255706, 0.42794338464736936, 0.48229725956916814, 0.5366511344909668], "correct_counts": [3, 3, 8, 18, 24, 25, 35, 44, 51, 45, 37, 35, 26, 22, 17, 14, 5, 0, 3, 1], "incorrect_counts": [0, 0, 0, 4, 4, 3, 11, 16, 11, 17, 19, 18, 25, 20, 14, 12, 5, 1, 3, 1]}, "energy/std": {"edges": [0.07470156430161684, 0.08958366354400084, 0.10446576278638484, 0.11934786202876882, 0.13422996127115283, 0.14911206051353681, 0.1639941597559208, 0.17887625899830478, 0.1937583582406888, 0.20864045748307278, 0.2235225567254568, 0.23840465596784077, 0.25328675521022476, 0.26816885445260874, 0.2830509536949928, 0.29793305293737676, 0.31281515217976075, 0.32769725142214473, 0.3425793506645287, 0.35746144990691275, 0.37234354914929674], "correct_counts": [2, 3, 10, 21, 30, 45, 56, 53, 49, 37, 39, 29, 16, 15, 4, 0, 4, 1, 1, 1], "incorrect_counts": [0, 0, 4, 7, 12, 22, 27, 26, 28, 16, 9, 13, 9, 4, 2, 4, 1, 0, 0, 0]}, "curv/lmax_mean": {"edges": [1.000376323858897, 1.0019486089547476, 1.0035208940505982, 1.0050931791464488, 1.0066654642422994, 1.00823774933815, 1.0098100344340006, 1.0113823195298512, 1.0129546046257019, 1.0145268897215525, 1.0160991748174033, 1.017671459913254, 1.0192437450091045, 1.0208160301049551, 1.0223883152008058, 1.0239606002966564, 1.025532885392507, 1.0271051704883576, 1.0286774555842082, 1.0302497406800588, 1.0318220257759094], "correct_counts": [40, 93, 93, 78, 31, 33, 12, 13, 9, 8, 2, 1, 0, 2, 0, 0, 0, 0, 0, 1], "incorrect_counts": [9, 39, 42, 33, 23, 16, 11, 0, 6, 1, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.000000238418579, 1.0040149986743927, 1.0080297589302063, 1.01204451918602, 1.0160592794418335, 1.020074039697647, 1.0240887999534607, 1.0281035602092743, 1.032118320465088, 1.0361330807209015, 1.040147840976715, 1.0441626012325287, 1.0481773614883423, 1.0521921217441559, 1.0562068819999695, 1.060221642255783, 1.0642364025115967, 1.0682511627674103, 1.0722659230232239, 1.0762806832790375, 1.080295443534851], "correct_counts": [260, 66, 30, 25, 15, 5, 4, 3, 3, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [112, 33, 17, 3, 4, 4, 6, 2, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.01093864440918, 32.016456858317056, 32.02197507222493, 32.02749328613281, 32.03301150004069, 32.03852971394857, 32.04404792785645, 32.04956614176432, 32.0550843556722, 32.06060256958008, 32.06612078348796, 32.07163899739584, 32.077157211303714, 32.08267542521159, 32.08819363911947, 32.093711853027344, 32.09923006693522, 32.104748280843104, 32.11026649475098, 32.11578470865886, 32.121302922566734], "correct_counts": [3, 1, 12, 19, 29, 44, 48, 53, 54, 50, 33, 21, 16, 11, 7, 8, 3, 1, 1, 2], "incorrect_counts": [0, 0, 0, 7, 12, 17, 25, 30, 20, 25, 17, 14, 4, 3, 4, 2, 2, 1, 1, 0]}, "curv/trace_best": {"edges": [32.00436782836914, 32.01764259338379, 32.03091735839844, 32.04419212341308, 32.05746688842773, 32.07074165344238, 32.08401641845703, 32.09729118347168, 32.110565948486325, 32.123840713500975, 32.137115478515625, 32.150390243530275, 32.163665008544925, 32.17693977355957, 32.19021453857422, 32.20348930358887, 32.21676406860352, 32.23003883361817, 32.24331359863281, 32.25658836364746, 32.26986312866211], "correct_counts": [22, 78, 72, 75, 64, 36, 21, 20, 10, 6, 4, 3, 1, 2, 0, 1, 0, 0, 0, 1], "incorrect_counts": [5, 30, 32, 33, 27, 17, 16, 6, 8, 4, 3, 2, 1, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6516666666666666, 0.6576666666666666, 0.6636666666666666, 0.6696666666666666, 0.6756666666666666, 0.6816666666666666, 0.6876666666666666, 0.6936666666666667, 0.6996666666666667, 0.7056666666666667, 0.7116666666666667, 0.7176666666666667, 0.7236666666666667, 0.7296666666666668, 0.7356666666666668, 0.7416666666666668, 0.7476666666666668, 0.7536666666666668, 0.7596666666666668, 0.7656666666666668, 0.7716666666666668], "correct_counts": [2, 8, 8, 19, 25, 39, 49, 45, 61, 51, 35, 24, 20, 10, 8, 4, 5, 0, 1, 2], "incorrect_counts": [1, 7, 8, 17, 12, 25, 30, 19, 22, 15, 15, 6, 0, 1, 1, 3, 1, 0, 1, 0]}, "dynamics/drop": {"edges": [13.908043444156647, 21.40915985889733, 28.91027627363801, 36.41139268837869, 43.912509103119376, 51.413625517860055, 58.914741932600734, 66.41585834734141, 73.9169747620821, 81.41809117682278, 88.91920759156346, 96.42032400630414, 103.92144042104482, 111.4225568357855, 118.92367325052619, 126.42478966526687, 133.92590608000756, 141.42702249474823, 148.92813890948892, 156.4292553242296, 163.93037173897028], "correct_counts": [76, 122, 102, 58, 24, 17, 8, 2, 1, 2, 2, 0, 0, 0, 1, 0, 0, 0, 0, 1], "incorrect_counts": [61, 68, 30, 16, 4, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471071392297745, 1.1626865026851496, 1.1782658661405245, 1.1938452295958997, 1.2094245930512746, 1.2250039565066497, 1.2405833199620246, 1.2561626834173998, 1.2717420468727747, 1.2873214103281498, 1.302900773783525, 1.3184801372389, 1.3340595006942748, 1.34963886414965, 1.365218227605025, 1.3807975910604, 1.3963769545157751, 1.41195631797115, 1.4275356814265252, 1.4431150448819001, 1.4586944083372753], "correct_counts": [2, 3, 7, 6, 24, 31, 42, 53, 54, 48, 56, 24, 33, 15, 10, 3, 4, 0, 0, 1], "incorrect_counts": [0, 0, 4, 3, 15, 13, 21, 18, 15, 27, 29, 14, 17, 4, 2, 1, 0, 0, 0, 1]}}}, "ece_geometry": 0.04224579057460222}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "fd77d5eea3e7", "timestamp": "2026-07-26T13:07:25.138708+00:00", "git_sha": "1e6727a", "config_hash": "ff22f3433ff5", "config": {"run": {"seed": 2, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6683333333333333, "base_error": 0.33166666666666667, "final_train_loss": 0.178430438041687, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.16709451083285565, "rho_basin": 0.28365045788141036, "energy_min": 0.37782573026863475, "energy_mean": 0.3915673002869955, "energy_std": 0.35511939449119495}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.2107312194357791, 0.16540577903174386, 0.2560328602083732], "delta_aurc_vs_best_energy": [0.1880248836583393, 0.14315050073946656, 0.23734906074468637], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.027777777777777887, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.09444444444444461, 0.10416666666666667, 0.10784313725490197, 0.12499999999999999, 0.13596491228070176, 0.15, 0.15079365079365076, 0.16287878787878787, 0.17028985507246377, 0.18402777777777776, 0.18666666666666668, 0.1891025641025641, 0.1882716049382716, 0.19345238095238096, 0.2011494252873563, 0.20833333333333334, 0.20967741935483872, 0.20833333333333334, 0.21212121212121213, 0.22549019607843138, 0.2333333333333333, 0.23842592592592604, 0.23648648648648649, 0.24342105263157895, 0.25, 0.25833333333333336, 0.27032520325203263, 0.2797619047619047, 0.28875968992248063, 0.29734848484848486, 0.3, 0.3061594202898551, 0.3085106382978723, 0.31597222222222227, 0.32482993197278914, 0.33166666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2787878787878788, 0.278787878787879, 0.27878787878787903, 0.2787878787878791, 0.2787878787878791, 0.2787878787878788, 0.27878787878787864, 0.2787878787878785, 0.27878787878787836, 0.2787878787878784, 0.27878787878787864, 0.27878787878787886, 0.27878787878787903, 0.27878787878787914, 0.2787878787878793, 0.2787878787878794, 0.2787878787878795, 0.2787878787878796, 0.27878787878787964, 0.27878787878787975, 0.2787878787878798, 0.27878787878787986, 0.2787878787878799, 0.2787878787878799, 0.27878787878788, 0.27878787878788003, 0.2787878787878801, 0.2787878787878801, 0.27878787878788014, 0.27878787878788014, 0.2787878787878802, 0.2787878787878802, 0.27878787878788025, 0.27878787878788025, 0.2787878787878803, 0.2787878787878803, 0.2787878787878803, 0.27878787878788036, 0.27878787878788036, 0.27878787878788036, 0.2787878787878804, 0.283928571428573, 0.29050387596899363, 0.29793123543123673, 0.30722222222222356, 0.3103260869565228, 0.31414267834793586, 0.31864316239316337, 0.3231292517006814, 0.33166666666666755]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.2916666666666667, 0.3611111111111111, 0.375, 0.35, 0.38888888888888884, 0.40476190476190466, 0.4166666666666667, 0.4351851851851852, 0.42499999999999993, 0.4318181818181818, 0.4166666666666667, 0.40384615384615385, 0.3869047619047619, 0.388888888888889, 0.390625, 0.4019607843137255, 0.40277777777777773, 0.39473684210526316, 0.3958333333333333, 0.3928571428571429, 0.4015151515151515, 0.39492753623188404, 0.39236111111111105, 0.3933333333333333, 0.3942307692307692, 0.3950617283950617, 0.3869047619047619, 0.37931034482758613, 0.37777777777777777, 0.3736559139784946, 0.3645833333333333, 0.3661616161616162, 0.3627450980392157, 0.3666666666666666, 0.3657407407407407, 0.36261261261261263, 0.36622807017543857, 0.36538461538461536, 0.36875, 0.36382113821138207, 0.3591269841269841, 0.3507751937984496, 0.3503787878787879, 0.35185185185185186, 0.3496376811594203, 0.3439716312056737, 0.34027777777777773, 0.33503401360544216, 0.33166666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4791666666666667, 0.45, 0.45833333333333326, 0.4166666666666668, 0.3958333333333333, 0.42592592592592593, 0.45833333333333326, 0.4621212121212121, 0.4513888888888889, 0.4423076923076923, 0.44047619047619047, 0.4333333333333333, 0.4270833333333333, 0.4117647058823529, 0.40740740740740733, 0.39473684210526316, 0.38333333333333336, 0.3928571428571428, 0.39015151515151514, 0.38768115942028986, 0.38541666666666663, 0.3933333333333333, 0.40384615384615385, 0.4074074074074074, 0.40773809523809523, 0.410919540229885, 0.40555555555555556, 0.4032258064516129, 0.3932291666666667, 0.39141414141414144, 0.38480392156862747, 0.3761904761904763, 0.3726851851851851, 0.3738738738738739, 0.3684210526315789, 0.36324786324786323, 0.36041666666666666, 0.3577235772357723, 0.3591269841269841, 0.35658914728682173, 0.3522727272727273, 0.35, 0.34601449275362317, 0.34219858156028365, 0.33854166666666674, 0.33503401360544216, 0.33166666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.4583333333333333, 0.5, 0.4583333333333333, 0.45, 0.41666666666666663, 0.4166666666666668, 0.3958333333333333, 0.3888888888888889, 0.39166666666666655, 0.36363636363636365, 0.3611111111111111, 0.36538461538461536, 0.35119047619047616, 0.3499999999999999, 0.3645833333333333, 0.3627450980392157, 0.3611111111111112, 0.3684210526315789, 0.3875, 0.3690476190476191, 0.35984848484848486, 0.35144927536231885, 0.34027777777777773, 0.3333333333333333, 0.3333333333333333, 0.3395061728395062, 0.33630952380952384, 0.3304597701149425, 0.325, 0.3279569892473118, 0.3255208333333333, 0.3282828282828283, 0.31862745098039214, 0.3166666666666666, 0.31481481481481494, 0.31981981981981983, 0.31798245614035087, 0.3162393162393162, 0.3145833333333333, 0.31504065040650403, 0.31349206349206343, 0.313953488372093, 0.3162878787878788, 0.32037037037037036, 0.3278985507246377, 0.3315602836879432, 0.32986111111111105, 0.3299319727891156, 0.33166666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.16541353383458646, 0.2751004016064257], "coverage": [0.0, 0.0, 0.0, 0.0, 0.44333333333333336, 0.83]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5988734194664094, "basin/entropy": 0.4017092946026893, "basin/dispersion": 0.5424253436759859, "energy/mean": 0.6074136267371771, "energy/min": 0.5868243962956929, "energy/std": 0.5071742753668592, "curv/lmax_mean": 0.3702302033860073, "curv/lmax_best": 0.38854496923520343, "curv/trace_mean": 0.3700798255617238, "curv/trace_best": 0.38669657514505196, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6523452674845549, "dynamics/drop": 0.7109612902417324, "dynamics/residual": 0.5072619957643579}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [0, 0, 3, 0, 0, 6, 0, 0, 8, 0, 0, 11, 0, 0, 3, 0, 0, 13, 0, 357], "incorrect_counts": [2, 0, 7, 0, 0, 7, 0, 0, 9, 0, 0, 9, 0, 0, 10, 0, 0, 17, 0, 138]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [357, 0, 0, 0, 0, 13, 0, 0, 3, 0, 10, 7, 8, 1, 0, 1, 0, 0, 1, 0], "incorrect_counts": [138, 0, 0, 0, 0, 17, 0, 0, 10, 0, 9, 9, 13, 0, 0, 0, 0, 1, 0, 2]}, "basin/dispersion": {"edges": [1.6439221991114048, 1.6628586470572428, 1.6817950950030807, 1.7007315429489187, 1.7196679908947567, 1.7386044388405946, 1.7575408867864326, 1.7764773347322707, 1.7954137826781085, 1.8143502306239465, 1.8332866785697846, 1.8522231265156224, 1.8711595744614604, 1.8900960224072985, 1.9090324703531363, 1.9279689182989743, 1.9469053662448124, 1.9658418141906502, 1.9847782621364882, 2.0037147100823263, 2.022651158028164], "correct_counts": [1, 2, 3, 13, 14, 13, 26, 31, 36, 41, 34, 51, 43, 31, 21, 18, 14, 5, 1, 3], "incorrect_counts": [1, 3, 2, 10, 7, 6, 16, 15, 19, 18, 20, 23, 22, 17, 6, 5, 4, 3, 1, 1]}, "energy/mean": {"edges": [0.41955216725667316, 0.4642234648267428, 0.5088947623968124, 0.5535660599668821, 0.5982373575369517, 0.6429086551070213, 0.6875799526770909, 0.7322512502471605, 0.7769225478172302, 0.8215938453873, 0.8662651429573696, 0.9109364405274392, 0.9556077380975088, 1.0002790356675784, 1.044950333237648, 1.0896216308077178, 1.1342929283777874, 1.178964225947857, 1.2236355235179266, 1.2683068210879962, 1.3129781186580658], "correct_counts": [5, 2, 3, 7, 13, 19, 17, 40, 47, 35, 45, 54, 40, 26, 26, 12, 4, 4, 1, 1], "incorrect_counts": [2, 1, 5, 5, 13, 9, 21, 25, 18, 28, 25, 16, 13, 10, 4, 2, 2, 0, 0, 0]}, "energy/min": {"edges": [0.02020469307899475, 0.07093883007764816, 0.12167296707630157, 0.17240710407495496, 0.2231412410736084, 0.2738753780722618, 0.3246095150709152, 0.3753436520695686, 0.426077789068222, 0.47681192606687545, 0.5275460630655289, 0.5782802000641822, 0.6290143370628356, 0.6797484740614891, 0.7304826110601425, 0.7812167480587959, 0.8319508850574493, 0.8826850220561027, 0.9334191590547561, 0.9841532960534095, 1.034887433052063], "correct_counts": [0, 2, 5, 10, 7, 16, 17, 35, 39, 31, 47, 49, 45, 41, 22, 16, 8, 7, 3, 1], "incorrect_counts": [1, 1, 2, 2, 9, 6, 22, 19, 25, 21, 22, 17, 30, 9, 8, 3, 0, 2, 0, 0]}, "energy/std": {"edges": [0.08805419068698145, 0.10022669760053629, 0.11239920451409115, 0.12457171142764599, 0.13674421834120085, 0.1489167252547557, 0.16108923216831053, 0.17326173908186537, 0.18543424599542024, 0.19760675290897506, 0.20977925982252993, 0.22195176673608477, 0.2341242736496396, 0.24629678056319446, 0.25846928747674935, 0.27064179439030417, 0.282814301303859, 0.29498680821741385, 0.3071593151309687, 0.3193318220445236, 0.3315043289580784], "correct_counts": [2, 1, 6, 14, 23, 31, 33, 30, 51, 42, 52, 35, 28, 18, 11, 10, 5, 3, 4, 2], "incorrect_counts": [0, 4, 7, 7, 14, 15, 15, 21, 15, 17, 18, 15, 10, 14, 14, 5, 2, 3, 3, 0]}, "curv/lmax_mean": {"edges": [0.9991512497266134, 0.9997431829571725, 1.0003351161877314, 1.0009270494182905, 1.0015189826488495, 1.0021109158794086, 1.0027028491099677, 1.0032947823405267, 1.0038867155710856, 1.0044786488016446, 1.0050705820322037, 1.0056625152627627, 1.0062544484933218, 1.0068463817238809, 1.00743831495444, 1.008030248184999, 1.008622181415558, 1.0092141146461169, 1.009806047876676, 1.010397981107235, 1.010989914337794], "correct_counts": [76, 176, 62, 38, 24, 9, 7, 4, 1, 0, 1, 2, 0, 0, 0, 0, 0, 1, 0, 0], "incorrect_counts": [22, 68, 31, 26, 21, 10, 5, 1, 0, 6, 2, 3, 1, 1, 0, 0, 0, 0, 1, 1]}, "curv/lmax_best": {"edges": [0.994626522064209, 0.996795529127121, 0.998964536190033, 1.001133543252945, 1.003302550315857, 1.005471557378769, 1.0076405644416808, 1.009809571504593, 1.0119785785675048, 1.014147585630417, 1.0163165926933289, 1.0184855997562408, 1.0206546068191529, 1.0228236138820648, 1.024992620944977, 1.0271616280078888, 1.0293306350708007, 1.0314996421337128, 1.0336686491966247, 1.0358376562595368, 1.0380066633224487], "correct_counts": [1, 11, 344, 27, 7, 5, 4, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 5, 149, 23, 9, 2, 2, 3, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/trace_mean": {"edges": [31.952821254730225, 31.960075608889262, 31.9673299630483, 31.974584317207338, 31.981838671366376, 31.989093025525413, 31.996347379684448, 32.00360173384349, 32.01085608800253, 32.018110442161564, 32.0253647963206, 32.03261915047963, 32.03987350463867, 32.04712785879771, 32.054382212956746, 32.061636567115784, 32.06889092127482, 32.07614527543386, 32.0833996295929, 32.090653983751935, 32.09790833791097], "correct_counts": [3, 2, 12, 26, 51, 59, 58, 53, 50, 35, 21, 17, 8, 2, 3, 0, 0, 1, 0, 0], "incorrect_counts": [0, 1, 4, 5, 17, 16, 28, 26, 22, 27, 19, 11, 12, 3, 5, 0, 2, 0, 0, 1]}, "curv/trace_best": {"edges": [31.93198013305664, 31.942170333862304, 31.952360534667967, 31.962550735473634, 31.972740936279298, 31.98293113708496, 31.993121337890624, 32.00331153869629, 32.013501739501955, 32.023691940307614, 32.03388214111328, 32.04407234191895, 32.05426254272461, 32.064452743530275, 32.074642944335935, 32.0848331451416, 32.09502334594727, 32.10521354675293, 32.115403747558595, 32.125593948364255, 32.13578414916992], "correct_counts": [2, 2, 4, 11, 33, 70, 97, 88, 39, 24, 19, 4, 3, 3, 1, 1, 0, 0, 0, 0], "incorrect_counts": [0, 1, 1, 3, 9, 24, 41, 39, 29, 21, 10, 8, 3, 3, 2, 1, 2, 0, 0, 2]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [401, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6266666666666666, 0.6331666666666667, 0.6396666666666666, 0.6461666666666666, 0.6526666666666666, 0.6591666666666667, 0.6656666666666666, 0.6721666666666666, 0.6786666666666666, 0.6851666666666667, 0.6916666666666667, 0.6981666666666666, 0.7046666666666667, 0.7111666666666667, 0.7176666666666667, 0.7241666666666666, 0.7306666666666667, 0.7371666666666667, 0.7436666666666667, 0.7501666666666666, 0.7566666666666667], "correct_counts": [0, 0, 0, 0, 5, 7, 13, 22, 27, 49, 58, 43, 51, 41, 32, 21, 16, 9, 3, 4], "incorrect_counts": [1, 0, 0, 0, 2, 6, 17, 28, 18, 27, 34, 20, 16, 18, 5, 1, 4, 2, 0, 0]}, "dynamics/drop": {"edges": [14.020859281222025, 18.926390168940028, 23.83192105665803, 28.737451944376033, 33.642982832094035, 38.548513719812036, 43.45404460753004, 48.35957549524804, 53.26510638296604, 58.17063727068404, 63.07616815840204, 67.98169904612004, 72.88722993383806, 77.79276082155606, 82.69829170927406, 87.60382259699206, 92.50935348471006, 97.41488437242806, 102.32041526014606, 107.22594614786406, 112.13147703558207], "correct_counts": [19, 90, 74, 61, 44, 39, 28, 17, 11, 9, 3, 3, 2, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [35, 66, 50, 28, 11, 4, 3, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1569731881221135, 1.1719582500557104, 1.1869433119893074, 1.2019283739229043, 1.2169134358565012, 1.2318984977900982, 1.2468835597236951, 1.261868621657292, 1.276853683590889, 1.291838745524486, 1.3068238074580827, 1.3218088693916799, 1.3367939313252766, 1.3517789932588735, 1.3667640551924705, 1.3817491171260674, 1.3967341790596643, 1.4117192409932613, 1.4267043029268582, 1.4416893648604552, 1.4566744267940521], "correct_counts": [0, 7, 3, 13, 26, 24, 43, 49, 49, 46, 46, 34, 24, 19, 10, 4, 2, 1, 0, 1], "incorrect_counts": [5, 2, 5, 2, 16, 11, 18, 23, 22, 25, 21, 20, 12, 9, 3, 4, 0, 1, 0, 0]}}}, "ece_geometry": 0.03781877930416236}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "2731d2172402", "timestamp": "2026-07-26T13:07:30.418664+00:00", "git_sha": "1e6727a", "config_hash": "fbed14eeb756", "config": {"run": {"seed": 3, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7133333333333334, "base_error": 0.2866666666666667, "final_train_loss": 0.16939383745193481, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.17019370187580518, "rho_basin": 0.23474252197420142, "energy_min": 0.3728384148545108, "energy_mean": 0.38232702405148095, "energy_std": 0.3016550018932722}, "best_energy_baseline": "energy_std", "delta_aurc_vs_energy_min": [0.20264471297870562, 0.14299561878502962, 0.2569749266815167], "delta_aurc_vs_best_energy": [0.13146130001746703, 0.08951962500452121, 0.17179871890451937], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.041666666666666664, 0.08333333333333333, 0.09722222222222221, 0.09523809523809505, 0.125, 0.1388888888888889, 0.13333333333333336, 0.13636363636363635, 0.13194444444444445, 0.12179487179487179, 0.11904761904761904, 0.12222222222222219, 0.13020833333333334, 0.13725490196078433, 0.12962962962962976, 0.14035087719298245, 0.14583333333333334, 0.1706349206349206, 0.17424242424242425, 0.17391304347826086, 0.18055555555555552, 0.18, 0.18269230769230768, 0.18209876543209877, 0.18154761904761904, 0.18678160919540227, 0.18055555555555555, 0.18548387096774194, 0.19270833333333334, 0.19696969696969696, 0.2034313725490196, 0.20476190476190473, 0.20833333333333331, 0.21396396396396397, 0.22807017543859648, 0.2329059829059829, 0.2375, 0.23780487804878045, 0.23611111111111108, 0.2441860465116279, 0.25, 0.2574074074074074, 0.26268115942028986, 0.27304964539007087, 0.2760416666666668, 0.28401360544217685, 0.2866666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.22920892494929004, 0.2292089249492901, 0.22920892494929015, 0.2292089249492902, 0.22920892494929024, 0.22920892494929024, 0.22920892494929027, 0.22920892494929027, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.23576097105508903, 0.2425900592795261, 0.2492715617715621, 0.2574074074074077, 0.26518952062430345, 0.27160904255319174, 0.27650462962962985, 0.28099017384731684, 0.28666666666666685]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.5, 0.5277777777777778, 0.5208333333333334, 0.4666666666666667, 0.4305555555555555, 0.4404761904761906, 0.40625, 0.4166666666666667, 0.4083333333333334, 0.4090909090909091, 0.4097222222222222, 0.40384615384615385, 0.39880952380952384, 0.39444444444444454, 0.3958333333333333, 0.39705882352941174, 0.3935185185185185, 0.38596491228070173, 0.375, 0.37698412698412703, 0.3787878787878788, 0.36594202898550726, 0.3680555555555555, 0.36, 0.36217948717948717, 0.3611111111111111, 0.3541666666666667, 0.3448275862068967, 0.34444444444444444, 0.34139784946236557, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.3261904761904763, 0.3240740740740742, 0.32207207207207206, 0.3157894736842105, 0.30982905982905984, 0.30625, 0.3028455284552845, 0.3015873015873017, 0.29651162790697677, 0.29924242424242425, 0.29444444444444445, 0.2916666666666667, 0.2943262411347518, 0.29166666666666663, 0.2891156462585034, 0.2866666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.6666666666666666, 0.5416666666666666, 0.5277777777777778, 0.5833333333333334, 0.5166666666666667, 0.48611111111111105, 0.4523809523809525, 0.4375, 0.39814814814814814, 0.3833333333333334, 0.4015151515151515, 0.4027777777777778, 0.41025641025641024, 0.4226190476190476, 0.4166666666666668, 0.4166666666666667, 0.4019607843137255, 0.4027777777777779, 0.39035087719298245, 0.38333333333333336, 0.3690476190476191, 0.35984848484848486, 0.36231884057971014, 0.36111111111111116, 0.36, 0.358974358974359, 0.3549382716049383, 0.34523809523809523, 0.3448275862068967, 0.3416666666666667, 0.3387096774193548, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.32380952380952377, 0.324074074074074, 0.31981981981981983, 0.3157894736842105, 0.31196581196581197, 0.3104166666666667, 0.30894308943089427, 0.3055555555555555, 0.3023255813953488, 0.29734848484848486, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.2899305555555555, 0.29081632653061223, 0.2866666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.2916666666666667, 0.3055555555555556, 0.3541666666666667, 0.36666666666666664, 0.33333333333333326, 0.29761904761904767, 0.3125, 0.3055555555555556, 0.29166666666666674, 0.2878787878787879, 0.2708333333333333, 0.2948717948717949, 0.2976190476190476, 0.2944444444444444, 0.2864583333333333, 0.29411764705882354, 0.3009259259259259, 0.29385964912280704, 0.3, 0.2936507936507936, 0.29924242424242425, 0.2898550724637681, 0.29513888888888895, 0.29333333333333333, 0.28846153846153844, 0.28703703703703703, 0.27976190476190477, 0.278735632183908, 0.2833333333333333, 0.28225806451612906, 0.2838541666666667, 0.28535353535353536, 0.2818627450980392, 0.28333333333333344, 0.287037037037037, 0.28603603603603606, 0.29385964912280704, 0.2948717948717949, 0.29791666666666666, 0.2947154471544715, 0.2916666666666668, 0.29069767441860467, 0.2897727272727273, 0.28703703703703703, 0.2916666666666667, 0.2907801418439717, 0.29340277777777773, 0.2925170068027211, 0.2866666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.09090909090909091, 0.2038369304556355], "coverage": [0.0, 0.0, 0.0, 0.0, 0.12833333333333333, 0.695]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6152942295153228, "basin/entropy": 0.38495707454901107, "basin/dispersion": 0.5045913931753967, "energy/mean": 0.6219707672245164, "energy/min": 0.6181264942403826, "energy/std": 0.5105955227124538, "curv/lmax_mean": 0.3801755053249294, "curv/lmax_best": 0.38900510758530754, "curv/trace_mean": 0.37158362312540755, "curv/trace_best": 0.3750747120191263, "dynamics/steps": 0.5, "dynamics/monotonic": 0.5918346555096718, "dynamics/drop": 0.6229623994783743, "dynamics/residual": 0.4728048250380352}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [3, 0, 0, 4, 0, 0, 8, 0, 0, 0, 7, 0, 0, 10, 0, 0, 16, 0, 0, 380], "incorrect_counts": [4, 0, 0, 5, 0, 0, 7, 0, 0, 0, 9, 0, 0, 16, 0, 0, 18, 0, 0, 113]}, "basin/entropy": {"edges": [0.0, 0.05057021323536759, 0.10114042647073518, 0.15171063970610277, 0.20228085294147036, 0.25285106617683795, 0.30342127941220554, 0.3539914926475731, 0.4045617058829407, 0.4551319191183083, 0.5057021323536759, 0.5562723455890435, 0.6068425588244111, 0.6574127720597787, 0.7079829852951462, 0.7585531985305138, 0.8091234117658814, 0.859693625001249, 0.9102638382366166, 0.9608340514719842, 1.0114042647073518], "correct_counts": [380, 0, 0, 0, 0, 16, 0, 0, 10, 0, 0, 5, 7, 7, 2, 0, 1, 0, 0, 0], "incorrect_counts": [113, 0, 0, 0, 0, 18, 0, 0, 16, 0, 0, 9, 5, 7, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.638197206426302, 1.6606953621248024, 1.6831935178233026, 1.705691673521803, 1.7281898292203033, 1.7506879849188037, 1.773186140617304, 1.7956842963158044, 1.8181824520143046, 1.840680607712805, 1.8631787634113053, 1.8856769191098057, 1.9081750748083062, 1.9306732305068064, 1.9531713862053068, 1.975669541903807, 1.9981676976023075, 2.020665853300808, 2.043164008999308, 2.0656621646978084, 2.088160320396309], "correct_counts": [2, 7, 9, 12, 27, 47, 52, 47, 61, 52, 32, 23, 29, 9, 6, 8, 4, 0, 0, 1], "incorrect_counts": [3, 0, 6, 12, 8, 15, 19, 17, 23, 20, 18, 13, 10, 0, 5, 2, 1, 0, 0, 0]}, "energy/mean": {"edges": [0.4687643547852834, 0.5167818620800972, 0.5647993693749109, 0.6128168766697248, 0.6608343839645385, 0.7088518912593524, 0.7568693985541661, 0.80488690584898, 0.8529044131437937, 0.9009219204386076, 0.9489394277334213, 0.9969569350282352, 1.044974442323049, 1.0929919496178628, 1.1410094569126765, 1.1890269642074904, 1.2370444715023041, 1.285061978797118, 1.3330794860919317, 1.3810969933867456, 1.4291145006815593], "correct_counts": [0, 0, 2, 9, 12, 33, 30, 23, 44, 31, 37, 43, 34, 39, 35, 24, 18, 5, 6, 3], "incorrect_counts": [1, 1, 3, 9, 15, 13, 15, 21, 14, 11, 13, 16, 10, 8, 8, 6, 5, 3, 0, 0]}, "energy/min": {"edges": [0.13711321353912354, 0.18594235181808472, 0.2347714900970459, 0.2836006283760071, 0.33242976665496826, 0.38125890493392944, 0.4300880432128906, 0.4789171814918518, 0.527746319770813, 0.5765754580497742, 0.6254045963287354, 0.6742337346076965, 0.7230628728866577, 0.7718920111656189, 0.8207211494445801, 0.8695502877235413, 0.9183794260025024, 0.9672085642814636, 1.0160377025604248, 1.064866840839386, 1.1136959791183472], "correct_counts": [0, 1, 4, 5, 10, 21, 25, 40, 32, 31, 30, 47, 36, 46, 32, 24, 13, 17, 11, 3], "incorrect_counts": [2, 3, 1, 1, 15, 9, 16, 24, 16, 13, 13, 14, 10, 9, 7, 6, 7, 3, 2, 1]}, "energy/std": {"edges": [0.07382017921196296, 0.09043346825542178, 0.10704675729888061, 0.12366004634233943, 0.14027333538579825, 0.15688662442925708, 0.1734999134727159, 0.19011320251617472, 0.20672649155963355, 0.22333978060309237, 0.2399530696465512, 0.25656635869001, 0.2731796477334688, 0.28979293677692763, 0.30640622582038646, 0.3230195148638453, 0.3396328039073041, 0.3562460929507629, 0.37285938199422175, 0.3894726710376806, 0.40608596008113945], "correct_counts": [0, 6, 13, 19, 47, 60, 63, 62, 47, 48, 25, 11, 6, 11, 6, 1, 2, 0, 0, 1], "incorrect_counts": [1, 4, 3, 14, 14, 25, 26, 20, 19, 24, 7, 10, 2, 3, 0, 0, 0, 0, 0, 0]}, "curv/lmax_mean": {"edges": [0.9988449911276499, 0.9992352331678073, 0.9996254752079645, 1.0000157172481219, 1.0004059592882792, 1.0007962013284366, 1.0011864433685937, 1.001576685408751, 1.0019669274489085, 1.0023571694890658, 1.0027474115292232, 1.0031376535693806, 1.0035278956095377, 1.003918137649695, 1.0043083796898524, 1.0046986217300098, 1.0050888637701672, 1.0054791058103245, 1.0058693478504817, 1.006259589890639, 1.0066498319307964], "correct_counts": [12, 134, 143, 44, 37, 20, 17, 5, 5, 3, 5, 2, 0, 0, 0, 1, 0, 0, 0, 0], "incorrect_counts": [2, 26, 55, 28, 25, 17, 7, 3, 1, 4, 0, 2, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.995161235332489, 0.9962190896272659, 0.9972769439220428, 0.9983347982168198, 0.9993926525115967, 1.0004505068063736, 1.0015083611011506, 1.0025662153959274, 1.0036240696907044, 1.0046819239854812, 1.0057397782802582, 1.0067976325750352, 1.007855486869812, 1.008913341164589, 1.0099711954593658, 1.0110290497541428, 1.0120869040489198, 1.0131447583436965, 1.0142026126384736, 1.0152604669332503, 1.0163183212280273], "correct_counts": [1, 3, 10, 64, 293, 33, 14, 3, 1, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 0, 6, 11, 120, 17, 7, 3, 3, 0, 2, 0, 0, 0, 0, 1, 0, 1, 0, 1]}, "curv/trace_mean": {"edges": [31.904047807057697, 31.911127193768817, 31.91820658047994, 31.92528596719106, 31.93236535390218, 31.9394447406133, 31.946524127324423, 31.953603514035542, 31.960682900746665, 31.967762287457784, 31.974841674168907, 31.981921060880026, 31.989000447591145, 31.996079834302268, 32.00315922101339, 32.01023860772451, 32.01731799443563, 32.02439738114675, 32.031476767857875, 32.038556154568994, 32.04563554128011], "correct_counts": [3, 2, 4, 13, 19, 14, 19, 32, 51, 37, 20, 31, 34, 31, 28, 36, 27, 14, 8, 5], "incorrect_counts": [0, 0, 0, 1, 3, 3, 10, 9, 5, 10, 13, 15, 10, 11, 19, 25, 18, 10, 8, 2]}, "curv/trace_best": {"edges": [31.868419647216797, 31.879567527770995, 31.890715408325196, 31.901863288879394, 31.913011169433595, 31.924159049987793, 31.93530693054199, 31.946454811096192, 31.95760269165039, 31.96875057220459, 31.97989845275879, 31.991046333312987, 32.002194213867185, 32.01334209442139, 32.02448997497559, 32.035637855529785, 32.04678573608398, 32.05793361663818, 32.069081497192386, 32.08022937774658, 32.09137725830078], "correct_counts": [1, 2, 3, 6, 11, 13, 24, 26, 46, 55, 60, 60, 53, 37, 14, 11, 3, 1, 2, 0], "incorrect_counts": [0, 0, 1, 1, 5, 2, 2, 8, 9, 18, 26, 20, 29, 26, 15, 6, 1, 2, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [428, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6449999999999999, 0.6505833333333333, 0.6561666666666666, 0.66175, 0.6673333333333332, 0.6729166666666666, 0.6785, 0.6840833333333333, 0.6896666666666667, 0.6952499999999999, 0.7008333333333333, 0.7064166666666667, 0.712, 0.7175833333333334, 0.7231666666666666, 0.72875, 0.7343333333333334, 0.7399166666666667, 0.7455, 0.7510833333333333, 0.7566666666666667], "correct_counts": [3, 3, 9, 15, 16, 32, 38, 31, 53, 40, 32, 54, 32, 20, 21, 10, 7, 10, 0, 2], "incorrect_counts": [1, 2, 5, 6, 12, 19, 17, 15, 31, 17, 10, 13, 9, 4, 5, 3, 2, 0, 1, 0]}, "dynamics/drop": {"edges": [14.044291233023008, 17.461781060944002, 20.879270888864994, 24.29676071678599, 27.71425054470698, 31.131740372627974, 34.54923020054897, 37.966720028469965, 41.38420985639095, 44.80169968431195, 48.219189512232944, 51.63667934015394, 55.05416916807493, 58.471658995995924, 61.88914882391692, 65.3066386518379, 68.7241284797589, 72.14161830767989, 75.55910813560088, 78.97659796352188, 82.39408779144287], "correct_counts": [20, 45, 59, 60, 64, 49, 39, 24, 17, 14, 13, 8, 3, 5, 4, 1, 0, 1, 0, 2], "incorrect_counts": [10, 26, 43, 29, 18, 19, 10, 11, 3, 2, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.157560517390569, 1.171275670826435, 1.184990824262301, 1.1987059776981672, 1.2124211311340332, 1.226136284569899, 1.2398514380057652, 1.2535665914416314, 1.2672817448774973, 1.2809968983133633, 1.2947120517492294, 1.3084272051850956, 1.3221423586209615, 1.3358575120568275, 1.3495726654926936, 1.3632878189285598, 1.3770029723644257, 1.3907181258002916, 1.4044332792361578, 1.418148432672024, 1.4318635861078899], "correct_counts": [2, 1, 7, 4, 14, 21, 36, 48, 45, 57, 40, 31, 33, 24, 24, 20, 10, 5, 3, 3], "incorrect_counts": [0, 2, 2, 5, 1, 6, 8, 20, 15, 26, 19, 19, 15, 13, 7, 7, 2, 2, 1, 2]}}}, "ece_geometry": 0.03179577510102621}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "c13df321889e", "timestamp": "2026-07-26T13:07:35.792696+00:00", "git_sha": "1e6727a", "config_hash": "609a4152cf1e", "config": {"run": {"seed": 4, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.21295076608657837, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.13764728016172775, "rho_basin": 0.24498732350693359, "energy_min": 0.21394670001342045, "energy_mean": 0.21459813139712064, "energy_std": 0.29739590638116215}, "best_energy_baseline": "energy_min", "delta_aurc_vs_energy_min": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "delta_aurc_vs_best_energy": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "geometry_wins": true, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.041666666666666664, 0.03333333333333333, 0.04166666666666666, 0.059523809523809534, 0.052083333333333336, 0.05555555555555555, 0.06666666666666668, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.08888888888888888, 0.09375, 0.09803921568627451, 0.10185185185185183, 0.10526315789473684, 0.10833333333333334, 0.11904761904761903, 0.125, 0.13043478260869565, 0.13194444444444442, 0.13666666666666666, 0.13782051282051283, 0.1419753086419753, 0.14583333333333334, 0.1436781609195402, 0.15555555555555556, 0.16129032258064516, 0.16666666666666666, 0.18181818181818182, 0.18872549019607843, 0.19523809523809538, 0.19444444444444456, 0.19369369369369369, 0.19956140350877194, 0.202991452991453, 0.20833333333333334, 0.21747967479674807, 0.22619047619047616, 0.23255813953488372, 0.24621212121212122, 0.2537037037037037, 0.2608695652173913, 0.2641843971631207, 0.2690972222222223, 0.27721088435374147, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.24151696606786424, 0.24151696606786435, 0.24151696606786446, 0.24151696606786452, 0.24151696606786455, 0.2415169660678644, 0.24151696606786419, 0.24151696606786402, 0.2415169660678639, 0.2415169660678638, 0.24151696606786371, 0.24151696606786366, 0.2415169660678636, 0.24151696606786355, 0.24151696606786352, 0.24151696606786346, 0.24151696606786344, 0.2415169660678634, 0.24151696606786338, 0.24151696606786335, 0.24151696606786333, 0.24151696606786333, 0.2415169660678633, 0.24151696606786327, 0.24151696606786327, 0.24151696606786324, 0.24151696606786324, 0.24151696606786321, 0.24151696606786321, 0.2415169660678632, 0.2415169660678632, 0.2415169660678632, 0.24151696606786316, 0.24151696606786316, 0.24151696606786316, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.2415169660678631, 0.24241780045351358, 0.2459163898117374, 0.2492559523809511, 0.2551146384479704, 0.26169301712779824, 0.2708594075928229, 0.27747140522875685, 0.2808956916099758, 0.2816666666666653]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.1111111111111111, 0.125, 0.15, 0.18055555555555552, 0.1785714285714286, 0.17708333333333334, 0.2037037037037037, 0.1916666666666667, 0.18181818181818182, 0.18055555555555555, 0.17307692307692307, 0.18452380952380953, 0.18333333333333346, 0.19270833333333334, 0.19117647058823528, 0.1851851851851853, 0.20175438596491227, 0.21666666666666667, 0.21428571428571425, 0.22348484848484848, 0.2210144927536232, 0.21527777777777776, 0.21, 0.21474358974358973, 0.21296296296296297, 0.2113095238095238, 0.21264367816091967, 0.21944444444444444, 0.22311827956989247, 0.23177083333333334, 0.23232323232323232, 0.24019607843137256, 0.24999999999999997, 0.25231481481481494, 0.2545045045045045, 0.2543859649122807, 0.25427350427350426, 0.25625, 0.26219512195121947, 0.2718253968253969, 0.27325581395348836, 0.2784090909090909, 0.2851851851851852, 0.2826086956521739, 0.2819148936170214, 0.28472222222222215, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.1388888888888889, 0.16666666666666666, 0.15, 0.13888888888888887, 0.11904761904761907, 0.125, 0.12037037037037036, 0.11666666666666668, 0.12121212121212122, 0.14583333333333334, 0.17307692307692307, 0.19642857142857142, 0.19444444444444442, 0.19270833333333334, 0.18627450980392157, 0.19444444444444456, 0.20614035087719298, 0.20833333333333334, 0.21428571428571425, 0.21212121212121213, 0.20652173913043478, 0.21874999999999997, 0.22666666666666666, 0.22435897435897437, 0.2222222222222222, 0.22321428571428573, 0.2212643678160919, 0.21944444444444444, 0.22043010752688172, 0.22395833333333334, 0.23484848484848486, 0.24019607843137256, 0.23809523809523805, 0.2384259259259259, 0.24324324324324326, 0.25, 0.2606837606837607, 0.26458333333333334, 0.27235772357723587, 0.2738095238095238, 0.2751937984496124, 0.2784090909090909, 0.2777777777777778, 0.27898550724637683, 0.2819148936170214, 0.28298611111111105, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.36111111111111105, 0.3452380952380951, 0.3229166666666667, 0.3148148148148148, 0.30000000000000004, 0.30303030303030304, 0.2986111111111111, 0.2948717948717949, 0.2857142857142857, 0.28333333333333327, 0.28125, 0.28431372549019607, 0.287037037037037, 0.2894736842105263, 0.2916666666666667, 0.2936507936507936, 0.2803030303030303, 0.2826086956521739, 0.28124999999999994, 0.28, 0.27884615384615385, 0.28703703703703703, 0.28273809523809523, 0.2816091954022988, 0.2916666666666667, 0.28225806451612906, 0.2760416666666667, 0.2803030303030303, 0.28431372549019607, 0.28571428571428564, 0.28703703703703715, 0.2905405405405405, 0.28289473684210525, 0.2905982905982906, 0.28958333333333336, 0.290650406504065, 0.2936507936507936, 0.29069767441860467, 0.2916666666666667, 0.28888888888888886, 0.28804347826086957, 0.2836879432624113, 0.28298611111111105, 0.282312925170068, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.13028169014084506, 0.2537037037037037], "coverage": [0.0, 0.0, 0.0, 0.0, 0.47333333333333333, 0.9]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5830942215022172, "basin/entropy": 0.4163291643213114, "basin/dispersion": 0.5237029613256634, "energy/mean": 0.39264679635909333, "energy/min": 0.39925040157058717, "energy/std": 0.5168522357528247, "curv/lmax_mean": 0.5929790359560125, "curv/lmax_best": 0.5272518842927553, "curv/trace_mean": 0.6412773376899737, "curv/trace_best": 0.6202446491577315, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6388473208034158, "dynamics/drop": 0.6872691827180494, "dynamics/residual": 0.5027938329740935}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [1, 0, 6, 0, 0, 7, 0, 0, 7, 0, 0, 3, 0, 0, 10, 0, 0, 17, 0, 380], "incorrect_counts": [0, 0, 3, 0, 0, 5, 0, 0, 10, 0, 0, 8, 0, 0, 11, 0, 0, 11, 0, 121]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [380, 0, 0, 0, 0, 17, 0, 0, 10, 0, 3, 7, 13, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [121, 0, 0, 0, 0, 11, 0, 0, 10, 0, 8, 10, 6, 1, 0, 0, 1, 0, 1, 0]}, "basin/dispersion": {"edges": [1.6070286588464016, 1.6272769139695735, 1.6475251690927455, 1.6677734242159175, 1.6880216793390894, 1.7082699344622614, 1.7285181895854331, 1.748766444708605, 1.769014699831777, 1.789262954954949, 1.809511210078121, 1.829759465201293, 1.850007720324465, 1.8702559754476369, 1.8905042305708089, 1.9107524856939806, 1.9310007408171526, 1.9512489959403245, 1.9714972510634965, 1.9917455061866685, 2.0119937613098404], "correct_counts": [1, 1, 5, 5, 15, 22, 20, 31, 44, 54, 51, 65, 43, 26, 14, 11, 11, 4, 5, 3], "incorrect_counts": [2, 0, 1, 3, 1, 14, 9, 18, 19, 20, 20, 17, 13, 13, 8, 5, 1, 3, 1, 1]}, "energy/mean": {"edges": [0.7167260174949964, 0.7730059100935857, 0.8292858026921749, 0.8855656952907642, 0.9418455878893535, 0.9981254804879427, 1.054405373086532, 1.1106852656851212, 1.1669651582837104, 1.2232450508822996, 1.2795249434808889, 1.335804836079478, 1.3920847286780673, 1.4483646212766565, 1.504644513875246, 1.5609244064738352, 1.6172042990724245, 1.6734841916710137, 1.7297640842696032, 1.7860439768681924, 1.8423238694667816], "correct_counts": [12, 33, 69, 58, 50, 36, 27, 17, 19, 17, 10, 9, 15, 11, 19, 9, 11, 4, 1, 4], "incorrect_counts": [2, 6, 6, 27, 20, 14, 6, 11, 8, 11, 11, 11, 8, 8, 7, 5, 4, 1, 3, 0]}, "energy/min": {"edges": [0.33279895782470703, 0.39212902784347536, 0.45145909786224364, 0.5107891678810119, 0.5701192378997803, 0.6294493079185486, 0.6887793779373169, 0.7481094479560852, 0.8074395179748535, 0.8667695879936218, 0.9260996580123901, 0.9854297280311585, 1.0447597980499268, 1.1040898680686952, 1.1634199380874635, 1.2227500081062317, 1.282080078125, 1.3414101481437684, 1.4007402181625366, 1.460070288181305, 1.5194003582000732], "correct_counts": [4, 17, 34, 53, 57, 47, 49, 24, 22, 14, 20, 16, 12, 13, 10, 15, 8, 8, 6, 2], "incorrect_counts": [1, 0, 11, 12, 15, 22, 9, 12, 10, 16, 8, 8, 15, 11, 5, 5, 5, 3, 0, 1]}, "energy/std": {"edges": [0.06888872658823433, 0.08357696314973125, 0.09826519971122816, 0.11295343627272506, 0.12764167283422195, 0.14232990939571888, 0.15701814595721578, 0.17170638251871267, 0.1863946190802096, 0.2010828556417065, 0.21577109220320342, 0.23045932876470032, 0.24514756532619722, 0.2598358018876942, 0.27452403844919104, 0.28921227501068797, 0.3039005115721849, 0.31858874813368177, 0.33327698469517864, 0.3479652212566756, 0.3626534578181725], "correct_counts": [1, 0, 4, 8, 21, 36, 52, 55, 56, 50, 45, 26, 23, 26, 14, 5, 2, 5, 1, 1], "incorrect_counts": [0, 0, 2, 5, 9, 17, 15, 24, 22, 17, 18, 17, 9, 4, 4, 3, 2, 1, 0, 0]}, "curv/lmax_mean": {"edges": [0.999078502257665, 0.9991983219981194, 0.9993181417385737, 0.9994379614790281, 0.9995577812194825, 0.9996776009599369, 0.9997974207003912, 0.9999172404408455, 1.0000370601812998, 1.0001568799217542, 1.0002766996622086, 1.000396519402663, 1.0005163391431173, 1.0006361588835717, 1.000755978624026, 1.0008757983644805, 1.0009956181049349, 1.001115437845389, 1.0012352575858434, 1.0013550773262978, 1.0014748970667522], "correct_counts": [3, 8, 21, 38, 61, 87, 93, 58, 24, 18, 10, 6, 2, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 5, 5, 21, 32, 48, 38, 10, 5, 3, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.995195746421814, 0.9959201753139496, 0.9966446042060852, 0.9973690330982208, 0.9980934619903564, 0.9988178908824921, 0.9995423197746277, 1.0002667486667634, 1.0009911775588989, 1.0017156064510346, 1.0024400353431702, 1.0031644642353057, 1.0038888931274415, 1.004613322019577, 1.0053377509117127, 1.0060621798038483, 1.0067866086959838, 1.0075110375881196, 1.008235466480255, 1.0089598953723908, 1.0096843242645264], "correct_counts": [2, 1, 2, 1, 13, 45, 337, 24, 3, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 1, 0, 1, 6, 26, 130, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [31.831074873606365, 31.84024980068207, 31.84942472775777, 31.858599654833476, 31.86777458190918, 31.876949508984886, 31.88612443606059, 31.895299363136292, 31.904474290211997, 31.913649217287702, 31.922824144363403, 31.931999071439108, 31.941173998514813, 31.950348925590518, 31.959523852666223, 31.968698779741924, 31.97787370681763, 31.987048633893334, 31.996223560969035, 32.00539848804474, 32.014573415120445], "correct_counts": [1, 4, 6, 12, 12, 19, 14, 18, 19, 15, 14, 9, 12, 16, 10, 25, 44, 90, 75, 16], "incorrect_counts": [1, 1, 4, 8, 7, 11, 15, 9, 8, 15, 11, 2, 3, 9, 9, 12, 12, 18, 11, 3]}, "curv/trace_best": {"edges": [31.66323471069336, 31.682111549377442, 31.700988388061525, 31.719865226745604, 31.738742065429687, 31.75761890411377, 31.776495742797852, 31.795372581481935, 31.814249420166014, 31.833126258850097, 31.85200309753418, 31.870879936218262, 31.889756774902345, 31.908633613586424, 31.927510452270507, 31.94638729095459, 31.965264129638673, 31.984140968322755, 32.003017807006835, 32.02189464569092, 32.040771484375], "correct_counts": [2, 0, 1, 0, 5, 4, 0, 5, 3, 10, 4, 19, 20, 21, 22, 43, 60, 148, 60, 4], "incorrect_counts": [0, 0, 0, 1, 2, 1, 0, 1, 2, 4, 8, 7, 12, 14, 24, 18, 32, 35, 8, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6349999999999999, 0.6424166666666666, 0.6498333333333333, 0.6572499999999999, 0.6646666666666666, 0.6720833333333334, 0.6795, 0.6869166666666666, 0.6943333333333334, 0.7017500000000001, 0.7091666666666667, 0.7165833333333333, 0.7240000000000001, 0.7314166666666668, 0.7388333333333335, 0.7462500000000001, 0.7536666666666668, 0.7610833333333336, 0.7685000000000002, 0.7759166666666668, 0.7833333333333335], "correct_counts": [1, 1, 1, 13, 22, 38, 53, 49, 83, 45, 40, 41, 19, 12, 4, 2, 4, 2, 0, 1], "incorrect_counts": [0, 0, 5, 7, 15, 13, 39, 25, 32, 15, 5, 10, 3, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/drop": {"edges": [14.843120073278746, 20.788069765021405, 26.73301945676406, 32.677969148506726, 38.62291884024938, 44.56786853199204, 50.512818223734705, 56.45776791547736, 62.40271760722002, 68.34766729896268, 74.29261699070533, 80.23756668244799, 86.18251637419066, 92.1274660659333, 98.07241575767597, 104.01736544941862, 109.96231514116128, 115.90726483290395, 121.8522145246466, 127.79716421638926, 133.74211390813193], "correct_counts": [50, 128, 104, 57, 41, 25, 11, 2, 3, 3, 3, 2, 1, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [48, 69, 30, 12, 7, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1786483426888783, 1.1918049529194832, 1.2049615631500878, 1.2181181733806927, 1.2312747836112976, 1.2444313938419023, 1.2575880040725071, 1.270744614303112, 1.283901224533717, 1.2970578347643216, 1.3102144449949265, 1.3233710552255313, 1.3365276654561362, 1.3496842756867409, 1.3628408859173458, 1.3759974961479506, 1.3891541063785553, 1.4023107166091602, 1.415467326839765, 1.4286239370703697, 1.4417805473009746], "correct_counts": [3, 1, 2, 11, 24, 23, 30, 42, 48, 52, 46, 40, 39, 31, 11, 11, 11, 4, 1, 1], "incorrect_counts": [2, 2, 2, 6, 8, 9, 10, 14, 22, 13, 22, 12, 20, 12, 7, 4, 3, 0, 0, 1]}}}, "ece_geometry": 0.06147257738915552}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "f63036690d31", "timestamp": "2026-07-27T09:35:28.796489+00:00", "git_sha": "3d7ce49", "config_hash": "cecbb6158c46", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 1500}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 1500}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 1500, "n_calib": 1500, "n_test": 1500, "k_restarts": 12, "accuracy_id": 0.81, "base_error": 0.19, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0711432604851103, "rho_basin": 0.1192048662021191, "energy_min": 0.1447332231633481, "energy_mean": 0.14862598663924864, "energy_std": 0.18501378708598182, "msp": 0.06933811307974048, "temp_msp": 0.08261243554210007, "entropy": 0.07082020691013495}, "temperature": 0.4068744349087813, "best_energy_baseline": "energy_min", "best_baseline": "msp", "delta_aurc_vs_energy_min": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "delta_aurc_vs_best_energy": [0.07358996267823781, 0.05406001270195325, 0.09343612485819916], "delta_aurc_vs_best_baseline": [-0.001805147405369814, -0.015438010117724282, 0.011245405317148913], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.05, 0.03333333333333333, 0.025, 0.02666666666666667, 0.033333333333333326, 0.052380952380952396, 0.05, 0.04814814814814815, 0.046666666666666676, 0.04242424242424243, 0.041666666666666664, 0.04358974358974359, 0.04047619047619048, 0.04444444444444444, 0.04375, 0.043137254901960784, 0.04444444444444444, 0.043859649122807015, 0.045, 0.04285714285714285, 0.04242424242424243, 0.04057971014492753, 0.04166666666666666, 0.044, 0.04358974358974359, 0.04938271604938271, 0.05238095238095238, 0.0586206896551724, 0.06444444444444444, 0.06451612903225806, 0.06458333333333334, 0.06666666666666667, 0.07058823529411765, 0.07714285714285712, 0.08148148148148163, 0.08558558558558559, 0.08947368421052632, 0.10170940170940171, 0.11, 0.11788617886178875, 0.12380952380952392, 0.12945736434108526, 0.13636363636363635, 0.14444444444444443, 0.1492753623188406, 0.15673758865248238, 0.16805555555555565, 0.1782312925170068, 0.19]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.10957642725598533, 0.10957642725598518, 0.10957642725598526, 0.1095764272559854, 0.10957642725598545, 0.10957642725598521, 0.10957642725598504, 0.10957642725598492, 0.10957642725598482, 0.10957642725598482, 0.10957642725598507, 0.10957642725598529, 0.10957642725598546, 0.10957642725598561, 0.10957642725598575, 0.10957642725598586, 0.10957642725598597, 0.10957642725598606, 0.10957642725598614, 0.1095764272559862, 0.10957642725598626, 0.10957642725598632, 0.10957642725598638, 0.10957642725598643, 0.10957642725598647, 0.10957642725598653, 0.10957642725598656, 0.1095764272559866, 0.10957642725598664, 0.10957642725598667, 0.1095764272559867, 0.10957642725598672, 0.10957642725598675, 0.10957642725598678, 0.10957642725598679, 0.10957642725598682, 0.11300813008130228, 0.11709456568250055, 0.12097144048363749, 0.12465447154471758, 0.12967479674796975, 0.1350907029478481, 0.14025470653377864, 0.14618844696969926, 0.15196759259259482, 0.15961352657005046, 0.1675650118203331, 0.17561111111111322, 0.18333333333333537, 0.19000000000000197]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03333333333333333, 0.16666666666666666, 0.16666666666666666, 0.15833333333333333, 0.16, 0.15555555555555553, 0.14285714285714268, 0.1375, 0.14074074074074075, 0.1399999999999999, 0.13636363636363635, 0.1361111111111111, 0.13333333333333333, 0.13095238095238096, 0.12888888888888886, 0.13541666666666666, 0.13725490196078433, 0.13518518518518516, 0.13859649122807016, 0.14333333333333334, 0.14126984126984124, 0.1409090909090909, 0.14202898550724638, 0.1444444444444444, 0.14533333333333334, 0.14743589743589744, 0.14938271604938272, 0.15, 0.14712643678160917, 0.15, 0.14623655913978495, 0.14375, 0.1404040404040404, 0.13725490196078433, 0.13619047619047617, 0.13425925925925924, 0.13423423423423422, 0.13508771929824562, 0.13931623931623932, 0.1425, 0.14552845528455283, 0.14841269841269839, 0.15426356589147286, 0.15757575757575756, 0.16296296296296298, 0.16666666666666666, 0.17163120567375897, 0.17638888888888887, 0.18503401360544217, 0.19]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.1, 0.15, 0.17777777777777778, 0.14166666666666666, 0.15333333333333332, 0.14999999999999997, 0.14761904761904765, 0.14166666666666666, 0.14814814814814814, 0.1466666666666667, 0.14545454545454545, 0.1388888888888889, 0.13333333333333333, 0.1357142857142857, 0.13999999999999999, 0.14375, 0.1411764705882353, 0.1444444444444444, 0.14385964912280702, 0.14333333333333334, 0.13968253968253966, 0.15, 0.15072463768115943, 0.15138888888888885, 0.14933333333333335, 0.15512820512820513, 0.15432098765432098, 0.15119047619047618, 0.14942528735632182, 0.14777777777777779, 0.14731182795698924, 0.14270833333333333, 0.1404040404040404, 0.13823529411764707, 0.1352380952380952, 0.13425925925925924, 0.13603603603603603, 0.13771929824561405, 0.14017094017094017, 0.14083333333333334, 0.1422764227642276, 0.146031746031746, 0.15193798449612403, 0.1590909090909091, 0.16296296296296298, 0.17028985507246377, 0.1730496453900709, 0.17638888888888898, 0.18435374149659864, 0.19]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.23333333333333334, 0.18333333333333332, 0.16666666666666666, 0.19166666666666668, 0.16666666666666666, 0.1611111111111111, 0.16666666666666669, 0.15833333333333333, 0.15925925925925927, 0.16666666666666669, 0.1696969696969697, 0.18611111111111112, 0.18461538461538463, 0.19523809523809524, 0.1911111111111111, 0.19166666666666668, 0.18627450980392157, 0.18888888888888886, 0.18947368421052632, 0.18833333333333332, 0.18888888888888886, 0.19242424242424241, 0.19130434782608696, 0.18888888888888886, 0.192, 0.18974358974358974, 0.18518518518518517, 0.18214285714285713, 0.18160919540229883, 0.17888888888888888, 0.18494623655913978, 0.18645833333333334, 0.18686868686868688, 0.18627450980392157, 0.1876190476190476, 0.18703703703703717, 0.1873873873873874, 0.18859649122807018, 0.18803418803418803, 0.1925, 0.19430894308943086, 0.19444444444444442, 0.1937984496124031, 0.19242424242424241, 0.18962962962962962, 0.18840579710144928, 0.18936170212765954, 0.19097222222222218, 0.19115646258503402, 0.19]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.011111111111111112, 0.03333333333333333, 0.03333333333333333, 0.033333333333333326, 0.03333333333333334, 0.029166666666666667, 0.02962962962962963, 0.026666666666666672, 0.024242424242424242, 0.03611111111111111, 0.03333333333333333, 0.03333333333333333, 0.033333333333333326, 0.0375, 0.0392156862745098, 0.04074074074074074, 0.042105263157894736, 0.04833333333333333, 0.04761904761904761, 0.051515151515151514, 0.04927536231884058, 0.049999999999999996, 0.052, 0.052564102564102565, 0.05308641975308642, 0.055952380952380955, 0.06436781609195401, 0.06777777777777778, 0.07311827956989247, 0.07395833333333333, 0.08282828282828283, 0.08529411764705883, 0.08952380952380966, 0.09537037037037036, 0.1009009009009009, 0.10526315789473684, 0.1094017094017094, 0.11583333333333333, 0.12357723577235771, 0.12619047619047616, 0.13023255813953488, 0.13787878787878788, 0.1437037037037037, 0.1471014492753623, 0.15602836879432622, 0.1659722222222222, 0.17755102040816326, 0.19]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.016666666666666666, 0.03333333333333333, 0.041666666666666664, 0.03333333333333333, 0.03888888888888888, 0.038095238095238106, 0.0375, 0.04814814814814815, 0.046666666666666676, 0.06363636363636363, 0.06111111111111111, 0.05897435897435897, 0.06190476190476191, 0.06222222222222221, 0.06458333333333334, 0.06666666666666667, 0.06296296296296294, 0.06666666666666667, 0.06666666666666667, 0.06349206349206347, 0.06363636363636363, 0.06521739130434782, 0.06805555555555554, 0.068, 0.07179487179487179, 0.07407407407407407, 0.07976190476190476, 0.08390804597701168, 0.08555555555555555, 0.08602150537634409, 0.08645833333333333, 0.0898989898989899, 0.0892156862745098, 0.09333333333333348, 0.10092592592592592, 0.1045045045045045, 0.10789473684210527, 0.11196581196581197, 0.11583333333333333, 0.1219512195121951, 0.1269841269841271, 0.1325581395348837, 0.14166666666666666, 0.1474074074074074, 0.15434782608695652, 0.16099290780141856, 0.16736111111111118, 0.1782312925170068, 0.19]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.011111111111111112, 0.008333333333333333, 0.02666666666666667, 0.02222222222222222, 0.02380952380952381, 0.025, 0.025925925925925925, 0.023333333333333338, 0.03636363636363636, 0.03333333333333333, 0.03333333333333333, 0.0380952380952381, 0.039999999999999994, 0.041666666666666664, 0.041176470588235294, 0.04444444444444444, 0.043859649122807015, 0.05, 0.057142857142857134, 0.06060606060606061, 0.06231884057971015, 0.06666666666666665, 0.068, 0.06923076923076923, 0.07037037037037037, 0.07261904761904762, 0.07241379310344827, 0.07111111111111111, 0.07526881720430108, 0.07604166666666666, 0.07575757575757576, 0.0784313725490196, 0.0809523809523811, 0.08425925925925924, 0.08828828828828829, 0.09385964912280702, 0.10085470085470086, 0.10583333333333333, 0.11707317073170745, 0.123015873015873, 0.13178294573643412, 0.14166666666666666, 0.1474074074074074, 0.15869565217391304, 0.1638297872340425, 0.17013888888888887, 0.17959183673469387, 0.19]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.19374220945381396, "coverage": 0.6933333333333334, "abstain_rate": 0.30666666666666664, "selective_risk": 0.075, "selective_accuracy": 0.925, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.075, 0.1295577967416602, 0.17525773195876287, 0.1879598662207358], "coverage": [0.0, 0.0, 0.6933333333333334, 0.8593333333333333, 0.97, 0.9966666666666667]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7041700960219479, "basin/entropy": 0.2946790845426323, "basin/dispersion": 0.5298130098909826, "energy/mean": 0.3736336726590138, "energy/min": 0.36821890116237094, "energy/std": 0.49345462421485814, "curv/lmax_mean": 0.4698808750270739, "curv/lmax_best": 0.4712165186629124, "curv/trace_mean": 0.4452761533463288, "curv/trace_best": 0.4445397444227854, "dynamics/steps": 0.5, "dynamics/monotonic": 0.341111833080644, "dynamics/drop": 0.3597834091401343, "dynamics/residual": 0.5355483358602267}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [2, 0, 0, 0, 0, 14, 0, 22, 0, 28, 0, 0, 38, 0, 54, 0, 0, 90, 0, 967], "incorrect_counts": [1, 0, 2, 0, 0, 14, 0, 28, 0, 32, 0, 0, 26, 0, 30, 0, 0, 33, 0, 119]}, "basin/entropy": {"edges": [0.0, 0.06789889274936621, 0.13579778549873242, 0.20369667824809862, 0.27159557099746484, 0.33949446374683107, 0.40739335649619723, 0.47529224924556346, 0.5431911419949297, 0.6110900347442959, 0.6789889274936621, 0.7468878202430284, 0.8147867129923945, 0.8826856057417607, 0.9505844984911269, 1.0184833912404931, 1.0863822839898594, 1.1542811767392256, 1.2221800694885918, 1.290078962237958, 1.3579778549873243], "correct_counts": [967, 0, 0, 0, 90, 0, 53, 0, 37, 25, 35, 0, 3, 1, 1, 1, 1, 0, 0, 1], "incorrect_counts": [119, 0, 0, 0, 33, 0, 26, 0, 27, 29, 35, 0, 3, 2, 6, 3, 2, 0, 0, 0]}, "basin/dispersion": {"edges": [1.592531476672375, 1.614116358729778, 1.6357012407871807, 1.6572861228445837, 1.6788710049019866, 1.7004558869593893, 1.7220407690167923, 1.7436256510741952, 1.7652105331315981, 1.7867954151890009, 1.8083802972464038, 1.8299651793038068, 1.8515500613612095, 1.8731349434186124, 1.8947198254760154, 1.9163047075334183, 1.937889589590821, 1.959474471648224, 1.981059353705627, 2.0026442357630296, 2.0242291178204326], "correct_counts": [3, 2, 8, 9, 23, 47, 59, 82, 96, 177, 167, 139, 126, 93, 70, 52, 31, 17, 10, 4], "incorrect_counts": [0, 0, 2, 1, 5, 11, 19, 21, 37, 36, 38, 29, 30, 20, 17, 12, 5, 1, 1, 0]}, "energy/mean": {"edges": [-0.30860641847054165, -0.24559052648643653, -0.18257463450233144, -0.11955874251822635, -0.056542850534121225, 0.006473041449983896, 0.06948893343408896, 0.13250482541819408, 0.1955207174022992, 0.25853660938640427, 0.32155250137050945, 0.3845683933546145, 0.4475842853387196, 0.5106001773228248, 0.5736160693069299, 0.6366319612910349, 0.69964785327514, 0.7626637452592451, 0.8256796372433501, 0.8886955292274554, 0.9517114212115606], "correct_counts": [10, 53, 216, 277, 172, 82, 46, 41, 37, 37, 39, 36, 43, 41, 43, 21, 18, 2, 0, 1], "incorrect_counts": [2, 11, 35, 49, 32, 8, 2, 2, 4, 7, 14, 6, 21, 33, 19, 24, 12, 3, 1, 0]}, "energy/min": {"edges": [-0.6984671354293823, -0.628644996881485, -0.5588228583335877, -0.48900071978569026, -0.41917858123779295, -0.34935644268989563, -0.27953430414199826, -0.20971216559410089, -0.13989002704620357, -0.07006788849830625, -0.00024574995040893555, 0.06957638859748849, 0.1393985271453858, 0.20922066569328313, 0.27904280424118055, 0.34886494278907776, 0.4186870813369752, 0.4885092198848726, 0.5583313584327698, 0.6281534969806672, 0.6979756355285645], "correct_counts": [5, 25, 85, 195, 264, 157, 92, 53, 48, 43, 42, 45, 44, 51, 33, 14, 13, 6, 0, 0], "incorrect_counts": [0, 3, 21, 25, 47, 32, 10, 2, 5, 6, 12, 17, 21, 28, 22, 19, 11, 3, 0, 1]}, "energy/std": {"edges": [0.058812152065186904, 0.07638791223625852, 0.09396367240733014, 0.11153943257840175, 0.12911519274947336, 0.14669095292054496, 0.1642667130916166, 0.1818424732626882, 0.1994182334337598, 0.21699399360483146, 0.23456975377590306, 0.25214551394697465, 0.2697212741180463, 0.2872970342891179, 0.3048727944601895, 0.32244855463126115, 0.34002431480233275, 0.35760007497340435, 0.375175835144476, 0.3927515953155476, 0.4103273554866192], "correct_counts": [3, 6, 20, 52, 77, 135, 180, 211, 168, 133, 90, 68, 39, 16, 9, 5, 1, 0, 1, 1], "incorrect_counts": [0, 3, 4, 12, 10, 38, 42, 44, 44, 39, 19, 10, 16, 1, 1, 1, 0, 1, 0, 0]}, "curv/lmax_mean": {"edges": [1.0000369946161907, 1.0007940803964934, 1.001551166176796, 1.0023082519570987, 1.0030653377374015, 1.003822423517704, 1.0045795092980068, 1.0053365950783095, 1.006093680858612, 1.0068507666389148, 1.0076078524192176, 1.0083649381995201, 1.0091220239798229, 1.0098791097601254, 1.0106361955404282, 1.011393281320731, 1.0121503671010335, 1.0129074528813362, 1.013664538661639, 1.0144216244419415, 1.0151787102222443], "correct_counts": [617, 173, 79, 69, 52, 45, 45, 33, 29, 19, 20, 8, 6, 5, 6, 2, 6, 0, 0, 1], "incorrect_counts": [119, 74, 29, 22, 10, 10, 4, 5, 3, 4, 3, 1, 0, 0, 0, 1, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0017322301864624, 1.0034644603729248, 1.0051966905593872, 1.0069289207458496, 1.008661150932312, 1.0103933811187744, 1.0121256113052368, 1.0138578414916992, 1.0155900716781616, 1.017322301864624, 1.0190545320510864, 1.0207867622375488, 1.0225189924240112, 1.0242512226104736, 1.025983452796936, 1.0277156829833984, 1.0294479131698608, 1.0311801433563232, 1.0329123735427856, 1.034644603729248], "correct_counts": [934, 93, 63, 35, 27, 11, 9, 7, 9, 7, 3, 3, 2, 5, 3, 2, 1, 0, 0, 1], "incorrect_counts": [213, 31, 16, 6, 6, 3, 3, 3, 2, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00269635518392, 32.00753688812256, 32.0123774210612, 32.01721795399984, 32.02205848693848, 32.02689901987712, 32.03173955281576, 32.036580085754395, 32.04142061869303, 32.046261151631676, 32.05110168457031, 32.05594221750895, 32.06078275044759, 32.06562328338623, 32.07046381632487, 32.075304349263504, 32.08014488220215, 32.084985415140785, 32.08982594807942, 32.094666481018066, 32.0995070139567], "correct_counts": [175, 275, 151, 82, 84, 56, 49, 58, 42, 42, 57, 38, 42, 21, 20, 15, 4, 2, 1, 1], "incorrect_counts": [9, 50, 53, 31, 37, 20, 19, 17, 9, 8, 7, 13, 4, 5, 2, 0, 1, 0, 0, 0]}, "curv/trace_best": {"edges": [32.0009880065918, 32.010338592529294, 32.0196891784668, 32.029039764404295, 32.0383903503418, 32.0477409362793, 32.057091522216794, 32.0664421081543, 32.075792694091795, 32.0851432800293, 32.0944938659668, 32.103844451904294, 32.1131950378418, 32.122545623779295, 32.1318962097168, 32.1412467956543, 32.150597381591794, 32.1599479675293, 32.169298553466795, 32.1786491394043, 32.1879997253418], "correct_counts": [402, 281, 150, 106, 83, 59, 46, 28, 17, 9, 10, 6, 3, 5, 2, 2, 2, 3, 0, 1], "incorrect_counts": [66, 67, 55, 31, 17, 13, 8, 8, 7, 6, 3, 2, 0, 1, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [1215, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [285, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7133333333333333, 0.7193333333333333, 0.7253333333333333, 0.7313333333333333, 0.7373333333333333, 0.7433333333333333, 0.7493333333333333, 0.7553333333333333, 0.7613333333333333, 0.7673333333333333, 0.7733333333333334, 0.7793333333333334, 0.7853333333333334, 0.7913333333333334, 0.7973333333333334, 0.8033333333333335, 0.8093333333333335, 0.8153333333333335, 0.8213333333333335, 0.8273333333333335, 0.8333333333333335], "correct_counts": [1, 0, 2, 23, 32, 66, 131, 145, 240, 187, 167, 125, 55, 25, 12, 3, 1, 0, 0, 0], "incorrect_counts": [1, 0, 0, 3, 8, 14, 10, 17, 33, 38, 46, 39, 19, 20, 14, 11, 6, 3, 2, 1]}, "dynamics/drop": {"edges": [63.47809874266386, 76.39078911232451, 89.30347948198516, 102.21616985164583, 115.12886022130648, 128.04155059096712, 140.9542409606278, 153.86693133028845, 166.7796216999491, 179.69231206960976, 192.6050024392704, 205.51769280893106, 218.43038317859174, 231.3430735482524, 244.25576391791304, 257.1684542875737, 270.08114465723435, 282.993835026895, 295.90652539655565, 308.8192157662163, 321.73190613587695], "correct_counts": [10, 31, 108, 203, 266, 281, 168, 89, 28, 12, 10, 3, 1, 2, 2, 0, 0, 1, 0, 0], "incorrect_counts": [1, 7, 23, 29, 45, 36, 31, 27, 17, 10, 15, 12, 4, 11, 5, 4, 0, 4, 1, 3]}, "dynamics/residual": {"edges": [1.1409447093804677, 1.1559197435776392, 1.1708947777748107, 1.1858698119719822, 1.2008448461691539, 1.2158198803663254, 1.2307949145634969, 1.2457699487606684, 1.2607449829578399, 1.2757200171550114, 1.2906950513521829, 1.3056700855493546, 1.320645119746526, 1.3356201539436976, 1.350595188140869, 1.3655702223380406, 1.380545256535212, 1.3955202907323838, 1.4104953249295553, 1.4254703591267268, 1.4404453933238983], "correct_counts": [2, 4, 6, 19, 33, 43, 85, 102, 112, 156, 148, 145, 122, 97, 61, 43, 24, 7, 4, 2], "incorrect_counts": [0, 2, 2, 1, 11, 8, 21, 34, 36, 27, 34, 40, 24, 26, 10, 6, 3, 0, 0, 0]}}}, "ece_geometry": 0.03153101157848922, "accuracy_ood": 0.20533333333333334, "aurc_ood": {"geometry": 0.8035703588171388, "rho_basin": 0.7533412924431383, "energy_min": 0.7509755512657853, "energy_mean": 0.740760170602411, "energy_std": 0.7992759846101417, "msp": 0.7106997166086145, "temp_msp": 0.709349201412843, "entropy": 0.7172185327581629}, "ood_ltt": {"alpha": 0.1, "lambda_hat": 0.19374220945381396, "selective_risk": 0.7619532044760936, "coverage": 0.6553333333333333, "risk_within_budget": false}, "ood_validity": {"target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "id_risk": [0.0, 0.0, 0.075, 0.1295577967416602, 0.17525773195876287, 0.1879598662207358], "ood_risk": [0.0, 0.0, 0.7619532044760936, 0.7915726109857035, 0.7924914675767918, 0.7949231796927188], "id_coverage": [0.0, 0.0, 0.6933333333333334, 0.8593333333333333, 0.97, 0.9966666666666667], "ood_coverage": [0.0, 0.0, 0.6553333333333333, 0.886, 0.9766666666666667, 0.998]}}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "d81d5fbf0f98", "timestamp": "2026-07-27T09:35:47.556874+00:00", "git_sha": "3d7ce49", "config_hash": "3643423663c7", "config": {"run": {"seed": 0, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.227946937084198, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1527490825609553, "rho_basin": 0.22247851187975562, "energy_min": 0.37245598613988906, "energy_mean": 0.3894178842534342, "energy_std": 0.28068693518661764, "msp": 0.12101715414067431, "temp_msp": 0.1204758682795991, "entropy": 0.11966617541102537}, "temperature": 2.9326538741941572, "best_energy_baseline": "energy_std", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.21970690357893377, 0.16353680784617466, 0.27315268065902204], "delta_aurc_vs_best_energy": [0.12793785262566235, 0.08274255159121499, 0.17357149948028552], "delta_aurc_vs_best_baseline": [-0.033082907149929924, -0.05463796025527004, -0.013340602564395215], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.0625, 0.06666666666666667, 0.08333333333333331, 0.10714285714285696, 0.09375, 0.08333333333333333, 0.09166666666666667, 0.09090909090909091, 0.09722222222222222, 0.11538461538461539, 0.125, 0.1333333333333333, 0.13020833333333334, 0.12745098039215685, 0.12499999999999999, 0.12280701754385964, 0.12916666666666668, 0.13095238095238093, 0.13257575757575757, 0.13043478260869565, 0.13888888888888887, 0.13333333333333333, 0.14102564102564102, 0.1419753086419753, 0.1488095238095238, 0.14942528735632182, 0.15, 0.16129032258064516, 0.1640625, 0.1691919191919192, 0.17892156862745098, 0.1833333333333333, 0.18287037037037035, 0.18468468468468469, 0.18421052631578946, 0.19017094017094016, 0.20416666666666666, 0.20934959349593493, 0.21428571428571425, 0.2248062015503876, 0.23484848484848486, 0.2462962962962963, 0.25181159420289856, 0.25886524822695045, 0.26736111111111105, 0.272108843537415, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2165991902834008, 0.2165991902834009, 0.21659919028340094, 0.21659919028340077, 0.21659919028340063, 0.21659919028340055, 0.21659919028340052, 0.21659919028340047, 0.21659919028340044, 0.2165991902834004, 0.21659919028340038, 0.21659919028340036, 0.21659919028340055, 0.21659919028340077, 0.21659919028340097, 0.21659919028340113, 0.21659919028340127, 0.2165991902834014, 0.21659919028340155, 0.21659919028340163, 0.21659919028340174, 0.21659919028340183, 0.2165991902834019, 0.216599190283402, 0.21659919028340202, 0.2165991902834021, 0.21659919028340216, 0.21659919028340222, 0.21659919028340227, 0.2165991902834023, 0.21659919028340235, 0.21659919028340238, 0.21659919028340244, 0.21659919028340247, 0.2165991902834025, 0.21659919028340255, 0.21659919028340258, 0.2165991902834026, 0.21659919028340263, 0.21659919028340266, 0.2165991902834027, 0.22372534872535066, 0.23191214470284427, 0.23948863636363826, 0.24416666666666834, 0.250762776506485, 0.259985069055619, 0.2673611111111122, 0.27494331065759725, 0.2816666666666674]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4583333333333333, 0.4666666666666667, 0.45833333333333337, 0.4523809523809525, 0.4479166666666667, 0.46296296296296297, 0.42500000000000004, 0.42424242424242425, 0.4027777777777778, 0.3974358974358974, 0.4107142857142857, 0.40000000000000013, 0.4010416666666667, 0.4019607843137255, 0.40740740740740733, 0.39473684210526316, 0.4, 0.39682539682539675, 0.38636363636363635, 0.391304347826087, 0.39236111111111105, 0.38333333333333336, 0.3717948717948718, 0.36419753086419754, 0.36607142857142855, 0.3706896551724139, 0.3638888888888889, 0.3521505376344086, 0.3515625, 0.3434343434343434, 0.3382352941176471, 0.33571428571428563, 0.32870370370370366, 0.32657657657657657, 0.3223684210526316, 0.3247863247863248, 0.3229166666666667, 0.3191056910569105, 0.3115079365079366, 0.3081395348837209, 0.30303030303030304, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.28819444444444436, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.5, 0.5, 0.4375, 0.5, 0.49999999999999994, 0.4999999999999999, 0.46875, 0.4722222222222222, 0.45833333333333326, 0.44696969696969696, 0.4236111111111111, 0.4166666666666667, 0.42857142857142855, 0.4277777777777777, 0.421875, 0.4215686274509804, 0.412037037037037, 0.41228070175438597, 0.3958333333333333, 0.39682539682539686, 0.4015151515151515, 0.39492753623188404, 0.39583333333333326, 0.3933333333333333, 0.38782051282051283, 0.37962962962962965, 0.37202380952380953, 0.36781609195402293, 0.3611111111111111, 0.3548387096774194, 0.3463541666666667, 0.3383838383838384, 0.33578431372549017, 0.3309523809523809, 0.33101851851851866, 0.32882882882882886, 0.3267543859649123, 0.32264957264957267, 0.32083333333333336, 0.31707317073170727, 0.31349206349206343, 0.3081395348837209, 0.30303030303030304, 0.30185185185185187, 0.2971014492753623, 0.29078014184397155, 0.28645833333333326, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.33333333333333326, 0.3214285714285715, 0.3020833333333333, 0.3055555555555556, 0.2833333333333334, 0.2727272727272727, 0.2847222222222222, 0.28846153846153844, 0.27976190476190477, 0.28333333333333327, 0.2760416666666667, 0.27450980392156865, 0.26388888888888884, 0.2675438596491228, 0.2791666666666667, 0.26984126984126994, 0.2803030303030303, 0.2753623188405797, 0.26736111111111105, 0.26, 0.266025641025641, 0.2623456790123457, 0.2648809523809524, 0.26724137931034475, 0.26944444444444443, 0.27419354838709675, 0.2734375, 0.2727272727272727, 0.27941176470588236, 0.2738095238095238, 0.2708333333333334, 0.2725225225225225, 0.2741228070175439, 0.27136752136752135, 0.27708333333333335, 0.2804878048780489, 0.28174603174603186, 0.2810077519379845, 0.2821969696969697, 0.2814814814814815, 0.28442028985507245, 0.2836879432624113, 0.27951388888888895, 0.2789115646258503, 0.2816666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.05555555555555555, 0.05128205128205128, 0.047619047619047616, 0.055555555555555546, 0.052083333333333336, 0.06372549019607843, 0.060185185185185175, 0.06140350877192982, 0.0625, 0.07142857142857155, 0.0946969696969697, 0.09420289855072464, 0.10416666666666666, 0.11, 0.125, 0.12962962962962962, 0.13392857142857142, 0.1379310344827586, 0.14166666666666666, 0.14516129032258066, 0.1484375, 0.1590909090909091, 0.15931372549019607, 0.16428571428571442, 0.1759259259259259, 0.18243243243243243, 0.19298245614035087, 0.20085470085470086, 0.20625, 0.20528455284552857, 0.21428571428571438, 0.22286821705426357, 0.22916666666666666, 0.24074074074074073, 0.2554347826086957, 0.26063829787234055, 0.26736111111111105, 0.27040816326530615, 0.2816666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.05555555555555555, 0.05000000000000001, 0.05303030303030303, 0.04861111111111111, 0.05128205128205128, 0.05357142857142857, 0.050000000000000176, 0.057291666666666664, 0.058823529411764705, 0.055555555555555546, 0.06578947368421052, 0.07083333333333333, 0.08333333333333331, 0.08712121212121213, 0.09420289855072464, 0.10763888888888888, 0.11333333333333333, 0.125, 0.12654320987654322, 0.12797619047619047, 0.1350574712643678, 0.1388888888888889, 0.13709677419354838, 0.14322916666666666, 0.16161616161616163, 0.16176470588235295, 0.1714285714285714, 0.17824074074074087, 0.18468468468468469, 0.18859649122807018, 0.19658119658119658, 0.20416666666666666, 0.20934959349593507, 0.2103174603174603, 0.22093023255813954, 0.22537878787878787, 0.23333333333333334, 0.24094202898550723, 0.24822695035461, 0.2621527777777779, 0.27380952380952384, 0.2816666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.05555555555555555, 0.05128205128205128, 0.047619047619047616, 0.055555555555555546, 0.052083333333333336, 0.06372549019607843, 0.060185185185185175, 0.06140350877192982, 0.0625, 0.07142857142857155, 0.09090909090909091, 0.09420289855072464, 0.10416666666666666, 0.11333333333333333, 0.125, 0.12962962962962962, 0.13392857142857142, 0.13505747126436798, 0.1388888888888889, 0.1424731182795699, 0.1484375, 0.15404040404040403, 0.16176470588235295, 0.1666666666666668, 0.17592592592592607, 0.18693693693693694, 0.18640350877192982, 0.19230769230769232, 0.19791666666666666, 0.20731707317073167, 0.21031746031746043, 0.22093023255813954, 0.22727272727272727, 0.2351851851851852, 0.2391304347826087, 0.2500000000000001, 0.2604166666666668, 0.27380952380952384, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.14285714285714285, 0.2527075812274368], "coverage": [0.0, 0.0, 0.0, 0.0, 0.525, 0.9233333333333333]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6337264377599913, "basin/entropy": 0.36597839069729127, "basin/dispersion": 0.5371984788368869, "energy/mean": 0.6647400431087741, "energy/min": 0.6541275964799077, "energy/std": 0.4885020387429811, "curv/lmax_mean": 0.3431952662721893, "curv/lmax_best": 0.3816018890978734, "curv/trace_mean": 0.3301253449388377, "curv/trace_best": 0.3412526256538393, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6490067134364832, "dynamics/drop": 0.6809401556858277, "dynamics/residual": 0.5138044179629044}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [4, 0, 0, 3, 0, 0, 6, 0, 0, 0, 6, 0, 0, 11, 0, 0, 14, 0, 0, 387], "incorrect_counts": [6, 0, 0, 6, 0, 0, 9, 0, 0, 0, 13, 0, 0, 9, 0, 0, 19, 0, 0, 107]}, "basin/entropy": {"edges": [0.0, 0.04592138924965487, 0.09184277849930975, 0.1377641677489646, 0.1836855569986195, 0.22960694624827438, 0.2755283354979292, 0.3214497247475841, 0.367371113997239, 0.41329250324689387, 0.45921389249654876, 0.5051352817462036, 0.5510566709958584, 0.5969780602455134, 0.6428994494951682, 0.6888208387448231, 0.734742227994478, 0.7806636172441328, 0.8265850064937877, 0.8725063957434426, 0.9184277849930975], "correct_counts": [387, 0, 0, 0, 0, 0, 14, 0, 0, 10, 0, 0, 7, 6, 3, 4, 0, 0, 0, 0], "incorrect_counts": [107, 0, 0, 0, 0, 0, 19, 0, 0, 9, 0, 0, 12, 7, 6, 6, 0, 2, 0, 1]}, "basin/dispersion": {"edges": [1.620605021245252, 1.641571274378292, 1.662537527511332, 1.683503780644372, 1.7044700337774121, 1.725436286910452, 1.746402540043492, 1.767368793176532, 1.788335046309572, 1.809301299442612, 1.830267552575652, 1.851233805708692, 1.872200058841732, 1.893166311974772, 1.914132565107812, 1.935098818240852, 1.9560650713738919, 1.977031324506932, 1.9979975776399719, 2.018963830773012, 2.039930083906052], "correct_counts": [0, 2, 2, 8, 13, 22, 28, 29, 46, 69, 50, 43, 35, 31, 20, 17, 7, 7, 1, 1], "incorrect_counts": [1, 3, 0, 5, 2, 6, 17, 14, 22, 23, 22, 15, 14, 8, 13, 2, 1, 1, 0, 0]}, "energy/mean": {"edges": [0.6107773731152216, 0.6431646155814329, 0.6755518580476443, 0.7079391005138556, 0.7403263429800669, 0.7727135854462782, 0.8051008279124896, 0.8374880703787009, 0.8698753128449122, 0.9022625553111235, 0.9346497977773349, 0.9670370402435462, 0.9994242827097575, 1.0318115251759687, 1.0641987676421802, 1.0965860101083915, 1.1289732525746028, 1.161360495040814, 1.1937477375070253, 1.2261349799732368, 1.2585222224394481], "correct_counts": [1, 3, 8, 10, 11, 27, 35, 32, 45, 36, 55, 40, 46, 27, 21, 12, 15, 3, 3, 1], "incorrect_counts": [4, 1, 7, 6, 18, 17, 16, 20, 24, 11, 12, 11, 11, 5, 1, 1, 2, 1, 0, 1]}, "energy/min": {"edges": [0.20125961303710938, 0.2415154755115509, 0.2817713379859924, 0.322027200460434, 0.3622830629348755, 0.402538925409317, 0.4427947878837586, 0.4830506503582001, 0.5233065128326416, 0.5635623753070831, 0.6038182377815247, 0.6440741002559662, 0.6843299627304078, 0.7245858252048493, 0.7648416876792908, 0.8050975501537323, 0.8453534126281739, 0.8856092751026154, 0.9258651375770569, 0.9661210000514985, 1.00637686252594], "correct_counts": [1, 1, 6, 2, 13, 14, 29, 37, 39, 39, 58, 46, 39, 44, 24, 21, 10, 1, 3, 4], "incorrect_counts": [1, 0, 2, 4, 12, 13, 19, 20, 22, 21, 17, 10, 14, 5, 4, 2, 1, 2, 0, 0]}, "energy/std": {"edges": [0.06508645292670491, 0.08179583921728364, 0.09850522550786239, 0.11521461179844114, 0.1319239980890199, 0.1486333843795986, 0.16534277067017736, 0.1820521569607561, 0.19876154325133483, 0.2154709295419136, 0.23218031583249232, 0.24888970212307104, 0.2655990884136498, 0.28230847470422854, 0.29901786099480726, 0.31572724728538604, 0.33243663357596476, 0.3491460198665435, 0.36585540615712225, 0.382564792447701, 0.3992741787382797], "correct_counts": [1, 3, 7, 18, 27, 54, 60, 71, 59, 43, 31, 26, 20, 6, 1, 2, 2, 0, 0, 0], "incorrect_counts": [1, 0, 3, 10, 13, 16, 22, 21, 29, 13, 18, 11, 6, 2, 1, 0, 1, 0, 1, 1]}, "curv/lmax_mean": {"edges": [0.9988154868284861, 0.9990137209494908, 0.9992119550704955, 0.9994101891915003, 0.9996084233125051, 0.9998066574335098, 1.0000048915545146, 1.0002031256755193, 1.000401359796524, 1.0005995939175287, 1.0007978280385335, 1.0009960621595382, 1.001194296280543, 1.0013925304015476, 1.0015907645225526, 1.0017889986435573, 1.001987232764562, 1.0021854668855668, 1.0023837010065715, 1.0025819351275762, 1.002780169248581], "correct_counts": [2, 2, 10, 38, 119, 132, 72, 24, 11, 6, 10, 2, 1, 0, 0, 1, 0, 0, 0, 1], "incorrect_counts": [1, 0, 3, 9, 21, 43, 36, 30, 11, 5, 3, 1, 3, 2, 1, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.9958722591400146, 0.9962057709693909, 0.9965392827987671, 0.9968727946281433, 0.9972063064575195, 0.9975398182868958, 0.997873330116272, 0.9982068419456482, 0.9985403537750244, 0.9988738656044006, 0.9992073774337769, 0.9995408892631531, 0.9998744010925293, 1.0002079129219055, 1.0005414247512818, 1.000874936580658, 1.001208448410034, 1.0015419602394104, 1.0018754720687866, 1.002208983898163, 1.002542495727539], "correct_counts": [0, 0, 0, 0, 0, 1, 3, 4, 8, 10, 25, 92, 246, 23, 5, 1, 6, 4, 2, 1], "incorrect_counts": [1, 0, 0, 0, 0, 0, 1, 0, 1, 6, 6, 26, 85, 24, 6, 6, 3, 2, 2, 0]}, "curv/trace_mean": {"edges": [31.95863151550293, 31.962152846654256, 31.96567417780558, 31.969195508956908, 31.972716840108234, 31.97623817125956, 31.97975950241089, 31.983280833562215, 31.98680216471354, 31.990323495864867, 31.993844827016193, 31.99736615816752, 32.00088748931885, 32.004408820470175, 32.0079301516215, 32.01145148277283, 32.01497281392415, 32.01849414507548, 32.022015476226805, 32.02553680737813, 32.02905813852946], "correct_counts": [1, 0, 2, 7, 14, 14, 24, 46, 62, 37, 62, 49, 47, 21, 19, 14, 3, 2, 5, 2], "incorrect_counts": [0, 0, 0, 0, 0, 3, 4, 7, 10, 20, 17, 21, 31, 18, 22, 9, 3, 4, 0, 0]}, "curv/trace_best": {"edges": [31.937339782714844, 31.943468475341795, 31.94959716796875, 31.955725860595702, 31.961854553222658, 31.96798324584961, 31.97411193847656, 31.980240631103516, 31.986369323730468, 31.992498016357423, 31.998626708984375, 32.00475540161133, 32.01088409423828, 32.01701278686524, 32.02314147949219, 32.02927017211914, 32.03539886474609, 32.041527557373044, 32.04765625, 32.053784942626955, 32.059913635253906], "correct_counts": [2, 1, 3, 7, 6, 15, 18, 43, 86, 101, 89, 32, 11, 11, 4, 1, 0, 0, 1, 0], "incorrect_counts": [0, 0, 0, 0, 1, 3, 6, 6, 17, 43, 37, 26, 15, 10, 4, 0, 0, 0, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6466666666666666, 0.6523333333333333, 0.6579999999999999, 0.6636666666666666, 0.6693333333333332, 0.6749999999999999, 0.6806666666666666, 0.6863333333333332, 0.692, 0.6976666666666665, 0.7033333333333333, 0.709, 0.7146666666666666, 0.7203333333333333, 0.7259999999999999, 0.7316666666666666, 0.7373333333333333, 0.7429999999999999, 0.7486666666666666, 0.7543333333333332, 0.7599999999999999], "correct_counts": [0, 5, 4, 8, 19, 36, 33, 63, 55, 44, 53, 28, 27, 22, 10, 12, 6, 2, 1, 3], "incorrect_counts": [2, 3, 7, 9, 14, 22, 20, 25, 18, 11, 18, 8, 5, 6, 0, 0, 1, 0, 0, 0]}, "dynamics/drop": {"edges": [13.273132185141245, 17.449723252654074, 21.626314320166905, 25.802905387679733, 29.97949645519256, 34.15608752270539, 38.33267859021822, 42.509269657731046, 46.68586072524388, 50.86245179275671, 55.03904286026954, 59.21563392778236, 63.392224995295194, 67.56881606280803, 71.74540713032086, 75.92199819783369, 80.09858926534652, 84.27518033285935, 88.45177140037218, 92.62836246788501, 96.80495353539784], "correct_counts": [12, 54, 92, 87, 56, 46, 29, 19, 11, 9, 4, 2, 5, 0, 2, 2, 0, 0, 0, 1], "incorrect_counts": [24, 44, 39, 23, 17, 11, 7, 0, 1, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471606890360515, 1.1608674337466558, 1.1745741784572603, 1.1882809231678646, 1.201987667878469, 1.2156944125890732, 1.2294011572996775, 1.243107902010282, 1.2568146467208863, 1.2705213914314906, 1.284228136142095, 1.2979348808526994, 1.3116416255633037, 1.325348370273908, 1.3390551149845122, 1.3527618596951168, 1.366468604405721, 1.3801753491163253, 1.3938820938269298, 1.4075888385375341, 1.4212955832481384], "correct_counts": [1, 1, 3, 4, 7, 18, 32, 34, 40, 41, 45, 44, 52, 38, 27, 26, 10, 4, 2, 2], "incorrect_counts": [1, 0, 0, 2, 4, 10, 10, 14, 15, 14, 24, 17, 18, 12, 10, 8, 5, 5, 0, 0]}}}, "ece_geometry": 0.04034562456459658}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "fbea61f24343", "timestamp": "2026-07-27T09:35:59.699553+00:00", "git_sha": "3d7ce49", "config_hash": "60da27bc4d66", "config": {"run": {"seed": 1, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6933333333333334, "base_error": 0.30666666666666664, "final_train_loss": 0.2080264687538147, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1675196272459423, "rho_basin": 0.25457099646739273, "energy_min": 0.19884075586381236, "energy_mean": 0.19759334670590525, "energy_std": 0.29045958157255325, "msp": 0.12979645957506092, "temp_msp": 0.12794473890187164, "entropy": 0.12914511701061224}, "temperature": 2.814063136720987, "best_energy_baseline": "energy_mean", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.031321128617870064, 0.011535510095268008, 0.05241587205968214], "delta_aurc_vs_best_energy": [0.030073719459962955, 0.008993436060296952, 0.05227588726640192], "delta_aurc_vs_best_baseline": [-0.03957488834407066, -0.057365171281437685, -0.021981574766360788], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.05555555555555555, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.0625, 0.05555555555555555, 0.08333333333333334, 0.09090909090909091, 0.10416666666666667, 0.09615384615384616, 0.10714285714285714, 0.11666666666666664, 0.11979166666666667, 0.12254901960784313, 0.13888888888888887, 0.13596491228070176, 0.14166666666666666, 0.14682539682539694, 0.15151515151515152, 0.15942028985507245, 0.16666666666666663, 0.16666666666666666, 0.1762820512820513, 0.17901234567901234, 0.1875, 0.1954022988505747, 0.2, 0.20430107526881722, 0.20572916666666666, 0.21212121212121213, 0.21568627450980393, 0.22142857142857153, 0.22916666666666663, 0.23198198198198197, 0.2412280701754386, 0.24572649572649571, 0.25833333333333336, 0.26219512195121947, 0.2658730158730158, 0.2693798449612403, 0.2784090909090909, 0.2796296296296296, 0.2807971014492754, 0.2854609929078015, 0.29166666666666674, 0.3010204081632653, 0.30666666666666664]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2489878542510122, 0.24898785425101208, 0.24898785425101208, 0.24898785425101222, 0.2489878542510123, 0.24898785425101236, 0.2489878542510124, 0.24898785425101244, 0.24898785425101247, 0.2489878542510125, 0.2489878542510125, 0.24898785425101252, 0.24898785425101252, 0.24898785425101252, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.25492831541218686, 0.26175293823455936, 0.2689393939393948, 0.2777777777777784, 0.28623188405797145, 0.29078014184397205, 0.29714209401709435, 0.3030990173847319, 0.3066666666666668]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.1111111111111111, 0.125, 0.11666666666666667, 0.11111111111111109, 0.09523809523809525, 0.11458333333333333, 0.1388888888888889, 0.1416666666666667, 0.1590909090909091, 0.1597222222222222, 0.16025641025641027, 0.16666666666666666, 0.17222222222222236, 0.18229166666666666, 0.19117647058823528, 0.19444444444444442, 0.18859649122807018, 0.19166666666666668, 0.18650793650793662, 0.19318181818181818, 0.1956521739130435, 0.20833333333333331, 0.2, 0.19230769230769232, 0.20679012345679013, 0.20833333333333334, 0.2155172413793103, 0.2222222222222222, 0.22311827956989247, 0.22395833333333334, 0.22474747474747475, 0.22794117647058823, 0.23571428571428582, 0.2384259259259259, 0.24549549549549549, 0.2543859649122807, 0.2606837606837607, 0.26666666666666666, 0.27032520325203246, 0.26785714285714296, 0.27325581395348836, 0.2840909090909091, 0.28888888888888886, 0.2898550724637681, 0.2943262411347517, 0.29513888888888895, 0.3010204081632653, 0.30666666666666664]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.05555555555555555, 0.0625, 0.08333333333333333, 0.11111111111111109, 0.10714285714285716, 0.11458333333333333, 0.12962962962962962, 0.12500000000000003, 0.12121212121212122, 0.11805555555555555, 0.14743589743589744, 0.14285714285714285, 0.1611111111111113, 0.17708333333333334, 0.18137254901960784, 0.18518518518518515, 0.17982456140350878, 0.17916666666666667, 0.17857142857142855, 0.17045454545454544, 0.18478260869565216, 0.18402777777777776, 0.19666666666666666, 0.20833333333333334, 0.2006172839506173, 0.2113095238095238, 0.2097701149425287, 0.21666666666666667, 0.22580645161290322, 0.22916666666666666, 0.2297979797979798, 0.2426470588235294, 0.24047619047619045, 0.24074074074074084, 0.24774774774774774, 0.2565789473684211, 0.26282051282051283, 0.26458333333333334, 0.26626016260162594, 0.2678571428571428, 0.2655038759689923, 0.2708333333333333, 0.2740740740740741, 0.2807971014492754, 0.28900709219858167, 0.29687499999999994, 0.30272108843537415, 0.30666666666666664]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.16666666666666666, 0.19444444444444445, 0.22916666666666666, 0.25, 0.26388888888888884, 0.261904761904762, 0.2708333333333333, 0.26851851851851855, 0.2833333333333334, 0.30303030303030304, 0.2986111111111111, 0.28846153846153844, 0.30357142857142855, 0.29999999999999993, 0.3177083333333333, 0.3235294117647059, 0.3101851851851851, 0.2982456140350877, 0.3, 0.2896825396825398, 0.29924242424242425, 0.30434782608695654, 0.30208333333333326, 0.31, 0.3108974358974359, 0.3148148148148148, 0.31845238095238093, 0.3189655172413793, 0.32222222222222224, 0.31451612903225806, 0.3229166666666667, 0.3181818181818182, 0.31862745098039214, 0.31904761904761897, 0.31481481481481477, 0.31756756756756754, 0.3157894736842105, 0.3141025641025641, 0.3125, 0.306910569105691, 0.3055555555555557, 0.3062015503875969, 0.3068181818181818, 0.3037037037037037, 0.3061594202898551, 0.30673758865248224, 0.3055555555555555, 0.304421768707483, 0.30666666666666664]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761906, 0.020833333333333332, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.02976190476190476, 0.027777777777777773, 0.052083333333333336, 0.05392156862745098, 0.0648148148148148, 0.07456140350877193, 0.07916666666666666, 0.09126984126984125, 0.09090909090909091, 0.09420289855072464, 0.11111111111111109, 0.12333333333333334, 0.13141025641025642, 0.14506172839506173, 0.15178571428571427, 0.16091954022988522, 0.1638888888888889, 0.1774193548387097, 0.18489583333333334, 0.1893939393939394, 0.20098039215686275, 0.19761904761904775, 0.20370370370370366, 0.20945945945945946, 0.21929824561403508, 0.2264957264957265, 0.23541666666666666, 0.23983739837398385, 0.2500000000000001, 0.2558139534883721, 0.26325757575757575, 0.2759259259259259, 0.2826086956521739, 0.2872340425531916, 0.29513888888888884, 0.29931972789115646, 0.30666666666666664]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.025000000000000005, 0.030303030303030304, 0.034722222222222224, 0.03205128205128205, 0.02976190476190476, 0.03888888888888888, 0.036458333333333336, 0.04411764705882353, 0.05092592592592607, 0.06140350877192982, 0.0625, 0.07539682539682552, 0.09090909090909091, 0.11231884057971014, 0.12152777777777776, 0.12333333333333334, 0.1346153846153846, 0.1419753086419753, 0.1488095238095238, 0.16091954022988522, 0.17222222222222222, 0.18010752688172044, 0.1875, 0.18686868686868688, 0.19117647058823528, 0.19999999999999998, 0.19907407407407418, 0.21396396396396397, 0.22149122807017543, 0.2264957264957265, 0.2375, 0.2439024390243902, 0.24404761904761915, 0.25387596899224807, 0.26136363636363635, 0.26666666666666666, 0.27355072463768115, 0.2836879432624115, 0.29166666666666674, 0.3010204081632653, 0.30666666666666664]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.03205128205128205, 0.02976190476190476, 0.027777777777777773, 0.052083333333333336, 0.05392156862745098, 0.06018518518518533, 0.07456140350877193, 0.07916666666666666, 0.08730158730158728, 0.09090909090909091, 0.09420289855072464, 0.1111111111111112, 0.12666666666666668, 0.1346153846153846, 0.1419753086419753, 0.15476190476190477, 0.16379310344827583, 0.1638888888888889, 0.1774193548387097, 0.1875, 0.1919191919191919, 0.19117647058823528, 0.19999999999999998, 0.20370370370370366, 0.2072072072072072, 0.21271929824561403, 0.22435897435897437, 0.23333333333333334, 0.2439024390243902, 0.2500000000000001, 0.2596899224806202, 0.26325757575757575, 0.26851851851851855, 0.27355072463768115, 0.28546099290780136, 0.29166666666666663, 0.3010204081632653, 0.30666666666666664]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.15985130111524162, 0.24301075268817204], "coverage": [0.0, 0.0, 0.0, 0.0, 0.4483333333333333, 0.775]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6111648202341137, "basin/entropy": 0.3888286475752508, "basin/dispersion": 0.505382525083612, "energy/mean": 0.32454535953177255, "energy/min": 0.3349576714046823, "energy/std": 0.5020641722408027, "curv/lmax_mean": 0.46143394648829433, "curv/lmax_best": 0.5073879076086957, "curv/trace_mean": 0.4757133152173913, "curv/trace_best": 0.4546535326086957, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6186833716555183, "dynamics/drop": 0.6482545986622074, "dynamics/residual": 0.4986151755852843}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [0, 0, 0, 0, 0, 4, 0, 5, 0, 4, 0, 0, 9, 0, 9, 0, 0, 14, 0, 371], "incorrect_counts": [1, 0, 0, 0, 0, 3, 0, 4, 0, 9, 0, 0, 9, 0, 18, 0, 0, 17, 0, 123]}, "basin/entropy": {"edges": [0.0, 0.05493061443340548, 0.10986122886681096, 0.16479184330021646, 0.21972245773362192, 0.2746530721670274, 0.3295836866004329, 0.3845143010338384, 0.43944491546724385, 0.4943755299006493, 0.5493061443340548, 0.6042367587674603, 0.6591673732008658, 0.7140979876342712, 0.7690286020676768, 0.8239592165010822, 0.8788898309344877, 0.9338204453678932, 0.9887510598012986, 1.0436816742347041, 1.0986122886681096], "correct_counts": [371, 0, 0, 0, 0, 14, 0, 0, 9, 0, 8, 3, 8, 1, 0, 1, 1, 0, 0, 0], "incorrect_counts": [123, 0, 0, 0, 0, 17, 0, 0, 18, 0, 8, 9, 4, 1, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.6302329015759547, 1.6519526525410289, 1.6736724035061028, 1.6953921544711767, 1.717111905436251, 1.738831656401325, 1.760551407366399, 1.782271158331473, 1.8039909092965472, 1.8257106602616213, 1.8474304112266953, 1.8691501621917692, 1.8908699131568434, 1.9125896641219176, 1.9343094150869915, 1.9560291660520654, 1.9777489170171396, 1.9994686679822138, 2.021188418947288, 2.0429081699123617, 2.064627920877436], "correct_counts": [0, 4, 8, 11, 19, 21, 38, 49, 52, 38, 51, 40, 27, 21, 15, 10, 5, 5, 1, 1], "incorrect_counts": [2, 1, 2, 3, 11, 7, 21, 24, 18, 17, 23, 18, 20, 8, 3, 3, 1, 1, 1, 0]}, "energy/mean": {"edges": [-0.2394584914048513, -0.1896760197977225, -0.13989354819059374, -0.09011107658346496, -0.04032860497633617, 0.009453866630792618, 0.05923633823792138, 0.10901880984505016, 0.15880128145217895, 0.20858375305930774, 0.25836622466643655, 0.3081486962735652, 0.357931167880694, 0.4077136394878228, 0.4574961110949516, 0.5072785827020804, 0.5570610543092092, 0.606843525916338, 0.6566259975234667, 0.7064084691305955, 0.7561909407377243], "correct_counts": [1, 1, 6, 20, 16, 22, 43, 36, 38, 42, 43, 28, 32, 32, 31, 9, 9, 4, 1, 2], "incorrect_counts": [0, 0, 2, 0, 0, 6, 8, 12, 12, 12, 20, 17, 19, 22, 14, 19, 14, 4, 1, 2]}, "energy/min": {"edges": [-0.5504263639450073, -0.4960724890232086, -0.4417186141014099, -0.38736473917961123, -0.3330108642578125, -0.2786569893360138, -0.22430311441421513, -0.16994923949241642, -0.1155953645706177, -0.06124148964881898, -0.006887614727020264, 0.0474662601947784, 0.10182013511657706, 0.15617401003837583, 0.2105278849601745, 0.26488175988197327, 0.31923563480377193, 0.3735895097255706, 0.42794338464736936, 0.48229725956916814, 0.5366511344909668], "correct_counts": [3, 3, 8, 18, 24, 25, 35, 44, 51, 45, 37, 35, 26, 22, 17, 14, 5, 0, 3, 1], "incorrect_counts": [0, 0, 0, 4, 4, 3, 11, 16, 11, 17, 19, 18, 25, 20, 14, 12, 5, 1, 3, 1]}, "energy/std": {"edges": [0.07470156430161684, 0.08958366354400084, 0.10446576278638484, 0.11934786202876882, 0.13422996127115283, 0.14911206051353681, 0.1639941597559208, 0.17887625899830478, 0.1937583582406888, 0.20864045748307278, 0.2235225567254568, 0.23840465596784077, 0.25328675521022476, 0.26816885445260874, 0.2830509536949928, 0.29793305293737676, 0.31281515217976075, 0.32769725142214473, 0.3425793506645287, 0.35746144990691275, 0.37234354914929674], "correct_counts": [2, 3, 10, 21, 30, 45, 56, 53, 49, 37, 39, 29, 16, 15, 4, 0, 4, 1, 1, 1], "incorrect_counts": [0, 0, 4, 7, 12, 22, 27, 26, 28, 16, 9, 13, 9, 4, 2, 4, 1, 0, 0, 0]}, "curv/lmax_mean": {"edges": [1.000376323858897, 1.0019486089547476, 1.0035208940505982, 1.0050931791464488, 1.0066654642422994, 1.00823774933815, 1.0098100344340006, 1.0113823195298512, 1.0129546046257019, 1.0145268897215525, 1.0160991748174033, 1.017671459913254, 1.0192437450091045, 1.0208160301049551, 1.0223883152008058, 1.0239606002966564, 1.025532885392507, 1.0271051704883576, 1.0286774555842082, 1.0302497406800588, 1.0318220257759094], "correct_counts": [40, 93, 93, 78, 31, 33, 12, 13, 9, 8, 2, 1, 0, 2, 0, 0, 0, 0, 0, 1], "incorrect_counts": [9, 39, 42, 33, 23, 16, 11, 0, 6, 1, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.000000238418579, 1.0040149986743927, 1.0080297589302063, 1.01204451918602, 1.0160592794418335, 1.020074039697647, 1.0240887999534607, 1.0281035602092743, 1.032118320465088, 1.0361330807209015, 1.040147840976715, 1.0441626012325287, 1.0481773614883423, 1.0521921217441559, 1.0562068819999695, 1.060221642255783, 1.0642364025115967, 1.0682511627674103, 1.0722659230232239, 1.0762806832790375, 1.080295443534851], "correct_counts": [260, 66, 30, 25, 15, 5, 4, 3, 3, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [112, 33, 17, 3, 4, 4, 6, 2, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.01093864440918, 32.016456858317056, 32.02197507222493, 32.02749328613281, 32.03301150004069, 32.03852971394857, 32.04404792785645, 32.04956614176432, 32.0550843556722, 32.06060256958008, 32.06612078348796, 32.07163899739584, 32.077157211303714, 32.08267542521159, 32.08819363911947, 32.093711853027344, 32.09923006693522, 32.104748280843104, 32.11026649475098, 32.11578470865886, 32.121302922566734], "correct_counts": [3, 1, 12, 19, 29, 44, 48, 53, 54, 50, 33, 21, 16, 11, 7, 8, 3, 1, 1, 2], "incorrect_counts": [0, 0, 0, 7, 12, 17, 25, 30, 20, 25, 17, 14, 4, 3, 4, 2, 2, 1, 1, 0]}, "curv/trace_best": {"edges": [32.00436782836914, 32.01764259338379, 32.03091735839844, 32.04419212341308, 32.05746688842773, 32.07074165344238, 32.08401641845703, 32.09729118347168, 32.110565948486325, 32.123840713500975, 32.137115478515625, 32.150390243530275, 32.163665008544925, 32.17693977355957, 32.19021453857422, 32.20348930358887, 32.21676406860352, 32.23003883361817, 32.24331359863281, 32.25658836364746, 32.26986312866211], "correct_counts": [22, 78, 72, 75, 64, 36, 21, 20, 10, 6, 4, 3, 1, 2, 0, 1, 0, 0, 0, 1], "incorrect_counts": [5, 30, 32, 33, 27, 17, 16, 6, 8, 4, 3, 2, 1, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6516666666666666, 0.6576666666666666, 0.6636666666666666, 0.6696666666666666, 0.6756666666666666, 0.6816666666666666, 0.6876666666666666, 0.6936666666666667, 0.6996666666666667, 0.7056666666666667, 0.7116666666666667, 0.7176666666666667, 0.7236666666666667, 0.7296666666666668, 0.7356666666666668, 0.7416666666666668, 0.7476666666666668, 0.7536666666666668, 0.7596666666666668, 0.7656666666666668, 0.7716666666666668], "correct_counts": [2, 8, 8, 19, 25, 39, 49, 45, 61, 51, 35, 24, 20, 10, 8, 4, 5, 0, 1, 2], "incorrect_counts": [1, 7, 8, 17, 12, 25, 30, 19, 22, 15, 15, 6, 0, 1, 1, 3, 1, 0, 1, 0]}, "dynamics/drop": {"edges": [13.908043444156647, 21.40915985889733, 28.91027627363801, 36.41139268837869, 43.912509103119376, 51.413625517860055, 58.914741932600734, 66.41585834734141, 73.9169747620821, 81.41809117682278, 88.91920759156346, 96.42032400630414, 103.92144042104482, 111.4225568357855, 118.92367325052619, 126.42478966526687, 133.92590608000756, 141.42702249474823, 148.92813890948892, 156.4292553242296, 163.93037173897028], "correct_counts": [76, 122, 102, 58, 24, 17, 8, 2, 1, 2, 2, 0, 0, 0, 1, 0, 0, 0, 0, 1], "incorrect_counts": [61, 68, 30, 16, 4, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471071392297745, 1.1626865026851496, 1.1782658661405245, 1.1938452295958997, 1.2094245930512746, 1.2250039565066497, 1.2405833199620246, 1.2561626834173998, 1.2717420468727747, 1.2873214103281498, 1.302900773783525, 1.3184801372389, 1.3340595006942748, 1.34963886414965, 1.365218227605025, 1.3807975910604, 1.3963769545157751, 1.41195631797115, 1.4275356814265252, 1.4431150448819001, 1.4586944083372753], "correct_counts": [2, 3, 7, 6, 24, 31, 42, 53, 54, 48, 56, 24, 33, 15, 10, 3, 4, 0, 0, 1], "incorrect_counts": [0, 0, 4, 3, 15, 13, 21, 18, 15, 27, 29, 14, 17, 4, 2, 1, 0, 0, 0, 1]}}}, "ece_geometry": 0.04224579057460222}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "36bff160c306", "timestamp": "2026-07-27T09:36:14.731952+00:00", "git_sha": "3d7ce49", "config_hash": "ff22f3433ff5", "config": {"run": {"seed": 2, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6683333333333333, "base_error": 0.33166666666666667, "final_train_loss": 0.178430438041687, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.16709451083285565, "rho_basin": 0.28365045788141036, "energy_min": 0.37782573026863475, "energy_mean": 0.3915673002869955, "energy_std": 0.35511939449119495, "msp": 0.14732856097974725, "temp_msp": 0.13701445964444764, "entropy": 0.14443399349080804}, "temperature": 3.139475832001513, "best_energy_baseline": "energy_std", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.2107312194357791, 0.16540577903174386, 0.2560328602083732], "delta_aurc_vs_best_energy": [0.1880248836583393, 0.14315050073946656, 0.23734906074468637], "delta_aurc_vs_best_baseline": [-0.030080051188408008, -0.04648240416942707, -0.014728028214055597], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.027777777777777887, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.09444444444444461, 0.10416666666666667, 0.10784313725490197, 0.12499999999999999, 0.13596491228070176, 0.15, 0.15079365079365076, 0.16287878787878787, 0.17028985507246377, 0.18402777777777776, 0.18666666666666668, 0.1891025641025641, 0.1882716049382716, 0.19345238095238096, 0.2011494252873563, 0.20833333333333334, 0.20967741935483872, 0.20833333333333334, 0.21212121212121213, 0.22549019607843138, 0.2333333333333333, 0.23842592592592604, 0.23648648648648649, 0.24342105263157895, 0.25, 0.25833333333333336, 0.27032520325203263, 0.2797619047619047, 0.28875968992248063, 0.29734848484848486, 0.3, 0.3061594202898551, 0.3085106382978723, 0.31597222222222227, 0.32482993197278914, 0.33166666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2787878787878788, 0.278787878787879, 0.27878787878787903, 0.2787878787878791, 0.2787878787878791, 0.2787878787878788, 0.27878787878787864, 0.2787878787878785, 0.27878787878787836, 0.2787878787878784, 0.27878787878787864, 0.27878787878787886, 0.27878787878787903, 0.27878787878787914, 0.2787878787878793, 0.2787878787878794, 0.2787878787878795, 0.2787878787878796, 0.27878787878787964, 0.27878787878787975, 0.2787878787878798, 0.27878787878787986, 0.2787878787878799, 0.2787878787878799, 0.27878787878788, 0.27878787878788003, 0.2787878787878801, 0.2787878787878801, 0.27878787878788014, 0.27878787878788014, 0.2787878787878802, 0.2787878787878802, 0.27878787878788025, 0.27878787878788025, 0.2787878787878803, 0.2787878787878803, 0.2787878787878803, 0.27878787878788036, 0.27878787878788036, 0.27878787878788036, 0.2787878787878804, 0.283928571428573, 0.29050387596899363, 0.29793123543123673, 0.30722222222222356, 0.3103260869565228, 0.31414267834793586, 0.31864316239316337, 0.3231292517006814, 0.33166666666666755]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.2916666666666667, 0.3611111111111111, 0.375, 0.35, 0.38888888888888884, 0.40476190476190466, 0.4166666666666667, 0.4351851851851852, 0.42499999999999993, 0.4318181818181818, 0.4166666666666667, 0.40384615384615385, 0.3869047619047619, 0.388888888888889, 0.390625, 0.4019607843137255, 0.40277777777777773, 0.39473684210526316, 0.3958333333333333, 0.3928571428571429, 0.4015151515151515, 0.39492753623188404, 0.39236111111111105, 0.3933333333333333, 0.3942307692307692, 0.3950617283950617, 0.3869047619047619, 0.37931034482758613, 0.37777777777777777, 0.3736559139784946, 0.3645833333333333, 0.3661616161616162, 0.3627450980392157, 0.3666666666666666, 0.3657407407407407, 0.36261261261261263, 0.36622807017543857, 0.36538461538461536, 0.36875, 0.36382113821138207, 0.3591269841269841, 0.3507751937984496, 0.3503787878787879, 0.35185185185185186, 0.3496376811594203, 0.3439716312056737, 0.34027777777777773, 0.33503401360544216, 0.33166666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4791666666666667, 0.45, 0.45833333333333326, 0.4166666666666668, 0.3958333333333333, 0.42592592592592593, 0.45833333333333326, 0.4621212121212121, 0.4513888888888889, 0.4423076923076923, 0.44047619047619047, 0.4333333333333333, 0.4270833333333333, 0.4117647058823529, 0.40740740740740733, 0.39473684210526316, 0.38333333333333336, 0.3928571428571428, 0.39015151515151514, 0.38768115942028986, 0.38541666666666663, 0.3933333333333333, 0.40384615384615385, 0.4074074074074074, 0.40773809523809523, 0.410919540229885, 0.40555555555555556, 0.4032258064516129, 0.3932291666666667, 0.39141414141414144, 0.38480392156862747, 0.3761904761904763, 0.3726851851851851, 0.3738738738738739, 0.3684210526315789, 0.36324786324786323, 0.36041666666666666, 0.3577235772357723, 0.3591269841269841, 0.35658914728682173, 0.3522727272727273, 0.35, 0.34601449275362317, 0.34219858156028365, 0.33854166666666674, 0.33503401360544216, 0.33166666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.4583333333333333, 0.5, 0.4583333333333333, 0.45, 0.41666666666666663, 0.4166666666666668, 0.3958333333333333, 0.3888888888888889, 0.39166666666666655, 0.36363636363636365, 0.3611111111111111, 0.36538461538461536, 0.35119047619047616, 0.3499999999999999, 0.3645833333333333, 0.3627450980392157, 0.3611111111111112, 0.3684210526315789, 0.3875, 0.3690476190476191, 0.35984848484848486, 0.35144927536231885, 0.34027777777777773, 0.3333333333333333, 0.3333333333333333, 0.3395061728395062, 0.33630952380952384, 0.3304597701149425, 0.325, 0.3279569892473118, 0.3255208333333333, 0.3282828282828283, 0.31862745098039214, 0.3166666666666666, 0.31481481481481494, 0.31981981981981983, 0.31798245614035087, 0.3162393162393162, 0.3145833333333333, 0.31504065040650403, 0.31349206349206343, 0.313953488372093, 0.3162878787878788, 0.32037037037037036, 0.3278985507246377, 0.3315602836879432, 0.32986111111111105, 0.3299319727891156, 0.33166666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.020833333333333332, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.05555555555555555, 0.0641025641025641, 0.05952380952380952, 0.06666666666666665, 0.06770833333333333, 0.06862745098039216, 0.06944444444444443, 0.07017543859649122, 0.07916666666666666, 0.08333333333333331, 0.10227272727272728, 0.10869565217391304, 0.11805555555555554, 0.13333333333333333, 0.14423076923076922, 0.16358024691358025, 0.16666666666666666, 0.18390804597701146, 0.18888888888888888, 0.1989247311827957, 0.21354166666666666, 0.21717171717171718, 0.22794117647058823, 0.23571428571428568, 0.2407407407407407, 0.25, 0.2565789473684211, 0.2606837606837607, 0.26666666666666666, 0.26829268292682923, 0.26984126984126994, 0.28294573643410853, 0.29545454545454547, 0.3, 0.3079710144927536, 0.3156028368794327, 0.32291666666666674, 0.32482993197278914, 0.33166666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.03125, 0.027777777777777776, 0.04166666666666667, 0.045454545454545456, 0.041666666666666664, 0.038461538461538464, 0.047619047619047616, 0.049999999999999996, 0.052083333333333336, 0.06372549019607843, 0.06481481481481495, 0.06578947368421052, 0.06666666666666667, 0.07936507936507935, 0.09090909090909091, 0.10144927536231885, 0.11111111111111109, 0.12, 0.13141025641025642, 0.1388888888888889, 0.1488095238095238, 0.16379310344827602, 0.16944444444444445, 0.18010752688172044, 0.19010416666666666, 0.19696969696969696, 0.20588235294117646, 0.211904761904762, 0.21990740740740752, 0.22972972972972974, 0.23684210526315788, 0.24145299145299146, 0.24583333333333332, 0.2520325203252032, 0.263888888888889, 0.27325581395348836, 0.2821969696969697, 0.28888888888888886, 0.3007246376811594, 0.3085106382978724, 0.31944444444444436, 0.32482993197278914, 0.33166666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.020833333333333332, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.04861111111111111, 0.057692307692307696, 0.05952380952380952, 0.06666666666666665, 0.06770833333333333, 0.06862745098039216, 0.0648148148148148, 0.07017543859649122, 0.07916666666666666, 0.08333333333333345, 0.0946969696969697, 0.10869565217391304, 0.11805555555555554, 0.13333333333333333, 0.14743589743589744, 0.15432098765432098, 0.1636904761904762, 0.17528735632183906, 0.18611111111111112, 0.19623655913978494, 0.203125, 0.21464646464646464, 0.22058823529411764, 0.22380952380952393, 0.23842592592592604, 0.24774774774774774, 0.25, 0.2606837606837607, 0.25833333333333336, 0.26422764227642287, 0.2738095238095238, 0.27906976744186046, 0.2803030303030303, 0.28888888888888886, 0.3007246376811594, 0.30673758865248235, 0.31597222222222227, 0.3231292517006803, 0.33166666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.16541353383458646, 0.2751004016064257], "coverage": [0.0, 0.0, 0.0, 0.0, 0.44333333333333336, 0.83]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5988734194664094, "basin/entropy": 0.4017092946026893, "basin/dispersion": 0.5424253436759859, "energy/mean": 0.6074136267371771, "energy/min": 0.5868243962956929, "energy/std": 0.5071742753668592, "curv/lmax_mean": 0.3702302033860073, "curv/lmax_best": 0.38854496923520343, "curv/trace_mean": 0.3700798255617238, "curv/trace_best": 0.38669657514505196, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6523452674845549, "dynamics/drop": 0.7109612902417324, "dynamics/residual": 0.5072619957643579}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [0, 0, 3, 0, 0, 6, 0, 0, 8, 0, 0, 11, 0, 0, 3, 0, 0, 13, 0, 357], "incorrect_counts": [2, 0, 7, 0, 0, 7, 0, 0, 9, 0, 0, 9, 0, 0, 10, 0, 0, 17, 0, 138]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [357, 0, 0, 0, 0, 13, 0, 0, 3, 0, 10, 7, 8, 1, 0, 1, 0, 0, 1, 0], "incorrect_counts": [138, 0, 0, 0, 0, 17, 0, 0, 10, 0, 9, 9, 13, 0, 0, 0, 0, 1, 0, 2]}, "basin/dispersion": {"edges": [1.6439221991114048, 1.6628586470572428, 1.6817950950030807, 1.7007315429489187, 1.7196679908947567, 1.7386044388405946, 1.7575408867864326, 1.7764773347322707, 1.7954137826781085, 1.8143502306239465, 1.8332866785697846, 1.8522231265156224, 1.8711595744614604, 1.8900960224072985, 1.9090324703531363, 1.9279689182989743, 1.9469053662448124, 1.9658418141906502, 1.9847782621364882, 2.0037147100823263, 2.022651158028164], "correct_counts": [1, 2, 3, 13, 14, 13, 26, 31, 36, 41, 34, 51, 43, 31, 21, 18, 14, 5, 1, 3], "incorrect_counts": [1, 3, 2, 10, 7, 6, 16, 15, 19, 18, 20, 23, 22, 17, 6, 5, 4, 3, 1, 1]}, "energy/mean": {"edges": [0.41955216725667316, 0.4642234648267428, 0.5088947623968124, 0.5535660599668821, 0.5982373575369517, 0.6429086551070213, 0.6875799526770909, 0.7322512502471605, 0.7769225478172302, 0.8215938453873, 0.8662651429573696, 0.9109364405274392, 0.9556077380975088, 1.0002790356675784, 1.044950333237648, 1.0896216308077178, 1.1342929283777874, 1.178964225947857, 1.2236355235179266, 1.2683068210879962, 1.3129781186580658], "correct_counts": [5, 2, 3, 7, 13, 19, 17, 40, 47, 35, 45, 54, 40, 26, 26, 12, 4, 4, 1, 1], "incorrect_counts": [2, 1, 5, 5, 13, 9, 21, 25, 18, 28, 25, 16, 13, 10, 4, 2, 2, 0, 0, 0]}, "energy/min": {"edges": [0.02020469307899475, 0.07093883007764816, 0.12167296707630157, 0.17240710407495496, 0.2231412410736084, 0.2738753780722618, 0.3246095150709152, 0.3753436520695686, 0.426077789068222, 0.47681192606687545, 0.5275460630655289, 0.5782802000641822, 0.6290143370628356, 0.6797484740614891, 0.7304826110601425, 0.7812167480587959, 0.8319508850574493, 0.8826850220561027, 0.9334191590547561, 0.9841532960534095, 1.034887433052063], "correct_counts": [0, 2, 5, 10, 7, 16, 17, 35, 39, 31, 47, 49, 45, 41, 22, 16, 8, 7, 3, 1], "incorrect_counts": [1, 1, 2, 2, 9, 6, 22, 19, 25, 21, 22, 17, 30, 9, 8, 3, 0, 2, 0, 0]}, "energy/std": {"edges": [0.08805419068698145, 0.10022669760053629, 0.11239920451409115, 0.12457171142764599, 0.13674421834120085, 0.1489167252547557, 0.16108923216831053, 0.17326173908186537, 0.18543424599542024, 0.19760675290897506, 0.20977925982252993, 0.22195176673608477, 0.2341242736496396, 0.24629678056319446, 0.25846928747674935, 0.27064179439030417, 0.282814301303859, 0.29498680821741385, 0.3071593151309687, 0.3193318220445236, 0.3315043289580784], "correct_counts": [2, 1, 6, 14, 23, 31, 33, 30, 51, 42, 52, 35, 28, 18, 11, 10, 5, 3, 4, 2], "incorrect_counts": [0, 4, 7, 7, 14, 15, 15, 21, 15, 17, 18, 15, 10, 14, 14, 5, 2, 3, 3, 0]}, "curv/lmax_mean": {"edges": [0.9991512497266134, 0.9997431829571725, 1.0003351161877314, 1.0009270494182905, 1.0015189826488495, 1.0021109158794086, 1.0027028491099677, 1.0032947823405267, 1.0038867155710856, 1.0044786488016446, 1.0050705820322037, 1.0056625152627627, 1.0062544484933218, 1.0068463817238809, 1.00743831495444, 1.008030248184999, 1.008622181415558, 1.0092141146461169, 1.009806047876676, 1.010397981107235, 1.010989914337794], "correct_counts": [76, 176, 62, 38, 24, 9, 7, 4, 1, 0, 1, 2, 0, 0, 0, 0, 0, 1, 0, 0], "incorrect_counts": [22, 68, 31, 26, 21, 10, 5, 1, 0, 6, 2, 3, 1, 1, 0, 0, 0, 0, 1, 1]}, "curv/lmax_best": {"edges": [0.994626522064209, 0.996795529127121, 0.998964536190033, 1.001133543252945, 1.003302550315857, 1.005471557378769, 1.0076405644416808, 1.009809571504593, 1.0119785785675048, 1.014147585630417, 1.0163165926933289, 1.0184855997562408, 1.0206546068191529, 1.0228236138820648, 1.024992620944977, 1.0271616280078888, 1.0293306350708007, 1.0314996421337128, 1.0336686491966247, 1.0358376562595368, 1.0380066633224487], "correct_counts": [1, 11, 344, 27, 7, 5, 4, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 5, 149, 23, 9, 2, 2, 3, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/trace_mean": {"edges": [31.952821254730225, 31.960075608889262, 31.9673299630483, 31.974584317207338, 31.981838671366376, 31.989093025525413, 31.996347379684448, 32.00360173384349, 32.01085608800253, 32.018110442161564, 32.0253647963206, 32.03261915047963, 32.03987350463867, 32.04712785879771, 32.054382212956746, 32.061636567115784, 32.06889092127482, 32.07614527543386, 32.0833996295929, 32.090653983751935, 32.09790833791097], "correct_counts": [3, 2, 12, 26, 51, 59, 58, 53, 50, 35, 21, 17, 8, 2, 3, 0, 0, 1, 0, 0], "incorrect_counts": [0, 1, 4, 5, 17, 16, 28, 26, 22, 27, 19, 11, 12, 3, 5, 0, 2, 0, 0, 1]}, "curv/trace_best": {"edges": [31.93198013305664, 31.942170333862304, 31.952360534667967, 31.962550735473634, 31.972740936279298, 31.98293113708496, 31.993121337890624, 32.00331153869629, 32.013501739501955, 32.023691940307614, 32.03388214111328, 32.04407234191895, 32.05426254272461, 32.064452743530275, 32.074642944335935, 32.0848331451416, 32.09502334594727, 32.10521354675293, 32.115403747558595, 32.125593948364255, 32.13578414916992], "correct_counts": [2, 2, 4, 11, 33, 70, 97, 88, 39, 24, 19, 4, 3, 3, 1, 1, 0, 0, 0, 0], "incorrect_counts": [0, 1, 1, 3, 9, 24, 41, 39, 29, 21, 10, 8, 3, 3, 2, 1, 2, 0, 0, 2]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [401, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6266666666666666, 0.6331666666666667, 0.6396666666666666, 0.6461666666666666, 0.6526666666666666, 0.6591666666666667, 0.6656666666666666, 0.6721666666666666, 0.6786666666666666, 0.6851666666666667, 0.6916666666666667, 0.6981666666666666, 0.7046666666666667, 0.7111666666666667, 0.7176666666666667, 0.7241666666666666, 0.7306666666666667, 0.7371666666666667, 0.7436666666666667, 0.7501666666666666, 0.7566666666666667], "correct_counts": [0, 0, 0, 0, 5, 7, 13, 22, 27, 49, 58, 43, 51, 41, 32, 21, 16, 9, 3, 4], "incorrect_counts": [1, 0, 0, 0, 2, 6, 17, 28, 18, 27, 34, 20, 16, 18, 5, 1, 4, 2, 0, 0]}, "dynamics/drop": {"edges": [14.020859281222025, 18.926390168940028, 23.83192105665803, 28.737451944376033, 33.642982832094035, 38.548513719812036, 43.45404460753004, 48.35957549524804, 53.26510638296604, 58.17063727068404, 63.07616815840204, 67.98169904612004, 72.88722993383806, 77.79276082155606, 82.69829170927406, 87.60382259699206, 92.50935348471006, 97.41488437242806, 102.32041526014606, 107.22594614786406, 112.13147703558207], "correct_counts": [19, 90, 74, 61, 44, 39, 28, 17, 11, 9, 3, 3, 2, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [35, 66, 50, 28, 11, 4, 3, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1569731881221135, 1.1719582500557104, 1.1869433119893074, 1.2019283739229043, 1.2169134358565012, 1.2318984977900982, 1.2468835597236951, 1.261868621657292, 1.276853683590889, 1.291838745524486, 1.3068238074580827, 1.3218088693916799, 1.3367939313252766, 1.3517789932588735, 1.3667640551924705, 1.3817491171260674, 1.3967341790596643, 1.4117192409932613, 1.4267043029268582, 1.4416893648604552, 1.4566744267940521], "correct_counts": [0, 7, 3, 13, 26, 24, 43, 49, 49, 46, 46, 34, 24, 19, 10, 4, 2, 1, 0, 1], "incorrect_counts": [5, 2, 5, 2, 16, 11, 18, 23, 22, 25, 21, 20, 12, 9, 3, 4, 0, 1, 0, 0]}}}, "ece_geometry": 0.03781877930416236}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "83458d9ef896", "timestamp": "2026-07-27T09:36:29.379451+00:00", "git_sha": "3d7ce49", "config_hash": "fbed14eeb756", "config": {"run": {"seed": 3, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7133333333333334, "base_error": 0.2866666666666667, "final_train_loss": 0.16939383745193481, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.17019370187580518, "rho_basin": 0.23474252197420142, "energy_min": 0.3728384148545108, "energy_mean": 0.38232702405148095, "energy_std": 0.3016550018932722, "msp": 0.12508580744878345, "temp_msp": 0.1281729327488173, "entropy": 0.12478032569162513}, "temperature": 3.40053609466433, "best_energy_baseline": "energy_std", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.20264471297870562, 0.14299561878502962, 0.2569749266815167], "delta_aurc_vs_best_energy": [0.13146130001746703, 0.08951962500452121, 0.17179871890451937], "delta_aurc_vs_best_baseline": [-0.04541337618418005, -0.06759052108205994, -0.02516698291343395], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.041666666666666664, 0.08333333333333333, 0.09722222222222221, 0.09523809523809505, 0.125, 0.1388888888888889, 0.13333333333333336, 0.13636363636363635, 0.13194444444444445, 0.12179487179487179, 0.11904761904761904, 0.12222222222222219, 0.13020833333333334, 0.13725490196078433, 0.12962962962962976, 0.14035087719298245, 0.14583333333333334, 0.1706349206349206, 0.17424242424242425, 0.17391304347826086, 0.18055555555555552, 0.18, 0.18269230769230768, 0.18209876543209877, 0.18154761904761904, 0.18678160919540227, 0.18055555555555555, 0.18548387096774194, 0.19270833333333334, 0.19696969696969696, 0.2034313725490196, 0.20476190476190473, 0.20833333333333331, 0.21396396396396397, 0.22807017543859648, 0.2329059829059829, 0.2375, 0.23780487804878045, 0.23611111111111108, 0.2441860465116279, 0.25, 0.2574074074074074, 0.26268115942028986, 0.27304964539007087, 0.2760416666666668, 0.28401360544217685, 0.2866666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.22920892494929004, 0.2292089249492901, 0.22920892494929015, 0.2292089249492902, 0.22920892494929024, 0.22920892494929024, 0.22920892494929027, 0.22920892494929027, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.23576097105508903, 0.2425900592795261, 0.2492715617715621, 0.2574074074074077, 0.26518952062430345, 0.27160904255319174, 0.27650462962962985, 0.28099017384731684, 0.28666666666666685]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.5, 0.5277777777777778, 0.5208333333333334, 0.4666666666666667, 0.4305555555555555, 0.4404761904761906, 0.40625, 0.4166666666666667, 0.4083333333333334, 0.4090909090909091, 0.4097222222222222, 0.40384615384615385, 0.39880952380952384, 0.39444444444444454, 0.3958333333333333, 0.39705882352941174, 0.3935185185185185, 0.38596491228070173, 0.375, 0.37698412698412703, 0.3787878787878788, 0.36594202898550726, 0.3680555555555555, 0.36, 0.36217948717948717, 0.3611111111111111, 0.3541666666666667, 0.3448275862068967, 0.34444444444444444, 0.34139784946236557, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.3261904761904763, 0.3240740740740742, 0.32207207207207206, 0.3157894736842105, 0.30982905982905984, 0.30625, 0.3028455284552845, 0.3015873015873017, 0.29651162790697677, 0.29924242424242425, 0.29444444444444445, 0.2916666666666667, 0.2943262411347518, 0.29166666666666663, 0.2891156462585034, 0.2866666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.6666666666666666, 0.5416666666666666, 0.5277777777777778, 0.5833333333333334, 0.5166666666666667, 0.48611111111111105, 0.4523809523809525, 0.4375, 0.39814814814814814, 0.3833333333333334, 0.4015151515151515, 0.4027777777777778, 0.41025641025641024, 0.4226190476190476, 0.4166666666666668, 0.4166666666666667, 0.4019607843137255, 0.4027777777777779, 0.39035087719298245, 0.38333333333333336, 0.3690476190476191, 0.35984848484848486, 0.36231884057971014, 0.36111111111111116, 0.36, 0.358974358974359, 0.3549382716049383, 0.34523809523809523, 0.3448275862068967, 0.3416666666666667, 0.3387096774193548, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.32380952380952377, 0.324074074074074, 0.31981981981981983, 0.3157894736842105, 0.31196581196581197, 0.3104166666666667, 0.30894308943089427, 0.3055555555555555, 0.3023255813953488, 0.29734848484848486, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.2899305555555555, 0.29081632653061223, 0.2866666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.2916666666666667, 0.3055555555555556, 0.3541666666666667, 0.36666666666666664, 0.33333333333333326, 0.29761904761904767, 0.3125, 0.3055555555555556, 0.29166666666666674, 0.2878787878787879, 0.2708333333333333, 0.2948717948717949, 0.2976190476190476, 0.2944444444444444, 0.2864583333333333, 0.29411764705882354, 0.3009259259259259, 0.29385964912280704, 0.3, 0.2936507936507936, 0.29924242424242425, 0.2898550724637681, 0.29513888888888895, 0.29333333333333333, 0.28846153846153844, 0.28703703703703703, 0.27976190476190477, 0.278735632183908, 0.2833333333333333, 0.28225806451612906, 0.2838541666666667, 0.28535353535353536, 0.2818627450980392, 0.28333333333333344, 0.287037037037037, 0.28603603603603606, 0.29385964912280704, 0.2948717948717949, 0.29791666666666666, 0.2947154471544715, 0.2916666666666668, 0.29069767441860467, 0.2897727272727273, 0.28703703703703703, 0.2916666666666667, 0.2907801418439717, 0.29340277777777773, 0.2925170068027211, 0.2866666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.057692307692307696, 0.05952380952380952, 0.061111111111111095, 0.057291666666666664, 0.06372549019607843, 0.07407407407407406, 0.07894736842105263, 0.08333333333333333, 0.08730158730158742, 0.10606060606060606, 0.11594202898550725, 0.11111111111111109, 0.12, 0.12179487179487179, 0.13271604938271606, 0.13988095238095238, 0.14942528735632182, 0.16111111111111112, 0.1639784946236559, 0.17447916666666666, 0.17676767676767677, 0.17647058823529413, 0.1833333333333333, 0.1898148148148148, 0.1891891891891892, 0.19736842105263158, 0.20726495726495728, 0.2125, 0.21951219512195133, 0.22817460317460328, 0.23449612403100775, 0.24431818181818182, 0.25, 0.2608695652173913, 0.2677304964539007, 0.27256944444444436, 0.282312925170068, 0.2866666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.04166666666666667, 0.045454545454545456, 0.05555555555555555, 0.057692307692307696, 0.05357142857142857, 0.055555555555555726, 0.078125, 0.0784313725490196, 0.07870370370370385, 0.09210526315789473, 0.09583333333333334, 0.10317460317460315, 0.10984848484848485, 0.11956521739130435, 0.12499999999999999, 0.13666666666666666, 0.14102564102564102, 0.1388888888888889, 0.13392857142857142, 0.13505747126436798, 0.14444444444444443, 0.1586021505376344, 0.15885416666666666, 0.16666666666666666, 0.17892156862745098, 0.18095238095238092, 0.1851851851851853, 0.1981981981981982, 0.21052631578947367, 0.22008547008547008, 0.22291666666666668, 0.23170731707317085, 0.23412698412698424, 0.23837209302325582, 0.24242424242424243, 0.2518518518518518, 0.2554347826086957, 0.26241134751773043, 0.2690972222222223, 0.282312925170068, 0.2866666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.05128205128205128, 0.05952380952380952, 0.061111111111111095, 0.057291666666666664, 0.06372549019607843, 0.07407407407407406, 0.07894736842105263, 0.0875, 0.09126984126984125, 0.10984848484848485, 0.11594202898550725, 0.11111111111111109, 0.11666666666666667, 0.12179487179487179, 0.13580246913580246, 0.14285714285714285, 0.14942528735632202, 0.15555555555555556, 0.1639784946236559, 0.171875, 0.17424242424242425, 0.17401960784313725, 0.1833333333333333, 0.18750000000000014, 0.19144144144144143, 0.19956140350877194, 0.2094017094017094, 0.21041666666666667, 0.2195121951219512, 0.2261904761904763, 0.24031007751937986, 0.24242424242424243, 0.25, 0.25181159420289856, 0.25886524822695045, 0.27256944444444436, 0.28061224489795916, 0.2866666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.09090909090909091, 0.2038369304556355], "coverage": [0.0, 0.0, 0.0, 0.0, 0.12833333333333333, 0.695]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6152942295153228, "basin/entropy": 0.38495707454901107, "basin/dispersion": 0.5045913931753967, "energy/mean": 0.6219707672245164, "energy/min": 0.6181264942403826, "energy/std": 0.5105955227124538, "curv/lmax_mean": 0.3801755053249294, "curv/lmax_best": 0.38900510758530754, "curv/trace_mean": 0.37158362312540755, "curv/trace_best": 0.3750747120191263, "dynamics/steps": 0.5, "dynamics/monotonic": 0.5918346555096718, "dynamics/drop": 0.6229623994783743, "dynamics/residual": 0.4728048250380352}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [3, 0, 0, 4, 0, 0, 8, 0, 0, 0, 7, 0, 0, 10, 0, 0, 16, 0, 0, 380], "incorrect_counts": [4, 0, 0, 5, 0, 0, 7, 0, 0, 0, 9, 0, 0, 16, 0, 0, 18, 0, 0, 113]}, "basin/entropy": {"edges": [0.0, 0.05057021323536759, 0.10114042647073518, 0.15171063970610277, 0.20228085294147036, 0.25285106617683795, 0.30342127941220554, 0.3539914926475731, 0.4045617058829407, 0.4551319191183083, 0.5057021323536759, 0.5562723455890435, 0.6068425588244111, 0.6574127720597787, 0.7079829852951462, 0.7585531985305138, 0.8091234117658814, 0.859693625001249, 0.9102638382366166, 0.9608340514719842, 1.0114042647073518], "correct_counts": [380, 0, 0, 0, 0, 16, 0, 0, 10, 0, 0, 5, 7, 7, 2, 0, 1, 0, 0, 0], "incorrect_counts": [113, 0, 0, 0, 0, 18, 0, 0, 16, 0, 0, 9, 5, 7, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.638197206426302, 1.6606953621248024, 1.6831935178233026, 1.705691673521803, 1.7281898292203033, 1.7506879849188037, 1.773186140617304, 1.7956842963158044, 1.8181824520143046, 1.840680607712805, 1.8631787634113053, 1.8856769191098057, 1.9081750748083062, 1.9306732305068064, 1.9531713862053068, 1.975669541903807, 1.9981676976023075, 2.020665853300808, 2.043164008999308, 2.0656621646978084, 2.088160320396309], "correct_counts": [2, 7, 9, 12, 27, 47, 52, 47, 61, 52, 32, 23, 29, 9, 6, 8, 4, 0, 0, 1], "incorrect_counts": [3, 0, 6, 12, 8, 15, 19, 17, 23, 20, 18, 13, 10, 0, 5, 2, 1, 0, 0, 0]}, "energy/mean": {"edges": [0.4687643547852834, 0.5167818620800972, 0.5647993693749109, 0.6128168766697248, 0.6608343839645385, 0.7088518912593524, 0.7568693985541661, 0.80488690584898, 0.8529044131437937, 0.9009219204386076, 0.9489394277334213, 0.9969569350282352, 1.044974442323049, 1.0929919496178628, 1.1410094569126765, 1.1890269642074904, 1.2370444715023041, 1.285061978797118, 1.3330794860919317, 1.3810969933867456, 1.4291145006815593], "correct_counts": [0, 0, 2, 9, 12, 33, 30, 23, 44, 31, 37, 43, 34, 39, 35, 24, 18, 5, 6, 3], "incorrect_counts": [1, 1, 3, 9, 15, 13, 15, 21, 14, 11, 13, 16, 10, 8, 8, 6, 5, 3, 0, 0]}, "energy/min": {"edges": [0.13711321353912354, 0.18594235181808472, 0.2347714900970459, 0.2836006283760071, 0.33242976665496826, 0.38125890493392944, 0.4300880432128906, 0.4789171814918518, 0.527746319770813, 0.5765754580497742, 0.6254045963287354, 0.6742337346076965, 0.7230628728866577, 0.7718920111656189, 0.8207211494445801, 0.8695502877235413, 0.9183794260025024, 0.9672085642814636, 1.0160377025604248, 1.064866840839386, 1.1136959791183472], "correct_counts": [0, 1, 4, 5, 10, 21, 25, 40, 32, 31, 30, 47, 36, 46, 32, 24, 13, 17, 11, 3], "incorrect_counts": [2, 3, 1, 1, 15, 9, 16, 24, 16, 13, 13, 14, 10, 9, 7, 6, 7, 3, 2, 1]}, "energy/std": {"edges": [0.07382017921196296, 0.09043346825542178, 0.10704675729888061, 0.12366004634233943, 0.14027333538579825, 0.15688662442925708, 0.1734999134727159, 0.19011320251617472, 0.20672649155963355, 0.22333978060309237, 0.2399530696465512, 0.25656635869001, 0.2731796477334688, 0.28979293677692763, 0.30640622582038646, 0.3230195148638453, 0.3396328039073041, 0.3562460929507629, 0.37285938199422175, 0.3894726710376806, 0.40608596008113945], "correct_counts": [0, 6, 13, 19, 47, 60, 63, 62, 47, 48, 25, 11, 6, 11, 6, 1, 2, 0, 0, 1], "incorrect_counts": [1, 4, 3, 14, 14, 25, 26, 20, 19, 24, 7, 10, 2, 3, 0, 0, 0, 0, 0, 0]}, "curv/lmax_mean": {"edges": [0.9988449911276499, 0.9992352331678073, 0.9996254752079645, 1.0000157172481219, 1.0004059592882792, 1.0007962013284366, 1.0011864433685937, 1.001576685408751, 1.0019669274489085, 1.0023571694890658, 1.0027474115292232, 1.0031376535693806, 1.0035278956095377, 1.003918137649695, 1.0043083796898524, 1.0046986217300098, 1.0050888637701672, 1.0054791058103245, 1.0058693478504817, 1.006259589890639, 1.0066498319307964], "correct_counts": [12, 134, 143, 44, 37, 20, 17, 5, 5, 3, 5, 2, 0, 0, 0, 1, 0, 0, 0, 0], "incorrect_counts": [2, 26, 55, 28, 25, 17, 7, 3, 1, 4, 0, 2, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.995161235332489, 0.9962190896272659, 0.9972769439220428, 0.9983347982168198, 0.9993926525115967, 1.0004505068063736, 1.0015083611011506, 1.0025662153959274, 1.0036240696907044, 1.0046819239854812, 1.0057397782802582, 1.0067976325750352, 1.007855486869812, 1.008913341164589, 1.0099711954593658, 1.0110290497541428, 1.0120869040489198, 1.0131447583436965, 1.0142026126384736, 1.0152604669332503, 1.0163183212280273], "correct_counts": [1, 3, 10, 64, 293, 33, 14, 3, 1, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 0, 6, 11, 120, 17, 7, 3, 3, 0, 2, 0, 0, 0, 0, 1, 0, 1, 0, 1]}, "curv/trace_mean": {"edges": [31.904047807057697, 31.911127193768817, 31.91820658047994, 31.92528596719106, 31.93236535390218, 31.9394447406133, 31.946524127324423, 31.953603514035542, 31.960682900746665, 31.967762287457784, 31.974841674168907, 31.981921060880026, 31.989000447591145, 31.996079834302268, 32.00315922101339, 32.01023860772451, 32.01731799443563, 32.02439738114675, 32.031476767857875, 32.038556154568994, 32.04563554128011], "correct_counts": [3, 2, 4, 13, 19, 14, 19, 32, 51, 37, 20, 31, 34, 31, 28, 36, 27, 14, 8, 5], "incorrect_counts": [0, 0, 0, 1, 3, 3, 10, 9, 5, 10, 13, 15, 10, 11, 19, 25, 18, 10, 8, 2]}, "curv/trace_best": {"edges": [31.868419647216797, 31.879567527770995, 31.890715408325196, 31.901863288879394, 31.913011169433595, 31.924159049987793, 31.93530693054199, 31.946454811096192, 31.95760269165039, 31.96875057220459, 31.97989845275879, 31.991046333312987, 32.002194213867185, 32.01334209442139, 32.02448997497559, 32.035637855529785, 32.04678573608398, 32.05793361663818, 32.069081497192386, 32.08022937774658, 32.09137725830078], "correct_counts": [1, 2, 3, 6, 11, 13, 24, 26, 46, 55, 60, 60, 53, 37, 14, 11, 3, 1, 2, 0], "incorrect_counts": [0, 0, 1, 1, 5, 2, 2, 8, 9, 18, 26, 20, 29, 26, 15, 6, 1, 2, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [428, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6449999999999999, 0.6505833333333333, 0.6561666666666666, 0.66175, 0.6673333333333332, 0.6729166666666666, 0.6785, 0.6840833333333333, 0.6896666666666667, 0.6952499999999999, 0.7008333333333333, 0.7064166666666667, 0.712, 0.7175833333333334, 0.7231666666666666, 0.72875, 0.7343333333333334, 0.7399166666666667, 0.7455, 0.7510833333333333, 0.7566666666666667], "correct_counts": [3, 3, 9, 15, 16, 32, 38, 31, 53, 40, 32, 54, 32, 20, 21, 10, 7, 10, 0, 2], "incorrect_counts": [1, 2, 5, 6, 12, 19, 17, 15, 31, 17, 10, 13, 9, 4, 5, 3, 2, 0, 1, 0]}, "dynamics/drop": {"edges": [14.044291233023008, 17.461781060944002, 20.879270888864994, 24.29676071678599, 27.71425054470698, 31.131740372627974, 34.54923020054897, 37.966720028469965, 41.38420985639095, 44.80169968431195, 48.219189512232944, 51.63667934015394, 55.05416916807493, 58.471658995995924, 61.88914882391692, 65.3066386518379, 68.7241284797589, 72.14161830767989, 75.55910813560088, 78.97659796352188, 82.39408779144287], "correct_counts": [20, 45, 59, 60, 64, 49, 39, 24, 17, 14, 13, 8, 3, 5, 4, 1, 0, 1, 0, 2], "incorrect_counts": [10, 26, 43, 29, 18, 19, 10, 11, 3, 2, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.157560517390569, 1.171275670826435, 1.184990824262301, 1.1987059776981672, 1.2124211311340332, 1.226136284569899, 1.2398514380057652, 1.2535665914416314, 1.2672817448774973, 1.2809968983133633, 1.2947120517492294, 1.3084272051850956, 1.3221423586209615, 1.3358575120568275, 1.3495726654926936, 1.3632878189285598, 1.3770029723644257, 1.3907181258002916, 1.4044332792361578, 1.418148432672024, 1.4318635861078899], "correct_counts": [2, 1, 7, 4, 14, 21, 36, 48, 45, 57, 40, 31, 33, 24, 24, 20, 10, 5, 3, 3], "incorrect_counts": [0, 2, 2, 5, 1, 6, 8, 20, 15, 26, 19, 19, 15, 13, 7, 7, 2, 2, 1, 2]}}}, "ece_geometry": 0.03179577510102621}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "1cc6f657fd8f", "timestamp": "2026-07-27T09:36:43.288073+00:00", "git_sha": "3d7ce49", "config_hash": "609a4152cf1e", "config": {"run": {"seed": 4, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.21295076608657837, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.13764728016172775, "rho_basin": 0.24498732350693359, "energy_min": 0.21394670001342045, "energy_mean": 0.21459813139712064, "energy_std": 0.29739590638116215, "msp": 0.12281914609984014, "temp_msp": 0.1165771595942391, "entropy": 0.11947392267830788}, "temperature": 2.500094145198995, "best_energy_baseline": "energy_min", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "delta_aurc_vs_best_energy": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "delta_aurc_vs_best_baseline": [-0.02107012056748865, -0.03689862693304643, -0.005932362296364964], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.041666666666666664, 0.03333333333333333, 0.04166666666666666, 0.059523809523809534, 0.052083333333333336, 0.05555555555555555, 0.06666666666666668, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.08888888888888888, 0.09375, 0.09803921568627451, 0.10185185185185183, 0.10526315789473684, 0.10833333333333334, 0.11904761904761903, 0.125, 0.13043478260869565, 0.13194444444444442, 0.13666666666666666, 0.13782051282051283, 0.1419753086419753, 0.14583333333333334, 0.1436781609195402, 0.15555555555555556, 0.16129032258064516, 0.16666666666666666, 0.18181818181818182, 0.18872549019607843, 0.19523809523809538, 0.19444444444444456, 0.19369369369369369, 0.19956140350877194, 0.202991452991453, 0.20833333333333334, 0.21747967479674807, 0.22619047619047616, 0.23255813953488372, 0.24621212121212122, 0.2537037037037037, 0.2608695652173913, 0.2641843971631207, 0.2690972222222223, 0.27721088435374147, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.24151696606786424, 0.24151696606786435, 0.24151696606786446, 0.24151696606786452, 0.24151696606786455, 0.2415169660678644, 0.24151696606786419, 0.24151696606786402, 0.2415169660678639, 0.2415169660678638, 0.24151696606786371, 0.24151696606786366, 0.2415169660678636, 0.24151696606786355, 0.24151696606786352, 0.24151696606786346, 0.24151696606786344, 0.2415169660678634, 0.24151696606786338, 0.24151696606786335, 0.24151696606786333, 0.24151696606786333, 0.2415169660678633, 0.24151696606786327, 0.24151696606786327, 0.24151696606786324, 0.24151696606786324, 0.24151696606786321, 0.24151696606786321, 0.2415169660678632, 0.2415169660678632, 0.2415169660678632, 0.24151696606786316, 0.24151696606786316, 0.24151696606786316, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.2415169660678631, 0.24241780045351358, 0.2459163898117374, 0.2492559523809511, 0.2551146384479704, 0.26169301712779824, 0.2708594075928229, 0.27747140522875685, 0.2808956916099758, 0.2816666666666653]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.1111111111111111, 0.125, 0.15, 0.18055555555555552, 0.1785714285714286, 0.17708333333333334, 0.2037037037037037, 0.1916666666666667, 0.18181818181818182, 0.18055555555555555, 0.17307692307692307, 0.18452380952380953, 0.18333333333333346, 0.19270833333333334, 0.19117647058823528, 0.1851851851851853, 0.20175438596491227, 0.21666666666666667, 0.21428571428571425, 0.22348484848484848, 0.2210144927536232, 0.21527777777777776, 0.21, 0.21474358974358973, 0.21296296296296297, 0.2113095238095238, 0.21264367816091967, 0.21944444444444444, 0.22311827956989247, 0.23177083333333334, 0.23232323232323232, 0.24019607843137256, 0.24999999999999997, 0.25231481481481494, 0.2545045045045045, 0.2543859649122807, 0.25427350427350426, 0.25625, 0.26219512195121947, 0.2718253968253969, 0.27325581395348836, 0.2784090909090909, 0.2851851851851852, 0.2826086956521739, 0.2819148936170214, 0.28472222222222215, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.1388888888888889, 0.16666666666666666, 0.15, 0.13888888888888887, 0.11904761904761907, 0.125, 0.12037037037037036, 0.11666666666666668, 0.12121212121212122, 0.14583333333333334, 0.17307692307692307, 0.19642857142857142, 0.19444444444444442, 0.19270833333333334, 0.18627450980392157, 0.19444444444444456, 0.20614035087719298, 0.20833333333333334, 0.21428571428571425, 0.21212121212121213, 0.20652173913043478, 0.21874999999999997, 0.22666666666666666, 0.22435897435897437, 0.2222222222222222, 0.22321428571428573, 0.2212643678160919, 0.21944444444444444, 0.22043010752688172, 0.22395833333333334, 0.23484848484848486, 0.24019607843137256, 0.23809523809523805, 0.2384259259259259, 0.24324324324324326, 0.25, 0.2606837606837607, 0.26458333333333334, 0.27235772357723587, 0.2738095238095238, 0.2751937984496124, 0.2784090909090909, 0.2777777777777778, 0.27898550724637683, 0.2819148936170214, 0.28298611111111105, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.36111111111111105, 0.3452380952380951, 0.3229166666666667, 0.3148148148148148, 0.30000000000000004, 0.30303030303030304, 0.2986111111111111, 0.2948717948717949, 0.2857142857142857, 0.28333333333333327, 0.28125, 0.28431372549019607, 0.287037037037037, 0.2894736842105263, 0.2916666666666667, 0.2936507936507936, 0.2803030303030303, 0.2826086956521739, 0.28124999999999994, 0.28, 0.27884615384615385, 0.28703703703703703, 0.28273809523809523, 0.2816091954022988, 0.2916666666666667, 0.28225806451612906, 0.2760416666666667, 0.2803030303030303, 0.28431372549019607, 0.28571428571428564, 0.28703703703703715, 0.2905405405405405, 0.28289473684210525, 0.2905982905982906, 0.28958333333333336, 0.290650406504065, 0.2936507936507936, 0.29069767441860467, 0.2916666666666667, 0.28888888888888886, 0.28804347826086957, 0.2836879432624113, 0.28298611111111105, 0.282312925170068, 0.2816666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.02380952380952381, 0.020833333333333332, 0.027777777777777776, 0.04166666666666653, 0.03787878787878788, 0.034722222222222224, 0.038461538461538464, 0.047619047619047616, 0.04444444444444444, 0.041666666666666664, 0.05392156862745098, 0.06481481481481495, 0.07456140350877193, 0.07916666666666666, 0.07936507936507949, 0.09090909090909091, 0.10507246376811594, 0.11458333333333331, 0.11666666666666667, 0.13141025641025642, 0.1419753086419753, 0.1488095238095238, 0.15517241379310343, 0.15, 0.15591397849462366, 0.15625, 0.16161616161616163, 0.16911764705882354, 0.18095238095238092, 0.18981481481481494, 0.19594594594594594, 0.19956140350877194, 0.2094017094017094, 0.21041666666666667, 0.2195121951219512, 0.22420634920634933, 0.23255813953488372, 0.24242424242424243, 0.2518518518518518, 0.25905797101449274, 0.2695035460992909, 0.2777777777777779, 0.27721088435374147, 0.2816666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.010416666666666666, 0.018518518518518517, 0.01666666666666667, 0.030303030303030304, 0.027777777777777776, 0.038461538461538464, 0.041666666666666664, 0.03888888888888888, 0.041666666666666664, 0.058823529411764705, 0.0648148148148148, 0.07456140350877193, 0.07916666666666666, 0.09126984126984139, 0.10227272727272728, 0.10869565217391304, 0.11111111111111109, 0.12333333333333334, 0.1282051282051282, 0.13271604938271606, 0.14285714285714285, 0.1408045977011494, 0.14444444444444443, 0.1424731182795699, 0.1484375, 0.15151515151515152, 0.1568627450980392, 0.16190476190476188, 0.17129629629629645, 0.17792792792792791, 0.18859649122807018, 0.19658119658119658, 0.20416666666666666, 0.2073170731707318, 0.21428571428571438, 0.22093023255813954, 0.23106060606060605, 0.2351851851851852, 0.24456521739130435, 0.25177304964539016, 0.2656250000000001, 0.27721088435374147, 0.2816666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.04166666666666653, 0.03787878787878788, 0.034722222222222224, 0.038461538461538464, 0.041666666666666664, 0.04444444444444444, 0.046875, 0.049019607843137254, 0.06481481481481495, 0.07456140350877193, 0.08333333333333333, 0.08333333333333345, 0.09090909090909091, 0.10869565217391304, 0.11458333333333331, 0.11666666666666667, 0.13141025641025642, 0.1388888888888889, 0.1488095238095238, 0.14942528735632202, 0.15, 0.15053763440860216, 0.15625, 0.15656565656565657, 0.16176470588235295, 0.17857142857142855, 0.18518518518518515, 0.18468468468468469, 0.19298245614035087, 0.19658119658119658, 0.20625, 0.20934959349593507, 0.21825396825396823, 0.22868217054263565, 0.23106060606060605, 0.23703703703703705, 0.24456521739130435, 0.25177304964539016, 0.263888888888889, 0.27380952380952384, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.13028169014084506, 0.2537037037037037], "coverage": [0.0, 0.0, 0.0, 0.0, 0.47333333333333333, 0.9]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5830942215022172, "basin/entropy": 0.4163291643213114, "basin/dispersion": 0.5237029613256634, "energy/mean": 0.39264679635909333, "energy/min": 0.39925040157058717, "energy/std": 0.5168522357528247, "curv/lmax_mean": 0.5929790359560125, "curv/lmax_best": 0.5272518842927553, "curv/trace_mean": 0.6412773376899737, "curv/trace_best": 0.6202446491577315, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6388473208034158, "dynamics/drop": 0.6872691827180494, "dynamics/residual": 0.5027938329740935}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [1, 0, 6, 0, 0, 7, 0, 0, 7, 0, 0, 3, 0, 0, 10, 0, 0, 17, 0, 380], "incorrect_counts": [0, 0, 3, 0, 0, 5, 0, 0, 10, 0, 0, 8, 0, 0, 11, 0, 0, 11, 0, 121]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [380, 0, 0, 0, 0, 17, 0, 0, 10, 0, 3, 7, 13, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [121, 0, 0, 0, 0, 11, 0, 0, 10, 0, 8, 10, 6, 1, 0, 0, 1, 0, 1, 0]}, "basin/dispersion": {"edges": [1.6070286588464016, 1.6272769139695735, 1.6475251690927455, 1.6677734242159175, 1.6880216793390894, 1.7082699344622614, 1.7285181895854331, 1.748766444708605, 1.769014699831777, 1.789262954954949, 1.809511210078121, 1.829759465201293, 1.850007720324465, 1.8702559754476369, 1.8905042305708089, 1.9107524856939806, 1.9310007408171526, 1.9512489959403245, 1.9714972510634965, 1.9917455061866685, 2.0119937613098404], "correct_counts": [1, 1, 5, 5, 15, 22, 20, 31, 44, 54, 51, 65, 43, 26, 14, 11, 11, 4, 5, 3], "incorrect_counts": [2, 0, 1, 3, 1, 14, 9, 18, 19, 20, 20, 17, 13, 13, 8, 5, 1, 3, 1, 1]}, "energy/mean": {"edges": [0.7167260174949964, 0.7730059100935857, 0.8292858026921749, 0.8855656952907642, 0.9418455878893535, 0.9981254804879427, 1.054405373086532, 1.1106852656851212, 1.1669651582837104, 1.2232450508822996, 1.2795249434808889, 1.335804836079478, 1.3920847286780673, 1.4483646212766565, 1.504644513875246, 1.5609244064738352, 1.6172042990724245, 1.6734841916710137, 1.7297640842696032, 1.7860439768681924, 1.8423238694667816], "correct_counts": [12, 33, 69, 58, 50, 36, 27, 17, 19, 17, 10, 9, 15, 11, 19, 9, 11, 4, 1, 4], "incorrect_counts": [2, 6, 6, 27, 20, 14, 6, 11, 8, 11, 11, 11, 8, 8, 7, 5, 4, 1, 3, 0]}, "energy/min": {"edges": [0.33279895782470703, 0.39212902784347536, 0.45145909786224364, 0.5107891678810119, 0.5701192378997803, 0.6294493079185486, 0.6887793779373169, 0.7481094479560852, 0.8074395179748535, 0.8667695879936218, 0.9260996580123901, 0.9854297280311585, 1.0447597980499268, 1.1040898680686952, 1.1634199380874635, 1.2227500081062317, 1.282080078125, 1.3414101481437684, 1.4007402181625366, 1.460070288181305, 1.5194003582000732], "correct_counts": [4, 17, 34, 53, 57, 47, 49, 24, 22, 14, 20, 16, 12, 13, 10, 15, 8, 8, 6, 2], "incorrect_counts": [1, 0, 11, 12, 15, 22, 9, 12, 10, 16, 8, 8, 15, 11, 5, 5, 5, 3, 0, 1]}, "energy/std": {"edges": [0.06888872658823433, 0.08357696314973125, 0.09826519971122816, 0.11295343627272506, 0.12764167283422195, 0.14232990939571888, 0.15701814595721578, 0.17170638251871267, 0.1863946190802096, 0.2010828556417065, 0.21577109220320342, 0.23045932876470032, 0.24514756532619722, 0.2598358018876942, 0.27452403844919104, 0.28921227501068797, 0.3039005115721849, 0.31858874813368177, 0.33327698469517864, 0.3479652212566756, 0.3626534578181725], "correct_counts": [1, 0, 4, 8, 21, 36, 52, 55, 56, 50, 45, 26, 23, 26, 14, 5, 2, 5, 1, 1], "incorrect_counts": [0, 0, 2, 5, 9, 17, 15, 24, 22, 17, 18, 17, 9, 4, 4, 3, 2, 1, 0, 0]}, "curv/lmax_mean": {"edges": [0.999078502257665, 0.9991983219981194, 0.9993181417385737, 0.9994379614790281, 0.9995577812194825, 0.9996776009599369, 0.9997974207003912, 0.9999172404408455, 1.0000370601812998, 1.0001568799217542, 1.0002766996622086, 1.000396519402663, 1.0005163391431173, 1.0006361588835717, 1.000755978624026, 1.0008757983644805, 1.0009956181049349, 1.001115437845389, 1.0012352575858434, 1.0013550773262978, 1.0014748970667522], "correct_counts": [3, 8, 21, 38, 61, 87, 93, 58, 24, 18, 10, 6, 2, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 5, 5, 21, 32, 48, 38, 10, 5, 3, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.995195746421814, 0.9959201753139496, 0.9966446042060852, 0.9973690330982208, 0.9980934619903564, 0.9988178908824921, 0.9995423197746277, 1.0002667486667634, 1.0009911775588989, 1.0017156064510346, 1.0024400353431702, 1.0031644642353057, 1.0038888931274415, 1.004613322019577, 1.0053377509117127, 1.0060621798038483, 1.0067866086959838, 1.0075110375881196, 1.008235466480255, 1.0089598953723908, 1.0096843242645264], "correct_counts": [2, 1, 2, 1, 13, 45, 337, 24, 3, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 1, 0, 1, 6, 26, 130, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [31.831074873606365, 31.84024980068207, 31.84942472775777, 31.858599654833476, 31.86777458190918, 31.876949508984886, 31.88612443606059, 31.895299363136292, 31.904474290211997, 31.913649217287702, 31.922824144363403, 31.931999071439108, 31.941173998514813, 31.950348925590518, 31.959523852666223, 31.968698779741924, 31.97787370681763, 31.987048633893334, 31.996223560969035, 32.00539848804474, 32.014573415120445], "correct_counts": [1, 4, 6, 12, 12, 19, 14, 18, 19, 15, 14, 9, 12, 16, 10, 25, 44, 90, 75, 16], "incorrect_counts": [1, 1, 4, 8, 7, 11, 15, 9, 8, 15, 11, 2, 3, 9, 9, 12, 12, 18, 11, 3]}, "curv/trace_best": {"edges": [31.66323471069336, 31.682111549377442, 31.700988388061525, 31.719865226745604, 31.738742065429687, 31.75761890411377, 31.776495742797852, 31.795372581481935, 31.814249420166014, 31.833126258850097, 31.85200309753418, 31.870879936218262, 31.889756774902345, 31.908633613586424, 31.927510452270507, 31.94638729095459, 31.965264129638673, 31.984140968322755, 32.003017807006835, 32.02189464569092, 32.040771484375], "correct_counts": [2, 0, 1, 0, 5, 4, 0, 5, 3, 10, 4, 19, 20, 21, 22, 43, 60, 148, 60, 4], "incorrect_counts": [0, 0, 0, 1, 2, 1, 0, 1, 2, 4, 8, 7, 12, 14, 24, 18, 32, 35, 8, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6349999999999999, 0.6424166666666666, 0.6498333333333333, 0.6572499999999999, 0.6646666666666666, 0.6720833333333334, 0.6795, 0.6869166666666666, 0.6943333333333334, 0.7017500000000001, 0.7091666666666667, 0.7165833333333333, 0.7240000000000001, 0.7314166666666668, 0.7388333333333335, 0.7462500000000001, 0.7536666666666668, 0.7610833333333336, 0.7685000000000002, 0.7759166666666668, 0.7833333333333335], "correct_counts": [1, 1, 1, 13, 22, 38, 53, 49, 83, 45, 40, 41, 19, 12, 4, 2, 4, 2, 0, 1], "incorrect_counts": [0, 0, 5, 7, 15, 13, 39, 25, 32, 15, 5, 10, 3, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/drop": {"edges": [14.843120073278746, 20.788069765021405, 26.73301945676406, 32.677969148506726, 38.62291884024938, 44.56786853199204, 50.512818223734705, 56.45776791547736, 62.40271760722002, 68.34766729896268, 74.29261699070533, 80.23756668244799, 86.18251637419066, 92.1274660659333, 98.07241575767597, 104.01736544941862, 109.96231514116128, 115.90726483290395, 121.8522145246466, 127.79716421638926, 133.74211390813193], "correct_counts": [50, 128, 104, 57, 41, 25, 11, 2, 3, 3, 3, 2, 1, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [48, 69, 30, 12, 7, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1786483426888783, 1.1918049529194832, 1.2049615631500878, 1.2181181733806927, 1.2312747836112976, 1.2444313938419023, 1.2575880040725071, 1.270744614303112, 1.283901224533717, 1.2970578347643216, 1.3102144449949265, 1.3233710552255313, 1.3365276654561362, 1.3496842756867409, 1.3628408859173458, 1.3759974961479506, 1.3891541063785553, 1.4023107166091602, 1.415467326839765, 1.4286239370703697, 1.4417805473009746], "correct_counts": [3, 1, 2, 11, 24, 23, 30, 42, 48, 52, 46, 40, 39, 31, 11, 11, 11, 4, 1, 1], "incorrect_counts": [2, 2, 2, 6, 8, 9, 10, 14, 22, 13, 22, 12, 20, 12, 7, 4, 3, 0, 0, 1]}}}, "ece_geometry": 0.06147257738915552}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "6b31bce04506", "timestamp": "2026-07-27T09:38:01.149599+00:00", "git_sha": "3d7ce49", "config_hash": "c6246ddaef7e", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.82, "base_error": 0.18, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0784767176926761, "rho_basin": 0.12751482332802438, "energy_min": 0.16347101991147492, "energy_mean": 0.1708695950340498, "energy_std": 0.1652682650135063, "msp": 0.07413344872937365, "temp_msp": 0.0954664480423542, "entropy": 0.07111874477317347}, "temperature": 0.370153554472176, "best_energy_baseline": "energy_min", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.08499430221879882, 0.05443909813421712, 0.11823769217310283], "delta_aurc_vs_best_energy": [0.08499430221879882, 0.05443909813421712, 0.11823769217310283], "delta_aurc_vs_best_baseline": [-0.007357972919502631, -0.027934250128952055, 0.012642038074947301], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.0625, 0.05555555555555555, 0.05833333333333334, 0.05303030303030303, 0.05555555555555555, 0.05128205128205128, 0.05357142857142857, 0.049999999999999996, 0.046875, 0.04411764705882353, 0.060185185185185175, 0.06140350877192982, 0.058333333333333334, 0.055555555555555546, 0.05303030303030303, 0.057971014492753624, 0.05902777777777777, 0.056666666666666664, 0.05448717948717949, 0.06481481481481481, 0.06845238095238096, 0.0718390804597701, 0.07222222222222222, 0.07795698924731183, 0.08333333333333333, 0.08585858585858586, 0.09558823529411764, 0.09523809523809522, 0.09722222222222221, 0.1036036036036036, 0.1074561403508772, 0.11538461538461539, 0.125, 0.12398373983739835, 0.12896825396825395, 0.13565891472868216, 0.14015151515151514, 0.14814814814814814, 0.15217391304347827, 0.15780141843971643, 0.16666666666666674, 0.1717687074829932, 0.18]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.12009237875288685, 0.12009237875288688, 0.12009237875288686, 0.12009237875288677, 0.12009237875288671, 0.12009237875288667, 0.12009237875288664, 0.12009237875288663, 0.1200923787528866, 0.1200923787528866, 0.12009237875288659, 0.12009237875288657, 0.12009237875288657, 0.12009237875288656, 0.12009237875288656, 0.12009237875288654, 0.12009237875288654, 0.12009237875288654, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.1200923787528865, 0.1200923787528865, 0.1233108108108105, 0.12664473684210495, 0.129807692307692, 0.1328124999999997, 0.13630999213217915, 0.14151305683563736, 0.14647411852963235, 0.14978590250329377, 0.1522544283413848, 0.15703227931488792, 0.16239522888459046, 0.16790674603174596, 0.17360020931449494, 0.1799999999999998]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.25, 0.2222222222222222, 0.1875, 0.16666666666666666, 0.16666666666666663, 0.1547619047619048, 0.15625, 0.16666666666666666, 0.15833333333333335, 0.1590909090909091, 0.1527777777777778, 0.15384615384615385, 0.15476190476190477, 0.14999999999999997, 0.15104166666666666, 0.16666666666666666, 0.162037037037037, 0.16228070175438597, 0.16666666666666666, 0.1587301587301587, 0.16287878787878787, 0.16666666666666666, 0.16319444444444453, 0.16666666666666666, 0.1762820512820513, 0.1728395061728395, 0.17857142857142858, 0.17816091954022986, 0.17777777777777778, 0.17204301075268819, 0.16666666666666666, 0.16161616161616163, 0.1568627450980392, 0.1571428571428571, 0.15509259259259256, 0.1554054054054054, 0.15789473684210525, 0.15598290598290598, 0.16041666666666668, 0.1605691056910569, 0.1587301587301587, 0.16279069767441862, 0.16477272727272727, 0.16666666666666666, 0.17028985507246377, 0.17198581560283685, 0.1736111111111112, 0.17857142857142858, 0.18]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.16666666666666666, 0.18333333333333332, 0.19444444444444442, 0.1904761904761903, 0.1875, 0.18518518518518517, 0.17500000000000002, 0.18181818181818182, 0.16666666666666666, 0.16666666666666666, 0.17261904761904762, 0.16666666666666663, 0.16666666666666666, 0.16176470588235295, 0.16666666666666663, 0.16666666666666666, 0.16666666666666666, 0.17460317460317457, 0.1856060606060606, 0.18478260869565216, 0.18402777777777776, 0.18, 0.18269230769230768, 0.17901234567901234, 0.17857142857142858, 0.17528735632183906, 0.17222222222222222, 0.1693548387096774, 0.16666666666666666, 0.16414141414141414, 0.15931372549019607, 0.15476190476190474, 0.15509259259259256, 0.15315315315315314, 0.15570175438596492, 0.15598290598290598, 0.15625, 0.15650406504065037, 0.1587301587301587, 0.16472868217054262, 0.16666666666666666, 0.1685185185185185, 0.17028985507246377, 0.1684397163120567, 0.17534722222222218, 0.17687074829931973, 0.18]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.1388888888888889, 0.16666666666666666, 0.15, 0.12499999999999999, 0.11904761904761907, 0.13541666666666666, 0.12037037037037036, 0.11666666666666668, 0.12121212121212122, 0.14583333333333334, 0.15384615384615385, 0.16666666666666666, 0.1611111111111111, 0.15104166666666666, 0.1568627450980392, 0.15740740740740738, 0.16228070175438597, 0.1625, 0.16269841269841268, 0.17045454545454544, 0.17391304347826086, 0.17361111111111108, 0.17666666666666667, 0.1794871794871795, 0.1728395061728395, 0.16964285714285715, 0.16666666666666663, 0.16666666666666666, 0.17473118279569894, 0.171875, 0.17424242424242425, 0.1715686274509804, 0.16904761904761903, 0.16898148148148145, 0.17117117117117117, 0.17105263157894737, 0.17307692307692307, 0.175, 0.1768292682926829, 0.1746031746031747, 0.17635658914728683, 0.17234848484848486, 0.17407407407407408, 0.1721014492753623, 0.17375886524822692, 0.17708333333333331, 0.17687074829931973, 0.18]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.04761904761904762, 0.041666666666666664, 0.037037037037037035, 0.04166666666666667, 0.03787878787878788, 0.041666666666666664, 0.038461538461538464, 0.041666666666666664, 0.03888888888888888, 0.041666666666666664, 0.04411764705882353, 0.04629629629629629, 0.04824561403508772, 0.05, 0.05158730158730158, 0.05303030303030303, 0.050724637681159424, 0.055555555555555546, 0.06333333333333334, 0.0641025641025641, 0.06172839506172839, 0.0625, 0.06321839080459789, 0.06944444444444445, 0.07526881720430108, 0.08072916666666667, 0.08585858585858586, 0.09313725490196079, 0.0976190476190476, 0.10185185185185183, 0.10585585585585586, 0.1118421052631579, 0.11538461538461539, 0.11875, 0.12398373983739835, 0.12896825396825395, 0.12790697674418605, 0.13446969696969696, 0.1425925925925926, 0.1431159420289855, 0.14893617021276606, 0.1562500000000001, 0.16666666666666666, 0.18]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.0625, 0.05, 0.055555555555555546, 0.04761904761904762, 0.0625, 0.05555555555555555, 0.05833333333333334, 0.09090909090909091, 0.08333333333333333, 0.07692307692307693, 0.07738095238095238, 0.07777777777777777, 0.08333333333333333, 0.08823529411764706, 0.08333333333333331, 0.07894736842105263, 0.075, 0.07142857142857141, 0.07575757575757576, 0.07971014492753623, 0.08680555555555554, 0.08666666666666667, 0.08333333333333333, 0.08950617283950617, 0.08928571428571429, 0.09482758620689653, 0.1, 0.09946236559139784, 0.09895833333333333, 0.10353535353535354, 0.10049019607843138, 0.10476190476190475, 0.10648148148148147, 0.11036036036036036, 0.11403508771929824, 0.11538461538461539, 0.12083333333333333, 0.1219512195121951, 0.13095238095238093, 0.13372093023255813, 0.14393939393939395, 0.15185185185185185, 0.1539855072463768, 0.15780141843971643, 0.16666666666666663, 0.1717687074829932, 0.18]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.041666666666666664, 0.04487179487179487, 0.05357142857142857, 0.055555555555555546, 0.052083333333333336, 0.05392156862745098, 0.055555555555555546, 0.05701754385964912, 0.0625, 0.06746031746031744, 0.06818181818181818, 0.06521739130434782, 0.06250000000000011, 0.07, 0.0673076923076923, 0.06481481481481481, 0.06547619047619048, 0.0689655172413793, 0.06666666666666667, 0.06989247311827956, 0.06770833333333333, 0.06565656565656566, 0.06862745098039216, 0.06904761904761904, 0.07638888888888888, 0.08108108108108109, 0.08333333333333333, 0.09188034188034189, 0.09791666666666667, 0.1138211382113821, 0.11706349206349217, 0.12596899224806202, 0.13636363636363635, 0.14444444444444443, 0.1503623188405797, 0.15602836879432636, 0.15798611111111108, 0.1683673469387755, 0.18]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.17016647548003197, "coverage": 0.6816666666666666, "abstain_rate": 0.31833333333333336, "selective_risk": 0.09535452322738386, "selective_accuracy": 0.9046454767726162, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.09535452322738386, 0.12896825396825398, 0.17147707979626486, 0.18030050083472454], "coverage": [0.0, 0.0, 0.6816666666666666, 0.84, 0.9816666666666667, 0.9983333333333333]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.657397997591087, "basin/entropy": 0.3412375790424571, "basin/dispersion": 0.5489686841312857, "energy/mean": 0.4601588376994881, "energy/min": 0.45007151460403494, "energy/std": 0.4628124059018368, "curv/lmax_mean": 0.4898562180066245, "curv/lmax_best": 0.4666892502258356, "curv/trace_mean": 0.4625301114122252, "curv/trace_best": 0.4725986148750376, "dynamics/steps": 0.5, "dynamics/monotonic": 0.3500639867509786, "dynamics/drop": 0.3683566696778079, "dynamics/residual": 0.549439174947305}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [1, 0, 0, 0, 0, 6, 0, 12, 0, 13, 0, 0, 17, 0, 20, 0, 0, 42, 0, 381], "incorrect_counts": [0, 0, 0, 0, 0, 7, 0, 9, 0, 9, 0, 0, 6, 0, 11, 0, 0, 14, 0, 52]}, "basin/entropy": {"edges": [0.0, 0.06789889274936621, 0.13579778549873242, 0.20369667824809862, 0.27159557099746484, 0.33949446374683107, 0.40739335649619723, 0.47529224924556346, 0.5431911419949297, 0.6110900347442959, 0.6789889274936621, 0.7468878202430284, 0.8147867129923945, 0.8826856057417607, 0.9505844984911269, 1.0184833912404931, 1.0863822839898594, 1.1542811767392256, 1.2221800694885918, 1.290078962237958, 1.3579778549873243], "correct_counts": [381, 0, 0, 0, 42, 0, 20, 0, 16, 13, 17, 0, 0, 1, 0, 1, 0, 0, 0, 1], "incorrect_counts": [52, 0, 0, 0, 14, 0, 10, 0, 5, 9, 15, 0, 0, 0, 2, 1, 0, 0, 0, 0]}, "basin/dispersion": {"edges": [1.6075309204036172, 1.6282759139338974, 1.6490209074641777, 1.6697659009944579, 1.690510894524738, 1.7112558880550184, 1.7320008815852985, 1.7527458751155787, 1.773490868645859, 1.7942358621761392, 1.8149808557064193, 1.8357258492366997, 1.8564708427669798, 1.87721583629726, 1.8979608298275403, 1.9187058233578205, 1.9394508168881006, 1.960195810418381, 1.9809408039486611, 2.0016857974789413, 2.0224307910092216], "correct_counts": [1, 1, 4, 3, 13, 17, 25, 42, 40, 69, 70, 48, 50, 34, 27, 22, 12, 9, 3, 2], "incorrect_counts": [0, 1, 1, 0, 2, 4, 11, 8, 18, 13, 9, 10, 8, 13, 5, 3, 2, 0, 0, 0]}, "energy/mean": {"edges": [-0.2607540686925252, -0.20895610700050987, -0.15715814530849456, -0.10536018361647925, -0.053562221924463915, -0.0017642602324485779, 0.050033701459566704, 0.10183166315158204, 0.15362962484359738, 0.20542758653561272, 0.25722554822762805, 0.3090235099196434, 0.3608214716116586, 0.41261943330367395, 0.4644173949956893, 0.5162153566877046, 0.56801331837972, 0.6198112800717352, 0.6716092417637507, 0.7234072034557659, 0.7752051651477814], "correct_counts": [11, 39, 91, 75, 69, 24, 21, 10, 17, 13, 9, 15, 8, 15, 15, 13, 13, 16, 10, 8], "incorrect_counts": [2, 10, 17, 21, 11, 3, 0, 1, 1, 1, 2, 4, 2, 2, 9, 3, 5, 5, 5, 4]}, "energy/min": {"edges": [-0.6984671354293823, -0.635711333155632, -0.5729555308818817, -0.5101997286081315, -0.4474439263343811, -0.3846881240606308, -0.3219323217868805, -0.25917651951313025, -0.19642071723937993, -0.1336649149656296, -0.07090911269187927, -0.008153310418129056, 0.05460249185562127, 0.1173582941293716, 0.18011409640312182, 0.24286989867687214, 0.30562570095062247, 0.3683815032243727, 0.4311373054981231, 0.49389310777187334, 0.5566489100456238], "correct_counts": [2, 4, 26, 59, 81, 84, 40, 30, 21, 19, 14, 15, 14, 20, 14, 20, 12, 8, 7, 2], "incorrect_counts": [0, 0, 9, 10, 15, 19, 11, 0, 0, 3, 3, 4, 4, 2, 9, 6, 5, 5, 1, 2]}, "energy/std": {"edges": [0.07567243770358024, 0.09125026233767754, 0.10682808697177484, 0.12240591160587214, 0.13798373623996943, 0.15356156087406675, 0.16913938550816404, 0.18471721014226133, 0.20029503477635863, 0.21587285941045592, 0.2314506840445532, 0.2470285086786505, 0.2626063333127478, 0.2781841579468451, 0.29376198258094244, 0.30933980721503973, 0.324917631849137, 0.3404954564832343, 0.3560732811173316, 0.3716511057514289, 0.3872289303855262], "correct_counts": [2, 3, 17, 20, 45, 54, 69, 72, 55, 52, 34, 33, 21, 9, 2, 2, 1, 0, 0, 1], "incorrect_counts": [0, 3, 0, 6, 4, 16, 11, 18, 12, 12, 9, 4, 8, 1, 1, 1, 1, 0, 1, 0]}, "curv/lmax_mean": {"edges": [1.0000369946161907, 1.0007940803964934, 1.001551166176796, 1.0023082519570987, 1.0030653377374015, 1.003822423517704, 1.0045795092980068, 1.0053365950783095, 1.006093680858612, 1.0068507666389148, 1.0076078524192176, 1.0083649381995201, 1.0091220239798229, 1.0098791097601254, 1.0106361955404282, 1.011393281320731, 1.0121503671010335, 1.0129074528813362, 1.013664538661639, 1.0144216244419415, 1.0151787102222443], "correct_counts": [255, 60, 35, 22, 22, 20, 20, 14, 13, 10, 9, 1, 2, 2, 2, 1, 3, 0, 0, 1], "incorrect_counts": [51, 28, 7, 9, 2, 5, 1, 2, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0017322301864624, 1.0034644603729248, 1.0051966905593872, 1.0069289207458496, 1.008661150932312, 1.0103933811187744, 1.0121256113052368, 1.0138578414916992, 1.0155900716781616, 1.017322301864624, 1.0190545320510864, 1.0207867622375488, 1.0225189924240112, 1.0242512226104736, 1.025983452796936, 1.0277156829833984, 1.0294479131698608, 1.0311801433563232, 1.0329123735427856, 1.034644603729248], "correct_counts": [373, 47, 26, 9, 6, 6, 3, 6, 6, 3, 2, 2, 0, 0, 2, 0, 0, 0, 0, 1], "incorrect_counts": [84, 12, 6, 2, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00349839528402, 32.00748820304871, 32.0114780108134, 32.01546781857809, 32.01945762634278, 32.02344743410747, 32.02743724187215, 32.03142704963684, 32.03541685740153, 32.03940666516622, 32.04339647293091, 32.0473862806956, 32.05137608846029, 32.05536589622498, 32.05935570398967, 32.06334551175435, 32.06733531951904, 32.07132512728373, 32.07531493504842, 32.07930474281311, 32.0832945505778], "correct_counts": [74, 97, 58, 30, 24, 24, 20, 19, 19, 16, 18, 13, 24, 9, 16, 9, 8, 7, 6, 1], "incorrect_counts": [3, 11, 28, 16, 9, 10, 4, 3, 4, 6, 2, 0, 0, 2, 3, 0, 4, 2, 0, 1]}, "curv/trace_best": {"edges": [32.0009880065918, 32.010338592529294, 32.0196891784668, 32.029039764404295, 32.0383903503418, 32.0477409362793, 32.057091522216794, 32.0664421081543, 32.075792694091795, 32.0851432800293, 32.0944938659668, 32.103844451904294, 32.1131950378418, 32.122545623779295, 32.1318962097168, 32.1412467956543, 32.150597381591794, 32.1599479675293, 32.169298553466795, 32.1786491394043, 32.1879997253418], "correct_counts": [159, 110, 61, 42, 43, 24, 16, 11, 6, 7, 3, 2, 2, 0, 2, 1, 1, 1, 0, 1], "incorrect_counts": [23, 33, 20, 12, 5, 1, 3, 1, 3, 3, 0, 2, 0, 1, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [492, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7333333333333334, 0.7383333333333334, 0.7433333333333334, 0.7483333333333334, 0.7533333333333334, 0.7583333333333334, 0.7633333333333334, 0.7683333333333334, 0.7733333333333334, 0.7783333333333334, 0.7833333333333334, 0.7883333333333334, 0.7933333333333334, 0.7983333333333335, 0.8033333333333335, 0.8083333333333335, 0.8133333333333335, 0.8183333333333335, 0.8233333333333335, 0.8283333333333335, 0.8333333333333335], "correct_counts": [4, 18, 16, 37, 64, 65, 60, 89, 54, 37, 23, 15, 6, 2, 1, 0, 1, 0, 0, 0], "incorrect_counts": [1, 3, 3, 7, 5, 9, 9, 13, 14, 13, 10, 6, 2, 4, 2, 2, 1, 2, 1, 1]}, "dynamics/drop": {"edges": [74.99134453634422, 87.32837261632085, 99.6654006962975, 112.00242877627413, 124.33945685625076, 136.67648493622738, 149.01351301620406, 161.35054109618068, 173.6875691761573, 186.02459725613394, 198.3616253361106, 210.6986534160872, 223.0356814960639, 235.3727095760405, 247.70973765601713, 260.04676573599374, 272.3837938159704, 284.72082189594704, 297.05784997592366, 309.39487805590034, 321.73190613587695], "correct_counts": [16, 27, 71, 89, 118, 93, 44, 17, 11, 1, 2, 1, 2, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [1, 7, 16, 10, 15, 12, 15, 7, 5, 3, 1, 1, 4, 3, 3, 1, 0, 3, 0, 1]}, "dynamics/residual": {"edges": [1.1565301517645519, 1.1707259138425192, 1.1849216759204866, 1.199117437998454, 1.213313200076421, 1.2275089621543884, 1.2417047242323558, 1.2559004863103231, 1.2700962483882905, 1.2842920104662578, 1.2984877725442252, 1.3126835346221923, 1.3268792967001597, 1.341075058778127, 1.3552708208560944, 1.3694665829340618, 1.3836623450120291, 1.3978581070899962, 1.4120538691679636, 1.426249631245931, 1.4404453933238983], "correct_counts": [2, 3, 8, 6, 16, 35, 42, 37, 52, 52, 62, 48, 46, 36, 22, 16, 6, 1, 0, 2], "incorrect_counts": [1, 0, 0, 1, 4, 7, 14, 15, 8, 14, 9, 15, 11, 7, 1, 0, 1, 0, 0, 0]}}}, "ece_geometry": 0.05242852914544991}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "14e97e539898", "timestamp": "2026-07-27T09:38:03.912065+00:00", "git_sha": "3d7ce49", "config_hash": "e65ef9e88f92", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.8066666666666666, "base_error": 0.19333333333333333, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.07459630695723383, "rho_basin": 0.09995473471609245, "energy_min": 0.18564962420122708, "energy_mean": 0.18679968604822153, "energy_std": 0.1962536701726144, "msp": 0.05417632514228234, "temp_msp": 0.05699145581322151, "entropy": 0.07471040623687418}, "temperature": 0.40914505573299365, "best_energy_baseline": "energy_min", "best_baseline": "msp", "delta_aurc_vs_energy_min": [0.11105331724399324, 0.07109286586589932, 0.149723039243916], "delta_aurc_vs_best_energy": [0.11105331724399324, 0.07109286586589932, 0.149723039243916], "delta_aurc_vs_best_baseline": [-0.020419981814951492, -0.03976815367328431, -0.002945817690407804], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.0625, 0.05, 0.04166666666666666, 0.04761904761904762, 0.07291666666666667, 0.06481481481481481, 0.06666666666666668, 0.06060606060606061, 0.0625, 0.057692307692307696, 0.05357142857142857, 0.055555555555555546, 0.052083333333333336, 0.05392156862745098, 0.055555555555555546, 0.05263157894736842, 0.05, 0.04761904761904761, 0.04924242424242424, 0.04710144927536232, 0.048611111111111105, 0.05, 0.04807692307692308, 0.05555555555555555, 0.05654761904761905, 0.060344827586206885, 0.06111111111111111, 0.06182795698924731, 0.0625, 0.06565656565656566, 0.06862745098039216, 0.07142857142857141, 0.07407407407407421, 0.08333333333333333, 0.08552631578947369, 0.09188034188034189, 0.1, 0.1097560975609756, 0.11706349206349205, 0.12209302325581395, 0.12878787878787878, 0.13703703703703704, 0.15217391304347827, 0.1631205673758865, 0.17361111111111108, 0.18197278911564627, 0.19333333333333333]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0845410628019324, 0.08454106280193235, 0.08454106280193234, 0.08454106280193234, 0.08454106280193233, 0.08454106280193233, 0.08454106280193233, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08823129251700677, 0.09530423280423275, 0.10199485199485187, 0.10833333333333317, 0.11434676434676416, 0.1200595238095236, 0.12340301974448298, 0.1255668934240361, 0.1296603912882981, 0.13753607503607498, 0.147962962962963, 0.15996376811594212, 0.16849691146190815, 0.17641129032258074, 0.18435374149659864, 0.19333333333333322]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.19444444444444445, 0.14583333333333334, 0.13333333333333333, 0.15277777777777776, 0.1547619047619048, 0.16666666666666666, 0.1574074074074074, 0.16666666666666655, 0.19696969696969696, 0.1875, 0.1858974358974359, 0.19642857142857142, 0.19444444444444442, 0.19270833333333334, 0.18627450980392157, 0.18518518518518515, 0.17543859649122806, 0.1875, 0.19444444444444442, 0.19696969696969696, 0.19202898550724637, 0.18749999999999997, 0.18333333333333332, 0.1794871794871795, 0.17592592592592593, 0.17857142857142858, 0.17241379310344845, 0.175, 0.17473118279569894, 0.17447916666666666, 0.18181818181818182, 0.17647058823529413, 0.1714285714285714, 0.17361111111111108, 0.17567567567567569, 0.18201754385964913, 0.18376068376068377, 0.18333333333333332, 0.18495934959349591, 0.1884920634920636, 0.187984496124031, 0.1875, 0.18888888888888888, 0.19202898550724637, 0.19148936170212763, 0.19270833333333331, 0.1921768707482993, 0.19333333333333333]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.20833333333333334, 0.2, 0.18055555555555552, 0.2023809523809524, 0.20833333333333334, 0.2037037037037037, 0.2166666666666667, 0.20454545454545456, 0.19444444444444445, 0.19230769230769232, 0.19642857142857142, 0.1833333333333333, 0.18229166666666666, 0.18627450980392157, 0.1898148148148148, 0.19298245614035087, 0.1875, 0.18650793650793648, 0.1893939393939394, 0.18840579710144928, 0.18402777777777776, 0.18, 0.1794871794871795, 0.18209876543209877, 0.18154761904761904, 0.18390804597701146, 0.18055555555555555, 0.18010752688172044, 0.17708333333333334, 0.17424242424242425, 0.17892156862745098, 0.18333333333333346, 0.18518518518518515, 0.18693693693693694, 0.18859649122807018, 0.1858974358974359, 0.18541666666666667, 0.18699186991869915, 0.1845238095238095, 0.18023255813953487, 0.18371212121212122, 0.18703703703703703, 0.19021739130434784, 0.1932624113475177, 0.19270833333333331, 0.19387755102040816, 0.19333333333333333]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.1111111111111111, 0.22916666666666666, 0.2, 0.2222222222222223, 0.23809523809523814, 0.22916666666666666, 0.2037037037037037, 0.19166666666666657, 0.17424242424242425, 0.1875, 0.20512820512820512, 0.19047619047619047, 0.1944444444444446, 0.203125, 0.20588235294117646, 0.20370370370370366, 0.19736842105263158, 0.1875, 0.19841269841269837, 0.19318181818181818, 0.2028985507246377, 0.20486111111111108, 0.20666666666666667, 0.20192307692307693, 0.20987654320987653, 0.20535714285714285, 0.2126436781609195, 0.20833333333333334, 0.21236559139784947, 0.20833333333333334, 0.20959595959595959, 0.2107843137254902, 0.20714285714285713, 0.20370370370370366, 0.20045045045045046, 0.19736842105263158, 0.19658119658119658, 0.19583333333333333, 0.1930894308943089, 0.19444444444444442, 0.1937984496124031, 0.19318181818181818, 0.18888888888888888, 0.19021739130434784, 0.18794326241134748, 0.18749999999999997, 0.1921768707482993, 0.19333333333333333]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.010416666666666666, 0.00980392156862745, 0.013888888888888886, 0.013157894736842105, 0.0125, 0.011904761904761902, 0.015151515151515152, 0.018115942028985508, 0.020833333333333447, 0.023333333333333334, 0.02564102564102564, 0.027777777777777776, 0.03571428571428571, 0.04022988505747126, 0.044444444444444446, 0.051075268817204304, 0.0625, 0.06818181818181818, 0.0784313725490196, 0.09047619047619046, 0.09490740740740755, 0.1036036036036036, 0.10964912280701754, 0.1111111111111111, 0.11666666666666667, 0.1219512195121951, 0.1250000000000001, 0.14147286821705427, 0.14772727272727273, 0.1537037037037037, 0.16304347826086957, 0.17021276595744692, 0.1753472222222223, 0.18197278911564627, 0.19333333333333333]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761904, 0.01111111111111111, 0.020833333333333332, 0.024509803921568627, 0.023148148148148143, 0.021929824561403508, 0.020833333333333332, 0.027777777777777773, 0.030303030303030304, 0.028985507246376812, 0.03472222222222222, 0.04, 0.04487179487179487, 0.043209876543209874, 0.050595238095238096, 0.05747126436781608, 0.058333333333333334, 0.056451612903225805, 0.0703125, 0.07575757575757576, 0.08333333333333333, 0.0857142857142857, 0.09027777777777776, 0.0945945945945946, 0.09868421052631579, 0.10683760683760683, 0.11041666666666666, 0.1158536585365855, 0.12499999999999999, 0.1298449612403101, 0.13825757575757575, 0.1425925925925926, 0.15217391304347827, 0.16489361702127656, 0.17361111111111108, 0.1836734693877551, 0.19333333333333333]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.025000000000000005, 0.030303030303030304, 0.027777777777777776, 0.038461538461538464, 0.041666666666666664, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.04166666666666666, 0.039473684210526314, 0.04583333333333333, 0.04761904761904761, 0.045454545454545456, 0.04710144927536232, 0.055555555555555663, 0.06333333333333334, 0.060897435897435896, 0.06481481481481481, 0.06845238095238096, 0.07758620689655171, 0.08611111111111111, 0.09408602150537634, 0.09895833333333333, 0.09848484848484848, 0.10049019607843138, 0.10476190476190475, 0.10879629629629628, 0.11261261261261261, 0.1206140350877193, 0.12606837606837606, 0.13125, 0.14024390243902438, 0.14484126984126997, 0.1511627906976744, 0.1571969696969697, 0.1648148148148148, 0.16666666666666666, 0.17021276595744692, 0.17708333333333343, 0.1836734693877551, 0.19333333333333333]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.10636963445059894, "coverage": 0.505, "abstain_rate": 0.495, "selective_risk": 0.04950495049504951, "selective_accuracy": 0.9504950495049505, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.04950495049504951, 0.08993576017130621, 0.13543599257884972, 0.19333333333333333], "coverage": [0.0, 0.0, 0.505, 0.7783333333333333, 0.8983333333333333, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7604641635793673, "basin/entropy": 0.23897477913935594, "basin/dispersion": 0.48822670276432034, "energy/mean": 0.47834140780849244, "energy/min": 0.4678861499002565, "energy/std": 0.5175797948133372, "curv/lmax_mean": 0.5627315474494158, "curv/lmax_best": 0.5241967084639498, "curv/trace_mean": 0.5586972784269022, "curv/trace_best": 0.5594097321174124, "dynamics/steps": 0.5, "dynamics/monotonic": 0.38306853804502705, "dynamics/drop": 0.39514462809917356, "dynamics/residual": 0.4876567398119122}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [0, 0, 6, 0, 0, 14, 0, 0, 6, 0, 0, 11, 0, 0, 22, 0, 0, 46, 0, 379], "incorrect_counts": [1, 0, 9, 0, 0, 17, 0, 0, 14, 0, 0, 10, 0, 0, 6, 0, 0, 24, 0, 35]}, "basin/entropy": {"edges": [0.0, 0.05387781635334003, 0.10775563270668007, 0.16163344906002008, 0.21551126541336013, 0.2693890817667002, 0.32326689812004017, 0.3771447144733802, 0.43102253082672026, 0.4849003471800603, 0.5387781635334004, 0.5926559798867403, 0.6465337962400803, 0.7004116125934204, 0.7542894289467604, 0.8081672453001005, 0.8620450616534405, 0.9159228780067805, 0.9698006943601206, 1.0236785107134607, 1.0775563270668007], "correct_counts": [379, 0, 0, 0, 0, 46, 0, 0, 22, 0, 11, 6, 18, 0, 0, 0, 0, 2, 0, 0], "incorrect_counts": [35, 0, 0, 0, 0, 24, 0, 0, 6, 0, 10, 13, 22, 0, 0, 0, 2, 2, 0, 2]}, "basin/dispersion": {"edges": [1.6209795073409872, 1.6420014502019364, 1.663023393062886, 1.6840453359238352, 1.7050672787847847, 1.726089221645734, 1.7471111645066832, 1.7681331073676327, 1.789155050228582, 1.8101769930895315, 1.8311989359504808, 1.8522208788114303, 1.8732428216723795, 1.8942647645333288, 1.9152867073942783, 1.9363086502552276, 1.9573305931161769, 1.9783525359771263, 1.9993744788380756, 2.020396421699025, 2.0414183645599744], "correct_counts": [0, 4, 7, 7, 25, 26, 39, 52, 59, 48, 51, 51, 44, 28, 14, 18, 3, 7, 0, 1], "incorrect_counts": [1, 1, 0, 2, 8, 5, 6, 13, 12, 10, 20, 12, 7, 9, 3, 2, 1, 2, 1, 1]}, "energy/mean": {"edges": [-0.2875423381725947, -0.26760066399971644, -0.2476589898268382, -0.22771731565395992, -0.20777564148108166, -0.1878339673082034, -0.1678922931353251, -0.14795061896244685, -0.1280089447895686, -0.10806727061669033, -0.08812559644381207, -0.06818392227093378, -0.048242248098055523, -0.028300573925177264, -0.008358899752298976, 0.011582774420579256, 0.031524448593457544, 0.05146612276633583, 0.07140779693921406, 0.09134947111209235, 0.1112911452849706], "correct_counts": [2, 6, 7, 14, 26, 29, 50, 53, 58, 39, 53, 43, 31, 22, 17, 11, 8, 9, 2, 4], "incorrect_counts": [0, 1, 2, 4, 6, 9, 10, 12, 10, 9, 10, 13, 7, 4, 10, 3, 1, 3, 1, 1]}, "energy/min": {"edges": [-0.6702444553375244, -0.6411247402429581, -0.6120050251483917, -0.5828853100538254, -0.553765594959259, -0.5246458798646927, -0.49552616477012634, -0.46640644967556, -0.43728673458099365, -0.4081670194864273, -0.37904730439186096, -0.3499275892972946, -0.32080787420272827, -0.2916881591081619, -0.2625684440135956, -0.23344872891902924, -0.2043290138244629, -0.17520929872989655, -0.1460895836353302, -0.11696986854076385, -0.08785015344619751], "correct_counts": [0, 10, 6, 12, 28, 30, 32, 58, 62, 59, 54, 41, 27, 26, 17, 14, 3, 3, 1, 1], "incorrect_counts": [2, 0, 3, 2, 1, 9, 10, 12, 16, 8, 11, 14, 9, 9, 4, 3, 0, 2, 1, 0]}, "energy/std": {"edges": [0.0869744242281087, 0.10181870487610015, 0.1166629855240916, 0.13150726617208308, 0.1463515468200745, 0.161195827468066, 0.17604010811605744, 0.1908843887640489, 0.20572866941204035, 0.2205729500600318, 0.23541723070802326, 0.2502615113560147, 0.2651057920040062, 0.2799500726519977, 0.2947943532999891, 0.3096386339479805, 0.324482914595972, 0.3393271952439635, 0.3541714758919549, 0.3690157565399464, 0.3838600371879378], "correct_counts": [4, 8, 20, 26, 58, 54, 66, 59, 56, 35, 34, 35, 14, 6, 0, 7, 1, 0, 0, 1], "incorrect_counts": [1, 1, 4, 11, 10, 17, 18, 17, 9, 6, 8, 3, 3, 6, 1, 0, 0, 1, 0, 0]}, "curv/lmax_mean": {"edges": [1.0000324149926503, 1.00056377997001, 1.0010951449473697, 1.0016265099247297, 1.0021578749020894, 1.0026892398794491, 1.0032206048568089, 1.0037519698341686, 1.0042833348115285, 1.0048146997888883, 1.005346064766248, 1.0058774297436077, 1.0064087947209674, 1.0069401596983274, 1.0074715246756871, 1.0080028896530469, 1.0085342546304066, 1.0090656196077663, 1.0095969845851263, 1.010128349562486, 1.0106597145398457], "correct_counts": [205, 99, 55, 28, 29, 16, 16, 10, 6, 6, 3, 1, 5, 3, 0, 0, 1, 0, 0, 1], "incorrect_counts": [62, 19, 11, 8, 9, 0, 4, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0011777698993682, 1.0023555397987365, 1.003533309698105, 1.0047110795974732, 1.0058888494968414, 1.0070666193962097, 1.008244389295578, 1.0094221591949464, 1.0105999290943146, 1.0117776989936829, 1.012955468893051, 1.0141332387924193, 1.0153110086917878, 1.016488778591156, 1.0176665484905243, 1.0188443183898925, 1.0200220882892608, 1.0211998581886292, 1.0223776280879975, 1.0235553979873657], "correct_counts": [346, 63, 26, 17, 10, 3, 3, 3, 3, 2, 1, 0, 3, 1, 0, 0, 0, 2, 0, 1], "incorrect_counts": [83, 18, 9, 2, 0, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00225226084391, 32.00583710670471, 32.00942195256551, 32.01300679842631, 32.01659164428711, 32.020176490147904, 32.023761336008704, 32.027346181869504, 32.030931027730304, 32.034515873591104, 32.038100719451904, 32.041685565312704, 32.045270411173504, 32.048855257034305, 32.052440102895105, 32.056024948755905, 32.0596097946167, 32.0631946404775, 32.0667794863383, 32.0703643321991, 32.0739491780599], "correct_counts": [49, 83, 57, 41, 41, 34, 27, 29, 23, 22, 23, 16, 13, 6, 7, 6, 4, 2, 0, 1], "incorrect_counts": [11, 30, 16, 14, 4, 7, 7, 4, 6, 6, 4, 2, 1, 2, 1, 0, 0, 1, 0, 0]}, "curv/trace_best": {"edges": [32.00101089477539, 32.006920433044435, 32.01282997131348, 32.01873950958252, 32.02464904785156, 32.030558586120605, 32.03646812438965, 32.042377662658694, 32.04828720092773, 32.054196739196776, 32.06010627746582, 32.066015815734865, 32.07192535400391, 32.077834892272946, 32.08374443054199, 32.089653968811035, 32.09556350708008, 32.101473045349124, 32.10738258361816, 32.113292121887206, 32.11920166015625], "correct_counts": [114, 104, 69, 52, 43, 25, 18, 9, 11, 10, 7, 9, 1, 2, 1, 2, 2, 2, 1, 2], "incorrect_counts": [32, 30, 22, 11, 6, 6, 2, 0, 1, 2, 0, 2, 1, 0, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [484, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [116, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7266666666666666, 0.7315833333333333, 0.7364999999999999, 0.7414166666666666, 0.7463333333333333, 0.75125, 0.7561666666666667, 0.7610833333333333, 0.766, 0.7709166666666667, 0.7758333333333334, 0.78075, 0.7856666666666666, 0.7905833333333333, 0.7955, 0.8004166666666667, 0.8053333333333333, 0.81025, 0.8151666666666667, 0.8200833333333334, 0.8250000000000001], "correct_counts": [1, 5, 12, 12, 33, 46, 59, 70, 69, 48, 51, 37, 18, 11, 5, 1, 3, 2, 1, 0], "incorrect_counts": [1, 0, 2, 4, 3, 6, 18, 6, 13, 13, 7, 10, 12, 7, 5, 3, 4, 1, 0, 1]}, "dynamics/drop": {"edges": [68.1073338240385, 81.69682680144906, 95.28631977885962, 108.87581275627016, 122.46530573368072, 136.05479871109128, 149.64429168850182, 163.23378466591237, 176.82327764332294, 190.4127706207335, 204.00226359814405, 217.5917565755546, 231.18124955296514, 244.7707425303757, 258.36023550778623, 271.94972848519683, 285.5392214626074, 299.1287144400179, 312.7182074174285, 326.30770039483906, 339.8971933722496], "correct_counts": [13, 35, 94, 106, 104, 50, 32, 14, 18, 8, 5, 0, 2, 0, 2, 0, 1, 0, 0, 0], "incorrect_counts": [1, 7, 15, 20, 24, 12, 9, 4, 2, 3, 3, 0, 4, 4, 3, 1, 1, 1, 1, 1]}, "dynamics/residual": {"edges": [1.158554623524348, 1.174299425383409, 1.1900442272424698, 1.2057890291015307, 1.2215338309605916, 1.2372786328196526, 1.2530234346787135, 1.2687682365377744, 1.2845130383968353, 1.3002578402558962, 1.3160026421149573, 1.331747443974018, 1.3474922458330791, 1.36323704769214, 1.378981849551201, 1.3947266514102619, 1.4104714532693228, 1.4262162551283837, 1.4419610569874446, 1.4577058588465055, 1.4734506607055664], "correct_counts": [2, 5, 7, 19, 34, 40, 58, 60, 49, 71, 50, 37, 30, 9, 8, 2, 2, 0, 0, 1], "incorrect_counts": [0, 0, 1, 5, 8, 7, 14, 14, 14, 23, 10, 7, 6, 2, 3, 1, 0, 1, 0, 0]}}}, "ece_geometry": 0.037362225019872054}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "1fe9129f1cdb", "timestamp": "2026-07-27T09:38:06.671169+00:00", "git_sha": "3d7ce49", "config_hash": "cacbb6ece843", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.875, "base_error": 0.125, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.027466089121522774, "rho_basin": 0.04398134938153453, "energy_min": 0.05234072522833359, "energy_mean": 0.04353111898523968, "energy_std": 0.1232672107794348, "msp": 0.02395206735255971, "temp_msp": 0.0231856584368455, "entropy": 0.03417904916905903}, "temperature": 0.27953871560488314, "best_energy_baseline": "energy_mean", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.02487463610681082, 0.00582124344497423, 0.048405336508425545], "delta_aurc_vs_best_energy": [0.01606502986371691, 0.0010850007001792562, 0.03158239388658305], "delta_aurc_vs_best_baseline": [-0.004280430684677275, -0.014613823709454883, 0.005013414288260376], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.009259259259259259, 0.008333333333333335, 0.007575757575757576, 0.013888888888888888, 0.01282051282051282, 0.017857142857142856, 0.016666666666666663, 0.015625, 0.014705882352941176, 0.013888888888888886, 0.013157894736842105, 0.0125, 0.011904761904761902, 0.015151515151515152, 0.014492753623188406, 0.013888888888888886, 0.013333333333333334, 0.01282051282051282, 0.012345679012345678, 0.011904761904761904, 0.01436781609195402, 0.013888888888888888, 0.01881720430107527, 0.018229166666666668, 0.017676767676767676, 0.01715686274509804, 0.019047619047619046, 0.018518518518518514, 0.02252252252252252, 0.02631578947368421, 0.03205128205128205, 0.035416666666666666, 0.04674796747967479, 0.06150793650793663, 0.07170542635658915, 0.07765151515151515, 0.08888888888888889, 0.09782608695652174, 0.10283687943262422, 0.1093750000000001, 0.11904761904761904, 0.125]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03424657534246576, 0.03424657534246577, 0.03424657534246575, 0.03424657534246573, 0.03424657534246572, 0.03424657534246571, 0.0342465753424657, 0.0342465753424657, 0.034246575342465696, 0.034246575342465696, 0.034246575342465696, 0.034246575342465696, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246568, 0.03424657534246568, 0.0342465753424657, 0.034246575342465745, 0.03424657534246578, 0.034246575342465814, 0.03424657534246585, 0.034246575342465876, 0.034246575342465904, 0.034246575342465925, 0.034246575342465946, 0.03424657534246597, 0.03424657534246599, 0.03424657534246601, 0.03424657534246602, 0.03424657534246604, 0.03424657534246606, 0.03424657534246607, 0.034246575342466085, 0.036240786240786575, 0.0400717703349286, 0.04370629370629411, 0.04715909090909134, 0.05044345898004483, 0.0564236111111116, 0.06237887596899269, 0.06925230566534957, 0.0793075684380036, 0.08861058601134253, 0.09690101757631865, 0.10375816993464093, 0.10884353741496634, 0.12500000000000044]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.02976190476190476, 0.03888888888888888, 0.036458333333333336, 0.0392156862745098, 0.03703703703703703, 0.039473684210526314, 0.0375, 0.03968253968253967, 0.045454545454545456, 0.043478260869565216, 0.04513888888888888, 0.04666666666666667, 0.04487179487179487, 0.043209876543209874, 0.044642857142857144, 0.043103448275862065, 0.04722222222222222, 0.0456989247311828, 0.046875, 0.045454545454545456, 0.049019607843137254, 0.04761904761904761, 0.050925925925925916, 0.05405405405405406, 0.05921052631578947, 0.06196581196581197, 0.06666666666666667, 0.06910569105691056, 0.06944444444444443, 0.0755813953488372, 0.07954545454545454, 0.09074074074074075, 0.09601449275362318, 0.10106382978723402, 0.10763888888888899, 0.11734693877551021, 0.125]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.027777777777777776, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.023809523809523808, 0.02222222222222222, 0.026041666666666668, 0.024509803921568627, 0.023148148148148143, 0.02631578947368421, 0.025, 0.023809523809523805, 0.026515151515151516, 0.03260869565217391, 0.03819444444444444, 0.03666666666666667, 0.035256410256410256, 0.037037037037037035, 0.03869047619047619, 0.037356321839080456, 0.03611111111111111, 0.04032258064516129, 0.041666666666666664, 0.047979797979797977, 0.049019607843137254, 0.04761904761904761, 0.050925925925925916, 0.0518018018018018, 0.05482456140350877, 0.057692307692307696, 0.06875, 0.06910569105691056, 0.07142857142857141, 0.07945736434108527, 0.08333333333333333, 0.08888888888888889, 0.09420289855072464, 0.10106382978723415, 0.10937499999999999, 0.11564625850340136, 0.125]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.25, 0.16666666666666666, 0.14583333333333334, 0.13333333333333333, 0.1250000000000001, 0.11904761904761907, 0.11458333333333333, 0.10185185185185185, 0.11666666666666668, 0.11363636363636363, 0.1111111111111111, 0.10256410256410256, 0.10714285714285714, 0.11111111111111109, 0.10416666666666667, 0.10294117647058823, 0.10185185185185183, 0.10087719298245613, 0.09583333333333334, 0.09523809523809522, 0.10606060606060606, 0.12681159420289856, 0.12152777777777787, 0.12333333333333334, 0.125, 0.12345679012345678, 0.12797619047619047, 0.12356321839080457, 0.12222222222222222, 0.12634408602150538, 0.12760416666666666, 0.12373737373737374, 0.12254901960784313, 0.1238095238095238, 0.12499999999999999, 0.12612612612612611, 0.12938596491228072, 0.13034188034188035, 0.12708333333333333, 0.12398373983739835, 0.12499999999999999, 0.1298449612403101, 0.13068181818181818, 0.12777777777777777, 0.13043478260869565, 0.12765957446808507, 0.12673611111111108, 0.12585034013605442, 0.125]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.002976190476190476, 0.002873563218390804, 0.002777777777777778, 0.005376344086021506, 0.010416666666666666, 0.010101010101010102, 0.01715686274509804, 0.026190476190476344, 0.039351851851851846, 0.04279279279279279, 0.05043859649122807, 0.057692307692307696, 0.0625, 0.06504065040650406, 0.07142857142857141, 0.07751937984496124, 0.07954545454545454, 0.08703703703703704, 0.09420289855072464, 0.09929078014184396, 0.10937499999999999, 0.12074829931972789, 0.125]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.0043859649122807015, 0.004166666666666667, 0.003968253968253967, 0.003787878787878788, 0.007246376811594203, 0.006944444444444443, 0.006666666666666667, 0.009615384615384616, 0.012345679012345678, 0.011904761904761904, 0.017241379310344824, 0.016666666666666666, 0.016129032258064516, 0.015625, 0.017676767676767676, 0.01715686274509804, 0.019047619047619046, 0.02083333333333333, 0.024774774774774775, 0.02850877192982456, 0.027777777777777776, 0.03958333333333333, 0.04268292682926829, 0.049603174603174593, 0.05813953488372093, 0.06818181818181818, 0.07962962962962963, 0.09057971014492754, 0.10460992907801417, 0.11458333333333331, 0.12074829931972789, 0.125]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.008771929824561403, 0.008333333333333333, 0.011904761904761902, 0.015151515151515152, 0.014492753623188406, 0.02083333333333333, 0.02, 0.03205128205128205, 0.040123456790123455, 0.041666666666666664, 0.043103448275862065, 0.05, 0.04838709677419355, 0.046875, 0.045454545454545456, 0.04656862745098039, 0.049999999999999996, 0.050925925925925916, 0.05405405405405406, 0.05701754385964912, 0.05555555555555555, 0.058333333333333334, 0.0650406504065042, 0.0734126984126984, 0.0755813953488372, 0.07954545454545454, 0.09074074074074075, 0.09782608695652174, 0.10106382978723402, 0.10763888888888899, 0.11394557823129252, 0.125]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.34844747383499997, "coverage": 0.875, "abstain_rate": 0.125, "selective_risk": 0.07809523809523809, "selective_accuracy": 0.9219047619047619, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.017811704834605598, 0.07809523809523809, 0.12374581939799331, 0.125, 0.125], "coverage": [0.0, 0.655, 0.875, 0.9966666666666667, 1.0, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.8335492063492064, "basin/entropy": 0.16746666666666668, "basin/dispersion": 0.437384126984127, "energy/mean": 0.19380317460317462, "energy/min": 0.21155555555555555, "energy/std": 0.48954920634920635, "curv/lmax_mean": 0.37255873015873014, "curv/lmax_best": 0.4954031746031746, "curv/trace_mean": 0.4040888888888889, "curv/trace_best": 0.4389968253968254, "dynamics/steps": 0.5, "dynamics/monotonic": 0.29546666666666666, "dynamics/drop": 0.3021968253968254, "dynamics/residual": 0.40071111111111113}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [1, 0, 0, 11, 0, 0, 12, 0, 0, 0, 11, 0, 0, 22, 0, 0, 45, 0, 0, 423], "incorrect_counts": [11, 0, 0, 6, 0, 0, 11, 0, 0, 0, 12, 0, 0, 10, 0, 0, 10, 0, 0, 15]}, "basin/entropy": {"edges": [0.0, 0.041197960825054114, 0.08239592165010823, 0.12359388247516234, 0.16479184330021646, 0.20598980412527057, 0.24718776495032468, 0.2883857257753788, 0.3295836866004329, 0.370781647425487, 0.41197960825054114, 0.45317756907559525, 0.49437552990064937, 0.5355734907257035, 0.5767714515507576, 0.6179694123758117, 0.6591673732008658, 0.7003653340259199, 0.741563294850974, 0.7827612556760282, 0.8239592165010823], "correct_counts": [423, 0, 0, 0, 0, 0, 45, 0, 0, 0, 21, 0, 0, 12, 0, 10, 12, 0, 0, 2], "incorrect_counts": [15, 0, 0, 0, 0, 0, 10, 0, 0, 0, 9, 0, 0, 13, 0, 11, 17, 0, 0, 0]}, "basin/dispersion": {"edges": [1.637185344543599, 1.655643833346576, 1.6741023221495528, 1.6925608109525296, 1.7110192997555065, 1.729477788558483, 1.74793627736146, 1.7663947661644368, 1.7848532549674136, 1.8033117437703905, 1.8217702325733673, 1.8402287213763442, 1.858687210179321, 1.8771456989822979, 1.8956041877852747, 1.9140626765882516, 1.9325211653912282, 1.9509796541942053, 1.969438142997182, 1.9878966318001587, 2.0063551206031356], "correct_counts": [3, 5, 5, 18, 17, 28, 29, 37, 53, 58, 47, 67, 48, 38, 30, 16, 13, 8, 3, 2], "incorrect_counts": [0, 1, 0, 1, 3, 4, 6, 2, 6, 5, 7, 10, 7, 10, 2, 3, 2, 3, 2, 1]}, "energy/mean": {"edges": [-0.2380810777346293, -0.18294511114557582, -0.12780914455652237, -0.07267317796746889, -0.017537211378415407, 0.037598755210638074, 0.09273472179969153, 0.14787068838874504, 0.2030066549777985, 0.25814262156685197, 0.3132785881559055, 0.368414554744959, 0.4235505213340124, 0.4786864879230659, 0.5338224545121194, 0.5889584211011728, 0.6440943876902263, 0.6992303542792798, 0.7543663208683332, 0.8095022874573868, 0.8646382540464401], "correct_counts": [11, 51, 71, 110, 65, 46, 24, 23, 18, 19, 10, 15, 11, 7, 14, 9, 8, 8, 4, 1], "incorrect_counts": [0, 1, 3, 2, 5, 4, 4, 1, 3, 2, 8, 1, 7, 3, 8, 3, 7, 6, 5, 2]}, "energy/min": {"edges": [-0.6323069334030151, -0.5714096486568451, -0.510512363910675, -0.449615079164505, -0.388717794418335, -0.3278205096721649, -0.26692322492599485, -0.20602594017982484, -0.14512865543365477, -0.08423137068748476, -0.023334085941314697, 0.03756319880485537, 0.09846048355102544, 0.1593577682971955, 0.22025505304336546, 0.2811523377895355, 0.3420496225357056, 0.40294690728187565, 0.4638441920280456, 0.5247414767742158, 0.5856387615203857], "correct_counts": [3, 17, 46, 75, 77, 67, 67, 41, 14, 19, 21, 10, 15, 7, 12, 17, 6, 9, 0, 2], "incorrect_counts": [1, 0, 1, 2, 5, 5, 3, 3, 2, 5, 4, 3, 3, 3, 9, 8, 9, 5, 2, 2]}, "energy/std": {"edges": [0.09172678671306, 0.10379040427150595, 0.11585402182995189, 0.1279176393883978, 0.13998125694684377, 0.1520448745052897, 0.16410849206373562, 0.17617210962218155, 0.1882357271806275, 0.20029934473907346, 0.2123629622975194, 0.2244265798559653, 0.23649019741441124, 0.2485538149728572, 0.2606174325313031, 0.2726810500897491, 0.284744667648195, 0.29680828520664093, 0.3088719027650869, 0.3209355203235328, 0.33299913788197877], "correct_counts": [4, 3, 7, 24, 34, 39, 53, 54, 41, 63, 47, 44, 35, 24, 19, 14, 8, 5, 5, 2], "incorrect_counts": [0, 0, 4, 2, 4, 5, 5, 4, 13, 10, 5, 9, 5, 4, 2, 1, 0, 2, 0, 0]}, "curv/lmax_mean": {"edges": [1.0001661280790966, 1.0006819412112238, 1.0011977543433508, 1.0017135674754778, 1.002229380607605, 1.0027451937397323, 1.0032610068718593, 1.0037768200039863, 1.0042926331361135, 1.0048084462682407, 1.0053242594003677, 1.0058400725324947, 1.006355885664622, 1.0068716987967492, 1.0073875119288762, 1.0079033250610032, 1.0084191381931304, 1.0089349513252577, 1.0094507644573847, 1.0099665775895117, 1.010482390721639], "correct_counts": [65, 100, 87, 66, 41, 40, 43, 27, 12, 17, 5, 5, 3, 5, 2, 2, 3, 1, 0, 1], "incorrect_counts": [2, 10, 9, 9, 10, 9, 8, 4, 5, 2, 3, 0, 2, 1, 1, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.001549595594406, 1.0030991911888123, 1.0046487867832183, 1.0061983823776246, 1.0077479779720306, 1.0092975735664367, 1.010847169160843, 1.012396764755249, 1.0139463603496552, 1.0154959559440613, 1.0170455515384673, 1.0185951471328736, 1.0201447427272796, 1.0216943383216859, 1.023243933916092, 1.024793529510498, 1.0263431251049042, 1.0278927206993103, 1.0294423162937165, 1.0309919118881226], "correct_counts": [311, 80, 40, 32, 16, 11, 9, 6, 5, 2, 3, 2, 3, 0, 2, 0, 1, 0, 1, 1], "incorrect_counts": [43, 14, 5, 1, 4, 2, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0]}, "curv/trace_mean": {"edges": [32.0079870223999, 32.01155290603638, 32.015118789672854, 32.01868467330932, 32.0222505569458, 32.025816440582275, 32.02938232421875, 32.03294820785523, 32.036514091491696, 32.04007997512817, 32.04364585876465, 32.047211742401124, 32.0507776260376, 32.05434350967407, 32.057909393310545, 32.06147527694702, 32.0650411605835, 32.06860704421997, 32.07217292785644, 32.07573881149292, 32.079304695129395], "correct_counts": [20, 28, 38, 43, 36, 59, 52, 37, 50, 24, 35, 27, 23, 17, 21, 8, 3, 3, 0, 1], "incorrect_counts": [2, 3, 3, 3, 5, 5, 4, 9, 6, 6, 6, 3, 10, 4, 1, 0, 4, 0, 1, 0]}, "curv/trace_best": {"edges": [32.001441955566406, 32.00756416320801, 32.01368637084961, 32.01980857849121, 32.02593078613281, 32.032052993774414, 32.03817520141602, 32.04429740905762, 32.050419616699216, 32.05654182434082, 32.06266403198242, 32.068786239624025, 32.07490844726563, 32.081030654907224, 32.08715286254883, 32.09327507019043, 32.09939727783203, 32.105519485473636, 32.11164169311523, 32.117763900756835, 32.12388610839844], "correct_counts": [21, 62, 75, 72, 60, 48, 55, 36, 25, 18, 16, 9, 9, 5, 4, 7, 2, 1, 0, 0], "incorrect_counts": [4, 4, 11, 5, 13, 6, 6, 5, 6, 3, 3, 4, 0, 1, 0, 1, 1, 0, 0, 2]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [525, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7250000000000001, 0.7297500000000001, 0.7345000000000002, 0.7392500000000001, 0.7440000000000001, 0.7487500000000001, 0.7535000000000001, 0.7582500000000001, 0.7630000000000001, 0.7677500000000002, 0.7725000000000002, 0.7772500000000001, 0.7820000000000001, 0.7867500000000002, 0.7915000000000001, 0.7962500000000001, 0.8010000000000002, 0.8057500000000002, 0.8105000000000002, 0.8152500000000001, 0.8200000000000002], "correct_counts": [1, 2, 10, 9, 19, 39, 46, 70, 84, 78, 57, 44, 33, 17, 11, 3, 1, 0, 1, 0], "incorrect_counts": [0, 0, 0, 1, 2, 7, 0, 6, 2, 8, 7, 10, 6, 8, 7, 1, 4, 3, 2, 1]}, "dynamics/drop": {"edges": [74.32381541033585, 85.35149599189559, 96.37917657345534, 107.40685715501508, 118.43453773657481, 129.46221831813455, 140.4898988996943, 151.51757948125402, 162.54526006281378, 173.5729406443735, 184.60062122593325, 195.628301807493, 206.65598238905272, 217.6836629706125, 228.7113435521722, 239.73902413373196, 250.76670471529167, 261.79438529685143, 272.82206587841114, 283.8497464599709, 294.8774270415306], "correct_counts": [5, 32, 66, 110, 104, 112, 44, 30, 9, 2, 2, 2, 2, 2, 1, 0, 0, 2, 0, 0], "incorrect_counts": [0, 4, 8, 7, 7, 7, 5, 6, 4, 6, 4, 2, 2, 2, 3, 4, 0, 1, 1, 2]}, "dynamics/residual": {"edges": [1.162097414334615, 1.177681430677573, 1.1932654470205306, 1.2088494633634885, 1.2244334797064462, 1.2400174960494041, 1.2556015123923618, 1.2711855287353198, 1.2867695450782775, 1.3023535614212354, 1.317937577764193, 1.333521594107151, 1.349105610450109, 1.3646896267930666, 1.3802736431360245, 1.3958576594789822, 1.4114416758219401, 1.4270256921648978, 1.4426097085078557, 1.4581937248508137, 1.4737777411937714], "correct_counts": [4, 6, 11, 24, 26, 38, 59, 74, 72, 54, 48, 48, 30, 20, 5, 2, 3, 0, 0, 1], "incorrect_counts": [0, 0, 1, 1, 4, 2, 8, 6, 10, 8, 11, 11, 6, 4, 1, 1, 1, 0, 0, 0]}}}, "ece_geometry": 0.057436118293730715}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "2a934d626111", "timestamp": "2026-07-27T09:38:09.473524+00:00", "git_sha": "3d7ce49", "config_hash": "054f01c6c637", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.825, "base_error": 0.175, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.06161855001684325, "rho_basin": 0.09229054743872664, "energy_min": 0.21136180624942313, "energy_mean": 0.21095193442795585, "energy_std": 0.17250412654357142, "msp": 0.05530041645573463, "temp_msp": 0.05305986961532728, "entropy": 0.07641415209991691}, "temperature": 0.28776072591113844, "best_energy_baseline": "energy_std", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.1497432562325799, 0.10842759072651144, 0.19249016133690647], "delta_aurc_vs_best_energy": [0.11088557652672817, 0.07324712187491986, 0.14959801044328674], "delta_aurc_vs_best_baseline": [-0.008558680401515972, -0.02338443781342537, 0.005027521049910705], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.037037037037037035, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.05128205128205128, 0.047619047619047616, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.04166666666666666, 0.039473684210526314, 0.041666666666666664, 0.03968253968253967, 0.03787878787878788, 0.03985507246376811, 0.03819444444444444, 0.04, 0.041666666666666664, 0.040123456790123455, 0.041666666666666664, 0.045977011494252866, 0.04722222222222222, 0.051075268817204304, 0.052083333333333336, 0.05808080808080808, 0.061274509803921566, 0.06666666666666665, 0.06712962962962962, 0.06981981981981981, 0.07894736842105263, 0.08547008547008547, 0.09166666666666666, 0.09756097560975609, 0.10714285714285712, 0.1182170542635659, 0.13257575757575757, 0.14074074074074075, 0.14855072463768115, 0.15602836879432636, 0.16319444444444442, 0.1683673469387755, 0.175]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.07894736842105264, 0.07894736842105264, 0.07894736842105264, 0.07894736842105264, 0.07894736842105259, 0.07894736842105252, 0.07894736842105247, 0.07894736842105245, 0.07894736842105247, 0.07894736842105253, 0.07894736842105259, 0.07894736842105263, 0.07894736842105267, 0.0789473684210527, 0.07894736842105272, 0.07894736842105275, 0.07894736842105275, 0.07894736842105267, 0.0789473684210526, 0.07894736842105253, 0.07894736842105247, 0.07894736842105243, 0.07894736842105238, 0.07894736842105234, 0.0789473684210523, 0.07894736842105225, 0.07894736842105222, 0.07894736842105218, 0.07894736842105216, 0.07894736842105213, 0.0789473684210521, 0.07963466183574824, 0.08161323378714626, 0.08347541915316793, 0.08523119392684551, 0.086889425657541, 0.0884580232406313, 0.0920032639738876, 0.09679984098588691, 0.10135658914728626, 0.10569105691056861, 0.1129785247432302, 0.11992704058367491, 0.1270396270396266, 0.1361823361823356, 0.1449275362318834, 0.15521783181357598, 0.1650793650793647, 0.17386185243328067, 0.17499999999999957]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.4166666666666667, 0.3888888888888889, 0.3333333333333333, 0.3333333333333333, 0.3194444444444445, 0.3452380952380953, 0.3020833333333333, 0.28703703703703703, 0.2666666666666667, 0.24242424242424243, 0.2222222222222222, 0.22435897435897437, 0.21428571428571427, 0.21666666666666665, 0.22395833333333334, 0.2107843137254902, 0.21296296296296308, 0.2149122807017544, 0.21666666666666667, 0.21825396825396837, 0.2159090909090909, 0.20652173913043478, 0.20833333333333331, 0.2, 0.19230769230769232, 0.1882716049382716, 0.1875, 0.18103448275862066, 0.17777777777777778, 0.17473118279569894, 0.171875, 0.16666666666666666, 0.16176470588235295, 0.15952380952380948, 0.15740740740740738, 0.15315315315315314, 0.15350877192982457, 0.14957264957264957, 0.15, 0.15243902439024387, 0.14880952380952378, 0.14922480620155038, 0.15151515151515152, 0.15185185185185185, 0.15217391304347827, 0.15602836879432636, 0.16319444444444442, 0.1717687074829932, 0.175]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.375, 0.3333333333333333, 0.2708333333333333, 0.23333333333333334, 0.3194444444444445, 0.3214285714285715, 0.28125, 0.28703703703703703, 0.2833333333333334, 0.2878787878787879, 0.2638888888888889, 0.25, 0.24404761904761904, 0.24999999999999994, 0.23958333333333334, 0.23529411764705882, 0.22685185185185183, 0.2236842105263158, 0.2125, 0.20238095238095236, 0.19318181818181818, 0.19202898550724637, 0.18749999999999997, 0.18333333333333332, 0.1858974358974359, 0.18209876543209877, 0.17857142857142858, 0.17528735632183906, 0.17222222222222222, 0.16666666666666666, 0.16145833333333334, 0.1590909090909091, 0.15441176470588236, 0.15238095238095237, 0.1504629629629631, 0.1554054054054054, 0.15350877192982457, 0.1517094017094017, 0.15, 0.14837398373983737, 0.1468253968253968, 0.1434108527131783, 0.14393939393939395, 0.14814814814814814, 0.1503623188405797, 0.1524822695035462, 0.1597222222222223, 0.1683673469387755, 0.175]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.20833333333333334, 0.16666666666666666, 0.16666666666666663, 0.1785714285714286, 0.16666666666666666, 0.17592592592592593, 0.16666666666666669, 0.15151515151515152, 0.1597222222222222, 0.15384615384615385, 0.17857142857142858, 0.1722222222222222, 0.171875, 0.16666666666666666, 0.162037037037037, 0.16666666666666666, 0.175, 0.17460317460317457, 0.17424242424242425, 0.17028985507246377, 0.17361111111111108, 0.17333333333333334, 0.1794871794871795, 0.18518518518518517, 0.18154761904761904, 0.17816091954022986, 0.17777777777777778, 0.18010752688172044, 0.1796875, 0.18181818181818182, 0.18137254901960784, 0.17619047619047618, 0.17361111111111108, 0.17117117117117117, 0.16885964912280702, 0.16666666666666666, 0.16458333333333333, 0.16666666666666663, 0.16666666666666663, 0.16279069767441862, 0.16666666666666666, 0.1648148148148148, 0.16666666666666666, 0.16666666666666663, 0.17013888888888887, 0.17346938775510204, 0.175]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005952380952380952, 0.005555555555555555, 0.015625, 0.0196078431372549, 0.018518518518518514, 0.017543859649122806, 0.016666666666666666, 0.019841269841269837, 0.022727272727272728, 0.028985507246376812, 0.03472222222222222, 0.03666666666666667, 0.035256410256410256, 0.040123456790123455, 0.044642857142857144, 0.04885057471264386, 0.06111111111111111, 0.06451612903225806, 0.06770833333333333, 0.07828282828282829, 0.0857843137254902, 0.09523809523809522, 0.10185185185185183, 0.1036036036036036, 0.10964912280701754, 0.11324786324786325, 0.11875, 0.1219512195121951, 0.12103174603174614, 0.1298449612403101, 0.13257575757575757, 0.13703703703703704, 0.1431159420289855, 0.1524822695035462, 0.15972222222222218, 0.16666666666666666, 0.175]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.008333333333333335, 0.007575757575757576, 0.006944444444444444, 0.00641025641025641, 0.005952380952380952, 0.016666666666666663, 0.015625, 0.014705882352941176, 0.018518518518518514, 0.02631578947368421, 0.025, 0.023809523809523805, 0.030303030303030304, 0.028985507246376812, 0.027777777777777887, 0.03333333333333333, 0.035256410256410256, 0.037037037037037035, 0.041666666666666664, 0.045977011494252866, 0.05277777777777778, 0.05913978494623656, 0.0625, 0.06565656565656566, 0.06862745098039216, 0.07619047619047618, 0.08333333333333331, 0.0945945945945946, 0.09649122807017543, 0.10042735042735043, 0.10625, 0.11382113821138223, 0.11507936507936518, 0.12403100775193798, 0.12878787878787878, 0.13518518518518519, 0.14130434782608695, 0.150709219858156, 0.15798611111111108, 0.1683673469387755, 0.175]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.009259259259259259, 0.008333333333333335, 0.007575757575757576, 0.027777777777777776, 0.04487179487179487, 0.041666666666666664, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.050925925925925916, 0.06140350877192982, 0.07083333333333333, 0.06746031746031758, 0.08712121212121213, 0.09420289855072464, 0.10069444444444443, 0.10333333333333333, 0.10256410256410256, 0.10185185185185185, 0.10119047619047619, 0.10057471264367815, 0.10277777777777777, 0.10483870967741936, 0.10416666666666667, 0.10353535353535354, 0.10049019607843138, 0.10238095238095236, 0.10185185185185183, 0.10585585585585586, 0.10964912280701754, 0.11324786324786325, 0.11041666666666666, 0.11178861788617898, 0.11904761904761903, 0.12209302325581395, 0.1268939393939394, 0.13333333333333333, 0.14130434782608695, 0.15425531914893628, 0.16319444444444453, 0.17006802721088435, 0.175]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.15723492252970733, "coverage": 0.705, "abstain_rate": 0.295, "selective_risk": 0.06619385342789598, "selective_accuracy": 0.933806146572104, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.06619385342789598, 0.11650485436893204, 0.175, 0.175], "coverage": [0.0, 0.0, 0.705, 0.8583333333333333, 1.0, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7495911495911496, "basin/entropy": 0.24934102934102934, "basin/dispersion": 0.49964405964405967, "energy/mean": 0.5078595478595479, "energy/min": 0.5175949975949976, "energy/std": 0.48817700817700815, "curv/lmax_mean": 0.5006637806637807, "curv/lmax_best": 0.5116305916305917, "curv/trace_mean": 0.49462241462241463, "curv/trace_best": 0.5089369889369889, "dynamics/steps": 0.5, "dynamics/monotonic": 0.3703030303030303, "dynamics/drop": 0.3482058682058682, "dynamics/residual": 0.5112265512265513}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [10, 0, 0, 13, 0, 0, 12, 0, 0, 0, 20, 0, 0, 31, 0, 0, 59, 0, 0, 350], "incorrect_counts": [3, 0, 0, 22, 0, 0, 14, 0, 0, 0, 14, 0, 0, 12, 0, 0, 10, 0, 0, 30]}, "basin/entropy": {"edges": [0.0, 0.053756966200268666, 0.10751393240053733, 0.161270898600806, 0.21502786480107466, 0.2687848310013433, 0.322541797201612, 0.3762987634018807, 0.43005572960214933, 0.483812695802418, 0.5375696620026866, 0.5913266282029553, 0.645083594403224, 0.6988405606034926, 0.7525975268037614, 0.80635449300403, 0.8601114592042987, 0.9138684254045674, 0.967625391604836, 1.0213823578051047, 1.0751393240053733], "correct_counts": [350, 0, 0, 0, 0, 59, 0, 0, 30, 0, 20, 11, 20, 1, 0, 1, 1, 2, 0, 0], "incorrect_counts": [30, 0, 0, 0, 0, 10, 0, 0, 12, 0, 11, 14, 22, 3, 0, 0, 1, 1, 0, 1]}, "basin/dispersion": {"edges": [1.6415780038768077, 1.6640313201520378, 1.6864846364272679, 1.708937952702498, 1.731391268977728, 1.753844585252958, 1.7762979015281881, 1.798751217803418, 1.821204534078648, 1.843657850353878, 1.8661111666291081, 1.8885644829043382, 1.9110177991795683, 1.9334711154547983, 1.9559244317300284, 1.9783777480052582, 2.0008310642804883, 2.0232843805557184, 2.0457376968309484, 2.0681910131061785, 2.0906443293814085], "correct_counts": [3, 8, 12, 19, 39, 47, 51, 67, 66, 56, 37, 35, 22, 13, 8, 6, 3, 2, 0, 1], "incorrect_counts": [2, 0, 3, 5, 5, 14, 12, 10, 15, 9, 11, 5, 7, 2, 2, 3, 0, 0, 0, 0]}, "energy/mean": {"edges": [0.2628147800763448, 0.3243880569934845, 0.38596133391062415, 0.44753461082776386, 0.5091078877449036, 0.5706811646620433, 0.6322544415791829, 0.6938277184963226, 0.7554009954134624, 0.8169742723306019, 0.8785475492477417, 0.9401208261648815, 1.001694103082021, 1.0632673799991608, 1.1248406569163005, 1.1864139338334403, 1.2479872107505798, 1.3095604876677196, 1.3711337645848591, 1.432707041501999, 1.4942803184191387], "correct_counts": [1, 6, 8, 19, 15, 18, 35, 58, 81, 101, 66, 37, 19, 8, 5, 3, 5, 2, 3, 5], "incorrect_counts": [1, 2, 6, 4, 10, 4, 11, 10, 6, 9, 10, 2, 6, 3, 2, 4, 3, 2, 8, 2]}, "energy/min": {"edges": [-0.1004953682422638, -0.03417549580335617, 0.03214437663555145, 0.09846424907445905, 0.1647841215133667, 0.23110399395227432, 0.2974238663911819, 0.36374373883008954, 0.43006361126899717, 0.49638348370790475, 0.5627033561468124, 0.62902322858572, 0.6953431010246276, 0.7616629734635353, 0.8279828459024429, 0.8943027183413506, 0.9606225907802581, 1.0269424632191657, 1.0932623356580733, 1.159582208096981, 1.2259020805358887], "correct_counts": [1, 2, 5, 8, 24, 18, 40, 46, 75, 91, 81, 44, 22, 7, 11, 4, 6, 3, 5, 2], "incorrect_counts": [0, 0, 2, 9, 9, 9, 3, 8, 17, 9, 4, 5, 7, 2, 3, 6, 2, 7, 2, 1]}, "energy/std": {"edges": [0.0727182563720062, 0.08928063387973476, 0.1058430113874633, 0.12240538889519184, 0.1389677664029204, 0.15553014391064895, 0.1720925214183775, 0.18865489892610604, 0.20521727643383458, 0.22177965394156313, 0.23834203144929167, 0.25490440895702027, 0.2714667864647488, 0.28802916397247735, 0.3045915414802059, 0.32115391898793444, 0.337716296495663, 0.3542786740033915, 0.37084105151112007, 0.3874034290188486, 0.4039658065265772], "correct_counts": [2, 4, 14, 30, 45, 73, 65, 67, 67, 53, 37, 9, 13, 10, 5, 0, 1, 0, 0, 0], "incorrect_counts": [1, 0, 3, 6, 10, 13, 15, 18, 10, 8, 7, 2, 5, 4, 1, 0, 1, 0, 0, 1]}, "curv/lmax_mean": {"edges": [0.9986756841341654, 0.9992095266779264, 0.9997433692216873, 1.0002772117654481, 1.0008110543092092, 1.0013448968529701, 1.001878739396731, 1.0024125819404919, 1.002946424484253, 1.0034802670280139, 1.0040141095717747, 1.0045479521155358, 1.0050817946592967, 1.0056156372030576, 1.0061494797468185, 1.0066833222905796, 1.0072171648343404, 1.0077510073781013, 1.0082848499218622, 1.0088186924656233, 1.0093525350093842], "correct_counts": [2, 47, 164, 112, 58, 41, 23, 15, 11, 7, 4, 6, 0, 2, 1, 1, 0, 0, 0, 1], "incorrect_counts": [1, 26, 21, 13, 9, 6, 4, 7, 4, 4, 1, 6, 0, 2, 0, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.9967403411865234, 0.9977234482765198, 0.9987065553665161, 0.9996896624565125, 1.0006727695465087, 1.0016558766365051, 1.0026389837265015, 1.003622090816498, 1.004605197906494, 1.0055883049964904, 1.0065714120864868, 1.0075545191764832, 1.0085376262664796, 1.0095207333564757, 1.0105038404464721, 1.0114869475364685, 1.0124700546264649, 1.0134531617164613, 1.0144362688064574, 1.0154193758964538, 1.0164024829864502], "correct_counts": [2, 2, 33, 345, 66, 19, 9, 6, 1, 2, 2, 1, 1, 1, 1, 1, 0, 0, 1, 2], "incorrect_counts": [1, 0, 12, 64, 10, 4, 1, 3, 2, 2, 0, 0, 0, 2, 2, 0, 0, 0, 2, 0]}, "curv/trace_mean": {"edges": [31.886516253153484, 31.8960693359375, 31.905622418721517, 31.915175501505534, 31.924728584289554, 31.93428166707357, 31.943834749857587, 31.953387832641603, 31.96294091542562, 31.972493998209636, 31.982047080993652, 31.991600163777672, 32.001153246561685, 32.0107063293457, 32.020259412129725, 32.02981249491374, 32.03936557769776, 32.048918660481775, 32.05847174326579, 32.06802482604981, 32.077577908833824], "correct_counts": [3, 0, 4, 5, 2, 4, 5, 3, 5, 8, 31, 68, 119, 97, 56, 45, 31, 9, 0, 0], "incorrect_counts": [3, 2, 4, 4, 1, 3, 2, 3, 2, 5, 3, 7, 13, 10, 11, 7, 14, 9, 0, 2]}, "curv/trace_best": {"edges": [31.816234588623047, 31.83113193511963, 31.84602928161621, 31.860926628112793, 31.875823974609375, 31.890721321105957, 31.90561866760254, 31.92051601409912, 31.935413360595703, 31.950310707092285, 31.965208053588867, 31.98010540008545, 31.99500274658203, 32.00990009307861, 32.024797439575195, 32.03969478607178, 32.05459213256836, 32.06948947906494, 32.08438682556152, 32.099284172058105, 32.11418151855469], "correct_counts": [1, 0, 0, 2, 3, 0, 1, 7, 10, 10, 15, 44, 187, 116, 53, 32, 8, 2, 3, 1], "incorrect_counts": [0, 1, 2, 0, 4, 1, 2, 3, 9, 4, 4, 6, 16, 22, 12, 8, 4, 3, 4, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [495, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7233333333333335, 0.7288333333333334, 0.7343333333333335, 0.7398333333333335, 0.7453333333333335, 0.7508333333333335, 0.7563333333333335, 0.7618333333333335, 0.7673333333333335, 0.7728333333333335, 0.7783333333333335, 0.7838333333333335, 0.7893333333333334, 0.7948333333333335, 0.8003333333333335, 0.8058333333333335, 0.8113333333333335, 0.8168333333333335, 0.8223333333333335, 0.8278333333333335, 0.8333333333333335], "correct_counts": [5, 6, 7, 29, 40, 57, 77, 73, 59, 56, 33, 23, 15, 9, 4, 1, 1, 0, 0, 0], "incorrect_counts": [0, 0, 4, 2, 4, 8, 14, 9, 11, 18, 9, 6, 5, 4, 8, 1, 1, 0, 0, 1]}, "dynamics/drop": {"edges": [61.334716603159904, 74.52057102819283, 87.70642545322578, 100.8922798782587, 114.07813430329165, 127.26398872832458, 140.4498431533575, 153.63569757839045, 166.8215520034234, 180.0074064284563, 193.19326085348925, 206.3791152785222, 219.5649697035551, 232.75082412858805, 245.936678553621, 259.1225329786539, 272.3083874036869, 285.4942418287198, 298.6800962537527, 311.8659506787857, 325.0518051038186], "correct_counts": [3, 27, 66, 113, 119, 86, 38, 14, 6, 7, 9, 2, 0, 2, 1, 0, 2, 0, 0, 0], "incorrect_counts": [1, 1, 13, 11, 18, 22, 9, 6, 5, 4, 3, 0, 3, 3, 1, 2, 0, 1, 0, 2]}, "dynamics/residual": {"edges": [1.154107744495074, 1.1687258986135325, 1.183344052731991, 1.1979622068504494, 1.2125803609689076, 1.227198515087366, 1.2418166692058246, 1.256434823324283, 1.2710529774427415, 1.2856711315612, 1.3002892856796584, 1.3149074397981166, 1.329525593916575, 1.3441437480350336, 1.358761902153492, 1.3733800562719505, 1.387998210390409, 1.4026163645088672, 1.4172345186273256, 1.431852672745784, 1.4464708268642426], "correct_counts": [3, 2, 4, 11, 16, 27, 45, 61, 66, 64, 50, 48, 28, 30, 22, 4, 8, 0, 5, 1], "incorrect_counts": [0, 1, 3, 2, 4, 8, 8, 11, 15, 9, 13, 10, 4, 7, 4, 3, 2, 0, 1, 0]}}}, "ece_geometry": 0.04520941743790561}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "9c2dbcfa6ae8", "timestamp": "2026-07-27T09:38:12.300151+00:00", "git_sha": "3d7ce49", "config_hash": "0b564d7f9e23", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.8266666666666667, "base_error": 0.17333333333333334, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.054893261179459335, "rho_basin": 0.08645638007796895, "energy_min": 0.1561569151577505, "energy_mean": 0.16005205383202037, "energy_std": 0.19277374311689105, "msp": 0.0412224839050244, "temp_msp": 0.042521440482154955, "entropy": 0.04802248425611829}, "temperature": 0.3184887138430314, "best_energy_baseline": "energy_min", "best_baseline": "msp", "delta_aurc_vs_energy_min": [0.10126365397829115, 0.06398265000827008, 0.1401163910892647], "delta_aurc_vs_best_energy": [0.10126365397829115, 0.06398265000827008, 0.1401163910892647], "delta_aurc_vs_best_baseline": [-0.013670777274434935, -0.030908977792358076, 0.00040221684223381557], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.03205128205128205, 0.02976190476190476, 0.027777777777777773, 0.026041666666666668, 0.024509803921568627, 0.023148148148148143, 0.021929824561403508, 0.029166666666666667, 0.027777777777777773, 0.030303030303030304, 0.03260869565217391, 0.031249999999999997, 0.03, 0.03205128205128205, 0.030864197530864196, 0.03273809523809524, 0.03160919540229885, 0.03333333333333333, 0.03225806451612903, 0.033854166666666664, 0.03787878787878788, 0.049019607843137254, 0.057142857142857134, 0.060185185185185175, 0.06756756756756757, 0.07675438596491228, 0.08547008547008547, 0.0875, 0.09552845528455298, 0.10515873015873015, 0.11046511627906977, 0.11931818181818182, 0.13148148148148148, 0.13768115942028986, 0.14007092198581572, 0.15277777777777776, 0.16156462585034015, 0.17333333333333334]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.07194244604316546, 0.07194244604316546, 0.07194244604316546, 0.07194244604316546, 0.07194244604316544, 0.07194244604316537, 0.07194244604316531, 0.07194244604316528, 0.07194244604316526, 0.07194244604316523, 0.07194244604316521, 0.07194244604316519, 0.07194244604316517, 0.07194244604316517, 0.07194244604316516, 0.07194244604316515, 0.07194244604316515, 0.07194244604316513, 0.07194244604316517, 0.07194244604316526, 0.07194244604316533, 0.0719424460431654, 0.07194244604316546, 0.07194244604316552, 0.07194244604316556, 0.07194244604316562, 0.07194244604316566, 0.0719424460431657, 0.07194244604316573, 0.07194244604316577, 0.0719424460431658, 0.07194244604316584, 0.07194244604316587, 0.0719424460431659, 0.07315668202765024, 0.07784498207885351, 0.0822798605056674, 0.08648132427843845, 0.0904673283705546, 0.09476744186046554, 0.10436755530346054, 0.11351052048726526, 0.12222823147647434, 0.13423295454545514, 0.1472934472934479, 0.14994425863991148, 0.15248226950354687, 0.15782828282828354, 0.16375121477162355, 0.17333333333333387]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.16666666666666666, 0.1388888888888889, 0.125, 0.1, 0.09722222222222232, 0.10714285714285716, 0.125, 0.12962962962962962, 0.1416666666666667, 0.15151515151515152, 0.1597222222222222, 0.14743589743589744, 0.1488095238095238, 0.16666666666666663, 0.171875, 0.17647058823529413, 0.17129629629629645, 0.17543859649122806, 0.175, 0.17460317460317457, 0.17424242424242425, 0.17391304347826086, 0.17361111111111108, 0.17, 0.16987179487179488, 0.16358024691358025, 0.16071428571428573, 0.15804597701149423, 0.15555555555555556, 0.15591397849462366, 0.15625, 0.15404040404040403, 0.1568627450980392, 0.15476190476190474, 0.15277777777777776, 0.14864864864864866, 0.14912280701754385, 0.1452991452991453, 0.14583333333333334, 0.14227642276422775, 0.14285714285714282, 0.14728682170542637, 0.14962121212121213, 0.15, 0.15760869565217392, 0.16134751773049658, 0.17187499999999997, 0.17346938775510204, 0.17333333333333334]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.1388888888888889, 0.125, 0.1, 0.11111111111111109, 0.11904761904761907, 0.125, 0.1574074074074074, 0.1416666666666667, 0.14393939393939395, 0.1388888888888889, 0.14102564102564102, 0.14285714285714285, 0.16666666666666663, 0.17708333333333334, 0.1715686274509804, 0.18055555555555552, 0.18421052631578946, 0.17916666666666667, 0.17857142857142855, 0.17424242424242425, 0.17753623188405798, 0.17013888888888898, 0.18, 0.1762820512820513, 0.1697530864197531, 0.1636904761904762, 0.16666666666666663, 0.16111111111111112, 0.16129032258064516, 0.1640625, 0.16161616161616163, 0.16176470588235295, 0.15952380952380948, 0.15740740740740738, 0.15315315315315314, 0.14912280701754385, 0.1452991452991453, 0.14166666666666666, 0.14024390243902438, 0.14087301587301584, 0.1434108527131783, 0.14393939393939395, 0.15, 0.15760869565217392, 0.1595744680851065, 0.16319444444444442, 0.16666666666666666, 0.17333333333333334]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.2916666666666667, 0.19444444444444445, 0.25, 0.2, 0.2222222222222223, 0.22619047619047603, 0.21875, 0.24074074074074073, 0.22500000000000003, 0.2196969696969697, 0.2222222222222222, 0.21153846153846154, 0.2261904761904762, 0.21666666666666665, 0.203125, 0.19607843137254902, 0.18518518518518515, 0.17543859649122806, 0.17083333333333334, 0.1706349206349206, 0.17424242424242425, 0.17391304347826086, 0.17013888888888887, 0.16666666666666666, 0.16025641025641027, 0.16358024691358025, 0.16666666666666666, 0.16379310344827602, 0.16666666666666666, 0.1639784946236559, 0.1640625, 0.16666666666666666, 0.16666666666666666, 0.17142857142857157, 0.17361111111111108, 0.17342342342342343, 0.17105263157894737, 0.17094017094017094, 0.16875, 0.17073170731707316, 0.16865079365079377, 0.17054263565891473, 0.17045454545454544, 0.1685185185185185, 0.16847826086956522, 0.16666666666666677, 0.17013888888888887, 0.17006802721088435, 0.17333333333333334]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.008771929824561403, 0.008333333333333333, 0.007936507936507934, 0.015151515151515152, 0.014492753623188406, 0.013888888888888886, 0.02, 0.019230769230769232, 0.024691358024691357, 0.026785714285714284, 0.02873563218390804, 0.030555555555555555, 0.02956989247311828, 0.03125, 0.03535353535353535, 0.041666666666666664, 0.04523809523809523, 0.053240740740740734, 0.06306306306306306, 0.07236842105263158, 0.0811965811965812, 0.09375, 0.09959349593495934, 0.10515873015873028, 0.11434108527131782, 0.125, 0.13333333333333333, 0.13768115942028986, 0.14184397163120566, 0.15277777777777776, 0.16326530612244897, 0.17333333333333334]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.004901960784313725, 0.0046296296296296285, 0.0043859649122807015, 0.004166666666666667, 0.007936507936507934, 0.011363636363636364, 0.014492753623188406, 0.013888888888888886, 0.013333333333333334, 0.019230769230769232, 0.018518518518518517, 0.023809523809523808, 0.03160919540229885, 0.03611111111111111, 0.03763440860215054, 0.044270833333333336, 0.050505050505050504, 0.049019607843137254, 0.057142857142857134, 0.060185185185185175, 0.06981981981981981, 0.07017543859649122, 0.0811965811965812, 0.09166666666666666, 0.10162601626016259, 0.11111111111111109, 0.12015503875968993, 0.12878787878787878, 0.13333333333333333, 0.13949275362318841, 0.1524822695035462, 0.15972222222222218, 0.16666666666666666, 0.17333333333333334]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.011904761904761904, 0.01111111111111111, 0.010416666666666666, 0.014705882352941176, 0.013888888888888886, 0.013157894736842105, 0.020833333333333332, 0.027777777777777773, 0.026515151515151516, 0.028985507246376812, 0.027777777777777773, 0.03, 0.035256410256410256, 0.037037037037037035, 0.041666666666666664, 0.043103448275862065, 0.044444444444444446, 0.04838709677419355, 0.052083333333333336, 0.05555555555555555, 0.06372549019607843, 0.0738095238095238, 0.07407407407407406, 0.08108108108108109, 0.08552631578947369, 0.0876068376068376, 0.08958333333333333, 0.09756097560975623, 0.10515873015873028, 0.10852713178294573, 0.11363636363636363, 0.12222222222222222, 0.1358695652173913, 0.14539007092198591, 0.15624999999999997, 0.16666666666666666, 0.17333333333333334]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.09716599190283401, 0.16835016835016836, 0.17333333333333334], "coverage": [0.0, 0.0, 0.0, 0.8233333333333334, 0.99, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7625717276674938, "basin/entropy": 0.23485964640198512, "basin/dispersion": 0.534080334987593, "energy/mean": 0.44639810794044665, "energy/min": 0.4427341811414392, "energy/std": 0.5076574131513648, "curv/lmax_mean": 0.48588709677419356, "curv/lmax_best": 0.5200352822580645, "curv/trace_mean": 0.6061569478908189, "curv/trace_best": 0.5317249534739454, "dynamics/steps": 0.5, "dynamics/monotonic": 0.4070932847394541, "dynamics/drop": 0.3674782878411911, "dynamics/residual": 0.5121355459057072}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [5, 0, 0, 13, 0, 0, 19, 0, 0, 0, 3, 0, 0, 22, 0, 0, 47, 0, 0, 387], "incorrect_counts": [9, 0, 0, 9, 0, 0, 7, 0, 0, 0, 13, 0, 0, 21, 0, 0, 15, 0, 0, 30]}, "basin/entropy": {"edges": [0.0, 0.05057021323536759, 0.10114042647073518, 0.15171063970610277, 0.20228085294147036, 0.25285106617683795, 0.30342127941220554, 0.3539914926475731, 0.4045617058829407, 0.4551319191183083, 0.5057021323536759, 0.5562723455890435, 0.6068425588244111, 0.6574127720597787, 0.7079829852951462, 0.7585531985305138, 0.8091234117658814, 0.859693625001249, 0.9102638382366166, 0.9608340514719842, 1.0114042647073518], "correct_counts": [387, 0, 0, 0, 0, 47, 0, 0, 21, 0, 0, 4, 18, 15, 0, 0, 1, 1, 2, 0], "incorrect_counts": [30, 0, 0, 0, 0, 15, 0, 0, 19, 0, 0, 12, 5, 15, 3, 0, 1, 1, 1, 2]}, "basin/dispersion": {"edges": [1.5954158897257813, 1.616709448349816, 1.6380030069738507, 1.6592965655978853, 1.6805901242219201, 1.7018836828459547, 1.7231772414699895, 1.744470800094024, 1.765764358718059, 1.7870579173420935, 1.8083514759661283, 1.829645034590163, 1.8509385932141975, 1.8722321518382323, 1.893525710462267, 1.9148192690863017, 1.9361128277103363, 1.9574063863343711, 1.9786999449584057, 1.9999935035824405, 2.021287062206475], "correct_counts": [1, 1, 0, 4, 6, 16, 30, 35, 42, 67, 63, 66, 65, 40, 25, 10, 13, 6, 4, 2], "incorrect_counts": [0, 1, 1, 1, 7, 4, 8, 5, 10, 14, 5, 13, 12, 9, 5, 2, 3, 0, 3, 1]}, "energy/mean": {"edges": [0.7320398216446241, 0.7937728473295769, 0.8555058730145296, 0.9172388986994824, 0.978971924384435, 1.040704950069388, 1.1024379757543405, 1.1641710014392932, 1.225904027124246, 1.2876370528091987, 1.3493700784941516, 1.4111031041791042, 1.4728361298640569, 1.5345691555490097, 1.5963021812339624, 1.6580352069189153, 1.719768232603868, 1.7815012582888206, 1.8432342839737732, 1.9049673096587263, 1.9667003353436787], "correct_counts": [0, 3, 19, 23, 31, 21, 28, 15, 17, 19, 28, 29, 59, 64, 70, 36, 15, 11, 6, 2], "incorrect_counts": [1, 2, 0, 3, 5, 6, 3, 4, 8, 5, 7, 5, 9, 9, 2, 13, 9, 6, 5, 2]}, "energy/min": {"edges": [0.34151938557624817, 0.41071730107069016, 0.47991521656513214, 0.5491131320595741, 0.6183110475540161, 0.6875089630484581, 0.7567068785429001, 0.8259047940373421, 0.8951027095317841, 0.964300625026226, 1.033498540520668, 1.10269645601511, 1.171894371509552, 1.241092287003994, 1.310290202498436, 1.379488117992878, 1.44868603348732, 1.517883948981762, 1.587081864476204, 1.656279779970646, 1.725477695465088], "correct_counts": [2, 5, 4, 11, 34, 27, 26, 28, 18, 22, 40, 40, 62, 70, 49, 27, 17, 9, 5, 0], "incorrect_counts": [1, 2, 1, 0, 2, 3, 11, 3, 9, 5, 9, 7, 7, 8, 6, 10, 15, 3, 1, 1]}, "energy/std": {"edges": [0.07718251081341526, 0.0912297277006377, 0.10527694458786013, 0.11932416147508257, 0.133371378362305, 0.14741859524952744, 0.16146581213674988, 0.1755130290239723, 0.18956024591119475, 0.20360746279841718, 0.21765467968563962, 0.23170189657286205, 0.2457491134600845, 0.2597963303473069, 0.27384354723452936, 0.2878907641217518, 0.30193798100897423, 0.31598519789619667, 0.3300324147834191, 0.34407963167064154, 0.358126848557864], "correct_counts": [0, 2, 4, 17, 27, 48, 55, 74, 52, 52, 46, 41, 30, 14, 17, 9, 2, 3, 1, 2], "incorrect_counts": [1, 1, 1, 4, 8, 12, 12, 8, 8, 11, 12, 7, 6, 2, 6, 1, 3, 0, 1, 0]}, "curv/lmax_mean": {"edges": [0.9986266444126765, 0.9987140096724033, 0.9988013749321302, 0.998888740191857, 0.9989761054515838, 0.9990634707113106, 0.9991508359710375, 0.9992382012307643, 0.9993255664904912, 0.9994129317502181, 0.9995002970099449, 0.9995876622696717, 0.9996750275293985, 0.9997623927891254, 0.9998497580488522, 0.9999371233085791, 1.0000244885683058, 1.0001118538280327, 1.0001992190877596, 1.0002865843474864, 1.0003739496072133], "correct_counts": [2, 1, 3, 6, 10, 12, 20, 35, 51, 75, 77, 73, 56, 41, 19, 6, 4, 4, 1, 0], "incorrect_counts": [0, 0, 0, 2, 2, 1, 5, 4, 10, 16, 17, 18, 18, 8, 2, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.9928669929504395, 0.9932882905006408, 0.9937095880508423, 0.9941308856010437, 0.9945521831512452, 0.9949734807014465, 0.9953947782516479, 0.9958160758018494, 0.9962373733520508, 0.9966586709022522, 0.9970799684524536, 0.997501266002655, 0.9979225635528565, 0.9983438611030578, 0.9987651586532593, 0.9991864562034607, 0.9996077537536621, 1.0000290513038634, 1.000450348854065, 1.0008716464042664, 1.0012929439544678], "correct_counts": [1, 0, 0, 0, 1, 0, 0, 2, 1, 5, 7, 6, 8, 14, 28, 87, 328, 4, 1, 3], "incorrect_counts": [0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 4, 3, 5, 20, 68, 0, 0, 0]}, "curv/trace_mean": {"edges": [31.902789910634358, 31.907989327112833, 31.913188743591306, 31.91838816006978, 31.923587576548258, 31.928786993026733, 31.93398640950521, 31.93918582598368, 31.944385242462157, 31.949584658940633, 31.954784075419106, 31.95998349189758, 31.965182908376057, 31.970382324854533, 31.97558174133301, 31.98078115781148, 31.985980574289957, 31.991179990768433, 31.996379407246906, 32.00157882372538, 32.00677824020386], "correct_counts": [0, 0, 0, 4, 6, 5, 11, 17, 24, 45, 66, 88, 71, 64, 27, 13, 23, 20, 7, 5], "incorrect_counts": [1, 1, 4, 2, 5, 4, 4, 8, 8, 6, 11, 10, 10, 8, 4, 8, 6, 3, 0, 1]}, "curv/trace_best": {"edges": [31.815509796142578, 31.82562713623047, 31.83574447631836, 31.84586181640625, 31.85597915649414, 31.86609649658203, 31.876213836669923, 31.886331176757814, 31.8964485168457, 31.906565856933593, 31.916683197021484, 31.926800537109376, 31.936917877197267, 31.947035217285155, 31.957152557373046, 31.967269897460938, 31.97738723754883, 31.98750457763672, 31.997621917724608, 32.0077392578125, 32.01785659790039], "correct_counts": [1, 0, 0, 0, 2, 2, 2, 2, 1, 10, 15, 25, 32, 68, 93, 93, 73, 53, 20, 4], "incorrect_counts": [0, 0, 0, 0, 0, 3, 1, 1, 1, 2, 7, 6, 9, 12, 12, 18, 15, 14, 2, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [496, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [104, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7266666666666667, 0.7319166666666667, 0.7371666666666666, 0.7424166666666667, 0.7476666666666667, 0.7529166666666667, 0.7581666666666667, 0.7634166666666666, 0.7686666666666667, 0.7739166666666667, 0.7791666666666667, 0.7844166666666667, 0.7896666666666667, 0.7949166666666667, 0.8001666666666667, 0.8054166666666667, 0.8106666666666666, 0.8159166666666666, 0.8211666666666667, 0.8264166666666667, 0.8316666666666667], "correct_counts": [2, 5, 11, 25, 31, 42, 83, 79, 57, 62, 44, 32, 17, 1, 5, 0, 0, 0, 0, 0], "incorrect_counts": [1, 0, 6, 3, 7, 7, 7, 10, 13, 9, 14, 9, 4, 7, 1, 0, 4, 0, 0, 2]}, "dynamics/drop": {"edges": [67.27662276228268, 83.98704353620607, 100.69746431012948, 117.40788508405288, 134.11830585797628, 150.82872663189966, 167.53914740582306, 184.24956817974646, 200.95998895366984, 217.67040972759327, 234.38083050151664, 251.09125127544002, 267.80167204936345, 284.5120928232868, 301.2225135972102, 317.9329343711336, 334.643355145057, 351.3537759189804, 368.0641966929038, 384.7746174668272, 401.4850382407506], "correct_counts": [20, 48, 122, 170, 88, 26, 8, 3, 3, 2, 3, 1, 1, 1, 0, 0, 0, 0, 0, 0], "incorrect_counts": [1, 9, 21, 17, 24, 7, 4, 6, 4, 1, 2, 2, 2, 2, 0, 0, 0, 1, 0, 1]}, "dynamics/residual": {"edges": [1.1687835951646168, 1.181517025331656, 1.1942504554986952, 1.2069838856657344, 1.2197173158327739, 1.232450745999813, 1.2451841761668523, 1.2579176063338915, 1.2706510365009307, 1.28338446666797, 1.296117896835009, 1.3088513270020485, 1.3215847571690877, 1.334318187336127, 1.3470516175031662, 1.3597850476702054, 1.3725184778372446, 1.385251908004284, 1.3979853381713232, 1.4107187683383624, 1.4234521985054016], "correct_counts": [1, 3, 3, 8, 17, 32, 32, 45, 51, 56, 56, 49, 37, 46, 23, 13, 9, 10, 2, 3], "incorrect_counts": [2, 1, 0, 2, 4, 7, 7, 11, 5, 10, 13, 12, 10, 7, 8, 1, 2, 1, 1, 0]}}}, "ece_geometry": 0.04766602442020401}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "46752558c5df", "timestamp": "2026-07-27T10:01:46.609759+00:00", "git_sha": "5207145", "config_hash": "c6246ddaef7e", "config": {"run": {"seed": 0, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.82, "base_error": 0.18, "final_train_loss": 1.010847568511963, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.0784767176926761, "rho_basin": 0.12751482332802438, "energy_min": 0.16347101991147492, "energy_mean": 0.1708695950340498, "energy_std": 0.1652682650135063, "msp": 0.07413344872937365, "temp_msp": 0.0954664480423542, "entropy": 0.07111874477317347}, "temperature": 0.370153554472176, "best_energy_baseline": "energy_min", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.08499430221879882, 0.05443909813421712, 0.11823769217310283], "delta_aurc_vs_best_energy": [0.08499430221879882, 0.05443909813421712, 0.11823769217310283], "delta_aurc_vs_best_baseline": [-0.007357972919502631, -0.027934250128952055, 0.012642038074947301], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.0625, 0.05555555555555555, 0.05833333333333334, 0.05303030303030303, 0.05555555555555555, 0.05128205128205128, 0.05357142857142857, 0.049999999999999996, 0.046875, 0.04411764705882353, 0.060185185185185175, 0.06140350877192982, 0.058333333333333334, 0.055555555555555546, 0.05303030303030303, 0.057971014492753624, 0.05902777777777777, 0.056666666666666664, 0.05448717948717949, 0.06481481481481481, 0.06845238095238096, 0.0718390804597701, 0.07222222222222222, 0.07795698924731183, 0.08333333333333333, 0.08585858585858586, 0.09558823529411764, 0.09523809523809522, 0.09722222222222221, 0.1036036036036036, 0.1074561403508772, 0.11538461538461539, 0.125, 0.12398373983739835, 0.12896825396825395, 0.13565891472868216, 0.14015151515151514, 0.14814814814814814, 0.15217391304347827, 0.15780141843971643, 0.16666666666666674, 0.1717687074829932, 0.18]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.12009237875288685, 0.12009237875288688, 0.12009237875288686, 0.12009237875288677, 0.12009237875288671, 0.12009237875288667, 0.12009237875288664, 0.12009237875288663, 0.1200923787528866, 0.1200923787528866, 0.12009237875288659, 0.12009237875288657, 0.12009237875288657, 0.12009237875288656, 0.12009237875288656, 0.12009237875288654, 0.12009237875288654, 0.12009237875288654, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288653, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.12009237875288652, 0.1200923787528865, 0.1200923787528865, 0.1233108108108105, 0.12664473684210495, 0.129807692307692, 0.1328124999999997, 0.13630999213217915, 0.14151305683563736, 0.14647411852963235, 0.14978590250329377, 0.1522544283413848, 0.15703227931488792, 0.16239522888459046, 0.16790674603174596, 0.17360020931449494, 0.1799999999999998]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.25, 0.2222222222222222, 0.1875, 0.16666666666666666, 0.16666666666666663, 0.1547619047619048, 0.15625, 0.16666666666666666, 0.15833333333333335, 0.1590909090909091, 0.1527777777777778, 0.15384615384615385, 0.15476190476190477, 0.14999999999999997, 0.15104166666666666, 0.16666666666666666, 0.162037037037037, 0.16228070175438597, 0.16666666666666666, 0.1587301587301587, 0.16287878787878787, 0.16666666666666666, 0.16319444444444453, 0.16666666666666666, 0.1762820512820513, 0.1728395061728395, 0.17857142857142858, 0.17816091954022986, 0.17777777777777778, 0.17204301075268819, 0.16666666666666666, 0.16161616161616163, 0.1568627450980392, 0.1571428571428571, 0.15509259259259256, 0.1554054054054054, 0.15789473684210525, 0.15598290598290598, 0.16041666666666668, 0.1605691056910569, 0.1587301587301587, 0.16279069767441862, 0.16477272727272727, 0.16666666666666666, 0.17028985507246377, 0.17198581560283685, 0.1736111111111112, 0.17857142857142858, 0.18]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.16666666666666666, 0.18333333333333332, 0.19444444444444442, 0.1904761904761903, 0.1875, 0.18518518518518517, 0.17500000000000002, 0.18181818181818182, 0.16666666666666666, 0.16666666666666666, 0.17261904761904762, 0.16666666666666663, 0.16666666666666666, 0.16176470588235295, 0.16666666666666663, 0.16666666666666666, 0.16666666666666666, 0.17460317460317457, 0.1856060606060606, 0.18478260869565216, 0.18402777777777776, 0.18, 0.18269230769230768, 0.17901234567901234, 0.17857142857142858, 0.17528735632183906, 0.17222222222222222, 0.1693548387096774, 0.16666666666666666, 0.16414141414141414, 0.15931372549019607, 0.15476190476190474, 0.15509259259259256, 0.15315315315315314, 0.15570175438596492, 0.15598290598290598, 0.15625, 0.15650406504065037, 0.1587301587301587, 0.16472868217054262, 0.16666666666666666, 0.1685185185185185, 0.17028985507246377, 0.1684397163120567, 0.17534722222222218, 0.17687074829931973, 0.18]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.1388888888888889, 0.16666666666666666, 0.15, 0.12499999999999999, 0.11904761904761907, 0.13541666666666666, 0.12037037037037036, 0.11666666666666668, 0.12121212121212122, 0.14583333333333334, 0.15384615384615385, 0.16666666666666666, 0.1611111111111111, 0.15104166666666666, 0.1568627450980392, 0.15740740740740738, 0.16228070175438597, 0.1625, 0.16269841269841268, 0.17045454545454544, 0.17391304347826086, 0.17361111111111108, 0.17666666666666667, 0.1794871794871795, 0.1728395061728395, 0.16964285714285715, 0.16666666666666663, 0.16666666666666666, 0.17473118279569894, 0.171875, 0.17424242424242425, 0.1715686274509804, 0.16904761904761903, 0.16898148148148145, 0.17117117117117117, 0.17105263157894737, 0.17307692307692307, 0.175, 0.1768292682926829, 0.1746031746031747, 0.17635658914728683, 0.17234848484848486, 0.17407407407407408, 0.1721014492753623, 0.17375886524822692, 0.17708333333333331, 0.17687074829931973, 0.18]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.04761904761904762, 0.041666666666666664, 0.037037037037037035, 0.04166666666666667, 0.03787878787878788, 0.041666666666666664, 0.038461538461538464, 0.041666666666666664, 0.03888888888888888, 0.041666666666666664, 0.04411764705882353, 0.04629629629629629, 0.04824561403508772, 0.05, 0.05158730158730158, 0.05303030303030303, 0.050724637681159424, 0.055555555555555546, 0.06333333333333334, 0.0641025641025641, 0.06172839506172839, 0.0625, 0.06321839080459789, 0.06944444444444445, 0.07526881720430108, 0.08072916666666667, 0.08585858585858586, 0.09313725490196079, 0.0976190476190476, 0.10185185185185183, 0.10585585585585586, 0.1118421052631579, 0.11538461538461539, 0.11875, 0.12398373983739835, 0.12896825396825395, 0.12790697674418605, 0.13446969696969696, 0.1425925925925926, 0.1431159420289855, 0.14893617021276606, 0.1562500000000001, 0.16666666666666666, 0.18]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.08333333333333333, 0.08333333333333333, 0.0625, 0.05, 0.055555555555555546, 0.04761904761904762, 0.0625, 0.05555555555555555, 0.05833333333333334, 0.09090909090909091, 0.08333333333333333, 0.07692307692307693, 0.07738095238095238, 0.07777777777777777, 0.08333333333333333, 0.08823529411764706, 0.08333333333333331, 0.07894736842105263, 0.075, 0.07142857142857141, 0.07575757575757576, 0.07971014492753623, 0.08680555555555554, 0.08666666666666667, 0.08333333333333333, 0.08950617283950617, 0.08928571428571429, 0.09482758620689653, 0.1, 0.09946236559139784, 0.09895833333333333, 0.10353535353535354, 0.10049019607843138, 0.10476190476190475, 0.10648148148148147, 0.11036036036036036, 0.11403508771929824, 0.11538461538461539, 0.12083333333333333, 0.1219512195121951, 0.13095238095238093, 0.13372093023255813, 0.14393939393939395, 0.15185185185185185, 0.1539855072463768, 0.15780141843971643, 0.16666666666666663, 0.1717687074829932, 0.18]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.041666666666666664, 0.04487179487179487, 0.05357142857142857, 0.055555555555555546, 0.052083333333333336, 0.05392156862745098, 0.055555555555555546, 0.05701754385964912, 0.0625, 0.06746031746031744, 0.06818181818181818, 0.06521739130434782, 0.06250000000000011, 0.07, 0.0673076923076923, 0.06481481481481481, 0.06547619047619048, 0.0689655172413793, 0.06666666666666667, 0.06989247311827956, 0.06770833333333333, 0.06565656565656566, 0.06862745098039216, 0.06904761904761904, 0.07638888888888888, 0.08108108108108109, 0.08333333333333333, 0.09188034188034189, 0.09791666666666667, 0.1138211382113821, 0.11706349206349217, 0.12596899224806202, 0.13636363636363635, 0.14444444444444443, 0.1503623188405797, 0.15602836879432636, 0.15798611111111108, 0.1683673469387755, 0.18]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.17016647548003197, "coverage": 0.6816666666666666, "abstain_rate": 0.31833333333333336, "selective_risk": 0.09535452322738386, "selective_accuracy": 0.9046454767726162, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.09535452322738386, 0.12896825396825398, 0.17147707979626486, 0.18030050083472454], "coverage": [0.0, 0.0, 0.6816666666666666, 0.84, 0.9816666666666667, 0.9983333333333333]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.657397997591087, "basin/entropy": 0.3412375790424571, "basin/dispersion": 0.5489686841312857, "energy/mean": 0.4601588376994881, "energy/min": 0.45007151460403494, "energy/std": 0.4628124059018368, "curv/lmax_mean": 0.4898562180066245, "curv/lmax_best": 0.4666892502258356, "curv/trace_mean": 0.4625301114122252, "curv/trace_best": 0.4725986148750376, "dynamics/steps": 0.5, "dynamics/monotonic": 0.3500639867509786, "dynamics/drop": 0.3683566696778079, "dynamics/residual": 0.549439174947305}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [1, 0, 0, 0, 0, 6, 0, 12, 0, 13, 0, 0, 17, 0, 20, 0, 0, 42, 0, 381], "incorrect_counts": [0, 0, 0, 0, 0, 7, 0, 9, 0, 9, 0, 0, 6, 0, 11, 0, 0, 14, 0, 52]}, "basin/entropy": {"edges": [0.0, 0.06789889274936621, 0.13579778549873242, 0.20369667824809862, 0.27159557099746484, 0.33949446374683107, 0.40739335649619723, 0.47529224924556346, 0.5431911419949297, 0.6110900347442959, 0.6789889274936621, 0.7468878202430284, 0.8147867129923945, 0.8826856057417607, 0.9505844984911269, 1.0184833912404931, 1.0863822839898594, 1.1542811767392256, 1.2221800694885918, 1.290078962237958, 1.3579778549873243], "correct_counts": [381, 0, 0, 0, 42, 0, 20, 0, 16, 13, 17, 0, 0, 1, 0, 1, 0, 0, 0, 1], "incorrect_counts": [52, 0, 0, 0, 14, 0, 10, 0, 5, 9, 15, 0, 0, 0, 2, 1, 0, 0, 0, 0]}, "basin/dispersion": {"edges": [1.6075309204036172, 1.6282759139338974, 1.6490209074641777, 1.6697659009944579, 1.690510894524738, 1.7112558880550184, 1.7320008815852985, 1.7527458751155787, 1.773490868645859, 1.7942358621761392, 1.8149808557064193, 1.8357258492366997, 1.8564708427669798, 1.87721583629726, 1.8979608298275403, 1.9187058233578205, 1.9394508168881006, 1.960195810418381, 1.9809408039486611, 2.0016857974789413, 2.0224307910092216], "correct_counts": [1, 1, 4, 3, 13, 17, 25, 42, 40, 69, 70, 48, 50, 34, 27, 22, 12, 9, 3, 2], "incorrect_counts": [0, 1, 1, 0, 2, 4, 11, 8, 18, 13, 9, 10, 8, 13, 5, 3, 2, 0, 0, 0]}, "energy/mean": {"edges": [-0.2607540686925252, -0.20895610700050987, -0.15715814530849456, -0.10536018361647925, -0.053562221924463915, -0.0017642602324485779, 0.050033701459566704, 0.10183166315158204, 0.15362962484359738, 0.20542758653561272, 0.25722554822762805, 0.3090235099196434, 0.3608214716116586, 0.41261943330367395, 0.4644173949956893, 0.5162153566877046, 0.56801331837972, 0.6198112800717352, 0.6716092417637507, 0.7234072034557659, 0.7752051651477814], "correct_counts": [11, 39, 91, 75, 69, 24, 21, 10, 17, 13, 9, 15, 8, 15, 15, 13, 13, 16, 10, 8], "incorrect_counts": [2, 10, 17, 21, 11, 3, 0, 1, 1, 1, 2, 4, 2, 2, 9, 3, 5, 5, 5, 4]}, "energy/min": {"edges": [-0.6984671354293823, -0.635711333155632, -0.5729555308818817, -0.5101997286081315, -0.4474439263343811, -0.3846881240606308, -0.3219323217868805, -0.25917651951313025, -0.19642071723937993, -0.1336649149656296, -0.07090911269187927, -0.008153310418129056, 0.05460249185562127, 0.1173582941293716, 0.18011409640312182, 0.24286989867687214, 0.30562570095062247, 0.3683815032243727, 0.4311373054981231, 0.49389310777187334, 0.5566489100456238], "correct_counts": [2, 4, 26, 59, 81, 84, 40, 30, 21, 19, 14, 15, 14, 20, 14, 20, 12, 8, 7, 2], "incorrect_counts": [0, 0, 9, 10, 15, 19, 11, 0, 0, 3, 3, 4, 4, 2, 9, 6, 5, 5, 1, 2]}, "energy/std": {"edges": [0.07567243770358024, 0.09125026233767754, 0.10682808697177484, 0.12240591160587214, 0.13798373623996943, 0.15356156087406675, 0.16913938550816404, 0.18471721014226133, 0.20029503477635863, 0.21587285941045592, 0.2314506840445532, 0.2470285086786505, 0.2626063333127478, 0.2781841579468451, 0.29376198258094244, 0.30933980721503973, 0.324917631849137, 0.3404954564832343, 0.3560732811173316, 0.3716511057514289, 0.3872289303855262], "correct_counts": [2, 3, 17, 20, 45, 54, 69, 72, 55, 52, 34, 33, 21, 9, 2, 2, 1, 0, 0, 1], "incorrect_counts": [0, 3, 0, 6, 4, 16, 11, 18, 12, 12, 9, 4, 8, 1, 1, 1, 1, 0, 1, 0]}, "curv/lmax_mean": {"edges": [1.0000369946161907, 1.0007940803964934, 1.001551166176796, 1.0023082519570987, 1.0030653377374015, 1.003822423517704, 1.0045795092980068, 1.0053365950783095, 1.006093680858612, 1.0068507666389148, 1.0076078524192176, 1.0083649381995201, 1.0091220239798229, 1.0098791097601254, 1.0106361955404282, 1.011393281320731, 1.0121503671010335, 1.0129074528813362, 1.013664538661639, 1.0144216244419415, 1.0151787102222443], "correct_counts": [255, 60, 35, 22, 22, 20, 20, 14, 13, 10, 9, 1, 2, 2, 2, 1, 3, 0, 0, 1], "incorrect_counts": [51, 28, 7, 9, 2, 5, 1, 2, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0017322301864624, 1.0034644603729248, 1.0051966905593872, 1.0069289207458496, 1.008661150932312, 1.0103933811187744, 1.0121256113052368, 1.0138578414916992, 1.0155900716781616, 1.017322301864624, 1.0190545320510864, 1.0207867622375488, 1.0225189924240112, 1.0242512226104736, 1.025983452796936, 1.0277156829833984, 1.0294479131698608, 1.0311801433563232, 1.0329123735427856, 1.034644603729248], "correct_counts": [373, 47, 26, 9, 6, 6, 3, 6, 6, 3, 2, 2, 0, 0, 2, 0, 0, 0, 0, 1], "incorrect_counts": [84, 12, 6, 2, 2, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00349839528402, 32.00748820304871, 32.0114780108134, 32.01546781857809, 32.01945762634278, 32.02344743410747, 32.02743724187215, 32.03142704963684, 32.03541685740153, 32.03940666516622, 32.04339647293091, 32.0473862806956, 32.05137608846029, 32.05536589622498, 32.05935570398967, 32.06334551175435, 32.06733531951904, 32.07132512728373, 32.07531493504842, 32.07930474281311, 32.0832945505778], "correct_counts": [74, 97, 58, 30, 24, 24, 20, 19, 19, 16, 18, 13, 24, 9, 16, 9, 8, 7, 6, 1], "incorrect_counts": [3, 11, 28, 16, 9, 10, 4, 3, 4, 6, 2, 0, 0, 2, 3, 0, 4, 2, 0, 1]}, "curv/trace_best": {"edges": [32.0009880065918, 32.010338592529294, 32.0196891784668, 32.029039764404295, 32.0383903503418, 32.0477409362793, 32.057091522216794, 32.0664421081543, 32.075792694091795, 32.0851432800293, 32.0944938659668, 32.103844451904294, 32.1131950378418, 32.122545623779295, 32.1318962097168, 32.1412467956543, 32.150597381591794, 32.1599479675293, 32.169298553466795, 32.1786491394043, 32.1879997253418], "correct_counts": [159, 110, 61, 42, 43, 24, 16, 11, 6, 7, 3, 2, 2, 0, 2, 1, 1, 1, 0, 1], "incorrect_counts": [23, 33, 20, 12, 5, 1, 3, 1, 3, 3, 0, 2, 0, 1, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [492, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [108, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7333333333333334, 0.7383333333333334, 0.7433333333333334, 0.7483333333333334, 0.7533333333333334, 0.7583333333333334, 0.7633333333333334, 0.7683333333333334, 0.7733333333333334, 0.7783333333333334, 0.7833333333333334, 0.7883333333333334, 0.7933333333333334, 0.7983333333333335, 0.8033333333333335, 0.8083333333333335, 0.8133333333333335, 0.8183333333333335, 0.8233333333333335, 0.8283333333333335, 0.8333333333333335], "correct_counts": [4, 18, 16, 37, 64, 65, 60, 89, 54, 37, 23, 15, 6, 2, 1, 0, 1, 0, 0, 0], "incorrect_counts": [1, 3, 3, 7, 5, 9, 9, 13, 14, 13, 10, 6, 2, 4, 2, 2, 1, 2, 1, 1]}, "dynamics/drop": {"edges": [74.99134453634422, 87.32837261632085, 99.6654006962975, 112.00242877627413, 124.33945685625076, 136.67648493622738, 149.01351301620406, 161.35054109618068, 173.6875691761573, 186.02459725613394, 198.3616253361106, 210.6986534160872, 223.0356814960639, 235.3727095760405, 247.70973765601713, 260.04676573599374, 272.3837938159704, 284.72082189594704, 297.05784997592366, 309.39487805590034, 321.73190613587695], "correct_counts": [16, 27, 71, 89, 118, 93, 44, 17, 11, 1, 2, 1, 2, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [1, 7, 16, 10, 15, 12, 15, 7, 5, 3, 1, 1, 4, 3, 3, 1, 0, 3, 0, 1]}, "dynamics/residual": {"edges": [1.1565301517645519, 1.1707259138425192, 1.1849216759204866, 1.199117437998454, 1.213313200076421, 1.2275089621543884, 1.2417047242323558, 1.2559004863103231, 1.2700962483882905, 1.2842920104662578, 1.2984877725442252, 1.3126835346221923, 1.3268792967001597, 1.341075058778127, 1.3552708208560944, 1.3694665829340618, 1.3836623450120291, 1.3978581070899962, 1.4120538691679636, 1.426249631245931, 1.4404453933238983], "correct_counts": [2, 3, 8, 6, 16, 35, 42, 37, 52, 52, 62, 48, 46, 36, 22, 16, 6, 1, 0, 2], "incorrect_counts": [1, 0, 0, 1, 4, 7, 14, 15, 8, 14, 9, 15, 11, 7, 1, 0, 1, 0, 0, 0]}}}, "feature_ablation": {"full": 0.0784767176926761, "drop_basin": 0.10372467534211892, "basin_only": 0.10557222478623578, "drop_energy": 0.08127342129547027, "energy_only": 0.17025474694687504, "drop_curv": 0.0818413107632023, "curv_only": 0.1285684647317494, "drop_dynamics": 0.09343312053024784, "dynamics_only": 0.1420026845893754}, "ece_geometry": 0.05242852914544991}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "b8dc1a068df5", "timestamp": "2026-07-27T10:01:49.305415+00:00", "git_sha": "5207145", "config_hash": "e65ef9e88f92", "config": {"run": {"seed": 1, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.8066666666666666, "base_error": 0.19333333333333333, "final_train_loss": 0.9732173085212708, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.07459630695723383, "rho_basin": 0.09995473471609245, "energy_min": 0.18564962420122708, "energy_mean": 0.18679968604822153, "energy_std": 0.1962536701726144, "msp": 0.05417632514228234, "temp_msp": 0.05699145581322151, "entropy": 0.07471040623687418}, "temperature": 0.40914505573299365, "best_energy_baseline": "energy_min", "best_baseline": "msp", "delta_aurc_vs_energy_min": [0.11105331724399324, 0.07109286586589932, 0.149723039243916], "delta_aurc_vs_best_energy": [0.11105331724399324, 0.07109286586589932, 0.149723039243916], "delta_aurc_vs_best_baseline": [-0.020419981814951492, -0.03976815367328431, -0.002945817690407804], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.0625, 0.05, 0.04166666666666666, 0.04761904761904762, 0.07291666666666667, 0.06481481481481481, 0.06666666666666668, 0.06060606060606061, 0.0625, 0.057692307692307696, 0.05357142857142857, 0.055555555555555546, 0.052083333333333336, 0.05392156862745098, 0.055555555555555546, 0.05263157894736842, 0.05, 0.04761904761904761, 0.04924242424242424, 0.04710144927536232, 0.048611111111111105, 0.05, 0.04807692307692308, 0.05555555555555555, 0.05654761904761905, 0.060344827586206885, 0.06111111111111111, 0.06182795698924731, 0.0625, 0.06565656565656566, 0.06862745098039216, 0.07142857142857141, 0.07407407407407421, 0.08333333333333333, 0.08552631578947369, 0.09188034188034189, 0.1, 0.1097560975609756, 0.11706349206349205, 0.12209302325581395, 0.12878787878787878, 0.13703703703703704, 0.15217391304347827, 0.1631205673758865, 0.17361111111111108, 0.18197278911564627, 0.19333333333333333]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0845410628019324, 0.08454106280193235, 0.08454106280193234, 0.08454106280193234, 0.08454106280193233, 0.08454106280193233, 0.08454106280193233, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08454106280193231, 0.08823129251700677, 0.09530423280423275, 0.10199485199485187, 0.10833333333333317, 0.11434676434676416, 0.1200595238095236, 0.12340301974448298, 0.1255668934240361, 0.1296603912882981, 0.13753607503607498, 0.147962962962963, 0.15996376811594212, 0.16849691146190815, 0.17641129032258074, 0.18435374149659864, 0.19333333333333322]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.19444444444444445, 0.14583333333333334, 0.13333333333333333, 0.15277777777777776, 0.1547619047619048, 0.16666666666666666, 0.1574074074074074, 0.16666666666666655, 0.19696969696969696, 0.1875, 0.1858974358974359, 0.19642857142857142, 0.19444444444444442, 0.19270833333333334, 0.18627450980392157, 0.18518518518518515, 0.17543859649122806, 0.1875, 0.19444444444444442, 0.19696969696969696, 0.19202898550724637, 0.18749999999999997, 0.18333333333333332, 0.1794871794871795, 0.17592592592592593, 0.17857142857142858, 0.17241379310344845, 0.175, 0.17473118279569894, 0.17447916666666666, 0.18181818181818182, 0.17647058823529413, 0.1714285714285714, 0.17361111111111108, 0.17567567567567569, 0.18201754385964913, 0.18376068376068377, 0.18333333333333332, 0.18495934959349591, 0.1884920634920636, 0.187984496124031, 0.1875, 0.18888888888888888, 0.19202898550724637, 0.19148936170212763, 0.19270833333333331, 0.1921768707482993, 0.19333333333333333]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.20833333333333334, 0.2, 0.18055555555555552, 0.2023809523809524, 0.20833333333333334, 0.2037037037037037, 0.2166666666666667, 0.20454545454545456, 0.19444444444444445, 0.19230769230769232, 0.19642857142857142, 0.1833333333333333, 0.18229166666666666, 0.18627450980392157, 0.1898148148148148, 0.19298245614035087, 0.1875, 0.18650793650793648, 0.1893939393939394, 0.18840579710144928, 0.18402777777777776, 0.18, 0.1794871794871795, 0.18209876543209877, 0.18154761904761904, 0.18390804597701146, 0.18055555555555555, 0.18010752688172044, 0.17708333333333334, 0.17424242424242425, 0.17892156862745098, 0.18333333333333346, 0.18518518518518515, 0.18693693693693694, 0.18859649122807018, 0.1858974358974359, 0.18541666666666667, 0.18699186991869915, 0.1845238095238095, 0.18023255813953487, 0.18371212121212122, 0.18703703703703703, 0.19021739130434784, 0.1932624113475177, 0.19270833333333331, 0.19387755102040816, 0.19333333333333333]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.1111111111111111, 0.22916666666666666, 0.2, 0.2222222222222223, 0.23809523809523814, 0.22916666666666666, 0.2037037037037037, 0.19166666666666657, 0.17424242424242425, 0.1875, 0.20512820512820512, 0.19047619047619047, 0.1944444444444446, 0.203125, 0.20588235294117646, 0.20370370370370366, 0.19736842105263158, 0.1875, 0.19841269841269837, 0.19318181818181818, 0.2028985507246377, 0.20486111111111108, 0.20666666666666667, 0.20192307692307693, 0.20987654320987653, 0.20535714285714285, 0.2126436781609195, 0.20833333333333334, 0.21236559139784947, 0.20833333333333334, 0.20959595959595959, 0.2107843137254902, 0.20714285714285713, 0.20370370370370366, 0.20045045045045046, 0.19736842105263158, 0.19658119658119658, 0.19583333333333333, 0.1930894308943089, 0.19444444444444442, 0.1937984496124031, 0.19318181818181818, 0.18888888888888888, 0.19021739130434784, 0.18794326241134748, 0.18749999999999997, 0.1921768707482993, 0.19333333333333333]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.010416666666666666, 0.00980392156862745, 0.013888888888888886, 0.013157894736842105, 0.0125, 0.011904761904761902, 0.015151515151515152, 0.018115942028985508, 0.020833333333333447, 0.023333333333333334, 0.02564102564102564, 0.027777777777777776, 0.03571428571428571, 0.04022988505747126, 0.044444444444444446, 0.051075268817204304, 0.0625, 0.06818181818181818, 0.0784313725490196, 0.09047619047619046, 0.09490740740740755, 0.1036036036036036, 0.10964912280701754, 0.1111111111111111, 0.11666666666666667, 0.1219512195121951, 0.1250000000000001, 0.14147286821705427, 0.14772727272727273, 0.1537037037037037, 0.16304347826086957, 0.17021276595744692, 0.1753472222222223, 0.18197278911564627, 0.19333333333333333]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761904, 0.01111111111111111, 0.020833333333333332, 0.024509803921568627, 0.023148148148148143, 0.021929824561403508, 0.020833333333333332, 0.027777777777777773, 0.030303030303030304, 0.028985507246376812, 0.03472222222222222, 0.04, 0.04487179487179487, 0.043209876543209874, 0.050595238095238096, 0.05747126436781608, 0.058333333333333334, 0.056451612903225805, 0.0703125, 0.07575757575757576, 0.08333333333333333, 0.0857142857142857, 0.09027777777777776, 0.0945945945945946, 0.09868421052631579, 0.10683760683760683, 0.11041666666666666, 0.1158536585365855, 0.12499999999999999, 0.1298449612403101, 0.13825757575757575, 0.1425925925925926, 0.15217391304347827, 0.16489361702127656, 0.17361111111111108, 0.1836734693877551, 0.19333333333333333]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.025000000000000005, 0.030303030303030304, 0.027777777777777776, 0.038461538461538464, 0.041666666666666664, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.04166666666666666, 0.039473684210526314, 0.04583333333333333, 0.04761904761904761, 0.045454545454545456, 0.04710144927536232, 0.055555555555555663, 0.06333333333333334, 0.060897435897435896, 0.06481481481481481, 0.06845238095238096, 0.07758620689655171, 0.08611111111111111, 0.09408602150537634, 0.09895833333333333, 0.09848484848484848, 0.10049019607843138, 0.10476190476190475, 0.10879629629629628, 0.11261261261261261, 0.1206140350877193, 0.12606837606837606, 0.13125, 0.14024390243902438, 0.14484126984126997, 0.1511627906976744, 0.1571969696969697, 0.1648148148148148, 0.16666666666666666, 0.17021276595744692, 0.17708333333333343, 0.1836734693877551, 0.19333333333333333]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.10636963445059894, "coverage": 0.505, "abstain_rate": 0.495, "selective_risk": 0.04950495049504951, "selective_accuracy": 0.9504950495049505, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.04950495049504951, 0.08993576017130621, 0.13543599257884972, 0.19333333333333333], "coverage": [0.0, 0.0, 0.505, 0.7783333333333333, 0.8983333333333333, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7604641635793673, "basin/entropy": 0.23897477913935594, "basin/dispersion": 0.48822670276432034, "energy/mean": 0.47834140780849244, "energy/min": 0.4678861499002565, "energy/std": 0.5175797948133372, "curv/lmax_mean": 0.5627315474494158, "curv/lmax_best": 0.5241967084639498, "curv/trace_mean": 0.5586972784269022, "curv/trace_best": 0.5594097321174124, "dynamics/steps": 0.5, "dynamics/monotonic": 0.38306853804502705, "dynamics/drop": 0.39514462809917356, "dynamics/residual": 0.4876567398119122}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [0, 0, 6, 0, 0, 14, 0, 0, 6, 0, 0, 11, 0, 0, 22, 0, 0, 46, 0, 379], "incorrect_counts": [1, 0, 9, 0, 0, 17, 0, 0, 14, 0, 0, 10, 0, 0, 6, 0, 0, 24, 0, 35]}, "basin/entropy": {"edges": [0.0, 0.05387781635334003, 0.10775563270668007, 0.16163344906002008, 0.21551126541336013, 0.2693890817667002, 0.32326689812004017, 0.3771447144733802, 0.43102253082672026, 0.4849003471800603, 0.5387781635334004, 0.5926559798867403, 0.6465337962400803, 0.7004116125934204, 0.7542894289467604, 0.8081672453001005, 0.8620450616534405, 0.9159228780067805, 0.9698006943601206, 1.0236785107134607, 1.0775563270668007], "correct_counts": [379, 0, 0, 0, 0, 46, 0, 0, 22, 0, 11, 6, 18, 0, 0, 0, 0, 2, 0, 0], "incorrect_counts": [35, 0, 0, 0, 0, 24, 0, 0, 6, 0, 10, 13, 22, 0, 0, 0, 2, 2, 0, 2]}, "basin/dispersion": {"edges": [1.6209795073409872, 1.6420014502019364, 1.663023393062886, 1.6840453359238352, 1.7050672787847847, 1.726089221645734, 1.7471111645066832, 1.7681331073676327, 1.789155050228582, 1.8101769930895315, 1.8311989359504808, 1.8522208788114303, 1.8732428216723795, 1.8942647645333288, 1.9152867073942783, 1.9363086502552276, 1.9573305931161769, 1.9783525359771263, 1.9993744788380756, 2.020396421699025, 2.0414183645599744], "correct_counts": [0, 4, 7, 7, 25, 26, 39, 52, 59, 48, 51, 51, 44, 28, 14, 18, 3, 7, 0, 1], "incorrect_counts": [1, 1, 0, 2, 8, 5, 6, 13, 12, 10, 20, 12, 7, 9, 3, 2, 1, 2, 1, 1]}, "energy/mean": {"edges": [-0.2875423381725947, -0.26760066399971644, -0.2476589898268382, -0.22771731565395992, -0.20777564148108166, -0.1878339673082034, -0.1678922931353251, -0.14795061896244685, -0.1280089447895686, -0.10806727061669033, -0.08812559644381207, -0.06818392227093378, -0.048242248098055523, -0.028300573925177264, -0.008358899752298976, 0.011582774420579256, 0.031524448593457544, 0.05146612276633583, 0.07140779693921406, 0.09134947111209235, 0.1112911452849706], "correct_counts": [2, 6, 7, 14, 26, 29, 50, 53, 58, 39, 53, 43, 31, 22, 17, 11, 8, 9, 2, 4], "incorrect_counts": [0, 1, 2, 4, 6, 9, 10, 12, 10, 9, 10, 13, 7, 4, 10, 3, 1, 3, 1, 1]}, "energy/min": {"edges": [-0.6702444553375244, -0.6411247402429581, -0.6120050251483917, -0.5828853100538254, -0.553765594959259, -0.5246458798646927, -0.49552616477012634, -0.46640644967556, -0.43728673458099365, -0.4081670194864273, -0.37904730439186096, -0.3499275892972946, -0.32080787420272827, -0.2916881591081619, -0.2625684440135956, -0.23344872891902924, -0.2043290138244629, -0.17520929872989655, -0.1460895836353302, -0.11696986854076385, -0.08785015344619751], "correct_counts": [0, 10, 6, 12, 28, 30, 32, 58, 62, 59, 54, 41, 27, 26, 17, 14, 3, 3, 1, 1], "incorrect_counts": [2, 0, 3, 2, 1, 9, 10, 12, 16, 8, 11, 14, 9, 9, 4, 3, 0, 2, 1, 0]}, "energy/std": {"edges": [0.0869744242281087, 0.10181870487610015, 0.1166629855240916, 0.13150726617208308, 0.1463515468200745, 0.161195827468066, 0.17604010811605744, 0.1908843887640489, 0.20572866941204035, 0.2205729500600318, 0.23541723070802326, 0.2502615113560147, 0.2651057920040062, 0.2799500726519977, 0.2947943532999891, 0.3096386339479805, 0.324482914595972, 0.3393271952439635, 0.3541714758919549, 0.3690157565399464, 0.3838600371879378], "correct_counts": [4, 8, 20, 26, 58, 54, 66, 59, 56, 35, 34, 35, 14, 6, 0, 7, 1, 0, 0, 1], "incorrect_counts": [1, 1, 4, 11, 10, 17, 18, 17, 9, 6, 8, 3, 3, 6, 1, 0, 0, 1, 0, 0]}, "curv/lmax_mean": {"edges": [1.0000324149926503, 1.00056377997001, 1.0010951449473697, 1.0016265099247297, 1.0021578749020894, 1.0026892398794491, 1.0032206048568089, 1.0037519698341686, 1.0042833348115285, 1.0048146997888883, 1.005346064766248, 1.0058774297436077, 1.0064087947209674, 1.0069401596983274, 1.0074715246756871, 1.0080028896530469, 1.0085342546304066, 1.0090656196077663, 1.0095969845851263, 1.010128349562486, 1.0106597145398457], "correct_counts": [205, 99, 55, 28, 29, 16, 16, 10, 6, 6, 3, 1, 5, 3, 0, 0, 1, 0, 0, 1], "incorrect_counts": [62, 19, 11, 8, 9, 0, 4, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.0011777698993682, 1.0023555397987365, 1.003533309698105, 1.0047110795974732, 1.0058888494968414, 1.0070666193962097, 1.008244389295578, 1.0094221591949464, 1.0105999290943146, 1.0117776989936829, 1.012955468893051, 1.0141332387924193, 1.0153110086917878, 1.016488778591156, 1.0176665484905243, 1.0188443183898925, 1.0200220882892608, 1.0211998581886292, 1.0223776280879975, 1.0235553979873657], "correct_counts": [346, 63, 26, 17, 10, 3, 3, 3, 3, 2, 1, 0, 3, 1, 0, 0, 0, 2, 0, 1], "incorrect_counts": [83, 18, 9, 2, 0, 2, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.00225226084391, 32.00583710670471, 32.00942195256551, 32.01300679842631, 32.01659164428711, 32.020176490147904, 32.023761336008704, 32.027346181869504, 32.030931027730304, 32.034515873591104, 32.038100719451904, 32.041685565312704, 32.045270411173504, 32.048855257034305, 32.052440102895105, 32.056024948755905, 32.0596097946167, 32.0631946404775, 32.0667794863383, 32.0703643321991, 32.0739491780599], "correct_counts": [49, 83, 57, 41, 41, 34, 27, 29, 23, 22, 23, 16, 13, 6, 7, 6, 4, 2, 0, 1], "incorrect_counts": [11, 30, 16, 14, 4, 7, 7, 4, 6, 6, 4, 2, 1, 2, 1, 0, 0, 1, 0, 0]}, "curv/trace_best": {"edges": [32.00101089477539, 32.006920433044435, 32.01282997131348, 32.01873950958252, 32.02464904785156, 32.030558586120605, 32.03646812438965, 32.042377662658694, 32.04828720092773, 32.054196739196776, 32.06010627746582, 32.066015815734865, 32.07192535400391, 32.077834892272946, 32.08374443054199, 32.089653968811035, 32.09556350708008, 32.101473045349124, 32.10738258361816, 32.113292121887206, 32.11920166015625], "correct_counts": [114, 104, 69, 52, 43, 25, 18, 9, 11, 10, 7, 9, 1, 2, 1, 2, 2, 2, 1, 2], "incorrect_counts": [32, 30, 22, 11, 6, 6, 2, 0, 1, 2, 0, 2, 1, 0, 1, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [484, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [116, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7266666666666666, 0.7315833333333333, 0.7364999999999999, 0.7414166666666666, 0.7463333333333333, 0.75125, 0.7561666666666667, 0.7610833333333333, 0.766, 0.7709166666666667, 0.7758333333333334, 0.78075, 0.7856666666666666, 0.7905833333333333, 0.7955, 0.8004166666666667, 0.8053333333333333, 0.81025, 0.8151666666666667, 0.8200833333333334, 0.8250000000000001], "correct_counts": [1, 5, 12, 12, 33, 46, 59, 70, 69, 48, 51, 37, 18, 11, 5, 1, 3, 2, 1, 0], "incorrect_counts": [1, 0, 2, 4, 3, 6, 18, 6, 13, 13, 7, 10, 12, 7, 5, 3, 4, 1, 0, 1]}, "dynamics/drop": {"edges": [68.1073338240385, 81.69682680144906, 95.28631977885962, 108.87581275627016, 122.46530573368072, 136.05479871109128, 149.64429168850182, 163.23378466591237, 176.82327764332294, 190.4127706207335, 204.00226359814405, 217.5917565755546, 231.18124955296514, 244.7707425303757, 258.36023550778623, 271.94972848519683, 285.5392214626074, 299.1287144400179, 312.7182074174285, 326.30770039483906, 339.8971933722496], "correct_counts": [13, 35, 94, 106, 104, 50, 32, 14, 18, 8, 5, 0, 2, 0, 2, 0, 1, 0, 0, 0], "incorrect_counts": [1, 7, 15, 20, 24, 12, 9, 4, 2, 3, 3, 0, 4, 4, 3, 1, 1, 1, 1, 1]}, "dynamics/residual": {"edges": [1.158554623524348, 1.174299425383409, 1.1900442272424698, 1.2057890291015307, 1.2215338309605916, 1.2372786328196526, 1.2530234346787135, 1.2687682365377744, 1.2845130383968353, 1.3002578402558962, 1.3160026421149573, 1.331747443974018, 1.3474922458330791, 1.36323704769214, 1.378981849551201, 1.3947266514102619, 1.4104714532693228, 1.4262162551283837, 1.4419610569874446, 1.4577058588465055, 1.4734506607055664], "correct_counts": [2, 5, 7, 19, 34, 40, 58, 60, 49, 71, 50, 37, 30, 9, 8, 2, 2, 0, 0, 1], "incorrect_counts": [0, 0, 1, 5, 8, 7, 14, 14, 14, 23, 10, 7, 6, 2, 3, 1, 0, 1, 0, 0]}}}, "feature_ablation": {"full": 0.07459630695723383, "drop_basin": 0.14680007392382874, "basin_only": 0.092294847589651, "drop_energy": 0.0767516512198465, "energy_only": 0.1878399181373265, "drop_curv": 0.06909324774054762, "curv_only": 0.1566104807264947, "drop_dynamics": 0.081970123644488, "dynamics_only": 0.15476700579479816}, "ece_geometry": 0.037362225019872054}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "aa60425c10e1", "timestamp": "2026-07-27T10:01:52.008241+00:00", "git_sha": "5207145", "config_hash": "cacbb6ece843", "config": {"run": {"seed": 2, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.875, "base_error": 0.125, "final_train_loss": 0.9444285035133362, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.027466089121522774, "rho_basin": 0.04398134938153453, "energy_min": 0.05234072522833359, "energy_mean": 0.04353111898523968, "energy_std": 0.1232672107794348, "msp": 0.02395206735255971, "temp_msp": 0.0231856584368455, "entropy": 0.03417904916905903}, "temperature": 0.27953871560488314, "best_energy_baseline": "energy_mean", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.02487463610681082, 0.00582124344497423, 0.048405336508425545], "delta_aurc_vs_best_energy": [0.01606502986371691, 0.0010850007001792562, 0.03158239388658305], "delta_aurc_vs_best_baseline": [-0.004280430684677275, -0.014613823709454883, 0.005013414288260376], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.009259259259259259, 0.008333333333333335, 0.007575757575757576, 0.013888888888888888, 0.01282051282051282, 0.017857142857142856, 0.016666666666666663, 0.015625, 0.014705882352941176, 0.013888888888888886, 0.013157894736842105, 0.0125, 0.011904761904761902, 0.015151515151515152, 0.014492753623188406, 0.013888888888888886, 0.013333333333333334, 0.01282051282051282, 0.012345679012345678, 0.011904761904761904, 0.01436781609195402, 0.013888888888888888, 0.01881720430107527, 0.018229166666666668, 0.017676767676767676, 0.01715686274509804, 0.019047619047619046, 0.018518518518518514, 0.02252252252252252, 0.02631578947368421, 0.03205128205128205, 0.035416666666666666, 0.04674796747967479, 0.06150793650793663, 0.07170542635658915, 0.07765151515151515, 0.08888888888888889, 0.09782608695652174, 0.10283687943262422, 0.1093750000000001, 0.11904761904761904, 0.125]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.03424657534246576, 0.03424657534246577, 0.03424657534246575, 0.03424657534246573, 0.03424657534246572, 0.03424657534246571, 0.0342465753424657, 0.0342465753424657, 0.034246575342465696, 0.034246575342465696, 0.034246575342465696, 0.034246575342465696, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246569, 0.03424657534246568, 0.03424657534246568, 0.0342465753424657, 0.034246575342465745, 0.03424657534246578, 0.034246575342465814, 0.03424657534246585, 0.034246575342465876, 0.034246575342465904, 0.034246575342465925, 0.034246575342465946, 0.03424657534246597, 0.03424657534246599, 0.03424657534246601, 0.03424657534246602, 0.03424657534246604, 0.03424657534246606, 0.03424657534246607, 0.034246575342466085, 0.036240786240786575, 0.0400717703349286, 0.04370629370629411, 0.04715909090909134, 0.05044345898004483, 0.0564236111111116, 0.06237887596899269, 0.06925230566534957, 0.0793075684380036, 0.08861058601134253, 0.09690101757631865, 0.10375816993464093, 0.10884353741496634, 0.12500000000000044]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.02976190476190476, 0.03888888888888888, 0.036458333333333336, 0.0392156862745098, 0.03703703703703703, 0.039473684210526314, 0.0375, 0.03968253968253967, 0.045454545454545456, 0.043478260869565216, 0.04513888888888888, 0.04666666666666667, 0.04487179487179487, 0.043209876543209874, 0.044642857142857144, 0.043103448275862065, 0.04722222222222222, 0.0456989247311828, 0.046875, 0.045454545454545456, 0.049019607843137254, 0.04761904761904761, 0.050925925925925916, 0.05405405405405406, 0.05921052631578947, 0.06196581196581197, 0.06666666666666667, 0.06910569105691056, 0.06944444444444443, 0.0755813953488372, 0.07954545454545454, 0.09074074074074075, 0.09601449275362318, 0.10106382978723402, 0.10763888888888899, 0.11734693877551021, 0.125]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.027777777777777776, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.023809523809523808, 0.02222222222222222, 0.026041666666666668, 0.024509803921568627, 0.023148148148148143, 0.02631578947368421, 0.025, 0.023809523809523805, 0.026515151515151516, 0.03260869565217391, 0.03819444444444444, 0.03666666666666667, 0.035256410256410256, 0.037037037037037035, 0.03869047619047619, 0.037356321839080456, 0.03611111111111111, 0.04032258064516129, 0.041666666666666664, 0.047979797979797977, 0.049019607843137254, 0.04761904761904761, 0.050925925925925916, 0.0518018018018018, 0.05482456140350877, 0.057692307692307696, 0.06875, 0.06910569105691056, 0.07142857142857141, 0.07945736434108527, 0.08333333333333333, 0.08888888888888889, 0.09420289855072464, 0.10106382978723415, 0.10937499999999999, 0.11564625850340136, 0.125]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.25, 0.16666666666666666, 0.14583333333333334, 0.13333333333333333, 0.1250000000000001, 0.11904761904761907, 0.11458333333333333, 0.10185185185185185, 0.11666666666666668, 0.11363636363636363, 0.1111111111111111, 0.10256410256410256, 0.10714285714285714, 0.11111111111111109, 0.10416666666666667, 0.10294117647058823, 0.10185185185185183, 0.10087719298245613, 0.09583333333333334, 0.09523809523809522, 0.10606060606060606, 0.12681159420289856, 0.12152777777777787, 0.12333333333333334, 0.125, 0.12345679012345678, 0.12797619047619047, 0.12356321839080457, 0.12222222222222222, 0.12634408602150538, 0.12760416666666666, 0.12373737373737374, 0.12254901960784313, 0.1238095238095238, 0.12499999999999999, 0.12612612612612611, 0.12938596491228072, 0.13034188034188035, 0.12708333333333333, 0.12398373983739835, 0.12499999999999999, 0.1298449612403101, 0.13068181818181818, 0.12777777777777777, 0.13043478260869565, 0.12765957446808507, 0.12673611111111108, 0.12585034013605442, 0.125]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.002976190476190476, 0.002873563218390804, 0.002777777777777778, 0.005376344086021506, 0.010416666666666666, 0.010101010101010102, 0.01715686274509804, 0.026190476190476344, 0.039351851851851846, 0.04279279279279279, 0.05043859649122807, 0.057692307692307696, 0.0625, 0.06504065040650406, 0.07142857142857141, 0.07751937984496124, 0.07954545454545454, 0.08703703703703704, 0.09420289855072464, 0.09929078014184396, 0.10937499999999999, 0.12074829931972789, 0.125]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.0043859649122807015, 0.004166666666666667, 0.003968253968253967, 0.003787878787878788, 0.007246376811594203, 0.006944444444444443, 0.006666666666666667, 0.009615384615384616, 0.012345679012345678, 0.011904761904761904, 0.017241379310344824, 0.016666666666666666, 0.016129032258064516, 0.015625, 0.017676767676767676, 0.01715686274509804, 0.019047619047619046, 0.02083333333333333, 0.024774774774774775, 0.02850877192982456, 0.027777777777777776, 0.03958333333333333, 0.04268292682926829, 0.049603174603174593, 0.05813953488372093, 0.06818181818181818, 0.07962962962962963, 0.09057971014492754, 0.10460992907801417, 0.11458333333333331, 0.12074829931972789, 0.125]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.008771929824561403, 0.008333333333333333, 0.011904761904761902, 0.015151515151515152, 0.014492753623188406, 0.02083333333333333, 0.02, 0.03205128205128205, 0.040123456790123455, 0.041666666666666664, 0.043103448275862065, 0.05, 0.04838709677419355, 0.046875, 0.045454545454545456, 0.04656862745098039, 0.049999999999999996, 0.050925925925925916, 0.05405405405405406, 0.05701754385964912, 0.05555555555555555, 0.058333333333333334, 0.0650406504065042, 0.0734126984126984, 0.0755813953488372, 0.07954545454545454, 0.09074074074074075, 0.09782608695652174, 0.10106382978723402, 0.10763888888888899, 0.11394557823129252, 0.125]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.34844747383499997, "coverage": 0.875, "abstain_rate": 0.125, "selective_risk": 0.07809523809523809, "selective_accuracy": 0.9219047619047619, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.017811704834605598, 0.07809523809523809, 0.12374581939799331, 0.125, 0.125], "coverage": [0.0, 0.655, 0.875, 0.9966666666666667, 1.0, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.8335492063492064, "basin/entropy": 0.16746666666666668, "basin/dispersion": 0.437384126984127, "energy/mean": 0.19380317460317462, "energy/min": 0.21155555555555555, "energy/std": 0.48954920634920635, "curv/lmax_mean": 0.37255873015873014, "curv/lmax_best": 0.4954031746031746, "curv/trace_mean": 0.4040888888888889, "curv/trace_best": 0.4389968253968254, "dynamics/steps": 0.5, "dynamics/monotonic": 0.29546666666666666, "dynamics/drop": 0.3021968253968254, "dynamics/residual": 0.40071111111111113}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [1, 0, 0, 11, 0, 0, 12, 0, 0, 0, 11, 0, 0, 22, 0, 0, 45, 0, 0, 423], "incorrect_counts": [11, 0, 0, 6, 0, 0, 11, 0, 0, 0, 12, 0, 0, 10, 0, 0, 10, 0, 0, 15]}, "basin/entropy": {"edges": [0.0, 0.041197960825054114, 0.08239592165010823, 0.12359388247516234, 0.16479184330021646, 0.20598980412527057, 0.24718776495032468, 0.2883857257753788, 0.3295836866004329, 0.370781647425487, 0.41197960825054114, 0.45317756907559525, 0.49437552990064937, 0.5355734907257035, 0.5767714515507576, 0.6179694123758117, 0.6591673732008658, 0.7003653340259199, 0.741563294850974, 0.7827612556760282, 0.8239592165010823], "correct_counts": [423, 0, 0, 0, 0, 0, 45, 0, 0, 0, 21, 0, 0, 12, 0, 10, 12, 0, 0, 2], "incorrect_counts": [15, 0, 0, 0, 0, 0, 10, 0, 0, 0, 9, 0, 0, 13, 0, 11, 17, 0, 0, 0]}, "basin/dispersion": {"edges": [1.637185344543599, 1.655643833346576, 1.6741023221495528, 1.6925608109525296, 1.7110192997555065, 1.729477788558483, 1.74793627736146, 1.7663947661644368, 1.7848532549674136, 1.8033117437703905, 1.8217702325733673, 1.8402287213763442, 1.858687210179321, 1.8771456989822979, 1.8956041877852747, 1.9140626765882516, 1.9325211653912282, 1.9509796541942053, 1.969438142997182, 1.9878966318001587, 2.0063551206031356], "correct_counts": [3, 5, 5, 18, 17, 28, 29, 37, 53, 58, 47, 67, 48, 38, 30, 16, 13, 8, 3, 2], "incorrect_counts": [0, 1, 0, 1, 3, 4, 6, 2, 6, 5, 7, 10, 7, 10, 2, 3, 2, 3, 2, 1]}, "energy/mean": {"edges": [-0.2380810777346293, -0.18294511114557582, -0.12780914455652237, -0.07267317796746889, -0.017537211378415407, 0.037598755210638074, 0.09273472179969153, 0.14787068838874504, 0.2030066549777985, 0.25814262156685197, 0.3132785881559055, 0.368414554744959, 0.4235505213340124, 0.4786864879230659, 0.5338224545121194, 0.5889584211011728, 0.6440943876902263, 0.6992303542792798, 0.7543663208683332, 0.8095022874573868, 0.8646382540464401], "correct_counts": [11, 51, 71, 110, 65, 46, 24, 23, 18, 19, 10, 15, 11, 7, 14, 9, 8, 8, 4, 1], "incorrect_counts": [0, 1, 3, 2, 5, 4, 4, 1, 3, 2, 8, 1, 7, 3, 8, 3, 7, 6, 5, 2]}, "energy/min": {"edges": [-0.6323069334030151, -0.5714096486568451, -0.510512363910675, -0.449615079164505, -0.388717794418335, -0.3278205096721649, -0.26692322492599485, -0.20602594017982484, -0.14512865543365477, -0.08423137068748476, -0.023334085941314697, 0.03756319880485537, 0.09846048355102544, 0.1593577682971955, 0.22025505304336546, 0.2811523377895355, 0.3420496225357056, 0.40294690728187565, 0.4638441920280456, 0.5247414767742158, 0.5856387615203857], "correct_counts": [3, 17, 46, 75, 77, 67, 67, 41, 14, 19, 21, 10, 15, 7, 12, 17, 6, 9, 0, 2], "incorrect_counts": [1, 0, 1, 2, 5, 5, 3, 3, 2, 5, 4, 3, 3, 3, 9, 8, 9, 5, 2, 2]}, "energy/std": {"edges": [0.09172678671306, 0.10379040427150595, 0.11585402182995189, 0.1279176393883978, 0.13998125694684377, 0.1520448745052897, 0.16410849206373562, 0.17617210962218155, 0.1882357271806275, 0.20029934473907346, 0.2123629622975194, 0.2244265798559653, 0.23649019741441124, 0.2485538149728572, 0.2606174325313031, 0.2726810500897491, 0.284744667648195, 0.29680828520664093, 0.3088719027650869, 0.3209355203235328, 0.33299913788197877], "correct_counts": [4, 3, 7, 24, 34, 39, 53, 54, 41, 63, 47, 44, 35, 24, 19, 14, 8, 5, 5, 2], "incorrect_counts": [0, 0, 4, 2, 4, 5, 5, 4, 13, 10, 5, 9, 5, 4, 2, 1, 0, 2, 0, 0]}, "curv/lmax_mean": {"edges": [1.0001661280790966, 1.0006819412112238, 1.0011977543433508, 1.0017135674754778, 1.002229380607605, 1.0027451937397323, 1.0032610068718593, 1.0037768200039863, 1.0042926331361135, 1.0048084462682407, 1.0053242594003677, 1.0058400725324947, 1.006355885664622, 1.0068716987967492, 1.0073875119288762, 1.0079033250610032, 1.0084191381931304, 1.0089349513252577, 1.0094507644573847, 1.0099665775895117, 1.010482390721639], "correct_counts": [65, 100, 87, 66, 41, 40, 43, 27, 12, 17, 5, 5, 3, 5, 2, 2, 3, 1, 0, 1], "incorrect_counts": [2, 10, 9, 9, 10, 9, 8, 4, 5, 2, 3, 0, 2, 1, 1, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.0, 1.001549595594406, 1.0030991911888123, 1.0046487867832183, 1.0061983823776246, 1.0077479779720306, 1.0092975735664367, 1.010847169160843, 1.012396764755249, 1.0139463603496552, 1.0154959559440613, 1.0170455515384673, 1.0185951471328736, 1.0201447427272796, 1.0216943383216859, 1.023243933916092, 1.024793529510498, 1.0263431251049042, 1.0278927206993103, 1.0294423162937165, 1.0309919118881226], "correct_counts": [311, 80, 40, 32, 16, 11, 9, 6, 5, 2, 3, 2, 3, 0, 2, 0, 1, 0, 1, 1], "incorrect_counts": [43, 14, 5, 1, 4, 2, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0]}, "curv/trace_mean": {"edges": [32.0079870223999, 32.01155290603638, 32.015118789672854, 32.01868467330932, 32.0222505569458, 32.025816440582275, 32.02938232421875, 32.03294820785523, 32.036514091491696, 32.04007997512817, 32.04364585876465, 32.047211742401124, 32.0507776260376, 32.05434350967407, 32.057909393310545, 32.06147527694702, 32.0650411605835, 32.06860704421997, 32.07217292785644, 32.07573881149292, 32.079304695129395], "correct_counts": [20, 28, 38, 43, 36, 59, 52, 37, 50, 24, 35, 27, 23, 17, 21, 8, 3, 3, 0, 1], "incorrect_counts": [2, 3, 3, 3, 5, 5, 4, 9, 6, 6, 6, 3, 10, 4, 1, 0, 4, 0, 1, 0]}, "curv/trace_best": {"edges": [32.001441955566406, 32.00756416320801, 32.01368637084961, 32.01980857849121, 32.02593078613281, 32.032052993774414, 32.03817520141602, 32.04429740905762, 32.050419616699216, 32.05654182434082, 32.06266403198242, 32.068786239624025, 32.07490844726563, 32.081030654907224, 32.08715286254883, 32.09327507019043, 32.09939727783203, 32.105519485473636, 32.11164169311523, 32.117763900756835, 32.12388610839844], "correct_counts": [21, 62, 75, 72, 60, 48, 55, 36, 25, 18, 16, 9, 9, 5, 4, 7, 2, 1, 0, 0], "incorrect_counts": [4, 4, 11, 5, 13, 6, 6, 5, 6, 3, 3, 4, 0, 1, 0, 1, 1, 0, 0, 2]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [525, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [75, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7250000000000001, 0.7297500000000001, 0.7345000000000002, 0.7392500000000001, 0.7440000000000001, 0.7487500000000001, 0.7535000000000001, 0.7582500000000001, 0.7630000000000001, 0.7677500000000002, 0.7725000000000002, 0.7772500000000001, 0.7820000000000001, 0.7867500000000002, 0.7915000000000001, 0.7962500000000001, 0.8010000000000002, 0.8057500000000002, 0.8105000000000002, 0.8152500000000001, 0.8200000000000002], "correct_counts": [1, 2, 10, 9, 19, 39, 46, 70, 84, 78, 57, 44, 33, 17, 11, 3, 1, 0, 1, 0], "incorrect_counts": [0, 0, 0, 1, 2, 7, 0, 6, 2, 8, 7, 10, 6, 8, 7, 1, 4, 3, 2, 1]}, "dynamics/drop": {"edges": [74.32381541033585, 85.35149599189559, 96.37917657345534, 107.40685715501508, 118.43453773657481, 129.46221831813455, 140.4898988996943, 151.51757948125402, 162.54526006281378, 173.5729406443735, 184.60062122593325, 195.628301807493, 206.65598238905272, 217.6836629706125, 228.7113435521722, 239.73902413373196, 250.76670471529167, 261.79438529685143, 272.82206587841114, 283.8497464599709, 294.8774270415306], "correct_counts": [5, 32, 66, 110, 104, 112, 44, 30, 9, 2, 2, 2, 2, 2, 1, 0, 0, 2, 0, 0], "incorrect_counts": [0, 4, 8, 7, 7, 7, 5, 6, 4, 6, 4, 2, 2, 2, 3, 4, 0, 1, 1, 2]}, "dynamics/residual": {"edges": [1.162097414334615, 1.177681430677573, 1.1932654470205306, 1.2088494633634885, 1.2244334797064462, 1.2400174960494041, 1.2556015123923618, 1.2711855287353198, 1.2867695450782775, 1.3023535614212354, 1.317937577764193, 1.333521594107151, 1.349105610450109, 1.3646896267930666, 1.3802736431360245, 1.3958576594789822, 1.4114416758219401, 1.4270256921648978, 1.4426097085078557, 1.4581937248508137, 1.4737777411937714], "correct_counts": [4, 6, 11, 24, 26, 38, 59, 74, 72, 54, 48, 48, 30, 20, 5, 2, 3, 0, 0, 1], "incorrect_counts": [0, 0, 1, 1, 4, 2, 8, 6, 10, 8, 11, 11, 6, 4, 1, 1, 1, 0, 0, 0]}}}, "feature_ablation": {"full": 0.027466089121522774, "drop_basin": 0.057062857928267645, "basin_only": 0.05405795250584025, "drop_energy": 0.030427040663415977, "energy_only": 0.03656546480044251, "drop_curv": 0.024640042328588217, "curv_only": 0.10271323906065064, "drop_dynamics": 0.024543098804670704, "dynamics_only": 0.09662840153616381}, "ece_geometry": 0.057436118293730715}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "c948f8130a3d", "timestamp": "2026-07-27T10:01:54.683564+00:00", "git_sha": "5207145", "config_hash": "054f01c6c637", "config": {"run": {"seed": 3, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.825, "base_error": 0.175, "final_train_loss": 0.943260669708252, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.06161855001684325, "rho_basin": 0.09229054743872664, "energy_min": 0.21136180624942313, "energy_mean": 0.21095193442795585, "energy_std": 0.17250412654357142, "msp": 0.05530041645573463, "temp_msp": 0.05305986961532728, "entropy": 0.07641415209991691}, "temperature": 0.28776072591113844, "best_energy_baseline": "energy_std", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.1497432562325799, 0.10842759072651144, 0.19249016133690647], "delta_aurc_vs_best_energy": [0.11088557652672817, 0.07324712187491986, 0.14959801044328674], "delta_aurc_vs_best_baseline": [-0.008558680401515972, -0.02338443781342537, 0.005027521049910705], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.03333333333333333, 0.027777777777777773, 0.03571428571428572, 0.03125, 0.037037037037037035, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.05128205128205128, 0.047619047619047616, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.04166666666666666, 0.039473684210526314, 0.041666666666666664, 0.03968253968253967, 0.03787878787878788, 0.03985507246376811, 0.03819444444444444, 0.04, 0.041666666666666664, 0.040123456790123455, 0.041666666666666664, 0.045977011494252866, 0.04722222222222222, 0.051075268817204304, 0.052083333333333336, 0.05808080808080808, 0.061274509803921566, 0.06666666666666665, 0.06712962962962962, 0.06981981981981981, 0.07894736842105263, 0.08547008547008547, 0.09166666666666666, 0.09756097560975609, 0.10714285714285712, 0.1182170542635659, 0.13257575757575757, 0.14074074074074075, 0.14855072463768115, 0.15602836879432636, 0.16319444444444442, 0.1683673469387755, 0.175]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.07894736842105264, 0.07894736842105264, 0.07894736842105264, 0.07894736842105264, 0.07894736842105259, 0.07894736842105252, 0.07894736842105247, 0.07894736842105245, 0.07894736842105247, 0.07894736842105253, 0.07894736842105259, 0.07894736842105263, 0.07894736842105267, 0.0789473684210527, 0.07894736842105272, 0.07894736842105275, 0.07894736842105275, 0.07894736842105267, 0.0789473684210526, 0.07894736842105253, 0.07894736842105247, 0.07894736842105243, 0.07894736842105238, 0.07894736842105234, 0.0789473684210523, 0.07894736842105225, 0.07894736842105222, 0.07894736842105218, 0.07894736842105216, 0.07894736842105213, 0.0789473684210521, 0.07963466183574824, 0.08161323378714626, 0.08347541915316793, 0.08523119392684551, 0.086889425657541, 0.0884580232406313, 0.0920032639738876, 0.09679984098588691, 0.10135658914728626, 0.10569105691056861, 0.1129785247432302, 0.11992704058367491, 0.1270396270396266, 0.1361823361823356, 0.1449275362318834, 0.15521783181357598, 0.1650793650793647, 0.17386185243328067, 0.17499999999999957]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.4166666666666667, 0.3888888888888889, 0.3333333333333333, 0.3333333333333333, 0.3194444444444445, 0.3452380952380953, 0.3020833333333333, 0.28703703703703703, 0.2666666666666667, 0.24242424242424243, 0.2222222222222222, 0.22435897435897437, 0.21428571428571427, 0.21666666666666665, 0.22395833333333334, 0.2107843137254902, 0.21296296296296308, 0.2149122807017544, 0.21666666666666667, 0.21825396825396837, 0.2159090909090909, 0.20652173913043478, 0.20833333333333331, 0.2, 0.19230769230769232, 0.1882716049382716, 0.1875, 0.18103448275862066, 0.17777777777777778, 0.17473118279569894, 0.171875, 0.16666666666666666, 0.16176470588235295, 0.15952380952380948, 0.15740740740740738, 0.15315315315315314, 0.15350877192982457, 0.14957264957264957, 0.15, 0.15243902439024387, 0.14880952380952378, 0.14922480620155038, 0.15151515151515152, 0.15185185185185185, 0.15217391304347827, 0.15602836879432636, 0.16319444444444442, 0.1717687074829932, 0.175]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.375, 0.3333333333333333, 0.2708333333333333, 0.23333333333333334, 0.3194444444444445, 0.3214285714285715, 0.28125, 0.28703703703703703, 0.2833333333333334, 0.2878787878787879, 0.2638888888888889, 0.25, 0.24404761904761904, 0.24999999999999994, 0.23958333333333334, 0.23529411764705882, 0.22685185185185183, 0.2236842105263158, 0.2125, 0.20238095238095236, 0.19318181818181818, 0.19202898550724637, 0.18749999999999997, 0.18333333333333332, 0.1858974358974359, 0.18209876543209877, 0.17857142857142858, 0.17528735632183906, 0.17222222222222222, 0.16666666666666666, 0.16145833333333334, 0.1590909090909091, 0.15441176470588236, 0.15238095238095237, 0.1504629629629631, 0.1554054054054054, 0.15350877192982457, 0.1517094017094017, 0.15, 0.14837398373983737, 0.1468253968253968, 0.1434108527131783, 0.14393939393939395, 0.14814814814814814, 0.1503623188405797, 0.1524822695035462, 0.1597222222222223, 0.1683673469387755, 0.175]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.16666666666666666, 0.19444444444444445, 0.20833333333333334, 0.16666666666666666, 0.16666666666666663, 0.1785714285714286, 0.16666666666666666, 0.17592592592592593, 0.16666666666666669, 0.15151515151515152, 0.1597222222222222, 0.15384615384615385, 0.17857142857142858, 0.1722222222222222, 0.171875, 0.16666666666666666, 0.162037037037037, 0.16666666666666666, 0.175, 0.17460317460317457, 0.17424242424242425, 0.17028985507246377, 0.17361111111111108, 0.17333333333333334, 0.1794871794871795, 0.18518518518518517, 0.18154761904761904, 0.17816091954022986, 0.17777777777777778, 0.18010752688172044, 0.1796875, 0.18181818181818182, 0.18137254901960784, 0.17619047619047618, 0.17361111111111108, 0.17117117117117117, 0.16885964912280702, 0.16666666666666666, 0.16458333333333333, 0.16666666666666663, 0.16666666666666663, 0.16279069767441862, 0.16666666666666666, 0.1648148148148148, 0.16666666666666666, 0.16666666666666663, 0.17013888888888887, 0.17346938775510204, 0.175]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005952380952380952, 0.005555555555555555, 0.015625, 0.0196078431372549, 0.018518518518518514, 0.017543859649122806, 0.016666666666666666, 0.019841269841269837, 0.022727272727272728, 0.028985507246376812, 0.03472222222222222, 0.03666666666666667, 0.035256410256410256, 0.040123456790123455, 0.044642857142857144, 0.04885057471264386, 0.06111111111111111, 0.06451612903225806, 0.06770833333333333, 0.07828282828282829, 0.0857843137254902, 0.09523809523809522, 0.10185185185185183, 0.1036036036036036, 0.10964912280701754, 0.11324786324786325, 0.11875, 0.1219512195121951, 0.12103174603174614, 0.1298449612403101, 0.13257575757575757, 0.13703703703703704, 0.1431159420289855, 0.1524822695035462, 0.15972222222222218, 0.16666666666666666, 0.175]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.008333333333333335, 0.007575757575757576, 0.006944444444444444, 0.00641025641025641, 0.005952380952380952, 0.016666666666666663, 0.015625, 0.014705882352941176, 0.018518518518518514, 0.02631578947368421, 0.025, 0.023809523809523805, 0.030303030303030304, 0.028985507246376812, 0.027777777777777887, 0.03333333333333333, 0.035256410256410256, 0.037037037037037035, 0.041666666666666664, 0.045977011494252866, 0.05277777777777778, 0.05913978494623656, 0.0625, 0.06565656565656566, 0.06862745098039216, 0.07619047619047618, 0.08333333333333331, 0.0945945945945946, 0.09649122807017543, 0.10042735042735043, 0.10625, 0.11382113821138223, 0.11507936507936518, 0.12403100775193798, 0.12878787878787878, 0.13518518518518519, 0.14130434782608695, 0.150709219858156, 0.15798611111111108, 0.1683673469387755, 0.175]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.009259259259259259, 0.008333333333333335, 0.007575757575757576, 0.027777777777777776, 0.04487179487179487, 0.041666666666666664, 0.04444444444444444, 0.041666666666666664, 0.04411764705882353, 0.050925925925925916, 0.06140350877192982, 0.07083333333333333, 0.06746031746031758, 0.08712121212121213, 0.09420289855072464, 0.10069444444444443, 0.10333333333333333, 0.10256410256410256, 0.10185185185185185, 0.10119047619047619, 0.10057471264367815, 0.10277777777777777, 0.10483870967741936, 0.10416666666666667, 0.10353535353535354, 0.10049019607843138, 0.10238095238095236, 0.10185185185185183, 0.10585585585585586, 0.10964912280701754, 0.11324786324786325, 0.11041666666666666, 0.11178861788617898, 0.11904761904761903, 0.12209302325581395, 0.1268939393939394, 0.13333333333333333, 0.14130434782608695, 0.15425531914893628, 0.16319444444444453, 0.17006802721088435, 0.175]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": 0.15723492252970733, "coverage": 0.705, "abstain_rate": 0.295, "selective_risk": 0.06619385342789598, "selective_accuracy": 0.933806146572104, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.06619385342789598, 0.11650485436893204, 0.175, 0.175], "coverage": [0.0, 0.0, 0.705, 0.8583333333333333, 1.0, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7495911495911496, "basin/entropy": 0.24934102934102934, "basin/dispersion": 0.49964405964405967, "energy/mean": 0.5078595478595479, "energy/min": 0.5175949975949976, "energy/std": 0.48817700817700815, "curv/lmax_mean": 0.5006637806637807, "curv/lmax_best": 0.5116305916305917, "curv/trace_mean": 0.49462241462241463, "curv/trace_best": 0.5089369889369889, "dynamics/steps": 0.5, "dynamics/monotonic": 0.3703030303030303, "dynamics/drop": 0.3482058682058682, "dynamics/residual": 0.5112265512265513}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [10, 0, 0, 13, 0, 0, 12, 0, 0, 0, 20, 0, 0, 31, 0, 0, 59, 0, 0, 350], "incorrect_counts": [3, 0, 0, 22, 0, 0, 14, 0, 0, 0, 14, 0, 0, 12, 0, 0, 10, 0, 0, 30]}, "basin/entropy": {"edges": [0.0, 0.053756966200268666, 0.10751393240053733, 0.161270898600806, 0.21502786480107466, 0.2687848310013433, 0.322541797201612, 0.3762987634018807, 0.43005572960214933, 0.483812695802418, 0.5375696620026866, 0.5913266282029553, 0.645083594403224, 0.6988405606034926, 0.7525975268037614, 0.80635449300403, 0.8601114592042987, 0.9138684254045674, 0.967625391604836, 1.0213823578051047, 1.0751393240053733], "correct_counts": [350, 0, 0, 0, 0, 59, 0, 0, 30, 0, 20, 11, 20, 1, 0, 1, 1, 2, 0, 0], "incorrect_counts": [30, 0, 0, 0, 0, 10, 0, 0, 12, 0, 11, 14, 22, 3, 0, 0, 1, 1, 0, 1]}, "basin/dispersion": {"edges": [1.6415780038768077, 1.6640313201520378, 1.6864846364272679, 1.708937952702498, 1.731391268977728, 1.753844585252958, 1.7762979015281881, 1.798751217803418, 1.821204534078648, 1.843657850353878, 1.8661111666291081, 1.8885644829043382, 1.9110177991795683, 1.9334711154547983, 1.9559244317300284, 1.9783777480052582, 2.0008310642804883, 2.0232843805557184, 2.0457376968309484, 2.0681910131061785, 2.0906443293814085], "correct_counts": [3, 8, 12, 19, 39, 47, 51, 67, 66, 56, 37, 35, 22, 13, 8, 6, 3, 2, 0, 1], "incorrect_counts": [2, 0, 3, 5, 5, 14, 12, 10, 15, 9, 11, 5, 7, 2, 2, 3, 0, 0, 0, 0]}, "energy/mean": {"edges": [0.2628147800763448, 0.3243880569934845, 0.38596133391062415, 0.44753461082776386, 0.5091078877449036, 0.5706811646620433, 0.6322544415791829, 0.6938277184963226, 0.7554009954134624, 0.8169742723306019, 0.8785475492477417, 0.9401208261648815, 1.001694103082021, 1.0632673799991608, 1.1248406569163005, 1.1864139338334403, 1.2479872107505798, 1.3095604876677196, 1.3711337645848591, 1.432707041501999, 1.4942803184191387], "correct_counts": [1, 6, 8, 19, 15, 18, 35, 58, 81, 101, 66, 37, 19, 8, 5, 3, 5, 2, 3, 5], "incorrect_counts": [1, 2, 6, 4, 10, 4, 11, 10, 6, 9, 10, 2, 6, 3, 2, 4, 3, 2, 8, 2]}, "energy/min": {"edges": [-0.1004953682422638, -0.03417549580335617, 0.03214437663555145, 0.09846424907445905, 0.1647841215133667, 0.23110399395227432, 0.2974238663911819, 0.36374373883008954, 0.43006361126899717, 0.49638348370790475, 0.5627033561468124, 0.62902322858572, 0.6953431010246276, 0.7616629734635353, 0.8279828459024429, 0.8943027183413506, 0.9606225907802581, 1.0269424632191657, 1.0932623356580733, 1.159582208096981, 1.2259020805358887], "correct_counts": [1, 2, 5, 8, 24, 18, 40, 46, 75, 91, 81, 44, 22, 7, 11, 4, 6, 3, 5, 2], "incorrect_counts": [0, 0, 2, 9, 9, 9, 3, 8, 17, 9, 4, 5, 7, 2, 3, 6, 2, 7, 2, 1]}, "energy/std": {"edges": [0.0727182563720062, 0.08928063387973476, 0.1058430113874633, 0.12240538889519184, 0.1389677664029204, 0.15553014391064895, 0.1720925214183775, 0.18865489892610604, 0.20521727643383458, 0.22177965394156313, 0.23834203144929167, 0.25490440895702027, 0.2714667864647488, 0.28802916397247735, 0.3045915414802059, 0.32115391898793444, 0.337716296495663, 0.3542786740033915, 0.37084105151112007, 0.3874034290188486, 0.4039658065265772], "correct_counts": [2, 4, 14, 30, 45, 73, 65, 67, 67, 53, 37, 9, 13, 10, 5, 0, 1, 0, 0, 0], "incorrect_counts": [1, 0, 3, 6, 10, 13, 15, 18, 10, 8, 7, 2, 5, 4, 1, 0, 1, 0, 0, 1]}, "curv/lmax_mean": {"edges": [0.9986756841341654, 0.9992095266779264, 0.9997433692216873, 1.0002772117654481, 1.0008110543092092, 1.0013448968529701, 1.001878739396731, 1.0024125819404919, 1.002946424484253, 1.0034802670280139, 1.0040141095717747, 1.0045479521155358, 1.0050817946592967, 1.0056156372030576, 1.0061494797468185, 1.0066833222905796, 1.0072171648343404, 1.0077510073781013, 1.0082848499218622, 1.0088186924656233, 1.0093525350093842], "correct_counts": [2, 47, 164, 112, 58, 41, 23, 15, 11, 7, 4, 6, 0, 2, 1, 1, 0, 0, 0, 1], "incorrect_counts": [1, 26, 21, 13, 9, 6, 4, 7, 4, 4, 1, 6, 0, 2, 0, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.9967403411865234, 0.9977234482765198, 0.9987065553665161, 0.9996896624565125, 1.0006727695465087, 1.0016558766365051, 1.0026389837265015, 1.003622090816498, 1.004605197906494, 1.0055883049964904, 1.0065714120864868, 1.0075545191764832, 1.0085376262664796, 1.0095207333564757, 1.0105038404464721, 1.0114869475364685, 1.0124700546264649, 1.0134531617164613, 1.0144362688064574, 1.0154193758964538, 1.0164024829864502], "correct_counts": [2, 2, 33, 345, 66, 19, 9, 6, 1, 2, 2, 1, 1, 1, 1, 1, 0, 0, 1, 2], "incorrect_counts": [1, 0, 12, 64, 10, 4, 1, 3, 2, 2, 0, 0, 0, 2, 2, 0, 0, 0, 2, 0]}, "curv/trace_mean": {"edges": [31.886516253153484, 31.8960693359375, 31.905622418721517, 31.915175501505534, 31.924728584289554, 31.93428166707357, 31.943834749857587, 31.953387832641603, 31.96294091542562, 31.972493998209636, 31.982047080993652, 31.991600163777672, 32.001153246561685, 32.0107063293457, 32.020259412129725, 32.02981249491374, 32.03936557769776, 32.048918660481775, 32.05847174326579, 32.06802482604981, 32.077577908833824], "correct_counts": [3, 0, 4, 5, 2, 4, 5, 3, 5, 8, 31, 68, 119, 97, 56, 45, 31, 9, 0, 0], "incorrect_counts": [3, 2, 4, 4, 1, 3, 2, 3, 2, 5, 3, 7, 13, 10, 11, 7, 14, 9, 0, 2]}, "curv/trace_best": {"edges": [31.816234588623047, 31.83113193511963, 31.84602928161621, 31.860926628112793, 31.875823974609375, 31.890721321105957, 31.90561866760254, 31.92051601409912, 31.935413360595703, 31.950310707092285, 31.965208053588867, 31.98010540008545, 31.99500274658203, 32.00990009307861, 32.024797439575195, 32.03969478607178, 32.05459213256836, 32.06948947906494, 32.08438682556152, 32.099284172058105, 32.11418151855469], "correct_counts": [1, 0, 0, 2, 3, 0, 1, 7, 10, 10, 15, 44, 187, 116, 53, 32, 8, 2, 3, 1], "incorrect_counts": [0, 1, 2, 0, 4, 1, 2, 3, 9, 4, 4, 6, 16, 22, 12, 8, 4, 3, 4, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [495, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [105, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7233333333333335, 0.7288333333333334, 0.7343333333333335, 0.7398333333333335, 0.7453333333333335, 0.7508333333333335, 0.7563333333333335, 0.7618333333333335, 0.7673333333333335, 0.7728333333333335, 0.7783333333333335, 0.7838333333333335, 0.7893333333333334, 0.7948333333333335, 0.8003333333333335, 0.8058333333333335, 0.8113333333333335, 0.8168333333333335, 0.8223333333333335, 0.8278333333333335, 0.8333333333333335], "correct_counts": [5, 6, 7, 29, 40, 57, 77, 73, 59, 56, 33, 23, 15, 9, 4, 1, 1, 0, 0, 0], "incorrect_counts": [0, 0, 4, 2, 4, 8, 14, 9, 11, 18, 9, 6, 5, 4, 8, 1, 1, 0, 0, 1]}, "dynamics/drop": {"edges": [61.334716603159904, 74.52057102819283, 87.70642545322578, 100.8922798782587, 114.07813430329165, 127.26398872832458, 140.4498431533575, 153.63569757839045, 166.8215520034234, 180.0074064284563, 193.19326085348925, 206.3791152785222, 219.5649697035551, 232.75082412858805, 245.936678553621, 259.1225329786539, 272.3083874036869, 285.4942418287198, 298.6800962537527, 311.8659506787857, 325.0518051038186], "correct_counts": [3, 27, 66, 113, 119, 86, 38, 14, 6, 7, 9, 2, 0, 2, 1, 0, 2, 0, 0, 0], "incorrect_counts": [1, 1, 13, 11, 18, 22, 9, 6, 5, 4, 3, 0, 3, 3, 1, 2, 0, 1, 0, 2]}, "dynamics/residual": {"edges": [1.154107744495074, 1.1687258986135325, 1.183344052731991, 1.1979622068504494, 1.2125803609689076, 1.227198515087366, 1.2418166692058246, 1.256434823324283, 1.2710529774427415, 1.2856711315612, 1.3002892856796584, 1.3149074397981166, 1.329525593916575, 1.3441437480350336, 1.358761902153492, 1.3733800562719505, 1.387998210390409, 1.4026163645088672, 1.4172345186273256, 1.431852672745784, 1.4464708268642426], "correct_counts": [3, 2, 4, 11, 16, 27, 45, 61, 66, 64, 50, 48, 28, 30, 22, 4, 8, 0, 5, 1], "incorrect_counts": [0, 1, 3, 2, 4, 8, 8, 11, 15, 9, 13, 10, 4, 7, 4, 3, 2, 0, 1, 0]}}}, "feature_ablation": {"full": 0.06161855001684325, "drop_basin": 0.1186795725541486, "basin_only": 0.10056958024140106, "drop_energy": 0.06409034702303915, "energy_only": 0.2158304639713504, "drop_curv": 0.07728564902993969, "curv_only": 0.13726777124827397, "drop_dynamics": 0.05955759606129707, "dynamics_only": 0.12849301837335375}, "ece_geometry": 0.04520941743790561}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "4a5576e4c244", "timestamp": "2026-07-27T10:01:57.359827+00:00", "git_sha": "5207145", "config_hash": "0b564d7f9e23", "config": {"run": {"seed": 4, "task": "arithmetic", "notes": "E1 selective-prediction main run"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 7, "batch_size": 128, "lr": 0.001, "n_train": 6000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"arithmetic": {"modular": false, "n_operands": 6, "ood_n_operands": 6, "max_operand": 7, "ood_max_operand": 9}}}, "task": "arithmetic", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.8266666666666667, "base_error": 0.17333333333333334, "final_train_loss": 0.8476110100746155, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.054893261179459335, "rho_basin": 0.08645638007796895, "energy_min": 0.1561569151577505, "energy_mean": 0.16005205383202037, "energy_std": 0.19277374311689105, "msp": 0.0412224839050244, "temp_msp": 0.042521440482154955, "entropy": 0.04802248425611829}, "temperature": 0.3184887138430314, "best_energy_baseline": "energy_min", "best_baseline": "msp", "delta_aurc_vs_energy_min": [0.10126365397829115, 0.06398265000827008, 0.1401163910892647], "delta_aurc_vs_best_energy": [0.10126365397829115, 0.06398265000827008, 0.1401163910892647], "delta_aurc_vs_best_baseline": [-0.013670777274434935, -0.030908977792358076, 0.00040221684223381557], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.03205128205128205, 0.02976190476190476, 0.027777777777777773, 0.026041666666666668, 0.024509803921568627, 0.023148148148148143, 0.021929824561403508, 0.029166666666666667, 0.027777777777777773, 0.030303030303030304, 0.03260869565217391, 0.031249999999999997, 0.03, 0.03205128205128205, 0.030864197530864196, 0.03273809523809524, 0.03160919540229885, 0.03333333333333333, 0.03225806451612903, 0.033854166666666664, 0.03787878787878788, 0.049019607843137254, 0.057142857142857134, 0.060185185185185175, 0.06756756756756757, 0.07675438596491228, 0.08547008547008547, 0.0875, 0.09552845528455298, 0.10515873015873015, 0.11046511627906977, 0.11931818181818182, 0.13148148148148148, 0.13768115942028986, 0.14007092198581572, 0.15277777777777776, 0.16156462585034015, 0.17333333333333334]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.07194244604316546, 0.07194244604316546, 0.07194244604316546, 0.07194244604316546, 0.07194244604316544, 0.07194244604316537, 0.07194244604316531, 0.07194244604316528, 0.07194244604316526, 0.07194244604316523, 0.07194244604316521, 0.07194244604316519, 0.07194244604316517, 0.07194244604316517, 0.07194244604316516, 0.07194244604316515, 0.07194244604316515, 0.07194244604316513, 0.07194244604316517, 0.07194244604316526, 0.07194244604316533, 0.0719424460431654, 0.07194244604316546, 0.07194244604316552, 0.07194244604316556, 0.07194244604316562, 0.07194244604316566, 0.0719424460431657, 0.07194244604316573, 0.07194244604316577, 0.0719424460431658, 0.07194244604316584, 0.07194244604316587, 0.0719424460431659, 0.07315668202765024, 0.07784498207885351, 0.0822798605056674, 0.08648132427843845, 0.0904673283705546, 0.09476744186046554, 0.10436755530346054, 0.11351052048726526, 0.12222823147647434, 0.13423295454545514, 0.1472934472934479, 0.14994425863991148, 0.15248226950354687, 0.15782828282828354, 0.16375121477162355, 0.17333333333333387]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.16666666666666666, 0.1388888888888889, 0.125, 0.1, 0.09722222222222232, 0.10714285714285716, 0.125, 0.12962962962962962, 0.1416666666666667, 0.15151515151515152, 0.1597222222222222, 0.14743589743589744, 0.1488095238095238, 0.16666666666666663, 0.171875, 0.17647058823529413, 0.17129629629629645, 0.17543859649122806, 0.175, 0.17460317460317457, 0.17424242424242425, 0.17391304347826086, 0.17361111111111108, 0.17, 0.16987179487179488, 0.16358024691358025, 0.16071428571428573, 0.15804597701149423, 0.15555555555555556, 0.15591397849462366, 0.15625, 0.15404040404040403, 0.1568627450980392, 0.15476190476190474, 0.15277777777777776, 0.14864864864864866, 0.14912280701754385, 0.1452991452991453, 0.14583333333333334, 0.14227642276422775, 0.14285714285714282, 0.14728682170542637, 0.14962121212121213, 0.15, 0.15760869565217392, 0.16134751773049658, 0.17187499999999997, 0.17346938775510204, 0.17333333333333334]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.125, 0.1388888888888889, 0.125, 0.1, 0.11111111111111109, 0.11904761904761907, 0.125, 0.1574074074074074, 0.1416666666666667, 0.14393939393939395, 0.1388888888888889, 0.14102564102564102, 0.14285714285714285, 0.16666666666666663, 0.17708333333333334, 0.1715686274509804, 0.18055555555555552, 0.18421052631578946, 0.17916666666666667, 0.17857142857142855, 0.17424242424242425, 0.17753623188405798, 0.17013888888888898, 0.18, 0.1762820512820513, 0.1697530864197531, 0.1636904761904762, 0.16666666666666663, 0.16111111111111112, 0.16129032258064516, 0.1640625, 0.16161616161616163, 0.16176470588235295, 0.15952380952380948, 0.15740740740740738, 0.15315315315315314, 0.14912280701754385, 0.1452991452991453, 0.14166666666666666, 0.14024390243902438, 0.14087301587301584, 0.1434108527131783, 0.14393939393939395, 0.15, 0.15760869565217392, 0.1595744680851065, 0.16319444444444442, 0.16666666666666666, 0.17333333333333334]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.2916666666666667, 0.19444444444444445, 0.25, 0.2, 0.2222222222222223, 0.22619047619047603, 0.21875, 0.24074074074074073, 0.22500000000000003, 0.2196969696969697, 0.2222222222222222, 0.21153846153846154, 0.2261904761904762, 0.21666666666666665, 0.203125, 0.19607843137254902, 0.18518518518518515, 0.17543859649122806, 0.17083333333333334, 0.1706349206349206, 0.17424242424242425, 0.17391304347826086, 0.17013888888888887, 0.16666666666666666, 0.16025641025641027, 0.16358024691358025, 0.16666666666666666, 0.16379310344827602, 0.16666666666666666, 0.1639784946236559, 0.1640625, 0.16666666666666666, 0.16666666666666666, 0.17142857142857157, 0.17361111111111108, 0.17342342342342343, 0.17105263157894737, 0.17094017094017094, 0.16875, 0.17073170731707316, 0.16865079365079377, 0.17054263565891473, 0.17045454545454544, 0.1685185185185185, 0.16847826086956522, 0.16666666666666677, 0.17013888888888887, 0.17006802721088435, 0.17333333333333334]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.005952380952380952, 0.005555555555555555, 0.005208333333333333, 0.004901960784313725, 0.0046296296296296285, 0.008771929824561403, 0.008333333333333333, 0.007936507936507934, 0.015151515151515152, 0.014492753623188406, 0.013888888888888886, 0.02, 0.019230769230769232, 0.024691358024691357, 0.026785714285714284, 0.02873563218390804, 0.030555555555555555, 0.02956989247311828, 0.03125, 0.03535353535353535, 0.041666666666666664, 0.04523809523809523, 0.053240740740740734, 0.06306306306306306, 0.07236842105263158, 0.0811965811965812, 0.09375, 0.09959349593495934, 0.10515873015873028, 0.11434108527131782, 0.125, 0.13333333333333333, 0.13768115942028986, 0.14184397163120566, 0.15277777777777776, 0.16326530612244897, 0.17333333333333334]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.004901960784313725, 0.0046296296296296285, 0.0043859649122807015, 0.004166666666666667, 0.007936507936507934, 0.011363636363636364, 0.014492753623188406, 0.013888888888888886, 0.013333333333333334, 0.019230769230769232, 0.018518518518518517, 0.023809523809523808, 0.03160919540229885, 0.03611111111111111, 0.03763440860215054, 0.044270833333333336, 0.050505050505050504, 0.049019607843137254, 0.057142857142857134, 0.060185185185185175, 0.06981981981981981, 0.07017543859649122, 0.0811965811965812, 0.09166666666666666, 0.10162601626016259, 0.11111111111111109, 0.12015503875968993, 0.12878787878787878, 0.13333333333333333, 0.13949275362318841, 0.1524822695035462, 0.15972222222222218, 0.16666666666666666, 0.17333333333333334]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.00641025641025641, 0.011904761904761904, 0.01111111111111111, 0.010416666666666666, 0.014705882352941176, 0.013888888888888886, 0.013157894736842105, 0.020833333333333332, 0.027777777777777773, 0.026515151515151516, 0.028985507246376812, 0.027777777777777773, 0.03, 0.035256410256410256, 0.037037037037037035, 0.041666666666666664, 0.043103448275862065, 0.044444444444444446, 0.04838709677419355, 0.052083333333333336, 0.05555555555555555, 0.06372549019607843, 0.0738095238095238, 0.07407407407407406, 0.08108108108108109, 0.08552631578947369, 0.0876068376068376, 0.08958333333333333, 0.09756097560975623, 0.10515873015873028, 0.10852713178294573, 0.11363636363636363, 0.12222222222222222, 0.1358695652173913, 0.14539007092198591, 0.15624999999999997, 0.16666666666666666, 0.17333333333333334]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.09716599190283401, 0.16835016835016836, 0.17333333333333334], "coverage": [0.0, 0.0, 0.0, 0.8233333333333334, 0.99, 1.0]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.7625717276674938, "basin/entropy": 0.23485964640198512, "basin/dispersion": 0.534080334987593, "energy/mean": 0.44639810794044665, "energy/min": 0.4427341811414392, "energy/std": 0.5076574131513648, "curv/lmax_mean": 0.48588709677419356, "curv/lmax_best": 0.5200352822580645, "curv/trace_mean": 0.6061569478908189, "curv/trace_best": 0.5317249534739454, "dynamics/steps": 0.5, "dynamics/monotonic": 0.4070932847394541, "dynamics/drop": 0.3674782878411911, "dynamics/residual": 0.5121355459057072}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [5, 0, 0, 13, 0, 0, 19, 0, 0, 0, 3, 0, 0, 22, 0, 0, 47, 0, 0, 387], "incorrect_counts": [9, 0, 0, 9, 0, 0, 7, 0, 0, 0, 13, 0, 0, 21, 0, 0, 15, 0, 0, 30]}, "basin/entropy": {"edges": [0.0, 0.05057021323536759, 0.10114042647073518, 0.15171063970610277, 0.20228085294147036, 0.25285106617683795, 0.30342127941220554, 0.3539914926475731, 0.4045617058829407, 0.4551319191183083, 0.5057021323536759, 0.5562723455890435, 0.6068425588244111, 0.6574127720597787, 0.7079829852951462, 0.7585531985305138, 0.8091234117658814, 0.859693625001249, 0.9102638382366166, 0.9608340514719842, 1.0114042647073518], "correct_counts": [387, 0, 0, 0, 0, 47, 0, 0, 21, 0, 0, 4, 18, 15, 0, 0, 1, 1, 2, 0], "incorrect_counts": [30, 0, 0, 0, 0, 15, 0, 0, 19, 0, 0, 12, 5, 15, 3, 0, 1, 1, 1, 2]}, "basin/dispersion": {"edges": [1.5954158897257813, 1.616709448349816, 1.6380030069738507, 1.6592965655978853, 1.6805901242219201, 1.7018836828459547, 1.7231772414699895, 1.744470800094024, 1.765764358718059, 1.7870579173420935, 1.8083514759661283, 1.829645034590163, 1.8509385932141975, 1.8722321518382323, 1.893525710462267, 1.9148192690863017, 1.9361128277103363, 1.9574063863343711, 1.9786999449584057, 1.9999935035824405, 2.021287062206475], "correct_counts": [1, 1, 0, 4, 6, 16, 30, 35, 42, 67, 63, 66, 65, 40, 25, 10, 13, 6, 4, 2], "incorrect_counts": [0, 1, 1, 1, 7, 4, 8, 5, 10, 14, 5, 13, 12, 9, 5, 2, 3, 0, 3, 1]}, "energy/mean": {"edges": [0.7320398216446241, 0.7937728473295769, 0.8555058730145296, 0.9172388986994824, 0.978971924384435, 1.040704950069388, 1.1024379757543405, 1.1641710014392932, 1.225904027124246, 1.2876370528091987, 1.3493700784941516, 1.4111031041791042, 1.4728361298640569, 1.5345691555490097, 1.5963021812339624, 1.6580352069189153, 1.719768232603868, 1.7815012582888206, 1.8432342839737732, 1.9049673096587263, 1.9667003353436787], "correct_counts": [0, 3, 19, 23, 31, 21, 28, 15, 17, 19, 28, 29, 59, 64, 70, 36, 15, 11, 6, 2], "incorrect_counts": [1, 2, 0, 3, 5, 6, 3, 4, 8, 5, 7, 5, 9, 9, 2, 13, 9, 6, 5, 2]}, "energy/min": {"edges": [0.34151938557624817, 0.41071730107069016, 0.47991521656513214, 0.5491131320595741, 0.6183110475540161, 0.6875089630484581, 0.7567068785429001, 0.8259047940373421, 0.8951027095317841, 0.964300625026226, 1.033498540520668, 1.10269645601511, 1.171894371509552, 1.241092287003994, 1.310290202498436, 1.379488117992878, 1.44868603348732, 1.517883948981762, 1.587081864476204, 1.656279779970646, 1.725477695465088], "correct_counts": [2, 5, 4, 11, 34, 27, 26, 28, 18, 22, 40, 40, 62, 70, 49, 27, 17, 9, 5, 0], "incorrect_counts": [1, 2, 1, 0, 2, 3, 11, 3, 9, 5, 9, 7, 7, 8, 6, 10, 15, 3, 1, 1]}, "energy/std": {"edges": [0.07718251081341526, 0.0912297277006377, 0.10527694458786013, 0.11932416147508257, 0.133371378362305, 0.14741859524952744, 0.16146581213674988, 0.1755130290239723, 0.18956024591119475, 0.20360746279841718, 0.21765467968563962, 0.23170189657286205, 0.2457491134600845, 0.2597963303473069, 0.27384354723452936, 0.2878907641217518, 0.30193798100897423, 0.31598519789619667, 0.3300324147834191, 0.34407963167064154, 0.358126848557864], "correct_counts": [0, 2, 4, 17, 27, 48, 55, 74, 52, 52, 46, 41, 30, 14, 17, 9, 2, 3, 1, 2], "incorrect_counts": [1, 1, 1, 4, 8, 12, 12, 8, 8, 11, 12, 7, 6, 2, 6, 1, 3, 0, 1, 0]}, "curv/lmax_mean": {"edges": [0.9986266444126765, 0.9987140096724033, 0.9988013749321302, 0.998888740191857, 0.9989761054515838, 0.9990634707113106, 0.9991508359710375, 0.9992382012307643, 0.9993255664904912, 0.9994129317502181, 0.9995002970099449, 0.9995876622696717, 0.9996750275293985, 0.9997623927891254, 0.9998497580488522, 0.9999371233085791, 1.0000244885683058, 1.0001118538280327, 1.0001992190877596, 1.0002865843474864, 1.0003739496072133], "correct_counts": [2, 1, 3, 6, 10, 12, 20, 35, 51, 75, 77, 73, 56, 41, 19, 6, 4, 4, 1, 0], "incorrect_counts": [0, 0, 0, 2, 2, 1, 5, 4, 10, 16, 17, 18, 18, 8, 2, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.9928669929504395, 0.9932882905006408, 0.9937095880508423, 0.9941308856010437, 0.9945521831512452, 0.9949734807014465, 0.9953947782516479, 0.9958160758018494, 0.9962373733520508, 0.9966586709022522, 0.9970799684524536, 0.997501266002655, 0.9979225635528565, 0.9983438611030578, 0.9987651586532593, 0.9991864562034607, 0.9996077537536621, 1.0000290513038634, 1.000450348854065, 1.0008716464042664, 1.0012929439544678], "correct_counts": [1, 0, 0, 0, 1, 0, 0, 2, 1, 5, 7, 6, 8, 14, 28, 87, 328, 4, 1, 3], "incorrect_counts": [0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 1, 4, 3, 5, 20, 68, 0, 0, 0]}, "curv/trace_mean": {"edges": [31.902789910634358, 31.907989327112833, 31.913188743591306, 31.91838816006978, 31.923587576548258, 31.928786993026733, 31.93398640950521, 31.93918582598368, 31.944385242462157, 31.949584658940633, 31.954784075419106, 31.95998349189758, 31.965182908376057, 31.970382324854533, 31.97558174133301, 31.98078115781148, 31.985980574289957, 31.991179990768433, 31.996379407246906, 32.00157882372538, 32.00677824020386], "correct_counts": [0, 0, 0, 4, 6, 5, 11, 17, 24, 45, 66, 88, 71, 64, 27, 13, 23, 20, 7, 5], "incorrect_counts": [1, 1, 4, 2, 5, 4, 4, 8, 8, 6, 11, 10, 10, 8, 4, 8, 6, 3, 0, 1]}, "curv/trace_best": {"edges": [31.815509796142578, 31.82562713623047, 31.83574447631836, 31.84586181640625, 31.85597915649414, 31.86609649658203, 31.876213836669923, 31.886331176757814, 31.8964485168457, 31.906565856933593, 31.916683197021484, 31.926800537109376, 31.936917877197267, 31.947035217285155, 31.957152557373046, 31.967269897460938, 31.97738723754883, 31.98750457763672, 31.997621917724608, 32.0077392578125, 32.01785659790039], "correct_counts": [1, 0, 0, 0, 2, 2, 2, 2, 1, 10, 15, 25, 32, 68, 93, 93, 73, 53, 20, 4], "incorrect_counts": [0, 0, 0, 0, 0, 3, 1, 1, 1, 2, 7, 6, 9, 12, 12, 18, 15, 14, 2, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [496, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [104, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.7266666666666667, 0.7319166666666667, 0.7371666666666666, 0.7424166666666667, 0.7476666666666667, 0.7529166666666667, 0.7581666666666667, 0.7634166666666666, 0.7686666666666667, 0.7739166666666667, 0.7791666666666667, 0.7844166666666667, 0.7896666666666667, 0.7949166666666667, 0.8001666666666667, 0.8054166666666667, 0.8106666666666666, 0.8159166666666666, 0.8211666666666667, 0.8264166666666667, 0.8316666666666667], "correct_counts": [2, 5, 11, 25, 31, 42, 83, 79, 57, 62, 44, 32, 17, 1, 5, 0, 0, 0, 0, 0], "incorrect_counts": [1, 0, 6, 3, 7, 7, 7, 10, 13, 9, 14, 9, 4, 7, 1, 0, 4, 0, 0, 2]}, "dynamics/drop": {"edges": [67.27662276228268, 83.98704353620607, 100.69746431012948, 117.40788508405288, 134.11830585797628, 150.82872663189966, 167.53914740582306, 184.24956817974646, 200.95998895366984, 217.67040972759327, 234.38083050151664, 251.09125127544002, 267.80167204936345, 284.5120928232868, 301.2225135972102, 317.9329343711336, 334.643355145057, 351.3537759189804, 368.0641966929038, 384.7746174668272, 401.4850382407506], "correct_counts": [20, 48, 122, 170, 88, 26, 8, 3, 3, 2, 3, 1, 1, 1, 0, 0, 0, 0, 0, 0], "incorrect_counts": [1, 9, 21, 17, 24, 7, 4, 6, 4, 1, 2, 2, 2, 2, 0, 0, 0, 1, 0, 1]}, "dynamics/residual": {"edges": [1.1687835951646168, 1.181517025331656, 1.1942504554986952, 1.2069838856657344, 1.2197173158327739, 1.232450745999813, 1.2451841761668523, 1.2579176063338915, 1.2706510365009307, 1.28338446666797, 1.296117896835009, 1.3088513270020485, 1.3215847571690877, 1.334318187336127, 1.3470516175031662, 1.3597850476702054, 1.3725184778372446, 1.385251908004284, 1.3979853381713232, 1.4107187683383624, 1.4234521985054016], "correct_counts": [1, 3, 3, 8, 17, 32, 32, 45, 51, 56, 56, 49, 37, 46, 23, 13, 9, 10, 2, 3], "incorrect_counts": [2, 1, 0, 2, 4, 7, 7, 11, 5, 10, 13, 12, 10, 7, 8, 1, 2, 1, 1, 0]}}}, "feature_ablation": {"full": 0.054893261179459335, "drop_basin": 0.1249973182878787, "basin_only": 0.10254852464985335, "drop_energy": 0.05537729278817232, "energy_only": 0.15682846485364488, "drop_curv": 0.06023839375152389, "curv_only": 0.14618149034503106, "drop_dynamics": 0.061816201439268734, "dynamics_only": 0.13024391473078728}, "ece_geometry": 0.04766602442020401}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "d1ae14a590d9", "timestamp": "2026-07-27T10:02:10.909225+00:00", "git_sha": "5207145", "config_hash": "3643423663c7", "config": {"run": {"seed": 0, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 0, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.227946937084198, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1527490825609553, "rho_basin": 0.22247851187975562, "energy_min": 0.37245598613988906, "energy_mean": 0.3894178842534342, "energy_std": 0.28068693518661764, "msp": 0.12101715414067431, "temp_msp": 0.1204758682795991, "entropy": 0.11966617541102537}, "temperature": 2.9326538741941572, "best_energy_baseline": "energy_std", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.21970690357893377, 0.16353680784617466, 0.27315268065902204], "delta_aurc_vs_best_energy": [0.12793785262566235, 0.08274255159121499, 0.17357149948028552], "delta_aurc_vs_best_baseline": [-0.033082907149929924, -0.05463796025527004, -0.013340602564395215], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.125, 0.08333333333333333, 0.0625, 0.06666666666666667, 0.08333333333333331, 0.10714285714285696, 0.09375, 0.08333333333333333, 0.09166666666666667, 0.09090909090909091, 0.09722222222222222, 0.11538461538461539, 0.125, 0.1333333333333333, 0.13020833333333334, 0.12745098039215685, 0.12499999999999999, 0.12280701754385964, 0.12916666666666668, 0.13095238095238093, 0.13257575757575757, 0.13043478260869565, 0.13888888888888887, 0.13333333333333333, 0.14102564102564102, 0.1419753086419753, 0.1488095238095238, 0.14942528735632182, 0.15, 0.16129032258064516, 0.1640625, 0.1691919191919192, 0.17892156862745098, 0.1833333333333333, 0.18287037037037035, 0.18468468468468469, 0.18421052631578946, 0.19017094017094016, 0.20416666666666666, 0.20934959349593493, 0.21428571428571425, 0.2248062015503876, 0.23484848484848486, 0.2462962962962963, 0.25181159420289856, 0.25886524822695045, 0.26736111111111105, 0.272108843537415, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2165991902834008, 0.2165991902834009, 0.21659919028340094, 0.21659919028340077, 0.21659919028340063, 0.21659919028340055, 0.21659919028340052, 0.21659919028340047, 0.21659919028340044, 0.2165991902834004, 0.21659919028340038, 0.21659919028340036, 0.21659919028340055, 0.21659919028340077, 0.21659919028340097, 0.21659919028340113, 0.21659919028340127, 0.2165991902834014, 0.21659919028340155, 0.21659919028340163, 0.21659919028340174, 0.21659919028340183, 0.2165991902834019, 0.216599190283402, 0.21659919028340202, 0.2165991902834021, 0.21659919028340216, 0.21659919028340222, 0.21659919028340227, 0.2165991902834023, 0.21659919028340235, 0.21659919028340238, 0.21659919028340244, 0.21659919028340247, 0.2165991902834025, 0.21659919028340255, 0.21659919028340258, 0.2165991902834026, 0.21659919028340263, 0.21659919028340266, 0.2165991902834027, 0.22372534872535066, 0.23191214470284427, 0.23948863636363826, 0.24416666666666834, 0.250762776506485, 0.259985069055619, 0.2673611111111122, 0.27494331065759725, 0.2816666666666674]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4583333333333333, 0.4666666666666667, 0.45833333333333337, 0.4523809523809525, 0.4479166666666667, 0.46296296296296297, 0.42500000000000004, 0.42424242424242425, 0.4027777777777778, 0.3974358974358974, 0.4107142857142857, 0.40000000000000013, 0.4010416666666667, 0.4019607843137255, 0.40740740740740733, 0.39473684210526316, 0.4, 0.39682539682539675, 0.38636363636363635, 0.391304347826087, 0.39236111111111105, 0.38333333333333336, 0.3717948717948718, 0.36419753086419754, 0.36607142857142855, 0.3706896551724139, 0.3638888888888889, 0.3521505376344086, 0.3515625, 0.3434343434343434, 0.3382352941176471, 0.33571428571428563, 0.32870370370370366, 0.32657657657657657, 0.3223684210526316, 0.3247863247863248, 0.3229166666666667, 0.3191056910569105, 0.3115079365079366, 0.3081395348837209, 0.30303030303030304, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.28819444444444436, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.5, 0.5, 0.4375, 0.5, 0.49999999999999994, 0.4999999999999999, 0.46875, 0.4722222222222222, 0.45833333333333326, 0.44696969696969696, 0.4236111111111111, 0.4166666666666667, 0.42857142857142855, 0.4277777777777777, 0.421875, 0.4215686274509804, 0.412037037037037, 0.41228070175438597, 0.3958333333333333, 0.39682539682539686, 0.4015151515151515, 0.39492753623188404, 0.39583333333333326, 0.3933333333333333, 0.38782051282051283, 0.37962962962962965, 0.37202380952380953, 0.36781609195402293, 0.3611111111111111, 0.3548387096774194, 0.3463541666666667, 0.3383838383838384, 0.33578431372549017, 0.3309523809523809, 0.33101851851851866, 0.32882882882882886, 0.3267543859649123, 0.32264957264957267, 0.32083333333333336, 0.31707317073170727, 0.31349206349206343, 0.3081395348837209, 0.30303030303030304, 0.30185185185185187, 0.2971014492753623, 0.29078014184397155, 0.28645833333333326, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.25, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.33333333333333326, 0.3214285714285715, 0.3020833333333333, 0.3055555555555556, 0.2833333333333334, 0.2727272727272727, 0.2847222222222222, 0.28846153846153844, 0.27976190476190477, 0.28333333333333327, 0.2760416666666667, 0.27450980392156865, 0.26388888888888884, 0.2675438596491228, 0.2791666666666667, 0.26984126984126994, 0.2803030303030303, 0.2753623188405797, 0.26736111111111105, 0.26, 0.266025641025641, 0.2623456790123457, 0.2648809523809524, 0.26724137931034475, 0.26944444444444443, 0.27419354838709675, 0.2734375, 0.2727272727272727, 0.27941176470588236, 0.2738095238095238, 0.2708333333333334, 0.2725225225225225, 0.2741228070175439, 0.27136752136752135, 0.27708333333333335, 0.2804878048780489, 0.28174603174603186, 0.2810077519379845, 0.2821969696969697, 0.2814814814814815, 0.28442028985507245, 0.2836879432624113, 0.27951388888888895, 0.2789115646258503, 0.2816666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.05555555555555555, 0.05128205128205128, 0.047619047619047616, 0.055555555555555546, 0.052083333333333336, 0.06372549019607843, 0.060185185185185175, 0.06140350877192982, 0.0625, 0.07142857142857155, 0.0946969696969697, 0.09420289855072464, 0.10416666666666666, 0.11, 0.125, 0.12962962962962962, 0.13392857142857142, 0.1379310344827586, 0.14166666666666666, 0.14516129032258066, 0.1484375, 0.1590909090909091, 0.15931372549019607, 0.16428571428571442, 0.1759259259259259, 0.18243243243243243, 0.19298245614035087, 0.20085470085470086, 0.20625, 0.20528455284552857, 0.21428571428571438, 0.22286821705426357, 0.22916666666666666, 0.24074074074074073, 0.2554347826086957, 0.26063829787234055, 0.26736111111111105, 0.27040816326530615, 0.2816666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.027777777777777776, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.05555555555555555, 0.05000000000000001, 0.05303030303030303, 0.04861111111111111, 0.05128205128205128, 0.05357142857142857, 0.050000000000000176, 0.057291666666666664, 0.058823529411764705, 0.055555555555555546, 0.06578947368421052, 0.07083333333333333, 0.08333333333333331, 0.08712121212121213, 0.09420289855072464, 0.10763888888888888, 0.11333333333333333, 0.125, 0.12654320987654322, 0.12797619047619047, 0.1350574712643678, 0.1388888888888889, 0.13709677419354838, 0.14322916666666666, 0.16161616161616163, 0.16176470588235295, 0.1714285714285714, 0.17824074074074087, 0.18468468468468469, 0.18859649122807018, 0.19658119658119658, 0.20416666666666666, 0.20934959349593507, 0.2103174603174603, 0.22093023255813954, 0.22537878787878787, 0.23333333333333334, 0.24094202898550723, 0.24822695035461, 0.2621527777777779, 0.27380952380952384, 0.2816666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.03333333333333333, 0.04166666666666666, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.05555555555555555, 0.05128205128205128, 0.047619047619047616, 0.055555555555555546, 0.052083333333333336, 0.06372549019607843, 0.060185185185185175, 0.06140350877192982, 0.0625, 0.07142857142857155, 0.09090909090909091, 0.09420289855072464, 0.10416666666666666, 0.11333333333333333, 0.125, 0.12962962962962962, 0.13392857142857142, 0.13505747126436798, 0.1388888888888889, 0.1424731182795699, 0.1484375, 0.15404040404040403, 0.16176470588235295, 0.1666666666666668, 0.17592592592592607, 0.18693693693693694, 0.18640350877192982, 0.19230769230769232, 0.19791666666666666, 0.20731707317073167, 0.21031746031746043, 0.22093023255813954, 0.22727272727272727, 0.2351851851851852, 0.2391304347826087, 0.2500000000000001, 0.2604166666666668, 0.27380952380952384, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.14285714285714285, 0.2527075812274368], "coverage": [0.0, 0.0, 0.0, 0.0, 0.525, 0.9233333333333333]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6337264377599913, "basin/entropy": 0.36597839069729127, "basin/dispersion": 0.5371984788368869, "energy/mean": 0.6647400431087741, "energy/min": 0.6541275964799077, "energy/std": 0.4885020387429811, "curv/lmax_mean": 0.3431952662721893, "curv/lmax_best": 0.3816018890978734, "curv/trace_mean": 0.3301253449388377, "curv/trace_best": 0.3412526256538393, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6490067134364832, "dynamics/drop": 0.6809401556858277, "dynamics/residual": 0.5138044179629044}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [4, 0, 0, 3, 0, 0, 6, 0, 0, 0, 6, 0, 0, 11, 0, 0, 14, 0, 0, 387], "incorrect_counts": [6, 0, 0, 6, 0, 0, 9, 0, 0, 0, 13, 0, 0, 9, 0, 0, 19, 0, 0, 107]}, "basin/entropy": {"edges": [0.0, 0.04592138924965487, 0.09184277849930975, 0.1377641677489646, 0.1836855569986195, 0.22960694624827438, 0.2755283354979292, 0.3214497247475841, 0.367371113997239, 0.41329250324689387, 0.45921389249654876, 0.5051352817462036, 0.5510566709958584, 0.5969780602455134, 0.6428994494951682, 0.6888208387448231, 0.734742227994478, 0.7806636172441328, 0.8265850064937877, 0.8725063957434426, 0.9184277849930975], "correct_counts": [387, 0, 0, 0, 0, 0, 14, 0, 0, 10, 0, 0, 7, 6, 3, 4, 0, 0, 0, 0], "incorrect_counts": [107, 0, 0, 0, 0, 0, 19, 0, 0, 9, 0, 0, 12, 7, 6, 6, 0, 2, 0, 1]}, "basin/dispersion": {"edges": [1.620605021245252, 1.641571274378292, 1.662537527511332, 1.683503780644372, 1.7044700337774121, 1.725436286910452, 1.746402540043492, 1.767368793176532, 1.788335046309572, 1.809301299442612, 1.830267552575652, 1.851233805708692, 1.872200058841732, 1.893166311974772, 1.914132565107812, 1.935098818240852, 1.9560650713738919, 1.977031324506932, 1.9979975776399719, 2.018963830773012, 2.039930083906052], "correct_counts": [0, 2, 2, 8, 13, 22, 28, 29, 46, 69, 50, 43, 35, 31, 20, 17, 7, 7, 1, 1], "incorrect_counts": [1, 3, 0, 5, 2, 6, 17, 14, 22, 23, 22, 15, 14, 8, 13, 2, 1, 1, 0, 0]}, "energy/mean": {"edges": [0.6107773731152216, 0.6431646155814329, 0.6755518580476443, 0.7079391005138556, 0.7403263429800669, 0.7727135854462782, 0.8051008279124896, 0.8374880703787009, 0.8698753128449122, 0.9022625553111235, 0.9346497977773349, 0.9670370402435462, 0.9994242827097575, 1.0318115251759687, 1.0641987676421802, 1.0965860101083915, 1.1289732525746028, 1.161360495040814, 1.1937477375070253, 1.2261349799732368, 1.2585222224394481], "correct_counts": [1, 3, 8, 10, 11, 27, 35, 32, 45, 36, 55, 40, 46, 27, 21, 12, 15, 3, 3, 1], "incorrect_counts": [4, 1, 7, 6, 18, 17, 16, 20, 24, 11, 12, 11, 11, 5, 1, 1, 2, 1, 0, 1]}, "energy/min": {"edges": [0.20125961303710938, 0.2415154755115509, 0.2817713379859924, 0.322027200460434, 0.3622830629348755, 0.402538925409317, 0.4427947878837586, 0.4830506503582001, 0.5233065128326416, 0.5635623753070831, 0.6038182377815247, 0.6440741002559662, 0.6843299627304078, 0.7245858252048493, 0.7648416876792908, 0.8050975501537323, 0.8453534126281739, 0.8856092751026154, 0.9258651375770569, 0.9661210000514985, 1.00637686252594], "correct_counts": [1, 1, 6, 2, 13, 14, 29, 37, 39, 39, 58, 46, 39, 44, 24, 21, 10, 1, 3, 4], "incorrect_counts": [1, 0, 2, 4, 12, 13, 19, 20, 22, 21, 17, 10, 14, 5, 4, 2, 1, 2, 0, 0]}, "energy/std": {"edges": [0.06508645292670491, 0.08179583921728364, 0.09850522550786239, 0.11521461179844114, 0.1319239980890199, 0.1486333843795986, 0.16534277067017736, 0.1820521569607561, 0.19876154325133483, 0.2154709295419136, 0.23218031583249232, 0.24888970212307104, 0.2655990884136498, 0.28230847470422854, 0.29901786099480726, 0.31572724728538604, 0.33243663357596476, 0.3491460198665435, 0.36585540615712225, 0.382564792447701, 0.3992741787382797], "correct_counts": [1, 3, 7, 18, 27, 54, 60, 71, 59, 43, 31, 26, 20, 6, 1, 2, 2, 0, 0, 0], "incorrect_counts": [1, 0, 3, 10, 13, 16, 22, 21, 29, 13, 18, 11, 6, 2, 1, 0, 1, 0, 1, 1]}, "curv/lmax_mean": {"edges": [0.9988154868284861, 0.9990137209494908, 0.9992119550704955, 0.9994101891915003, 0.9996084233125051, 0.9998066574335098, 1.0000048915545146, 1.0002031256755193, 1.000401359796524, 1.0005995939175287, 1.0007978280385335, 1.0009960621595382, 1.001194296280543, 1.0013925304015476, 1.0015907645225526, 1.0017889986435573, 1.001987232764562, 1.0021854668855668, 1.0023837010065715, 1.0025819351275762, 1.002780169248581], "correct_counts": [2, 2, 10, 38, 119, 132, 72, 24, 11, 6, 10, 2, 1, 0, 0, 1, 0, 0, 0, 1], "incorrect_counts": [1, 0, 3, 9, 21, 43, 36, 30, 11, 5, 3, 1, 3, 2, 1, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.9958722591400146, 0.9962057709693909, 0.9965392827987671, 0.9968727946281433, 0.9972063064575195, 0.9975398182868958, 0.997873330116272, 0.9982068419456482, 0.9985403537750244, 0.9988738656044006, 0.9992073774337769, 0.9995408892631531, 0.9998744010925293, 1.0002079129219055, 1.0005414247512818, 1.000874936580658, 1.001208448410034, 1.0015419602394104, 1.0018754720687866, 1.002208983898163, 1.002542495727539], "correct_counts": [0, 0, 0, 0, 0, 1, 3, 4, 8, 10, 25, 92, 246, 23, 5, 1, 6, 4, 2, 1], "incorrect_counts": [1, 0, 0, 0, 0, 0, 1, 0, 1, 6, 6, 26, 85, 24, 6, 6, 3, 2, 2, 0]}, "curv/trace_mean": {"edges": [31.95863151550293, 31.962152846654256, 31.96567417780558, 31.969195508956908, 31.972716840108234, 31.97623817125956, 31.97975950241089, 31.983280833562215, 31.98680216471354, 31.990323495864867, 31.993844827016193, 31.99736615816752, 32.00088748931885, 32.004408820470175, 32.0079301516215, 32.01145148277283, 32.01497281392415, 32.01849414507548, 32.022015476226805, 32.02553680737813, 32.02905813852946], "correct_counts": [1, 0, 2, 7, 14, 14, 24, 46, 62, 37, 62, 49, 47, 21, 19, 14, 3, 2, 5, 2], "incorrect_counts": [0, 0, 0, 0, 0, 3, 4, 7, 10, 20, 17, 21, 31, 18, 22, 9, 3, 4, 0, 0]}, "curv/trace_best": {"edges": [31.937339782714844, 31.943468475341795, 31.94959716796875, 31.955725860595702, 31.961854553222658, 31.96798324584961, 31.97411193847656, 31.980240631103516, 31.986369323730468, 31.992498016357423, 31.998626708984375, 32.00475540161133, 32.01088409423828, 32.01701278686524, 32.02314147949219, 32.02927017211914, 32.03539886474609, 32.041527557373044, 32.04765625, 32.053784942626955, 32.059913635253906], "correct_counts": [2, 1, 3, 7, 6, 15, 18, 43, 86, 101, 89, 32, 11, 11, 4, 1, 0, 0, 1, 0], "incorrect_counts": [0, 0, 0, 0, 1, 3, 6, 6, 17, 43, 37, 26, 15, 10, 4, 0, 0, 0, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6466666666666666, 0.6523333333333333, 0.6579999999999999, 0.6636666666666666, 0.6693333333333332, 0.6749999999999999, 0.6806666666666666, 0.6863333333333332, 0.692, 0.6976666666666665, 0.7033333333333333, 0.709, 0.7146666666666666, 0.7203333333333333, 0.7259999999999999, 0.7316666666666666, 0.7373333333333333, 0.7429999999999999, 0.7486666666666666, 0.7543333333333332, 0.7599999999999999], "correct_counts": [0, 5, 4, 8, 19, 36, 33, 63, 55, 44, 53, 28, 27, 22, 10, 12, 6, 2, 1, 3], "incorrect_counts": [2, 3, 7, 9, 14, 22, 20, 25, 18, 11, 18, 8, 5, 6, 0, 0, 1, 0, 0, 0]}, "dynamics/drop": {"edges": [13.273132185141245, 17.449723252654074, 21.626314320166905, 25.802905387679733, 29.97949645519256, 34.15608752270539, 38.33267859021822, 42.509269657731046, 46.68586072524388, 50.86245179275671, 55.03904286026954, 59.21563392778236, 63.392224995295194, 67.56881606280803, 71.74540713032086, 75.92199819783369, 80.09858926534652, 84.27518033285935, 88.45177140037218, 92.62836246788501, 96.80495353539784], "correct_counts": [12, 54, 92, 87, 56, 46, 29, 19, 11, 9, 4, 2, 5, 0, 2, 2, 0, 0, 0, 1], "incorrect_counts": [24, 44, 39, 23, 17, 11, 7, 0, 1, 2, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471606890360515, 1.1608674337466558, 1.1745741784572603, 1.1882809231678646, 1.201987667878469, 1.2156944125890732, 1.2294011572996775, 1.243107902010282, 1.2568146467208863, 1.2705213914314906, 1.284228136142095, 1.2979348808526994, 1.3116416255633037, 1.325348370273908, 1.3390551149845122, 1.3527618596951168, 1.366468604405721, 1.3801753491163253, 1.3938820938269298, 1.4075888385375341, 1.4212955832481384], "correct_counts": [1, 1, 3, 4, 7, 18, 32, 34, 40, 41, 45, 44, 52, 38, 27, 26, 10, 4, 2, 2], "incorrect_counts": [1, 0, 0, 2, 4, 10, 10, 14, 15, 14, 24, 17, 18, 12, 10, 8, 5, 5, 0, 0]}}}, "feature_ablation": {"full": 0.1527490825609553, "drop_basin": 0.16710255496589052, "basin_only": 0.19644399803458065, "drop_energy": 0.1497772096515114, "energy_only": 0.1918777359298754, "drop_curv": 0.15631908398785066, "curv_only": 0.17337743240377687, "drop_dynamics": 0.16367135018146997, "dynamics_only": 0.17732263226119244}, "ece_geometry": 0.04034562456459658}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "452500308f3b", "timestamp": "2026-07-27T10:02:16.480055+00:00", "git_sha": "5207145", "config_hash": "60da27bc4d66", "config": {"run": {"seed": 1, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 1, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6933333333333334, "base_error": 0.30666666666666664, "final_train_loss": 0.2080264687538147, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.1675196272459423, "rho_basin": 0.25457099646739273, "energy_min": 0.19884075586381236, "energy_mean": 0.19759334670590525, "energy_std": 0.29045958157255325, "msp": 0.12979645957506092, "temp_msp": 0.12794473890187164, "entropy": 0.12914511701061224}, "temperature": 2.814063136720987, "best_energy_baseline": "energy_mean", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.031321128617870064, 0.011535510095268008, 0.05241587205968214], "delta_aurc_vs_best_energy": [0.030073719459962955, 0.008993436060296952, 0.05227588726640192], "delta_aurc_vs_best_baseline": [-0.03957488834407066, -0.057365171281437685, -0.021981574766360788], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.08333333333333333, 0.05555555555555555, 0.041666666666666664, 0.05, 0.04166666666666666, 0.03571428571428572, 0.0625, 0.05555555555555555, 0.08333333333333334, 0.09090909090909091, 0.10416666666666667, 0.09615384615384616, 0.10714285714285714, 0.11666666666666664, 0.11979166666666667, 0.12254901960784313, 0.13888888888888887, 0.13596491228070176, 0.14166666666666666, 0.14682539682539694, 0.15151515151515152, 0.15942028985507245, 0.16666666666666663, 0.16666666666666666, 0.1762820512820513, 0.17901234567901234, 0.1875, 0.1954022988505747, 0.2, 0.20430107526881722, 0.20572916666666666, 0.21212121212121213, 0.21568627450980393, 0.22142857142857153, 0.22916666666666663, 0.23198198198198197, 0.2412280701754386, 0.24572649572649571, 0.25833333333333336, 0.26219512195121947, 0.2658730158730158, 0.2693798449612403, 0.2784090909090909, 0.2796296296296296, 0.2807971014492754, 0.2854609929078015, 0.29166666666666674, 0.3010204081632653, 0.30666666666666664]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2489878542510122, 0.24898785425101208, 0.24898785425101208, 0.24898785425101222, 0.2489878542510123, 0.24898785425101236, 0.2489878542510124, 0.24898785425101244, 0.24898785425101247, 0.2489878542510125, 0.2489878542510125, 0.24898785425101252, 0.24898785425101252, 0.24898785425101252, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101255, 0.24898785425101255, 0.24898785425101255, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.24898785425101258, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.2489878542510126, 0.25492831541218686, 0.26175293823455936, 0.2689393939393948, 0.2777777777777784, 0.28623188405797145, 0.29078014184397205, 0.29714209401709435, 0.3030990173847319, 0.3066666666666668]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.1111111111111111, 0.125, 0.11666666666666667, 0.11111111111111109, 0.09523809523809525, 0.11458333333333333, 0.1388888888888889, 0.1416666666666667, 0.1590909090909091, 0.1597222222222222, 0.16025641025641027, 0.16666666666666666, 0.17222222222222236, 0.18229166666666666, 0.19117647058823528, 0.19444444444444442, 0.18859649122807018, 0.19166666666666668, 0.18650793650793662, 0.19318181818181818, 0.1956521739130435, 0.20833333333333331, 0.2, 0.19230769230769232, 0.20679012345679013, 0.20833333333333334, 0.2155172413793103, 0.2222222222222222, 0.22311827956989247, 0.22395833333333334, 0.22474747474747475, 0.22794117647058823, 0.23571428571428582, 0.2384259259259259, 0.24549549549549549, 0.2543859649122807, 0.2606837606837607, 0.26666666666666666, 0.27032520325203246, 0.26785714285714296, 0.27325581395348836, 0.2840909090909091, 0.28888888888888886, 0.2898550724637681, 0.2943262411347517, 0.29513888888888895, 0.3010204081632653, 0.30666666666666664]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.08333333333333333, 0.05555555555555555, 0.0625, 0.08333333333333333, 0.11111111111111109, 0.10714285714285716, 0.11458333333333333, 0.12962962962962962, 0.12500000000000003, 0.12121212121212122, 0.11805555555555555, 0.14743589743589744, 0.14285714285714285, 0.1611111111111113, 0.17708333333333334, 0.18137254901960784, 0.18518518518518515, 0.17982456140350878, 0.17916666666666667, 0.17857142857142855, 0.17045454545454544, 0.18478260869565216, 0.18402777777777776, 0.19666666666666666, 0.20833333333333334, 0.2006172839506173, 0.2113095238095238, 0.2097701149425287, 0.21666666666666667, 0.22580645161290322, 0.22916666666666666, 0.2297979797979798, 0.2426470588235294, 0.24047619047619045, 0.24074074074074084, 0.24774774774774774, 0.2565789473684211, 0.26282051282051283, 0.26458333333333334, 0.26626016260162594, 0.2678571428571428, 0.2655038759689923, 0.2708333333333333, 0.2740740740740741, 0.2807971014492754, 0.28900709219858167, 0.29687499999999994, 0.30272108843537415, 0.30666666666666664]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.16666666666666666, 0.19444444444444445, 0.22916666666666666, 0.25, 0.26388888888888884, 0.261904761904762, 0.2708333333333333, 0.26851851851851855, 0.2833333333333334, 0.30303030303030304, 0.2986111111111111, 0.28846153846153844, 0.30357142857142855, 0.29999999999999993, 0.3177083333333333, 0.3235294117647059, 0.3101851851851851, 0.2982456140350877, 0.3, 0.2896825396825398, 0.29924242424242425, 0.30434782608695654, 0.30208333333333326, 0.31, 0.3108974358974359, 0.3148148148148148, 0.31845238095238093, 0.3189655172413793, 0.32222222222222224, 0.31451612903225806, 0.3229166666666667, 0.3181818181818182, 0.31862745098039214, 0.31904761904761897, 0.31481481481481477, 0.31756756756756754, 0.3157894736842105, 0.3141025641025641, 0.3125, 0.306910569105691, 0.3055555555555557, 0.3062015503875969, 0.3068181818181818, 0.3037037037037037, 0.3061594202898551, 0.30673758865248224, 0.3055555555555555, 0.304421768707483, 0.30666666666666664]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761906, 0.020833333333333332, 0.037037037037037035, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.02564102564102564, 0.02976190476190476, 0.027777777777777773, 0.052083333333333336, 0.05392156862745098, 0.0648148148148148, 0.07456140350877193, 0.07916666666666666, 0.09126984126984125, 0.09090909090909091, 0.09420289855072464, 0.11111111111111109, 0.12333333333333334, 0.13141025641025642, 0.14506172839506173, 0.15178571428571427, 0.16091954022988522, 0.1638888888888889, 0.1774193548387097, 0.18489583333333334, 0.1893939393939394, 0.20098039215686275, 0.19761904761904775, 0.20370370370370366, 0.20945945945945946, 0.21929824561403508, 0.2264957264957265, 0.23541666666666666, 0.23983739837398385, 0.2500000000000001, 0.2558139534883721, 0.26325757575757575, 0.2759259259259259, 0.2826086956521739, 0.2872340425531916, 0.29513888888888884, 0.29931972789115646, 0.30666666666666664]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.010416666666666666, 0.018518518518518517, 0.025000000000000005, 0.030303030303030304, 0.034722222222222224, 0.03205128205128205, 0.02976190476190476, 0.03888888888888888, 0.036458333333333336, 0.04411764705882353, 0.05092592592592607, 0.06140350877192982, 0.0625, 0.07539682539682552, 0.09090909090909091, 0.11231884057971014, 0.12152777777777776, 0.12333333333333334, 0.1346153846153846, 0.1419753086419753, 0.1488095238095238, 0.16091954022988522, 0.17222222222222222, 0.18010752688172044, 0.1875, 0.18686868686868688, 0.19117647058823528, 0.19999999999999998, 0.19907407407407418, 0.21396396396396397, 0.22149122807017543, 0.2264957264957265, 0.2375, 0.2439024390243902, 0.24404761904761915, 0.25387596899224807, 0.26136363636363635, 0.26666666666666666, 0.27355072463768115, 0.2836879432624115, 0.29166666666666674, 0.3010204081632653, 0.30666666666666664]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.030303030303030304, 0.027777777777777776, 0.03205128205128205, 0.02976190476190476, 0.027777777777777773, 0.052083333333333336, 0.05392156862745098, 0.06018518518518533, 0.07456140350877193, 0.07916666666666666, 0.08730158730158728, 0.09090909090909091, 0.09420289855072464, 0.1111111111111112, 0.12666666666666668, 0.1346153846153846, 0.1419753086419753, 0.15476190476190477, 0.16379310344827583, 0.1638888888888889, 0.1774193548387097, 0.1875, 0.1919191919191919, 0.19117647058823528, 0.19999999999999998, 0.20370370370370366, 0.2072072072072072, 0.21271929824561403, 0.22435897435897437, 0.23333333333333334, 0.2439024390243902, 0.2500000000000001, 0.2596899224806202, 0.26325757575757575, 0.26851851851851855, 0.27355072463768115, 0.28546099290780136, 0.29166666666666663, 0.3010204081632653, 0.30666666666666664]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.15985130111524162, 0.24301075268817204], "coverage": [0.0, 0.0, 0.0, 0.0, 0.4483333333333333, 0.775]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6111648202341137, "basin/entropy": 0.3888286475752508, "basin/dispersion": 0.505382525083612, "energy/mean": 0.32454535953177255, "energy/min": 0.3349576714046823, "energy/std": 0.5020641722408027, "curv/lmax_mean": 0.46143394648829433, "curv/lmax_best": 0.5073879076086957, "curv/trace_mean": 0.4757133152173913, "curv/trace_best": 0.4546535326086957, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6186833716555183, "dynamics/drop": 0.6482545986622074, "dynamics/residual": 0.4986151755852843}, "hist": {"basin/rho": {"edges": [0.3333333333333333, 0.36666666666666664, 0.4, 0.43333333333333335, 0.4666666666666667, 0.5, 0.5333333333333333, 0.5666666666666667, 0.6000000000000001, 0.6333333333333333, 0.6666666666666667, 0.7000000000000001, 0.7333333333333334, 0.7666666666666667, 0.8, 0.8333333333333335, 0.8666666666666667, 0.9000000000000001, 0.9333333333333333, 0.9666666666666668, 1.0], "correct_counts": [0, 0, 0, 0, 0, 4, 0, 5, 0, 4, 0, 0, 9, 0, 9, 0, 0, 14, 0, 371], "incorrect_counts": [1, 0, 0, 0, 0, 3, 0, 4, 0, 9, 0, 0, 9, 0, 18, 0, 0, 17, 0, 123]}, "basin/entropy": {"edges": [0.0, 0.05493061443340548, 0.10986122886681096, 0.16479184330021646, 0.21972245773362192, 0.2746530721670274, 0.3295836866004329, 0.3845143010338384, 0.43944491546724385, 0.4943755299006493, 0.5493061443340548, 0.6042367587674603, 0.6591673732008658, 0.7140979876342712, 0.7690286020676768, 0.8239592165010822, 0.8788898309344877, 0.9338204453678932, 0.9887510598012986, 1.0436816742347041, 1.0986122886681096], "correct_counts": [371, 0, 0, 0, 0, 14, 0, 0, 9, 0, 8, 3, 8, 1, 0, 1, 1, 0, 0, 0], "incorrect_counts": [123, 0, 0, 0, 0, 17, 0, 0, 18, 0, 8, 9, 4, 1, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.6302329015759547, 1.6519526525410289, 1.6736724035061028, 1.6953921544711767, 1.717111905436251, 1.738831656401325, 1.760551407366399, 1.782271158331473, 1.8039909092965472, 1.8257106602616213, 1.8474304112266953, 1.8691501621917692, 1.8908699131568434, 1.9125896641219176, 1.9343094150869915, 1.9560291660520654, 1.9777489170171396, 1.9994686679822138, 2.021188418947288, 2.0429081699123617, 2.064627920877436], "correct_counts": [0, 4, 8, 11, 19, 21, 38, 49, 52, 38, 51, 40, 27, 21, 15, 10, 5, 5, 1, 1], "incorrect_counts": [2, 1, 2, 3, 11, 7, 21, 24, 18, 17, 23, 18, 20, 8, 3, 3, 1, 1, 1, 0]}, "energy/mean": {"edges": [-0.2394584914048513, -0.1896760197977225, -0.13989354819059374, -0.09011107658346496, -0.04032860497633617, 0.009453866630792618, 0.05923633823792138, 0.10901880984505016, 0.15880128145217895, 0.20858375305930774, 0.25836622466643655, 0.3081486962735652, 0.357931167880694, 0.4077136394878228, 0.4574961110949516, 0.5072785827020804, 0.5570610543092092, 0.606843525916338, 0.6566259975234667, 0.7064084691305955, 0.7561909407377243], "correct_counts": [1, 1, 6, 20, 16, 22, 43, 36, 38, 42, 43, 28, 32, 32, 31, 9, 9, 4, 1, 2], "incorrect_counts": [0, 0, 2, 0, 0, 6, 8, 12, 12, 12, 20, 17, 19, 22, 14, 19, 14, 4, 1, 2]}, "energy/min": {"edges": [-0.5504263639450073, -0.4960724890232086, -0.4417186141014099, -0.38736473917961123, -0.3330108642578125, -0.2786569893360138, -0.22430311441421513, -0.16994923949241642, -0.1155953645706177, -0.06124148964881898, -0.006887614727020264, 0.0474662601947784, 0.10182013511657706, 0.15617401003837583, 0.2105278849601745, 0.26488175988197327, 0.31923563480377193, 0.3735895097255706, 0.42794338464736936, 0.48229725956916814, 0.5366511344909668], "correct_counts": [3, 3, 8, 18, 24, 25, 35, 44, 51, 45, 37, 35, 26, 22, 17, 14, 5, 0, 3, 1], "incorrect_counts": [0, 0, 0, 4, 4, 3, 11, 16, 11, 17, 19, 18, 25, 20, 14, 12, 5, 1, 3, 1]}, "energy/std": {"edges": [0.07470156430161684, 0.08958366354400084, 0.10446576278638484, 0.11934786202876882, 0.13422996127115283, 0.14911206051353681, 0.1639941597559208, 0.17887625899830478, 0.1937583582406888, 0.20864045748307278, 0.2235225567254568, 0.23840465596784077, 0.25328675521022476, 0.26816885445260874, 0.2830509536949928, 0.29793305293737676, 0.31281515217976075, 0.32769725142214473, 0.3425793506645287, 0.35746144990691275, 0.37234354914929674], "correct_counts": [2, 3, 10, 21, 30, 45, 56, 53, 49, 37, 39, 29, 16, 15, 4, 0, 4, 1, 1, 1], "incorrect_counts": [0, 0, 4, 7, 12, 22, 27, 26, 28, 16, 9, 13, 9, 4, 2, 4, 1, 0, 0, 0]}, "curv/lmax_mean": {"edges": [1.000376323858897, 1.0019486089547476, 1.0035208940505982, 1.0050931791464488, 1.0066654642422994, 1.00823774933815, 1.0098100344340006, 1.0113823195298512, 1.0129546046257019, 1.0145268897215525, 1.0160991748174033, 1.017671459913254, 1.0192437450091045, 1.0208160301049551, 1.0223883152008058, 1.0239606002966564, 1.025532885392507, 1.0271051704883576, 1.0286774555842082, 1.0302497406800588, 1.0318220257759094], "correct_counts": [40, 93, 93, 78, 31, 33, 12, 13, 9, 8, 2, 1, 0, 2, 0, 0, 0, 0, 0, 1], "incorrect_counts": [9, 39, 42, 33, 23, 16, 11, 0, 6, 1, 4, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [1.000000238418579, 1.0040149986743927, 1.0080297589302063, 1.01204451918602, 1.0160592794418335, 1.020074039697647, 1.0240887999534607, 1.0281035602092743, 1.032118320465088, 1.0361330807209015, 1.040147840976715, 1.0441626012325287, 1.0481773614883423, 1.0521921217441559, 1.0562068819999695, 1.060221642255783, 1.0642364025115967, 1.0682511627674103, 1.0722659230232239, 1.0762806832790375, 1.080295443534851], "correct_counts": [260, 66, 30, 25, 15, 5, 4, 3, 3, 1, 3, 0, 0, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [112, 33, 17, 3, 4, 4, 6, 2, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [32.01093864440918, 32.016456858317056, 32.02197507222493, 32.02749328613281, 32.03301150004069, 32.03852971394857, 32.04404792785645, 32.04956614176432, 32.0550843556722, 32.06060256958008, 32.06612078348796, 32.07163899739584, 32.077157211303714, 32.08267542521159, 32.08819363911947, 32.093711853027344, 32.09923006693522, 32.104748280843104, 32.11026649475098, 32.11578470865886, 32.121302922566734], "correct_counts": [3, 1, 12, 19, 29, 44, 48, 53, 54, 50, 33, 21, 16, 11, 7, 8, 3, 1, 1, 2], "incorrect_counts": [0, 0, 0, 7, 12, 17, 25, 30, 20, 25, 17, 14, 4, 3, 4, 2, 2, 1, 1, 0]}, "curv/trace_best": {"edges": [32.00436782836914, 32.01764259338379, 32.03091735839844, 32.04419212341308, 32.05746688842773, 32.07074165344238, 32.08401641845703, 32.09729118347168, 32.110565948486325, 32.123840713500975, 32.137115478515625, 32.150390243530275, 32.163665008544925, 32.17693977355957, 32.19021453857422, 32.20348930358887, 32.21676406860352, 32.23003883361817, 32.24331359863281, 32.25658836364746, 32.26986312866211], "correct_counts": [22, 78, 72, 75, 64, 36, 21, 20, 10, 6, 4, 3, 1, 2, 0, 1, 0, 0, 0, 1], "incorrect_counts": [5, 30, 32, 33, 27, 17, 16, 6, 8, 4, 3, 2, 1, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [416, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [184, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6516666666666666, 0.6576666666666666, 0.6636666666666666, 0.6696666666666666, 0.6756666666666666, 0.6816666666666666, 0.6876666666666666, 0.6936666666666667, 0.6996666666666667, 0.7056666666666667, 0.7116666666666667, 0.7176666666666667, 0.7236666666666667, 0.7296666666666668, 0.7356666666666668, 0.7416666666666668, 0.7476666666666668, 0.7536666666666668, 0.7596666666666668, 0.7656666666666668, 0.7716666666666668], "correct_counts": [2, 8, 8, 19, 25, 39, 49, 45, 61, 51, 35, 24, 20, 10, 8, 4, 5, 0, 1, 2], "incorrect_counts": [1, 7, 8, 17, 12, 25, 30, 19, 22, 15, 15, 6, 0, 1, 1, 3, 1, 0, 1, 0]}, "dynamics/drop": {"edges": [13.908043444156647, 21.40915985889733, 28.91027627363801, 36.41139268837869, 43.912509103119376, 51.413625517860055, 58.914741932600734, 66.41585834734141, 73.9169747620821, 81.41809117682278, 88.91920759156346, 96.42032400630414, 103.92144042104482, 111.4225568357855, 118.92367325052619, 126.42478966526687, 133.92590608000756, 141.42702249474823, 148.92813890948892, 156.4292553242296, 163.93037173897028], "correct_counts": [76, 122, 102, 58, 24, 17, 8, 2, 1, 2, 2, 0, 0, 0, 1, 0, 0, 0, 0, 1], "incorrect_counts": [61, 68, 30, 16, 4, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1471071392297745, 1.1626865026851496, 1.1782658661405245, 1.1938452295958997, 1.2094245930512746, 1.2250039565066497, 1.2405833199620246, 1.2561626834173998, 1.2717420468727747, 1.2873214103281498, 1.302900773783525, 1.3184801372389, 1.3340595006942748, 1.34963886414965, 1.365218227605025, 1.3807975910604, 1.3963769545157751, 1.41195631797115, 1.4275356814265252, 1.4431150448819001, 1.4586944083372753], "correct_counts": [2, 3, 7, 6, 24, 31, 42, 53, 54, 48, 56, 24, 33, 15, 10, 3, 4, 0, 0, 1], "incorrect_counts": [0, 0, 4, 3, 15, 13, 21, 18, 15, 27, 29, 14, 17, 4, 2, 1, 0, 0, 0, 1]}}}, "feature_ablation": {"full": 0.1675196272459423, "drop_basin": 0.1955759205956289, "basin_only": 0.25967990160782134, "drop_energy": 0.18472321594142074, "energy_only": 0.19478984592070658, "drop_curv": 0.1689688648439061, "curv_only": 0.30469050438624096, "drop_dynamics": 0.16453164505536297, "dynamics_only": 0.21399042145048885}, "ece_geometry": 0.04224579057460222}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "b1e83969abe5", "timestamp": "2026-07-27T10:02:22.133565+00:00", "git_sha": "5207145", "config_hash": "ff22f3433ff5", "config": {"run": {"seed": 2, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 2, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.6683333333333333, "base_error": 0.33166666666666667, "final_train_loss": 0.178430438041687, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.16709451083285565, "rho_basin": 0.28365045788141036, "energy_min": 0.37782573026863475, "energy_mean": 0.3915673002869955, "energy_std": 0.35511939449119495, "msp": 0.14732856097974725, "temp_msp": 0.13701445964444764, "entropy": 0.14443399349080804}, "temperature": 3.139475832001513, "best_energy_baseline": "energy_std", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.2107312194357791, 0.16540577903174386, 0.2560328602083732], "delta_aurc_vs_best_energy": [0.1880248836583393, 0.14315050073946656, 0.23734906074468637], "delta_aurc_vs_best_baseline": [-0.030080051188408008, -0.04648240416942707, -0.014728028214055597], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.027777777777777776, 0.020833333333333332, 0.016666666666666666, 0.027777777777777887, 0.03571428571428572, 0.041666666666666664, 0.046296296296296294, 0.05000000000000001, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.09444444444444461, 0.10416666666666667, 0.10784313725490197, 0.12499999999999999, 0.13596491228070176, 0.15, 0.15079365079365076, 0.16287878787878787, 0.17028985507246377, 0.18402777777777776, 0.18666666666666668, 0.1891025641025641, 0.1882716049382716, 0.19345238095238096, 0.2011494252873563, 0.20833333333333334, 0.20967741935483872, 0.20833333333333334, 0.21212121212121213, 0.22549019607843138, 0.2333333333333333, 0.23842592592592604, 0.23648648648648649, 0.24342105263157895, 0.25, 0.25833333333333336, 0.27032520325203263, 0.2797619047619047, 0.28875968992248063, 0.29734848484848486, 0.3, 0.3061594202898551, 0.3085106382978723, 0.31597222222222227, 0.32482993197278914, 0.33166666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.2787878787878788, 0.278787878787879, 0.27878787878787903, 0.2787878787878791, 0.2787878787878791, 0.2787878787878788, 0.27878787878787864, 0.2787878787878785, 0.27878787878787836, 0.2787878787878784, 0.27878787878787864, 0.27878787878787886, 0.27878787878787903, 0.27878787878787914, 0.2787878787878793, 0.2787878787878794, 0.2787878787878795, 0.2787878787878796, 0.27878787878787964, 0.27878787878787975, 0.2787878787878798, 0.27878787878787986, 0.2787878787878799, 0.2787878787878799, 0.27878787878788, 0.27878787878788003, 0.2787878787878801, 0.2787878787878801, 0.27878787878788014, 0.27878787878788014, 0.2787878787878802, 0.2787878787878802, 0.27878787878788025, 0.27878787878788025, 0.2787878787878803, 0.2787878787878803, 0.2787878787878803, 0.27878787878788036, 0.27878787878788036, 0.27878787878788036, 0.2787878787878804, 0.283928571428573, 0.29050387596899363, 0.29793123543123673, 0.30722222222222356, 0.3103260869565228, 0.31414267834793586, 0.31864316239316337, 0.3231292517006814, 0.33166666666666755]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.2916666666666667, 0.3611111111111111, 0.375, 0.35, 0.38888888888888884, 0.40476190476190466, 0.4166666666666667, 0.4351851851851852, 0.42499999999999993, 0.4318181818181818, 0.4166666666666667, 0.40384615384615385, 0.3869047619047619, 0.388888888888889, 0.390625, 0.4019607843137255, 0.40277777777777773, 0.39473684210526316, 0.3958333333333333, 0.3928571428571429, 0.4015151515151515, 0.39492753623188404, 0.39236111111111105, 0.3933333333333333, 0.3942307692307692, 0.3950617283950617, 0.3869047619047619, 0.37931034482758613, 0.37777777777777777, 0.3736559139784946, 0.3645833333333333, 0.3661616161616162, 0.3627450980392157, 0.3666666666666666, 0.3657407407407407, 0.36261261261261263, 0.36622807017543857, 0.36538461538461536, 0.36875, 0.36382113821138207, 0.3591269841269841, 0.3507751937984496, 0.3503787878787879, 0.35185185185185186, 0.3496376811594203, 0.3439716312056737, 0.34027777777777773, 0.33503401360544216, 0.33166666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.3333333333333333, 0.4166666666666667, 0.3888888888888889, 0.4791666666666667, 0.45, 0.45833333333333326, 0.4166666666666668, 0.3958333333333333, 0.42592592592592593, 0.45833333333333326, 0.4621212121212121, 0.4513888888888889, 0.4423076923076923, 0.44047619047619047, 0.4333333333333333, 0.4270833333333333, 0.4117647058823529, 0.40740740740740733, 0.39473684210526316, 0.38333333333333336, 0.3928571428571428, 0.39015151515151514, 0.38768115942028986, 0.38541666666666663, 0.3933333333333333, 0.40384615384615385, 0.4074074074074074, 0.40773809523809523, 0.410919540229885, 0.40555555555555556, 0.4032258064516129, 0.3932291666666667, 0.39141414141414144, 0.38480392156862747, 0.3761904761904763, 0.3726851851851851, 0.3738738738738739, 0.3684210526315789, 0.36324786324786323, 0.36041666666666666, 0.3577235772357723, 0.3591269841269841, 0.35658914728682173, 0.3522727272727273, 0.35, 0.34601449275362317, 0.34219858156028365, 0.33854166666666674, 0.33503401360544216, 0.33166666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.4583333333333333, 0.5, 0.4583333333333333, 0.45, 0.41666666666666663, 0.4166666666666668, 0.3958333333333333, 0.3888888888888889, 0.39166666666666655, 0.36363636363636365, 0.3611111111111111, 0.36538461538461536, 0.35119047619047616, 0.3499999999999999, 0.3645833333333333, 0.3627450980392157, 0.3611111111111112, 0.3684210526315789, 0.3875, 0.3690476190476191, 0.35984848484848486, 0.35144927536231885, 0.34027777777777773, 0.3333333333333333, 0.3333333333333333, 0.3395061728395062, 0.33630952380952384, 0.3304597701149425, 0.325, 0.3279569892473118, 0.3255208333333333, 0.3282828282828283, 0.31862745098039214, 0.3166666666666666, 0.31481481481481494, 0.31981981981981983, 0.31798245614035087, 0.3162393162393162, 0.3145833333333333, 0.31504065040650403, 0.31349206349206343, 0.313953488372093, 0.3162878787878788, 0.32037037037037036, 0.3278985507246377, 0.3315602836879432, 0.32986111111111105, 0.3299319727891156, 0.33166666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.020833333333333332, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.05555555555555555, 0.0641025641025641, 0.05952380952380952, 0.06666666666666665, 0.06770833333333333, 0.06862745098039216, 0.06944444444444443, 0.07017543859649122, 0.07916666666666666, 0.08333333333333331, 0.10227272727272728, 0.10869565217391304, 0.11805555555555554, 0.13333333333333333, 0.14423076923076922, 0.16358024691358025, 0.16666666666666666, 0.18390804597701146, 0.18888888888888888, 0.1989247311827957, 0.21354166666666666, 0.21717171717171718, 0.22794117647058823, 0.23571428571428568, 0.2407407407407407, 0.25, 0.2565789473684211, 0.2606837606837607, 0.26666666666666666, 0.26829268292682923, 0.26984126984126994, 0.28294573643410853, 0.29545454545454547, 0.3, 0.3079710144927536, 0.3156028368794327, 0.32291666666666674, 0.32482993197278914, 0.33166666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.03125, 0.027777777777777776, 0.04166666666666667, 0.045454545454545456, 0.041666666666666664, 0.038461538461538464, 0.047619047619047616, 0.049999999999999996, 0.052083333333333336, 0.06372549019607843, 0.06481481481481495, 0.06578947368421052, 0.06666666666666667, 0.07936507936507935, 0.09090909090909091, 0.10144927536231885, 0.11111111111111109, 0.12, 0.13141025641025642, 0.1388888888888889, 0.1488095238095238, 0.16379310344827602, 0.16944444444444445, 0.18010752688172044, 0.19010416666666666, 0.19696969696969696, 0.20588235294117646, 0.211904761904762, 0.21990740740740752, 0.22972972972972974, 0.23684210526315788, 0.24145299145299146, 0.24583333333333332, 0.2520325203252032, 0.263888888888889, 0.27325581395348836, 0.2821969696969697, 0.28888888888888886, 0.3007246376811594, 0.3085106382978724, 0.31944444444444436, 0.32482993197278914, 0.33166666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.011904761904761705, 0.020833333333333332, 0.046296296296296294, 0.04166666666666667, 0.045454545454545456, 0.04861111111111111, 0.057692307692307696, 0.05952380952380952, 0.06666666666666665, 0.06770833333333333, 0.06862745098039216, 0.0648148148148148, 0.07017543859649122, 0.07916666666666666, 0.08333333333333345, 0.0946969696969697, 0.10869565217391304, 0.11805555555555554, 0.13333333333333333, 0.14743589743589744, 0.15432098765432098, 0.1636904761904762, 0.17528735632183906, 0.18611111111111112, 0.19623655913978494, 0.203125, 0.21464646464646464, 0.22058823529411764, 0.22380952380952393, 0.23842592592592604, 0.24774774774774774, 0.25, 0.2606837606837607, 0.25833333333333336, 0.26422764227642287, 0.2738095238095238, 0.27906976744186046, 0.2803030303030303, 0.28888888888888886, 0.3007246376811594, 0.30673758865248235, 0.31597222222222227, 0.3231292517006803, 0.33166666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.16541353383458646, 0.2751004016064257], "coverage": [0.0, 0.0, 0.0, 0.0, 0.44333333333333336, 0.83]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5988734194664094, "basin/entropy": 0.4017092946026893, "basin/dispersion": 0.5424253436759859, "energy/mean": 0.6074136267371771, "energy/min": 0.5868243962956929, "energy/std": 0.5071742753668592, "curv/lmax_mean": 0.3702302033860073, "curv/lmax_best": 0.38854496923520343, "curv/trace_mean": 0.3700798255617238, "curv/trace_best": 0.38669657514505196, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6523452674845549, "dynamics/drop": 0.7109612902417324, "dynamics/residual": 0.5072619957643579}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [0, 0, 3, 0, 0, 6, 0, 0, 8, 0, 0, 11, 0, 0, 3, 0, 0, 13, 0, 357], "incorrect_counts": [2, 0, 7, 0, 0, 7, 0, 0, 9, 0, 0, 9, 0, 0, 10, 0, 0, 17, 0, 138]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [357, 0, 0, 0, 0, 13, 0, 0, 3, 0, 10, 7, 8, 1, 0, 1, 0, 0, 1, 0], "incorrect_counts": [138, 0, 0, 0, 0, 17, 0, 0, 10, 0, 9, 9, 13, 0, 0, 0, 0, 1, 0, 2]}, "basin/dispersion": {"edges": [1.6439221991114048, 1.6628586470572428, 1.6817950950030807, 1.7007315429489187, 1.7196679908947567, 1.7386044388405946, 1.7575408867864326, 1.7764773347322707, 1.7954137826781085, 1.8143502306239465, 1.8332866785697846, 1.8522231265156224, 1.8711595744614604, 1.8900960224072985, 1.9090324703531363, 1.9279689182989743, 1.9469053662448124, 1.9658418141906502, 1.9847782621364882, 2.0037147100823263, 2.022651158028164], "correct_counts": [1, 2, 3, 13, 14, 13, 26, 31, 36, 41, 34, 51, 43, 31, 21, 18, 14, 5, 1, 3], "incorrect_counts": [1, 3, 2, 10, 7, 6, 16, 15, 19, 18, 20, 23, 22, 17, 6, 5, 4, 3, 1, 1]}, "energy/mean": {"edges": [0.41955216725667316, 0.4642234648267428, 0.5088947623968124, 0.5535660599668821, 0.5982373575369517, 0.6429086551070213, 0.6875799526770909, 0.7322512502471605, 0.7769225478172302, 0.8215938453873, 0.8662651429573696, 0.9109364405274392, 0.9556077380975088, 1.0002790356675784, 1.044950333237648, 1.0896216308077178, 1.1342929283777874, 1.178964225947857, 1.2236355235179266, 1.2683068210879962, 1.3129781186580658], "correct_counts": [5, 2, 3, 7, 13, 19, 17, 40, 47, 35, 45, 54, 40, 26, 26, 12, 4, 4, 1, 1], "incorrect_counts": [2, 1, 5, 5, 13, 9, 21, 25, 18, 28, 25, 16, 13, 10, 4, 2, 2, 0, 0, 0]}, "energy/min": {"edges": [0.02020469307899475, 0.07093883007764816, 0.12167296707630157, 0.17240710407495496, 0.2231412410736084, 0.2738753780722618, 0.3246095150709152, 0.3753436520695686, 0.426077789068222, 0.47681192606687545, 0.5275460630655289, 0.5782802000641822, 0.6290143370628356, 0.6797484740614891, 0.7304826110601425, 0.7812167480587959, 0.8319508850574493, 0.8826850220561027, 0.9334191590547561, 0.9841532960534095, 1.034887433052063], "correct_counts": [0, 2, 5, 10, 7, 16, 17, 35, 39, 31, 47, 49, 45, 41, 22, 16, 8, 7, 3, 1], "incorrect_counts": [1, 1, 2, 2, 9, 6, 22, 19, 25, 21, 22, 17, 30, 9, 8, 3, 0, 2, 0, 0]}, "energy/std": {"edges": [0.08805419068698145, 0.10022669760053629, 0.11239920451409115, 0.12457171142764599, 0.13674421834120085, 0.1489167252547557, 0.16108923216831053, 0.17326173908186537, 0.18543424599542024, 0.19760675290897506, 0.20977925982252993, 0.22195176673608477, 0.2341242736496396, 0.24629678056319446, 0.25846928747674935, 0.27064179439030417, 0.282814301303859, 0.29498680821741385, 0.3071593151309687, 0.3193318220445236, 0.3315043289580784], "correct_counts": [2, 1, 6, 14, 23, 31, 33, 30, 51, 42, 52, 35, 28, 18, 11, 10, 5, 3, 4, 2], "incorrect_counts": [0, 4, 7, 7, 14, 15, 15, 21, 15, 17, 18, 15, 10, 14, 14, 5, 2, 3, 3, 0]}, "curv/lmax_mean": {"edges": [0.9991512497266134, 0.9997431829571725, 1.0003351161877314, 1.0009270494182905, 1.0015189826488495, 1.0021109158794086, 1.0027028491099677, 1.0032947823405267, 1.0038867155710856, 1.0044786488016446, 1.0050705820322037, 1.0056625152627627, 1.0062544484933218, 1.0068463817238809, 1.00743831495444, 1.008030248184999, 1.008622181415558, 1.0092141146461169, 1.009806047876676, 1.010397981107235, 1.010989914337794], "correct_counts": [76, 176, 62, 38, 24, 9, 7, 4, 1, 0, 1, 2, 0, 0, 0, 0, 0, 1, 0, 0], "incorrect_counts": [22, 68, 31, 26, 21, 10, 5, 1, 0, 6, 2, 3, 1, 1, 0, 0, 0, 0, 1, 1]}, "curv/lmax_best": {"edges": [0.994626522064209, 0.996795529127121, 0.998964536190033, 1.001133543252945, 1.003302550315857, 1.005471557378769, 1.0076405644416808, 1.009809571504593, 1.0119785785675048, 1.014147585630417, 1.0163165926933289, 1.0184855997562408, 1.0206546068191529, 1.0228236138820648, 1.024992620944977, 1.0271616280078888, 1.0293306350708007, 1.0314996421337128, 1.0336686491966247, 1.0358376562595368, 1.0380066633224487], "correct_counts": [1, 11, 344, 27, 7, 5, 4, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 5, 149, 23, 9, 2, 2, 3, 3, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/trace_mean": {"edges": [31.952821254730225, 31.960075608889262, 31.9673299630483, 31.974584317207338, 31.981838671366376, 31.989093025525413, 31.996347379684448, 32.00360173384349, 32.01085608800253, 32.018110442161564, 32.0253647963206, 32.03261915047963, 32.03987350463867, 32.04712785879771, 32.054382212956746, 32.061636567115784, 32.06889092127482, 32.07614527543386, 32.0833996295929, 32.090653983751935, 32.09790833791097], "correct_counts": [3, 2, 12, 26, 51, 59, 58, 53, 50, 35, 21, 17, 8, 2, 3, 0, 0, 1, 0, 0], "incorrect_counts": [0, 1, 4, 5, 17, 16, 28, 26, 22, 27, 19, 11, 12, 3, 5, 0, 2, 0, 0, 1]}, "curv/trace_best": {"edges": [31.93198013305664, 31.942170333862304, 31.952360534667967, 31.962550735473634, 31.972740936279298, 31.98293113708496, 31.993121337890624, 32.00331153869629, 32.013501739501955, 32.023691940307614, 32.03388214111328, 32.04407234191895, 32.05426254272461, 32.064452743530275, 32.074642944335935, 32.0848331451416, 32.09502334594727, 32.10521354675293, 32.115403747558595, 32.125593948364255, 32.13578414916992], "correct_counts": [2, 2, 4, 11, 33, 70, 97, 88, 39, 24, 19, 4, 3, 3, 1, 1, 0, 0, 0, 0], "incorrect_counts": [0, 1, 1, 3, 9, 24, 41, 39, 29, 21, 10, 8, 3, 3, 2, 1, 2, 0, 0, 2]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [401, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [199, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6266666666666666, 0.6331666666666667, 0.6396666666666666, 0.6461666666666666, 0.6526666666666666, 0.6591666666666667, 0.6656666666666666, 0.6721666666666666, 0.6786666666666666, 0.6851666666666667, 0.6916666666666667, 0.6981666666666666, 0.7046666666666667, 0.7111666666666667, 0.7176666666666667, 0.7241666666666666, 0.7306666666666667, 0.7371666666666667, 0.7436666666666667, 0.7501666666666666, 0.7566666666666667], "correct_counts": [0, 0, 0, 0, 5, 7, 13, 22, 27, 49, 58, 43, 51, 41, 32, 21, 16, 9, 3, 4], "incorrect_counts": [1, 0, 0, 0, 2, 6, 17, 28, 18, 27, 34, 20, 16, 18, 5, 1, 4, 2, 0, 0]}, "dynamics/drop": {"edges": [14.020859281222025, 18.926390168940028, 23.83192105665803, 28.737451944376033, 33.642982832094035, 38.548513719812036, 43.45404460753004, 48.35957549524804, 53.26510638296604, 58.17063727068404, 63.07616815840204, 67.98169904612004, 72.88722993383806, 77.79276082155606, 82.69829170927406, 87.60382259699206, 92.50935348471006, 97.41488437242806, 102.32041526014606, 107.22594614786406, 112.13147703558207], "correct_counts": [19, 90, 74, 61, 44, 39, 28, 17, 11, 9, 3, 3, 2, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [35, 66, 50, 28, 11, 4, 3, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1569731881221135, 1.1719582500557104, 1.1869433119893074, 1.2019283739229043, 1.2169134358565012, 1.2318984977900982, 1.2468835597236951, 1.261868621657292, 1.276853683590889, 1.291838745524486, 1.3068238074580827, 1.3218088693916799, 1.3367939313252766, 1.3517789932588735, 1.3667640551924705, 1.3817491171260674, 1.3967341790596643, 1.4117192409932613, 1.4267043029268582, 1.4416893648604552, 1.4566744267940521], "correct_counts": [0, 7, 3, 13, 26, 24, 43, 49, 49, 46, 46, 34, 24, 19, 10, 4, 2, 1, 0, 1], "incorrect_counts": [5, 2, 5, 2, 16, 11, 18, 23, 22, 25, 21, 20, 12, 9, 3, 4, 0, 1, 0, 0]}}}, "feature_ablation": {"full": 0.16709451083285565, "drop_basin": 0.1764619824600932, "basin_only": 0.2526067320324128, "drop_energy": 0.1691897321697063, "energy_only": 0.27713326412513645, "drop_curv": 0.1719733602062839, "curv_only": 0.27348129472080446, "drop_dynamics": 0.22842807279427982, "dynamics_only": 0.18359324587631137}, "ece_geometry": 0.03781877930416236}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "70d97e02bed0", "timestamp": "2026-07-27T10:02:27.587625+00:00", "git_sha": "5207145", "config_hash": "fbed14eeb756", "config": {"run": {"seed": 3, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 3, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7133333333333334, "base_error": 0.2866666666666667, "final_train_loss": 0.16939383745193481, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.17019370187580518, "rho_basin": 0.23474252197420142, "energy_min": 0.3728384148545108, "energy_mean": 0.38232702405148095, "energy_std": 0.3016550018932722, "msp": 0.12508580744878345, "temp_msp": 0.1281729327488173, "entropy": 0.12478032569162513}, "temperature": 3.40053609466433, "best_energy_baseline": "energy_std", "best_baseline": "entropy", "delta_aurc_vs_energy_min": [0.20264471297870562, 0.14299561878502962, 0.2569749266815167], "delta_aurc_vs_best_energy": [0.13146130001746703, 0.08951962500452121, 0.17179871890451937], "delta_aurc_vs_best_baseline": [-0.04541337618418005, -0.06759052108205994, -0.02516698291343395], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.041666666666666664, 0.05555555555555555, 0.041666666666666664, 0.08333333333333333, 0.09722222222222221, 0.09523809523809505, 0.125, 0.1388888888888889, 0.13333333333333336, 0.13636363636363635, 0.13194444444444445, 0.12179487179487179, 0.11904761904761904, 0.12222222222222219, 0.13020833333333334, 0.13725490196078433, 0.12962962962962976, 0.14035087719298245, 0.14583333333333334, 0.1706349206349206, 0.17424242424242425, 0.17391304347826086, 0.18055555555555552, 0.18, 0.18269230769230768, 0.18209876543209877, 0.18154761904761904, 0.18678160919540227, 0.18055555555555555, 0.18548387096774194, 0.19270833333333334, 0.19696969696969696, 0.2034313725490196, 0.20476190476190473, 0.20833333333333331, 0.21396396396396397, 0.22807017543859648, 0.2329059829059829, 0.2375, 0.23780487804878045, 0.23611111111111108, 0.2441860465116279, 0.25, 0.2574074074074074, 0.26268115942028986, 0.27304964539007087, 0.2760416666666668, 0.28401360544217685, 0.2866666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.22920892494929004, 0.2292089249492901, 0.22920892494929015, 0.2292089249492902, 0.22920892494929024, 0.22920892494929024, 0.22920892494929027, 0.22920892494929027, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.2292089249492903, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.22920892494929032, 0.23576097105508903, 0.2425900592795261, 0.2492715617715621, 0.2574074074074077, 0.26518952062430345, 0.27160904255319174, 0.27650462962962985, 0.28099017384731684, 0.28666666666666685]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.5, 0.5, 0.5277777777777778, 0.5208333333333334, 0.4666666666666667, 0.4305555555555555, 0.4404761904761906, 0.40625, 0.4166666666666667, 0.4083333333333334, 0.4090909090909091, 0.4097222222222222, 0.40384615384615385, 0.39880952380952384, 0.39444444444444454, 0.3958333333333333, 0.39705882352941174, 0.3935185185185185, 0.38596491228070173, 0.375, 0.37698412698412703, 0.3787878787878788, 0.36594202898550726, 0.3680555555555555, 0.36, 0.36217948717948717, 0.3611111111111111, 0.3541666666666667, 0.3448275862068967, 0.34444444444444444, 0.34139784946236557, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.3261904761904763, 0.3240740740740742, 0.32207207207207206, 0.3157894736842105, 0.30982905982905984, 0.30625, 0.3028455284552845, 0.3015873015873017, 0.29651162790697677, 0.29924242424242425, 0.29444444444444445, 0.2916666666666667, 0.2943262411347518, 0.29166666666666663, 0.2891156462585034, 0.2866666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.6666666666666666, 0.5416666666666666, 0.5277777777777778, 0.5833333333333334, 0.5166666666666667, 0.48611111111111105, 0.4523809523809525, 0.4375, 0.39814814814814814, 0.3833333333333334, 0.4015151515151515, 0.4027777777777778, 0.41025641025641024, 0.4226190476190476, 0.4166666666666668, 0.4166666666666667, 0.4019607843137255, 0.4027777777777779, 0.39035087719298245, 0.38333333333333336, 0.3690476190476191, 0.35984848484848486, 0.36231884057971014, 0.36111111111111116, 0.36, 0.358974358974359, 0.3549382716049383, 0.34523809523809523, 0.3448275862068967, 0.3416666666666667, 0.3387096774193548, 0.3359375, 0.3333333333333333, 0.3284313725490196, 0.32380952380952377, 0.324074074074074, 0.31981981981981983, 0.3157894736842105, 0.31196581196581197, 0.3104166666666667, 0.30894308943089427, 0.3055555555555555, 0.3023255813953488, 0.29734848484848486, 0.2962962962962963, 0.29347826086956524, 0.29255319148936165, 0.2899305555555555, 0.29081632653061223, 0.2866666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.2916666666666667, 0.3055555555555556, 0.3541666666666667, 0.36666666666666664, 0.33333333333333326, 0.29761904761904767, 0.3125, 0.3055555555555556, 0.29166666666666674, 0.2878787878787879, 0.2708333333333333, 0.2948717948717949, 0.2976190476190476, 0.2944444444444444, 0.2864583333333333, 0.29411764705882354, 0.3009259259259259, 0.29385964912280704, 0.3, 0.2936507936507936, 0.29924242424242425, 0.2898550724637681, 0.29513888888888895, 0.29333333333333333, 0.28846153846153844, 0.28703703703703703, 0.27976190476190477, 0.278735632183908, 0.2833333333333333, 0.28225806451612906, 0.2838541666666667, 0.28535353535353536, 0.2818627450980392, 0.28333333333333344, 0.287037037037037, 0.28603603603603606, 0.29385964912280704, 0.2948717948717949, 0.29791666666666666, 0.2947154471544715, 0.2916666666666668, 0.29069767441860467, 0.2897727272727273, 0.28703703703703703, 0.2916666666666667, 0.2907801418439717, 0.29340277777777773, 0.2925170068027211, 0.2866666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.057692307692307696, 0.05952380952380952, 0.061111111111111095, 0.057291666666666664, 0.06372549019607843, 0.07407407407407406, 0.07894736842105263, 0.08333333333333333, 0.08730158730158742, 0.10606060606060606, 0.11594202898550725, 0.11111111111111109, 0.12, 0.12179487179487179, 0.13271604938271606, 0.13988095238095238, 0.14942528735632182, 0.16111111111111112, 0.1639784946236559, 0.17447916666666666, 0.17676767676767677, 0.17647058823529413, 0.1833333333333333, 0.1898148148148148, 0.1891891891891892, 0.19736842105263158, 0.20726495726495728, 0.2125, 0.21951219512195133, 0.22817460317460328, 0.23449612403100775, 0.24431818181818182, 0.25, 0.2608695652173913, 0.2677304964539007, 0.27256944444444436, 0.282312925170068, 0.2866666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.04166666666666667, 0.045454545454545456, 0.05555555555555555, 0.057692307692307696, 0.05357142857142857, 0.055555555555555726, 0.078125, 0.0784313725490196, 0.07870370370370385, 0.09210526315789473, 0.09583333333333334, 0.10317460317460315, 0.10984848484848485, 0.11956521739130435, 0.12499999999999999, 0.13666666666666666, 0.14102564102564102, 0.1388888888888889, 0.13392857142857142, 0.13505747126436798, 0.14444444444444443, 0.1586021505376344, 0.15885416666666666, 0.16666666666666666, 0.17892156862745098, 0.18095238095238092, 0.1851851851851853, 0.1981981981981982, 0.21052631578947367, 0.22008547008547008, 0.22291666666666668, 0.23170731707317085, 0.23412698412698424, 0.23837209302325582, 0.24242424242424243, 0.2518518518518518, 0.2554347826086957, 0.26241134751773043, 0.2690972222222223, 0.282312925170068, 0.2866666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.0, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.03333333333333334, 0.045454545454545456, 0.04861111111111111, 0.05128205128205128, 0.05952380952380952, 0.061111111111111095, 0.057291666666666664, 0.06372549019607843, 0.07407407407407406, 0.07894736842105263, 0.0875, 0.09126984126984125, 0.10984848484848485, 0.11594202898550725, 0.11111111111111109, 0.11666666666666667, 0.12179487179487179, 0.13580246913580246, 0.14285714285714285, 0.14942528735632202, 0.15555555555555556, 0.1639784946236559, 0.171875, 0.17424242424242425, 0.17401960784313725, 0.1833333333333333, 0.18750000000000014, 0.19144144144144143, 0.19956140350877194, 0.2094017094017094, 0.21041666666666667, 0.2195121951219512, 0.2261904761904763, 0.24031007751937986, 0.24242424242424243, 0.25, 0.25181159420289856, 0.25886524822695045, 0.27256944444444436, 0.28061224489795916, 0.2866666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.09090909090909091, 0.2038369304556355], "coverage": [0.0, 0.0, 0.0, 0.0, 0.12833333333333333, 0.695]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.6152942295153228, "basin/entropy": 0.38495707454901107, "basin/dispersion": 0.5045913931753967, "energy/mean": 0.6219707672245164, "energy/min": 0.6181264942403826, "energy/std": 0.5105955227124538, "curv/lmax_mean": 0.3801755053249294, "curv/lmax_best": 0.38900510758530754, "curv/trace_mean": 0.37158362312540755, "curv/trace_best": 0.3750747120191263, "dynamics/steps": 0.5, "dynamics/monotonic": 0.5918346555096718, "dynamics/drop": 0.6229623994783743, "dynamics/residual": 0.4728048250380352}, "hist": {"basin/rho": {"edges": [0.5, 0.525, 0.55, 0.575, 0.6, 0.625, 0.65, 0.675, 0.7, 0.725, 0.75, 0.775, 0.8, 0.825, 0.8500000000000001, 0.875, 0.9, 0.925, 0.95, 0.9750000000000001, 1.0], "correct_counts": [3, 0, 0, 4, 0, 0, 8, 0, 0, 0, 7, 0, 0, 10, 0, 0, 16, 0, 0, 380], "incorrect_counts": [4, 0, 0, 5, 0, 0, 7, 0, 0, 0, 9, 0, 0, 16, 0, 0, 18, 0, 0, 113]}, "basin/entropy": {"edges": [0.0, 0.05057021323536759, 0.10114042647073518, 0.15171063970610277, 0.20228085294147036, 0.25285106617683795, 0.30342127941220554, 0.3539914926475731, 0.4045617058829407, 0.4551319191183083, 0.5057021323536759, 0.5562723455890435, 0.6068425588244111, 0.6574127720597787, 0.7079829852951462, 0.7585531985305138, 0.8091234117658814, 0.859693625001249, 0.9102638382366166, 0.9608340514719842, 1.0114042647073518], "correct_counts": [380, 0, 0, 0, 0, 16, 0, 0, 10, 0, 0, 5, 7, 7, 2, 0, 1, 0, 0, 0], "incorrect_counts": [113, 0, 0, 0, 0, 18, 0, 0, 16, 0, 0, 9, 5, 7, 0, 0, 2, 0, 1, 1]}, "basin/dispersion": {"edges": [1.638197206426302, 1.6606953621248024, 1.6831935178233026, 1.705691673521803, 1.7281898292203033, 1.7506879849188037, 1.773186140617304, 1.7956842963158044, 1.8181824520143046, 1.840680607712805, 1.8631787634113053, 1.8856769191098057, 1.9081750748083062, 1.9306732305068064, 1.9531713862053068, 1.975669541903807, 1.9981676976023075, 2.020665853300808, 2.043164008999308, 2.0656621646978084, 2.088160320396309], "correct_counts": [2, 7, 9, 12, 27, 47, 52, 47, 61, 52, 32, 23, 29, 9, 6, 8, 4, 0, 0, 1], "incorrect_counts": [3, 0, 6, 12, 8, 15, 19, 17, 23, 20, 18, 13, 10, 0, 5, 2, 1, 0, 0, 0]}, "energy/mean": {"edges": [0.4687643547852834, 0.5167818620800972, 0.5647993693749109, 0.6128168766697248, 0.6608343839645385, 0.7088518912593524, 0.7568693985541661, 0.80488690584898, 0.8529044131437937, 0.9009219204386076, 0.9489394277334213, 0.9969569350282352, 1.044974442323049, 1.0929919496178628, 1.1410094569126765, 1.1890269642074904, 1.2370444715023041, 1.285061978797118, 1.3330794860919317, 1.3810969933867456, 1.4291145006815593], "correct_counts": [0, 0, 2, 9, 12, 33, 30, 23, 44, 31, 37, 43, 34, 39, 35, 24, 18, 5, 6, 3], "incorrect_counts": [1, 1, 3, 9, 15, 13, 15, 21, 14, 11, 13, 16, 10, 8, 8, 6, 5, 3, 0, 0]}, "energy/min": {"edges": [0.13711321353912354, 0.18594235181808472, 0.2347714900970459, 0.2836006283760071, 0.33242976665496826, 0.38125890493392944, 0.4300880432128906, 0.4789171814918518, 0.527746319770813, 0.5765754580497742, 0.6254045963287354, 0.6742337346076965, 0.7230628728866577, 0.7718920111656189, 0.8207211494445801, 0.8695502877235413, 0.9183794260025024, 0.9672085642814636, 1.0160377025604248, 1.064866840839386, 1.1136959791183472], "correct_counts": [0, 1, 4, 5, 10, 21, 25, 40, 32, 31, 30, 47, 36, 46, 32, 24, 13, 17, 11, 3], "incorrect_counts": [2, 3, 1, 1, 15, 9, 16, 24, 16, 13, 13, 14, 10, 9, 7, 6, 7, 3, 2, 1]}, "energy/std": {"edges": [0.07382017921196296, 0.09043346825542178, 0.10704675729888061, 0.12366004634233943, 0.14027333538579825, 0.15688662442925708, 0.1734999134727159, 0.19011320251617472, 0.20672649155963355, 0.22333978060309237, 0.2399530696465512, 0.25656635869001, 0.2731796477334688, 0.28979293677692763, 0.30640622582038646, 0.3230195148638453, 0.3396328039073041, 0.3562460929507629, 0.37285938199422175, 0.3894726710376806, 0.40608596008113945], "correct_counts": [0, 6, 13, 19, 47, 60, 63, 62, 47, 48, 25, 11, 6, 11, 6, 1, 2, 0, 0, 1], "incorrect_counts": [1, 4, 3, 14, 14, 25, 26, 20, 19, 24, 7, 10, 2, 3, 0, 0, 0, 0, 0, 0]}, "curv/lmax_mean": {"edges": [0.9988449911276499, 0.9992352331678073, 0.9996254752079645, 1.0000157172481219, 1.0004059592882792, 1.0007962013284366, 1.0011864433685937, 1.001576685408751, 1.0019669274489085, 1.0023571694890658, 1.0027474115292232, 1.0031376535693806, 1.0035278956095377, 1.003918137649695, 1.0043083796898524, 1.0046986217300098, 1.0050888637701672, 1.0054791058103245, 1.0058693478504817, 1.006259589890639, 1.0066498319307964], "correct_counts": [12, 134, 143, 44, 37, 20, 17, 5, 5, 3, 5, 2, 0, 0, 0, 1, 0, 0, 0, 0], "incorrect_counts": [2, 26, 55, 28, 25, 17, 7, 3, 1, 4, 0, 2, 1, 0, 0, 0, 0, 0, 0, 1]}, "curv/lmax_best": {"edges": [0.995161235332489, 0.9962190896272659, 0.9972769439220428, 0.9983347982168198, 0.9993926525115967, 1.0004505068063736, 1.0015083611011506, 1.0025662153959274, 1.0036240696907044, 1.0046819239854812, 1.0057397782802582, 1.0067976325750352, 1.007855486869812, 1.008913341164589, 1.0099711954593658, 1.0110290497541428, 1.0120869040489198, 1.0131447583436965, 1.0142026126384736, 1.0152604669332503, 1.0163183212280273], "correct_counts": [1, 3, 10, 64, 293, 33, 14, 3, 1, 5, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [0, 0, 6, 11, 120, 17, 7, 3, 3, 0, 2, 0, 0, 0, 0, 1, 0, 1, 0, 1]}, "curv/trace_mean": {"edges": [31.904047807057697, 31.911127193768817, 31.91820658047994, 31.92528596719106, 31.93236535390218, 31.9394447406133, 31.946524127324423, 31.953603514035542, 31.960682900746665, 31.967762287457784, 31.974841674168907, 31.981921060880026, 31.989000447591145, 31.996079834302268, 32.00315922101339, 32.01023860772451, 32.01731799443563, 32.02439738114675, 32.031476767857875, 32.038556154568994, 32.04563554128011], "correct_counts": [3, 2, 4, 13, 19, 14, 19, 32, 51, 37, 20, 31, 34, 31, 28, 36, 27, 14, 8, 5], "incorrect_counts": [0, 0, 0, 1, 3, 3, 10, 9, 5, 10, 13, 15, 10, 11, 19, 25, 18, 10, 8, 2]}, "curv/trace_best": {"edges": [31.868419647216797, 31.879567527770995, 31.890715408325196, 31.901863288879394, 31.913011169433595, 31.924159049987793, 31.93530693054199, 31.946454811096192, 31.95760269165039, 31.96875057220459, 31.97989845275879, 31.991046333312987, 32.002194213867185, 32.01334209442139, 32.02448997497559, 32.035637855529785, 32.04678573608398, 32.05793361663818, 32.069081497192386, 32.08022937774658, 32.09137725830078], "correct_counts": [1, 2, 3, 6, 11, 13, 24, 26, 46, 55, 60, 60, 53, 37, 14, 11, 3, 1, 2, 0], "incorrect_counts": [0, 0, 1, 1, 5, 2, 2, 8, 9, 18, 26, 20, 29, 26, 15, 6, 1, 2, 0, 1]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [428, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [172, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6449999999999999, 0.6505833333333333, 0.6561666666666666, 0.66175, 0.6673333333333332, 0.6729166666666666, 0.6785, 0.6840833333333333, 0.6896666666666667, 0.6952499999999999, 0.7008333333333333, 0.7064166666666667, 0.712, 0.7175833333333334, 0.7231666666666666, 0.72875, 0.7343333333333334, 0.7399166666666667, 0.7455, 0.7510833333333333, 0.7566666666666667], "correct_counts": [3, 3, 9, 15, 16, 32, 38, 31, 53, 40, 32, 54, 32, 20, 21, 10, 7, 10, 0, 2], "incorrect_counts": [1, 2, 5, 6, 12, 19, 17, 15, 31, 17, 10, 13, 9, 4, 5, 3, 2, 0, 1, 0]}, "dynamics/drop": {"edges": [14.044291233023008, 17.461781060944002, 20.879270888864994, 24.29676071678599, 27.71425054470698, 31.131740372627974, 34.54923020054897, 37.966720028469965, 41.38420985639095, 44.80169968431195, 48.219189512232944, 51.63667934015394, 55.05416916807493, 58.471658995995924, 61.88914882391692, 65.3066386518379, 68.7241284797589, 72.14161830767989, 75.55910813560088, 78.97659796352188, 82.39408779144287], "correct_counts": [20, 45, 59, 60, 64, 49, 39, 24, 17, 14, 13, 8, 3, 5, 4, 1, 0, 1, 0, 2], "incorrect_counts": [10, 26, 43, 29, 18, 19, 10, 11, 3, 2, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.157560517390569, 1.171275670826435, 1.184990824262301, 1.1987059776981672, 1.2124211311340332, 1.226136284569899, 1.2398514380057652, 1.2535665914416314, 1.2672817448774973, 1.2809968983133633, 1.2947120517492294, 1.3084272051850956, 1.3221423586209615, 1.3358575120568275, 1.3495726654926936, 1.3632878189285598, 1.3770029723644257, 1.3907181258002916, 1.4044332792361578, 1.418148432672024, 1.4318635861078899], "correct_counts": [2, 1, 7, 4, 14, 21, 36, 48, 45, 57, 40, 31, 33, 24, 24, 20, 10, 5, 3, 3], "incorrect_counts": [0, 2, 2, 5, 1, 6, 8, 20, 15, 26, 19, 19, 15, 13, 7, 7, 2, 2, 1, 2]}}}, "feature_ablation": {"full": 0.17019370187580518, "drop_basin": 0.1894123038073051, "basin_only": 0.24263792613475368, "drop_energy": 0.16235662247824972, "energy_only": 0.22078843230036388, "drop_curv": 0.16363978075451233, "curv_only": 0.20402026223124906, "drop_dynamics": 0.1930144138670838, "dynamics_only": 0.19918492310239777}, "ece_geometry": 0.03179577510102621}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
+{"run_id": "46bb116f8c7c", "timestamp": "2026-07-27T10:02:33.120221+00:00", "git_sha": "5207145", "config_hash": "609a4152cf1e", "config": {"run": {"seed": 4, "task": "graph_planning", "notes": "E2 graph selective-prediction"}, "model": {"latent_dim": 32, "hidden_dim": 128, "context_dim": 64}, "inference": {"k_restarts": 12, "steps": 50, "step_size": 0.1, "temperature": 0.05, "init_scale": 1.0, "grad_tol": 0.001}, "train": {"epochs": 25, "batch_size": 128, "lr": 0.001, "n_train": 8000, "n_neg": 4, "neg_noise": 0.5}, "eval": {"n_eval": 600}, "conformal": {"alpha": 0.1, "delta": 0.05, "n_calib": 800}, "task": {"graph_planning": {"n_nodes": 7, "ood_n_nodes": 10, "edge_prob": 0.4, "max_len": 4}}}, "task": "graph_planning", "split": "selective", "seed": 4, "metrics": {"n_fit": 600, "n_calib": 800, "n_test": 600, "k_restarts": 12, "accuracy_id": 0.7183333333333334, "base_error": 0.2816666666666667, "final_train_loss": 0.21295076608657837, "feature_names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "aurc": {"geometry": 0.13764728016172775, "rho_basin": 0.24498732350693359, "energy_min": 0.21394670001342045, "energy_mean": 0.21459813139712064, "energy_std": 0.29739590638116215, "msp": 0.12281914609984014, "temp_msp": 0.1165771595942391, "entropy": 0.11947392267830788}, "temperature": 2.500094145198995, "best_energy_baseline": "energy_min", "best_baseline": "temp_msp", "delta_aurc_vs_energy_min": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "delta_aurc_vs_best_energy": [0.0762994198516927, 0.04508991076645607, 0.10959431704920404], "delta_aurc_vs_best_baseline": [-0.02107012056748865, -0.03689862693304643, -0.005932362296364964], "geometry_wins": true, "geometry_wins_vs_baseline": false, "risk_coverage": {"geometry": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.041666666666666664, 0.03333333333333333, 0.04166666666666666, 0.059523809523809534, 0.052083333333333336, 0.05555555555555555, 0.06666666666666668, 0.06060606060606061, 0.06944444444444445, 0.07692307692307693, 0.08333333333333333, 0.08888888888888888, 0.09375, 0.09803921568627451, 0.10185185185185183, 0.10526315789473684, 0.10833333333333334, 0.11904761904761903, 0.125, 0.13043478260869565, 0.13194444444444442, 0.13666666666666666, 0.13782051282051283, 0.1419753086419753, 0.14583333333333334, 0.1436781609195402, 0.15555555555555556, 0.16129032258064516, 0.16666666666666666, 0.18181818181818182, 0.18872549019607843, 0.19523809523809538, 0.19444444444444456, 0.19369369369369369, 0.19956140350877194, 0.202991452991453, 0.20833333333333334, 0.21747967479674807, 0.22619047619047616, 0.23255813953488372, 0.24621212121212122, 0.2537037037037037, 0.2608695652173913, 0.2641843971631207, 0.2690972222222223, 0.27721088435374147, 0.2816666666666667]}, "rho_basin": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.24151696606786424, 0.24151696606786435, 0.24151696606786446, 0.24151696606786452, 0.24151696606786455, 0.2415169660678644, 0.24151696606786419, 0.24151696606786402, 0.2415169660678639, 0.2415169660678638, 0.24151696606786371, 0.24151696606786366, 0.2415169660678636, 0.24151696606786355, 0.24151696606786352, 0.24151696606786346, 0.24151696606786344, 0.2415169660678634, 0.24151696606786338, 0.24151696606786335, 0.24151696606786333, 0.24151696606786333, 0.2415169660678633, 0.24151696606786327, 0.24151696606786327, 0.24151696606786324, 0.24151696606786324, 0.24151696606786321, 0.24151696606786321, 0.2415169660678632, 0.2415169660678632, 0.2415169660678632, 0.24151696606786316, 0.24151696606786316, 0.24151696606786316, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.24151696606786313, 0.2415169660678631, 0.24241780045351358, 0.2459163898117374, 0.2492559523809511, 0.2551146384479704, 0.26169301712779824, 0.2708594075928229, 0.27747140522875685, 0.2808956916099758, 0.2816666666666653]}, "energy_min": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.08333333333333333, 0.041666666666666664, 0.1111111111111111, 0.125, 0.15, 0.18055555555555552, 0.1785714285714286, 0.17708333333333334, 0.2037037037037037, 0.1916666666666667, 0.18181818181818182, 0.18055555555555555, 0.17307692307692307, 0.18452380952380953, 0.18333333333333346, 0.19270833333333334, 0.19117647058823528, 0.1851851851851853, 0.20175438596491227, 0.21666666666666667, 0.21428571428571425, 0.22348484848484848, 0.2210144927536232, 0.21527777777777776, 0.21, 0.21474358974358973, 0.21296296296296297, 0.2113095238095238, 0.21264367816091967, 0.21944444444444444, 0.22311827956989247, 0.23177083333333334, 0.23232323232323232, 0.24019607843137256, 0.24999999999999997, 0.25231481481481494, 0.2545045045045045, 0.2543859649122807, 0.25427350427350426, 0.25625, 0.26219512195121947, 0.2718253968253969, 0.27325581395348836, 0.2784090909090909, 0.2851851851851852, 0.2826086956521739, 0.2819148936170214, 0.28472222222222215, 0.28401360544217685, 0.2816666666666667]}, "energy_mean": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.16666666666666666, 0.20833333333333334, 0.1388888888888889, 0.16666666666666666, 0.15, 0.13888888888888887, 0.11904761904761907, 0.125, 0.12037037037037036, 0.11666666666666668, 0.12121212121212122, 0.14583333333333334, 0.17307692307692307, 0.19642857142857142, 0.19444444444444442, 0.19270833333333334, 0.18627450980392157, 0.19444444444444456, 0.20614035087719298, 0.20833333333333334, 0.21428571428571425, 0.21212121212121213, 0.20652173913043478, 0.21874999999999997, 0.22666666666666666, 0.22435897435897437, 0.2222222222222222, 0.22321428571428573, 0.2212643678160919, 0.21944444444444444, 0.22043010752688172, 0.22395833333333334, 0.23484848484848486, 0.24019607843137256, 0.23809523809523805, 0.2384259259259259, 0.24324324324324326, 0.25, 0.2606837606837607, 0.26458333333333334, 0.27235772357723587, 0.2738095238095238, 0.2751937984496124, 0.2784090909090909, 0.2777777777777778, 0.27898550724637683, 0.2819148936170214, 0.28298611111111105, 0.28061224489795916, 0.2816666666666667]}, "energy_std": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.4166666666666667, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.3333333333333333, 0.36111111111111105, 0.3452380952380951, 0.3229166666666667, 0.3148148148148148, 0.30000000000000004, 0.30303030303030304, 0.2986111111111111, 0.2948717948717949, 0.2857142857142857, 0.28333333333333327, 0.28125, 0.28431372549019607, 0.287037037037037, 0.2894736842105263, 0.2916666666666667, 0.2936507936507936, 0.2803030303030303, 0.2826086956521739, 0.28124999999999994, 0.28, 0.27884615384615385, 0.28703703703703703, 0.28273809523809523, 0.2816091954022988, 0.2916666666666667, 0.28225806451612906, 0.2760416666666667, 0.2803030303030303, 0.28431372549019607, 0.28571428571428564, 0.28703703703703715, 0.2905405405405405, 0.28289473684210525, 0.2905982905982906, 0.28958333333333336, 0.290650406504065, 0.2936507936507936, 0.29069767441860467, 0.2916666666666667, 0.28888888888888886, 0.28804347826086957, 0.2836879432624113, 0.28298611111111105, 0.282312925170068, 0.2816666666666667]}, "msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.02380952380952381, 0.020833333333333332, 0.027777777777777776, 0.04166666666666653, 0.03787878787878788, 0.034722222222222224, 0.038461538461538464, 0.047619047619047616, 0.04444444444444444, 0.041666666666666664, 0.05392156862745098, 0.06481481481481495, 0.07456140350877193, 0.07916666666666666, 0.07936507936507949, 0.09090909090909091, 0.10507246376811594, 0.11458333333333331, 0.11666666666666667, 0.13141025641025642, 0.1419753086419753, 0.1488095238095238, 0.15517241379310343, 0.15, 0.15591397849462366, 0.15625, 0.16161616161616163, 0.16911764705882354, 0.18095238095238092, 0.18981481481481494, 0.19594594594594594, 0.19956140350877194, 0.2094017094017094, 0.21041666666666667, 0.2195121951219512, 0.22420634920634933, 0.23255813953488372, 0.24242424242424243, 0.2518518518518518, 0.25905797101449274, 0.2695035460992909, 0.2777777777777779, 0.27721088435374147, 0.2816666666666667]}, "temp_msp": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.0, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.010416666666666666, 0.018518518518518517, 0.01666666666666667, 0.030303030303030304, 0.027777777777777776, 0.038461538461538464, 0.041666666666666664, 0.03888888888888888, 0.041666666666666664, 0.058823529411764705, 0.0648148148148148, 0.07456140350877193, 0.07916666666666666, 0.09126984126984139, 0.10227272727272728, 0.10869565217391304, 0.11111111111111109, 0.12333333333333334, 0.1282051282051282, 0.13271604938271606, 0.14285714285714285, 0.1408045977011494, 0.14444444444444443, 0.1424731182795699, 0.1484375, 0.15151515151515152, 0.1568627450980392, 0.16190476190476188, 0.17129629629629645, 0.17792792792792791, 0.18859649122807018, 0.19658119658119658, 0.20416666666666666, 0.2073170731707318, 0.21428571428571438, 0.22093023255813954, 0.23106060606060605, 0.2351851851851852, 0.24456521739130435, 0.25177304964539016, 0.2656250000000001, 0.27721088435374147, 0.2816666666666667]}, "entropy": {"coverage": [0.02, 0.04, 0.06, 0.08, 0.1, 0.12000000000000001, 0.13999999999999999, 0.16, 0.18, 0.19999999999999998, 0.22, 0.24, 0.26, 0.28, 0.30000000000000004, 0.32, 0.34, 0.36000000000000004, 0.38, 0.4, 0.42000000000000004, 0.44, 0.46, 0.48000000000000004, 0.5, 0.52, 0.54, 0.56, 0.5800000000000001, 0.6, 0.62, 0.64, 0.66, 0.68, 0.7000000000000001, 0.7200000000000001, 0.74, 0.76, 0.78, 0.8, 0.8200000000000001, 0.8400000000000001, 0.86, 0.88, 0.9, 0.92, 0.9400000000000001, 0.9600000000000001, 0.98, 1.0], "risk": [0.0, 0.0, 0.0, 0.020833333333333332, 0.016666666666666666, 0.013888888888888886, 0.011904761904761906, 0.020833333333333332, 0.027777777777777776, 0.04166666666666653, 0.03787878787878788, 0.034722222222222224, 0.038461538461538464, 0.041666666666666664, 0.04444444444444444, 0.046875, 0.049019607843137254, 0.06481481481481495, 0.07456140350877193, 0.08333333333333333, 0.08333333333333345, 0.09090909090909091, 0.10869565217391304, 0.11458333333333331, 0.11666666666666667, 0.13141025641025642, 0.1388888888888889, 0.1488095238095238, 0.14942528735632202, 0.15, 0.15053763440860216, 0.15625, 0.15656565656565657, 0.16176470588235295, 0.17857142857142855, 0.18518518518518515, 0.18468468468468469, 0.19298245614035087, 0.19658119658119658, 0.20625, 0.20934959349593507, 0.21825396825396823, 0.22868217054263565, 0.23106060606060605, 0.23703703703703705, 0.24456521739130435, 0.25177304964539016, 0.263888888888889, 0.27380952380952384, 0.2816666666666667]}}, "ltt": {"alpha": 0.1, "delta": 0.05, "lambda_hat": null, "coverage": 0.0, "abstain_rate": 1.0, "selective_risk": 0.0, "selective_accuracy": null, "risk_within_budget": true}, "coverage_validity": {"alpha_grid": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "target": [0.02, 0.05, 0.1, 0.15, 0.2, 0.3], "achieved_risk": [0.0, 0.0, 0.0, 0.0, 0.13028169014084506, 0.2537037037037037], "coverage": [0.0, 0.0, 0.0, 0.0, 0.47333333333333333, 0.9]}, "feature_diagnostics": {"names": ["basin/rho", "basin/entropy", "basin/dispersion", "energy/mean", "energy/min", "energy/std", "curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best", "dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"], "auroc": {"basin/rho": 0.5830942215022172, "basin/entropy": 0.4163291643213114, "basin/dispersion": 0.5237029613256634, "energy/mean": 0.39264679635909333, "energy/min": 0.39925040157058717, "energy/std": 0.5168522357528247, "curv/lmax_mean": 0.5929790359560125, "curv/lmax_best": 0.5272518842927553, "curv/trace_mean": 0.6412773376899737, "curv/trace_best": 0.6202446491577315, "dynamics/steps": 0.5, "dynamics/monotonic": 0.6388473208034158, "dynamics/drop": 0.6872691827180494, "dynamics/residual": 0.5027938329740935}, "hist": {"basin/rho": {"edges": [0.4166666666666667, 0.44583333333333336, 0.47500000000000003, 0.5041666666666667, 0.5333333333333333, 0.5625, 0.5916666666666667, 0.6208333333333333, 0.65, 0.6791666666666667, 0.7083333333333333, 0.7375, 0.7666666666666666, 0.7958333333333334, 0.825, 0.8541666666666666, 0.8833333333333333, 0.9125, 0.9416666666666667, 0.9708333333333332, 1.0], "correct_counts": [1, 0, 6, 0, 0, 7, 0, 0, 7, 0, 0, 3, 0, 0, 10, 0, 0, 17, 0, 380], "incorrect_counts": [0, 0, 3, 0, 0, 5, 0, 0, 10, 0, 0, 8, 0, 0, 11, 0, 0, 11, 0, 121]}, "basin/entropy": {"edges": [0.0, 0.05387781635334005, 0.1077556327066801, 0.16163344906002014, 0.2155112654133602, 0.26938908176670023, 0.3232668981200403, 0.3771447144733803, 0.4310225308267204, 0.4849003471800604, 0.5387781635334005, 0.5926559798867406, 0.6465337962400806, 0.7004116125934206, 0.7542894289467607, 0.8081672453001008, 0.8620450616534407, 0.9159228780067807, 0.9698006943601208, 1.023678510713461, 1.077556327066801], "correct_counts": [380, 0, 0, 0, 0, 17, 0, 0, 10, 0, 3, 7, 13, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [121, 0, 0, 0, 0, 11, 0, 0, 10, 0, 8, 10, 6, 1, 0, 0, 1, 0, 1, 0]}, "basin/dispersion": {"edges": [1.6070286588464016, 1.6272769139695735, 1.6475251690927455, 1.6677734242159175, 1.6880216793390894, 1.7082699344622614, 1.7285181895854331, 1.748766444708605, 1.769014699831777, 1.789262954954949, 1.809511210078121, 1.829759465201293, 1.850007720324465, 1.8702559754476369, 1.8905042305708089, 1.9107524856939806, 1.9310007408171526, 1.9512489959403245, 1.9714972510634965, 1.9917455061866685, 2.0119937613098404], "correct_counts": [1, 1, 5, 5, 15, 22, 20, 31, 44, 54, 51, 65, 43, 26, 14, 11, 11, 4, 5, 3], "incorrect_counts": [2, 0, 1, 3, 1, 14, 9, 18, 19, 20, 20, 17, 13, 13, 8, 5, 1, 3, 1, 1]}, "energy/mean": {"edges": [0.7167260174949964, 0.7730059100935857, 0.8292858026921749, 0.8855656952907642, 0.9418455878893535, 0.9981254804879427, 1.054405373086532, 1.1106852656851212, 1.1669651582837104, 1.2232450508822996, 1.2795249434808889, 1.335804836079478, 1.3920847286780673, 1.4483646212766565, 1.504644513875246, 1.5609244064738352, 1.6172042990724245, 1.6734841916710137, 1.7297640842696032, 1.7860439768681924, 1.8423238694667816], "correct_counts": [12, 33, 69, 58, 50, 36, 27, 17, 19, 17, 10, 9, 15, 11, 19, 9, 11, 4, 1, 4], "incorrect_counts": [2, 6, 6, 27, 20, 14, 6, 11, 8, 11, 11, 11, 8, 8, 7, 5, 4, 1, 3, 0]}, "energy/min": {"edges": [0.33279895782470703, 0.39212902784347536, 0.45145909786224364, 0.5107891678810119, 0.5701192378997803, 0.6294493079185486, 0.6887793779373169, 0.7481094479560852, 0.8074395179748535, 0.8667695879936218, 0.9260996580123901, 0.9854297280311585, 1.0447597980499268, 1.1040898680686952, 1.1634199380874635, 1.2227500081062317, 1.282080078125, 1.3414101481437684, 1.4007402181625366, 1.460070288181305, 1.5194003582000732], "correct_counts": [4, 17, 34, 53, 57, 47, 49, 24, 22, 14, 20, 16, 12, 13, 10, 15, 8, 8, 6, 2], "incorrect_counts": [1, 0, 11, 12, 15, 22, 9, 12, 10, 16, 8, 8, 15, 11, 5, 5, 5, 3, 0, 1]}, "energy/std": {"edges": [0.06888872658823433, 0.08357696314973125, 0.09826519971122816, 0.11295343627272506, 0.12764167283422195, 0.14232990939571888, 0.15701814595721578, 0.17170638251871267, 0.1863946190802096, 0.2010828556417065, 0.21577109220320342, 0.23045932876470032, 0.24514756532619722, 0.2598358018876942, 0.27452403844919104, 0.28921227501068797, 0.3039005115721849, 0.31858874813368177, 0.33327698469517864, 0.3479652212566756, 0.3626534578181725], "correct_counts": [1, 0, 4, 8, 21, 36, 52, 55, 56, 50, 45, 26, 23, 26, 14, 5, 2, 5, 1, 1], "incorrect_counts": [0, 0, 2, 5, 9, 17, 15, 24, 22, 17, 18, 17, 9, 4, 4, 3, 2, 1, 0, 0]}, "curv/lmax_mean": {"edges": [0.999078502257665, 0.9991983219981194, 0.9993181417385737, 0.9994379614790281, 0.9995577812194825, 0.9996776009599369, 0.9997974207003912, 0.9999172404408455, 1.0000370601812998, 1.0001568799217542, 1.0002766996622086, 1.000396519402663, 1.0005163391431173, 1.0006361588835717, 1.000755978624026, 1.0008757983644805, 1.0009956181049349, 1.001115437845389, 1.0012352575858434, 1.0013550773262978, 1.0014748970667522], "correct_counts": [3, 8, 21, 38, 61, 87, 93, 58, 24, 18, 10, 6, 2, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 5, 5, 21, 32, 48, 38, 10, 5, 3, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/lmax_best": {"edges": [0.995195746421814, 0.9959201753139496, 0.9966446042060852, 0.9973690330982208, 0.9980934619903564, 0.9988178908824921, 0.9995423197746277, 1.0002667486667634, 1.0009911775588989, 1.0017156064510346, 1.0024400353431702, 1.0031644642353057, 1.0038888931274415, 1.004613322019577, 1.0053377509117127, 1.0060621798038483, 1.0067866086959838, 1.0075110375881196, 1.008235466480255, 1.0089598953723908, 1.0096843242645264], "correct_counts": [2, 1, 2, 1, 13, 45, 337, 24, 3, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1], "incorrect_counts": [0, 1, 0, 1, 6, 26, 130, 4, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "curv/trace_mean": {"edges": [31.831074873606365, 31.84024980068207, 31.84942472775777, 31.858599654833476, 31.86777458190918, 31.876949508984886, 31.88612443606059, 31.895299363136292, 31.904474290211997, 31.913649217287702, 31.922824144363403, 31.931999071439108, 31.941173998514813, 31.950348925590518, 31.959523852666223, 31.968698779741924, 31.97787370681763, 31.987048633893334, 31.996223560969035, 32.00539848804474, 32.014573415120445], "correct_counts": [1, 4, 6, 12, 12, 19, 14, 18, 19, 15, 14, 9, 12, 16, 10, 25, 44, 90, 75, 16], "incorrect_counts": [1, 1, 4, 8, 7, 11, 15, 9, 8, 15, 11, 2, 3, 9, 9, 12, 12, 18, 11, 3]}, "curv/trace_best": {"edges": [31.66323471069336, 31.682111549377442, 31.700988388061525, 31.719865226745604, 31.738742065429687, 31.75761890411377, 31.776495742797852, 31.795372581481935, 31.814249420166014, 31.833126258850097, 31.85200309753418, 31.870879936218262, 31.889756774902345, 31.908633613586424, 31.927510452270507, 31.94638729095459, 31.965264129638673, 31.984140968322755, 32.003017807006835, 32.02189464569092, 32.040771484375], "correct_counts": [2, 0, 1, 0, 5, 4, 0, 5, 3, 10, 4, 19, 20, 21, 22, 43, 60, 148, 60, 4], "incorrect_counts": [0, 0, 0, 1, 2, 1, 0, 1, 2, 4, 8, 7, 12, 14, 24, 18, 32, 35, 8, 0]}, "dynamics/steps": {"edges": [1.0, 1.05, 1.1, 1.15, 1.2, 1.25, 1.3, 1.35, 1.4, 1.45, 1.5, 1.55, 1.6, 1.65, 1.7000000000000002, 1.75, 1.8, 1.85, 1.9, 1.9500000000000002, 2.0], "correct_counts": [431, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], "incorrect_counts": [169, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/monotonic": {"edges": [0.6349999999999999, 0.6424166666666666, 0.6498333333333333, 0.6572499999999999, 0.6646666666666666, 0.6720833333333334, 0.6795, 0.6869166666666666, 0.6943333333333334, 0.7017500000000001, 0.7091666666666667, 0.7165833333333333, 0.7240000000000001, 0.7314166666666668, 0.7388333333333335, 0.7462500000000001, 0.7536666666666668, 0.7610833333333336, 0.7685000000000002, 0.7759166666666668, 0.7833333333333335], "correct_counts": [1, 1, 1, 13, 22, 38, 53, 49, 83, 45, 40, 41, 19, 12, 4, 2, 4, 2, 0, 1], "incorrect_counts": [0, 0, 5, 7, 15, 13, 39, 25, 32, 15, 5, 10, 3, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/drop": {"edges": [14.843120073278746, 20.788069765021405, 26.73301945676406, 32.677969148506726, 38.62291884024938, 44.56786853199204, 50.512818223734705, 56.45776791547736, 62.40271760722002, 68.34766729896268, 74.29261699070533, 80.23756668244799, 86.18251637419066, 92.1274660659333, 98.07241575767597, 104.01736544941862, 109.96231514116128, 115.90726483290395, 121.8522145246466, 127.79716421638926, 133.74211390813193], "correct_counts": [50, 128, 104, 57, 41, 25, 11, 2, 3, 3, 3, 2, 1, 0, 0, 0, 0, 0, 0, 1], "incorrect_counts": [48, 69, 30, 12, 7, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}, "dynamics/residual": {"edges": [1.1786483426888783, 1.1918049529194832, 1.2049615631500878, 1.2181181733806927, 1.2312747836112976, 1.2444313938419023, 1.2575880040725071, 1.270744614303112, 1.283901224533717, 1.2970578347643216, 1.3102144449949265, 1.3233710552255313, 1.3365276654561362, 1.3496842756867409, 1.3628408859173458, 1.3759974961479506, 1.3891541063785553, 1.4023107166091602, 1.415467326839765, 1.4286239370703697, 1.4417805473009746], "correct_counts": [3, 1, 2, 11, 24, 23, 30, 42, 48, 52, 46, 40, 39, 31, 11, 11, 11, 4, 1, 1], "incorrect_counts": [2, 2, 2, 6, 8, 9, 10, 14, 22, 13, 22, 12, 20, 12, 7, 4, 3, 0, 0, 1]}}}, "feature_ablation": {"full": 0.13764728016172775, "drop_basin": 0.1472579859743771, "basin_only": 0.24918451421966334, "drop_energy": 0.14477184372770538, "energy_only": 0.20943845240356984, "drop_curv": 0.13728290417794947, "curv_only": 0.19597353911884235, "drop_dynamics": 0.18465596709524087, "dynamics_only": 0.16587248520081196}, "ece_geometry": 0.06147257738915552}, "env": {"python": "3.12.13", "platform": "macOS-26.5.2-arm64-arm-64bit", "jax": "0.11.0", "jax_backend": "cpu"}, "artifact_paths": []}
diff --git a/src/edc/cli.py b/src/edc/cli.py
index d3095cd..88a0c73 100644
--- a/src/edc/cli.py
+++ b/src/edc/cli.py
@@ -14,10 +14,13 @@
 import datetime as _dt
 import uuid
 
+import jax
 import jax.numpy as jnp
 import numpy as np
 
 from edc.config import load_config
+from edc.eval import metrics
+from edc.geometry.features import geometry_features
 from edc.inference import restarts
 from edc.ledger import RunRecord, append
 from edc.registry import build_task
@@ -79,6 +82,85 @@ def cmd_smoke(args: argparse.Namespace) -> int:
     return 0
 
 
+def _geometry_report(fns, params, cfg, task, split, key) -> dict:
+    """Compute geometry features on a split and score each feature's correct-vs-incorrect AUROC."""
+    k_solve, k_geom = jax.random.split(key)
+    rng = numpy_rng(cfg.run.seed, 8, 0 if split == "id" else 1)
+    batch = task.sample(rng, cfg.eval.n_eval, split=split)
+    traj = restarts.solve(fns, params, jnp.asarray(batch.x), cfg, k_solve)
+
+    feats, names = geometry_features(fns=fns, params=params, traj=traj, key=k_geom,
+                                     grad_tol=cfg.inference.grad_tol)
+    pred_boN, _ = restarts.best_of_n_energy(traj)
+    correct = task.evaluate(np.asarray(pred_boN), batch.y)
+
+    auroc = {name: metrics.single_feature_auroc(feats[:, j], correct)
+             for j, name in enumerate(names)}
+    return {"accuracy": float(np.mean(correct)), "feature_auroc": auroc, "feature_names": names}
+
+
+def _print_geometry_table(split: str, report: dict) -> None:
+    sep = lambda a: abs(a - 0.5)  # noqa: E731  distance from chance = separation strength
+    ranked = sorted(report["feature_auroc"].items(), key=lambda kv: sep(kv[1]), reverse=True)
+    print(f"\n[edc] {split.upper()} base acc={report['accuracy']:.3f} "
+          f"(best-of-N energy) — per-feature correct-vs-incorrect AUROC:")
+    for name, a in ranked:
+        group = "energy" if name.startswith("energy/") else "geom  "
+        print(f"       {group}  {name:<20} AUROC={a:.3f}  |sep|={sep(a):.3f}")
+
+
+def cmd_geometry(args: argparse.Namespace) -> int:
+    cfg = load_config(args.config)
+    task = build_task(cfg.run.task, **cfg.to_dict().get("task", {}).get(cfg.run.task, {}))
+
+    print(f"[edc] training on '{task.name}' (seed={cfg.run.seed}) ...")
+    params, fns, history = train(cfg, task)
+    print(f"[edc] final train loss={history['epochs'][-1]['loss']:.4f}")
+
+    key = root_key(cfg.run.seed)
+    k_id, k_ood = jax.random.split(key)
+    reports = {"id": _geometry_report(fns, params, cfg, task, "id", k_id),
+               "ood": _geometry_report(fns, params, cfg, task, "ood", k_ood)}
+
+    sep = lambda a: abs(a - 0.5)  # noqa: E731
+    for split in ("id", "ood"):
+        _print_geometry_table(split, reports[split])
+
+    # Falsification smell test: does the best geometry feature separate better than the best
+    # raw-energy feature (the EBT baseline)? Full ΔAURC bootstrap arrives in Phase 3.
+    auroc = reports["id"]["feature_auroc"]
+    geom = {n: a for n, a in auroc.items() if not n.startswith("energy/")}
+    energy = {n: a for n, a in auroc.items() if n.startswith("energy/")}
+    best_geom = max(geom, key=lambda n: sep(geom[n]))
+    best_energy = max(energy, key=lambda n: sep(energy[n]))
+    ahead = sep(geom[best_geom]) > sep(energy[best_energy])
+    verdict = "geometry ahead" if ahead else "energy ahead"
+    print(f"\n[edc] ID best geometry '{best_geom}' |sep|={sep(geom[best_geom]):.3f}  vs  "
+          f"best energy '{best_energy}' |sep|={sep(energy[best_energy]):.3f}  ({verdict})")
+
+    metrics_row = {
+        "id": reports["id"],
+        "ood": reports["ood"],
+        "best_geometry_feature": best_geom,
+        "best_geometry_sep": sep(geom[best_geom]),
+        "best_energy_feature": best_energy,
+        "best_energy_sep": sep(energy[best_energy]),
+        "k_restarts": cfg.inference.k_restarts,
+        "steps": cfg.inference.steps,
+    }
+    row = append(RunRecord(
+        run_id=uuid.uuid4().hex[:12],
+        timestamp=_dt.datetime.now(_dt.UTC).isoformat(),
+        resolved_config=cfg.to_dict(),
+        task=task.name,
+        split="geometry-diagnostic",
+        seed=cfg.run.seed,
+        metrics=metrics_row,
+    ))
+    print(f"[edc] appended ledger row run_id={row['run_id']} -> results/ledger.jsonl")
+    return 0
+
+
 def main(argv: list[str] | None = None) -> int:
     p = argparse.ArgumentParser(prog="edc", description="Energy Descent Certificates")
     sub = p.add_subparsers(dest="cmd", required=True)
@@ -87,6 +169,10 @@ def main(argv: list[str] | None = None) -> int:
     ps.add_argument("--config", default="configs/smoke.toml")
     ps.set_defaults(func=cmd_smoke)
 
+    pg = sub.add_parser("geometry", help="Phase-2 geometry-feature separation diagnostic (CPU)")
+    pg.add_argument("--config", default="configs/smoke.toml")
+    pg.set_defaults(func=cmd_geometry)
+
     args = p.parse_args(argv)
     return args.func(args)
 
diff --git a/src/edc/conformal/crc.py b/src/edc/conformal/crc.py
index 32a16df..c50dcd9 100644
--- a/src/edc/conformal/crc.py
+++ b/src/edc/conformal/crc.py
@@ -5,10 +5,62 @@
 errors). CRC (Angelopoulos et al., arXiv:2208.02814) then picks ``lambda_hat`` so
 ``E[L(lambda_hat)] <= alpha`` in finite samples. Monotonicity is exactly why CRC — not LTT —
 is the right tool here.
+
+This module is the pure calibrator. The halting *policy* that produces ``loss_by_lambda`` from a
+trajectory (``halting.adaptive``) and figure F4 are deferred (they need per-step decoded answers
+on ``TrajectoryRecord``). NumPy only (invariant 1).
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+
+def _normalise(loss_by_lambda, lambdas):
+    """Return ``(lambdas ascending, loss_matrix (n_cal, n_lambda) aligned to them)``.
+
+    Accepts a ``dict {lambda: per-point loss vector}`` or a 2D array with an explicit ``lambdas``.
+    Lambdas are sorted ascending; by convention a larger lambda halts earlier => saves more
+    compute => weakly higher loss.
+    """
+    if isinstance(loss_by_lambda, dict):
+        lams = np.array(sorted(loss_by_lambda), dtype=float)
+        mat = np.stack([np.asarray(loss_by_lambda[k], dtype=float) for k in lams], axis=1)
+        return lams, mat
+    mat = np.asarray(loss_by_lambda, dtype=float)
+    if mat.ndim != 2:
+        raise ValueError("loss_by_lambda must be a dict or a (n_cal, n_lambda) 2D array")
+    lams = np.arange(mat.shape[1], dtype=float) if lambdas is None else np.asarray(lambdas, float)
+    order = np.argsort(lams)
+    return lams[order], mat[:, order]
+
+
+def calibrate(loss_by_lambda, alpha: float, lambdas=None):
+    """Pick the most compute-saving ``lambda_hat`` whose CRC bound holds, or ``None`` if infeasible.
+
+    Finite-sample CRC bound with bounded loss ``B = 1`` (mirrors the ``(n+1)`` correction in
+    ``split_conformal``)::
+
+        (n / (n + 1)) * R_hat(lambda) + 1 / (n + 1) <= alpha
+
+    where ``R_hat`` is the mean loss over the ``n`` calibration points. Because loss is monotone
+    non-decreasing in ``lambda`` (more compute saved => weakly more errors), the admissible set is
+    a prefix in ascending-lambda order; we scan that order and return the **largest admissible**
+    lambda (maximal compute saving while safe). Returns ``None`` when even the most conservative
+    lambda violates the bound — including the structurally infeasible regime ``alpha < 1/(n+1)``,
+    where no risk (not even 0) can satisfy it.
+    """
+    lams, mat = _normalise(loss_by_lambda, lambdas)
+    n = mat.shape[0]
+    if n == 0:
+        raise ValueError("need at least one calibration point")
+    rhs = (alpha * (n + 1) - 1.0) / n  # R_hat must be <= this
+    r_hat = mat.mean(axis=0)
 
-def calibrate(loss_by_lambda, alpha: float):
-    raise NotImplementedError("Phase 3: choose the smallest-compute lambda with E[L] <= alpha.")
+    lambda_hat = None
+    for lam, r in zip(lams, r_hat, strict=True):
+        if r <= rhs:
+            lambda_hat = float(lam)  # extend the admissible prefix
+        else:
+            break  # monotone: once it fails, larger lambdas only get worse
+    return lambda_hat
diff --git a/src/edc/conformal/ltt.py b/src/edc/conformal/ltt.py
index bbfb6d5..3389cc3 100644
--- a/src/edc/conformal/ltt.py
+++ b/src/edc/conformal/ltt.py
@@ -5,14 +5,84 @@
 candidate ``lambda`` as a hypothesis ``H_lambda: R_sel(lambda) > alpha``, computes a valid
 Hoeffding-Bentkus p-value, and returns the FWER-controlled admissible set. Guarantee:
 ``P(R_sel(lambda_hat) <= alpha) >= 1 - delta``, distribution-free, under exchangeability.
+
+NumPy / scipy only (invariant 1).
 """
 
 from __future__ import annotations
 
+import math
+
+import numpy as np
+from scipy.special import rel_entr
+from scipy.stats import binom
+
 
 def hoeffding_bentkus_pvalue(risk_hat: float, alpha: float, n: int) -> float:
-    raise NotImplementedError("Phase 3: valid p-value for H: R_sel > alpha.")
+    """Valid p-value for ``H: R > alpha`` given empirical mean loss ``risk_hat`` over ``n`` points.
+
+    ``p = min(p_Hoeffding, p_Bentkus, 1)``. Rejecting ``H`` at level ``delta`` certifies
+    ``R <= alpha``. The loss is in ``[0, 1]`` (here a selective error rate).
+
+    - Hoeffding (relative-entropy / Chernoff form, tighter than the additive bound):
+      ``exp(-n * KL(risk_hat || alpha))`` with ``KL`` via ``rel_entr`` so ``0*log 0 -> 0``.
+    - Bentkus: ``e * P(Bin(n, alpha) <= ceil(n * risk_hat))``.
+
+    Returns ``1.0`` when ``risk_hat >= alpha`` (no evidence to reject) or ``n <= 0`` (an empty
+    sample certifies nothing) — the Hoeffding form is only valid for ``risk_hat < alpha``.
+    """
+    if n <= 0 or risk_hat >= alpha:
+        return 1.0
+    risk_hat = max(float(risk_hat), 0.0)
+    kl = float(rel_entr(risk_hat, alpha) + rel_entr(1.0 - risk_hat, 1.0 - alpha))
+    p_hoeffding = math.exp(-n * kl)
+    p_bentkus = math.e * float(binom.cdf(math.ceil(n * risk_hat), n, alpha))
+    return float(min(p_hoeffding, p_bentkus, 1.0))
+
+
+def selective_risk(scores_cal, correct_cal, lam: float) -> tuple[float, int]:
+    """``(R_sel, n_answered)`` at threshold ``lam``: error rate among answered inputs (s <= lam)."""
+    scores = np.asarray(scores_cal, dtype=float)
+    correct = np.asarray(correct_cal).astype(bool)
+    answered = scores <= lam
+    n_ans = int(answered.sum())
+    if n_ans == 0:
+        return float("nan"), 0
+    return float((~correct[answered]).mean()), n_ans
+
+
+def calibrate(scores_cal, correct_cal, alpha: float, delta: float, lambdas=None,
+              n_grid: int = 100) -> dict:
+    """Return the FWER-controlled admissible thresholds and the max-coverage pick.
+
+    For each candidate threshold ``lambda`` the selective risk ``R_sel`` is the error rate among
+    answered inputs (``s <= lambda``) over ``n_ans`` points, with p-value ``HB(R_sel, alpha,
+    n_ans)`` for ``H_lambda: R_sel > alpha``. Because ``R_sel`` is **non-monotone** in ``lambda``,
+    fixed-sequence testing does not apply (the smallest thresholds answer too few points to have
+    any statistical power, so a sequence would stall immediately); we instead use **Bonferroni**
+    over a bounded quantile grid of ``m`` candidates: admissible ``= {lambda : p(lambda) <=
+    delta/m}``, which controls FWER at ``delta`` for arbitrary thresholds. ``lambda_hat`` is the
+    largest admissible threshold (max coverage), or ``None`` if none pass — abstain on everything
+    at this ``(alpha, delta)`` given ``n``.
+
+    An empty answered set (``lambda`` below every score) gives ``p = 1`` (never admissible; no 0/0).
+    Pass explicit ``lambdas`` to override the grid.
+    """
+    scores = np.asarray(scores_cal, dtype=float)
+    correct = np.asarray(correct_cal).astype(bool)
+    if lambdas is None:
+        # Quantile grid keeps the multiplicity m bounded (so delta/m is not crushed) while still
+        # placing candidates where the scores actually are.
+        lambdas = np.unique(np.quantile(scores, np.linspace(0.0, 1.0, n_grid)))
+    lambdas = np.sort(np.asarray(lambdas, dtype=float))
+    m = lambdas.shape[0]
 
+    pvalues = np.empty(m, dtype=float)
+    for i, lam in enumerate(lambdas):
+        r_sel, n_ans = selective_risk(scores, correct, lam)
+        pvalues[i] = 1.0 if n_ans == 0 else hoeffding_bentkus_pvalue(r_sel, alpha, n_ans)
 
-def calibrate(scores_cal, correct_cal, alpha: float, delta: float):
-    raise NotImplementedError("Phase 3: return admissible thresholds controlling selective risk.")
+    admissible = lambdas[pvalues <= delta / m]  # Bonferroni
+    lambda_hat = float(admissible.max()) if admissible.size else None
+    return {"admissible": admissible, "lambda_hat": lambda_hat, "pvalues": pvalues,
+            "lambdas": lambdas, "m": m}
diff --git a/src/edc/conformal/nonconformity.py b/src/edc/conformal/nonconformity.py
index 9f42822..67f7fb8 100644
--- a/src/edc/conformal/nonconformity.py
+++ b/src/edc/conformal/nonconformity.py
@@ -1,17 +1,78 @@
 """Nonconformity scores from geometry features. [Phase 3]
 
-Fits a small monotone mapper (logistic regression / gradient-boosted trees) from geometry
-features to ``p_hat(correct)`` on a fold **disjoint** from the calibration fold (invariant 7),
-then scores ``s(x) = 1 - p_hat(correct)``. A hand-defined ``1 - rho_basin`` variant is kept so
-conformal validity does not hinge on the learned mapper.
+Fits a small mapper (logistic regression) from geometry features to ``p_hat(correct)`` on a fold
+**disjoint** from the calibration fold (invariant 7), then scores ``s(x) = 1 - p_hat(correct)``
+(low ``s`` = confident). A hand-defined ``1 - rho_basin`` variant is kept so conformal validity
+does not hinge on the learned mapper.
+
+Plain NumPy / scikit-learn (invariant 1): no JAX. The standardisation statistics travel *inside*
+the returned sklearn ``Pipeline`` so ``score`` is a pure function of the fitted object.
 """
 
 from __future__ import annotations
 
+import numpy as np
+from sklearn.linear_model import LogisticRegression
+from sklearn.pipeline import Pipeline
+from sklearn.preprocessing import StandardScaler
+
+
+class _ConstantMapper:
+    """Degenerate mapper for a single-class fit fold: ``p_hat(correct)`` is a constant.
+
+    Exposes the minimal sklearn surface (``classes_`` + ``predict_proba``) that ``score`` uses, so
+    the all-correct / all-wrong fold produces a consistent constant nonconformity score instead of
+    crashing ``LogisticRegression`` (which needs >= 2 classes).
+    """
+
+    def __init__(self, cls: int) -> None:
+        self.classes_ = np.array([cls])
+
+    def predict_proba(self, x):
+        return np.ones((np.asarray(x).shape[0], 1))
+
 
 def fit_mapper(features_train, correct_train):
-    raise NotImplementedError("Phase 3: fit logistic/GBT features -> p(correct) on the fit fold.")
+    """Fit ``features -> p_hat(correct)`` on the FIT fold (disjoint from calibration, invariant 7).
+
+    Returns a standardise-then-logistic ``Pipeline``; the scaler's mean/scale are stored on the
+    object so ``score`` needs nothing else. A single-class fit fold (all-correct or all-wrong)
+    yields a ``_ConstantMapper`` so ``score`` stays well-defined.
+    """
+    x = np.asarray(features_train, dtype=float)
+    y = np.asarray(correct_train).astype(int)
+    classes = np.unique(y)
+    if classes.shape[0] < 2:
+        return _ConstantMapper(int(classes[0]))
+    mapper = Pipeline(
+        [("scaler", StandardScaler()), ("clf", LogisticRegression(max_iter=1000))]
+    )
+    mapper.fit(x, y)
+    return mapper
+
+
+def score(mapper: Pipeline, features) -> np.ndarray:
+    """Nonconformity score ``s = 1 - p_hat(correct)`` for each row of ``features``.
+
+    Looks up the positive-class (``correct == 1``) column via ``classes_`` rather than assuming
+    column 1; if the fit fold saw only one class, returns a constant score consistent with it.
+    """
+    x = np.asarray(features, dtype=float)
+    classes = mapper.classes_
+    proba = mapper.predict_proba(x)
+    if 1 in classes:
+        p_correct = proba[:, list(classes).index(1)]
+    else:
+        # Fit fold had no correct examples -> model always predicts incorrect -> p_correct = 0.
+        p_correct = np.zeros(x.shape[0])
+    return 1.0 - p_correct
+
 
+def rho_basin_score(rho) -> np.ndarray:
+    """Hand-defined fallback score ``s = 1 - rho_basin`` (no fit).
 
-def score(mapper, features):
-    raise NotImplementedError("Phase 3: s(x) = 1 - p_hat(correct).")
+    ``rho`` is the plurality fraction of decoded answers across restarts
+    (``basin_features(traj)[0][:, 0]``). High agreement => confident => low score. Kept so
+    distribution-free validity never depends on the learned mapper (invariant 7).
+    """
+    return 1.0 - np.asarray(rho, dtype=float)
diff --git a/src/edc/conformal/selective.py b/src/edc/conformal/selective.py
index 9d99d1d..82e409a 100644
--- a/src/edc/conformal/selective.py
+++ b/src/edc/conformal/selective.py
@@ -7,8 +7,20 @@
 
 from __future__ import annotations
 
+import numpy as np
+
 ABSTAIN = -1
 
 
-def selective_predict(pred, scores, threshold):
-    raise NotImplementedError("Phase 3: return pred where score<=threshold else ABSTAIN.")
+def selective_predict(pred, scores, threshold) -> np.ndarray:
+    """Return ``pred`` where ``score <= threshold`` (answer), else ``ABSTAIN`` (-1).
+
+    Boundary answers (``s == threshold``), consistent with ``split_conformal.empirical_coverage``
+    and the ``s <= lambda`` convention used across the conformal layer. ``threshold`` may be
+    ``inf`` (admit all) or ``-inf`` (abstain all).
+    """
+    pred = np.asarray(pred)
+    scores = np.asarray(scores, dtype=float)
+    if pred.shape[0] != scores.shape[0]:
+        raise ValueError("pred and scores must have the same length")
+    return np.where(scores <= threshold, pred, ABSTAIN)
diff --git a/src/edc/eval/baselines.py b/src/edc/eval/baselines.py
new file mode 100644
index 0000000..716831c
--- /dev/null
+++ b/src/edc/eval/baselines.py
@@ -0,0 +1,65 @@
+"""Standard softmax-confidence nonconformity baselines. [Phase 4e]
+
+Geometry must beat not only the EBT scalar energy (invariant 8) but also the obvious decoder-softmax
+confidence signals. These are cheap, standard baselines computed from the mean decoder logits over
+the K restarts (a logit-averaging ensemble read-off), conformalized identically to geometry:
+
+* **MSP** — ``1 - max_c softmax``: the classic maximum-softmax-probability confidence.
+* **temperature-scaled MSP** — MSP after a single scalar temperature fit on the (disjoint) fit fold.
+* **predictive entropy** — entropy of the softmax (higher = less confident).
+
+All return nonconformity scores (low = confident), so ``eval.metrics.aurc`` consumes them directly.
+NumPy/scipy only (invariant 1); temperature fit is deterministic.
+"""
+
+from __future__ import annotations
+
+import numpy as np
+from scipy.optimize import minimize_scalar
+
+
+def softmax(logits: np.ndarray) -> np.ndarray:
+    """Row-wise softmax of ``(n, C)`` logits."""
+    z = np.asarray(logits, dtype=float)
+    z = z - z.max(axis=-1, keepdims=True)
+    e = np.exp(z)
+    return e / e.sum(axis=-1, keepdims=True)
+
+
+def msp_score(mean_logits: np.ndarray) -> np.ndarray:
+    """``1 - max softmax`` — low when the predictive distribution is peaked (confident)."""
+    return 1.0 - softmax(mean_logits).max(axis=-1)
+
+
+def entropy_score(mean_logits: np.ndarray) -> np.ndarray:
+    """Predictive entropy in nats — high when the distribution is flat (unconfident)."""
+    p = softmax(mean_logits)
+    with np.errstate(divide="ignore", invalid="ignore"):
+        logp = np.where(p > 0, np.log(p), 0.0)
+    return -(p * logp).sum(axis=-1)
+
+
+def _nll(temp: float, logits: np.ndarray, y: np.ndarray) -> float:
+    """Mean negative log-likelihood of ``y`` under ``softmax(logits / temp)``."""
+    p = softmax(logits / temp)
+    n = logits.shape[0]
+    return float(-np.mean(np.log(p[np.arange(n), y] + 1e-12)))
+
+
+def fit_temperature(mean_logits_fit: np.ndarray, y_fit: np.ndarray) -> float:
+    """Scalar temperature minimising NLL on the fit fold (invariant 7); ``1.0`` fallback.
+
+    Standard temperature scaling (Guo et al. 2017), fit on a fold disjoint from calibration so the
+    resulting nonconformity score stays exchangeable with the calibration set.
+    """
+    logits = np.asarray(mean_logits_fit, dtype=float)
+    y = np.asarray(y_fit).astype(int)
+    if logits.shape[0] == 0 or len(np.unique(y)) < 2:
+        return 1.0
+    res = minimize_scalar(_nll, bounds=(0.05, 20.0), args=(logits, y), method="bounded")
+    return float(res.x) if res.success else 1.0
+
+
+def temp_msp_score(mean_logits: np.ndarray, temp: float) -> np.ndarray:
+    """MSP after temperature scaling: ``1 - max softmax(logits / temp)``."""
+    return 1.0 - softmax(np.asarray(mean_logits, dtype=float) / temp).max(axis=-1)
diff --git a/src/edc/eval/evaluate.py b/src/edc/eval/evaluate.py
index aac4b18..5e6662a 100644
--- a/src/edc/eval/evaluate.py
+++ b/src/edc/eval/evaluate.py
@@ -1,14 +1,282 @@
-"""End-to-end evaluation orchestration. [Phase 3/4]
+"""End-to-end selective-prediction evaluation. [Phase 3]
 
-Trains (or loads) a reasoner, runs K-restart inference on the eval + calibration splits,
-extracts geometry features, computes every nonconformity score (geometry vs the energy/MSP/
-MC-dropout/ensemble baselines), calibrates the conformal layer, and returns the metrics dict
-that ``experiments/run_experiment.py`` writes to the ledger. Phase-1 ``edc.cli smoke`` covers
-the train -> infer -> accuracy path; this module adds the geometry + conformal evaluation.
+Trains a reasoner, runs K-restart inference on three **disjoint** folds (fit / calib / test,
+invariant 7), extracts geometry features, computes every nonconformity score (the learned
+geometry mapper, the ``1 - rho_basin`` fallback, and the raw-energy baselines that geometry must
+beat — invariant 8), and returns the metrics dict that ``experiments/run_experiment.py`` writes to
+the ledger. The centrepiece is the paired-bootstrap ΔAURC(raw energy − geometry) falsification
+test and the Learn-then-Test abstention guarantee.
+
+Plain NumPy over JAX arrays pulled to host (invariant 1): the only JAX is inside the reused
+``train`` / ``restarts.solve`` / curvature core.
 """
 
 from __future__ import annotations
 
+import jax
+import jax.numpy as jnp
+import numpy as np
+
+from edc.conformal import ltt
+from edc.conformal import nonconformity as nc
+from edc.conformal.selective import ABSTAIN, selective_predict
+from edc.eval import baselines, metrics
+from edc.geometry.features import geometry_features
+from edc.inference import restarts
+from edc.seeding import numpy_rng, root_key
+from edc.train.train_ebm import train
+
+# alpha levels swept for the coverage-validity figure (F3): achieved selective risk must sit
+# at or below the nominal target for a valid distribution-free guarantee.
+_ALPHA_GRID = (0.02, 0.05, 0.1, 0.15, 0.2, 0.3)
+_COVERAGE_GRID = np.linspace(0.02, 1.0, 50)  # F2 risk-coverage sampling grid
+
+
+def _fold(fns, params, cfg, task, split: str, data_stream: int, key) -> dict:
+    """Run inference on one data fold and return its geometry features + baselines + correctness."""
+    rng = numpy_rng(cfg.run.seed, 20, data_stream)
+    n = cfg.conformal.n_calib if data_stream == 1 else cfg.eval.n_eval
+    batch = task.sample(rng, n, split=split)
+
+    k_solve, k_geom = jax.random.split(key)
+    traj = restarts.solve(fns, params, jnp.asarray(batch.x), cfg, k_solve)
+    feats, names = geometry_features(traj, fns, params, k_geom, grad_tol=cfg.inference.grad_tol)
+
+    pred, _ = restarts.best_of_n_energy(traj)
+    correct = np.asarray(task.evaluate(np.asarray(pred), batch.y)).astype(bool)
+
+    terminal = np.asarray(traj.terminal_energy)  # (B, K)
+    return {
+        "features": np.asarray(feats),
+        "names": names,
+        "correct": correct,
+        "accuracy": float(correct.mean()),
+        "y": np.asarray(batch.y),
+        # Raw-energy baselines (EBT-style scalar-energy confidence) — higher energy = less
+        # confident = higher nonconformity score. These are the signals geometry must beat.
+        "energy_min": terminal.min(axis=1),
+        "energy_mean": terminal.mean(axis=1),
+        "energy_std": terminal.std(axis=1),
+        # Mean decoder logits over restarts -> softmax-confidence baselines (MSP/entropy/temp).
+        "mean_logits": np.asarray(traj.logits).mean(axis=1),   # (B, C)
+        "rho": np.asarray(feats)[:, names.index("basin/rho")],
+    }
+
+
+def _resample_curve(scores, correct) -> dict:
+    """Risk-coverage curve resampled onto ``_COVERAGE_GRID`` (compact, ledger-friendly, F2)."""
+    coverage, risk = metrics.risk_coverage_curve(scores, correct)
+    risk_grid = np.interp(_COVERAGE_GRID, coverage, risk)
+    return {"coverage": _COVERAGE_GRID.tolist(), "risk": risk_grid.tolist()}
+
+
+def _feature_diagnostics(features, names, correct, n_bins: int = 20) -> dict:
+    """Per-feature correct-vs-incorrect separation (AUROC) + histograms — the F5 mechanism data.
+
+    Compact and fully regenerable: per feature, the single-feature AUROC and 20-bin histograms of
+    the correct and incorrect subsets over the feature's observed range.
+    """
+    features = np.asarray(features, dtype=float)
+    correct = np.asarray(correct).astype(bool)
+    auroc, hist = {}, {}
+    for j, name in enumerate(names):
+        col = features[:, j]
+        auroc[name] = metrics.single_feature_auroc(col, correct)
+        lo, hi = float(col.min()), float(col.max())
+        if hi <= lo:
+            hi = lo + 1.0                                   # degenerate/constant feature
+        edges = np.linspace(lo, hi, n_bins + 1)
+        c_counts, _ = np.histogram(col[correct], bins=edges)
+        i_counts, _ = np.histogram(col[~correct], bins=edges)
+        hist[name] = {"edges": edges.tolist(),
+                      "correct_counts": c_counts.tolist(),
+                      "incorrect_counts": i_counts.tolist()}
+    return {"names": list(names), "auroc": auroc, "hist": hist}
+
+
+_FEATURE_GROUPS = ("basin", "energy", "curv", "dynamics")
+
+
+def _feature_ablation(fit_feats, fit_correct, test_feats, test_correct, names) -> dict:
+    """Leave-one-group-out and group-only AURC — which geometry features drive the win (T2/A1).
+
+    Refits the logistic mapper on feature subsets (fit fold, invariant 7) and scores the test fold.
+    ``drop_energy`` is the key one: does geometry *without* energy columns still beat raw energy?
+    """
+    fit_feats = np.asarray(fit_feats, dtype=float)
+    test_feats = np.asarray(test_feats, dtype=float)
+    groups = {g: [j for j, n in enumerate(names) if n.split("/")[0] == g] for g in _FEATURE_GROUPS}
+
+    def aurc_on(cols: list[int]) -> float:
+        if not cols:
+            return 0.5
+        m = nc.fit_mapper(fit_feats[:, cols], fit_correct)
+        return metrics.aurc(nc.score(m, test_feats[:, cols]), test_correct)
+
+    out = {"full": aurc_on(list(range(len(names))))}
+    for g, cols in groups.items():
+        out[f"drop_{g}"] = aurc_on([j for j in range(len(names)) if j not in cols])
+        out[f"{g}_only"] = aurc_on(cols)
+    return out
+
+
+def evaluate(cfg, task, include_ood: bool = True) -> dict:
+    """Train, calibrate, and score geometry vs raw energy on disjoint folds. Returns a metrics dict.
+
+    Folds: ``fit`` (train the nonconformity mapper), ``calib`` (choose the LTT threshold), ``test``
+    (report AURC, ΔAURC, and the abstention guarantee). Disjoint and independently seeded so the
+    conformal guarantees are valid (invariants 7). ΔAURC and the guarantee are read on ``test``;
+    the mapper never sees it.
+
+    ``include_ood=False`` skips the OOD fold (and its ``accuracy_ood``/``aurc_ood`` metrics), one
+    quarter of the inference — used by the K-sweep, which only needs the ID falsification numbers.
+    """
+    params, fns, history = train(cfg, task)
+
+    key = root_key(cfg.run.seed)
+    k_fit, k_cal, k_test, k_ood = jax.random.split(key, 4)
+    fit = _fold(fns, params, cfg, task, "id", 0, k_fit)
+    cal = _fold(fns, params, cfg, task, "id", 1, k_cal)
+    test = _fold(fns, params, cfg, task, "id", 2, k_test)
+    ood = _fold(fns, params, cfg, task, "ood", 3, k_ood) if include_ood else None
+
+    # --- Fit geometry mapper + temperature on the FIT fold only (invariant 7) ------------------
+    mapper = nc.fit_mapper(fit["features"], fit["correct"])
+    temperature = baselines.fit_temperature(fit["mean_logits"], fit["y"])
+
+    def scores_for(fold: dict) -> dict:
+        """All nonconformity scores on a fold. Low = confident; answer iff score <= threshold."""
+        return {
+            "geometry": nc.score(mapper, fold["features"]),
+            "rho_basin": nc.rho_basin_score(fold["rho"]),
+            "energy_min": fold["energy_min"],
+            "energy_mean": fold["energy_mean"],
+            "energy_std": fold["energy_std"],
+            # standard softmax-confidence baselines (Phase 4e)
+            "msp": baselines.msp_score(fold["mean_logits"]),
+            "temp_msp": baselines.temp_msp_score(fold["mean_logits"], temperature),
+            "entropy": baselines.entropy_score(fold["mean_logits"]),
+        }
+
+    test_scores = scores_for(test)
+    cal_scores = scores_for(cal)
+    correct = test["correct"]
+
+    # --- AURC per score + falsification ΔAURC vs (a) raw energy [invariant 8] and (b) the best of
+    #     ALL baselines (energy + softmax-confidence), the stronger claim -----------------------
+    aurc = {name: metrics.aurc(s, correct) for name, s in test_scores.items()}
+    energy_names = ("energy_min", "energy_mean", "energy_std")
+    baseline_names = (*energy_names, "msp", "temp_msp", "entropy")  # everything geometry must beat
+    best_energy = min(energy_names, key=lambda k: aurc[k])
+    best_baseline = min(baseline_names, key=lambda k: aurc[k])
+
+    n_boot = 2000
+    d_min, lo_min, hi_min = metrics.paired_bootstrap_delta_aurc(
+        test_scores["energy_min"], test_scores["geometry"], correct, n_boot, cfg.run.seed)
+    d_best, lo_best, hi_best = metrics.paired_bootstrap_delta_aurc(
+        test_scores[best_energy], test_scores["geometry"], correct, n_boot, cfg.run.seed)
+    d_bl, lo_bl, hi_bl = metrics.paired_bootstrap_delta_aurc(
+        test_scores[best_baseline], test_scores["geometry"], correct, n_boot, cfg.run.seed)
+    geometry_wins = bool(d_best > 0 and lo_best > 0)
+    geometry_wins_vs_baseline = bool(d_bl > 0 and lo_bl > 0)
+
+    # --- LTT abstention guarantee at the configured (alpha, delta) ------------------------------
+    alpha, delta = cfg.conformal.alpha, cfg.conformal.delta
+    out = ltt.calibrate(cal_scores["geometry"], cal["correct"], alpha, delta)
+    ltt_block = _ltt_report(out["lambda_hat"], test_scores["geometry"], correct, alpha, delta)
+
+    # --- Coverage-validity sweep (F3: ID) + OOD stress (F6) -------------------------------------
+    # The LTT threshold is always calibrated on the ID calib fold; F6 applies that same threshold
+    # to the OOD test fold, where exchangeability is broken, to exhibit the guarantee failing.
+    ood_geo = scores_for(ood)["geometry"] if include_ood else None
+    ood_correct = ood["correct"] if include_ood else None
+    validity = {"alpha_grid": [], "target": [], "achieved_risk": [], "coverage": []}
+    ood_validity = ({"target": [], "id_risk": [], "ood_risk": [], "id_coverage": [],
+                     "ood_coverage": []} if include_ood else None)
+    for a in _ALPHA_GRID:
+        lam = ltt.calibrate(cal_scores["geometry"], cal["correct"], a, delta)["lambda_hat"]
+        r_sel, cov = _selective_risk_coverage(lam, test_scores["geometry"], correct)
+        validity["alpha_grid"].append(a)
+        validity["target"].append(a)
+        validity["achieved_risk"].append(r_sel)
+        validity["coverage"].append(cov)
+        if include_ood:
+            r_ood, cov_ood = _selective_risk_coverage(lam, ood_geo, ood_correct)
+            ood_validity["target"].append(a)
+            ood_validity["id_risk"].append(r_sel)
+            ood_validity["ood_risk"].append(r_ood)
+            ood_validity["id_coverage"].append(cov)
+            ood_validity["ood_coverage"].append(cov_ood)
+
+    result = {
+        "n_fit": int(fit["correct"].shape[0]),
+        "n_calib": int(cal["correct"].shape[0]),
+        "n_test": int(test["correct"].shape[0]),
+        "k_restarts": int(cfg.inference.k_restarts),
+        "accuracy_id": test["accuracy"],
+        "base_error": float((~correct).mean()),
+        "final_train_loss": float(history["epochs"][-1]["loss"]),
+        "feature_names": test["names"],
+        "aurc": aurc,
+        "temperature": temperature,
+        "best_energy_baseline": best_energy,
+        "best_baseline": best_baseline,
+        "delta_aurc_vs_energy_min": [d_min, lo_min, hi_min],
+        "delta_aurc_vs_best_energy": [d_best, lo_best, hi_best],
+        "delta_aurc_vs_best_baseline": [d_bl, lo_bl, hi_bl],
+        "geometry_wins": geometry_wins,
+        "geometry_wins_vs_baseline": geometry_wins_vs_baseline,
+        "risk_coverage": {name: _resample_curve(s, correct) for name, s in test_scores.items()},
+        "ltt": ltt_block,
+        "coverage_validity": validity,
+        "feature_diagnostics": _feature_diagnostics(test["features"], test["names"], correct),
+        "feature_ablation": _feature_ablation(
+            fit["features"], fit["correct"], test["features"], correct, test["names"]),
+        "ece_geometry": metrics.ece(1.0 - test_scores["geometry"], correct),
+    }
+    if include_ood:
+        result["accuracy_ood"] = ood["accuracy"]
+        result["aurc_ood"] = {n: metrics.aurc(s, ood["correct"])
+                              for n, s in scores_for(ood).items()}
+        # F6: the ID-calibrated threshold at the configured alpha, evaluated on OOD (expect break).
+        r_ood, cov_ood = _selective_risk_coverage(out["lambda_hat"], ood_geo, ood_correct)
+        result["ood_ltt"] = {
+            "alpha": alpha,
+            "lambda_hat": out["lambda_hat"],
+            "selective_risk": r_ood,
+            "coverage": cov_ood,
+            "risk_within_budget": bool(cov_ood == 0 or r_ood <= alpha),
+        }
+        result["ood_validity"] = ood_validity
+    return result
+
+
+def _selective_risk_coverage(lam, scores, correct):
+    """``(selective_risk, coverage)`` at threshold ``lam`` on a fold; abstain-everything if None."""
+    if lam is None:
+        return 0.0, 0.0  # abstain on everything -> no answered errors, zero coverage
+    scores = np.asarray(scores, dtype=float)
+    answered = scores <= lam
+    cov = float(answered.mean())
+    if not answered.any():
+        return 0.0, 0.0
+    r_sel = float((~np.asarray(correct).astype(bool)[answered]).mean())
+    return r_sel, cov
+
 
-def evaluate(cfg, task):
-    raise NotImplementedError("Phase 3/4: full geometry + conformal evaluation pipeline.")
+def _ltt_report(lam, scores, correct, alpha, delta) -> dict:
+    """Selective-prediction metrics at the LTT-chosen threshold on the test fold."""
+    r_sel, cov = _selective_risk_coverage(lam, scores, correct)
+    answers = selective_predict(np.zeros_like(np.asarray(correct), dtype=int),
+                                scores, lam if lam is not None else float("-inf"))
+    abstain_rate = float((answers == ABSTAIN).mean())
+    return {
+        "alpha": alpha,
+        "delta": delta,
+        "lambda_hat": lam,
+        "coverage": cov,
+        "abstain_rate": abstain_rate,
+        "selective_risk": r_sel,
+        "selective_accuracy": 1.0 - r_sel if cov > 0 else None,
+        "risk_within_budget": bool(cov == 0 or r_sel <= alpha),
+    }
diff --git a/src/edc/eval/evaluate_halting.py b/src/edc/eval/evaluate_halting.py
new file mode 100644
index 0000000..adb30e4
--- /dev/null
+++ b/src/edc/eval/evaluate_halting.py
@@ -0,0 +1,98 @@
+"""End-to-end adaptive-halting evaluation (the CRC guarantee). [Phase 4b]
+
+Trains a reasoner, records the per-step decoded answers on disjoint calib/test folds
+(``restarts.solve(record_steps=True)``), CRC-calibrates the basin-agreement halting threshold on
+calib, and reports on test: compute saved, halting risk (disagreement with the full-budget answer)
+vs the budget ``alpha``, end-task accuracy retained, and a tau-sweep for the F4 Pareto. Returns the
+JSON-safe metrics dict that ``experiments/run_halting.py`` writes to the ledger.
+
+Plain NumPy over JAX arrays pulled to host (invariant 1); the only JAX is the reused train/solve
+core. Randomness via ``edc.seeding`` (invariant 4).
+"""
+
+from __future__ import annotations
+
+import jax
+import numpy as np
+
+from edc.halting import adaptive
+from edc.inference import restarts
+from edc.seeding import numpy_rng, root_key
+from edc.train.train_ebm import train
+
+
+def _fold(fns, params, cfg, task, data_stream: int, key) -> dict:
+    """Sample a fold and run K-restart inference with per-step decoding recorded."""
+    rng = numpy_rng(cfg.run.seed, 30, data_stream)
+    n = cfg.conformal.n_calib if data_stream == 0 else cfg.eval.n_eval
+    batch = task.sample(rng, n, split="id")
+    import jax.numpy as jnp
+
+    traj = restarts.solve(fns, params, jnp.asarray(batch.x), cfg, key, record_steps=True)
+    pred_full, _ = restarts.best_of_n_energy(traj)
+    return {
+        "step_pred": np.asarray(traj.step_pred),      # (B, K, T+1)
+        "energies": np.asarray(traj.energies),        # (B, K, T+1)
+        "y": np.asarray(batch.y),
+        "pred_full": np.asarray(pred_full),           # best-of-N at full budget
+    }
+
+
+def _accuracy(pred, y) -> float:
+    return float(np.mean(np.asarray(pred) == np.asarray(y)))
+
+
+def evaluate_halting(cfg, task) -> dict:
+    """Train, CRC-calibrate the halting threshold on calib, and report the test compute/accuracy."""
+    params, fns, history = train(cfg, task)
+    key = root_key(cfg.run.seed)
+    k_cal, k_test = jax.random.split(key)
+    cal = _fold(fns, params, cfg, task, 0, k_cal)
+    test = _fold(fns, params, cfg, task, 1, k_test)
+
+    K = int(cfg.inference.k_restarts)
+    # Candidate agreement thresholds: multiples of 1/K span the achievable agreement range.
+    taus = np.round(np.linspace(1.0 / K, 1.0, K), 4)
+
+    cal_out = adaptive.calibrate(cal["step_pred"], cal["energies"], cfg.conformal.alpha, taus)
+    tau_hat = cal_out["tau_hat"]
+
+    full_acc = _accuracy(test["pred_full"], test["y"])
+
+    # tau-sweep on the TEST fold for F4: compute used, accuracy vs labels, disagreement with full.
+    sweep = {"taus": taus.tolist(), "compute_used": [], "accuracy": [], "disagreement": []}
+    for tau in taus:
+        early, compute = adaptive.halted_predictions(test["step_pred"], test["energies"], tau)
+        sweep["compute_used"].append(float(compute.mean()))
+        sweep["accuracy"].append(_accuracy(early, test["y"]))
+        sweep["disagreement"].append(float(np.mean(early != test["pred_full"])))
+
+    # The CRC-chosen operating point on test.
+    if tau_hat is not None:
+        early, compute = adaptive.halted_predictions(test["step_pred"], test["energies"], tau_hat)
+        halting_risk = float(np.mean(early != test["pred_full"]))
+        halted_acc = _accuracy(early, test["y"])
+        compute_used = float(compute.mean())
+    else:
+        halting_risk, halted_acc, compute_used = 0.0, full_acc, 1.0   # infeasible => never halt
+
+    return {
+        "k_restarts": K,
+        "steps": int(cfg.inference.steps),
+        "alpha": float(cfg.conformal.alpha),
+        "n_calib": int(cal["y"].shape[0]),
+        "n_test": int(test["y"].shape[0]),
+        "final_train_loss": float(history["epochs"][-1]["loss"]),
+        "tau_hat": tau_hat,
+        "compute_used": compute_used,
+        "compute_saved": 1.0 - compute_used,
+        "halting_risk": halting_risk,
+        "risk_within_budget": bool(halting_risk <= cfg.conformal.alpha),
+        "full_accuracy": full_acc,
+        "halted_accuracy": halted_acc,
+        "accuracy_drop": full_acc - halted_acc,
+        "risk_monotone_in_tau": cal_out["risk_monotone_in_tau"],
+        "calib_risk_by_tau": cal_out["risk_by_tau"],
+        "calib_compute_by_tau": cal_out["compute_by_tau"],
+        "tau_sweep": sweep,
+    }
diff --git a/src/edc/eval/metrics.py b/src/edc/eval/metrics.py
index b72935d..6cee494 100644
--- a/src/edc/eval/metrics.py
+++ b/src/edc/eval/metrics.py
@@ -8,19 +8,136 @@
 from __future__ import annotations
 
 import numpy as np
+from scipy.stats import rankdata
+
+from edc.seeding import numpy_rng
 
 
 def accuracy(pred: np.ndarray, y: np.ndarray) -> float:
     return float(np.mean(np.asarray(pred) == np.asarray(y)))
 
 
+def single_feature_auroc(feature: np.ndarray, correct: np.ndarray) -> float:
+    """AUROC of a single scalar ``feature`` separating correct from incorrect answers.
+
+    ``P(feature[correct] > feature[incorrect])`` via the Mann-Whitney rank statistic (ties
+    count as 0.5). A value near 0.5 means no separation; far from 0.5 (in either direction —
+    the feature's sign is not assumed) means the feature carries correctness signal. This is a
+    Phase-2 *diagnostic*; the Phase-3 selective-prediction metric is ``aurc`` (still stubbed).
+    Returns 0.5 when either class is empty (undefined).
+    """
+    feature = np.asarray(feature, dtype=float)
+    correct = np.asarray(correct).astype(bool)
+    n_pos = int(correct.sum())
+    n_neg = int((~correct).sum())
+    if n_pos == 0 or n_neg == 0:
+        return 0.5
+    ranks = rankdata(feature)  # average ranks, ties shared
+    return float((ranks[correct].sum() - n_pos * (n_pos + 1) / 2) / (n_pos * n_neg))
+
+
+def _errors_by_confidence(scores: np.ndarray, correct: np.ndarray) -> np.ndarray:
+    """Per-point error (``~correct``) reordered most-confident first, tie-robust.
+
+    Convention: the nonconformity ``score`` is LOW when confident, so an ascending sort puts the
+    answers we would keep first. **Tied scores share their group-mean error** — the expected risk
+    under random tie-breaking — so the curve (and AURC) does not depend on incidental input order
+    within a tie group. This matters for scores with few distinct values (e.g. ``rho_basin`` over
+    K restarts) so the geometry-vs-energy comparison stays fair.
+    """
+    scores = np.asarray(scores, dtype=float)
+    correct = np.asarray(correct).astype(bool)
+    if scores.shape[0] != correct.shape[0]:
+        raise ValueError("scores and correct must have the same length")
+    if scores.shape[0] == 0:
+        raise ValueError("need at least one point")
+    order = np.argsort(scores, kind="stable")
+    err = (~correct[order]).astype(float)
+    # Replace each point's error by its tie-group mean (groups are contiguous after the sort).
+    _, inv = np.unique(scores[order], return_inverse=True)
+    inv = inv.ravel()
+    group_mean = np.bincount(inv, weights=err) / np.bincount(inv)
+    return group_mean[inv]
+
+
 def risk_coverage_curve(scores, correct):  # F2 / AURC
-    raise NotImplementedError("Phase 3: sort by score, sweep coverage, return (coverage, risk).")
+    """Return ``(coverage, risk)``, each ``(n,)``, sweeping coverage from the most-confident point.
+
+    At coverage ``c = (i+1)/n`` we answer the ``i+1`` lowest-score inputs; ``risk`` is the error
+    rate among them (``cumsum(error)/(i+1)``). ``risk[-1]`` is the base error rate (full coverage).
+    Ties in ``scores`` split across the prefix boundary — fine for the scalar AURC summary and for
+    a diagnostic curve (conformal thresholds admit whole tie-groups anyway).
+    """
+    e = _errors_by_confidence(scores, correct)
+    n = e.shape[0]
+    coverage = np.arange(1, n + 1, dtype=float) / n
+    risk = np.cumsum(e) / np.arange(1, n + 1)
+    return coverage, risk
 
 
 def aurc(scores, correct) -> float:
-    raise NotImplementedError("Phase 3: area under the risk-coverage curve.")
+    """Area under the risk-coverage curve = mean selective risk over the uniform coverage grid.
+
+    Lower is better: a ranker that pushes all errors to the least-confident tail minimises every
+    prefix risk. Equals the base error rate when scores carry no ranking signal (all ties).
+    """
+    e = _errors_by_confidence(scores, correct)
+    return float(np.mean(e.cumsum() / np.arange(1, e.shape[0] + 1)))
+
+
+def selective_accuracy_at_coverage(scores, correct, coverage: float) -> float:
+    """Selective accuracy (``1 - risk``) when answering the most-confident ``coverage`` fraction."""
+    if not 0.0 < coverage <= 1.0:
+        raise ValueError("coverage must be in (0, 1]")
+    e = _errors_by_confidence(scores, correct)
+    k = max(1, int(round(coverage * e.shape[0])))
+    return float(1.0 - e[:k].mean())
+
+
+def ece(prob_correct, correct, n_bins: int = 10) -> float:
+    """Expected calibration error of a ``p_hat(correct)`` estimate, equal-width binning.
+
+    ``sum_b (n_b/n) * |acc_b - conf_b|`` — the standard reliability-diagram summary. Used as a
+    reported calibration number alongside the distribution-free guarantees.
+    """
+    p = np.asarray(prob_correct, dtype=float)
+    y = np.asarray(correct).astype(float)
+    if p.shape[0] == 0:
+        raise ValueError("need at least one point")
+    edges = np.linspace(0.0, 1.0, n_bins + 1)
+    idx = np.clip(np.digitize(p, edges[1:-1], right=False), 0, n_bins - 1)
+    total = 0.0
+    for b in range(n_bins):
+        mask = idx == b
+        if not mask.any():
+            continue
+        total += mask.mean() * abs(y[mask].mean() - p[mask].mean())
+    return float(total)
+
+
+def paired_bootstrap_delta_aurc(scores_a, scores_b, correct, n_boot: int = 1000, seed: int = 0):
+    """Paired-bootstrap CI on ``ΔAURC = AURC(a) - AURC(b)`` — the falsification test (invariant 8).
 
+    ``a`` = raw terminal energy, ``b`` = geometry. Each bootstrap iterate resamples one index
+    vector with replacement and applies it to *both* scorers and ``correct`` (the pairing cancels
+    shared test-set variance). Returns ``(delta, lo, hi)`` with a 95% percentile CI. **Geometry
+    wins iff ``delta > 0`` and ``lo > 0``** (its AURC is lower); a CI containing 0 falsifies the
+    core claim. Randomness flows only through ``edc.seeding`` (invariant 4).
+    """
+    if n_boot <= 0:
+        raise ValueError("n_boot must be positive")
+    sa = np.asarray(scores_a, dtype=float)
+    sb = np.asarray(scores_b, dtype=float)
+    c = np.asarray(correct).astype(bool)
+    n = c.shape[0]
+    if not (sa.shape[0] == sb.shape[0] == n):
+        raise ValueError("scores_a, scores_b, correct must have equal length")
 
-def paired_bootstrap_delta_aurc(scores_a, scores_b, correct, n_boot: int = 1000):
-    raise NotImplementedError("Phase 3: bootstrap CI on AURC(a) - AURC(b); the falsification test.")
+    delta = aurc(sa, c) - aurc(sb, c)
+    rng = numpy_rng(seed, 101)  # 101 = the ΔAURC bootstrap substream
+    deltas = np.empty(n_boot, dtype=float)
+    for b in range(n_boot):
+        idx = rng.integers(0, n, size=n)
+        deltas[b] = aurc(sa[idx], c[idx]) - aurc(sb[idx], c[idx])
+    lo, hi = np.quantile(deltas, [0.025, 0.975])
+    return float(delta), float(lo), float(hi)
diff --git a/src/edc/geometry/basin.py b/src/edc/geometry/basin.py
index 8d400af..68b204c 100644
--- a/src/edc/geometry/basin.py
+++ b/src/edc/geometry/basin.py
@@ -2,11 +2,50 @@
 
 Do independent restarts land in the same answer / same latent basin? Plurality fraction
 ``rho_basin``, decoded-answer entropy, and latent-cluster separation. Hypothesis: high
-agreement => reliable. Must be computable under ``vmap`` over restarts.
+agreement => reliable.
+
+Plain NumPy (invariant 1), fixed-shape and vectorized over the K axis.
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+from edc.inference.trajectory import TrajectoryRecord
+
+
+def basin_features(traj: TrajectoryRecord) -> tuple[np.ndarray, list[str]]:
+    """``(B, 3)`` basin-agreement features plus names.
+
+    * ``basin/rho``        — plurality fraction of decoded answers across restarts (1 => all
+      restarts agree).
+    * ``basin/entropy``    — Shannon entropy (nats) of the decoded-answer distribution
+      (0 => unanimous, higher => scattered).
+    * ``basin/dispersion`` — mean pairwise Euclidean distance between restart endpoints
+      ``z*`` (small => restarts converge to one basin).
+    """
+    pred = np.asarray(traj.pred)                                # (B, K)
+    z = np.asarray(traj.z_star, dtype=float)                   # (B, K, d)
+    B, K = pred.shape
+    n_classes = int(np.asarray(traj.logits).shape[-1])
+
+    # Per-input class counts across restarts -> (B, C).
+    counts = np.zeros((B, n_classes), dtype=float)
+    np.add.at(counts, (np.arange(B)[:, None], pred), 1.0)
+
+    rho = counts.max(axis=1) / K
+    probs = counts / K
+    with np.errstate(divide="ignore", invalid="ignore"):
+        logp = np.where(probs > 0, np.log(probs), 0.0)
+    entropy = -(probs * logp).sum(axis=1)
+
+    # Mean pairwise distance between the K endpoints, per input.
+    diff = z[:, :, None, :] - z[:, None, :, :]                  # (B, K, K, d)
+    dist = np.linalg.norm(diff, axis=-1)                       # (B, K, K)
+    if K > 1:
+        dispersion = dist.sum(axis=(1, 2)) / (K * (K - 1))     # off-diagonal mean
+    else:
+        dispersion = np.zeros(B)
 
-def basin_features(traj):
-    raise NotImplementedError("Phase 2: rho_basin, answer entropy, latent cluster separation.")
+    feats = np.stack([rho, entropy, dispersion], axis=1)
+    return feats, ["basin/rho", "basin/entropy", "basin/dispersion"]
diff --git a/src/edc/geometry/curvature.py b/src/edc/geometry/curvature.py
index e5bf073..4a3432d 100644
--- a/src/edc/geometry/curvature.py
+++ b/src/edc/geometry/curvature.py
@@ -28,7 +28,7 @@ def hvp(f, z, v):
 
 def lambda_max(f, z, key, iters: int = 12) -> jnp.ndarray:
     """Top Hessian eigenvalue at ``z`` via power iteration (sharpness of the basin)."""
-    v = jax.random.normal(key, z.shape)
+    v = jax.random.normal(key, z.shape, dtype=z.dtype)
     v = v / (jnp.linalg.norm(v) + 1e-12)
     lam = jnp.array(0.0)
     for _ in range(iters):
@@ -57,3 +57,25 @@ def f(z):
         return jnp.squeeze(energy(params, h_x_row[None, :], z[None, :]))
 
     return f
+
+
+def batched_curvature(energy, params, contexts, zs, key, iters: int = 12, n_probes: int = 8):
+    """Per-particle ``(lambda_max, tr(H))`` for a flat batch of ``N`` particles.
+
+    This is the JAX-side entry point the (plain-NumPy) Phase-2 feature assembler calls, so all
+    per-input Hessian-vector products and their ``vmap`` over restarts stay confined to the JAX
+    core (invariant 1). ``contexts`` is ``(N, dc)`` and ``zs`` is ``(N, d)`` — flattened
+    ``B*K`` particles exactly as ``inference.restarts.solve`` lays them out. Each particle gets
+    its own PRNG stream (one key for power iteration, one for the Hutchinson probes) so the
+    estimate is a pure function of ``key``. Returns ``(lmax (N,), trace (N,))``.
+    """
+    n = zs.shape[0]
+    keys = jax.random.split(key, n)
+
+    def one(h_row, z_row, k):
+        k_lam, k_tr = jax.random.split(k)
+        f = energy_at(energy, params, h_row)
+        return (lambda_max(f, z_row, k_lam, iters=iters),
+                hutchinson_trace(f, z_row, k_tr, n_probes=n_probes))
+
+    return jax.vmap(one)(contexts, zs, keys)
diff --git a/src/edc/geometry/dynamics.py b/src/edc/geometry/dynamics.py
index 972f0ef..a16997e 100644
--- a/src/edc/geometry/dynamics.py
+++ b/src/edc/geometry/dynamics.py
@@ -1,14 +1,44 @@
 """Descent-dynamics features from the energy/gradient trajectories. [Phase 2]
 
 Steps-to-convergence (first step with ``||grad|| < grad_tol``), monotonicity fraction
-(fraction of steps with ΔE < 0), area under the energy curve, and terminal gradient norm
-(residual stationarity). Slow, non-monotone, high-residual descents => hypothesised unreliable.
+(fraction of steps with ΔE < 0), total energy drop, and terminal gradient norm (residual
+stationarity). Slow, non-monotone, high-residual descents => hypothesised unreliable.
+
+Computed per restart from the ``(B, K, ...)`` curves, then averaged over the K restarts to a
+per-input ``(B, ...)`` value. Plain NumPy (invariant 1).
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+from edc.inference.trajectory import TrajectoryRecord
+
+
+def dynamics_features(traj: TrajectoryRecord, grad_tol: float) -> tuple[np.ndarray, list[str]]:
+    """``(B, 4)`` descent-dynamics features (mean over restarts) plus names."""
+    energies = np.asarray(traj.energies, dtype=float)          # (B, K, T+1)
+    gnorms = np.asarray(traj.grad_norms, dtype=float)          # (B, K, T)
+    T = gnorms.shape[-1]
+
+    # Steps-to-converge: first step under grad_tol, else T. Normalise by T to keep it in [0, 1].
+    below = gnorms < grad_tol                                  # (B, K, T)
+    any_below = below.any(axis=-1)
+    first = below.argmax(axis=-1)                              # 0 when none are below
+    steps = np.where(any_below, first, T).astype(float) / T
+
+    # Monotonicity: fraction of steps where energy decreased.
+    de = np.diff(energies, axis=-1)                            # (B, K, T)
+    monotonic = (de < 0).mean(axis=-1)
+
+    # Total energy drop from the initial point to the endpoint.
+    drop = energies[..., 0] - energies[..., -1]               # (B, K)
+
+    # Terminal residual: last-step gradient norm (stationarity proxy at ~z*).
+    residual = np.asarray(traj.terminal_grad_norm, dtype=float)  # (B, K)
 
-def dynamics_features(traj, grad_tol: float):
-    raise NotImplementedError(
-        "Phase 2: steps-to-converge, monotonicity fraction, AUC, terminal residual."
+    feats = np.stack(
+        [steps.mean(axis=1), monotonic.mean(axis=1), drop.mean(axis=1), residual.mean(axis=1)],
+        axis=1,
     )
+    return feats, ["dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"]
diff --git a/src/edc/geometry/energy_stats.py b/src/edc/geometry/energy_stats.py
index 24a1a32..e9ad8cd 100644
--- a/src/edc/geometry/energy_stats.py
+++ b/src/edc/geometry/energy_stats.py
@@ -3,10 +3,19 @@
 Mean, min, and spread of the terminal energy over the K restarts. These reproduce the
 **scalar-energy / best-of-N** signal that Energy-Based Transformers use — here they are just
 features, and the falsification test is whether the geometry features beat them (ΔAURC).
+
+Plain NumPy (invariant 1): JAX stays in ``energy/``, ``inference/`` and ``curvature.py``.
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+from edc.inference.trajectory import TrajectoryRecord
+
 
-def energy_features(traj):
-    raise NotImplementedError("Phase 2: mean/min/std of terminal energy across restarts.")
+def energy_features(traj: TrajectoryRecord) -> tuple[np.ndarray, list[str]]:
+    """``(B, 3)`` mean/min/std of terminal energy across restarts, plus feature names."""
+    e = np.asarray(traj.terminal_energy, dtype=float)          # (B, K)
+    feats = np.stack([e.mean(axis=1), e.min(axis=1), e.std(axis=1)], axis=1)
+    return feats, ["energy/mean", "energy/min", "energy/std"]
diff --git a/src/edc/geometry/features.py b/src/edc/geometry/features.py
index fbbc657..85e7812 100644
--- a/src/edc/geometry/features.py
+++ b/src/edc/geometry/features.py
@@ -3,15 +3,61 @@
 Combines: basin agreement (``basin``), terminal-energy statistics (``energy_stats``),
 curvature (``curvature``), and descent dynamics (``dynamics``) into the nonconformity
 feature vector that Phase-3 conformal calibration consumes. This is the paper's contribution.
+
+Plain NumPy assembler (invariant 1): the only JAX touched here is behind
+``curvature.batched_curvature``, whose output is immediately pulled back to NumPy.
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+from edc.geometry import curvature
+from edc.geometry.basin import basin_features
+from edc.geometry.dynamics import dynamics_features
+from edc.geometry.energy_stats import energy_features
 from edc.inference.trajectory import TrajectoryRecord
 
 
-def geometry_features(traj: TrajectoryRecord, fns, params, key):
-    """Return ``(B, n_features)`` array + feature-name list. Implemented in Phase 2."""
-    raise NotImplementedError(
-        "Phase 2: assemble basin + energy_stats + curvature + dynamics features."
+def _curvature_features(traj, fns, params, key) -> tuple[np.ndarray, list[str]]:
+    """``(B, 4)`` curvature summaries: mean over restarts and value at the min-energy restart."""
+    z = np.asarray(traj.z_star, dtype=float)                   # (B, K, d)
+    B, K, d = z.shape
+    h_x = np.asarray(traj.h_x, dtype=float)                    # (B, dc)
+
+    # Flatten to N = B*K particles exactly as inference.restarts lays them out (particle
+    # (b, k) at index b*K + k), tiling the shared per-input context to match.
+    contexts = np.repeat(h_x, K, axis=0)                       # (N, dc)
+    zs = z.reshape(B * K, d)                                   # (N, d)
+
+    lmax, trace = curvature.batched_curvature(fns.energy, params, contexts, zs, key)
+    lmax = np.asarray(lmax, dtype=float).reshape(B, K)
+    trace = np.asarray(trace, dtype=float).reshape(B, K)
+
+    # "Best" restart = lowest terminal energy (the best-of-N answer we would report).
+    best = np.asarray(traj.terminal_energy, dtype=float).argmin(axis=1)   # (B,)
+    rows = np.arange(B)
+    feats = np.stack(
+        [lmax.mean(axis=1), lmax[rows, best], trace.mean(axis=1), trace[rows, best]],
+        axis=1,
     )
+    return feats, ["curv/lmax_mean", "curv/lmax_best", "curv/trace_mean", "curv/trace_best"]
+
+
+def geometry_features(
+    traj: TrajectoryRecord, fns, params, key, grad_tol: float = 1e-3
+) -> tuple[np.ndarray, list[str]]:
+    """Return ``((B, n_features) array, feature-name list)``.
+
+    Concatenates the four feature groups (basin, energy stats, curvature, dynamics) in a
+    fixed order. ``key`` seeds the curvature HVP estimators; ``grad_tol`` sets the
+    convergence threshold for the dynamics ``steps`` feature (pass ``cfg.inference.grad_tol``).
+    """
+    b_feats, b_names = basin_features(traj)
+    e_feats, e_names = energy_features(traj)
+    c_feats, c_names = _curvature_features(traj, fns, params, key)
+    d_feats, d_names = dynamics_features(traj, grad_tol)
+
+    feats = np.concatenate([b_feats, e_feats, c_feats, d_feats], axis=1)
+    names = [*b_names, *e_names, *c_names, *d_names]
+    return feats, names
diff --git a/src/edc/halting/adaptive.py b/src/edc/halting/adaptive.py
index ff17c77..88ba0fe 100644
--- a/src/edc/halting/adaptive.py
+++ b/src/edc/halting/adaptive.py
@@ -1,13 +1,129 @@
-"""Geometry-driven adaptive halting policy. [Phase 3]
+"""Geometry-driven adaptive halting policy. [Phase 4b]
 
-Stop the descent as soon as a geometry-based certificate (e.g. basin agreement / curvature
-crossing a CRC-calibrated threshold) says the answer is settled, saving compute versus a fixed
-step budget — with a guaranteed error budget from ``edc.conformal.crc``. Produces the
-compute-vs-accuracy Pareto (figure F4).
+Stop the K-restart descent as soon as **basin agreement** — the plurality fraction of decoded
+answers across restarts — crosses a threshold ``tau``, saving compute versus the fixed step budget.
+The halting loss ``L(tau) = 1[early-stopped best-of-N answer != full-budget best-of-N answer]`` is
+bounded in [0,1]; Conformal Risk Control (``edc.conformal.crc``) picks the most compute-saving
+``tau`` with ``E[L] <= alpha`` in finite samples. Produces the compute-vs-accuracy Pareto (F4).
+
+Plain NumPy over ``TrajectoryRecord.step_pred`` / ``energies`` (invariant 1); the reparametrisation
+``lambda = 1 - tau`` orients the sweep so CRC's "largest admissible lambda = most compute saved"
+convention selects the most aggressive safe halt.
 """
 
 from __future__ import annotations
 
+import numpy as np
+
+from edc.conformal import crc
+
+
+def per_step_agreement(step_pred) -> np.ndarray:
+    """``(B, T+1)`` basin agreement at each descent step: plurality fraction across the K restarts.
+
+    ``step_pred`` is ``(B, K, T+1)`` decoded classes (from ``restarts.solve(record_steps=True)``).
+    1.0 => all restarts decode the same answer at that step; low => scattered. Same plurality
+    definition as ``geometry.basin.basin_features``, vectorised over the step axis.
+    """
+    step_pred = np.asarray(step_pred)
+    B, K, T1 = step_pred.shape
+    n_classes = int(step_pred.max()) + 1
+    counts = np.zeros((B, T1, n_classes), dtype=float)
+    b_idx = np.arange(B)[:, None, None]
+    t_idx = np.arange(T1)[None, None, :]
+    np.add.at(counts, (b_idx, t_idx, step_pred), 1.0)         # (B, T+1, C)
+    return counts.max(axis=2) / K                            # (B, T+1)
+
+
+def halting_policy(agreement_row, tau: float) -> int:
+    """First step index whose agreement ``>= tau`` for one input; else the full budget ``T``.
+
+    ``agreement_row`` is ``(T+1,)``. Returns an index in ``[0, T]`` (T = last = full budget).
+    """
+    a = np.asarray(agreement_row, dtype=float)
+    hits = np.nonzero(a >= tau)[0]
+    return int(hits[0]) if hits.size else a.shape[0] - 1
+
+
+def _stop_steps(agreement: np.ndarray, tau: float) -> np.ndarray:
+    """Vectorised ``halting_policy`` over inputs: ``(B,)`` stop indices at threshold ``tau``."""
+    mask = agreement >= tau                                   # (B, T+1)
+    T = agreement.shape[1] - 1
+    return np.where(mask.any(axis=1), mask.argmax(axis=1), T)
+
+
+def _best_of_n_at(step_pred: np.ndarray, energies: np.ndarray, stop_t: np.ndarray) -> np.ndarray:
+    """Best-of-N (lowest-energy restart) decoded answer at per-input step ``stop_t`` -> ``(B,)``."""
+    B, K, _ = step_pred.shape
+    rows = np.arange(B)
+    e_at = energies[rows[:, None], np.arange(K)[None, :], stop_t[:, None]]   # (B, K)
+    k_best = e_at.argmin(axis=1)                                             # (B,)
+    return step_pred[rows, k_best, stop_t]                                   # (B,)
+
+
+def halted_predictions(step_pred, energies, tau: float):
+    """``(early_pred (B,), compute_used (B,))`` under the halting policy at threshold ``tau``.
+
+    The reported answer is best-of-N (lowest energy) at the agreement-triggered stop step; compute
+    used is the fraction of the step budget consumed. Used by the evaluator (accuracy vs labels) and
+    the F4 Pareto.
+    """
+    step_pred = np.asarray(step_pred)
+    energies = np.asarray(energies, dtype=float)
+    T = step_pred.shape[2] - 1
+    stop_t = _stop_steps(per_step_agreement(step_pred), tau)
+    return _best_of_n_at(step_pred, energies, stop_t), stop_t / T
+
+
+def halting_losses(step_pred, energies, taus):
+    """Per-input halting loss and compute fraction for each ``tau``.
+
+    Returns ``(loss, compute_used)`` each ``(B, n_tau)``: ``loss = 1[early answer != full answer]``
+    where the early answer is best-of-N at the agreement-triggered stop step and the full answer is
+    best-of-N at the final step; ``compute_used = stop_step / T`` (fraction of the step budget).
+    """
+    step_pred = np.asarray(step_pred)
+    energies = np.asarray(energies, dtype=float)
+    B, K, T1 = step_pred.shape
+    T = T1 - 1
+    agreement = per_step_agreement(step_pred)
+    full = _best_of_n_at(step_pred, energies, np.full(B, T))     # best-of-N at the final step
+
+    taus = np.asarray(taus, dtype=float)
+    loss = np.empty((B, taus.shape[0]))
+    compute = np.empty((B, taus.shape[0]))
+    for j, tau in enumerate(taus):
+        stop_t = _stop_steps(agreement, tau)
+        early = _best_of_n_at(step_pred, energies, stop_t)
+        loss[:, j] = (early != full).astype(float)
+        compute[:, j] = stop_t / T
+    return loss, compute
+
+
+def calibrate(step_pred_cal, energies_cal, alpha: float, taus) -> dict:
+    """CRC-calibrate the halting threshold ``tau`` at error budget ``alpha`` on a calibration fold.
+
+    Reparametrises ``lambda = 1 - tau`` so ascending ``lambda`` means a lower agreement bar => stop
+    earlier => weakly more disagreement and more compute saved, matching ``crc.calibrate``'s
+    "largest admissible lambda = most compute saved". Returns ``tau_hat`` (most aggressive safe
+    threshold, or ``None`` if the budget is infeasible) plus the per-tau risk/compute curves and a
+    monotonicity flag (CRC assumes the risk is monotone in the threshold).
+    """
+    taus = np.asarray(taus, dtype=float)
+    loss, compute = halting_losses(step_pred_cal, energies_cal, taus)
+    risk_by_tau = loss.mean(axis=0)
+    compute_by_tau = compute.mean(axis=0)
+
+    lambda_hat = crc.calibrate(loss, alpha, lambdas=1.0 - taus)
+    tau_hat = None if lambda_hat is None else float(1.0 - lambda_hat)
 
-def halting_policy(traj, threshold):
-    raise NotImplementedError("Phase 3: per-input stop step from geometry crossing a threshold.")
+    # Risk should be non-increasing as tau rises (later stop). Flag material violations.
+    monotone = bool(np.all(np.diff(risk_by_tau) <= 1e-9))
+    return {
+        "tau_hat": tau_hat,
+        "alpha": float(alpha),
+        "taus": taus.tolist(),
+        "risk_by_tau": risk_by_tau.tolist(),
+        "compute_by_tau": compute_by_tau.tolist(),
+        "risk_monotone_in_tau": monotone,
+    }
diff --git a/src/edc/inference/optimizer.py b/src/edc/inference/optimizer.py
index 38174ec..e028973 100644
--- a/src/edc/inference/optimizer.py
+++ b/src/edc/inference/optimizer.py
@@ -25,7 +25,8 @@ def total_energy(z):
     return jax.grad(total_energy)
 
 
-def langevin_descent(energy, params, h_x, z0, key, step_size, temperature, steps):
+def langevin_descent(energy, params, h_x, z0, key, step_size, temperature, steps,
+                     record_z: bool = False):
     """Descend ``N`` particles for ``steps`` steps.
 
     Args:
@@ -33,26 +34,46 @@ def langevin_descent(energy, params, h_x, z0, key, step_size, temperature, steps
       h_x: ``(N, context_dim)``.
       z0:  ``(N, d)`` initial latents.
       key: JAX PRNG key for the Langevin noise.
+      record_z: also return the full latent trajectory (for per-step decoding / halting).
 
-    Returns ``(z_final, energies, grad_norms)`` with
+    Returns ``(z_final, energies, grad_norms, z_traj)`` with
       ``energies``   shape ``(steps+1, N)`` (row 0 = initial),
-      ``grad_norms`` shape ``(steps, N)``   (norm of the gradient used at each step).
+      ``grad_norms`` shape ``(steps, N)``   (norm of the gradient used at each step),
+      ``z_traj``     shape ``(steps+1, N, d)`` (row 0 = ``z0``) when ``record_z`` else ``None``.
     """
     grad_fn = _per_particle_grad(energy, params, h_x)
     noise_scale = jnp.sqrt(2.0 * step_size * temperature)
 
-    def step(carry, _):
-        z, k = carry
+    def _update(z, k):
         k, sub = jax.random.split(k)
         g = grad_fn(z)                                   # (N, d)
         z_new = z - step_size * g + noise_scale * jax.random.normal(sub, z.shape)
         e_new = energy(params, h_x, z_new)               # (N,)
         gnorm = jnp.linalg.norm(g, axis=-1)              # (N,)
-        return (z_new, k), (e_new, gnorm)
+        return z_new, k, e_new, gnorm
 
     e0 = energy(params, h_x, z0)                          # (N,)
-    (z_final, _), (e_steps, gnorms) = jax.lax.scan(
-        step, (z0, key), None, length=steps
-    )
+
+    if record_z:
+        def step(carry, _):
+            z, k = carry
+            z_new, k, e_new, gnorm = _update(z, k)
+            return (z_new, k), (e_new, gnorm, z_new)
+
+        (z_final, _), (e_steps, gnorms, z_steps) = jax.lax.scan(
+            step, (z0, key), None, length=steps
+        )
+        z_traj = jnp.concatenate([z0[None], z_steps], axis=0)   # (steps+1, N, d)
+    else:
+        def step(carry, _):
+            z, k = carry
+            z_new, k, e_new, gnorm = _update(z, k)
+            return (z_new, k), (e_new, gnorm)
+
+        (z_final, _), (e_steps, gnorms) = jax.lax.scan(
+            step, (z0, key), None, length=steps
+        )
+        z_traj = None
+
     energies = jnp.concatenate([e0[None, :], e_steps], axis=0)  # (steps+1, N)
-    return z_final, energies, gnorms
+    return z_final, energies, gnorms, z_traj
diff --git a/src/edc/inference/restarts.py b/src/edc/inference/restarts.py
index abf6e35..2b7511a 100644
--- a/src/edc/inference/restarts.py
+++ b/src/edc/inference/restarts.py
@@ -18,8 +18,14 @@
 from edc.inference.trajectory import TrajectoryRecord
 
 
-def solve(fns: ReasonerFns, params, x, cfg, key, k_restarts: int | None = None) -> TrajectoryRecord:
-    """Run K-restart inference on a batch of raw inputs ``x`` (``(B, feature_dim)``)."""
+def solve(fns: ReasonerFns, params, x, cfg, key, k_restarts: int | None = None,
+          record_steps: bool = False) -> TrajectoryRecord:
+    """Run K-restart inference on a batch of raw inputs ``x`` (``(B, feature_dim)``).
+
+    ``record_steps=True`` additionally decodes the latent at *every* descent step and stores the
+    per-step class in ``TrajectoryRecord.step_pred`` (B, K, T+1) — the raw material adaptive halting
+    needs. It costs one extra decode over all ``(T+1)*N`` intermediate latents, so it is opt-in.
+    """
     K = int(k_restarts if k_restarts is not None else cfg.inference.k_restarts)
     d = fns.latent_dim
 
@@ -32,11 +38,12 @@ def solve(fns: ReasonerFns, params, x, cfg, key, k_restarts: int | None = None)
     key_init, key_lan = jax.random.split(key)
     z0 = cfg.inference.init_scale * jax.random.normal(key_init, (B * K, d))
 
-    z_final, energies, gnorms = langevin_descent(
+    z_final, energies, gnorms, z_traj = langevin_descent(
         fns.energy, params, h_rep, z0, key_lan,
         step_size=cfg.inference.step_size,
         temperature=cfg.inference.temperature,
         steps=cfg.inference.steps,
+        record_z=record_steps,
     )
 
     logits = fns.decode(params, z_final)                 # (N, C)
@@ -45,12 +52,21 @@ def solve(fns: ReasonerFns, params, x, cfg, key, k_restarts: int | None = None)
     def to_bk(a):
         return jnp.reshape(a, (B, K) + a.shape[1:])
 
+    step_pred = None
+    if record_steps:
+        T1 = z_traj.shape[0]                             # steps + 1
+        step_logits = fns.decode(params, z_traj.reshape(T1 * B * K, d))   # ((T+1)*N, C)
+        step_cls = jnp.argmax(step_logits, axis=-1).reshape(T1, B * K)    # (T+1, N)
+        step_pred = to_bk(step_cls.T)                                     # (N, T+1) -> (B, K, T+1)
+
     return TrajectoryRecord(
         z_star=to_bk(z_final),          # (B, K, d)
         energies=to_bk(energies.T),     # (N, T+1) -> (B, K, T+1)
         grad_norms=to_bk(gnorms.T),     # (N, T)   -> (B, K, T)
         logits=to_bk(logits),           # (B, K, C)
         pred=to_bk(pred),               # (B, K)
+        h_x=h_x,                        # (B, dc)  one context per input (shared across restarts)
+        step_pred=step_pred,            # (B, K, T+1) when record_steps else None
     )
 
 
diff --git a/src/edc/inference/trajectory.py b/src/edc/inference/trajectory.py
index d49e88c..3c1039d 100644
--- a/src/edc/inference/trajectory.py
+++ b/src/edc/inference/trajectory.py
@@ -20,6 +20,8 @@ class TrajectoryRecord:
     grad_norms: Array     # (B, K, T)   ||grad_z E|| at each step
     logits: Array         # (B, K, C)   decoder logits at z_star
     pred: Array           # (B, K)      argmax class per restart
+    h_x: Array            # (B, dc)     per-input context; needed to recompute curvature at z*
+    step_pred: Array | None = None  # (B, K, T+1) per-step decode; set by solve(record_steps=True)
 
     @property
     def terminal_energy(self) -> Array:
diff --git a/src/edc/plotting.py b/src/edc/plotting.py
index 8c0e034..26faece 100644
--- a/src/edc/plotting.py
+++ b/src/edc/plotting.py
@@ -11,6 +11,8 @@
 
 from typing import Any
 
+import numpy as np
+
 STYLE = {
     "figure.dpi": 150,
     "savefig.bbox": "tight",
@@ -26,6 +28,276 @@ def use_style() -> None:
     plt.rcParams.update(STYLE)
 
 
+# Nonconformity scores rendered in F2, in draw order (geometry emphasised, energy baselines
+# muted — the falsification is "geometry beats energy").
+_SCORE_STYLE = {
+    "geometry": {"color": "#1b6ca8", "lw": 2.4, "label": "geometry (learned)"},
+    "rho_basin": {"color": "#5aa469", "lw": 1.6, "label": "1 − ρ_basin (fallback)"},
+    "energy_min": {"color": "#b0794a", "lw": 1.4, "label": "energy Eₘᵢₙ (EBT)"},
+    "energy_mean": {"color": "#c9a66b", "lw": 1.2, "label": "energy Ē"},
+    "energy_std": {"color": "#a05195", "lw": 1.2, "label": "energy spread"},
+    "msp": {"color": "#d1495b", "lw": 1.2, "label": "MSP (softmax)"},
+    "temp_msp": {"color": "#e8825a", "lw": 1.2, "label": "MSP + temperature"},
+    "entropy": {"color": "#8d6e63", "lw": 1.2, "label": "predictive entropy"},
+}
+
+
+def _selective_row(rows: list[dict[str, Any]]) -> dict[str, Any]:
+    """Metrics of the headline ``split == 'selective'`` run for F2/F3 (raises if none).
+
+    Prefers the full-fold run — largest ``n_test`` (E1), ties broken by ledger order — so the
+    single-run figures reflect the authoritative experiment, not whatever reduced-fold sweep cell
+    happened to land last in the ledger.
+    """
+    sel = [r for r in rows if r.get("split") == "selective"]
+    if not sel:
+        raise ValueError("no split='selective' ledger row; run experiments/run_experiment.py first")
+    # largest n_test wins; ties broken toward the latest ledger row (freshest full-fold E1).
+    headline = max(enumerate(sel), key=lambda iv: (iv[1]["metrics"].get("n_test", 0), iv[0]))[1]
+    return headline["metrics"]
+
+
+def _halting_row(rows: list[dict[str, Any]]) -> dict[str, Any]:
+    """Metrics of the latest ``split == 'halting'`` run for F4 (raises if none)."""
+    sel = [r for r in rows if r.get("split") == "halting"]
+    if not sel:
+        raise ValueError("no split='halting' ledger row; run experiments/run_halting.py first")
+    return sel[-1]["metrics"]
+
+
 def risk_coverage_curve(rows: list[dict[str, Any]], out_path: str) -> str:  # F2
-    """Risk-coverage curves per nonconformity score. Implemented in Phase 3."""
-    raise NotImplementedError("Phase 3: risk-coverage figure (F2).")
+    """Risk-coverage curves per nonconformity score — the core selective-prediction result.
+
+    Reads the sampled ``risk_coverage`` grids stored on the latest selective ledger row (invariant
+    6: figures come only from the ledger). Lower curve = better ranker; the geometry line beating
+    the energy lines is the visual falsification test.
+    """
+    import matplotlib.pyplot as plt
+
+    use_style()
+    m = _selective_row(rows)
+    rc = m["risk_coverage"]
+    fig, ax = plt.subplots(figsize=(5.2, 4.0))
+    for name, style in _SCORE_STYLE.items():
+        if name not in rc:
+            continue
+        aurc = m["aurc"].get(name)
+        lbl = f"{style['label']}  (AURC={aurc:.3f})" if aurc is not None else style["label"]
+        ax.plot(rc[name]["coverage"], rc[name]["risk"], color=style["color"],
+                lw=style["lw"], label=lbl)
+    ax.set_xlabel("coverage (fraction answered)")
+    ax.set_ylabel("selective risk (error | answered)")
+    ax.set_title(f"F2 · risk–coverage  (ID acc {m['accuracy_id']:.2f})")
+    ax.set_xlim(0, 1)
+    ax.set_ylim(bottom=0)
+    ax.legend(fontsize=7, frameon=False)
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
+
+
+def k_restart_lift(rows: list[dict[str, Any]], out_path: str) -> str:  # S1 / T2 companion
+    """ΔAURC(best energy − geometry) vs number of restarts K — the restart-ablation figure.
+
+    Per-seed points + the seed-mean line, with a dashed y=0 (below ⇒ geometry loses to energy).
+    A lift that grows with K is evidence the *restart* geometry (not just a single descent) carries
+    the signal. Needs ≥2 distinct K in the ledger; raises otherwise. Data is aggregated in
+    ``analysis.aggregate`` but re-read here from the raw rows to keep plotting self-contained.
+    """
+    import matplotlib.pyplot as plt
+
+    use_style()
+    # Restrict to the single task that actually has a K-sweep, so a second task's fixed-K rows
+    # cannot contaminate the ablation (the K-sweep is run on arithmetic).
+    sel = [r for r in rows if r.get("split") == "selective"]
+
+    def _k(r):
+        m = r["metrics"]
+        return int(m.get("k_restarts", r["config"]["inference"]["k_restarts"]))
+
+    per_task_ks: dict[str, set] = {}
+    for r in sel:
+        per_task_ks.setdefault(r.get("task"), set()).add(_k(r))
+    sweep_task = max(per_task_ks, key=lambda t: len(per_task_ks[t])) if per_task_ks else None
+
+    by_k: dict[int, list[float]] = {}
+    for r in sel:
+        if r.get("task") != sweep_task:
+            continue
+        by_k.setdefault(_k(r), []).append(r["metrics"]["delta_aurc_vs_best_energy"][0])
+    if len(by_k) < 2:
+        raise ValueError("k_restart_lift needs >=2 distinct K in the ledger (run the K-sweep)")
+
+    ks = sorted(by_k)
+    means = [float(np.mean(by_k[k])) for k in ks]
+    fig, ax = plt.subplots(figsize=(5.0, 3.8))
+    ax.axhline(0.0, ls="--", color="0.6", lw=1.0, label="no lift (geometry = energy)")
+    for k in ks:
+        ax.scatter([k] * len(by_k[k]), by_k[k], color="#1b6ca8", alpha=0.5, s=22, zorder=2)
+    ax.plot(ks, means, "-o", color="#1b6ca8", lw=2.0, zorder=3, label="seed mean")
+    ax.set_xscale("log", base=2)
+    ax.set_xticks(ks)
+    ax.set_xticklabels([str(k) for k in ks])
+    ax.set_xlabel("restarts K  (K=1 ≈ EBT, no basin geometry)")
+    ax.set_ylabel("ΔAURC (energy − geometry)")
+    ax.set_title("S1 · restart ablation: does geometry lift grow with K?")
+    ax.legend(fontsize=7, frameon=False)
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
+
+
+def coverage_validity(rows: list[dict[str, Any]], out_path: str) -> str:  # F3
+    """Achieved selective risk vs the nominal target α — the calibration-validity check.
+
+    Distribution-free validity means every point sits at or below the diagonal (achieved risk ≤
+    target). Reads the ``coverage_validity`` α-sweep from the latest selective ledger row.
+    """
+    import matplotlib.pyplot as plt
+
+    use_style()
+    v = _selective_row(rows)["coverage_validity"]
+    fig, ax = plt.subplots(figsize=(4.4, 4.0))
+    lim = max(v["target"]) * 1.1
+    ax.plot([0, lim], [0, lim], ls="--", color="0.6", lw=1.0, label="nominal (achieved = target)")
+    ax.scatter(v["target"], v["achieved_risk"], color="#1b6ca8", zorder=3, label="achieved")
+    for a, r, c in zip(v["target"], v["achieved_risk"], v["coverage"], strict=True):
+        ax.annotate(f"cov {c:.2f}", (a, r), fontsize=6, xytext=(3, 3),
+                    textcoords="offset points", color="0.4")
+    ax.set_xlabel("target selective risk α")
+    ax.set_ylabel("achieved selective risk (test)")
+    ax.set_title("F3 · coverage validity (on/below diagonal = valid)")
+    ax.set_xlim(0, lim)
+    ax.set_ylim(0, lim)
+    ax.legend(fontsize=7, frameon=False)
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
+
+
+_FAMILY_COLOR = {"basin": "#5aa469", "energy": "#b0794a", "curv": "#1b6ca8", "dynamics": "#a05195"}
+
+
+def _family(name: str) -> str:
+    key = name.split("/")[0]
+    return "curv" if key == "curv" else key
+
+
+def _bars_from_hist(h: dict) -> tuple[np.ndarray, np.ndarray, np.ndarray, float]:
+    """Return ``(centers, correct_density, incorrect_density, width)`` from a stored histogram."""
+    edges = np.asarray(h["edges"], dtype=float)
+    centers = 0.5 * (edges[:-1] + edges[1:])
+    c = np.asarray(h["correct_counts"], dtype=float)
+    i = np.asarray(h["incorrect_counts"], dtype=float)
+    c = c / c.sum() if c.sum() else c                     # per-class density (class-balanced view)
+    i = i / i.sum() if i.sum() else i
+    return centers, c, i, float(edges[1] - edges[0])
+
+
+def feature_diagnostics(rows: list[dict[str, Any]], out_path: str) -> str:  # F5
+    """Per-feature separation + the correct-vs-incorrect distributions of the mechanism features.
+
+    Left: |AUROC−0.5| per feature (distance from chance), coloured by family. Right: overlaid
+    correct/incorrect histograms of the most-separating basin and curvature features — the geometry
+    signal the thesis rests on. Reads ``feature_diagnostics`` from the headline selective row.
+    """
+    import matplotlib.pyplot as plt
+
+    fd = _selective_row(rows).get("feature_diagnostics")
+    if fd is None:
+        raise ValueError("no feature_diagnostics on the selective row; re-run run_experiment.py")
+    use_style()
+    names = fd["names"]
+    sep = {n: abs(fd["auroc"][n] - 0.5) for n in names}
+    order = sorted(names, key=lambda n: sep[n])            # ascending -> strongest at top
+
+    def top(family: str) -> str:
+        fam = [n for n in names if _family(n) == family]
+        return max(fam, key=lambda n: sep[n]) if fam else names[0]
+
+    fig, (ax0, ax1, ax2) = plt.subplots(
+        1, 3, figsize=(11.0, 4.2), gridspec_kw={"width_ratios": [1.5, 1, 1]})
+
+    ax0.barh(range(len(order)), [sep[n] for n in order],
+             color=[_FAMILY_COLOR[_family(n)] for n in order])
+    ax0.set_yticks(range(len(order)))
+    ax0.set_yticklabels(order, fontsize=7)
+    ax0.set_xlabel("|AUROC − 0.5|  (correct-vs-incorrect separation)")
+    ax0.set_title("F5 · which geometry features separate")
+
+    for ax, family in ((ax1, "basin"), (ax2, "curv")):
+        name = top(family)
+        centers, c, i, w = _bars_from_hist(fd["hist"][name])
+        ax.bar(centers, c, width=w, color="#1b6ca8", alpha=0.6, label="correct")
+        ax.bar(centers, i, width=w, color="#d1495b", alpha=0.6, label="incorrect")
+        ax.set_title(f"{name}\n(AUROC {fd['auroc'][name]:.2f})", fontsize=9)
+        ax.set_xlabel("feature value")
+        ax.set_ylabel("density")
+        ax.legend(fontsize=7, frameon=False)
+
+    fig.tight_layout()
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
+
+
+def ood_stress(rows: list[dict[str, Any]], out_path: str) -> str:  # F6
+    """Selective-risk validity in-distribution vs under distribution shift.
+
+    The LTT threshold is calibrated ID; achieved risk vs target α is plotted for the ID test fold
+    (on/below the diagonal = valid) and the OOD test fold (above the diagonal = the guarantee breaks
+    because exchangeability is violated). This is the argument for abstention/routing in critical
+    systems. Reads ``ood_validity`` from the headline selective row.
+    """
+    import matplotlib.pyplot as plt
+
+    v = _selective_row(rows).get("ood_validity")
+    if v is None:
+        raise ValueError("no ood_validity on the selective row; re-run with include_ood=True")
+    use_style()
+    lim = max(v["target"]) * 1.1
+    fig, ax = plt.subplots(figsize=(4.8, 4.2))
+    ax.plot([0, lim], [0, lim], ls="--", color="0.6", lw=1.0, label="valid (achieved ≤ target)")
+    ax.plot(v["target"], v["id_risk"], "-o", color="#1b6ca8", lw=1.8, label="in-distribution")
+    ax.plot(v["target"], v["ood_risk"], "-s", color="#d1495b", lw=1.8, label="out-of-distribution")
+    ax.set_xlabel("target selective risk α")
+    ax.set_ylabel("achieved selective risk (test)")
+    ax.set_title("F6 · guarantee holds ID, breaks under shift")
+    ax.set_xlim(0, lim)
+    ax.set_ylim(0, max(1.0, max(v["ood_risk"]) * 1.05) if v["ood_risk"] else 1.0)
+    ax.legend(fontsize=7, frameon=False, loc="upper left")
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
+
+
+def halting_pareto(rows: list[dict[str, Any]], out_path: str) -> str:  # F4
+    """Compute-vs-accuracy Pareto under adaptive halting.
+
+    Sweeps the agreement threshold τ: end-task accuracy (y) vs mean fraction of the step budget
+    used (x). Marks the CRC-chosen operating point (guaranteed halting risk ≤ α) and the full-budget
+    reference. Reads the latest ``split=='halting'`` ledger row (invariant 6).
+    """
+    import matplotlib.pyplot as plt
+
+    use_style()
+    m = _halting_row(rows)
+    s = m["tau_sweep"]
+    fig, ax = plt.subplots(figsize=(5.2, 4.0))
+    ax.plot(s["compute_used"], s["accuracy"], "-o", color="#1b6ca8", lw=1.8, ms=4,
+            label="halting τ-sweep", zorder=2)
+    ax.axhline(m["full_accuracy"], ls="--", color="0.6", lw=1.0,
+               label=f"full-budget acc {m['full_accuracy']:.2f}")
+    if m["tau_hat"] is not None:
+        lbl = (f"CRC τ̂={m['tau_hat']:.2f} (risk {m['halting_risk']:.2f} ≤ α={m['alpha']}), "
+               f"{m['compute_saved']:.0%} compute saved")
+        ax.scatter([m["compute_used"]], [m["halted_accuracy"]], color="#d1495b", s=70, zorder=3,
+                   label=lbl)
+    ax.set_xlabel("compute used (fraction of step budget)")
+    ax.set_ylabel("end-task accuracy")
+    ax.set_title("F4 · adaptive halting: accuracy vs compute")
+    ax.set_xlim(0, 1.02)
+    ax.legend(fontsize=7, frameon=False, loc="lower right")
+    fig.savefig(out_path)
+    plt.close(fig)
+    return out_path
diff --git a/src/edc/tasks/__init__.py b/src/edc/tasks/__init__.py
index 5a57f21..e53a432 100644
--- a/src/edc/tasks/__init__.py
+++ b/src/edc/tasks/__init__.py
@@ -1,10 +1,12 @@
 """Task families. Importing this package registers every built task.
 
-Phase 1 ships ``arithmetic``. ``graph_planning``, ``logic``, and ``hard_task`` are stubbed
-(Phase 4) and import lazily without erroring so the registry can list them.
+Phase 1 ships ``arithmetic``; Phase 4d adds ``graph_planning``. ``logic`` and ``hard_task`` remain
+stubbed and are not imported until implemented.
 """
 
-from edc.tasks import arithmetic  # noqa: F401  (side effect: registers "arithmetic")
+from edc.tasks import (  # noqa: F401  (side effects: register the tasks)
+    arithmetic,
+    graph_planning,
+)
 
-# Phase 4 task families are intentionally not imported here until implemented.
-__all__ = ["arithmetic"]
+__all__ = ["arithmetic", "graph_planning"]
diff --git a/src/edc/tasks/arithmetic.py b/src/edc/tasks/arithmetic.py
index d6534e7..05bac11 100644
--- a/src/edc/tasks/arithmetic.py
+++ b/src/edc/tasks/arithmetic.py
@@ -29,6 +29,7 @@ def __init__(
         n_operands: int = 3,
         max_operand: int = 9,
         ood_n_operands: int = 5,
+        ood_max_operand: int | None = None,
         modular: bool = False,
     ) -> None:
         if max(n_operands, ood_n_operands) > _MAX_OPERANDS:
@@ -37,23 +38,33 @@ def __init__(
         self.n_operands = n_operands
         self.max_operand = max_operand
         self.ood_n_operands = ood_n_operands
+        # OOD shift 1: more operands (ood_n_operands). OOD shift 2: larger operand magnitudes
+        # (ood_max_operand > max_operand) — a covariate shift that keeps the reasoning identical
+        # and the low-sum region in-distribution, so accuracy degrades *gracefully* rather than
+        # collapsing. Defaults to max_operand (no magnitude shift).
+        self.ood_max_operand = max_operand if ood_max_operand is None else ood_max_operand
         # modular=True is the harder "grokking" variant (answer = sum mod modulus);
         # modular=False (default) is plain sum classification — smooth in the inputs and the
         # right demonstrator that descent lands in the correct basin. Both share feature_dim.
         self.modular = modular
-        self.n_classes = modulus if modular else _MAX_OPERANDS * max_operand + 1
-        self.feature_dim = _MAX_OPERANDS * (max_operand + 1)
+        # Encoding width and class count must cover both splits so one model handles ID + OOD.
+        self._slot_max = max(max_operand, self.ood_max_operand)
+        self.n_classes = modulus if modular else _MAX_OPERANDS * self._slot_max + 1
+        self.feature_dim = _MAX_OPERANDS * (self._slot_max + 1)
 
     def _k(self, split: str) -> int:
         return self.ood_n_operands if split == "ood" else self.n_operands
 
+    def _max(self, split: str) -> int:
+        return self.ood_max_operand if split == "ood" else self.max_operand
+
     def sample(self, rng: np.random.Generator, n: int, split: str = "id") -> Batch:
         k = self._k(split)
-        operands = rng.integers(0, self.max_operand + 1, size=(n, k))
+        operands = rng.integers(0, self._max(split) + 1, size=(n, k))
         total = operands.sum(axis=1)
         y = (total % self.modulus) if self.modular else total
 
-        slot = self.max_operand + 1
+        slot = self._slot_max + 1
         x = np.zeros((n, self.feature_dim), dtype=np.float32)
         for j in range(k):
             cols = j * slot + operands[:, j]
@@ -65,7 +76,7 @@ def evaluate(self, pred: np.ndarray, y: np.ndarray) -> np.ndarray:
 
     def difficulty(self, batch: Batch) -> np.ndarray:
         # Number of active operand slots ~ proxy for carry complexity.
-        slot = self.max_operand + 1
+        slot = self._slot_max + 1
         blocks = batch.x.reshape(batch.x.shape[0], _MAX_OPERANDS, slot)
         return blocks.max(axis=2).sum(axis=1).astype(np.float32)
 
@@ -73,5 +84,6 @@ def difficulty(self, batch: Batch) -> np.ndarray:
 @register_task("arithmetic")
 def _build(**kwargs) -> ArithmeticTask:
     # Accept keys from the [task.arithmetic] TOML block; ignore unrelated ones.
-    known = {"modulus", "n_operands", "max_operand", "ood_n_operands", "modular"}
+    known = {"modulus", "n_operands", "max_operand", "ood_n_operands", "ood_max_operand",
+             "modular"}
     return ArithmeticTask(**{k: v for k, v in kwargs.items() if k in known})
diff --git a/src/edc/tasks/graph_planning.py b/src/edc/tasks/graph_planning.py
index 965907c..eac2102 100644
--- a/src/edc/tasks/graph_planning.py
+++ b/src/edc/tasks/graph_planning.py
@@ -1,14 +1,102 @@
-"""Graph shortest-path / connectivity planning family (port of EBRM). [Phase 4]
+"""Graph shortest-path planning family. [Phase 4d]
 
-Fixed-size adjacency + source/target encoded into ``x``; answer is a path/next-hop/length
-label. OOD split = larger graphs than trained on (IRED-style size generalisation).
+A relational/combinatorial reasoning family, deliberately distinct from additive arithmetic, to
+test whether the geometry-beats-energy discovery *generalizes*. Each problem is a random undirected
+Erdős–Rényi graph with a random ``(source, target)`` pair; the answer is the shortest-path length
+(BFS), capped: classes ``1..max_len`` are the exact length, class ``0`` = unreachable-or-farther.
+
+Features are fixed-size (padded to ``max_nodes``) so the in-distribution split (``n_nodes``) and the
+OOD split (``ood_n_nodes`` > it — IRED-style size generalization) share one model:
+flattened ``max_nodes × max_nodes`` adjacency ⊕ source one-hot ⊕ target one-hot.
+
+Pure NumPy (invariant 1); BFS is a small per-sample loop (graphs are tiny).
 """
 
 from __future__ import annotations
 
+from collections import deque
+
+import numpy as np
+
+from edc.registry import register_task
+from edc.tasks.base import Batch
+
+_MAX_NODES = 12  # padding capacity; both ID and OOD node counts must be <= this
+
+
+def _shortest_len(adj: np.ndarray, src: int, dst: int, cap: int) -> int:
+    """BFS shortest-path length in an undirected 0/1 adjacency; 0 if unreachable or > ``cap``."""
+    if src == dst:
+        return 0
+    seen = {src}
+    q = deque([(src, 0)])
+    while q:
+        node, dist = q.popleft()
+        if dist >= cap:
+            continue
+        for nbr in np.nonzero(adj[node])[0]:
+            if nbr == dst:
+                return dist + 1
+            if nbr not in seen:
+                seen.add(int(nbr))
+                q.append((int(nbr), dist + 1))
+    return 0  # unreachable-or-farther
+
 
 class GraphPlanningTask:
     name = "graph_planning"
 
-    def __init__(self, **kwargs) -> None:
-        raise NotImplementedError("Phase 4: port EBRM graph_reasoning.jl generator.")
+    def __init__(
+        self,
+        n_nodes: int = 7,
+        ood_n_nodes: int = 10,
+        edge_prob: float = 0.3,
+        max_len: int = 4,
+    ) -> None:
+        if max(n_nodes, ood_n_nodes) > _MAX_NODES:
+            raise ValueError(f"node count exceeds capacity {_MAX_NODES}")
+        self.n_nodes = n_nodes
+        self.ood_n_nodes = ood_n_nodes
+        self.edge_prob = edge_prob
+        self.max_len = max_len
+        # class 0 = unreachable/farther, 1..max_len = exact shortest-path length.
+        self.n_classes = max_len + 1
+        self.feature_dim = _MAX_NODES * _MAX_NODES + 2 * _MAX_NODES
+
+    def _n(self, split: str) -> int:
+        return self.ood_n_nodes if split == "ood" else self.n_nodes
+
+    def sample(self, rng: np.random.Generator, n: int, split: str = "id") -> Batch:
+        k = self._n(split)
+        x = np.zeros((n, self.feature_dim), dtype=np.float32)
+        y = np.zeros(n, dtype=np.int64)
+        adj_block = _MAX_NODES * _MAX_NODES
+        for i in range(n):
+            # Random undirected graph on the k active nodes (upper triangle -> symmetric).
+            upper = (rng.random((k, k)) < self.edge_prob).astype(np.float32)
+            upper = np.triu(upper, 1)
+            adj = upper + upper.T
+            src, dst = rng.choice(k, size=2, replace=False)
+            y[i] = _shortest_len(adj, int(src), int(dst), self.max_len)
+
+            padded = np.zeros((_MAX_NODES, _MAX_NODES), dtype=np.float32)
+            padded[:k, :k] = adj
+            x[i, :adj_block] = padded.reshape(-1)
+            x[i, adj_block + int(src)] = 1.0                       # source one-hot
+            x[i, adj_block + _MAX_NODES + int(dst)] = 1.0          # target one-hot
+        return Batch(x=x, y=y)
+
+    def evaluate(self, pred: np.ndarray, y: np.ndarray) -> np.ndarray:
+        return np.asarray(pred) == np.asarray(y)
+
+    def difficulty(self, batch: Batch) -> np.ndarray:
+        # Edge count (undirected) ~ graph density proxy.
+        adj_block = _MAX_NODES * _MAX_NODES
+        adj = batch.x[:, :adj_block]
+        return (adj.sum(axis=1) / 2.0).astype(np.float32)
+
+
+@register_task("graph_planning")
+def _build(**kwargs) -> GraphPlanningTask:
+    known = {"n_nodes", "ood_n_nodes", "edge_prob", "max_len"}
+    return GraphPlanningTask(**{k: v for k, v in kwargs.items() if k in known})
diff --git a/tests/conftest.py b/tests/conftest.py
index 39d4f51..5a416b3 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -1,5 +1,13 @@
 """Shared test configuration: force JAX onto CPU so the suite never needs a GPU."""
 
 import os
+import sys
+from pathlib import Path
 
 os.environ.setdefault("JAX_PLATFORMS", "cpu")
+
+# The experiments/ and analysis/ scripts are not a package; expose them for direct import in tests
+# (the same way they resolve sibling imports when run as scripts).
+_ROOT = Path(__file__).resolve().parents[1]
+for _d in ("experiments", "analysis"):
+    sys.path.insert(0, str(_ROOT / _d))
diff --git a/tests/test_aggregate.py b/tests/test_aggregate.py
new file mode 100644
index 0000000..9ed5d83
--- /dev/null
+++ b/tests/test_aggregate.py
@@ -0,0 +1,93 @@
+"""Ledger aggregation across seeds: grouping by K + the multi-seed falsification summary.
+
+Offline + CPU-only, on hand-built synthetic ledger rows (no training).
+"""
+
+import aggregate
+
+
+def _row(k, seed, delta, lo, geo=0.08, best=0.14, task="arithmetic", baseline_delta=None):
+    m = {
+        "k_restarts": k, "accuracy_id": 0.8, "base_error": 0.2,
+        "best_energy_baseline": "energy_min",
+        "aurc": {"geometry": geo, "energy_min": best},
+        "delta_aurc_vs_best_energy": [delta, lo, delta + 0.02],
+        "ltt": {"coverage": 0.7, "selective_risk": 0.07, "abstain_rate": 0.3},
+    }
+    if baseline_delta is not None:                          # Phase 4e field (older rows omit it)
+        bd, blo = baseline_delta
+        m["best_baseline"] = "temp_msp"
+        m["delta_aurc_vs_best_baseline"] = [bd, blo, bd + 0.02]
+        m["feature_ablation"] = {                            # Phase 4f field
+            "full": geo, "drop_basin": geo + 0.03, "drop_energy": geo + 0.005,
+            "drop_curv": geo + 0.001, "drop_dynamics": geo + 0.002,
+        }
+    return {
+        "run_id": f"{task[:3]}{k}_{seed}", "seed": seed, "split": "selective", "task": task,
+        "config_hash": f"{task}h{k}_{seed}", "git_sha": "abc1234",
+        "env": {"python": "3.12", "jax": "0.11", "jax_backend": "cpu"},
+        "config": {"inference": {"k_restarts": k}},
+        "metrics": m,
+    }
+
+
+def test_by_k_groups_and_dedups():
+    rows = [_row(1, 0, 0.0, -0.01), _row(1, 1, 0.0, -0.01), _row(4, 0, 0.06, 0.05)]
+    # a re-run of (K=4, seed 0) with the SAME config_hash must replace, not duplicate
+    rows.append(_row(4, 0, 0.07, 0.06))
+    grouped = aggregate.by_k(rows)
+    assert set(grouped) == {1, 4}
+    assert len(grouped[1]) == 2 and len(grouped[4]) == 1
+    assert grouped[4][0]["metrics"]["delta_aurc_vs_best_energy"][0] == 0.07   # latest kept
+
+
+def test_aggregate_cell_verdict():
+    # K=1: no lift (CI includes 0). K=16: clear lift on all seeds.
+    k1 = [_row(1, s, 0.001, -0.01) for s in range(5)]
+    a1 = aggregate.aggregate_cell(k1)
+    assert a1["seeds_ci_excludes_0"] == 0 and a1["geometry_wins_all"] is False
+    assert a1["n_seeds"] == 5 and a1["k_restarts"] == 1
+
+    k16 = [_row(16, s, 0.09, 0.07, geo=0.05) for s in range(5)]
+    a16 = aggregate.aggregate_cell(k16)
+    assert a16["seeds_ci_excludes_0"] == 5 and a16["geometry_wins_all"] is True
+    assert a16["delta_aurc"][0] > a1["delta_aurc"][0]           # lift grows with K
+    assert a16["aurc_geometry"][0] < a16["aurc_best_energy"][0]  # geometry lower AURC = better
+
+
+def test_baseline_delta_aggregation_and_fallback():
+    # Rows carrying the Phase-4e field aggregate its own value + seeds-win count.
+    withbl = [_row(12, s, 0.09, 0.07, baseline_delta=(0.05, 0.03)) for s in range(4)]
+    a = aggregate.aggregate_cell(withbl)
+    assert a["delta_aurc_baseline"][0] == 0.05 and a["seeds_ci_excludes_0_baseline"] == 4
+    assert a["delta_aurc"][0] == 0.09                       # energy ΔAURC still separate
+    assert "temp_msp" in a["best_baseline_names"]
+
+    # Older rows without the field fall back to the energy ΔAURC (no crash).
+    old = [_row(12, s, 0.09, 0.07) for s in range(3)]
+    a_old = aggregate.aggregate_cell(old)
+    assert a_old["delta_aurc_baseline"] == a_old["delta_aurc"]
+    assert a_old["feature_ablation"] == {}                  # absent -> empty, no crash
+
+
+def test_feature_ablation_aggregates():
+    rows = [_row(12, s, 0.09, 0.07, geo=0.08, baseline_delta=(0.05, 0.03)) for s in range(4)]
+    fa = aggregate.aggregate_cell(rows)["feature_ablation"]
+    assert set(fa) == {"full", "drop_basin", "drop_energy", "drop_curv", "drop_dynamics"}
+    assert fa["full"][0] == 0.08 and fa["drop_basin"][0] == 0.11   # mean AURC per subset
+    assert fa["full"][1] == 0.0                                    # zero std (identical seeds)
+
+
+def test_task_filter_no_cross_contamination():
+    # Two tasks share K=12 in the ledger; aggregation must not mix them.
+    rows = [_row(12, s, 0.08, 0.06, task="arithmetic") for s in range(3)]
+    rows += [_row(12, s, 0.02, -0.02, task="graph_planning") for s in range(4)]
+    assert aggregate.tasks_present(rows) == ["arithmetic", "graph_planning"]
+
+    ar = aggregate.aggregate_by_k(rows, task="arithmetic")[12]
+    gr = aggregate.aggregate_by_k(rows, task="graph_planning")[12]
+    assert ar["n_seeds"] == 3 and gr["n_seeds"] == 4
+    assert ar["task"] == "arithmetic" and gr["task"] == "graph_planning"
+    assert ar["delta_aurc"][0] != gr["delta_aurc"][0]           # distinct aggregates
+    # unfiltered mixes both (7 rows at K=12) — the bug the filter prevents
+    assert aggregate.aggregate_by_k(rows)[12]["n_seeds"] == 7
diff --git a/tests/test_baselines.py b/tests/test_baselines.py
new file mode 100644
index 0000000..52e46e3
--- /dev/null
+++ b/tests/test_baselines.py
@@ -0,0 +1,43 @@
+"""Softmax-confidence baselines: MSP / entropy exact values, temperature scaling reduces NLL, and
+degenerate-fold fallbacks. Offline + CPU-only, NumPy/scipy.
+"""
+
+import numpy as np
+
+from edc.eval import baselines as B
+
+
+def test_msp_and_entropy_exact():
+    peaked = np.array([[10.0, 0.0, 0.0, 0.0]])
+    uniform = np.array([[1.0, 1.0, 1.0, 1.0]])
+    assert B.msp_score(peaked)[0] < 1e-3                    # confident -> low nonconformity
+    assert np.isclose(B.msp_score(uniform)[0], 0.75)       # 1 - 1/4
+    assert B.entropy_score(peaked)[0] < 1e-2               # peaked -> ~0 entropy
+    assert np.isclose(B.entropy_score(uniform)[0], np.log(4), atol=1e-6)   # uniform -> ln C
+
+
+def test_softmax_normalises():
+    p = B.softmax(np.random.default_rng(0).standard_normal((5, 7)))
+    assert np.allclose(p.sum(axis=1), 1.0) and (p >= 0).all()
+
+
+def test_fit_temperature_reduces_nll():
+    rng = np.random.default_rng(0)
+    n, c = 800, 5
+    y = rng.integers(0, c, n)
+    onehot = np.zeros((n, c))
+    onehot[np.arange(n), y] = 1.0
+    logits = (onehot + rng.standard_normal((n, c)) * 0.5) * 6.0   # mis-scaled (overconfident)
+    T = B.fit_temperature(logits, y)
+    assert 0.05 <= T <= 20.0
+    assert B._nll(T, logits, y) <= B._nll(1.0, logits, y) + 1e-9  # temperature never hurts NLL
+
+
+def test_temp_msp_and_single_class_fallback():
+    logits = np.array([[3.0, 1.0, 0.0]])
+    # higher temperature flattens the softmax -> higher nonconformity (less confident)
+    assert B.temp_msp_score(logits, 5.0)[0] > B.temp_msp_score(logits, 0.5)[0]
+    # a single-class fit fold cannot calibrate -> T = 1.0 fallback, scores still finite
+    T = B.fit_temperature(logits.repeat(10, 0), np.zeros(10, int))
+    assert T == 1.0
+    assert np.all(np.isfinite(B.temp_msp_score(logits, T)))
diff --git a/tests/test_conformal_crc.py b/tests/test_conformal_crc.py
new file mode 100644
index 0000000..5309ed4
--- /dev/null
+++ b/tests/test_conformal_crc.py
@@ -0,0 +1,51 @@
+"""Conformal Risk Control calibrator: exact finite-sample boundary, infeasible-alpha guard, and
+the monotone-halting risk guarantee ``E[L(lambda_hat)] <= alpha``. Offline + CPU-only.
+"""
+
+import numpy as np
+
+from edc.conformal.crc import calibrate
+
+
+def test_crc_boundary_pick():
+    # n=9, alpha=0.2 -> RHS = ((n+1)*alpha - 1)/n = (2 - 1)/9 = 0.1111.
+    # Three lambdas with R_hat = {0, 1/9=0.111, 1}. Admissible prefix is {0, 0.111}; pick the
+    # most compute-saving (largest index) admissible -> lambda index 1.
+    n = 9
+    loss = np.zeros((n, 3))
+    loss[0, 1] = 1.0          # column 1: one error -> R_hat = 1/9 = 0.111 <= 0.111 (admissible)
+    loss[:, 2] = 1.0          # column 2: R_hat = 1.0 (fails)
+    assert calibrate(loss, alpha=0.2, lambdas=[0.0, 0.5, 1.0]) == 0.5
+
+
+def test_crc_infeasible_alpha_returns_none():
+    # n=3, alpha=0.2 -> (n+1)*alpha - 1 = -0.2 < 0: no R_hat (not even 0) can satisfy the bound.
+    assert calibrate(np.zeros((3, 2)), alpha=0.2) is None
+
+
+def test_crc_dict_input_and_monotone_stop():
+    # dict form; risks increase with lambda. RHS for n=20, alpha=0.2 = (21*0.2-1)/20 = 0.16.
+    n = 20
+    loss_by_lambda = {
+        0.1: np.zeros(n),                                    # R_hat 0.00 -> admissible
+        0.2: np.array([1] * 3 + [0] * (n - 3)),              # R_hat 0.15 -> admissible
+        0.3: np.array([1] * 5 + [0] * (n - 5)),              # R_hat 0.25 -> fails, stop
+    }
+    assert calibrate(loss_by_lambda, alpha=0.2) == 0.2
+
+
+def test_crc_risk_guarantee():
+    # Monte-Carlo: E[L(lambda_hat)] <= alpha over draws. Loss at each lambda is Bernoulli with a
+    # rate that increases with lambda; CRC must never pick a lambda whose true rate exceeds alpha.
+    alpha = 0.1
+    lambdas = np.array([0.0, 1.0, 2.0, 3.0])
+    true_rates = np.array([0.02, 0.06, 0.12, 0.30])
+    picked_true_rate = []
+    for t in range(400):
+        rng = np.random.default_rng(500 + t)
+        loss = (rng.random((300, len(lambdas))) < true_rates).astype(float)
+        lam = calibrate(loss, alpha, lambdas=lambdas)
+        if lam is not None:
+            picked_true_rate.append(true_rates[list(lambdas).index(lam)])
+    # the finite-sample bound should keep the expected picked risk at or below alpha
+    assert np.mean(picked_true_rate) <= alpha
diff --git a/tests/test_conformal_ltt.py b/tests/test_conformal_ltt.py
new file mode 100644
index 0000000..91282ad
--- /dev/null
+++ b/tests/test_conformal_ltt.py
@@ -0,0 +1,75 @@
+"""Learn-then-Test: Hoeffding-Bentkus p-value validity + LTT selective-risk coverage guarantee.
+
+The two properties that matter: (1) HB is a valid super-uniform p-value for ``H: R > alpha``, so
+``P(p <= delta) <= delta`` when ``R == alpha``; (2) the calibrated threshold controls selective
+risk at ``1 - delta`` confidence over fresh data. Offline + CPU-only.
+"""
+
+import numpy as np
+
+from edc.conformal.ltt import calibrate, hoeffding_bentkus_pvalue, selective_risk
+
+
+def test_hb_edges():
+    assert hoeffding_bentkus_pvalue(0.3, 0.2, 200) == 1.0     # risk >= alpha: no evidence
+    assert hoeffding_bentkus_pvalue(0.2, 0.2, 200) == 1.0     # boundary
+    assert hoeffding_bentkus_pvalue(0.0, 0.2, 0) == 1.0       # empty sample certifies nothing
+    p0 = hoeffding_bentkus_pvalue(0.0, 0.2, 5)               # finite, no nan at risk_hat=0
+    assert np.isfinite(p0) and p0 == min(0.8 ** 5, np.e * 0.8 ** 5)
+
+
+def test_hb_monotone_in_risk_hat():
+    # p-value INCREASES as risk_hat rises toward alpha (evidence against H: R>alpha weakens).
+    ps = [hoeffding_bentkus_pvalue(r, 0.3, 500) for r in np.linspace(0.0, 0.29, 30)]
+    assert all(a <= b + 1e-12 for a, b in zip(ps[:-1], ps[1:], strict=True))  # non-decreasing
+
+
+def test_hb_pvalue_is_valid_superuniform():
+    # With true risk == alpha, the p-value must satisfy P(p <= delta) <= delta (validity).
+    alpha, delta, n, trials = 0.2, 0.1, 200, 4000
+    rng = np.random.default_rng(0)
+    x = rng.binomial(n, alpha, size=trials)              # X ~ Bin(n, alpha)
+    ps = np.array([hoeffding_bentkus_pvalue(xi / n, alpha, n) for xi in x])
+    assert (ps <= delta).mean() <= delta + 0.02          # small Monte-Carlo slack
+
+
+def test_selective_risk_empty_answered():
+    r, n = selective_risk([0.5, 0.6], [True, True], lam=0.1)  # nothing answered
+    assert n == 0 and np.isnan(r)
+
+
+def _synth(rng, n, base_err):
+    correct = rng.random(n) > base_err
+    scores = np.where(correct, rng.beta(2, 5, n), rng.beta(5, 2, n))  # low score when correct
+    return scores, correct
+
+
+def test_ltt_coverage_guarantee():
+    # Over many calib/test draws, the empirical P(R_sel(lambda_hat) <= alpha) must be >= 1 - delta.
+    alpha, delta, trials = 0.1, 0.05, 250
+    violations = 0
+    coverages = []
+    for t in range(trials):
+        rng = np.random.default_rng(2000 + t)
+        sc, cc = _synth(rng, 800, base_err=0.15)
+        st, ct = _synth(rng, 4000, base_err=0.15)
+        lam = calibrate(sc, cc, alpha, delta)["lambda_hat"]
+        if lam is None:
+            coverages.append(0.0)
+            continue
+        answered = st <= lam
+        coverages.append(answered.mean())
+        if answered.any() and (~ct[answered]).mean() > alpha:
+            violations += 1
+    assert violations / trials <= delta                   # distribution-free guarantee holds
+    assert np.mean(coverages) > 0.5                        # and it is not vacuous (answers a lot)
+
+
+def test_ltt_abstains_when_base_error_exceeds_alpha():
+    # base error 25% > alpha 10%: must abstain on hard inputs -> coverage < 1, risk pulled down.
+    rng = np.random.default_rng(7)
+    sc, cc = _synth(rng, 3000, base_err=0.25)
+    lam = calibrate(sc, cc, alpha=0.1, delta=0.05)["lambda_hat"]
+    answered = sc <= lam
+    assert 0.0 < answered.mean() < 1.0
+    assert (~cc[answered]).mean() <= 0.15                 # selective risk near/below target
diff --git a/tests/test_eval_metrics.py b/tests/test_eval_metrics.py
new file mode 100644
index 0000000..8880cce
--- /dev/null
+++ b/tests/test_eval_metrics.py
@@ -0,0 +1,83 @@
+"""Selective-prediction metrics: exact AURC values, risk-coverage shape, and the paired-bootstrap
+ΔAURC falsification test (directionality + determinism).
+
+Offline + CPU-only. NumPy-only maths; the only reason JAX loads is the seeding import.
+"""
+
+import numpy as np
+import pytest
+
+from edc.eval.metrics import (
+    aurc,
+    paired_bootstrap_delta_aurc,
+    risk_coverage_curve,
+    selective_accuracy_at_coverage,
+)
+
+T, F = True, False
+
+
+def test_risk_coverage_exact():
+    # scores ascending == confidence order; errors are the two least-confident points.
+    cov, risk = risk_coverage_curve([0.1, 0.2, 0.3, 0.4], [T, T, F, F])
+    assert np.allclose(cov, [0.25, 0.5, 0.75, 1.0])
+    assert np.allclose(risk, [0.0, 0.0, 1 / 3, 0.5])
+
+
+def test_risk_coverage_full_risk_is_base_error():
+    rng = np.random.default_rng(0)
+    scores = rng.standard_normal(500)
+    correct = rng.random(500) < 0.7
+    _, risk = risk_coverage_curve(scores, correct)
+    assert np.isclose(risk[-1], np.mean(~correct))  # full coverage == base error rate
+
+
+def test_aurc_exact_value():
+    assert aurc([0.1, 0.2, 0.3, 0.4], [T, T, F, F]) == pytest.approx(0.2083333, abs=1e-6)
+
+
+def test_aurc_ranking_bounds():
+    rng = np.random.default_rng(1)
+    n = 400
+    correct = rng.random(n) < 0.6
+    error = (~correct).astype(float)
+    perfect = error + rng.random(n) * 1e-6           # errors ranked LEAST confident (high score)
+    worst = -perfect                                 # errors ranked MOST confident
+    random_scores = rng.standard_normal(n)
+    # perfect ranking pushes errors to the tail -> every prefix risk <= base error -> AURC minimal
+    assert aurc(perfect, correct) < aurc(random_scores, correct) < aurc(worst, correct)
+    assert aurc(perfect, correct) <= error.mean() and aurc(worst, correct) >= error.mean()
+    # constant scores => all ties => tie-robust AURC == base error rate exactly
+    assert aurc(np.ones(n), correct) == pytest.approx(error.mean())
+
+
+def test_selective_accuracy_at_coverage():
+    # answer the 2 most confident of 4 -> both correct -> selective acc 1.0
+    assert selective_accuracy_at_coverage([0.1, 0.2, 0.3, 0.4], [T, T, F, F], 0.5) == 1.0
+    assert selective_accuracy_at_coverage([0.1, 0.2, 0.3, 0.4], [T, T, F, F], 1.0) == 0.5
+
+
+def test_delta_aurc_deterministic_via_seeding():
+    rng = np.random.default_rng(2)
+    n = 300
+    correct = rng.random(n) < 0.6
+    a, b = rng.random(n), rng.random(n)
+    r1 = paired_bootstrap_delta_aurc(a, b, correct, n_boot=200, seed=5)
+    r2 = paired_bootstrap_delta_aurc(a, b, correct, n_boot=200, seed=5)
+    assert r1 == r2
+    assert paired_bootstrap_delta_aurc(a, b, correct, n_boot=200, seed=6) != r1
+
+
+def test_delta_aurc_directionality():
+    rng = np.random.default_rng(3)
+    n = 600
+    correct = rng.random(n) < 0.6
+    # b (geometry) ranks well; a (energy) is noise. delta = AURC(a) - AURC(b) > 0 => b wins.
+    good = np.where(correct, rng.random(n) * 0.4, 0.6 + rng.random(n) * 0.4)
+    noise = rng.random(n)
+    delta, lo, hi = paired_bootstrap_delta_aurc(noise, good, correct, n_boot=1000, seed=0)
+    assert delta > 0 and lo > 0                       # geometry strictly wins, CI excludes 0
+
+    # identical scorers => delta 0 and CI brackets 0 (a genuine tie / null result)
+    d0, lo0, hi0 = paired_bootstrap_delta_aurc(noise, noise, correct, n_boot=500, seed=0)
+    assert d0 == 0.0 and lo0 <= 0.0 <= hi0
diff --git a/tests/test_evaluate_halting.py b/tests/test_evaluate_halting.py
new file mode 100644
index 0000000..0e6f9f3
--- /dev/null
+++ b/tests/test_evaluate_halting.py
@@ -0,0 +1,31 @@
+"""End-to-end adaptive-halting evaluation on the smoke config: complete, finite, JSON-serialisable
+metrics with the F4 tau-sweep. Marked slow (it trains). Offline + CPU-only.
+"""
+
+import json
+
+import numpy as np
+import pytest
+
+from edc.config import load_config
+from edc.eval.evaluate_halting import evaluate_halting
+from edc.registry import build_task
+
+pytestmark = pytest.mark.slow
+
+
+def test_halting_pipeline_metrics():
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    m = evaluate_halting(cfg, task)
+
+    for key in ("tau_hat", "compute_used", "halting_risk", "risk_within_budget",
+                "full_accuracy", "halted_accuracy", "tau_sweep", "risk_monotone_in_tau"):
+        assert key in m
+
+    s = m["tau_sweep"]
+    n = len(s["taus"])
+    assert len(s["compute_used"]) == n and len(s["accuracy"]) == n and len(s["disagreement"]) == n
+    assert all(0.0 <= c <= 1.0 for c in s["compute_used"])
+    assert 0.0 <= m["full_accuracy"] <= 1.0 and np.isfinite(m["halted_accuracy"])
+    json.dumps(m)  # ledger-serialisable
diff --git a/tests/test_evaluate_include_ood.py b/tests/test_evaluate_include_ood.py
new file mode 100644
index 0000000..f2a8e1f
--- /dev/null
+++ b/tests/test_evaluate_include_ood.py
@@ -0,0 +1,37 @@
+"""``evaluate(include_ood=False)`` skips the OOD fold but still returns the falsification + LTT
+blocks. Marked slow (it trains a tiny reasoner). Offline + CPU-only.
+"""
+
+import numpy as np
+import pytest
+
+from edc.config import load_config
+from edc.eval.evaluate import evaluate
+from edc.registry import build_task
+
+pytestmark = pytest.mark.slow
+
+
+def test_include_ood_flag_toggles_ood_metrics():
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    m = evaluate(cfg, task, include_ood=False)
+
+    assert "accuracy_ood" not in m and "aurc_ood" not in m   # OOD fold skipped
+    # ID falsification + guarantee still present and finite
+    assert set(m) >= {"aurc", "delta_aurc_vs_best_energy", "geometry_wins", "ltt", "k_restarts"}
+    d, lo, hi = m["delta_aurc_vs_best_energy"]
+    assert lo <= d <= hi
+    assert np.isfinite(m["accuracy_id"]) and "geometry" in m["aurc"]
+
+    # Phase 4e softmax-confidence baselines + the vs-best-baseline falsification
+    assert {"msp", "temp_msp", "entropy"} <= set(m["aurc"])
+    assert m["best_baseline"] in m["aurc"]
+    db, lob, hib = m["delta_aurc_vs_best_baseline"]
+    assert lob <= db <= hib
+    assert isinstance(m["geometry_wins_vs_baseline"], bool) and np.isfinite(m["temperature"])
+
+    # Phase 4f feature-group leave-one-out ablation
+    fa = m["feature_ablation"]
+    assert {"full", "drop_basin", "drop_energy", "drop_curv", "drop_dynamics"} <= set(fa)
+    assert all(0.0 <= v <= 1.0 for v in fa.values())
diff --git a/tests/test_evaluate_pipeline.py b/tests/test_evaluate_pipeline.py
new file mode 100644
index 0000000..34098ef
--- /dev/null
+++ b/tests/test_evaluate_pipeline.py
@@ -0,0 +1,57 @@
+"""End-to-end selective-prediction pipeline on the tiny smoke config.
+
+Trains a small reasoner, runs the disjoint fit/calib/test folds, and checks the metrics dict is
+complete, finite, JSON-serialisable, and keeps the raw-energy baseline wired in (invariant 8).
+Marked ``slow`` (it trains) but still offline + CPU-only. Values are not asserted — the smoke
+config saturates; correctness of the numbers is covered by the unit tests.
+"""
+
+import json
+
+import numpy as np
+import pytest
+
+from edc.config import load_config
+from edc.eval.evaluate import evaluate
+from edc.registry import build_task
+from edc.seeding import numpy_rng
+
+pytestmark = pytest.mark.slow
+
+
+def test_pipeline_returns_complete_finite_metrics():
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    m = evaluate(cfg, task)
+
+    # folds are the three disjoint sizes from the config
+    assert m["n_fit"] == cfg.eval.n_eval
+    assert m["n_calib"] == cfg.conformal.n_calib
+    assert m["n_test"] == cfg.eval.n_eval
+
+    # raw-energy baselines present alongside geometry (invariant 8) + the fallback
+    for name in ("geometry", "rho_basin", "energy_min", "energy_mean", "energy_std"):
+        assert name in m["aurc"]
+        assert 0.0 <= m["aurc"][name] <= 1.0
+
+    # falsification test wired: ΔAURC point estimate + ordered CI
+    d, lo, hi = m["delta_aurc_vs_best_energy"]
+    assert lo <= d <= hi
+    assert isinstance(m["geometry_wins"], bool)
+
+    # LTT block present and self-consistent
+    ltt = m["ltt"]
+    assert set(ltt) >= {"alpha", "lambda_hat", "coverage", "selective_risk", "risk_within_budget"}
+    assert 0.0 <= ltt["coverage"] <= 1.0
+
+    assert np.isfinite(m["accuracy_id"]) and np.isfinite(m["ece_geometry"])
+    json.dumps(m)  # must be ledger-serialisable
+
+
+def test_folds_are_independently_sampled():
+    # The three ID folds draw from distinct seeding substreams, so they are not the same batch.
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    batches = [task.sample(numpy_rng(cfg.run.seed, 20, s), cfg.eval.n_eval, "id").x
+               for s in (0, 2)]
+    assert not np.array_equal(batches[0], batches[1])
diff --git a/tests/test_geometry_features.py b/tests/test_geometry_features.py
new file mode 100644
index 0000000..63f25c3
--- /dev/null
+++ b/tests/test_geometry_features.py
@@ -0,0 +1,215 @@
+"""Phase-2 geometry features: exact values on hand-built trajectories, shape/finiteness on a
+real (untrained) tiny model, determinism, and batched-curvature correctness.
+
+Offline + CPU-only (conftest forces JAX_PLATFORMS=cpu). No training needed: the geometry
+features are a pure function of a TrajectoryRecord, so we exercise them on a freshly-built
+model plus small hand-constructed records.
+"""
+
+import numpy as np
+
+from edc.config import load_config
+from edc.energy import mlp_ebm
+from edc.eval.metrics import single_feature_auroc
+from edc.geometry.basin import basin_features
+from edc.geometry.curvature import (
+    batched_curvature,
+    energy_at,
+    hutchinson_trace,
+    lambda_max,
+)
+from edc.geometry.dynamics import dynamics_features
+from edc.geometry.energy_stats import energy_features
+from edc.geometry.features import geometry_features
+from edc.inference import restarts
+from edc.inference.trajectory import TrajectoryRecord
+from edc.seeding import numpy_rng, root_key
+from edc.tasks.arithmetic import ArithmeticTask
+
+N_EXPECTED_FEATURES = 14  # basin 3 + energy 3 + curvature 4 + dynamics 4
+
+
+def _toy_traj(pred, z_star, energies, grad_norms, n_classes):
+    """Build a TrajectoryRecord from NumPy arrays (logits/h_x are placeholders for the
+    basin/energy/dynamics numpy features, which never touch them)."""
+    pred = np.asarray(pred)
+    B, K = pred.shape
+    logits = np.zeros((B, K, n_classes))
+    return TrajectoryRecord(
+        z_star=np.asarray(z_star, dtype=float),
+        energies=np.asarray(energies, dtype=float),
+        grad_norms=np.asarray(grad_norms, dtype=float),
+        logits=logits,
+        pred=pred,
+        h_x=np.zeros((B, 4)),
+    )
+
+
+# ------------------------------- basin / energy / dynamics exact values -----------------------
+
+
+def test_basin_unanimous_vs_split():
+    # Two inputs: input 0 all restarts agree (class 2); input 1 splits 2-2.
+    pred = np.array([[2, 2, 2, 2], [0, 0, 1, 1]])
+    z = np.zeros((2, 4, 3))            # identical endpoints => zero dispersion
+    feats, names = basin_features(_toy_traj(pred, z, np.zeros((2, 4, 2)), np.zeros((2, 4, 1)), 3))
+    assert names == ["basin/rho", "basin/entropy", "basin/dispersion"]
+
+    rho, entropy, dispersion = feats[:, 0], feats[:, 1], feats[:, 2]
+    assert rho[0] == 1.0 and entropy[0] == 0.0            # unanimous
+    assert np.isclose(rho[1], 0.5)
+    assert np.isclose(entropy[1], np.log(2))             # 2-2 split -> ln 2 nats
+    assert np.allclose(dispersion, 0.0)
+
+
+def test_basin_dispersion_grows_with_spread():
+    pred = np.zeros((1, 2), dtype=int)
+    z_close = np.array([[[0.0, 0.0], [0.1, 0.0]]])
+    z_far = np.array([[[0.0, 0.0], [3.0, 4.0]]])          # pairwise distance 5
+    zeros = (np.zeros((1, 2, 2)), np.zeros((1, 2, 1)))
+    d_close = basin_features(_toy_traj(pred, z_close, *zeros, 1))[0][0, 2]
+    d_far = basin_features(_toy_traj(pred, z_far, *zeros, 1))[0][0, 2]
+    assert np.isclose(d_close, 0.1)
+    assert np.isclose(d_far, 5.0)
+    assert d_far > d_close
+
+
+def test_energy_stats_order():
+    # terminal energy = energies[..., -1]. Input 0 restarts end at {1, 3}; input 1 at {2, 2}.
+    energies = np.array([[[9, 1], [9, 3]], [[9, 2], [9, 2]]], dtype=float)  # (B=2, K=2, T+1=2)
+    feats, names = energy_features(_toy_traj(np.zeros((2, 2), int), np.zeros((2, 2, 1)),
+                                             energies, np.zeros((2, 2, 1)), 1))
+    assert names == ["energy/mean", "energy/min", "energy/std"]
+    emean, emin, estd = feats[:, 0], feats[:, 1], feats[:, 2]
+    assert np.allclose(emean, [2.0, 2.0]) and np.allclose(emin, [1.0, 2.0])
+    assert np.all(emin <= emean + 1e-9)
+    assert np.isclose(estd[1], 0.0)                       # identical -> zero spread
+
+
+def test_dynamics_values():
+    # One input, one restart. Energy strictly decreasing 4->3->2->1 (T=3 steps): monotonic=1.
+    energies = np.array([[[4.0, 3.0, 2.0, 1.0]]])         # (1, 1, 4)
+    grad_norms = np.array([[[0.5, 0.4, 1e-4]]])           # converges at step 2 (< tol)
+    traj = _toy_traj(np.zeros((1, 1), int), np.zeros((1, 1, 1)), energies, grad_norms, 1)
+    feats, names = dynamics_features(traj, grad_tol=1e-3)
+    assert names == ["dynamics/steps", "dynamics/monotonic", "dynamics/drop", "dynamics/residual"]
+    steps, monotonic, drop, residual = feats[0]
+    assert np.isclose(steps, 2 / 3)                       # first below-tol step is index 2, /T
+    assert np.isclose(monotonic, 1.0)
+    assert np.isclose(drop, 3.0)                          # 4 - 1
+    assert np.isclose(residual, 1e-4)                     # terminal grad norm
+
+
+def test_dynamics_steps_defaults_to_T_when_never_converged():
+    grad_norms = np.array([[[1.0, 1.0, 1.0]]])            # never below tol
+    traj = _toy_traj(np.zeros((1, 1), int), np.zeros((1, 1, 1)),
+                     np.zeros((1, 1, 4)), grad_norms, 1)
+    steps = dynamics_features(traj, grad_tol=1e-3)[0][0, 0]
+    assert np.isclose(steps, 1.0)                         # T/T
+
+
+# ------------------------------- single-feature AUROC -----------------------------------------
+
+
+def test_single_feature_auroc():
+    correct = np.array([True, True, False, False])
+    assert single_feature_auroc(np.array([3.0, 4.0, 1.0, 2.0]), correct) == 1.0   # ordered
+    assert single_feature_auroc(np.array([1.0, 2.0, 3.0, 4.0]), correct) == 0.0   # reversed
+    assert single_feature_auroc(np.array([1.0, 1.0, 1.0, 1.0]), correct) == 0.5   # all ties
+    assert single_feature_auroc(np.array([1.0, 2.0, 3.0]), np.array([True, True, True])) == 0.5
+
+
+def test_auroc_near_half_on_noise():
+    rng = np.random.default_rng(0)
+    feature = rng.standard_normal(2000)
+    correct = rng.integers(0, 2, size=2000).astype(bool)
+    assert abs(single_feature_auroc(feature, correct) - 0.5) < 0.05
+
+
+# ------------------------------- batched curvature --------------------------------------------
+
+
+def test_batched_curvature_matches_quadratic():
+    d, n = 5, 3
+    A = np.random.default_rng(0).standard_normal((d, d)).astype(np.float32)
+    A = A @ A.T + np.eye(d, dtype=np.float32)             # SPD
+
+    def energy(_params, _h, z):                           # ignores params/context
+        return 0.5 * np.einsum("ni,ij,nj->n", np.asarray(z), A, np.asarray(z))
+
+    import jax.numpy as jnp
+
+    def jax_energy(_params, _h, z):
+        return 0.5 * jnp.einsum("ni,ij,nj->n", z, jnp.asarray(A), z)
+
+    zs = jnp.asarray(np.random.default_rng(1).standard_normal((n, d)).astype(np.float32))
+    contexts = jnp.zeros((n, 1))
+    lmax, trace = batched_curvature(jax_energy, {}, contexts, zs, root_key(3),
+                                    iters=60, n_probes=256)
+    top = float(np.linalg.eigvalsh(A)[-1])
+    assert np.allclose(np.asarray(lmax), top, rtol=2e-2)  # Hessian is constant A for every z
+    assert np.allclose(np.asarray(trace), float(np.trace(A)), rtol=0.15)
+    _ = energy  # numpy reference kept for documentation
+
+
+def test_batched_curvature_matches_per_particle_loop():
+    # Batched wrapper must equal the per-particle primitives with the same derived keys.
+    import jax
+
+    cfg = load_config("configs/smoke.toml")
+    task = ArithmeticTask()
+    params, fns, _ = mlp_ebm.build(cfg, task.n_classes, task.feature_dim, root_key(0))
+
+    import jax.numpy as jnp
+
+    batch = task.sample(numpy_rng(0, 1), 3, "id")
+    traj = restarts.solve(fns, params, jnp.asarray(batch.x), cfg, root_key(0))
+
+    B, K, dlat = np.asarray(traj.z_star).shape
+    contexts = jnp.asarray(np.repeat(np.asarray(traj.h_x), K, axis=0))
+    zs = jnp.asarray(np.asarray(traj.z_star).reshape(B * K, dlat))
+    key = root_key(7)
+
+    lmax_b, trace_b = batched_curvature(fns.energy, params, contexts, zs, key)
+
+    N = contexts.shape[0]
+    keys = jax.random.split(key, N)
+    lmax_ref, trace_ref = [], []
+    for i in range(N):
+        k_lam, k_tr = jax.random.split(keys[i])
+        f = energy_at(fns.energy, params, contexts[i])
+        lmax_ref.append(float(lambda_max(f, zs[i], k_lam)))
+        trace_ref.append(float(hutchinson_trace(f, zs[i], k_tr)))
+    assert np.allclose(np.asarray(lmax_b), lmax_ref, atol=1e-4)
+    assert np.allclose(np.asarray(trace_b), trace_ref, atol=1e-4)
+
+
+# ------------------------------- full assembly on a real model --------------------------------
+
+
+def _real_traj(seed=0, n=8):
+    import jax.numpy as jnp
+
+    cfg = load_config("configs/smoke.toml")
+    task = ArithmeticTask()
+    params, fns, _ = mlp_ebm.build(cfg, task.n_classes, task.feature_dim, root_key(seed))
+    batch = task.sample(numpy_rng(seed, 1), n, "id")
+    traj = restarts.solve(fns, params, jnp.asarray(batch.x), cfg, root_key(seed))
+    return traj, fns, params, cfg
+
+
+def test_geometry_features_shape_and_names():
+    traj, fns, params, cfg = _real_traj(n=8)
+    feats, names = geometry_features(traj, fns, params, root_key(1),
+                                     grad_tol=cfg.inference.grad_tol)
+    assert feats.shape == (8, N_EXPECTED_FEATURES)
+    assert len(names) == N_EXPECTED_FEATURES
+    assert len(set(names)) == N_EXPECTED_FEATURES        # unique
+    assert np.all(np.isfinite(feats))
+
+
+def test_geometry_features_deterministic():
+    traj, fns, params, cfg = _real_traj(n=6)
+    f1, _ = geometry_features(traj, fns, params, root_key(2), grad_tol=cfg.inference.grad_tol)
+    f2, _ = geometry_features(traj, fns, params, root_key(2), grad_tol=cfg.inference.grad_tol)
+    assert np.array_equal(f1, f2)
diff --git a/tests/test_graph_task.py b/tests/test_graph_task.py
new file mode 100644
index 0000000..35160e7
--- /dev/null
+++ b/tests/test_graph_task.py
@@ -0,0 +1,67 @@
+"""GraphPlanningTask: BFS shortest-path labels, fixed-size padded encoding, size-shift OOD, and
+determinism. Offline + CPU-only, pure NumPy (no training).
+"""
+
+import numpy as np
+
+from edc.registry import build_task
+from edc.tasks.graph_planning import _MAX_NODES, GraphPlanningTask, _shortest_len
+
+
+def _chain(n):
+    adj = np.zeros((n, n))
+    for a in range(n - 1):
+        adj[a, a + 1] = adj[a + 1, a] = 1.0
+    return adj
+
+
+def test_bfs_shortest_len():
+    adj = _chain(5)                                   # 0-1-2-3-4
+    assert _shortest_len(adj, 0, 4, cap=5) == 4
+    assert _shortest_len(adj, 1, 3, cap=5) == 2
+    assert _shortest_len(adj, 0, 1, cap=5) == 1
+    # cap: a path longer than cap reads as 0 (unreachable-or-farther)
+    assert _shortest_len(adj, 0, 4, cap=3) == 0
+    # genuinely unreachable
+    disconnected = np.array([[0, 1, 0, 0], [1, 0, 0, 0], [0, 0, 0, 1], [0, 0, 1, 0]], float)
+    assert _shortest_len(disconnected, 0, 3, cap=5) == 0
+
+
+def test_sample_shapes_and_labels_in_range():
+    task = GraphPlanningTask(n_nodes=7, ood_n_nodes=10, edge_prob=0.4, max_len=4)
+    assert task.feature_dim == _MAX_NODES * _MAX_NODES + 2 * _MAX_NODES
+    assert task.n_classes == 5
+    b = task.sample(np.random.default_rng(0), 200, "id")
+    assert b.x.shape == (200, task.feature_dim)
+    assert b.y.shape == (200,)
+    assert b.y.min() >= 0 and b.y.max() < task.n_classes
+
+
+def test_encoded_adjacency_symmetric_and_onehots():
+    task = GraphPlanningTask(n_nodes=7)
+    b = task.sample(np.random.default_rng(1), 5, "id")
+    adj_block = _MAX_NODES * _MAX_NODES
+    for i in range(5):
+        adj = b.x[i, :adj_block].reshape(_MAX_NODES, _MAX_NODES)
+        assert np.allclose(adj, adj.T)                # undirected
+        src = b.x[i, adj_block:adj_block + _MAX_NODES]
+        dst = b.x[i, adj_block + _MAX_NODES:]
+        assert src.sum() == 1.0 and dst.sum() == 1.0  # one source, one target
+        assert int(src.argmax()) != int(dst.argmax())  # distinct
+
+
+def test_ood_uses_more_nodes_same_feature_dim():
+    task = GraphPlanningTask(n_nodes=7, ood_n_nodes=10)
+    id_b = task.sample(np.random.default_rng(2), 300, "id")
+    ood_b = task.sample(np.random.default_rng(2), 300, "ood")
+    assert id_b.x.shape[1] == ood_b.x.shape[1] == task.feature_dim
+    # OOD graphs use more nodes, so on average more edges are active
+    assert task.difficulty(ood_b).mean() > task.difficulty(id_b).mean()
+
+
+def test_determinism_and_registration():
+    task = build_task("graph_planning", n_nodes=7, edge_prob=0.4)
+    assert isinstance(task, GraphPlanningTask)
+    a = task.sample(np.random.default_rng(3), 50, "id")
+    b = task.sample(np.random.default_rng(3), 50, "id")
+    assert np.array_equal(a.x, b.x) and np.array_equal(a.y, b.y)
diff --git a/tests/test_halting.py b/tests/test_halting.py
new file mode 100644
index 0000000..023cfa6
--- /dev/null
+++ b/tests/test_halting.py
@@ -0,0 +1,79 @@
+"""Adaptive halting: exact per-step agreement, stop-step policy, halting losses, and CRC
+calibration (with the lambda = 1 - tau orientation). Offline + CPU-only, hand-built trajectories.
+"""
+
+import numpy as np
+
+from edc.halting import adaptive as H
+
+
+def test_per_step_agreement_exact():
+    # 1 input, K=4 restarts, T+1=3 steps. t0: {0,1,2,3} all differ -> 1/4; t1: {5,5,5,6} -> 3/4;
+    # t2: all 5 -> 1.0.
+    step_pred = np.array([[[0, 5, 5], [1, 5, 5], [2, 5, 5], [3, 6, 5]]])   # (1,4,3)
+    agr = H.per_step_agreement(step_pred)
+    assert np.allclose(agr[0], [0.25, 0.75, 1.0])
+
+
+def test_halting_policy_first_crossing_else_full():
+    a = np.array([0.25, 0.5, 0.75, 1.0])
+    assert H.halting_policy(a, 0.75) == 2          # first step reaching 0.75
+    assert H.halting_policy(a, 1.0) == 3           # only the last step is unanimous
+    assert H.halting_policy(a, 1.5) == 3           # never reached -> full budget (last index)
+
+
+def test_halting_losses_early_vs_full():
+    # K=2, T+1=3. Restart 0 always lowest energy => best-of-N follows restart 0.
+    # Restart-0 answers: step0=9, step1=9, step2=7 (full). Agreement: t0 {9,1}->0.5, t1 {9,2}->0.5,
+    # t2 {7,7}->1.0. At tau=1.0 we stop at t2 => early == full (7) => loss 0. At tau=0.5 we stop at
+    # t0 => early=9 != full=7 => loss 1, compute 0.
+    step_pred = np.array([[[9, 9, 7], [1, 2, 7]]])                          # (1,2,3)
+    energies = np.array([[[0.0, 0.0, 0.0], [5.0, 5.0, 5.0]]])               # restart 0 lower
+    loss, compute = H.halting_losses(step_pred, energies, taus=[1.0, 0.5])
+    assert loss[0, 0] == 0.0 and np.isclose(compute[0, 0], 1.0)             # tau=1.0: full budget
+    assert loss[0, 1] == 1.0 and np.isclose(compute[0, 1], 0.0)             # tau=0.5: halt at t0
+
+
+def test_halted_predictions_matches_policy():
+    step_pred = np.array([[[9, 9, 7], [1, 2, 7]]])
+    energies = np.array([[[0.0, 0.0, 0.0], [5.0, 5.0, 5.0]]])
+    early, compute = H.halted_predictions(step_pred, energies, tau=1.0)
+    assert early[0] == 7 and np.isclose(compute[0], 1.0)
+
+
+def _synthetic_cal(n=400, seed=0):
+    """Build (step_pred, energies) where higher tau (later stop) strictly lowers disagreement:
+    each input agrees early with prob p; a late-stop always matches the full answer."""
+    rng = np.random.default_rng(seed)
+    K, T1 = 3, 5
+    step_pred = np.zeros((n, K, T1), dtype=int)
+    energies = np.zeros((n, K, T1))
+    energies[:, 1:, :] = 1.0                       # restart 0 always lowest -> best-of-N follows it
+    full = rng.integers(0, 5, size=n)
+    for b in range(n):
+        step_pred[b, 0, -1] = full[b]             # full answer at last step
+        early_ok = rng.random() < 0.7
+        step_pred[b, 0, :-1] = full[b] if early_ok else (full[b] + 1) % 5   # early maybe wrong
+        step_pred[b, 1:, :] = (full[b] + 2) % 5  # other restarts disagree -> low early agreement
+        step_pred[b, :, -1] = full[b]            # all agree at the end -> agreement 1.0 at end
+    return step_pred, energies
+
+
+def test_crc_calibration_controls_halting_risk():
+    step_pred, energies = _synthetic_cal(n=600, seed=1)
+    alpha, K = 0.1, 3
+    taus = np.round(np.linspace(1.0 / K, 1.0, K), 4)
+    out = H.calibrate(step_pred, energies, alpha, taus)
+    # A feasible budget picks a tau whose empirical risk is within alpha; risk falls as tau rises.
+    assert out["tau_hat"] is not None
+    risk = np.array(out["risk_by_tau"])
+    assert np.all(np.diff(risk) <= 1e-9)                     # non-increasing in tau (monotone)
+    chosen_idx = list(taus).index(out["tau_hat"])
+    assert risk[chosen_idx] <= alpha                         # chosen point respects the budget
+
+
+def test_calibrate_infeasible_budget_returns_none():
+    step_pred, energies = _synthetic_cal(n=200, seed=2)
+    out = H.calibrate(step_pred, energies, alpha=0.0, taus=[0.34, 0.67, 1.0])
+    # alpha=0 with finite calibration is structurally infeasible (CRC (n+1) correction) -> None.
+    assert out["tau_hat"] is None
diff --git a/tests/test_inference_record_steps.py b/tests/test_inference_record_steps.py
new file mode 100644
index 0000000..3e23d0e
--- /dev/null
+++ b/tests/test_inference_record_steps.py
@@ -0,0 +1,44 @@
+"""solve(record_steps=True) populates a correct per-step decode trajectory; default leaves it None
+and does not perturb the descent. Marked slow (builds a tiny model). Offline + CPU-only.
+"""
+
+import jax.numpy as jnp
+import numpy as np
+import pytest
+
+from edc.config import load_config
+from edc.energy import mlp_ebm
+from edc.inference import restarts
+from edc.seeding import numpy_rng, root_key
+from edc.tasks.arithmetic import ArithmeticTask
+
+pytestmark = pytest.mark.slow
+
+
+def _model():
+    cfg = load_config("configs/smoke.toml")
+    task = ArithmeticTask()
+    params, fns, _ = mlp_ebm.build(cfg, task.n_classes, task.feature_dim, root_key(0))
+    x = jnp.asarray(task.sample(numpy_rng(0, 1), 6, "id").x)
+    return cfg, fns, params, x
+
+
+def test_record_steps_shape_and_final_consistency():
+    cfg, fns, params, x = _model()
+    traj = restarts.solve(fns, params, x, cfg, root_key(0), record_steps=True)
+    B, K = np.asarray(traj.pred).shape
+    T1 = np.asarray(traj.energies).shape[-1]
+    sp = np.asarray(traj.step_pred)
+    assert sp.shape == (B, K, T1)
+    # the last recorded step is exactly the final decoded prediction
+    assert np.array_equal(sp[:, :, -1], np.asarray(traj.pred))
+
+
+def test_record_steps_off_is_none_and_nonintrusive():
+    cfg, fns, params, x = _model()
+    on = restarts.solve(fns, params, x, cfg, root_key(0), record_steps=True)
+    off = restarts.solve(fns, params, x, cfg, root_key(0))
+    assert off.step_pred is None
+    # recording must not change the descent itself
+    assert np.allclose(np.asarray(on.energies), np.asarray(off.energies))
+    assert np.array_equal(np.asarray(on.pred), np.asarray(off.pred))
diff --git a/tests/test_make_tables.py b/tests/test_make_tables.py
new file mode 100644
index 0000000..9b364e3
--- /dev/null
+++ b/tests/test_make_tables.py
@@ -0,0 +1,51 @@
+"""LaTeX table generation smoke test: T1/T2/T3 are well-formed from synthetic ledger rows.
+
+Offline + CPU-only. Checks structure (booktabs rules, labels, per-task T1, per-K T2) without
+asserting exact numbers.
+"""
+
+import aggregate
+import make_tables
+from test_aggregate import _row
+
+
+def _rows():
+    rows = []
+    for k in (1, 4, 16):                                     # arithmetic K-sweep
+        lift = 0.0 if k == 1 else 0.05
+        for s in range(3):
+            rows.append(_row(k, s, lift, lift - 0.01, geo=0.1 - lift, best=0.14,
+                             baseline_delta=(lift - 0.02, lift - 0.03)))
+    for s in range(3):                                       # a second task at one K
+        rows.append(_row(12, s, 0.03, 0.01, geo=0.09, best=0.13, task="graph_planning",
+                         baseline_delta=(0.01, -0.01)))
+    return rows
+
+
+def test_tables_well_formed():
+    rows = _rows()
+
+    t1 = make_tables._t1(rows)                               # per-task
+    t2 = make_tables._t2(aggregate.aggregate_by_k(rows, task="arithmetic"))
+    t2b = make_tables._t2b(rows)                             # feature ablation
+    t3 = make_tables._t3(rows)
+
+    for tex in (t1, t2, t2b, t3):
+        assert r"\toprule" in tex and r"\bottomrule" in tex and r"\begin{tabular}" in tex
+
+    assert r"\label{tab:main}" in t1
+    assert r"\label{tab:kablation}" in t2
+    assert r"\label{tab:ablation}" in t2b and "drop\\_energy" in t2b
+    # T1 has one row per task (arithmetic + graph_planning)
+    t1_body = t1.split(r"\midrule")[1].split(r"\bottomrule")[0]
+    assert t1_body.count(r"\\") == 2
+    assert "graph" in t1 and "arithmetic" in t1
+    # T2 has one data row per arithmetic K (1, 4, 16)
+    t2_body = t2.split(r"\midrule")[1].split(r"\bottomrule")[0]
+    assert t2_body.count(r"\\") == 3
+    # T3 lists every run
+    assert t3.count(r"\\") >= len(rows)
+
+
+def test_delta_pm_formatting():
+    assert make_tables._pm((0.0736, 0.012)) == r"$0.074 \pm 0.012$"
diff --git a/tests/test_nonconformity.py b/tests/test_nonconformity.py
new file mode 100644
index 0000000..df20c67
--- /dev/null
+++ b/tests/test_nonconformity.py
@@ -0,0 +1,42 @@
+"""Nonconformity mapper: learned score is monotone in a planted signal, standardisation travels
+with the pickled object, the rho-basin fallback is exact, and a single-class fit fold degrades
+gracefully. Offline + CPU-only.
+"""
+
+import pickle
+
+import numpy as np
+
+from edc.conformal.nonconformity import fit_mapper, rho_basin_score, score
+
+
+def test_score_monotone_in_planted_signal():
+    rng = np.random.default_rng(0)
+    x = rng.standard_normal((400, 3))
+    correct = (x[:, 0] + rng.standard_normal(400) * 0.3) > 0   # feature 0 predicts correctness
+    mapper = fit_mapper(x, correct)
+    s = score(mapper, x)
+    # higher feature 0 -> more likely correct -> higher p_hat -> LOWER nonconformity score
+    assert np.corrcoef(s, x[:, 0])[0, 1] < -0.5
+    assert np.all((s >= 0.0) & (s <= 1.0))
+
+
+def test_standardisation_travels_through_pickle():
+    rng = np.random.default_rng(1)
+    x = rng.standard_normal((200, 4)) * 100.0 + 5.0            # off-scale features
+    correct = x[:, 1] > x[:, 1].mean()
+    mapper = fit_mapper(x, correct)
+    s = score(mapper, x)
+    reloaded = pickle.loads(pickle.dumps(mapper))
+    assert np.allclose(s, score(reloaded, x))                 # scaler stats persist with the model
+
+
+def test_rho_basin_score_exact():
+    assert np.allclose(rho_basin_score([1.0, 0.5, 0.25]), [0.0, 0.5, 0.75])
+
+
+def test_single_class_fit_fold():
+    rng = np.random.default_rng(2)
+    x = rng.standard_normal((50, 3))
+    assert np.all(score(fit_mapper(x, np.zeros(50, bool)), x) == 1.0)   # all-wrong  -> s = 1
+    assert np.all(score(fit_mapper(x, np.ones(50, bool)), x) == 0.0)    # all-correct -> s = 0
diff --git a/tests/test_ood_validity.py b/tests/test_ood_validity.py
new file mode 100644
index 0000000..a1462ee
--- /dev/null
+++ b/tests/test_ood_validity.py
@@ -0,0 +1,47 @@
+"""F5/F6 metric blocks from the evaluate pipeline: OOD selective-risk validity (guarantee applied
+under shift) and per-feature diagnostics. Marked slow (it trains). Offline + CPU-only.
+"""
+
+import json
+
+import numpy as np
+import pytest
+
+from edc.config import load_config
+from edc.eval.evaluate import _ALPHA_GRID, evaluate
+from edc.registry import build_task
+
+pytestmark = pytest.mark.slow
+
+
+def test_ood_validity_and_feature_diagnostics_present():
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    m = evaluate(cfg, task, include_ood=True)
+
+    # F6: ood_validity mirrors the alpha grid with ID + OOD achieved risk, and an ood_ltt block.
+    ov = m["ood_validity"]
+    n = len(_ALPHA_GRID)
+    for key in ("target", "id_risk", "ood_risk", "id_coverage", "ood_coverage"):
+        assert len(ov[key]) == n
+    assert set(m["ood_ltt"]) >= {"alpha", "lambda_hat", "selective_risk", "coverage",
+                                 "risk_within_budget"}
+
+    # F5: 14 named per-feature AUROCs + histograms whose counts sum to the class sizes.
+    fd = m["feature_diagnostics"]
+    assert len(fd["auroc"]) == 14 and len(fd["names"]) == 14
+    n_correct = int(np.round(m["accuracy_id"] * m["n_test"]))
+    h = fd["hist"][fd["names"][0]]
+    assert len(h["edges"]) == 21                                  # 20 bins
+    assert sum(h["correct_counts"]) == n_correct
+    assert sum(h["correct_counts"]) + sum(h["incorrect_counts"]) == m["n_test"]
+
+    json.dumps(m)  # ledger-serialisable
+
+
+def test_include_ood_false_omits_f6_blocks():
+    cfg = load_config("configs/smoke.toml")
+    task = build_task(cfg.run.task)
+    m = evaluate(cfg, task, include_ood=False)
+    assert "ood_validity" not in m and "ood_ltt" not in m
+    assert "feature_diagnostics" in m                             # F5 does not need the OOD fold
diff --git a/tests/test_plotting_f5_f6.py b/tests/test_plotting_f5_f6.py
new file mode 100644
index 0000000..e6f3ca4
--- /dev/null
+++ b/tests/test_plotting_f5_f6.py
@@ -0,0 +1,52 @@
+"""F5/F6 plotting: renders a PNG from a synthetic ledger row and skips cleanly when the block is
+absent. Offline + CPU-only; skipped if matplotlib (the optional plot extra) is unavailable.
+"""
+
+import numpy as np
+import pytest
+
+pytest.importorskip("matplotlib")
+
+from edc import plotting  # noqa: E402
+
+
+def _row_with_blocks():
+    names = ["basin/rho", "basin/entropy", "curv/lmax_best", "energy/min", "dynamics/steps"]
+    rng = np.random.default_rng(0)
+    hist = {n: {"edges": np.linspace(0, 1, 21).tolist(),
+                "correct_counts": rng.integers(0, 30, 20).tolist(),
+                "incorrect_counts": rng.integers(0, 10, 20).tolist()} for n in names}
+    metrics = {
+        "n_test": 500,
+        "feature_diagnostics": {
+            "names": names,
+            "auroc": {"basin/rho": 0.78, "basin/entropy": 0.30, "curv/lmax_best": 0.66,
+                      "energy/min": 0.55, "dynamics/steps": 0.50},
+            "hist": hist,
+        },
+        "ood_validity": {
+            "target": [0.05, 0.1, 0.2, 0.3],
+            "id_risk": [0.03, 0.08, 0.15, 0.22],       # ID stays on/below diagonal
+            "ood_risk": [0.4, 0.5, 0.6, 0.7],          # OOD breaks above diagonal
+            "id_coverage": [0.2, 0.5, 0.8, 0.95],
+            "ood_coverage": [0.3, 0.6, 0.85, 0.97],
+        },
+    }
+    return [{"split": "selective", "metrics": metrics}]
+
+
+def test_f5_f6_render(tmp_path):
+    rows = _row_with_blocks()
+    f5 = plotting.feature_diagnostics(rows, str(tmp_path / "f5.png"))
+    f6 = plotting.ood_stress(rows, str(tmp_path / "f6.png"))
+    assert (tmp_path / "f5.png").stat().st_size > 0
+    assert (tmp_path / "f6.png").stat().st_size > 0
+    assert f5.endswith("f5.png") and f6.endswith("f6.png")
+
+
+def test_missing_blocks_raise_valueerror(tmp_path):
+    rows = [{"split": "selective", "metrics": {"n_test": 10}}]   # no F5/F6 blocks
+    with pytest.raises(ValueError):
+        plotting.feature_diagnostics(rows, str(tmp_path / "f5.png"))
+    with pytest.raises(ValueError):
+        plotting.ood_stress(rows, str(tmp_path / "f6.png"))
diff --git a/tests/test_run_sweep.py b/tests/test_run_sweep.py
new file mode 100644
index 0000000..36f198d
--- /dev/null
+++ b/tests/test_run_sweep.py
@@ -0,0 +1,41 @@
+"""Sweep grid expansion and dotted-key overrides — pure config manipulation, no training.
+
+Offline + CPU-only: exercises only the parsing/expansion helpers of experiments/run_sweep.py
+(the per-cell training is covered by the slow pipeline test).
+"""
+
+import run_sweep
+
+
+def test_set_dotted_nested():
+    d = {"inference": {"k_restarts": 8}}
+    run_sweep._set_dotted(d, "inference.k_restarts", 16)
+    run_sweep._set_dotted(d, "run.seed", 3)          # creates the intermediate dict
+    assert d["inference"]["k_restarts"] == 16
+    assert d["run"]["seed"] == 3
+
+
+def test_expand_grid_cartesian():
+    cells = run_sweep.expand_grid({"a": [1, 2], "b": [10, 20, 30]})
+    assert len(cells) == 6
+    assert {"a": 1, "b": 10} in cells and {"a": 2, "b": 30} in cells
+    assert run_sweep.expand_grid({}) == [{}]          # empty grid -> one no-op cell
+
+
+def test_cell_config_applies_override_and_cell():
+    base = {"inference": {"k_restarts": 8, "steps": 50}, "eval": {"n_eval": 1500},
+            "run": {"seed": 0}}
+    override = {"eval.n_eval": 600}
+    cell = {"inference.k_restarts": 4, "run.seed": 2}
+    d = run_sweep.cell_config_dict(base, override, cell)
+    assert d["eval"]["n_eval"] == 600                 # override applied
+    assert d["inference"]["k_restarts"] == 4 and d["run"]["seed"] == 2   # cell applied
+    assert d["inference"]["steps"] == 50              # untouched key preserved
+    assert base["inference"]["k_restarts"] == 8       # base not mutated (deep-copied)
+
+
+def test_load_sweep_reads_real_config():
+    base, override, cells = run_sweep.load_sweep("configs/sweeps/k_restarts.toml")
+    assert len(cells) == 25                           # 5 K x 5 seeds
+    assert override["eval.n_eval"] == 600
+    assert base["task"]["arithmetic"]["n_operands"] == 6   # base experiment config resolved
diff --git a/tests/test_selective.py b/tests/test_selective.py
new file mode 100644
index 0000000..ef827ad
--- /dev/null
+++ b/tests/test_selective.py
@@ -0,0 +1,28 @@
+"""Selective-prediction wrapper: exact abstain behaviour (boundary answers) and the coverage
+identity linking it to the conformal layer. Offline + CPU-only.
+"""
+
+import numpy as np
+
+from edc.conformal.selective import ABSTAIN, selective_predict
+from edc.conformal.split_conformal import empirical_coverage
+
+
+def test_selective_predict_exact_with_boundary():
+    ans = selective_predict([3, 1, 4, 1], [0.1, 0.9, 0.5, 0.5], threshold=0.5)
+    assert list(ans) == [3, ABSTAIN, 4, 1]                 # s == threshold answers
+
+
+def test_selective_predict_extremes():
+    pred = np.array([7, 8, 9])
+    assert list(selective_predict(pred, [0.1, 0.2, 0.3], float("inf"))) == [7, 8, 9]  # admit all
+    assert list(selective_predict(pred, [0.1, 0.2, 0.3], float("-inf"))) == [ABSTAIN] * 3
+
+
+def test_coverage_matches_empirical_coverage():
+    rng = np.random.default_rng(0)
+    scores = rng.random(500)
+    pred = rng.integers(0, 5, size=500)
+    thr = 0.4
+    answers = selective_predict(pred, scores, thr)
+    assert np.isclose((answers != ABSTAIN).mean(), empirical_coverage(scores, thr))