MinishLab · Pringled · Aug 1, 2026 · Jul 25, 2026 · Jul 25, 2026 · Jul 27, 2026
diff --git a/README.md b/README.md
@@ -1,7 +1,7 @@
 <h2 align="center">
   <img width="30%" alt="semble logo" src="https://raw.githubusercontent.com/MinishLab/semble/main/assets/images/semble_logo.png"><br/>
   Fast and Accurate Code Search for Agents<br/>
-  <sub>Uses ~98% fewer tokens than grep+read</sub>
+  <sub>Uses ~99% fewer tokens than grep+read</sub>
 </h2>
 
 <div align="center">
@@ -24,7 +24,7 @@
 
 </div>
 
-Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~98% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, with ~200x faster indexing and ~10x faster queries than a code-specialized transformer, at 99% of its retrieval quality (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo.
+Semble is a code search library built for agents. It returns the exact code snippets they need instantly, using ~99% fewer tokens than grep+read. Indexing and searching a full codebase end-to-end takes under a second, matching the retrieval quality of a code-specialized transformer while indexing ~220x faster and querying ~17x faster (see [benchmarks](#benchmarks)). Everything runs on CPU with no API keys, GPU, or external services. Use it as an MCP server, a CLI tool via AGENTS.md, or a dedicated sub-agent, and any coding agent (Claude Code, Cursor, Codex, OpenCode, etc.) gets instant access to any repo.
 
 ## Quickstart
 
@@ -72,9 +72,9 @@ semble install --agent claude --type mcp subagent --yes
 
 ## Main Features
 
-- **Fast**: indexes an average repo in ~250 ms and answers queries in ~1.5 ms, all on CPU.
+- **Fast**: indexes an average repo in ~500 ms and answers queries in ~1 ms, all on CPU.
 - **Accurate**: NDCG@10 of 0.854 on our [benchmarks](#benchmarks), on par with code-specialized transformer models, at a fraction of the size and cost.
-- **Token-efficient**: returns only the relevant chunks, using [~98% fewer tokens than grep+read](#benchmarks).
+- **Token-efficient**: returns only the relevant chunks, using [~99% fewer tokens than grep+read](#benchmarks).
 - **Zero setup**: runs on CPU with no API keys, GPU, or external services required.
 - **MCP server**: works with Claude Code, Cursor, Codex, OpenCode, VS Code, and any other MCP-compatible agent.
 - **Local and remote**: pass a local path or a git URL.
@@ -235,7 +235,7 @@ We benchmark quality and speed across ~1,250 queries over 63 repositories in 19
 </tr>
 </table>
 
-The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble achieves 99% of the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) Hybrid while indexing 218x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 98% fewer tokens on average and hits 94% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology.
+The quality benchmark (left) scores retrieval quality (NDCG@10) against total latency; semble matches the quality of the 137M-parameter [CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed) while indexing 220x faster. The token efficiency benchmark (right) measures how many tokens each method needs to reach a given recall level; semble uses 99% fewer tokens on average and hits 97% recall at only 2k tokens, while grep+read needs a full 100k context window to reach 85%. See [benchmarks](benchmarks/README.md) for per-language results, ablations, and full methodology.
 
 ## How it works
 

diff --git a/assets/images/speed_vs_ndcg_cold.png b/assets/images/speed_vs_ndcg_cold.png
diff --git a/assets/images/speed_vs_ndcg_warm.png b/assets/images/speed_vs_ndcg_warm.png
diff --git a/assets/images/token_efficiency.png b/assets/images/token_efficiency.png
diff --git a/benchmarks/README.md b/benchmarks/README.md
@@ -15,22 +15,24 @@ Quality and speed benchmarks for `semble`.
 
 Quality and speed across all methods.
 
-| Method | NDCG@10 | Index | Query p50 |
-|---|---:|---:|---:|
-| CodeRankEmbed Hybrid | 0.862 | 57 s | 16 ms |
-| **semble** | **0.854** | **263 ms** | **1.5 ms** |
-| CodeRankEmbed | 0.765 | 57 s | 16 ms |
-| ColGREP | 0.693 | 5.8 s | 124 ms |
-| BM25 | 0.673 | 263 ms | 0.02 ms |
-| grepai | 0.561 | 35 s | 48 ms |
-| probe | 0.387 | — | 207 ms |
-| ripgrep | 0.126 | — | 12 ms |
+| Method               |   NDCG@10 |      Index |   Query p50 |
+| -------------------- | --------: | ---------: | ----------: |
+| **semble**           | **0.854** | **518 ms** | **0.91 ms** |
+| CodeRankEmbed        |     0.839 |      116 s |       16 ms |
+| ColGREP              |     0.693 |      5.4 s |      122 ms |
+| BM25                 |     0.673 |      47 ms |     0.17 ms |
+| ck                   |     0.642 |       96 s |      187 ms |
+| codebase-memory-mcp  |     0.630 |     454 ms |       46 ms |
+| grepai               |     0.561 |       35 s |       48 ms |
+| probe                |     0.387 |          — |      207 ms |
+| cs                   |     0.200 |          — |       22 ms |
+| ripgrep              |     0.126 |          — |       14 ms |
 
 | ![Speed vs quality (cold)](../assets/images/speed_vs_ndcg_cold.png) | ![Speed vs quality (warm)](../assets/images/speed_vs_ndcg_warm.png) |
-|:--:|:--:|
-| *Time to first result (index + query) vs NDCG@10* | *Query latency on a warm index vs NDCG@10* |
+| :-----------------------------------------------------------------: | :-----------------------------------------------------------------: |
+|          _Time to first result (index + query) vs NDCG@10_          |             _Query latency on a warm index vs NDCG@10_              |
 
-The 137M-param CodeRankEmbed Hybrid wins NDCG@10 by 0.008. semble wins index time by 218x and query latency by 11x.
+semble matches the NDCG@10 of the 137M-param CodeRankEmbed while winning index time by ~220x and query latency by ~17x.
 
 NDCG@10 is averaged across all queries. Speed numbers use one repo per language, CPU only: cold-start index time and warm query p50 (median across 5 consecutive runs).
 
@@ -44,19 +46,19 @@ Coding agents (Claude Code, OpenCode, etc.) typically find code by running `grep
 
 For each query: tokens consumed at first relevant hit, or 32k if the method never finds anything. Averaged across all 1251 queries.
 
-| Method | Expected tokens | Savings |
-|---|---:|---:|
-| ripgrep + read file | 45,692 | baseline |
-| **semble** | **566** | **98% fewer** |
+| Method              | Expected tokens |       Savings |
+| ------------------- | --------------: | ------------: |
+| ripgrep + read file |          45,587 |      baseline |
+| **semble**          |         **348** | **99% fewer** |
 
 ### Recall at fixed token budgets
 
 A relevant file is "covered" once any retrieved unit comes from it.
 
-| Method | 500 | 1k | 2k | 4k | 8k | 16k | 32k |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| **semble** | **0.685** | **0.849** | **0.938** | **0.976** | **0.991** | **0.996** | **0.996** |
-| ripgrep + read file | 0.001 | 0.008 | 0.037 | 0.088 | 0.212 | 0.379 | 0.583 |
+| Method              |       500 |        1k |        2k |        4k |        8k |       16k |       32k |
+| ------------------- | --------: | --------: | --------: | --------: | --------: | --------: | --------: |
+| **semble**          | **0.842** | **0.923** | **0.967** | **0.988** | **0.995** | **0.995** | **0.995** |
+| ripgrep + read file |     0.001 |     0.008 |     0.037 |     0.086 |     0.207 |     0.374 |     0.583 |
 
 <details>
 <summary>Methodology</summary>
@@ -67,63 +69,65 @@ semble returns the top-50 ranked chunks. `ripgrep+read` splits the query into ke
 
 ## By language
 
-NDCG@10 per language, sorted by CodeRankEmbed Hybrid (CRE in the table). Best score per row is bolded.
-
-| Language | semble | CRE Hybrid | CRE | ColGREP | grepai | probe | ripgrep |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| scala | 0.909 | **0.922** | 0.845 | 0.765 | 0.330 | 0.392 | 0.180 |
-| cpp | **0.915** | 0.913 | 0.846 | 0.626 | 0.731 | 0.375 | 0.126 |
-| ruby | **0.909** | **0.909** | 0.769 | 0.708 | 0.643 | 0.382 | 0.230 |
-| elixir | 0.894 | **0.905** | 0.869 | 0.808 | 0.669 | 0.412 | 0.134 |
-| javascript | 0.917 | 0.903 | **0.920** | 0.823 | 0.675 | 0.588 | 0.176 |
-| zig | **0.913** | 0.901 | 0.807 | 0.474 | 0.755 | 0.369 | 0.000 |
-| csharp | 0.885 | **0.889** | 0.743 | 0.614 | 0.277 | 0.392 | 0.117 |
-| go | **0.895** | 0.884 | 0.676 | 0.785 | 0.722 | 0.410 | 0.133 |
-| python | 0.867 | **0.880** | 0.794 | 0.777 | 0.634 | 0.488 | 0.202 |
-| php | 0.858 | **0.874** | 0.758 | 0.663 | 0.402 | 0.340 | 0.123 |
-| swift | 0.860 | **0.873** | 0.721 | 0.710 | 0.429 | 0.280 | 0.160 |
-| bash | 0.825 | 0.852 | **0.892** | 0.706 | 0.723 | 0.226 | 0.000 |
-| lua | 0.823 | **0.847** | 0.803 | 0.798 | 0.699 | 0.336 | 0.000 |
-| java | **0.849** | 0.841 | 0.706 | 0.641 | 0.386 | 0.536 | 0.198 |
-| kotlin | 0.821 | **0.830** | 0.670 | 0.637 | 0.478 | 0.335 | 0.166 |
-| rust | **0.856** | 0.827 | 0.627 | 0.662 | 0.519 | 0.242 | 0.162 |
-| c | 0.741 | **0.806** | 0.706 | 0.676 | 0.555 | 0.384 | 0.000 |
-| haskell | 0.765 | 0.771 | **0.776** | 0.683 | 0.483 | 0.313 | 0.000 |
-| typescript | 0.706 | **0.708** | 0.545 | 0.430 | 0.394 | 0.354 | 0.128 |
-| **overall** | **0.854** | **0.862** | **0.765** | **0.693** | **0.561** | **0.387** | **0.126** |
+NDCG@10 per language, sorted by CodeRankEmbed (CRE in the table). Best score per row is bolded.
+
+| Language    |    semble |       CRE |   ColGREP |        ck |       cbm |    grepai |     probe |        cs |   ripgrep |
+| ----------- | --------: | --------: | --------: | --------: | --------: | --------: | --------: | --------: | --------: |
+| javascript  |     0.917 | **0.925** |     0.823 |     0.772 |     0.770 |     0.675 |     0.588 |     0.171 |     0.176 |
+| scala       |     0.909 | **0.925** |     0.765 |     0.717 |     0.704 |     0.330 |     0.392 |     0.111 |     0.180 |
+| zig         | **0.913** |     0.911 |     0.474 |     0.511 |     0.766 |     0.755 |     0.369 |     0.121 |     0.000 |
+| ruby        | **0.909** |     0.905 |     0.708 |     0.738 |     0.689 |     0.643 |     0.382 |     0.255 |     0.230 |
+| cpp         | **0.915** |     0.897 |     0.626 |     0.687 |     0.630 |     0.731 |     0.375 |     0.262 |     0.126 |
+| elixir      | **0.894** |     0.893 |     0.808 |     0.786 |     0.506 |     0.669 |     0.412 |     0.397 |     0.134 |
+| python      |     0.867 | **0.878** |     0.777 |     0.721 |     0.643 |     0.634 |     0.488 |     0.305 |     0.202 |
+| csharp      | **0.885** |     0.848 |     0.614 |     0.548 |     0.775 |     0.277 |     0.392 |     0.248 |     0.117 |
+| php         | **0.858** |     0.847 |     0.663 |     0.615 |     0.608 |     0.402 |     0.340 |     0.180 |     0.123 |
+| swift       | **0.860** |     0.845 |     0.710 |     0.672 |     0.630 |     0.429 |     0.280 |     0.151 |     0.160 |
+| bash        |     0.825 | **0.834** |     0.706 |     0.677 |     0.768 |     0.723 |     0.226 |     0.170 |     0.000 |
+| lua         |     0.823 | **0.829** |     0.798 |     0.738 |     0.591 |     0.699 |     0.336 |     0.050 |     0.000 |
+| kotlin      |     0.821 | **0.823** |     0.637 |     0.587 |     0.611 |     0.478 |     0.335 |     0.170 |     0.166 |
+| haskell     |     0.765 | **0.811** |     0.683 |     0.733 |     0.624 |     0.483 |     0.313 |     0.160 |     0.000 |
+| java        | **0.849** |     0.790 |     0.641 |     0.606 |     0.554 |     0.386 |     0.536 |     0.136 |     0.198 |
+| c           |     0.741 | **0.771** |     0.676 |     0.606 |     0.655 |     0.555 |     0.384 |     0.175 |     0.000 |
+| rust        | **0.856** |     0.754 |     0.662 |     0.419 |     0.454 |     0.519 |     0.242 |     0.193 |     0.162 |
+| go          | **0.895** |     0.713 |     0.785 |     0.458 |     0.506 |     0.722 |     0.410 |     0.183 |     0.133 |
+| typescript  | **0.706** |     0.671 |     0.430 |     0.456 |     0.455 |     0.394 |     0.354 |     0.145 |     0.128 |
+| **overall** | **0.854** |     0.839 | **0.693** | **0.634** | **0.630** | **0.561** | **0.387** | **0.200** | **0.126** |
+
+cbm = [codebase-memory-mcp](#methods).
 
 ## Ablations
 
 `raw` returns retrieval scores directly; `+ ranking` feeds them through semble's hybrid ranker.
 
-| Retrieval | Raw | + ranking |
-|---|---:|---:|
-| BM25 | 0.675 | 0.834 |
-| potion-code-16M | 0.650 | 0.821 |
-| BM25 + potion-code-16M | — | **0.854** |
+| Retrieval              |   Raw | + ranking |
+| ---------------------- | ----: | --------: |
+| BM25                   | 0.675 |     0.834 |
+| potion-code-16M        | 0.650 |     0.821 |
+| BM25 + potion-code-16M |     — | **0.854** |
 
 <details>
 <summary>By query category</summary>
 
-| Mode | Architecture | Semantic | Symbol |
-|---|---:|---:|---:|
-| BM25 raw | 0.628 | 0.676 | 0.719 |
-| potion-code-16M raw | 0.626 | 0.666 | 0.629 |
-| semble BM25 (+ ranking) | 0.770 | 0.819 | 0.957 |
-| semble potion-code-16M (+ ranking) | 0.757 | 0.808 | 0.943 |
-| **semble hybrid** | **0.802** | **0.846** | **0.958** |
+| Mode                               | Architecture |  Semantic |    Symbol |
+| ---------------------------------- | -----------: | --------: | --------: |
+| BM25 raw                           |        0.628 |     0.676 |     0.719 |
+| potion-code-16M raw                |        0.626 |     0.666 |     0.629 |
+| semble BM25 (+ ranking)            |        0.770 |     0.819 |     0.957 |
+| semble potion-code-16M (+ ranking) |        0.757 |     0.808 |     0.943 |
+| **semble hybrid**                  |    **0.802** | **0.846** | **0.958** |
 
 </details>
 
 ## Dataset
 
 ~1,250 queries over 63 repositories in 19 languages, grouped into three categories:
 
-| Category | Queries | What it tests |
-|---|---:|---|
-| semantic | 711 | Code that implements a specific behavior or concept |
-| architecture | 343 | Design decisions, module boundaries, structural patterns |
-| symbol | 204 | Named entity lookup (function, class, type, variable) |
+| Category     | Queries | What it tests                                            |
+| ------------ | ------: | -------------------------------------------------------- |
+| semantic     |     711 | Code that implements a specific behavior or concept      |
+| architecture |     343 | Design decisions, module boundaries, structural patterns |
+| symbol       |     204 | Named entity lookup (function, class, type, variable)    |
 
 <details>
 <summary>Notes</summary>
@@ -137,18 +141,23 @@ NDCG@10 per language, sorted by CodeRankEmbed Hybrid (CRE in the table). Best sc
 ## Methods
 
 - **[ripgrep](https://github.com/BurntSushi/ripgrep)**: fast regex search over files, included as a raw keyword-match baseline.
-- **[probe](https://github.com/buger/probe)**: BM25 keyword ranking backed by tree-sitter parse trees. No persistent index; scans on the fly.
+- **[probe](https://github.com/buger/probe)**: BM25 keyword ranking backed by tree-sitter parse trees. We use its free `probe search` mode; its `probe agent` mode (which does query rewriting) needs a paid LLM API key and isn't tested here.
+- **[cs (Code Spelunker)](https://github.com/boyter/cs)**: structural BM25 ranker which ranks matches differently in code/comments/strings).
 - **[ColGREP](https://github.com/lightonai/next-plaid/tree/main/colgrep)**: late-interaction code retrieval built on next-plaid with the [LateOn-Code-edge](https://huggingface.co/lightonai/LateOn-Code-edge) model.
 - **[grepai](https://github.com/nicholasgasior/grepai)**: semantic search using [nomic-embed-text](https://huggingface.co/nomic-ai/nomic-embed-text-v1) (137M params) via a local Ollama daemon.
-- **[CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed)**: 137M-param transformer embedding model for code retrieval. *CodeRankEmbed Hybrid* fuses its dense scores with BM25.
+- **[codebase-memory-mcp](https://github.com/DeusData/codebase-memory-mcp)**: code intelligence engine that indexes a repo into a SQLite/graph store. We benchmark its `search_graph` tool in `fast` mode, which does BM25 full-text search with structural boosting.
+- **[ck](https://github.com/BeaconBay/ck)**: hybrid regex + semantic search using [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5).
+- **[CodeRankEmbed](https://huggingface.co/nomic-ai/CodeRankEmbed)**: 137M-param transformer embedding model for code retrieval, used for semantic-only dense search.
 - **[semble](https://github.com/your-repo/semble)**: this library. [potion-code-16M](https://huggingface.co/minishlab/potion-code-16M) static embeddings + BM25 + the semble reranking stack.
 
 ## Excluded methods
 
-Two tools were considered but not included in the benchmark:
+The following tools were considered but not included in the main comparison:
 
-- **[codanna](https://codanna.io)**: symbol-level semantic search with fastembed. Excluded because it does not support Haskell, Bash, Zig, Scala, Elixir, or Ruby (6 of the 19 benchmark languages, covering 20 of 63 repos (~38% of tasks)).
+- **[codanna](https://codanna.io)**: symbol-level semantic search with fastembed. Excluded because it does not support Haskell, Bash, Zig, Scala, Elixir, or Ruby (6 of the 19 benchmark languages).
 - **[claude-context](https://github.com/zilliztech/claude-context)**: retrieval-augmented code search using OpenAI embeddings and a vector database. Excluded because it requires a paid OpenAI API key and a running vector-DB service.
+- **[GitNexus](https://github.com/abhigyanpatwari/GitNexus)**: knowledge-graph code search (BM25 + local embeddings + RRF). Excluded because it does not support Bash, Elixir, Haskell, Lua, Scala, or Zig (6 of the 19 benchmark languages).
+- **[codegraph](https://github.com/colbymchenry/codegraph)**: SQLite FTS5 symbol search + graph traversal. Excluded because it does not support Haskell, Bash, Elixir, or Zig (4 of the 19 benchmark languages).
 
 ## Running the benchmarks
 
@@ -232,6 +241,44 @@ The `--output` flag enables resume mode: already-completed repos are skipped on
 
 </details>
 
+<details>
+<summary>cs (Code Spelunker)</summary>
+
+Needs `cs` on `$PATH` (`go install github.com/boyter/cs/v3@latest`).
+
+```bash
+uv run python -m benchmarks.baselines.cs
+uv run python -m benchmarks.baselines.cs --repo fastapi --repo axios
+```
+
+</details>
+
+<details>
+<summary>codebase-memory-mcp</summary>
+
+Needs `codebase-memory-mcp` on `$PATH` (`uv tool install codebase-memory-mcp`, or `pip`/`npm`/`brew`). First run downloads a small platform binary.
+
+```bash
+uv run python -m benchmarks.baselines.codebase_memory
+uv run python -m benchmarks.baselines.codebase_memory --repo fastapi --repo axios
+```
+
+Each repo is indexed under a `semble-bench-<repo>` project name and deleted again after evaluation, so it doesn't collide with any projects you have indexed for real use.
+
+</details>
+
+<details>
+<summary>ck</summary>
+
+Needs `ck` on `$PATH` (`cargo install ck-search` or `npm install -g @beaconbay/ck-search`). Requires outbound HTTPS to Hugging Face to download its `BAAI/bge-small-en-v1.5` embedding model on first run.
+
+```bash
+uv run python -m benchmarks.baselines.ck
+uv run python -m benchmarks.baselines.ck --repo fastapi --repo axios
+```
+
+</details>
+
 <details>
 <summary>ripgrep</summary>
 
@@ -265,7 +312,7 @@ Requires the `benchmark` extra (`uv sync --extra benchmark`).
 
 ```bash
 uv run python -m benchmarks.baselines.coderankembed
-uv run python -m benchmarks.baselines.coderankembed --mode semantic
+uv run python -m benchmarks.baselines.coderankembed --repo fastapi --repo axios
 ```
 
 </details>