Draft
feat(detect): index Jupyter notebooks with kernel-language fenced code blocks#1
Conversation
… code blocks
- Add NOTEBOOK_EXTENSIONS = {'.ipynb'}
- Add ipynb_to_markdown() using metadata.language_info.name / kernelspec.language
for fenced code type (falls back to 'code' when metadata absent)
- Add convert_notebook_file() sidecar logic mirroring office files
- classify_file() now classifies .ipynb as DOCUMENT
- detect() converts notebooks to markdown sidecars
- Tests for all new behaviour including language metadata resolution
The Jupyter format allows 'source' to be either a str or list[str]. Also adds a test for the string-source case.
Copilot
AI
changed the title
feat(detect): add Jupyter notebook support with language-aware fenced code blocks
feat(detect): index Jupyter notebooks with kernel-language fenced code blocks
Jun 28, 2026
Copilot created this pull request from a session on behalf of
jimwhite
June 28, 2026 18:55
View session
jimwhite
pushed a commit
that referenced
this pull request
Aug 2, 2026
…languages (Graphify-Labs#1581) Cross-file name resolution folded case for every language, so `from pathlib import Path` resolved to a shell script's `export PATH=...` node — one variable becoming the corpus's #1 god-node (266 false incoming edges on a real repo), polluting god-node rankings, affected blast-radius, and clustering. Reported with a precise diagnosis by @sheik-hiiobd. Case is semantic in Python/Rust/Go/Java/C#/Kotlin/Swift/Ruby/C/C++/JS/TS: `Path` (class), `PATH` (env var), `path` (variable) are distinct. Fix gates folding by language at the two resolution sites the repro exercised: - global cross-file CALL resolver: index by exact case; a folded index is built only for case-insensitive-language nodes (PHP/SQL/Nim) and consulted only when the calling file is such a language. - type-reference STUB rewire (_rewire_unique_stub_nodes): match stubs to real defs by exact case, with a folded fallback restricted to case-insensitive- language definitions — so a case-sensitive `PATH` can never absorb a `Path`. For case-sensitive languages this only ever removes false edges. Concept/doc dedup (dedup.py, guarded to non-code nodes) is intentionally left folding. Regression tests: Python `Path` no longer hits shell `PATH`; a case-differing cross-file ref doesn't resolve; exact-case resolution still works; PHP fold preserved. Full suite 2777. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
jimwhite
pushed a commit
that referenced
this pull request
Aug 2, 2026
…raphify-Labs#1749) The extraction spec forbids cross-language `calls` edges, and build already dropped cross-language INFERRED `calls`. But `imports`/`references` had no such guard: an unresolved Python `import time` resolved by bare stem (the Graphify-Labs#1504 old-stem alias) onto a `src/time.ts` file node, welding a polyglot repo's two language halves together. In the reporter's repo three such edges were the only bridge between 2409 Python and 1403 TS nodes, so every backend<->frontend shortest path routed through time.ts, inflating its betweenness ~90x and making it the #1 reported god node. Hoist the interop-family map to a module constant and extend the edge-loop guard to `imports`/`imports_from`/`references`. For these relations the edge is dropped only when BOTH endpoints are known code languages of different families, so a config/manifest -> code reference (unknown ext) is never mistaken for a phantom. `calls` behavior is unchanged (still INFERRED-only, still drops when either family differs). Regression tests: py->ts import dropped, ts->ts import kept, config->code reference kept. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports notebook indexing from Graphify-Labs#1498 with the core fix: fenced code blocks use the actual kernel language from notebook metadata instead of hardcoded
code.Changes
graphify/detect.pyNOTEBOOK_EXTENSIONS = {'.ipynb'}— kept separate fromCODE_EXTENSIONS/DOC_EXTENSIONSsince notebooks go through sidecar conversionipynb_to_markdown(path)— converts cells to markdown; resolves fence language viametadata.language_info.name→kernelspec.language→"code"; handlessourceas eitherstrorlist[str](both valid per Jupyter spec)convert_notebook_file(path, out_dir)— sidecar logic mirroringconvert_office_file(); uses content-equality check so output-only re-runs don't churn sidecar mtimeclassify_file()—.ipynb→DOCUMENTdetect()— converts notebooks to markdown sidecars before indexingtests/test_detect.py— 14 new tests covering language resolution priority, mixed cells, mtime stability on output-only changes,detect()/detect_incremental()integration, and string vs. list source formats.Language resolution
A standard Python notebook produces
```pythonblocks; the"code"fallback applies only when metadata is absent entirely.