Deprecate NeMo2

[jstjohn]

Description

Remove all NeMo2/Megatron-LM dependencies from the repository, delete stale sub-packages, and modernize the framework CI to use a lightweight per-sub-package matrix strategy matching the recipes CI pattern.

What was deleted (~100 files)

• 3rdparty submodules: Megatron-LM and NeMo git submodules removed.
• 3 sub-packages: bionemo-example_model, bionemo-llm, bionemo-testing — all depended on NeMo2/Megatron-Core and are no longer maintained.
• Root build artifacts: Dockerfile, pyproject.toml, requirements-cve.txt, requirements-dev.txt, requirements-test.txt — packages now define their own dependencies in their pyproject.toml.
• Framework devcontainer: .devcontainer/framework/ removed (only one devcontainer remains).
• Monolithic CI scripts: ci/scripts/pytest_runner.sh, run_pytest_unittests.sh, run_pytest_slow.sh, run_pytest_notebooks.sh — replaced by per-package install in the workflow itself.

CI overhaul

• unit-tests-framework.yml rewritten to match the unit-tests-recipes.yml pattern: each sub-package runs in its own matrix job with a lightweight PyTorch container image, installing only its own dependencies.
• Dependency graph is derived from tach.toml at CI time instead of being hardcoded. When a package changes, its downstream dependents are automatically included in the test matrix.
• bionemo-noodles gets a .ci_build.sh for Rust toolchain installation (maturin/pyo3 build).
• Codecov stabilized: each matrix job uploads with a per-package flag and carryforward: true so partial runs don't cause coverage to appear to drop.
• Sparse checkout now includes LICENSE/ directory so sub-package LICENSE symlinks resolve correctly.

Docs cleanup

• Overview page rewritten to describe "BioNeMo Recipes" and "Sub-package Utilities" instead of the generic "BioNeMo Framework."
• Stale NeMo2 references removed across getting-started guides, FAQ, contributing docs, model pages.
• bionemo2 path references updated to bionemo throughout.
• gen_ref_pages.py now rewrites relative links when copying recipe READMEs into the docs tree, so source files can use relative paths while the compiled docs still resolve correctly.
• MolMIM marked as TODO (not yet ported to recipes).
• GitHub raw URLs in recipe READMEs reverted to relative paths; CI script references changed to plain backtick file paths.

Structure changes

• Devcontainer moved from .devcontainer/recipes/ to .devcontainer/ (single devcontainer).
• Root pyproject.toml split: ruff config → .ruff.toml, pytest config → pytest.ini.
• tach.toml rewritten with accurate module definitions and dependencies for the remaining 8 sub-packages.
• bionemo-webdatamodule: lightning added as an explicit dependency in its pyproject.toml.

Type of changes

• Refactor
• Documentation update
• Other: Remove deprecated NeMo2 dependencies, sub-packages, and CI infrastructure

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: efe10fa1-d952-4905-be55-73e83b19f0ef

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch jstjohn/deprecate-nemo2

📝 Coding Plan

• Generate coding plan for human review comments

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.62%. Comparing base (4272f4a) to head (3fabcb1).
⚠️ Report is 6 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1517      +/-   ##
==========================================
+ Coverage   79.25%   81.62%   +2.36%     
==========================================
  Files          93       68      -25     
  Lines        6767     5196    -1571     
==========================================
- Hits         5363     4241    -1122     
+ Misses       1404      955     -449     

Flag	Coverage Δ
bionemo-core	83.33% <ø> (?)
bionemo-moco	75.93% <ø> (?)
bionemo-noodles	89.65% <ø> (?)
bionemo-recipeutils	80.95% <ø> (?)
bionemo-scdl	87.12% <ø> (?)
bionemo-scspeedtest	69.78% <ø> (?)
bionemo-size-aware-batching	96.95% <ø> (?)
bionemo-webdatamodule	88.59% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...nemo-scspeedtest/src/bionemo/scspeedtest/common.py	56.17% <ø> (+12.92%)	⬆️

... and 6 files with indirect coverage changes

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[jomitchellnv]

I suggest we do not merge until post GTC

[jwilber]

A lot of this is edited out, but I wonder if we still need this file at all?