Skip to content

Commit d1649bb

Browse files
KleinBlueCclaude
andcommitted
[None][chore] Inkling NVFP4: code-quality pass over the bring-up
Removes the bring-up's debug scaffolding and completes the deliverables the Inkling NVFP4 change was missing. No model behaviour changes. Deleted 41 debug/localization test files (divergence probes, dump/isolate helpers, teacher-forcing and graph-capture localizers) that existed to find the bring-up's defects and have no role now that it works. Nothing imports them: every deleted basename was grepped across the tree, zero references remain. The two environment variables the model still reads are the ones worth keeping — INKLING_ALLREDUCE_STRATEGY (the escape hatch for the ONESHOT all-reduce mitigation) and INKLING_MOE_BACKEND (the trtllm-gen MoE kernel select) — and both are documented; the 19 debug-only INKLING_* knobs are gone. Completed the deliverables: - docs/source/models/supported-models.md — architecture row, multimodal feature-matrix row, and footnote [^14] covering modality coverage, the unsupported set (MTP, LoRA, function calling, constrained decoding, EPD, mm-hash caching), and the all-reduce mitigation plus its escape hatch. - TestInkling_NVFP4::test_nvfp4 added to the shared multimodal accuracy file rather than a standalone per-model test, with a sourced MMMU reference. - Registered that id in test-db/l0_b200.yml and qa/llm_function_core.txt. - Dropped the orphaned inkling_vision_tower_artifact.json; the surviving regression test consumes the generated artifact instead. Static checks: git diff --check clean, every modified Python file compiles, no stale references to the deleted files. Runtime coverage as of the last completed suite: unit tiers green (vision 43, audio 10, video 12, text 31, collect 95 with zero residual debug files) and GSM8K cg0ov0 parity at delta=0.0. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1 parent 35a4a46 commit d1649bb

62 files changed

Lines changed: 586 additions & 15499 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,9 +73,13 @@ docs/source/**/*.rst
7373

7474
# Testing
7575
.coverage.*
76+
.bootstrap_build.lock
7677
results_trt/
7778
llm-test-workspace/
7879
ad-test-workspace/
80+
tests/unittest/_torch/modeling/_mmmu_cache/
81+
tests/unittest/_torch/modeling/mmmu_align_artifact.json
82+
tests/unittest/_torch/modeling/inkling_vision_tower_generated_artifact.json
7983

8084
# FetchContent reference cache
8185
3rdparty/.cache_3rdparty/

docs/source/models/supported-models.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ The following is a table of supported models for the PyTorch backend:
2828
| `GptOssForCausalLM` | GPT-OSS | `openai/gpt-oss-20b`, `openai/gpt-oss-120b` |
2929
| `HunYuanDenseForCausalLM` [^5] | Hunyuan Dense | `tencent/Hunyuan-7B-Instruct` |
3030
| `HunYuanMoEForCausalLM` [^5] | Hunyuan MoE | `tencent/Hunyuan-A13B-Instruct` |
31+
| `InklingForConditionalGeneration` [^14] | Inkling | `thinkingmachines/Inkling-NVFP4` |
3132
| `InternLM3ForCausalLM` [^5] | InternLM3 | `internlm/internlm3-8b-instruct` |
3233
| `KimiK25ForConditionalGeneration` | Kimi-K2.5 | `moonshotai/Kimi-K2.5` |
3334
| `LagunaForCausalLM` | Laguna-XS | `poolside/laguna-XS.2` |
@@ -73,6 +74,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
7374
| `Qwen3_5MoeForCausalLM` | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No | Yes | Untested | Yes | N/A | Untested | Untested |
7475
| `Llama4ForConditionalGeneration` | Yes | Yes | Yes | Yes | Yes | No | Yes | Yes | No | Yes | Yes | Untested | N/A | Yes | Yes |
7576
| `GptOssForCausalLM` | Yes | Yes | Yes | Yes | Yes | No | Yes | No | Yes | Yes | Yes | Yes | N/A | Yes | Yes |
77+
| `InklingForConditionalGeneration` [^14] | Yes | Yes | Untested | No | Untested | No | No | No | No | Yes | Untested | No | Yes | Yes | No |
7678
| `Glm4MoeLiteForCausalLM` [^5] | Yes | Yes | Untested | Untested | Yes | No | No | No | No | Yes | Untested | Untested | N/A | Untested | Untested |
7779
| `NemotronHForCausalLM` | Yes | Yes | Yes | Yes | Yes | Yes | No | No | No | Yes | Yes | Yes | N/A | Untested | Untested |
7880
| `Gemma4ForConditionalGeneration` | Untested | Yes | Untested | No | Yes | No | No | No | No | Yes | Untested | No | Yes | Untested | Untested |
@@ -92,6 +94,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
9294
[^11]: DeepSeek-V4 is only supported on Blackwell GPUs (`SM100+`). See the [DeepSeek-V4 example README](../../../examples/models/core/deepseek_v4/README.md) for setup and parallelism.
9395
[^12]: Supports text, image, and video inputs over the block-sparse attention path. The published MXFP8 checkpoint is dequantized on load so the runtime sees an effectively BF16 model. The text decoder is also usable standalone (text-only) via the `MiniMaxM3SparseForCausalLM` architecture. KV cache reuse and MTP are not supported on the sparse-attention path in this release.
9496
[^13]: The Cosmos 3 family also supports visual generation through the VisualGen API. See [Visual Generation Models](#visual-generation-models).
97+
[^14]: Inkling is an NVFP4 checkpoint (`thinkingmachines/Inkling-NVFP4`) with a RoPE-free hybrid-attention text decoder plus a BF16 hMLP vision tower and dMel audio tower; video is handled as multi-frame images. The vision and audio towers stay BF16 even though the text decoder and routed experts are NVFP4. Accuracy is gated on the text decoder (GSM8K) and the vision path (MMMU); the audio and video paths are covered by module unit tests only, not an accuracy benchmark. The text decoder is also usable standalone (text-only) via the `InklingForCausalLM` architecture. A reasoning parser is registered (`--reasoning_parser inkling`) for Inkling's typed-content thinking/content blocks. Not supported in this release: MTP / speculative decoding (a static MTP config tier exists in the checkpoint and is weight-accounted, but no runtime path consumes it), LoRA, function calling, constrained/guided decoding, EPD disaggregated serving, and multimodal-hash prefix caching. Every Inkling all-reduce is pinned to the `ONESHOT` strategy after construction to avoid a captured symmetric-all-reduce defect under CUDA graph (a 12288 B one-token decode all-reduce goes non-finite and decode collapses to a repeated token); set `INKLING_ALLREDUCE_STRATEGY=AUTO` to restore the stock strategy — and re-expose the defect — for A/B measurement.
9598

9699
# Multimodal Feature Support Matrix (PyTorch Backend)
97100

@@ -102,6 +105,7 @@ Note: Support for other models may vary. Features marked "N/A" are not applicabl
102105
| `Gemma4ForConditionalGeneration` | Untested | Yes | Yes | Yes | Untested | No | Untested | No | L + I + V + A [^9] |
103106
| `Gemma4UnifiedForConditionalGeneration` | Untested | Untested | Untested | Yes | Untested | No | Untested | No | L + I + A |
104107
| `HCXVisionForCausalLM` | Yes | Yes | No | Yes | Yes | Yes | Yes | No | L + I |
108+
| `InklingForConditionalGeneration` | Yes | Yes | Untested | Yes | Untested | No | Yes | No | L + I + V + A [^14] |
105109
| `LlavaLlamaModel (VILA)` | Yes | Yes | No | Yes | Yes | No | Yes | No | L + I + V |
106110
| `LlavaNextForConditionalGeneration` | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | L + I |
107111
| `Llama4ForConditionalGeneration` | Yes | Yes | No | Yes | Yes | No | Yes | No | L + I |

0 commit comments

Comments
 (0)