Add Torch and Triton kernels for Torch2FlyDSL and Triton2FlyDSL translation#45
Add Torch and Triton kernels for Torch2FlyDSL and Triton2FlyDSL translation#45amd-yashagar wants to merge 8 commits into
Conversation
amd-yashagar
requested review from
irvineoy,
sharareh-y and
sinarafati-amd
as code owners
|
Validation All 99 new tasks were run through the project's Task Validator checks (the 10 checks in agents/task_validator) on gfx942/MI300X: config schema, source files, target symbols, compile, correctness, performance, correctness-implementation review, self-containedness, hang check, and result-template compatibility. Result: 99/99 PASS (48 torch2flydsl + 51 triton2flydsl), 0 FAIL. The 11 FP4/MX gfx950-only tasks validate as a clean skip on gfx942 (exit 0) — the expected platform-support behavior; all other tasks pass compile + correctness (against the real aiter op / in-source reference at the tight gate) + performance. Per-task validation_report.yaml and the aggregate validation_summary.yaml are available in a separate commit, these should be removed from the PR once it is reviewed and approved. |
|
Hi, quick heads-up: we are currently working on a larger refactor #48 that moves the benchmark/runtime flow to a Docker-first environment. This touches a lot of shared infrastructure, so reviews and merges may be slower than usual for a bit. Once that refactor lands, I’ll help rebase open PRs onto that one. Sorry for the delay, and thanks for your patience. |
Introduce the torch2flydsl task type (a PyTorch `class Model` in KernelBench format as the source, a FlyDSL kernel as the target) and register both the torch2flydsl and triton2flydsl task types in the prompt builder. Add the torch2flydsl task corpus: pure-torch, AMD-runtime-faithful references validated against the real aiter ops at a tight gate, covering GEMM, MoE routing/gating/sorting, normalization, RoPE, activation, and quantization. FP8 references select the arch-appropriate dtype (e4m3fnuz on gfx942, e4m3fn on gfx950); FP4/MX-only tasks are tagged gfx950-only with a harness skip-guard. Routing is made deterministic so the reference and the op select the same experts. Ignore build/ artifacts.
Standalone Triton-source tasks (depend only on triton/torch) faithfully extracted from aiter, covering bf16/fp8 GEMM (incl. block-scale, batched, feed-forward), MoE routing and grouped GEMM, normalization, softmax, RoPE, quantization, and attention (MHA / MLA / unified / sage / fp8 MQA logits). Each task ships the Triton source, a harness that runs the Triton kernel and gates it against an in-source reference at the upstream tolerance, and a config. MX scaled-dot tasks are tagged gfx950-only with a skip-guard. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from SGLang's linear-attention / Gated-DeltaNet chunk pipeline: chunk state update (chunk_fwd_h), chunk output (chunk_fwd_o), fused recurrent decode, and the L2-norm forward. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source task faithfully extracted from SGLang's fused-MoE grouped-GEMM kernel, with a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from SGLang: the native RadixAttention prefill/decode/extend backend and state-merge, the rest of the Gated-DeltaNet chunk pipeline (gating, cumsum, scaled-dot-kkt, norm-gate, wy_fast), Mamba2 SSD chunk-state, lightning linear attention, MoE routing, gated and dual-residual normalization, expert combine, multimodal RoPE, and FP4 indexer quantization. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
Standalone Triton-source tasks faithfully extracted from the Meta generative-recommenders (HSTU) project: jagged-dense BMM with broadcast add, jagged-dense broadcast add, layer norm, and fused SwiGLU. Each ships the Triton source, a harness that runs the Triton kernel against an in-source reference at the upstream tolerance, and a config. Upstream license headers are retained.
irvineoy
force-pushed
the
torch_triton_kernels
branch
from
de5418d to
b069e2b
Compare
irvineoy
left a comment
irvineoy
left a comment
There was a problem hiding this comment.
Hi, first of all, thank you for your PR.
I rebased this branch on top of the latest main and fixed a few known issues in the task configs / validation setup.
I also ran the Validator Agent on all newly added tasks in this PR on gfx950/MI355X. Some tasks did not pass validation, and I collected the non-PASS results in the report below.
pr45_validation_non_pass_report.md
Please use this report to fix the remaining issues, then rerun the Validator Agent after the fixes. A few failures look like dependency/API mismatches in the Docker validation environment, some are harness issues, and some are weaker validation coverage warnings.
Feel free to discuss with me if anything in the report is unclear or if you think any item should be handled differently.
|
Hey @irvineoy thank you for the review and the changes. I am not able to access pr45_validation_non_pass_report.md, it shows 404 error. |
…eal references - Guard unimplemented FlyDSL kernel.py stubs (NotImplementedError -> SKIP) in model-only torch2flydsl correctness and benchmark paths; keep validating model.py against the aiter op at full tolerance. - Fix gemm_a8w8_bpreshuffle for flydsl 0.2.2 (cast idx2crd inputs to i32). - Add real numerical references to 11 triton2flydsl harnesses that were finiteness-only (mha, mla, unified_attention[+sparse_mla], fp8_mqa_logits, moe_fused_gemm, fav3_sage, moe_routing_sigmoid_top1, jagged_dense_bmm_broadcast_add, rmsnorm). - moe_topk_softmax: compare absolute matched-id weight error and document the intrinsic bf16 routing-floor exception for biased large-E shapes. - Resolve validator WARNs (per_tensor_fp8_quant, gelu_fast, rope_thd_fwd, aiter/rmsnorm large-N shape) and add the silu_and_mul benchmark stub-guard. Co-authored-by: Cursor <cursoragent@cursor.com>
amd-yashagar
force-pushed
the
torch_triton_kernels
branch
from
b069e2b to
137b0ba
Compare
|
Ah there were 2 commits added by Vincent Ouyang after which I force pushed some changes. I think I may have overwritten those. |
3 participants
Summary
Adds two task suites for FlyDSL kernel translation:
Together they cover the operators current models run on AMD: GEMM (incl. FP8 block‑scale), attention/MLA, MoE routing + fused MoE, normalization, RoPE, quantization, and linear‑attention/SSD. The PR also registers the torch2flydsl and triton2flydsl task types in the prompt builder.
Sources & licenses
Kernels are extracted from upstream (aiter, SGLang, Meta generative‑recommenders); each file retains its upstream license/copyright header. Modifications are limited to making each kernel standalone