Skip to content

Bot Commands

Jump to bottom
Wang, Xiao edited this page Feb 27, 2025 · 32 revisions

(This page is generated. Don't directly edit here. Your changes may be overwritten in future updates.)

Bot commands

You may comment !build or !test on your PR to trigger internal CI pipelines to run builds and tests for your PR. This is necessary as there are currently several required checks (build_image_20, c++ and python lint) that have to be passed before your PR can get merged. Both checks are only created from internal pipelines and not automatic GitHub actions.

CI pipelines can only be triggered by authorized users. If you're not an authorized user, please ping one of our maintainers to get help.

Note

Checks with Expected — Waiting for status to be reported will not be reported automatically. Authorized users need to use bot commands to trigger internal pipelines that report those job statuses.

Details about bot commands

We have mechanisms to cancel redundant pipelines from a same PR. A newly triggered pipeline will cancel all old existing pipelines from a same PR.

If you need to keep multiple pipelines running to compare different implementations, you may open additional draft [DO NOT MERGE] PRs to test your implementations.

Whenever a PR is closed (merged or not), all pipelines for that PR will be cancelled.

!build command

The !build command triggers a pipeline that runs only build jobs. No tests will be triggered.

Use this command if your PR change is small, or safe to be merged while not causing funcitonality or performance issues.

!test command

The !test command triggers a full pipeline, that runs build jobs first and test jobs afterwards.

The !test command without any option runs a default set of jobs, including C++ and python tests, examples build and run, sanity checks, etc.

The default set of jobs always run regardless of extra options, unless specified otherwise.

You may use the options below to run extra optional tests.

CI bot commands:

Usage: !test [options]
  Options:
    --diff
      Run codegen diff jobs (fast): python tests and binary tests
    --diff-bench
      Run codegen diff jobs (slow): binary benchmark
    --matmul-bench
      Run python and binary matmul benchmark tests with internal heuristic 
      plugin 
    --pybench
      Run a fraction of python benchmark jobs
    --pybench-full
      Run full python benchmark jobs. Very slow. May take days. It's 
      recommended to use --pybench in general
    --serde
      Run serialization and deserialization jobs

Example:

!test --diff --diff-bench

This runs the codegen diff jobs (fast) and codegen diff jobs (slow) in addition to the default set of tests.

Note

If you see an error similar like nvfuser-ci — CI script failed. If retry doesn't work, please contact maintainers with reference A#1234. after trying to start a pipeline, first check if your PR has a merge conflict with the main branch. If so, resolve the conflict first before starting a new pipeline. If not, check mailing list/team and direct messages and see if there are any anouncement about CI infra outages. Wait for 5 - 10 minutes and retry a new pipeline. If it still doesn't work, raise an issue with CI maintainers with the reference number.

Details about CI jobs

jit_python_bc_advisory

This job is created to detect backward compatibility (BC) breaking changes in python API. The job will build nvfuser from your PR head commit and run python tests from its merge base with the main branch, to catch any BC-breaking changes in python API.

If your PR has python tests passed but python_bc_advisory tests failed, you have a few options:

  1. Update the version.txt file. With that, the python_bc_advisory job will be skipped. We have a convention that a patch version update indicates python API changes.

    Sometimes when two PRs are both bumping version.txt at the same time and the other was merged before yours, you may see this error. This reminds you that you may need to bump version.txt again.

  2. Ignore the failure in python_bc_advisory job, if your PR change is small or you believe it's safe. This is why the job is named advisory.

  3. There are rare cases that you updated some python tests in your PR, and those (updated) tests passed in jit_python_tests job but failed in the jit_python_bc_advisory job.

    You may want to check if the old python tests are valid tests and if they pass on main. If so, you may need to check why those (old) tests failed on your PR build.

Remember, you always have to make sure the jit_python_tests passed. Those are built and tested on your current PR head commit.

other jobs

🚧 under construction 🚧

Clone this wiki locally