Alternatives to Slurm

[kimminw00]

Considering the Slurm or other cluster management systems may be highly customized and the task-submit command may be variant, the forementioned example is just a naive example. The key point is to submit the Python script with the MPI runtime, and the HLAPI will take care of the rest.

Are there any cluster management systems that can be used instead of slurm?

[Advait251206]

Yes. Slurm is not a requirement for TensorRT-LLM itself—it's simply one of the most common workload managers used to launch distributed jobs.

The key requirement is that all participating processes are launched correctly (typically via an MPI launcher or another distributed runtime), so that TensorRT-LLM can initialize the distributed environment. As mentioned in the documentation, the HLAPI expects your Python script to be launched under an MPI runtime, while the cluster manager is primarily responsible for resource allocation and process startup.

Some common alternatives include:

• Open MPI / MPICH launched directly with mpirun or mpiexec (without any scheduler).
• Kubernetes, typically using the MPI Operator or custom job controllers for distributed GPU workloads.
• Ray, if your infrastructure already uses Ray for distributed execution (although integration may require additional orchestration).
• Docker Swarm or Nomad, for simpler container orchestration environments.
• PBS Professional, Torque, LSF, or other HPC schedulers that can launch MPI jobs.
• Cloud-native batch schedulers such as AWS Batch, Google Cloud Batch, or Azure Batch, provided they correctly launch the distributed processes.

Ultimately, TensorRT-LLM is generally agnostic to the scheduler itself. As long as your launcher correctly starts all ranks, sets up the required communication environment (e.g., MPI/NCCL), and allocates the requested GPUs, the choice of cluster manager is largely independent of TensorRT-LLM.

One question for the TensorRT-LLM team:

• Are there any officially tested or recommended cluster managers besides Slurm, or is any scheduler that can correctly launch MPI-based distributed jobs considered fully supported?

---

If this answer helped or pointed you in the right direction, I'd appreciate it if you could mark it as the accepted answer so it's easier for others with the same issue to find.

Also, if you found my contribution useful, I'd appreciate it if you could check out my GitHub profile, follow me, and star any repositories you find interesting.

GitHub: https://github.com/Advait251206