-
Notifications
You must be signed in to change notification settings - Fork 13
home
The RoCE Networking-Based LLM Post-Training Solution with Intel Enterprise AI Foundation for OpenShift delivers a end-to-end solution for large language model (LLM) post-training workloads for enterprise AI area. By harnessing Intel's advanced hardware, such as Gaudi accelerators, and the built-in RoCE, it provides a cost-efficient, scalable and high-performance solution.
The seamless integration with Red Hat OpenShift and OpenShift AI provides a production-grade platform for deploying and managing AI workloads, offering enterprises a robust, flexible, and user-friendly environment.
The Intel Technology Enabling for OpenShift project provides Intel Data Center hardware feature-provisioning technologies with the Red Hat OpenShift Container Platform (RHOCP). The technology to deploy and manage Intel Enterprise AI End-to-End (E2E) solutions and the related reference workloads for these features are also included in the project.
Fast GPU Provisioning technology enables GPU provisioning less than 1 second with no reboots using pre-built driver containers. The feature eliminates any dependency on machine configuration which triggers reboot, an expensive operation. Instead, the required operations are performed at runtime. This leads to a simplified and accelerated deployment process.
When the containers need access to device files they usually need to run as root UID/GID as 0/0. But when the device plugins make the device files available to the workload containers, it is owned by root and thus the containers need to run as root. But it is not a good security practice. So its always a good idea to run containers as rootless. Here is short tutorial on how to run the Intel Device plugins so they the workload containers can run as rootless. By default this is not turned on.