-
Notifications
You must be signed in to change notification settings - Fork 1.9k
microshift AI jtbd #118684
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
microshift AI jtbd #118684
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: configure-hardware-accelerators-for-ai-workloads | ||
|
|
||
| include::modules/microshift-configure-hardware-accelerators-for-ai-workloads.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-workflow.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-install.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-inferenceservice-more-options.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: customize-serving-environment | ||
|
|
||
| include::modules/microshift-customize-serving-environment.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-override-kserve-config.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: expose-and-query-ai-models | ||
|
|
||
| include::modules/microshift-expose-and-query-ai-models.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-create-route.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-query-model-con.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-query-model.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-verify-model-connected.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-get-model-ready-inference.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: install-ai-capabilities | ||
|
|
||
| include::modules/microshift-install-ai-capabilities.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-con.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-workflow.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-install.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: monitor-model-performance | ||
|
|
||
| include::modules/microshift-monitor-model-performance.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-get-model-server-metrics.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-export-metrics-otel.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: prepare-and-package-ai-models | ||
|
|
||
| include::modules/microshift-prepare-and-package-ai-models.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-create-ns.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-model-package-oci.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-workflow.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: remove-ai-model-serving-components | ||
|
|
||
| include::modules/microshift-remove-ai-model-serving-components.adoc[leveloffset=+0,chunk="to-content"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: secure-ai-inference-endpoints | ||
|
|
||
| include::modules/microshift-secure-ai-inference-endpoints.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-create-route.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: serve-ai-models | ||
|
|
||
| include::modules/microshift-serve-ai-models.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-serving-ai-models-con.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-supported-crds.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-supported-mserv-runtimes.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-servingruntimes-ex.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-inferenceservice-ex.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-model-serving-rt-verify.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: update-running-ai-model | ||
|
|
||
| include::modules/microshift-update-running-ai-model.adoc[leveloffset=+0,chunk="to-content"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| :_mod-docs-content-type: MAP | ||
| :context: use-alternative-storage-for-ai-models | ||
|
|
||
| include::modules/microshift-use-alternative-storage-for-ai-models.adoc[leveloffset=+0,chunk="to-content"] | ||
|
|
||
| include::modules/microshift-rhoai-model-package-oci.adoc[leveloffset=+1,toc="no"] | ||
|
|
||
| include::modules/microshift-rhoai-workflow.adoc[leveloffset=+1,toc="no"] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| :_mod-docs-content-type: MAP | ||
|
|
||
| = AI | ||
|
|
||
| include::jobs/install-ai-capabilities.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/prepare-and-package-ai-models.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/serve-ai-models.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/expose-and-query-ai-models.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/monitor-model-performance.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/customize-serving-environment.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/use-alternative-storage-for-ai-models.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/configure-hardware-accelerators-for-ai-workloads.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/update-running-ai-model.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/secure-ai-inference-endpoints.adoc[leveloffset=+1] | ||
|
|
||
| include::jobs/remove-ai-model-serving-components.adoc[leveloffset=+1] |
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/configure-hardware-accelerators-for-ai-workloads.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-configure-hardware-accelerators-for-ai-workloads_{context}"] | ||
| = Configure hardware accelerators for AI workloads | ||
|
|
||
| [role="_abstract"] | ||
| Configure GPU or NPU accelerators and device plugins so your {microshift-short} node can run inference workloads that require specialized hardware. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/customize-serving-environment.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-customize-serving-environment_{context}"] | ||
| = Adapt AI model serving to your edge constraints | ||
|
|
||
| [role="_abstract"] | ||
| Override the KServe configuration when the default model-serving environment does not meet your edge deployment constraints. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/expose-and-query-ai-models.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-expose-and-query-ai-models_{context}"] | ||
| = Access your AI model from external applications | ||
|
|
||
| [role="_abstract"] | ||
| Create a network route and query your running model so you can make external inferences and test model responses. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/install-ai-capabilities.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-install-ai-capabilities_{context}"] | ||
| = Enable AI model serving on {microshift-short} | ||
|
|
||
| [role="_abstract"] | ||
| Install the {rhoai-full} RPM on your {microshift-short} node so you can deploy and serve AI models in edge environments. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/monitor-model-performance.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-monitor-model-performance_{context}"] | ||
| = Monitor model performance | ||
|
|
||
| [role="_abstract"] | ||
| Export and view model-server metrics so you can identify bottlenecks, optimize resource allocation, and ensure efficient infrastructure utilization. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/prepare-and-package-ai-models.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-prepare-and-package-ai-models_{context}"] | ||
| = Package your AI model for edge deployment | ||
|
|
||
| [role="_abstract"] | ||
| Create a namespace and package your AI model into an OCI image in ModelCar format so you can securely store and deploy the model to your {microshift-short} node. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/remove-ai-model-serving-components.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-remove-ai-model-serving-components_{context}"] | ||
| = Remove AI model serving components and free resources | ||
|
|
||
| [role="_abstract"] | ||
| When you no longer need to serve a model, delete the `InferenceService`, clean up OCI images, and uninstall the `microshift-ai-model-serving` RPM to manage resources on constrained edge nodes. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| Documentation for uninstalling AI model serving components is not yet available in the {microshift-short} AI chapter. | ||
| ==== |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/secure-ai-inference-endpoints.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-secure-ai-inference-endpoints_{context}"] | ||
| = Secure your AI inference endpoints | ||
|
|
||
| [role="_abstract"] | ||
| Add TLS encryption and authentication to inference routes so you can protect model access and meet security requirements. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/serve-ai-models.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-serve-ai-models_{context}"] | ||
| = Serve AI models on {microshift-short} | ||
|
|
||
| [role="_abstract"] | ||
| Configure a model-serving runtime and create an `InferenceService` custom resource so you can serve your AI model on {microshift-short}. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/update-running-ai-model.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-update-running-ai-model_{context}"] | ||
| = Update a running AI model to a newer version | ||
|
|
||
| [role="_abstract"] | ||
| When a newer version of your AI model is available, update the served model without downtime so you can deliver improved inference results. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| Documentation for updating a running model is not yet available in the {microshift-short} AI chapter. | ||
| ==== |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * maps/jobs/use-alternative-storage-for-ai-models.adoc | ||
|
|
||
| :_mod-docs-content-type: CONCEPT | ||
| [id="microshift-use-alternative-storage-for-ai-models_{context}"] | ||
| = Use alternative storage for your AI models | ||
|
|
||
| [role="_abstract"] | ||
| When you already have S3-compatible storage or a persistent volume claim, you can point your `InferenceService` to that storage instead of packaging a ModelCar OCI image. Only the ModelCar approach is tested and supported for {microshift-short}. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🤖 [error] OpenShiftAsciiDoc.ModuleContainsContentType: Module is missing the '_mod-docs-content-type' variable.