[WIP] Training: Initial Documentation for Kubeflow Trainer V2 #3958
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| +++ | ||
| title = "Kubeflow Trainer" | ||
| description = "Documentation for Kubeflow Trainer" | ||
| weight = 20 | ||
| +++ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| +++ | ||
| title = "Contributor Guides" | ||
| description = "Documentation for Kubeflow Trainer contributors" | ||
| weight = 60 | ||
| +++ | ||
|
|
||
| This doc is in progress... |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| +++ | ||
| title = "Community Guide" | ||
| description = "How to get involved to Kubeflow Trainer community" | ||
| weight = 20 | ||
| +++ | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| +++ | ||
|
There was a problem hiding this comment. this is under discussion, given that other components do not have this content on the website. we want to ensure this is consistent across the website. |
||
| title = "Contributing Guide" | ||
| description = "How to contribute to Kubeflow Trainer project" | ||
| weight = 10 | ||
| +++ | ||
|
|
||
| This doc is in progress... | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,29 @@ | ||||||
| +++ | ||||||
| title = "Getting Started" | ||||||
| description = "Get Started with Kubeflow Trainer" | ||||||
| weight = 30 | ||||||
| +++ | ||||||
|
|
||||||
| This guide describes how to get started with Kubeflow Trainer and run distributed training | ||||||
| with PyTorch. | ||||||
|
|
||||||
| ## Prerequisites | ||||||
|
|
||||||
| Ensure that you have access to a Kubernetes cluster with Kubeflow Trainer | ||||||
| control plane installed. If it is not set up yet, followÍ | ||||||
|
There was a problem hiding this comment.
Suggested change
|
||||||
| [the installation guide](/docs/components/trainer/operator-guides/installation) to quickly deploy | ||||||
| Kubeflow Trainer on your local Kind cluster. | ||||||
|
There was a problem hiding this comment.
Suggested change
It may be just better straight "quickly deploy Kubeflow Trainer". The provided link gives the specifics. |
||||||
|
|
||||||
| ### Installing the Kubeflow Python SDK | ||||||
|
|
||||||
| Install the latest Kubeflow Python SDK version directly from the source repository: | ||||||
|
|
||||||
| ```bash | ||||||
| pip install git+https://github.com/kubeflow/training-operator.git@master#subdirectory=sdk_v2 | ||||||
| ``` | ||||||
|
|
||||||
| TODO (andreyvelich): Add command once we release SDK to PyPI: https://pypi.org/project/kubeflow | ||||||
|
|
||||||
| ## Getting Started with PyTorch | ||||||
|
|
||||||
| TODO (andreyvelich): Add example from the Notebook | ||||||
|
There was a problem hiding this comment. what about or just remove the section until is fully ready There was a problem hiding this comment. I will add the getting started example once we finish this PR with @astefanutti: kubeflow/training-operator#2387 |
||||||
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| +++ | ||
| title = "Legacy Kubeflow Training Operator (v1)" | ||
| description = "Kubeflow Training Operator V1 Documentation" | ||
| weight = 999 | ||
| +++ | ||
|
|
||
| {{% alert title="Old Version" color="warning" %}} | ||
| This page is about **Kubeflow Training Operator V1**, for the latest information check | ||
| [the Kubeflow Trainer V2 documentation](/docs/components/trainer). | ||
|
|
||
| Follow [this guide for migrating to Kubeflow Trainer V2](/docs/components/trainer/operator-guides/migration) | ||
|
There was a problem hiding this comment. my two cents here is that given that the component's name changed should we said V2? or just Kubeflow Trainer There was a problem hiding this comment. Maybe we could just say: |
||
| {{% /alert %}} | ||
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
File renamed without changes
|
There was a problem hiding this comment. in installing the control plane, https://v1-9-branch.kubeflow.org/docs/started/installing-kubeflow/ There was a problem hiding this comment. Actually, the latest version of Kubeflow Platform 1.10 will also include Training Operator v1. There was a problem hiding this comment. In the Next Steps section , the link is associated with the latest getting started guide instead of v1 Run your first Training Operator Job by following the Getting Started guide. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,13 +5,13 @@ weight = 10 | |
| +++ | ||
|
|
||
| This page shows how Training Operator implements the | ||
| [API to fine-tune LLMs](/docs/components/trainer/legacy-v1/user-guides/fine-tuning). | ||
|
|
||
| ## Architecture | ||
|
|
||
| In the following diagram you can see how `train` Python API works: | ||
|
|
||
| <img src="/docs/components/trainer/legacy-v1/images/fine-tune-llm-api.drawio.svg" | ||
|
There was a problem hiding this comment. It'l be great if we can update the links to the kubernetes documentation to the kubernetes version supported by the V1 legacy https://v1-28.docs.kubernetes.io/docs/concepts/storage/persistent-volumes/ for eadOnlyMany access mode There was a problem hiding this comment. Do we really need to do it here ? |
||
| alt="Fine-Tune API for LLMs" | ||
| class="mt-3 mb-3"> | ||
|
|
||
|
|
||
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is under discussion, given that other components do not have this content on the website.we want to ensure this is consistent across the website.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I created an issue to reflect the conversation we had and made a few updates, feel free to make any suggestions
#3971
but the main idea is to not have individual pages on each project on the website, but continue one centralized place on the website and links to the git repos.