[WIP] Training: Initial Documentation for Kubeflow Trainer V2 #3958
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,5 @@ | ||
| +++ | ||
| title = "Kubeflow Training" | ||
|
There was a problem hiding this comment. What do you think about calling it "Kubeflow Trainer"? 🤔 There was a problem hiding this comment. Hmm, this could be one of the options to name this project Kubeflow Trainer/KFTrainer. There was a problem hiding this comment. It would follow HuggingFace's and Lightning.ai's, "trainer" convention: https://huggingface.co/docs/transformers/en/main_classes/trainer There was a problem hiding this comment. So we have these: CRD Names:
SDK APIs: from kubeflow.trainer import TrainerClient, Trainer
TrainerClient().train(
Trainer(
func=train_func,
),
num_nodes=5,
runtime_ref="torch-distributed"
)
# For LLMs
from kubeflow.trainer import TrainerClient, Trainer, FineTuningConfig, LoraConfig
TrainerClient().train(
Trainer(
fine_tuning_config=FineTuningConfig(
peft_config=LoraConfig(
r=4,
)
),
num_nodes=5,
runtime_ref="llama-3.2-8b",
)What do we think about these names ? There was a problem hiding this comment. cc @astefanutti @kubeflow/wg-training-leads @Electronic-Waste @deepanker13 @saileshd1402 @seanlaii @kannon92 There was a problem hiding this comment. @andreyvelich I don't like that Kubeflow platform combines the products together? Tell me more! I love the idea of bringing components together but also love the idea of installing them separately. If I remember correctly, there is someone dying on that hill :-). That being said, we still don't have conformance.. we still don't have a unified definition of Kubeflow..so it might be a bit early to rename anything tbh. It feels like procrastination chores when we've got big fish to fry here. Also,
Why not? What are you optimizing for and based on what demand? Have we opened this up to the greater community? Who are our "customers" so to speak and how would they want APIs/Components labeled? This seems like a big decision to make in a vacuum and thought leadership is service (Kelsey has for sure played this game well). What if we opened it up to the greater community? Posted some options on poles via socials we as a community publish? We can then be more data driven. I bet @StefanoFioravanzo has some opinions! There was a problem hiding this comment. One concern to me is that training operator is not limited to training use case but can be expanded to parallel computing (via MPIJob). Similarly, There was a problem hiding this comment.
My points don't imply that Kubeflow products cannot function as standalone applications. However, for users interested in seeing how these individual open-source projects integrate seamlessly, the Kubeflow Platform provides a comprehensive, end-to-end machine learning experience.
I believe it will take our community significantly more time to discuss this thoroughly (approximately 1–2 years given the user base of Kubeflow Pipelines).
That is correct, but right now it is out of scope of supported CRDs (TrainJob and TrainingRuntime). Theoretically, users can leverage TrainingRuntimes and TrainJob for distributed inference with MPI, but it would be better if we create dedicated CRDs for it. Also, our All in all, naming this project
from kubeflow.trainer import TrainerClient
# Get available runtimes.
TrainerClient().list_runtimes()
# Train my ML model
TrainerClient().train(
runtime_ref="torch-distributed",
trainer=Trainer(
func=train_func,
func_args={"lr": 0.01},
num_nodes=100,
resources_per_node={"gpu": 5},
),
)We can always revisit the project name in the future if users tell us that this experience is bad. There was a problem hiding this comment. I would support this naming convention, it would be more clear to users than There was a problem hiding this comment. I'm ok with But I agree with here discussions since Note that Previously, we were seeking the Kubeflow |
||
| description = "Documentation for Kubeflow Training" | ||
| weight = 20 | ||
| +++ | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| +++ | ||
| title = "Contributor Guides" | ||
| description = "Documentation for Kubeflow Training contributors" | ||
| weight = 60 | ||
| +++ |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| +++ | ||
| title = "Contributing Guide" | ||
| description = "How to contribute to Kubeflow Training project" | ||
| weight = 10 | ||
| +++ | ||
|
|
||
| This document describes how to contribute to Kubeflow Training project. | ||
|
There was a problem hiding this comment. I see inconsistent use of "This document". "This guide", "This page" throughout this PR. We can just start the sentence with something like "To contribute to the Kubeflow Training project... " or something similar. Just a suggestion. There was a problem hiding this comment. I think, historically we've been using this in the beginning of the page: However, some pages have various messages. There was a problem hiding this comment. @andreyvelich based on previous conversation, we will not be having a specific page for the contribution/community. There was a problem hiding this comment. What do you mean @varodrig ? Even right now, individual Kubeflow projects have their own contributor guides:
There was a problem hiding this comment. I created an issue to reflect the conversation we had and made a few updates, feel free to make any suggestions |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,158 +1,34 @@ | ||
| +++ | ||
| title = "Getting Started" | ||
| description = "Get Started with Kubeflow Training" | ||
| weight = 30 | ||
| +++ | ||
|
|
||
| This guide describes how to get started with Kubeflow Training and run distributed training | ||
| with PyTorch. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Ensure that you have access to a Kubernetes cluster with Kubeflow Training | ||
| control plane installed. If it is not set up yet, follow | ||
| [the installation guide](/docs/components/training/operator-guides/installation) to quickly deploy | ||
| Kubeflow Training on a local Kind cluster. | ||
|
|
||
| ### Installing the Kubeflow Python SDK | ||
|
|
||
| Install the Kubeflow Python SDK to interact with Kubeflow Training APIs: | ||
|
|
||
| ```bash | ||
| pip install kubeflow | ||
| ``` | ||
|
|
||
| Alternatively, you can install the latest Kubeflow Python SDK version directly | ||
|
There was a problem hiding this comment. We can eliminate "you can" to reduce wordiness. |
||
| from the source repository: | ||
|
|
||
| ```bash | ||
| pip install git+https://github.com/kubeflow/training-operator.git@master#subdirectory=sdk_v2 | ||
| ``` | ||
|
|
||
| ## Getting Started with PyTorch | ||
|
|
||
| TODO (andreyvelich): Add example from the Notebook | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| +++ | ||
| title = "Legacy (v1)" | ||
| description = "Kubeflow Training V1 Documentation" | ||
| weight = 999 | ||
| +++ | ||
|
|
||
| {{% alert title="Old Version" color="warning" %}} | ||
| This page is about **Kubeflow Training V1**, please see the [V2 documentation](/docs/components/training) for the latest information. | ||
|
There was a problem hiding this comment. Style guides typically recommend against using "please". |
||
|
|
||
| Please follow [this guide for migrating to Kubeflow Training V2](/docs/components/training/operator-guides/migration) | ||
| {{% /alert %}} | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should there be a comma between "template" and "check"?