You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Firstly, thanks a lot for this community effort! We are currently running several KF1.2 distributions on AWS where we integrate as tightly as possible with AWS managed services and using your solution for S3/RDS in Pipelines is an important part of that. We offload on-cluster services to AWS as far as possible. This means in particular:
Using Cognito for user pool management
Using RDS for all metadata storage (pipelines, metadb, cachedb, katibdb) instead of an on-cluster MySQL db
Using S3 for all pipeline artifact storage instead of using Minio for on-cluster storage
Using IRSA to manage granular pod-level access to AWS resources
In addition, we use ArgoCD as GitOps operator try to avoid any middleware such as kftctl / Kubeflow Operator we rolling out. We have started a new community effort that aims to do that for Kubeflow 1.3 here: https://github.com/argoflow/argoflow-aws (currently still under construction). We plan to integrate the solution that you have developed here as well, and would also welcome any direct contributions from you!
Most important for us right now though is that KF 1.3 used Pipelines 1.5.0. Your latest version is for Pipelines 1.4.1. Are you aware of any changes that are necessary to upgrade to 1.5.0 and do you intend to upgrade soon?
The text was updated successfully, but these errors were encountered:
@karlschriek
This sound exciting! RDS support was what I always wanted to do but never found time for it.
Would be very happy to collab and contribute!
Haven't been following the recent updates to kfp, so not sure what are the major changes. But let me spend some time to have a look at the recent commits.
Firstly, thanks a lot for this community effort! We are currently running several KF1.2 distributions on AWS where we integrate as tightly as possible with AWS managed services and using your solution for S3/RDS in Pipelines is an important part of that. We offload on-cluster services to AWS as far as possible. This means in particular:
In addition, we use ArgoCD as GitOps operator try to avoid any middleware such as kftctl / Kubeflow Operator we rolling out. We have started a new community effort that aims to do that for Kubeflow 1.3 here: https://github.com/argoflow/argoflow-aws (currently still under construction). We plan to integrate the solution that you have developed here as well, and would also welcome any direct contributions from you!
Most important for us right now though is that KF 1.3 used Pipelines 1.5.0. Your latest version is for Pipelines 1.4.1. Are you aware of any changes that are necessary to upgrade to 1.5.0 and do you intend to upgrade soon?
The text was updated successfully, but these errors were encountered: