Kubernetes reference implementation

[doberst]

LLMWare provides several Docker implementation scripts and a devcontainer reference script.

We would welcome contributions from Kubernetes experts to provide a reference Kubernetes configuration and 'fast start' script to deploy llmware in a Kubernetes cluster as well as advising us on additional steps and capabilities that will facilitate Kubernetes scalable deployments.

This is a great first issue if you are an expert in Kubernetes and just starting to learn llmware.

[Lelin07]

@doberst I'm planning to work on the Kubernetes deployment for LLMWare. Given the multi-container setup and the complexity of the project, I have a few questions to ensure the solution meets your expectations. Since this is a "good first issue" and for a "reference" Kubernetes configuration, I want to make sure we align on the requirements.

The project would involve deploying multiple services (MongoDB, Milvus, Neo4j, Pgvector, Qdrant, Redis Stack) and managing inter-service communication, resource allocation, configuration management, external access, scalability (HPA) and monitoring/logging.

Follow-Ups:

1. Scope of Initial Setup: Should the initial Kubernetes configuration focus on a basic setup with Deployments and Services?
2. Resource Requirements: Are there any specific resource requirements (CPU, memory) that should be considered for the deployments to ensure optimal performance?

Thank you for your time and assistance.

[doberst]

@Lelin07 - I am so sorry that this message slipped through the cracks - we have been completely buried the last couple of weeks. Yes, we really appreciate your interest in this, and would welcome a contribution. In terms of scope, we would look for a basic recipe that could be the foundation for further customization depending upon a specific deployment pattern - and so we would encourage it to be more "universal" as a starting point. In terms of CPU/memory, I don't have a specific guideline - but per the comment above, I would aim for a practical basic implementation that could always be scaled up if needed. Please let me know if you have other questions/clarifications - and promise faster replies! 👍

[jothilal22]

Hey @doberst
I came across the issue regarding the reference Kubernetes configuration and 'fast start' script for deploying LLMWare and am very interested in contributing. This seems like a great first step to start working on the project while applying my Kubernetes expertise.Before I begin, could you confirm if this issue is still open and if I can proceed? My plan is to create a basic, universal configuration and fast-start script, as suggested, to serve as a foundation for scalable deployments.

Please let me know if there are any additional guidelines or details to consider. I look forward to your confirmation and further instructions!

[Lelin07]

@doberst Currently I'm not working on this issue. Consider assigning @jothilal22