Sync main branch changes to release-0.1 for rc4 release (#312)

* Update manifests version to v0.1.0-rc.3 (#287)

* [Misc] Add sync images step and scripts in release process (#283)

Add sync images step and scripts in release process

* [batch] E2E works with driver and request proxy  (#272)

* e2e driver and test

* comment functions

* check job status in test

* format update

* update copyright

* add examples with instructions and interfaces

* move batch tutorial

---------

Co-authored-by: xin.chen <[email protected]>

* Fix address already in use when AIRuntime start in pod (#289)

add uvicorn startup into file entrypoint

* Read model  name from request body (#290)

* Use model name from request body

* rename dummy to reserved router

* Fix redis bootstrap flaky connection issue (#293)

* skip docs CI if no changes in /docs dir (#294)

* skip docs CI if no changes in /docs dir

* test docs build

* Improve Rayclusterreplicaset Status (#295)

* improve rayclusterreplicaset status
* nit
* fix lint error
* improve isClusterActive logic
* fix lint error
* remove redundant isRayPodCreateOrDeleteFailed check
---------

Signed-off-by: Yicheng-Lu-llll <[email protected]>

* Add request trace for profiling (#291)

* Add request trace for profiling

* add to redis at 10 second interval

* nit

* round to nearest 10s interval

* round timestamp to nearest 10s interval and aggregate data by model

* add go routine to add request trace

* Update the crd definiton due to runtime upgrade (#298)

#295 introduce the latest kuberay api and the dependencies bumps sigs.k8s.io/controller-runtime from v0.17.3 to v0.17.5. Due to that change, make manifest update the CRD definitions

* Push images to Github registry in release pipeline (#301)

* Disable docker build github workflow to cut CI cost

* Push images to Github registry in release pipeline

* Build autoscaler abstractions like fetcher, client and scaler (#300)

* minor clean up on the autoscaler controller

* Extract the algorithm package

algorithm is extracted to distinguish with the scaler.

* Refactor scaler interface

1. Split the Scaler interface and BaseAutoscaler implementation
2. Create APA/KPA scaler separately and adopt the corresponding algorithms

* Introduce the scalingContext in algorithm

* Introduce k8s.io/metrics for resource & custom metrics fetching

* Extract metric fetcher to cover the fetching logic

* Optimize the scaler workflow to adopt fetch and client interface

* Further refactor the code structure

* Support pod autoscaler periodically check (#306)

* Support pod autoscaler periodically check

* Fix the error case

* Add timeout in nc check for redis bootstrap (#309)

* Refactor AutoScaler: metricClient, context, reconcile (#308)

* Refactor AutoScaler: optimize metric client, context, and reconcile processes.

* fix make lint-all

* fix typos

---------

Signed-off-by: Yicheng-Lu-llll <[email protected]>
Co-authored-by: xinchen384 <[email protected]>
Co-authored-by: xin.chen <[email protected]>
Co-authored-by: brosoul <[email protected]>
Co-authored-by: Varun Gupta <[email protected]>
Co-authored-by: Yicheng-Lu-llll <[email protected]>
Co-authored-by: Rong-Kang <[email protected]>

.github/workflows/docker-build-images.yml

@@ -6,6 +6,9 @@ on:
 
 jobs:
   build:
+    # This prevents the job from running as other steps cover its functionality.
+    # We use 'if: false' to keep the file for future reference without deleting it.
+    if: false
     runs-on: ubuntu-latest
     steps:
     - name: Check out code

.github/workflows/release-build.yaml

@@ -24,16 +24,34 @@ jobs:
           username: ${{ secrets.DOCKER_HUB_USERNAME }}
           password: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
 
-      # Build container images
+      # Log in to Github Registry
+      - name: Login to the Container registry
+        uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      # Build container images with docker registry namespace
       - name: Build Container Images
         run: |
           GIT_COMMIT_HASH=${{ github.ref_name }} make docker-build-all
 
-      # Push container image to container registry
+      # Push container image to DockerHub
       - name: Push container image to container registry
         run: |
           GIT_COMMIT_HASH=${{ github.ref_name }} make docker-push-all
 
+      # Build container images with Github registry namespace
+      - name: Build Container Images with Github Container Registry prefix
+        run: |
+          GIT_COMMIT_HASH=${{ github.ref_name }} AIBRIX_CONTAINER_REGISTRY_NAMESPACE=ghcr.io/aibrix make docker-build-all
+
+      # Push container image to Github container registry
+      - name: Push Container Images to Github Container Registry
+        run: |
+          GIT_COMMIT_HASH=${{ github.ref_name }} AIBRIX_CONTAINER_REGISTRY_NAMESPACE=ghcr.io/aibrix make docker-push-all
+
   python-wheel-release:
     runs-on: ubuntu-latest
     strategy:

.readthedocs.yaml

@@ -10,6 +10,19 @@ build:
   os: ubuntu-22.04
   tools:
     python: "3.10"
+  jobs:
+    post_checkout:
+      # Cancel building pull requests when there aren't changed in the docs directory or YAML file.
+      # You can add any other files or directories that you'd like here as well,
+      # like your docs requirements file, or other files that will change your docs build.
+      #
+      # If there are no changes (git diff exits with 0) we force the command to return with 183.
+      # This is a special exit code on Read the Docs that will cancel the build immediately.
+      - |
+        if [ "$READTHEDOCS_VERSION_TYPE" = "external" ] && git diff --quiet origin/main -- docs/ .readthedocs.yaml;
+        then
+          exit 183;
+        fi
 
 # Build documentation in the "docs/" directory with Sphinx
 sphinx:

README.md

@@ -34,10 +34,10 @@ kubectl create -k config/default
 Install stable distribution
 ```shell
 # Install component dependencies
-kubectl create -k "github.com/aibrix/aibrix/config/dependency?ref=v0.1.0-rc.1"
+kubectl create -k "github.com/aibrix/aibrix/config/dependency?ref=v0.1.0-rc.3"
 
 # Install aibrix components
-kubectl create -k "github.com/aibrix/aibrix/config/default?ref=v0.1.0-rc.1"
+kubectl create -k "github.com/aibrix/aibrix/config/default?ref=v0.1.0-rc.3"
 ```
 
 ## Documentation

cmd/controllers/main.go

@@ -186,7 +186,7 @@ func main() {
 		panic(err)
 	}
 
-	cache.NewCache(config, stopCh)
+	cache.NewCache(config, stopCh, nil)
 
 	// Kind controller registration is encapsulated inside the pkg/controller/controller.go
 	// So here we can use more clean registration flow and there's no need to change logics in future.

cmd/plugins/main.go

@@ -51,7 +51,7 @@ func main() {
 	// Connect to Redis
 	redisClient := utils.GetRedisClient()
 
-	fmt.Println("Starting cache")
+	fmt.Println("starting cache")
 	stopCh := make(chan struct{})
 	defer close(stopCh)
 	var config *rest.Config
@@ -71,7 +71,7 @@ func main() {
 		panic(err)
 	}
 
-	cache.NewCache(config, stopCh)
+	cache.NewCache(config, stopCh, redisClient)
 
 	// Connect to K8s cluster
 	k8sClient, err := kubernetes.NewForConfig(config)
@@ -90,7 +90,7 @@ func main() {
 	extProcPb.RegisterExternalProcessorServer(s, gateway.NewServer(redisClient, k8sClient))
 	healthPb.RegisterHealthServer(s, &gateway.HealthServer{})
 
-	klog.Info("Starting gRPC server on port :50052")
+	klog.Info("starting gRPC server on port :50052")
 
 	// shutdown
 	var gracefulStop = make(chan os.Signal, 1)

config/crd/bases/model.aibrix.ai_modeladapters.yaml

@@ -37,6 +37,7 @@ spec:
               credentialsSecretRef:
                 properties:
                   name:
+                    default: ""
                     type: string
                 type: object
                 x-kubernetes-map-type: atomic
@@ -53,11 +54,13 @@ spec:
                           items:
                             type: string
                           type: array
+                          x-kubernetes-list-type: atomic
                       required:
                       - key
                       - operator
                       type: object
                     type: array
+                    x-kubernetes-list-type: atomic
                   matchLabels:
                     additionalProperties:
                       type: string