Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add yaml config file with configurable k8s knobs (#87)
Right now if the user wants to run `weaver kube deploy` it has to
specify various kubernetes knobs in weaver.toml. However, more knobs we
would like the user to control, harder is to make them work using
weaver.toml. We've been trying to expose some knobs (e.g., resource
requirements, probes) in toml. To do that, we had to write wrapper
datastructures that mimic the corresponding kubernetes types. This is
not only ugly, but it's hard to capture all the knobs and makes the code
unnecessarily complicated.
With this PR, the user should be able to write a YAML config where it
can specify deployment information for kubernetes using native go k8s
types. We let the user configure the k8s knobs that are the most
commonly used:
1) resource requirements (cpu/mem requests/limits)
2) scaling spec (min/max replicas, target utilization threshold,
resource to monitor, etc for the horizontal pod autoscaler)
3) probes (liveness, readiness, startup)
4) volumes (any kind of volume like pvc, config maps, secrets, etc.)
For each of these knobs we let the user specify native go k8s types. The
user can copy paste these values directly from their existing yaml
files, or can type it by themselves.
The kube deployer takes a single YAML config that contains a reference
to the TOML app config. The TOML app config can contain only the name of
the binary.
The YAML config also allows the user to specify groups of collocated
components and to fine tune various configs for each group (e.g.,
resource requirements, scaling spec, volumes).
The simplest YAML config to run `weaver kube` deployer looks like:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
```
where the weaver.toml looks like
``` weaver.toml
[serviceweaver]
binary = "./collatz"
```
To add a set of colocated components you can specify a group as follows:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
groups:
- name: mygroup
components:
- component1
- component2
```
To specify resource requirements that apply to all the pods in the app,
you can add a k8s resource spec as follows:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
resourceSpec:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
groups:
- name: mygroup
components:
- component1
- component2
```
If you want to have different resource requirements for the components
in `mygroup` you can set the resourceSpec for `mygroup` as follows:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
resourceSpec:
requests:
memory: "64Mi"
cpu: "250m"
limits:
memory: "128Mi"
cpu: "500m"
groups:
- name: mygroup
components:
- component1
- component2
resourceSpec:
requests:
memory: "72Mi"
cpu: "290m"
limits:
memory: "189Mi"
cpu: "700m"
```
We can do the same thing with scaling spec. E.g., scaling spec:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
scalingSpec:
minReplicas: 1
maxReplicas: 3
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 80
```
To add volumes (per app or per group or both), you have to specify the
name of the volume, the volume source and the volume mount (both volume
source and volume mount are go k8s types). E.g., to add a secret
`my-secret` to all the pods, you can do:
``` kube.yaml
appConfig: weaver.toml
repo: docker.io/your_docker_hub_username
listeners:
- name: foo
public: true
volumes:
- name: robert-secret
volumeSource:
secret:
secretName: my-secret
volumeMount:
mountPath: /etc/secret
```
Fixed Michaels' comments- Loading branch information
