Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: update examples documentation to use of loading from ENV #350

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 63 additions & 28 deletions docs/examples.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,21 @@
## Example NodeSpec Supporting Mulitple Key/Values
# Druid Operator Examples

```
## Example NodeSpec Supporting Multiple Key/Values

```yaml
middlemanagers:
podAnnotations:
type: middlemanager
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
-
matchExpressions:
-
key: node-type
- matchExpressions:
- key: node-type
operator: In
values:
- druid-data
tolerations:
-
effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Exists
Expand All @@ -29,8 +28,7 @@
podDisruptionBudgetSpec:
maxUnavailable: 1
ports:
-
containerPort: 8100
- containerPort: 8100
name: peon-0
replicas: 1
resources:
Expand Down Expand Up @@ -63,18 +61,15 @@
druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000
druid.indexer.fork.property.druid.processing.numThreads=1
services:
-
spec:
- spec:
clusterIP: None
ports:
-
name: tcp-service-port
- name: tcp-service-port
port: 8091
targetPort: 8091
type: ClusterIP
volumeClaimTemplates:
-
metadata:
- metadata:
name: data-volume
spec:
accessModes:
Expand All @@ -84,8 +79,7 @@
storage: 30Gi
storageClassName: zone-a-storage
volumeMounts:
-
mountPath: /druid/data
- mountPath: /druid/data
name: data-volume
containerSecurityContext:
fsGroup: 1001
Expand All @@ -108,13 +102,12 @@
target:
type: Utilization
averageUtilization: 50

```

## Configure Ingress

```
brokers:
```yaml
brokers:
nodeType: "broker"
druid.port: 8080
ingressAnnotations:
Expand All @@ -138,9 +131,9 @@
secretName: tls-broker-druid-cluster
```

## Configure Deployments
## Configure Deployments

```
```yaml
nodes:
brokers:
kind: Deployment
Expand All @@ -155,8 +148,8 @@

## Configure Hot/Cold for Historicals

```
hot:
```yaml
hot:
druid.port: 8083
env:
- name: DRUID_XMS
Expand Down Expand Up @@ -190,7 +183,7 @@
requests:
cpu: 1
memory: 1Gi
runtime.properties:
runtime.properties:
druid.plaintextPort=8083
druid.service=druid/historical/hot
...
Expand Down Expand Up @@ -231,16 +224,17 @@
requests:
cpu: 1
memory: 1Gi
runtime.properties:
runtime.properties:
druid.plaintextPort=8083
druid.service=druid/historical/cold
...
...
...
```

## Configure Additional Containers

```
```yaml
additionalContainer:
- image: universalforwarder-sidekick:next
containerName: forwarder
Expand Down Expand Up @@ -269,4 +263,45 @@
- -application=application
- -instance=instance
- -logFiles=logFiles
```
```

## Using Environment Variables in common.runtime.properties

You can use environment variables in the `*.properties` section of your Druid configuration. This feature allows for more flexible and secure configuration management, especially for sensitive information or values that might change between environments.

To use an environment variable, use the following format:

```yaml
property_name={"type":"environment","variable":"ENVIRONMENT_VARIABLE_NAME"}
```

Here's an example of how you might use this in your `common.runtime.properties`:

```yaml
common.runtime.properties: |
# Zookeeper
druid.zk.service.host=tiny-cluster-zk-0.tiny-cluster-zk
druid.zk.paths.base=/druid
druid.zk.service.compress=false

# Metadata Store
druid.metadata.storage.type=derby
druid.metadata.storage.connector.connectURI={"type":"environment","variable":"CONNECTOR_URI"}
druid.metadata.storage.connector.host={"type":"environment","variable":"DB_HOST"}
druid.metadata.storage.connector.port={"type":"environment","variable":"DB_PORT"}
druid.metadata.storage.connector.createTables=true

```

In this example, values for `CONNECTOR_URI`, `DB_HOST`, `DB_PORT` will be pulled from environment variables, allowing you to set these values separately from your Druid configuration file.

Remember to ensure that these environment variables are properly set in your Kubernetes environment, typically through Secrets or ConfigMaps, before deploying your Druid cluster.

```yaml
env:
- name: CONNECTOR_URI
valueFrom:
secretKeyRef:
key: connector-uri
name: druid-secrets
```
6 changes: 4 additions & 2 deletions examples/tiny-cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ spec:
druid.metadata.storage.connector.host=localhost
druid.metadata.storage.connector.port=1527
druid.metadata.storage.connector.createTables=true
# alternatively, you can load from the environment variable using the following
# druid.metadata.storage.connector.connectURI={"type":"environment","variable":"CONNECTOR_URI"}

# Deep Storage
druid.storage.type=local
Expand Down Expand Up @@ -334,7 +336,7 @@ spec:
extra.jvm.options: |-
-Xmx512M
-Xms512M

routers:
nodeType: "router"
druid.port: 8088
Expand All @@ -354,7 +356,7 @@ spec:
druid.router.coordinatorServiceName=druid/coordinator

# Management proxy to coordinator / overlord: required for unified web console.
druid.router.managementProxy.enabled=true
druid.router.managementProxy.enabled=true
extra.jvm.options: |-
-Xmx512M
-Xms512M