Skip to content

Commit

Permalink
various minor updates (#1071)
Browse files Browse the repository at this point in the history
* update image tag
* recommend kubectl creation for sc
* headsup on preStop
  • Loading branch information
timfeirg authored Aug 8, 2024
1 parent 1cea65b commit 1f02d36
Show file tree
Hide file tree
Showing 8 changed files with 104 additions and 64 deletions.
15 changes: 12 additions & 3 deletions docs/en/guide/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,20 @@ globalConfig:
matchLabels:
...
annotations:
# delayed mount pod deletion
# Delayed mount pod deletion
juicefs-delete-delay: 5m
# clean cache when mount pod exits
# Clean cache when mount pod exits
juicefs-clean-cache: "true"


# Select by StorageClass
- pvcSelector:
matchStorageClassName: juicefs-sc
terminationGracePeriodSeconds: 60

# Select by PVC
- pvcSelector:
matchName: pvc-name
terminationGracePeriodSeconds: 60
```
### Inherit from CSI Node (deprecated) {#inherit-from-csi-node}
Expand Down
18 changes: 9 additions & 9 deletions docs/en/guide/custom-image.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ To avoid misuse and reduce image size, from CSI Driver 0.19.0, separated image i

```shell
# Tag of community mount image begin with ce-
juicedata/mount:ce-v1.1.2
juicedata/mount:ce-v1.2.0

# Tag of enterprise mount image begin with ee-
juicedata/mount:ee-5.0.17-8ba7611
juicedata/mount:ee-5.0.23-8c7c134

# Prior to 0.19.0, tag contains both CE and EE version string
# This won't be maintained and updated in the future
Expand All @@ -44,7 +44,7 @@ If you use Helm to manage CSI Driver, changing mount image is as simple as the f
```yaml
defaultMountImage:
# Community Edition
ce: "juicedata/mount:ce-v1.1.2"
ce: "juicedata/mount:ce-v1.2.0"
# Enterprise Edition
ee: "juicedata/mount:ee-5.0.10-10fbc97"
```
Expand All @@ -53,12 +53,12 @@ But if you use kubectl to directly install a `k8s.yaml`, you'll have to set envi

```shell
# Community Edition
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.1.2
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.1.2
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.2.0
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.2.0
# Enterprise Edition
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.17-8ba7611
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.17-8ba7611
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.23-8c7c134
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.23-8c7c134
```

Also, don't forget to put these changes into `k8s.yaml`, to avoid losing these changes after the next installation. This is why we always recommend you use [Helm installation](../getting_started.md#helm) for production environments.
Expand All @@ -78,7 +78,7 @@ parameters:
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
juicefs/mount-image: juicedata/mount:ce-v1.1.2
juicefs/mount-image: juicedata/mount:ce-v1.2.0
```

And then in PVC definitions, reference the needed StorageClass via the `storageClassName` field, so that you may use different mount pod image for different applications.
Expand Down Expand Up @@ -109,7 +109,7 @@ spec:
name: juicefs-secret
namespace: default
volumeAttributes:
juicefs/mount-image: juicedata/mount:ce-v1.1.2
juicefs/mount-image: juicedata/mount:ce-v1.2.0
```

## Build image
Expand Down
41 changes: 24 additions & 17 deletions docs/en/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,8 @@ spec:
volumeMounts:
- mountPath: /data
name: data
# Propagation must be added for automatic mount point recovery (if mount pod ever fails)
mountPropagation: HostToContainer
resources:
requests:
cpu: 10m
Expand All @@ -339,6 +341,26 @@ After pod is up and running, you'll see `out.txt` being created by the container

Due to StorageClass being the template used for creating PVs, **modifying mount options in StorageClass will not affect existing PVs,** if you need to adjust mount options under dynamic provisioning, you'll have to delete existing PVCs, or [directly modify mount options in existing PVs](./configurations.md#static-mount-options).

### Create via kubectl {#kubectl-sc}

Different from our usual recommendations, we advise you against managing StorageClass via Helm, because StorageClass is associated with volume credentials in `values.yaml`, and people usually don't want plain text credentials in `values.yaml`.

To create a StorageClass with kubectl, you must first prepare the [volume credentials](#volume-credentials) referenced in the StorageClass definition, create them in advance, and then fill in its information into the StorageClass definition shown below.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
reclaimPolicy: Retain
```

### Create via Helm {#helm-sc}

:::tip
Expand Down Expand Up @@ -375,23 +397,6 @@ storageClasses:
memory: "5Gi"
```

### Create via kubectl

[Volume credentials](#volume-credentials) is referenced in the StorageClass definition, so you'll have to create them in advance, and then fill in its information into the StorageClass definition shown below.

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
```

## Dynamic provisioning {#dynamic-provisioning}

Read [Usage](../introduction.md#usage) to learn about dynamic provisioning. Dynamic provisioning automatically creates PV for you, each corresponds to a sub-directory inside the JuiceFS volume, and the parameters needed by PV resides in StorageClass, thus you'll have to [create a StorageClass](#create-storage-class) in advance.
Expand Down Expand Up @@ -431,6 +436,7 @@ spec:
volumeMounts:
- mountPath: /data
name: juicefs-pv
mountPropagation: HostToContainer
volumes:
- name: juicefs-pv
persistentVolumeClaim:
Expand Down Expand Up @@ -472,6 +478,7 @@ spec:
volumeMounts:
- mountPath: /data
name: juicefs-pv
mountPropagation: HostToContainer
volumes:
- name: juicefs-pv
ephemeral:
Expand Down
6 changes: 5 additions & 1 deletion docs/zh_cn/administration/going-production.md
Original file line number Diff line number Diff line change
Expand Up @@ -386,10 +386,14 @@ authorization:
* 启用[延迟删除](../guide/resource-optimization.md#delayed-mount-pod-deletion),即便应用 pod 退出,mount pod 也会等待指定时间后,才由 CSI Node 销毁。合理设置延时,保证数据及时上传完成;
* 自 v0.24 起,CSI 驱动支持定制 mount pod 的方方面面,因此可以修改 `terminationGracePeriodSeconds`,再配合 `preStop` 实现等待数据上传完成后,mount pod 才退出,示范如下:

:::warning
仔细阅读并合理调整下方代码块的各项设置和检测脚本,不当的设置可能导致 mount pod 长时间无法退出,请充分测试并理解对应的风险。
:::

```yaml title="values-mycluster.yaml"
globalConfig:
mountPodPatch:
- terminationGracePeriodSeconds: 3600
- terminationGracePeriodSeconds: 600
lifecycle:
preStop:
exec:
Expand Down
10 changes: 10 additions & 0 deletions docs/zh_cn/guide/configurations.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,16 @@ globalConfig:
juicefs-delete-delay: 5m
# 退出时清理 cache
juicefs-clean-cache: "true"

# 选择特定的 StorageClass
- pvcSelector:
matchStorageClassName: juicefs-sc
terminationGracePeriodSeconds: 60

# 选择特定的 PVC
- pvcSelector:
matchName: pvc-name
terminationGracePeriodSeconds: 60
```
### 通过继承 CSI Node 配置进行定制(不推荐) {#inherit-from-csi-node}
Expand Down
20 changes: 10 additions & 10 deletions docs/zh_cn/guide/custom-image.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ Mount Pod 中运行着 JuiceFS 客户端,而 JuiceFS 又提供[「社区版」

```shell
# 社区版镜像标签以 ce- 开头
juicedata/mount:ce-v1.1.2
juicedata/mount:ce-v1.2.0

# 商业版镜像标签以 ee- 开头
juicedata/mount:ee-5.0.17-8ba7611
juicedata/mount:ee-5.0.23-8c7c134

# 在 0.19.0 以前,镜像标签中包含社区版和商业版客户端的版本号
# 该系列镜像不再继续更新维护
Expand All @@ -44,21 +44,21 @@ JuiceFS CSI 驱动 0.17.1 及以上版本支持自定义 Mount Pod 镜像,有
```yaml
defaultMountImage:
# 社区版
ce: "juicedata/mount:ce-v1.1.2"
ce: "juicedata/mount:ce-v1.2.0"
# 企业版
ee: "juicedata/mount:ee-5.0.17-8ba7611"
ee: "juicedata/mount:ee-5.0.23-8c7c134"
```
而如果是 kubectl 直接安装,那么需要在 CSI 驱动的组件中设置环境变量:
```shell
# 社区版
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.1.2
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.1.2
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.2.0
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_CE_MOUNT_IMAGE=juicedata/mount:ce-v1.2.0

# 企业版
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.17-8ba7611
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.17-8ba7611
kubectl -n kube-system set env daemonset/juicefs-csi-node -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.23-8c7c134
kubectl -n kube-system set env statefulset/juicefs-csi-controller -c juicefs-plugin JUICEFS_EE_MOUNT_IMAGE=juicedata/mount:ee-5.0.23-8c7c134
```

修改完毕以后,别忘了将这些配置同时加入 `k8s.yaml`,避免下次安装时配置丢失。正因为 kubectl 的安装方式管理配置不方便,所以建议在生产集群采用 [Helm 安装方式](../getting_started.md#helm)
Expand All @@ -78,7 +78,7 @@ parameters:
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
juicefs/mount-image: juicedata/mount:ce-v1.1.2
juicefs/mount-image: juicedata/mount:ce-v1.2.0
```
配置完成后,在不同的 PVC 中,通过 `storageClassName` 指定不同的 StorageClass,便能为不同的应用设置不同的 Mount Pod 镜像了。
Expand Down Expand Up @@ -109,7 +109,7 @@ spec:
name: juicefs-secret
namespace: default
volumeAttributes:
juicefs/mount-image: juicedata/mount:ce-v1.1.2
juicefs/mount-image: juicedata/mount:ce-v1.2.0
```

## 构建镜像
Expand Down
46 changes: 28 additions & 18 deletions docs/zh_cn/guide/pv.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ spec:
volumeMounts:
- mountPath: /data
name: data
mountPropagation: HostToContainer
resources:
requests:
cpu: 10m
Expand All @@ -337,7 +338,30 @@ Pod 创建完成后,你就能在 JuiceFS 挂载点看到上方容器写入的

[StorageClass](https://kubernetes.io/zh-cn/docs/concepts/storage/storage-classes)(存储类)里指定了创建 PV 所需的各类配置,你可以将其理解为动态配置下的「Profile」:不同的 StorageClass 就是不同的 Profile,可以在其中指定不同的文件系统认证信息、挂载配置,让动态配置下可以同时使用不同的文件系统,或者指定不同的挂载。因此如果你打算以[「动态配置」](#dynamic-provisioning)或[「通用临时卷」](#general-ephemeral-storage)的方式使用 JuiceFS CSI 驱动,那么你需要提前创建 StorageClass。

注意,StorageClass 仅仅是动态配置下用于创建 PV 的「模板」,也正因此,**在 StorageClass 中修改挂载配置,不影响已经创建的 PV。** 如果你需要调整挂载配置,需要删除 PVC 重建,或者直接[在 PV 级别调整挂载配置](./configurations.md#static-mount-options)。
注意,StorageClass 仅仅是动态配置下用于创建 PV 的「模板」,也正因此,使用的时候必须注意:

* **在 StorageClass 中修改挂载配置,不影响已经创建的 PV。** 对于已有的 PV,需要直接[在 PV 级别调整挂载配置](./configurations.md#static-mount-options),或者干脆删除 PVC 重建;
* 自 v0.24.3 开始,可以在 [ConfigMap](./configurations.md#configmap) 中使用 `matchStorageClassName` 来方便地选择已有 PVC,更推荐用这种方式来修改 StorageClass 相关配置。

### 通过 kubectl 创建 {#kubectl-sc}

大部分情况下,我们推荐用 Helm 安装和管理 CSI 驱动,但 StorageClass 除外,这是由于他和文件系统认证信息存在引用关系,而 `values.yaml` 往往不便用明文保存认证信息。因此对于 StorageClass 和 Secret,我们优先推荐用 kubectl 手动创建和管理。

在创建 StorageClass 之前,需要提前创建好[「文件系统认证信息」](#volume-credentials),然后将相关信息按照下方示范填入对应字段。

```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
reclaimPolicy: Retain
```

### 通过 Helm 创建 {#helm-sc}

Expand Down Expand Up @@ -375,23 +399,6 @@ storageClasses:
memory: "5Gi"
```
### 通过 kubectl 创建 {#kubectl-sc}
用 kubectl 创建 StorageClass,需要提前创建好[「文件系统认证信息」](#volume-credentials),创建完毕后,将相关信息按照下方示范填入对应字段。
```yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: juicefs-sc
provisioner: csi.juicefs.com
parameters:
csi.storage.k8s.io/provisioner-secret-name: juicefs-secret
csi.storage.k8s.io/provisioner-secret-namespace: default
csi.storage.k8s.io/node-publish-secret-name: juicefs-secret
csi.storage.k8s.io/node-publish-secret-namespace: default
```
## 动态配置 {#dynamic-provisioning}
阅读[「使用方式」](../introduction.md#usage)以了解什么是「动态配置」。动态配置方式会自动为你创建 PV,每一个 PV 会最终对应 JuiceFS 文件系统中的一个子目录,而创建 PV 的基础配置参数在 StorageClass 中定义,因此你需要先行[创建 StorageClass](#create-storage-class)。
Expand Down Expand Up @@ -431,6 +438,8 @@ spec:
volumeMounts:
- mountPath: /data
name: juicefs-pv
# 必须配置好传播,以防 mount pod 异常以后,挂载点无法自动恢复
mountPropagation: HostToContainer
volumes:
- name: juicefs-pv
persistentVolumeClaim:
Expand Down Expand Up @@ -472,6 +481,7 @@ spec:
volumeMounts:
- mountPath: /data
name: juicefs-pv
mountPropagation: HostToContainer
volumes:
- name: juicefs-pv
ephemeral:
Expand Down
12 changes: 6 additions & 6 deletions juicefs-csi-driver-config.example.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ data:
config.yaml: |-
# Set to true to schedule mount pod to node with via nodeSelector, rather than nodeName
enableNodeSelector: false
# The mountPodPatch section defines the mount pod spec
# Each item will be recursively merged into PVC settings according to its pvcSelector
# If pvcSelector isn't set, the patch will be applied to all PVCs
Expand All @@ -21,19 +21,19 @@ data:
- -c
- +e
- umount -l ${MOUNT_POINT}; rmdir ${MOUNT_POINT}; exit 0
# Example configurations:
# - pvcSelector:
# matchLabels:
# disable-host-network: "true"
# hostNetwork: false
# - pvcSelector:
# matchLabels:
# apply-labels: "true"
# labels:
# custom-labels: "asasasa"
# - pvcSelector:
# matchLabels:
# custom-resources: "true"
Expand All @@ -55,12 +55,12 @@ data:
# select pvc by storageClassName
# - pvcSelector:
# matchStorageClassName: juicefs-sc
# matchStorageClassName: juicefs-sc
# terminationGracePeriodSeconds: 60
# select pvc by pvc name
# - pvcSelector:
# matchName: pvc-name
# matchName: pvc-name
# terminationGracePeriodSeconds: 60
# - pvcSelector:
Expand Down

0 comments on commit 1f02d36

Please sign in to comment.