Skip to content

Commit f01416a

Browse files
QeasQeas
committed
updated master
1 parent e84586b commit f01416a

File tree

5 files changed

+98
-67
lines changed

5 files changed

+98
-67
lines changed

‎README.md‎

Lines changed: 86 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -2,18 +2,24 @@
22

33
Releases can be found here - https://github.com/DDNStorage/exa-csi-driver/releases
44

5+
## Compatibility matrix
6+
|CSI driver version|EXAScaler client version|EXA Scaler server version|
7+
|--- |---|---|
8+
|>=v2.3.0|>=2.14.0-ddn182|>=6.3.2|
9+
510
## Feature List
6-
|Feature|Feature Status|CSI Driver Version|CSI Spec Version|Kubernetes Version|
7-
|--- |--- |--- |--- |--- |
8-
|Static Provisioning|GA|>= 1.0.0|>= 1.0.0|>=1.18|
9-
|Dynamic Provisioning|GA|>= 1.0.0|>= 1.0.0|>=1.18|
10-
|RW mode|GA|>= 1.0.0|>= 1.0.0|>=1.18|
11-
|RO mode|GA|>= 1.0.0|>= 1.0.0|>=1.18|
12-
|Expand volume|GA|>= 1.0.0|>= 1.1.0|>=1.18|
13-
|StorageClass Secrets|GA|>= 1.0.0|>=1.0.0|>=1.18|
14-
|Mount options|GA|>= 1.0.0|>= 1.0.0|>=1.18|
15-
|Topology|GA|>= 2.0.0|>= 1.0.0|>=1.17|
16-
|Snapshots|GA|>= 2.2.6|>= 1.0.0|>=1.17|
11+
|Feature|Feature Status|CSI Driver Version|CSI Spec Version|Kubernetes Version|Openshift Version|
12+
|--- |--- |--- |--- |--- |--- |
13+
|Static Provisioning|GA|>= 1.0.0|>= 1.0.0|>=1.18|>=4.13|
14+
|Dynamic Provisioning|GA|>= 1.0.0|>= 1.0.0|>=1.18|>=4.13|
15+
|RW mode|GA|>= 1.0.0|>= 1.0.0|>=1.18|>=4.13|
16+
|RO mode|GA|>= 1.0.0|>= 1.0.0|>=1.18|>=4.13|
17+
|Expand volume|GA|>= 1.0.0|>= 1.1.0|>=1.18|>=4.13|
18+
|StorageClass Secrets|GA|>= 1.0.0|>=1.0.0|>=1.18|>=4.13|
19+
|Mount options|GA|>= 1.0.0|>= 1.0.0|>=1.18|>=4.13|
20+
|Topology|GA|>= 2.0.0|>= 1.0.0|>=1.17|>=4.13|
21+
|Snapshots|GA|>= 2.2.6|>= 1.0.0|>=1.17|>=4.13|
22+
|Exascaler Hot Nodes|GA|>= 2.3.0|>= 1.0.0|>=1.18| Not supported yet|
1723

1824
## Access Modes support
1925
|Access mode| Supported in version|
@@ -26,40 +32,9 @@ Releases can be found here - https://github.com/DDNStorage/exa-csi-driver/releas
2632
## OpenShift Certification
2733
|OpenShift Version| CSI driver Version| EXA Version|
2834
|---|---|---|
29-
|v4.13| >=v2.2.3|v6.3.0|
30-
|v4.14| >=v2.2.4|v6.3.0|
31-
|v4.15| >=v2.2.4|v6.3.0|
32-
33-
## Requirements
34-
35-
- Kubernetes cluster must allow privileged pods, this flag must be set for the API server and the kubelet
36-
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enable-privileged-pods)):
37-
```
38-
--allow-privileged=true
39-
```
40-
- Required API server and kubelet feature gates for k8s version < 1.16 (skip this step for k8s >= 1.16)
41-
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enabling-features)):
42-
```
43-
--feature-gates=ExpandInUsePersistentVolumes=true,ExpandCSIVolumes=true,ExpandPersistentVolumes=true
44-
```
45-
- Mount propagation must be enabled, the Docker daemon for the cluster must allow shared mounts
46-
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enabling-mount-propagation))
47-
48-
- MpiFileUtils dtar must be installed on all kubernetes nodes in order to use `dtar` as snapshot utility. Not required if `tar` is used for snapshots (default).
49-
50-
## Installation
51-
Clone or untar driver (depending on where you get the driver)
52-
```bash
53-
git clone -b <driver version> https://github.com/DDNStorage/exa-csi-driver.git /opt/exascaler-csi-file-driver
54-
```
55-
e.g:-
56-
```bash
57-
git clone -b 2.2.4 https://github.com/DDNStorage/exa-csi-driver.git /opt/exascaler-csi-file-driver
58-
```
59-
or
60-
```bash
61-
rpm -Uvh exa-csi-driver-1.0-1.el7.x86_64.rpm
62-
```
35+
|v4.13|>=v2.2.3|>=v6.3.0|
36+
|v4.14|>=v2.2.4|>=v6.3.0|
37+
|v4.15|>=v2.2.4|>=v6.3.0|
6338

6439
## OpenShift
6540
### Prerequisites
@@ -194,6 +169,40 @@ To enable `dtar` set `snapshotUtility: dtar` in config.
194169

195170

196171
## Kubernetes
172+
### Requirements
173+
174+
- Kubernetes cluster must allow privileged pods, this flag must be set for the API server and the kubelet
175+
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enable-privileged-pods)):
176+
```
177+
--allow-privileged=true
178+
```
179+
- Required API server and kubelet feature gates for k8s version < 1.16 (skip this step for k8s >= 1.16)
180+
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enabling-features)):
181+
```
182+
--feature-gates=ExpandInUsePersistentVolumes=true,ExpandCSIVolumes=true,ExpandPersistentVolumes=true
183+
```
184+
- Mount propagation must be enabled, the Docker daemon for the cluster must allow shared mounts
185+
([instructions](https://github.com/kubernetes-csi/docs/blob/735f1ef4adfcb157afce47c64d750b71012c8151/book/src/Setup.md#enabling-mount-propagation))
186+
187+
- MpiFileUtils dtar must be installed on all kubernetes nodes in order to use `dtar` as snapshot utility. Not required if `tar` is used for snapshots (default).
188+
189+
### Prerequisites
190+
EXAScaler client must be installed and configured on all kubernetes nodes. Please refer to EXAScaler Installation and Administration Guide.
191+
192+
### Installation
193+
Clone or untar driver (depending on where you get the driver)
194+
```bash
195+
git clone -b <driver version> https://github.com/DDNStorage/exa-csi-driver.git /opt/exascaler-csi-file-driver
196+
```
197+
e.g:-
198+
```bash
199+
git clone -b 2.2.4 https://github.com/DDNStorage/exa-csi-driver.git /opt/exascaler-csi-file-driver
200+
```
201+
or
202+
```bash
203+
rpm -Uvh exa-csi-driver-1.0-1.el7.x86_64.rpm
204+
```
205+
197206
### Using helm chart
198207

199208
Make changes to `deploy/helm-chart/values.yaml` and `deploy/helm-chart/exascaler-csi-file-driver-config.yaml` according to your Kubernetes and EXAScaler clusters environment
@@ -276,7 +285,7 @@ parameters:
276285
projectId: "100001" # Points to EXA project id to be used to set volume quota. Automatically generated by the driver if not provided.
277286
exaMountUid: "1001" # Uid which will be used to access the volume in pod. Should be synced between EXA server and clients.
278287
exaMountGid: "1002" # Gid which will be used to access the volume in pod. Should be synced between EXA server and clients.
279-
bindMount: "false" # Determines, whether volume will bind mounted or as a separate lustre mount.
288+
bindMount: "true" # Determines, whether volume will bind mounted or as a separate lustre mount.
280289
exaFS: "10.204.86.114@tcp:/testfs" # Overrides exaFS value from config. Use this to support multiple EXA filesystems.
281290
mountPoint: /exaFS # Overrides mountPoint value from config. Use this to support multiple EXA filesystems.
282291
mountOptions: ro,noflock
@@ -356,25 +365,37 @@ spec:
356365
#mountOptions: ro, flock # list of options for `mount` command
357366
```
358367

359-
CSI Parameters:
360-
361-
| Name | Description | Example |
362-
|----------------|-------------------------------------------------------------------|--------------------------------------|
363-
| `driver` | [required] installed driver name " exa.csi.ddn.com" | `exa.csi.ddn.com` |
364-
| `volumeHandle` | [required] EXAScaler name and path to volume on EXAScaler filesystem | `exa1:/nginx-persistent` |
365-
| `exaMountUid` | Uid which will be used to access the volume from the pod. | `1015` |
366-
| `exaMountGid` | Gid which will be used to access the volume from the pod. | `1015` |
367-
| `projectId` | Points to EXA project id to be used to set volume quota. Automatically generated by the driver if not provided. | `100001` |
368-
| `managementIp` | Should be used if there is a separate network configured for management operations, such as create/delete volumes. This network should have access to all Exa filesystems in a isolated zones environment | `192.168.10.20@tcp2` |
369-
| `bindMount` | Determines, whether volume will bind mounted or as a separate lustre mount. | `true` |
370-
| `mountOptions` | Options that will be passed to mount command (-o <opt1,opt2,opt3>) | `ro,flock` |
371-
| `minProjectId` | Minimum project ID number for automatic generation. Only used when projectId is not provided. | 10000 |
372-
| `maxProjectId` | Maximum project ID number for automatic generation. Only used when projectId is not provided. | 4294967295 |
373-
| `generateProjectIdRetries` | Maximum retry count for generating random project ID. Only used when projectId is not provided. | `5` |
374-
| `zone` | Topology zone to control where the volume should be created. Should match topology.exa.csi.ddn.com/zone label on node(s). | `us-west` |
375-
| `v1xCompatible` | [Optional] Only used when upgrading the driver from v1.x.x to v2.x.x. Provides compatibility for volumes that were created beore the upgrade. Set it to `true` to point to the Exa cluster that was configured before the upgrade | `false` |
376-
| `tempMountPoint` | [Optional] Used when `exaFS` points to a subdirectory that does not exist on Exascaler and will be automatically created by the driver. This parameter sets the directory where Exascaler filesystem will be temporarily mounted to create the subdirectory. | `/tmp/exafs-mnt` |
377-
| `volumeDirPermissions` | [Optional] Defines file permissions for mounted volumes. | `0777` |
368+
Available CSI Parameters:
369+
370+
|Config|Storage class| Description | Example |
371+
|--------|------------|------------------------------------------------------------------|--------------------------------------|
372+
| `exaFS` | `exaFS` | [required] Full path to EXAScaler filesystem | `10.3.3.200@tcp:/csi-fs` |
373+
| `mountPoint` [required] | - | Mountpoint on Kubernetes host where the exaFS will be mounted | `/exa-csi-mnt` |
374+
| - | `driver` [required] | Installed driver name " exa.csi.ddn.com" | `exa.csi.ddn.com` |
375+
| - | `volumeHandle` | [required for static volumes] EXAScaler name and path to volume on EXAScaler filesystem | `exa1:/nginx-persistent` |
376+
| - | `exaMountUid` | Uid which will be used to access the volume from the pod. | `1015` |
377+
| - | `exaMountGid` | Gid which will be used to access the volume from the pod. | `1015` |
378+
| - | `projectId` | Points to EXA project id to be used to set volume quota. Automatically generated by the driver if not provided. | `100001` |
379+
| `managementIp` | `managementIp` | Should be used if there is a separate network configured for management operations, such as create/delete volumes. This network should have access to all Exa filesystems in a isolated zones environment | `192.168.10.20@tcp2` |
380+
| `bindMount` | `bindMount` | Determines, whether volume will bind mounted or as a separate lustre mount. Default is `true` | `true` |
381+
| `defaultMountOptions` | `mountOptions` | Options that will be passed to mount command (-o <opt1,opt2,opt3>) | `ro,flock` |
382+
| - | `minProjectId` | Minimum project ID number for automatic generation. Only used when projectId is not provided. | 10000 |
383+
| - | `maxProjectId` | Maximum project ID number for automatic generation. Only used when projectId is not provided. | 4294967295 |
384+
| - | `generateProjectIdRetries` | Maximum retry count for generating random project ID. Only used when projectId is not provided. | `5` |
385+
| `zone` | `zone` | Topology zone to control where the volume should be created. Should match topology.exa.csi.ddn.com/zone label on node(s). | `us-west` |
386+
| `v1xCompatible` | - | [Optional] Only used when upgrading the driver from v1.x.x to v2.x.x. Provides compatibility for volumes that were created beore the upgrade. Set it to `true` to point to the Exa cluster that was configured before the upgrade | `false` |
387+
|`tempMountPoint` | `tempMountPoint` | [Optional] Used when `exaFS` points to a subdirectory that does not exist on Exascaler and will be automatically created by the driver. This parameter sets the directory where Exascaler filesystem will be temporarily mounted to create the subdirectory. | `/tmp/exafs-mnt` |
388+
| `volumeDirPermissions` | `volumeDirPermissions` | [Optional] Defines file permissions for mounted volumes. | `0777` |
389+
| `hotNodes` | `hotNodes` | Determines whether `HotNodes` feature should be used. This feature can only be used by the driver when Hot Nodes (PCC) service is disabled and not used manually on the kubernetes workers. | `false` |
390+
| `pccCache` | `pccCache` | Directory for cached files of the file system. Note that lpcc does not recognize directories with a
391+
trailing slash (“/” at the end). | `/csi-pcc` |
392+
| `pccAutocache` | `pccAutocache` | Condition for automatic file attachment (caching) | `projid={500}` |
393+
| `pccPurgeHighUsage` | `pccPurgeHighUsage` | If the disk usage of cache device is higher than high_usage, start detaching the files. Defaults to 90 (90% disk/inode usage). | `90` |
394+
| `pccPurgeLowUsage` | `pccPurgeLowUsage` | If the disk usage of cache device is lower than low_usage, stop detaching the files. Defaults to 75 (75% disk/inode usage). | `70` |
395+
| `pccPurgeScanThreads` | `pccPurgeScanThreads` | Threads to use for scanning cache device in parallel. Defaults to 1. | `1` |
396+
| `pccPurgeInterval` | `pccPurgeInterval` | Interval for lpcc_purge to check cache device usage, in seconds. Defaults to 30. | `30` |
397+
| `pccPurgeLogLevel` | `pccPurgeLogLevel` | Log level for lpcc_purge: either “fatal”, “error”, “warn”, “normal”, “info” (default), or “debug”. | `info` |
398+
| `pccPurgeForceScanInterval` | `pccPurgeForceScanInterval` | Scan PCC backends forcefully after this number of seconds to refresh statistic data. | `30` |
378399

379400
#### _PersistentVolumeClaim_ (pointing to created _PersistentVolume_)
380401

‎bin/exascaler-csi-file-driver.tar‎

495 KB
Binary file not shown.

‎deploy/helm-chart/values.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ provisioner:
2121
repository: registry.k8s.io/sig-storage/csi-provisioner
2222
tag: v4.0.1
2323
pullPolicy: IfNotPresent
24+
timeout: 120m
2425
resources:
2526

2627
attacher:

‎examples/exa-dynamic-nginx-2.yaml‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,20 @@ metadata:
99
provisioner: exa.csi.ddn.com
1010
allowVolumeExpansion: true
1111
parameters:
12-
projectId: "100001" # Points to a project id to be used to set volume quota.
12+
# projectId: "100001" # Points to a project id to be used to set volume quota.
1313
exaMountUid: "1001" # Uid which will be used to access the volume in pod. Should be synced between EXA server and clients.
1414
exaMountGid: "1001" # Gid which will be used to access the volume in pod. Should be synced between EXA server and clients.
1515
bindMount: "false"
1616
volumeDirPermissions: "1750"
17+
hotNodes: "true"
18+
pccCache: "/mnt/csi-1"
19+
# pccAutocache: ""
20+
# pccPurgeHighUsage: ""
21+
# pccPurgeLowUsage: ""
22+
# pccPurgeScanThreads: ""
23+
# pccPurgeInterval: ""
24+
# pccPurgeLogLevel: ""
25+
# pccPurgeForceScanInterval: ""
1726
---
1827

1928
# ------------------------------------------------

‎examples/snapshot-from-dynamic.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,5 +13,5 @@ metadata:
1313
spec:
1414
volumeSnapshotClassName: exascaler-csi-snapshot-class
1515
source:
16-
persistentVolumeClaimName: exascaler-csi-file-driver-pvc-2
16+
persistentVolumeClaimName: exascaler-csi-file-driver-pvc
1717
# kind: PersistentVolumeClaim

0 commit comments

Comments
 (0)