Skip to content
This repository was archived by the owner on Apr 17, 2025. It is now read-only.

Optimize the code framework #79

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

garenchan
Copy link

@garenchan garenchan commented Aug 16, 2022

What problem are we solving?

When we use the s3fs mounter, even if multiple pods use the same volume, the driver will start a process for each of them to mount the volume. This is wasteful of resources and bad for management.

o ps -ef | grep s3fs | grep pod
root     26654  3594  0 15:47 ?        00:00:00 s3fs custom / /var/lib/kubelet/pods/7e72ff6c-8096-4de3-973f-9d1a97fc3a89/volumes/kubernetes.io~csi/pvc-1a2313d1-ae74-45ee-bfa4-63106d7084ec/mount -o use_path_request_style -o url=http://10.134.80.223:9000 -o endpoint= -o allow_other -o mp_umask=000
root     34834  3594  0 19:32 ?        00:00:00 s3fs custom / /var/lib/kubelet/pods/5b8a4cb7-3897-42eb-94cf-049d0485a11b/volumes/kubernetes.io~csi/pvc-1a2313d1-ae74-45ee-bfa4-63106d7084ec/mount -o use_path_request_style -o url=http://10.134.80.223:9000 -o endpoint= -o allow_other -o mp_umask=000

How are we solving the problem?

The idea is that pods using the same volume need to share a mount process.
We currently solve this problem based on the CSI specification.

  1. When a volume is first used on a node, NodeStageVolume will start a process to mount it to host's /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount directory.
  2. Later when other pod uses the volume, NodePublishVolume just needs to bind mount /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount to /var/lib/kubelet/pods/<pod>/volumes/kubernetes.io~csi/<pvc>/mount directory.
  3. If the pod using the volume on the node is deleted, then NodeUnpublishVolume only needs to unmount /var/lib/kubelet/pods/<pod>/volumes/kubernetes.io~csi/<pvc>/mount.
  4. Finally, when the volume is not used by any pods on the node, NodeUnstageVolume will unmount the volume from /var/lib/kubelet/plugins/kubernetes.io/csi/pv/<pvc>/globalmount directory.

We made a significant change to the code to make it easier to perform similar optimizations for other mounters.

Additionally, according to the specification, we also consider scenarios for handling concurrent calls.
https://github.com/container-storage-interface/spec/blob/master/spec.md#concurrency

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant