Kubernetes: Taming Volume Snapshot

Many times one would have a need to template-ize a volume and would like to clone it for every new pod creation. ‘Volume Snapshot’ is the name for this.

In this article I have put together the steps required to

• create a master template as PVC
• create a volume snapshot
• restore a volume snapshot as PVC

In order to use VolumeSnapshot one needs to enable CSI in their kubernetes. In this attempt of my article I have tested this on GKE. And one requires to run atleast 1.17.x version of kubernetes master plane.

Note (1): I have biased this article for GKE. Hope to make it generic enough so that it can work for other cloud vendors.

Note (2): At the time of writing this article, the GKE documentation on https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/volume-snapshots did NOT work for me (root caused to storageClassName being defaulted to standard )

Note (3): One needs to enable CSI Driver in their GKE using https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver

Step 1: Creating a master template

Here, we create a StorageClass to adhere to CSI. Note, if the PVC for master-template is not based CSI storage, the VolumeSnapsot creation would fail.

# 01_step.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: pd-storage-standard
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
  type: pd-standard
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-template
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: pd-storage-standard
  resources:
    requests:
      storage: 20Gi

This is applied in kubernetes…

kubectl apply -f 01_step.yaml

The next step is to load data into this Volume using a throw-away pod.

# copy_files.yaml
apiVersion: v1
kind: Pod
metadata:
  name: copy-files
spec:
  volumes:
    - name: src-volume
      persistentVolumeClaim:
        claimName: pvc-template-1-test #one can use an existing source here.
    - name: template-volume
      persistentVolumeClaim:
        claimName: pvc-template
  containers:
    - name: copy-files
      image: nginx
      volumeMounts:
        - mountPath: "/source"
          name: src-volume
        - mountPath: "/template"
          name: template-volume
  nodeSelector:
    node-type: 1-core

This is applied using

kubectl apply -f copy_files.yaml

Once the above pod is ready, one can connect to the pod and copy files using

kubectl exec -it pod/copy-files -- /bin/bash

After which the pod can be destroyed using

kubectl delete -f copy_files.yaml

Step 2: Create Volume Snapshot

Here, we create a VolumeSnapshot resource based on the PVC created from Step 1.

# create_volume.yaml
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshotClass
metadata:
  name: pd-snapshot
driver: pd.csi.storage.gke.io
deletionPolicy: Delete
---
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshot
metadata:
  name: snap-template
spec:
  volumeSnapshotClassName: pd-snapshot
  source:
    persistentVolumeClaimName: pvc-template

It can be noted that, the deletion policy of the instance of the VolumeSnapshot’s storage is delete once the cloned PVC is deleted.

kubectl apply -f create_volume.yaml

Step 3: Using VolumeSnapshot in another pod

This is the restore step, where a pod uses the VolumneSnapshot using a PVC.

Using PVC in pod / deployment /…

# restore_test.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: restore-pvc-1
  namespace: pipeline
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: pd-storage-standard
  resources:
    requests:
      storage: 20Gi
  dataSource:
    name: snap-template
    kind: VolumeSnapshot
    apiGroup: snapshot.storage.k8s.io
---
apiVersion: v1
kind: Pod
metadata:
  name: template-instance-test-1
spec:
  volumes:
    - name: my-storage
      persistentVolumeClaim:
        claimName: restore-pvc-1
        readOnly: false
  containers:
    - name: task-pv-container
      image: nginx
      ports:
        - containerPort: 80
          name: "test-port1"
      volumeMounts:
        - mountPath: "/template"
          name: my-storage

Executed using

kubectl apply -f restore_test.yaml

The /template mount in the PVC can be verified to see if the contents are same as what was setup in Step 1

Alternatives

CSI Volume Cloning — https://kubernetes.io/docs/concepts/storage/volume-pvc-datasource/ is a potential alternative to VolumeSnapshot.

Also using bucket to copy files into the init step is another option. But, can cause slowness if a large burst of pods are created.