Kubernetes: Taming Volume Snapshot

Many times one would have a need to template-ize a volume and would like to clone it for every new pod creation. ‘Volume Snapshot’ is the name for this.

In this article I have put together the steps required to

  • create a master template as PVC
  • create a volume snapshot
  • restore a volume snapshot as PVC

In order to use VolumeSnapshot one needs to enable CSI in their kubernetes. In this attempt of my article I have tested this on GKE. And one requires to run atleast 1.17.x version of kubernetes master plane.

Note (1): I have biased this article for GKE. Hope to make it generic enough so that it can work for other cloud vendors.

Note (2): At the time of writing this article, the GKE documentation on https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/volume-snapshots did NOT work for me (root caused to storageClassName being defaulted to standard )

Note (3): One needs to enable CSI Driver in their GKE using https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/gce-pd-csi-driver

Step 1: Creating a master template

Here, we create a StorageClass to adhere to CSI. Note, if the PVC for master-template is not based CSI storage, the VolumeSnapsot creation would fail.

# 01_step.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: pd-storage-standard
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
parameters:
type: pd-standard
---
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: pvc-template
spec:
accessModes:
- ReadWriteOnce
storageClassName: pd-storage-standard
resources:
requests:
storage: 20Gi

This is applied in kubernetes…

kubectl apply -f 01_step.yaml

The next step is to load data into this Volume using a throw-away pod.

# copy_files.yaml
apiVersion: v1
kind: Pod
metadata:
name: copy-files
spec:
volumes:
- name: src-volume
persistentVolumeClaim:
claimName: pvc-template-1-test #one can use an existing source here.
- name: template-volume
persistentVolumeClaim:
claimName: pvc-template
containers:
- name: copy-files
image: nginx
volumeMounts:
- mountPath: "/source"
name: src-volume
- mountPath: "/template"
name: template-volume
nodeSelector:
node-type: 1-core

This is applied using

kubectl apply -f copy_files.yaml

Once the above pod is ready, one can connect to the pod and copy files using

kubectl exec -it pod/copy-files -- /bin/bash

After which the pod can be destroyed using

kubectl delete -f copy_files.yaml

Step 2: Create Volume Snapshot

Here, we create a VolumeSnapshot resource based on the PVC created from Step 1.

# create_volume.yaml
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshotClass
metadata:
name: pd-snapshot
driver: pd.csi.storage.gke.io
deletionPolicy: Delete
---
apiVersion: snapshot.storage.k8s.io/v1beta1
kind: VolumeSnapshot
metadata:
name: snap-template
spec:
volumeSnapshotClassName: pd-snapshot
source:
persistentVolumeClaimName: pvc-template

It can be noted that, the deletion policy of the instance of the VolumeSnapshot’s storage is delete once the cloned PVC is deleted.

kubectl apply -f create_volume.yaml

Step 3: Using VolumeSnapshot in another pod

This is the restore step, where a pod uses the VolumneSnapshot using a PVC.

Using PVC in pod / deployment /…
# restore_test.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: restore-pvc-1
namespace: pipeline
spec:
accessModes:
- ReadWriteOnce
storageClassName: pd-storage-standard
resources:
requests:
storage: 20Gi
dataSource:
name: snap-template
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
---
apiVersion: v1
kind: Pod
metadata:
name: template-instance-test-1
spec:
volumes:
- name: my-storage
persistentVolumeClaim:
claimName: restore-pvc-1
readOnly: false
containers:
- name: task-pv-container
image: nginx
ports:
- containerPort: 80
name: "test-port1"
volumeMounts:
- mountPath: "/template"
name: my-storage

Executed using

kubectl apply -f restore_test.yaml

The /template mount in the PVC can be verified to see if the contents are same as what was setup in Step 1

Alternatives

CSI Volume Cloning — https://kubernetes.io/docs/concepts/storage/volume-pvc-datasource/ is a potential alternative to VolumeSnapshot.

Also using bucket to copy files into the init step is another option. But, can cause slowness if a large burst of pods are created.