Use dedicated Persistent Disks as ephemeral volumes

This page shows you how to use external storage hardware, such as Compute Engine Persistent Disks, as ephemeral volumes in your Google Kubernetes Engine (GKE) workloads. You should already be familiar with Kubernetes Volumes and StorageClasses.

When to use ephemeral storage in Kubernetes

Ephemeral storage is useful in any situation where your workloads only need the data during the lifecycle of the application, such as for data-processing pipelines, machine learning jobs, batch processing, local caching, or analytics. By default, part of the GKE node boot disk is available to use as ephemeral storage in your Pods. This approach often requires careful space planning.

Kubernetes generic ephemeral volumes let you explicitly request ephemeral storage for your Pods by using PersistentVolumeClaims. GKE dynamically provisions Compute Engine Persistent Disks and attaches the disks to your nodes. This type of ephemeral storage is useful in situations like the following:

• Your workloads have high performance requirements, so you need to control the storage hardware.
• You need short-term, container-specific ephemeral storage.
• You want to avoid using emptyDir to provision ephemeral storage. emptyDir volumes are still useful in situations where you want multiple containers to share the data in the ephemeral storage.
• You want more ephemeral storage capacity than the GKE built-in defaults.
• You want to avoid having to plan your node boot disk size and type in advance for Standard mode GKE clusters.

Ephemeral storage types in GKE

In general, you can use boot disk storage capacity or dedicated Persistent Disks as ephemeral storage in your Pods and containers. The following table describes the differences:

Pricing

Storage that you provision through generic ephemeral volumes as described in this guide is billed based on Compute Engine disk pricing.

Before you begin

Before you start, make sure you have performed the following tasks:

• Enable the Google Kubernetes Engine API.
Enable Google Kubernetes Engine API
• If you want to use the Google Cloud CLI for this task, install and then initialize the gcloud CLI. If you previously installed the gcloud CLI, get the latest version by running gcloud components update.

• Ensure that you have a GKE Autopilot or Standard cluster running version 1.23 or later.
• Ensure that you have enough quota in your Google Cloud project for the storage hardware. To manage your quota, see View the quotas for your project.

Create a StorageClass

Creating a custom Kubernetes StorageClass lets you specify the type of storage to provision based on your price and performance requirements. This step is optional but recommended. If you want to use the GKE default StorageClass, which has the pd-balanced Persistent Disk type, skip this step.

1. Save the following manifest as ephemeral-pd-class.yaml:

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: ephemeral-ssd
   provisioner: pd.csi.storage.gke.io
   volumeBindingMode: WaitForFirstConsumer
   allowVolumeExpansion: true
   parameters:
     type: STORAGE_TYPE
   

   Replace STORAGE_TYPE with the name of the Persistent Disk type that you want, like pd-ssd. For a list of supported types, see Persistent Disk types in the Compute Engine documentation.

2. Create the StorageClass:

   kubectl create -f ephemeral-pd-class.yaml
   

Request ephemeral storage capacity in a Pod

To provision, attach, and use external hardware as ephemeral storage, add the corresponding volume to your Pod manifest and add a volume mount to the container specification.

1. Save the following manifest as ephemeral-ssd-deployment.yaml:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: ephemeral-deployment
   spec:
     replicas: 1
     selector:
       matchLabels:
         storage: ephemeral
     template:
       metadata:
         labels:
           storage: ephemeral
       spec:
         containers:
         - name: ephemeral-container
           image: nginx
           resources:
             requests:
               cpu: 500m
               memory: 2Gi
               ephemeral-storage: 2Gi
           volumeMounts:
           - mountPath: "/short-term"
             name: ephemeral-volume
         volumes:
         - name: ephemeral-volume
           ephemeral:
             volumeClaimTemplate:
               metadata:
                 labels:
                   type: ephemeral
               spec:
                 accessModes: ["ReadWriteOnce"]
                 storageClassName: "ephemeral-ssd"
                 resources:
                   requests:
                     storage: 1Ti
   

   This manifest creates a new Kubernetes PersistentVolumeClaim that requests a new PersistentVolume named ephemeral-volume with the following properties:

   • spec.volumes.ephemeral: The ephemeral volume type.
   • .spec.accessModes: The volume access mode, which determines read-write access from Pods and volume sharing between nodes. This examples uses ReadWriteOnce, which mounts the PersistentVolume to a single node for access by one or more Pods on the node. For details, see Access modes.
   • .spec.storageClassName: Optionally, the name of the StorageClass that you created. If you omit this field, GKE uses the default StorageClass and provisions a pd-balanced Persistent Disk.
   • .spec.resources.requests.storage: The storage capacity that you want.

2. Create the Deployment:

   kubectl create -f ephemeral-ssd-deployment.yaml
   

GKE provisions a Compute Engine disk that meets the requirements of the PersistentVolumeClaim and attaches the disk to the node. GKE mounts the volume into the Pod and provides the requested capacity to the container.

Verify that GKE mounted an ephemeral volume

1. Create a shell session in the Pod:

   kubectl exec -it deploy/ephemeral-deployment -- bash
   

2. Check the mounted volumes:

   df -h
   

   The output is similar to the following:

   Filesystem                Size      Used Available Use% Mounted on
   ...
   /dev/sdb               1006.9G     28.0K   1006.8G   0% /short-term
   /dev/sda1                94.3G      3.6G     90.6G   4% /etc/hosts
   /dev/sda1                94.3G      3.6G     90.6G   4% /dev/termination-log
   /dev/sda1                94.3G      3.6G     90.6G   4% /etc/hostname
   /dev/sda1                94.3G      3.6G     90.6G   4% /etc/resolv.conf
   ...
   

3. Exit the shell session:

   exit
   

What's next

• Learn about the available storage options in GKE