Bring in the Volumes!

Trident 19.04 has just been released! Volume import is one of the main features included with v19.04. It makes the move to containerized environments and disaster recovery scenarios much easier.   

 

What is Volume Import? 

Volume import provides the ability to bring an existing external storage volume into a Kubernetes environment and mounts it to the appropriate container. 

With volume import, you can also choose whether to import and manage the volume, or just import it without managing it. By default, Trident manages the volume. If manage is on, Trident will also rename the volume on the backend to Trident’s conventions which include the storage prefix and Kubernetes namespace.  If it is not managed, Trident will not rename the volume nor perform any volume clone or resize. It also will not listen for the Persistent Volume (PV) deletion, and therefore not take any action if the PV is deleted. 

Trident v19.04 volume import is supported with the ontap-nas, ontap-nas-flexgroup, solidfire-san and the aws-cvs drivers. The remaining drivers (ontap-san, ontap-nas-economy, and eseries-iscsi) manage volumes differently. It is very convoluted to import them into a form Trident can manage if they were not initially created by Trident. For example, the ontap-nas-economy driver manages qtrees, which are part of the volumes it also manages. Existing qtrees cannot be easily moved into one of these volumes, so there is no straightforward import process to implement.

 

Why would I use this feature? 

Volume Import is useful for many scenarios. A few are listed below: 

• Migrate existing monolithic applications into containers – Volume Import allows you to test and migrate current applications into containers without losing your data.  

• Volume migration – Migrate your data from one Kubernetes cluster to another.

• Kubernetes re-build - If the Kubernetes cluster has a catastrophe and needs to be rebuilt from scratch, don’t lose your data! Keep the data and use volume import to retrieve the data and bring into the new re-built cluster.

• Data Management disaster recovery – When using SnapMirror, volume import can be used to import volumes from the backup site into the Kubernetes cluster.

Watch this video and see how volume import can be used to import a populated MySQL database into a containerized environment. 

 

How do I import a volume?

There are five steps to import a volume:

1. With Trident installed and running, create a Trident backend definition for the external volume location if it doesn’t already exist
2. Retrieve the unique volume name on the backend
3. Create a Persistent Volume Claim (PVC) definition file for the volume’s application
4. Issue the volume import command
5. Validate the PV and PVC are bound and the volume can be mounted

We use an example of importing existing blog content into a new pod on a new Kubernetes cluster. We start with the blog application deployment and service already added to the new Kubernetes cluster. The pod remains in the pending state until Trident imports the volume into Kubernetes, the PVC/PV pair are bound and there are no other issues. A storage class is already applied to the new cluster that uses Trident as the provisioner.

$ kubectl get pods -n blog
NAME                    READY   STATUS    RESTARTS   AGE
blog-588d7f5ddc-sbhkk   0/1     Pending   0          10s

 

Step 1:  Create Trident backend definition

With Trident installed and running, add the backend definition to Trident that corresponds to the external volume’s location if it doesn’t already exist. We used the following JSON template as shown below.

{
    "version": 1,
    "storageDriverName": "ontap-nas",
    "backendName": "backend_with_existing_volume",
    "managementLIF": "<ip_address>",
    "dataLIF": "<ip_address>",
    "svm": "TME",
    "username": "vsadmin",
    "password": "password",
    "storagePrefix": "Imported_volume"
}

Note the IP addresses and password must be replaced with real values.

Use tridentctl to create the backend:

$ tridentctl create backend -f backends/ontapnas.json -n trident
+------------------------------+----------------+--------+---------+
|             NAME             | STORAGE DRIVER | STATE  | VOLUMES |
+------------------------------+----------------+--------+---------+
| backend_with_existing_volume | ontap-nas      | online |       0 |
+------------------------------+----------------+--------+---------+

 

Step 2:  Retrieve volume name

We retrieve the current volume name from the backend. The volume name needs to be unique on that backend. We will import the volume called TME_blog_content_7a2d5 and can see its size is 5GB on the OnCommand System Manager as shown below.

 

Step 3:  Create a PVC

Create a PVC definition file (YAML or JSON) to associate with the application that mounts the volume. The PVC identifies a previously applied storage class with Trident as the provisioner. The PVC does not need to have the correct volume size, as Trident will adjust it based upon the actual volume size on the backend. In our example, we use the below YAML file:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: content-blog
  namespace: blog
spec:
  accessModes:
  - ReadWriteMany
  resources:
    requests:
      storage: 4Gi
  storageClassName: storage-class-nas-blog

Step 4:  Issue the Volume Import command

Import the backend volume which will create the PVC and PV. We use the command tridentctl import volume <backend name> <volume name> -f /path/to/pvc.yaml -n trident. See example below:

$ tridentctl import volume backend_with_existing_volume TME_blog_content_7a2d5 -f ../k8s/ghost_pvc.yaml -n trident
+-------------------------+---------+------------------------+----------+------------------------------+------+
|          NAME           |  SIZE   |     STORAGE CLASS      | PROTOCOL |           BACKEND            | POOL |
+-------------------------+---------+------------------------+----------+------------------------------+------+
| blog-content-blog-76e66 | 5.0 GiB | storage-class-nas-blog | file     | backend_with_existing_volume |      |
+-------------------------+---------+------------------------+----------+------------------------------+------+

Step 5:  Validate the PVC is bound and the volume can be mounted

Validate the PVC is bound, and the application pod is up. Note the volume name has changed on the backend to match the PV. Additionally, validate the data is still present and correct.

$ kubectl get pvc -n blog

NAME                               STATUS VOLUME                  CAPACITY   ACCESS MODES STORAGECLASS           AGE
persistentvolumeclaim/content-blog Bound  blog-content-blog-76e66 5368709120 RWX          storage-class-nas-blog 25m

If we look at the backend, the volume name changed to Imported_volume_blog_content_blog_76e66, to match the PV above and Trident’s conventions. (The storage prefix in the new backend definition file was called “Imported_volume” and the Kubernetes namespace is “blog”):

We can see the pod came up!

$ kubectl get pods -n blog
NAME                    READY   STATUS    RESTARTS   AGE
blog-588d7f5ddc-sbhkk   1/1     Running   0          71s

 

Last, check to be sure the content is accessible by the new pod on the new cluster. Since we are using NodePort service, we will retrieve the port (31135) to use.

 

$ kubectl get svc -n blog
NAME        TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
ghost-svc   NodePort   10.111.185.4   <none>        2368:31135/TCP   98s

 

Going to the webpage, we can see the content is accessible from the new cluster

In conclusion:

Volume Import is now available to bring existing volumes into a Kubernetes environment. It is useful when moving from a monolithic environment to a containerized environment, when moving applications between clusters, and in disaster recovery scenarios.

Try it out and let us know how you like it on slack! We’d love to hear from you.