We have been busy this summer making some improvements to Tridents plumbing. With it comes Trident v19.07 Alpha! Two major changes are happening, the introduction of CSI Trident and the introduction of Kubernetes Custom Resource Definitions (CRD) for Trident. This blog will discuss the new CRDs.

Although Trident will continue to function as it always has, Trident v19.07 Alpha and beyond will now utilize Kubernetes CRDs to store and manage its own state. With this change, Trident will store its metadata in the Kubernetes clusters’s etcd database just like other Kubernetes ecosystem tools. This simplifies install, makes it easier to debug, and provides a tighter integration with Kubernetes.

Trident v19.07 Alpha is for preview and greenfield deployments (new installations) only and has support for Kubernetes 1.14. You also cannot downgrade to a prior Trident version from Trident v19.07 Alpha. However, we will be out with a General Availability (GA) version soon!

What are Kubernetes CRDs anyway? How does Trident use them?

Kubernetes Custom Resource Definitions (CRDs) extend the Kubernetes API and add new custom resource types stored in the Kubernetes cluster’s etcd. As we know, Kubernetes comes with many pre-defined resource types. Some well-known resource types include pods, deployments, persistentvolumeclaims, persistentvolumes, storageclasses, services, etc. When you define a CRD, you create your own resource type based on what YOU want and use it like any other resource type. You can even use kubectl’ to view and troubleshoot it. The ability to create your own resource definitions gives Kubernetes great flexibility and extensibility.

Prior to Trident version 19.07 Alpha, Trident stored its metadata in its own etcd database, keeping it separate from Kubernetes. This also required a separate volume for Trident’s etcd itself. Now, with 19.07 Alpha (and beyond), the need for this extra etcd database/volume goes away. Trident now takes advantage of the CRDs to store its state instead. Of course, please verify your Kubernetes etcd volume is backed up regularly as it should be.

We have developed and implemented seven CRDs for Trident. They will be created automatically during Trident 19.07 Alpha installation and include:

  • tridentbackends.trident.netapp.io: Records the backends you have configured.
  • tridentnodes.trident.netapp.io: Records the CSI nodes in your cluster.
  • tridentstorageclasses.trident.netapp.io: Records the storage classes that Trident uses.
  • tridenttransactions.trident.netapp.io: Records a temporary structure to track volume create and removal.
  • tridentversions.trident.netapp.io: Records the installed version of Trident.
  • tridentvolumes.trident.netapp.io: Records the volumes.
  • tridentsnapshots.trident.netapp.io: Records the CSI snapshots.

Want to see them listed yourself? Simply use kubectl get crd -n trident in your cluster running Trident v19.07 Alpha or higher.

How does this simplify Trident’s installation?

Since Trident’s own etcd database doesn’t exist anymore, there is no need to initially create a volume for it. This means no /setup/backend.json file is needed before installing Trident. Thus, let's compare the installation procedure between 19.07 Alpha and earlier versions.

Prior to Trident version 19.07 Alpha, the steps were as follows:

  1. wget the installer from github
  2. Extract the files
  3. Change to the trident-installer directory
  4. Copy a sample-input file to setup/backend.json
  5. Edit the setup/backend.json file with IPs and credentials
  6. Install Trident

Trident version 19.07 Alpha and beyond requires less steps:

  1. wget the installer from github
  2. Extract the files
  3. Change to the trident-installer directory
  4. Install trident

It’s now even easier than before!

What if I already have Trident Installed?

Although we are currently only previewing new installations with v19.07 Alpha, an automatic upgrade from previous Trident versions will be available with 19.07 GA.

When you upgrade from a pre-19.07 Alpha release to a 19.07 GA or later release, the installation process will create the new CRDs, read all the data from Trident’s etcd volume, and import it directly into the new resources. There is no manual work required. However, this migration may take just a bit longer than a normal upgrade, depending on the number of objects that need converting.

Additionally, we leave Trident’s old etcd volume in place, just in case you need to go back. However, reversing upgrade requires a uninstall of the 19.07 GA, a Trident re-install of the prior version, and a manual removal of Trident’s CRDs.

How can I troubleshoot Trident using the CRDs?

Install the backends the same as you do today using tridentctl. Backends can then be viewed using either tridentctl or kubectl. However, please do not use the kubectl edit command, as Trident will not be aware of any changes made using kubectl edit until Trident restarts.

As an example, to see which backends are installed:

 $ kubectl get tridentbackends -n trident


tbe-q95l2   nas_blog   a3fa93d8-243d-4403-b299-7c7cf6e3d590

Which volumes is Trident managing?

$ kubectl get tridentvolumes -n trident

NAME                      AGE
blog-blog-content-2a68f   27h
blog-blog-content-83753   29h
blog-blog-content-d3032   8d

You can also use the kubectl describe command. In this case, I used volume import to import the volume. You can see the original name and the new name created by the import!

Please note CSI-Trident has not implemented volume import in the 19.07 Alpha release.

$ kubectl describe tridentvolumes blog-blog-content-2a68f -n trident
Name:          blog-blog-content-2a68f
Namespace:     trident
Labels:        <none>
Annotations:   <none>
API Version:   trident.netapp.io/v1
Backend UUID:  a3fa93d8-243d-4403-b299-7c7cf6e3d590
  Access Information:
    Nfs Path:                    /cluster1_blog_content_271f8
    Nfs Server Ip:               X.X.X.X [IP address removed]
  Access Mode:                   ReadWriteMany
  Block Size:                   
  Clone Source Snapshot:        
  Clone Source Volume:          
  Clone Source Volume Internal: 
  File System:                  
  Import Original Name:          cluster1_blog_content_271f8
  Internal Name:                 cluster2_blog_blog_content_2a68f
  Name:                          blog-blog-content-2a68f
  Protocol:                      file
  Security Style:               
  Size:                          5368709120
  Space Reserve:                
  Split On Clone:               
  Storage Class:                 storage-class-nas-blog
Kind:                            TridentVolume
 Creation Timestamp:  2019-06-14T18:13:53Z
 Generation:        1
 Resource Version:  2118935
 Self Link:         /apis/trident.netapp.io/v1/namespaces/trident/tridentvolumes/blog-blog-content-2a68f
 UID:               2ab4a829-8ed0-11e9-902c-fa163e6f3abf
Orphaned:            false
Events:              <none>

Taking a look at the storageclass as another example:

$ kubectl get tridentstorageclass -n trident
NAME                     AGE
storage-class-nas-blog   8d

$ kubectl describe tridentstorageclass storage-class-nas-blog -n trident
Name:         storage-class-nas-blog
Namespace:    trident
Labels:       <none>
Annotations:  <none>
API Version:  trident.netapp.io/v1
Kind:         TridentStorageClass
  Creation Timestamp:  2019-06-07T16:28:21Z
  Generation:        1
  Resource Version:  1233990
  Self Link:         /apis/trident.netapp.io/v1/.../.../tridentstorageclasses/storage-class-nas-blog
  UID:               44158502-8941-11e9-902c-fa163e6f3abf
    Backend Type:  ontap-nas
  Name:            storage-class-nas-blog
  Storage Pools:
    Nas Blog:
  Version:  1
Events:     <none>

In Conclusion

Try out 19.07 Alpha in your test environment and let us know how it goes! We’d love to hear from you on our containers slack channel!  Have a great day!




About Diane Patton

Diane is a Technical Marketing Engineer with NetApp, in the open eco-systems group supporting Docker, Kubernetes, and OpenShift integration with NetApp products. She works with product management, marketing, and development to evangelize, support, and help drive new technologies into Trident, the open source storage provisioner for containers maintained by NetApp. Diane has over 20 years experience in the IT industry (Optical DWDM, Layer 2, Layer 3, container networking, storage), holds CCIE 2537 Emeritus, and has a BS and MS in Electrical Engineering. When not working on open technologies, you will find Diane on a tennis court.

Pin It on Pinterest