Kubernetes makes managing applications very easy and flexible. Pods can be deployed anywhere, on your local PC for testing, on-prem or in the cloud at anytime. Kubernetes can guarantee a certain number of pods are deployed and running at any given time, and may even automatically scale your application up or down based upon demand. While the Kubernetes cluster is very flexible by creating and destroying pods, the data must remain persistent. This is where Trident comes in, Trident makes provisioning and de-provisioning Persistent Volumes (PV) across a cluster very easy, automatic in fact, just like Kubernetes.

However, some enterprises have multiple groups in an organization. Each group needs to keep their volume space allocated and managed separately from each other, while sharing the same Kubernetes cluster and same Trident instance.  Some enterprises may also want to segregate even further by keeping each application separate from each other. In this case, each application has its own volume space without access to any allocated volume space used by another application. Trident and Kubernetes make achieving this goal simple.

This blog post outlines how to configure a multi-tenant data management environment using the Trident provisioner, Network File Storage (NFS) and separate storage virtual machines (SVMs) on the backend.

So, how do I provide complete data separation with Kubernetes and Trident?

Construct and manage multi-tenancy with Trident and Kubernetes in three steps:

  • Create separate backend files per tenant
  • Generate separate storage classes per tenant
  • Deploy the applications and Persistent Volume Claims (PVCs) in separate Kubernetes namespaces

Taking these steps achieves data management isolation between tenants and guarantees one tenant will not use volume space allocated for a different tenant.

Create backend files for Trident per tenant

In this case, each tenant uses one or more unique backend files. The backend files for each tenant references its own SVM on the backend ONTAP cluster. Be sure to give a unique “BackendName” and “StoragePrefix” per file to reference later.  Two examples are shown below.

As seen above, although both backends are using the same NAS driver, they log into separate SVMs on the same ONTAP cluster and keep the data separate. The storagePrefix will help the administrator distinguish between them when looking at the ONTAP SystemManager or via CLI.

Create separate storage classes per tenant

Each tenant has its own set (one or more) of StorageClasses. The Kubernetes StorageClasses are defined with a Kubernetes manifest file and they name Trident as the provisioner. StoragePools are a filter that Trident uses that can define which backend to use. As can be seen below, we are using the same string for StoragePools as defined in the BackendName in the above backend file. The “.*” with the StoragePools value is a wildcard that means “all pools”, but individual pools can be specified as well if desired.

Sample Storage manifest files to use with the above backends are shown below.

Since Kubernetes does not namespace StorageClasses, ResourceQuotas can be used to prevent tenants from using each others StorageClasses. Control access to ResourceQuotas via Kubernetes RBAC.

Create and deploy in separate Kubernetes namespaces per tenant

Each tenant should have its own applications and persistent volume claims deployed in their own namespace to maintain separation. The namespaces are defined in the manifest files under the metadata field. Here, I have combined the PVC for the application and the application itself in the same manifest file. The PVC calls out the storage class allocated for that tenant, and the application calls out the PVC as shown below.

The above example creates a deployment with one pod. The Kubernetes deployment guarantees one pod always exists, even if the host is removed from the cluster. We are able to use ReadWriteOnce because only one pod will ever exist at a time for that specific tenant in this example.

How would I determine who is using my storage?

Provided different Kubernetes namespaces and/or StoragePrefix’s are used, managing the separate tenants on the ONTAP backend is easy. The volumes displayed in ONTAP are prefaced by the Kubernetes namespace. Additionally, the same PVC name used in Kubernetes is shown right in ONTAP. The storage usage and availability per tenant is easily seen below.

What now?

Keeping the tenants separate is easy with Trident. Try it out for yourself in a lab and let us know how it goes. We would love to hear from you on our Slack #containers channel.


Diane Patton on Email
Diane Patton
Diane is a Technical Marketing Engineer with NetApp, in the open eco-systems group supporting Docker, Kubernetes, and OpenShift integration with NetApp products. She works with product management, marketing, and development to evangelize, support, and help drive new technologies into Trident, the open source storage provisioner for containers maintained by NetApp. Diane has over 20 years experience in the IT industry (Optical DWDM, Layer 2, Layer 3, container networking, storage), holds CCIE 2537 Emeritus, and has a BS and MS in Electrical Engineering. When not working on open technologies, you will find Diane on a tennis court.