Overview

In this post, we will extract the Docker images from the Ubuntu/Debian bare metal distribution packages, make the images available to our Kubernetes worker nodes and deploy StorageGRID onto our Kubernetes cluster. Our example shows a minimal, single site StorageGRID deployment with one primary Admin Node and three Storage Nodes.

Networking

Kubernetes pods within the same cluster can communicate directly with each other; this is perfect for a single StorageGRID site as all the StorageGRID nodes need to intercommunicate.  Additionally, the three Storage Nodes will register with the Admin Node by providing the Admin Node’s DNS name during deployment.  A Kubernetes headless service will provide DNS mappings for the StorageGRID nodes.  Using the DNS name, as opposed to the IP, will also allow the Storage Nodes to be deployed in parallel with the Admin Node.

Note: The existence of multiple sites would require StorageGRID nodes at each site be able to route directly to StorageGRID nodes at the other sites.  This could be accomplished with a VPN or other tunneling technology and is outside the scope of this blog.

StorageGRID Nodes

Admin Node (one required)
A primary Admin Node is required for every StorageGRID deployment (secondary Admin Nodes are optional).  Admin Nodes provide the Grid Manager Interface (GMI) for administering your StorageGRID deployment using a web browser.

Storage Node (three required)
Three Storage Nodes are required per StorageGRID site.  Storage Nodes are the backbone of a StorageGRID deployment

Gateway Node (optional)
The API Gateway Node monitors the health of the grid and the usage of each Storage Node.  Since this is a minimalistic deployment, we will forego the Gateway Node’s health checks and use Kubernetes basic round-robin built-in load-balancer.

For further discussion on StorageGRID load balancing, see this Technical Report.

Prerequisites

Kubernetes

• Command line access to Master and Worker nodes of the cluster.
• Resources for four StorageGRID nodes (1 primary Admin Node, 3 Storage Nodes) Each StorageGRID node will require:
  • 24Gi of memory (total of 96Gi)
  • 8 CPUs (total of 32 CPUs)
• Persistent Volumes for the StorageGRID primary Admin Node:
  • 100Gi mounted at /var/local
  • 200Gi mounted at /var/local/mysql_ibdata
  • 200Gi mounted at /var/local/audit/export
• Persistent Volumes for three StorageGRID Storage Nodes:
  • 100Gi mounted at /var/local (per Storage Node)
  • 4Ti mounted at /var/local/rangedb/0 (per Storage Node)
  • 4Ti mounted at /var/local/rangedb/1 (per Storage Node)
  • 4Ti mounted at /var/local/rangedb/2 (per Storage Node)

StorageGRID Docker Images

StorageGRID 11.5.0 (SG1000 or SG100 Primary Admin Node tgz)

• Direct your browser to (requires registration): https://mysupport.netapp.com/site/products/all/details/storagegrid/downloads-tab/download/61023/11.5.0 Click CONTINUE and accept the Caution and EULA
• Under SG1000 or SG100 Primary Admin Node, Select Download STORAGEGRID 11.5.0 DEB.TGZ this file will be downloaded: StorageGRID-Webscale-11.5.0-DEB-20210511.2328.e9368a4.tgz

Kubernetes Worker Nodes Preparation Load the StorageGRID Docker images on each Kubernetes node.

Note: Alternatively, the StorageGRID images could be hosted on a private Docker repository; just make sure to update the primary-admin.yaml and the storage.yaml files with the image path/names.

Deploy StorageGRID on Kubernetes

Follow the detailed kubectl commands documented on GitHub:
https://github.com/NetApp-StorageGRID/Kubernetes/blob/master/README.md

• The Admin Node service (GMI HTTPS – 443) will be mapped to an external IP.
• The Storage Node service will map S3 protocol (port 18082) from the Storage Nodes to an external, load balanced IP.

Install StorageGRID

• Direct your browser to https://<EXTERNAL-IP> (of the admin-node-service)
• Accept the default StorageGRID insecure certificate.
• From this point forward, you can follow the StorageGRID 11.5.0 installation documentation.
• Click Install a StorageGRID Webscale system.

Follow the installation wizard – paying special attention to:

• Step 1: License – Use the demo license that came with the download: /tmp/StorageGRID-Webscale-11.5.0/debs/NLF000000.txt
• Step 2: Grid Network – Use the network available to the Kubernetes pods
• Step 3: Grid Nodes – Make sure all four StorageGRID nodes get approved.

Note: NTP, DNS, and other fields will be specific to your environment, fill them in accordingly. Click the Install button and monitor the installation.

After installation, simply:

• Log in as the administrator (use the password created during installation).
• Add Tenants and Buckets (making note of AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID).
• Don’t forget to direct your S3 client to port 18082 of the EXTERNAL-IP of the service.

Summary

At this point you have a fully functional StorageGRID system; albeit very minimum isolated single site grid. You can now leverage information lifecycle management (ILM) rules to manage object data.

Kubernetes and its deployment model have greatly simplified this whole process. We’re specifying compute resources, storage resources, DNS names, services to expose, etc.. Also, we’re deploying all the pods as StatefulSets, this makes all the PVCs stick their pods.

The yaml files can be found on GitHub. Go kick the tires!!!