Automating StorageGRID Operations with Ansible

The first release of the Ansible Modules for StorageGRID is centered around automation and orchestration of Day-1 and Day-2 operations.

If you’ve been following the many Ansible blog posts here at netapp.io, you will already be familiar with how Ansible makes it easy to perform repeatable tasks quickly and consistently.

With the introduction of StorageGRID modules for Ansible, administrators will benefit from simplified, templated and idempotent operations for NetApp StorageGRID.

In this post, I am going to focus on a typical StorageGRID workflow of provisioning a new Tenant within the Grid, right up to the point of creating S3 Buckets.

Requirements

To get started, we need a StorageGRID management account with at least the “Tenant Accounts” privilege. The minimum required version of StorageGRID is 11.2.

Next, we need to configure our Ansible environment with the NetApp StorageGRID modules. Like the other NetApp Ansible modules, these are packaged and distributed through Ansible Galaxy.

From your Ansible host, install the modules from Ansible Galaxy with the following command:

$ ansible-galaxy collection install netapp.storagegrid

Module Overview

Each StorageGRID module is either Grid-scoped or Tenant-scoped, and are named as follows:

• na_sg_grid_* - perform grid operations such as creating Tenant Accounts or setting global DNS and NTP configuration. Requires Grid user credentials.
• na_sg_org_* - perform tenant operations such as creating users, groups, S3 keys and buckets. Requires Tenant user credentials.

Playbook Structure

Here we are going to create a playbook YAML file to perform a sequence of tasks against a StorageGRID environment. The full playbook example is shown below, and we’ll look at key sections in further detail.

---
- hosts: localhost
  collections:
    - netapp.storagegrid
  vars_files:
    /home/ansible/sg_vars.yaml
  gather_facts: no
  tasks:
  - name: Get grid authorization token
    uri:
      url: "{{ grid_admin_base_url }}/api/v3/authorize"
      method: POST
      body: {
        "username": "{{ grid_username }}",
        "password": "{{ grid_password }}",
        "cookie": false,
        "csrfToken": false
      }
      body_format: json
      validate_certs: false
    register: auth

  - name: Create a StorageGRID Tenant
    na_sg_grid_account:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      name: thePub
      protocol: s3
      management: true
      use_own_identity_source: false
      allow_platform_services: false
      password: "{{ tenant_password }}"
      quota_size: 100
    register: account

  - name: Get user authorization token
    uri:
      url: "{{ grid_admin_base_url }}/api/v3/authorize"
      method: POST
      body: {
        "accountId": "{{ account.resp.id }}",
        "username": "{{ tenant_username }}",
        "password": "{{ tenant_password }}",
        "cookie": false,
        "csrfToken": false
      }
      body_format: json
      validate_certs: false
    register: auth

  - name: Create a group
    na_sg_org_group:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      display_name: thePub-Patrons
      unique_name: group/thepub-patrons
      management_policy:
        manage_all_containers: true
        manage_endpoints: true
        manage_own_s3_credentials: false
        root_access: false
      s3_policy: {"Statement":[{"Effect":"Allow","Action":"s3:*","Resource":"arn:aws:s3:::*"}]}

  - name: Create users
    na_sg_org_user:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      full_name: "{{ item }}"
      unique_name: "user/{{ item }}"
      member_of: "group/thepub-patrons"
    with_items: "{{ users }}"

  - name: Create a s3 key for our users
    na_sg_org_user_s3_key:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      unique_user_name: "user/{{ item }}"
    with_items: "{{ users }}"

  - name: Create a s3 bucket
    na_sg_org_container:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      name: thepub-bucket1

At the very start of the playbook we’re going to import a variable file and also declare that we are using the “netapp.storagegrid” collection. This shortcut allows us to just specify the module name in our playbook tasks rather than specifying the fully qualified collection name each time.

- hosts: localhost
  collections:
    - netapp.storagegrid
  vars_files:
    /home/ansible/sg_vars.yml

This is my variable file

    grid_admin_base_url: https://sg-adm1.example.com
    grid_username: myadminaccount
    grid_password: netapp123
    tenant_username: root
    tenant_password: changeme
    users:
      - bill
      - ted

The StorageGRID REST API requires an authorization token when invoking a Grid or Tenant operation.

The first step is to obtain a valid token, which we can perform with the URI module. This is a built-in Ansible module which allows us to make a simple HTTP request. Using the URI module, we will make a REST API call to StorageGRID’s authorize endpoint and specify our Grid credentials in the request body in order to obtain an authorization token for subsequent tasks. We are registering the response from this REST API call into a variable called “auth”, which allows us to reference the resulting token through the variable “auth.json.data”

  - name: Get user authorization token
    uri:
      url: "{{ grid_admin_base_url }}/api/v3/authorize"
      method: POST
      body: {
        "accountId": "{{ account.resp.id }}",
        "username": "{{ tenant_user }}",
        "password": "{{ tenant_pass }}",
        "cookie": false,
        "csrfToken": false
      }
      body_format: json
      validate_certs: false
    register: auth

The next two tasks require a little bit more explanation...

To summarize the workflow, the purpose of this playbook is to create a new StorageGRID tenancy; populate some users and groups; assign an S3 key; and finally create an S3 bucket. This sequence commences with Grid-scoped tasks and then crosses over to Tenant-scoped tasks, which means that we need to use Tenant-admin credentials to perform the rest of the tasks after creating the Tenant Account. The common pattern which is used here is registering the output of a task to use as an input parameter for  subsequent tasks.

The first StorageGRID module we’re calling is “na_sg_grid_account” to create a new Tenant Account. The result from this task is registered as the variable “account” which will contain the unique ID generated by StorageGRID for the resulting Tenant Account.

  - name: Create a StorageGRID Tenant
    na_sg_grid_account:
      api_url: "{{ grid_admin_base_url }}"
      auth_token: "{{ auth.json.data }}"
      validate_certs: false
      state: present
      name: thePub
      protocol: s3
      management: true
      use_own_identity_source: false
      allow_platform_services: false
      password: "{{ tenant_password }}"
      quota_size: 100
    register: account

The Account ID needs to be used when obtaining a new authorization token for a Tenant-scoped task. This is very similar to the very first step of obtaining a Grid authorization token via the URI module, except that it contains an extra parameter in the body for “accountId”.

  - name: Get user authorization token
    uri:
      url: "{{ grid_admin_base_url }}/api/v3/authorize"
      method: POST
      body: {
        "accountId": "{{ account.resp.id }}",
        "username": "{{ tenant_user }}",
        "password": "{{ tenant_pass }}",
        "cookie": false,
        "csrfToken": false
      }
      body_format: json
      validate_certs: false
    register: auth

From there, the playbook uses the following modules to provision the respective StorageGRID resources within the Tenant Account:

• na_sg_org_group
• na_sg_org_user [notice we’re looping over a set of users and assigning them to the group]
• na_sg_org_user_s3_key
• na_sg_org_container

Playbook Execution

To execute this playbook, simply run:

$ ansible-playbook sg_tenant_workflow.yaml

There are no other command line options required as the "vars_file" is referenced in the playbook directly. However, in order to template this, you may wish to experiment with the "--extra-vars" command line parameter instead.

You can also increase the output verbosity with "-v" or "-vvv" to inspect the result output when building your playbooks and roles for StorageGRID.

Checkout our website for regular updates on Ansible modules for all NetApp products. For support related questions and updates, join our Slack channel #configurationmgmt at netapp.io.