Sites trying to move into a DevOps model often want to implement a Continuous Integration (CI)/Continuous Deployment (CD) process.  This can be challenging without any frames of reference.  For our Ansible development we use such a process and want to share that with you. We will describe the continuous integration and practices that were used to develop, test, and validate the Ansible modules. The DevOps team delivered 66 Ansible modules as part of Red Hat Ansible Engine 2.6 & 2.7 release by referring to this test model. 

This model was used to run functional tests against various versions of ONTAP and resolve the bugs internally, before publication to GitHub as part of the Ansible releases. These practices helped to attain a higher level of quality and rapidly develop modules. 

Achieving Agile Testing  

Ansible Engine modules are developed by using Agile practices. The Jenkins pipeline runs each time a code is submitted to Bitbucket. The pipeline is set up such that shorter tests, which are likely to fail, are run first. The longer tests that are not likely to fail, are run last.  

There are three main sections in the Jenkins pipeline: 

  • Static Analysis: this step concurrently runs pylint, pep8, Ansible validate-modules, and sanity tests 
  • Unit Test: this step runs all our unit tests in just minutes 
  • System Tests: these steps set up a virtual ONTAP system, runs a test playbook on the virtual system and verifies the results. This tests typically last between half an hour to one hour 

Our Pipeline uses the following tools during its execution  

  • Bitbucket (GIT) for storing source code and code reviews 
  • JIRA for tracking for collecting the customer requirements, planning the work items, and for tracking progress 
  • Confluence for documentation  
  • Jenkins with Blue Ocean for automated testing. 
  • Ansible for playbooks 
  • Docker for unit tests 
  • ONTAP simulator (Simulate ONTAP) for validating the test cases 

Setup Environment 

The first part of the pipeline is to set up the environment. 

Static Analysis 

The Static analysis section is a Parallel set in our Jenkins pipeline, which means each of the jobs in that column will run at the same time as its own job. Instead of running ansible-test sanity, which runs all of Ansible tests on all modules, we run each test separately in parallel. The tests are forced to run only on our modules. This greatly speeds up the runtime from over 10 minutes to just seconds. This approach enables the developer to ascertain issues with Static Analysis seconds after submitting their code to Bitbucket and can submit a fix right away. 

Example of parallel pipeline  

    stage('Static Analysis') { 
      parallel { 
        stage('validate-modules') { 
          steps { 
            sh ''' 
              . ansible/hacking/env-setup 
              pip install jinja2 
              pip install mock 
              pip install voluptuous 
              ansible-test sanity --test validate-modules lib/ansible/modules/storage/netapp/na* 
            ''' 
          } 
        } 
      } 
    }  

Unit Test and Coverage 

After the Static Analysis section completes successfully, we proceed to the Unit Test. Because the Unit Test takes a little longer to complete, we put them on a separate stage after Unit Tests. For the Unit Test, we use Ansible standard Unit Test command—ansible-test units— to run the unit tests inside a Docker container. Like the Static Analysis, we specify only the NetApp modules on which to run, bringing down the runtime from over 20 minutes to about 1 minute.  

    stage('Unit Test') { 
      steps { 
        sh ''' 
          . ansible/hacking/env-setup 
          ansible-test units --docker=default --python 2.7 --coverage <List of modules to run> 
        ''' 
      } 
    }

Virtual ONTAP Simulator (VSIM) Creation 

For system tests, we wanted to make sure that each time we ran the test, we were on a fresh machine so that the previous commands do not affect the next run of the tests. To enable this, we used the ONTAP simulator to create a for each run.  

Playbooks and Requisites for Running Playbooks 

Playbooks helps to validate module features and options. It also helps to verify the options that are not supported with developed modules. 

  1. Write a playbook to test the module inside the directory: ansible-playbooks/ansible_playbooks/test_playbooks/
  2. If there is any need or dependency (license) on resource to run the newly written playbook, add the same to “prerequisites.yml.”

For example: >File: ansible-playbooks/ansible_playbooks/sample_playbooks/prerequisites.yml  

Use case 1: A Vserver (SVM) is required to test a volume module. 

A Vserver is created in this playbook, which will be referred in test_volume.yml.file. 

Use case 2: Creating an NFS server and adding an NFS license to it. 

Add steps to run the playbook in the “Playbook” stage of Jenkins file. 

File: ansible-playbooks/Jenkinsfile 

Virtual ONTAP Simulator (VSIM) Deletion 

After the system tests have completed, Simulate ONTAP is deleted since it is no longer needed.  

Hopefully this information helps you along your own CI/CD process.  Explore www.netapp.io more to find other elements that can help you achieve your goals faster.  If you aren’t already a part of our thePub Slack Workspace get an invitation at www.netapp.io/slack.  We discuss quite a few topics there, so there is always something to learn. 

Good luck, and good developing.