Setting up an Edge system All in One with OpenStack and ONTAP Select

Some sites aren’t large enough to need or have an entire datacenter infrastructure. Sometimes you want to be able to run just a datacenter in a box. Remote locations, small sites, any place where you might need the entire features of a data center, but only have one box. How do you get all those features with only a single system?

We call these ‘Edge’ systems and with ONTAP Select 2.5, now you can deploy on KVM. This means you can run OpenStack and ONTAP on a single box, giving you all the advantages of both OpenStack and ONTAP.

ONTAP Select is an easily deployed data management solution that runs on your current hardware, existing servers, transforming it into a software-defined storage (SDS) infrastructure. ONTAP Select can be quickly installed on any commodity hardware and be up in running in minutes.

I am going to walk through setting up a single node like that and will use ONTAP Select as the backend for both Cinder and Manila. ONTAP Select is setup using the Select deploy KVM kvmge. All of the steps necessary for this are detailed here. More information on the setup process of Select can be found in the “Installation and Cluster Deployment Guide for KVM using Deploy 2.6” found on the NetApp Support site. OpenStack will be deployed using the RDO distribution. RDO is the upstream repository of the Red Hat OpenStack Platform. Packstack will be the tool we use to install OpenStack. Packstack is a collection of Puppet scripts that make setting up a single all in one OpenStack box trivial. Information about RDO and Packstack can be found at http://rdoproject.org. The minimum requirements for the system you set this up on are;

  • 8 CPUs (vCPUs)
  • 32 gigs of RAM
  • 2 network interfaces, the first of which should be hardcoded with an IP and the second can just be left to start, but doesn’t need an IP.
  • 1.2TB or more to present as a single device for ONTAP Select. This can be either a RAID of disks presented as a single device, or if you have enough space, a single drive, or partition on a single drive. This cannot be a LVM volume, it must be a raw SCSI device.
  • Minimum of 5 IP addresses plus a pool of how ever many you want for OpenStack instances to use.

For my setup I am using the following IPs and conventions;

  • The host OS is CentOS 7.2 with the RDO distro of OpenStack
  • My hostname is AIO.local, and I added that to /etc/hosts
  • My username is netapp
  • My device for my select storage pool is /dev/sdb
  • My first NIC is device ens160 with address 172.32.0.155
  • My second NIC is device ens192 which I will attach to my bridge, br-ex
  • My network is 172.32.0.0/24
  • My router/gateway/DNS is 172.32.0.1
  • My deploy system IP is 172.32.0.156
  • My select cluster uses IPs;
    • 172.32.0.157 – cluster management
    • 172.32.0.158 – node management
    • 172.32.0.159 – SVM LIF
  • I use 172.32.0.200-172.32.0.240 for my float IP range

There are some major differences between the CentOS repository qemu-kvm and the qemu-kvm-ev that OpenStack uses in RDO, so ONTAP Select has to be setup first.

  1. The first step is to prepare the host for running the deploy KVM conflict between deploy and qemu.

  1. The bridge that will be used to allow deploy, Select, and any instances that a run to be accessed from the network needs to be called br-ex because packstack looks for that later.

  1. Download the Select Deploy for KVM raw image file from the NetApp support site, and move it to the host for deployment.

  1. Now the deploy KVM instance can be created.

  • If you happen to get an error like;

“ERROR unsupported configuration: CPU mode ‘custom’ for x86_64 kvm domain on x86_64 host is not supported by hypervisor
Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
virsh –connect qemu:///system start deploy-kvm
otherwise, please restart your installation.”

You will need to change the KVM group to id 78 and run the virt-install command again. You can change the group id like this;

  1. Now that the deploy system is booting the console needs to be attached to and the system configured.

  1. Select relies on a virsh storage pool for its space. A virsh storage pool is just a dedicated collection of disk used by libvirt for creating LVM resources for KVM instances.

  1. The web gui can be used to deploy Select or connect via ssh or the console and run the CLI commands to configure and deploy Select. I am covering the CLI instructions. I am also using the evaluation version of Select. It allows a 90 day testing period. Please refer to the ONTAP Select Administration Guide for information on how to apply your license if you purchase a license.
  • For this part, and the packstack install, root needs to have a password so that deploy can validate and install the Select instance, and puppet can ssh to itself.

Use the host show-all command to see that the node was authenticated properly.

Now that the host is configured to be able to host Select, the cluster (1 node in this case) is created.

This sets the admin password to netapp123, change it if desired.

After the command is run the cluster starts to deploy. This process can take from 15 minutes to 2 hours based on a number of factors. Check the status with cluster show-all.

Once the cluster is online, exit from the ssh session.

  1. Now that Select is up and running, configure the host to be able to run OpenStack.

  1. After the reboot, the yum update that wasn’t done earlier is run, followed by another reboot.

At this point Select will no longer boot and you will get this error in the qemu logs;

You don’t need to worry about this as this will be fixed in a later step.

  1. Now using PackStack, OpenStack will be installed on this node. Root’s local ssh key will also be added locally as a key to allow Puppet to run without asking for root’s password constantly.

The PackStack commmand here sets Neutron to use the openvswitch port of br-ex with ens192 as its interface, and calls it extnet for reference later in configs. Also, the demo project and user are disabled, and the configurations for ONTAP Select as a backend for Manila are done.

  1. Now that OpenStack is installed, Select needs to be modified to run.

Change the entry host-passthrough to host-model.

Watch till boot finishes and login.

  1. Now the Select cluster will be configured. This covers creating a data aggregate, adding a SVM and the data LIF for it, as well as activating NFS, making the necessary changes to the default export policy to allow connections, and finally allowing HTTP communication for Manila.

  1. Manila will need a few modifications still to use the backend, as well as the GUI portions will need to be added to the Horizon dashboard.

When this is done verify the backend is setup properly with manila pool-list.

  1. So that as much as possible is managed through OpenStack, the Cinder backend NFS export will be created using Manila. This will allow administrators to resize the Cinder volume from the OpenStack interface. There is also the added bonus that this verifies Manila is working properly.

  1. Using the path from step 14 Cinder can now be configured to use a FlexVol from ONTAP Select as its backend storage.

  1. Test that the Cinder backend is working

There should be a file in the Cinder mnt named the same as the test volumes id number.  After you verify that the volume was indeed created, it can be deleted.

  1. Networking for the provider network and an image to test with are added now.

  1. Lastly a project for use by end users is created. These steps can be run every time a new project is needed. Be sure to replace all instances of the word “test” with the name of your project, and the password with the password you would like the default user to have. That password is “temp” in this example.  The user is created with the same name as the project. Note that ‘default’ cannot be used as a project name.

Also note that the exported values for some of the OS variables are being changed so that commands will be run as the temp user.  If you want to go back to full admin rights after, you will need to source the keystonerc_admin file again.

    • If the system ever has to be rebooted, wait for ONTAP Select to fully boot and then restart the Cinder and Manila services

Now there is a running single node system that can provide compute, shares, and block storage for an Edge private cloud. In addition, SnapMirror can be used to backup all the Manila shares to any other ONTAP system, including ONTAP Cloud, and access them. Also, the Cinder FlexVol can be backed up and, if needed, the presented Cinder volumes can be mounted as loop back devices remotely to get read only access to them in case of disaster, or verifications.

For more information on ONTAP Select visit the Select information site on NetApp’s website at http://www.netapp.com/us/products/data-management-software/ontap-select-sds.aspx

In a follow up post on the topic of this Edge system, I will demonstrate how you can use ONTAP Select to manage the configurations, lib files, and database for OpenStack making restoring or rebuilding an edge node back to its full configuration, with all shares, volumes, and instances a minor matter.

David Blackwell
Technical Marketing Engineer for OpenStack at NetApp
David is a twenty year IT veteran who has been an admin for just about every aspect of a DataCenter at one time or another. Currently he is the Technical Marketing Engineer for OpenStack at NetApp. When not working, or tinkering with new software at home, David spends most of his free time on his hobby of 3D printing.

Leave a Reply