Developers and data scientists frequently use NetApp’s volume cloning technology to clone development workspaces almost instantaneously. This can lead to operational challenges as end users often don’t have superuser privileges and, thus, are not able to run a mount operation in order to mount their newly created clones within their development environment. In this post, I am going to show you how to use the NetApp DataOps Toolkit and custom junction paths to safely and automatically mount all of your newly-created clones without having to run a mount command every time.

Note: This guide assumes that you perform your development or data science work within a Linux-based environment.

Step 1 – Install and configure the NetApp DataOps Toolkit

First, you will need to install and configure the NetApp DataOps Toolkit by following the instructions outlined in the README.

Run the following command to install the DataOps Toolkit.

python3 -m pip install netapp-dataops-traditional

Next, run the following command to configure the toolkit. You will be prompted to enter details regarding your specific NetApp storage system or service.

netapp_dataops_cli.py config

Step 2 – Create top-level volume

Once the DataOps Toolkit has been installed and configured, the next step is to create a top-level volume. You can name this volume whatever you want, but we recommend giving it a descriptive name that conveys its purpose. For example, if you are part of a development team, you may want to name it something like “dev_<team_name>” or “<project_name>”.

To create the top-level volume, run the following command.

netapp_dataops_cli.py create volume --name=<volume_name> --size=<volume_size>

The example that follows shows the creation of a volume named “team1” of size 500 GB.

$ netapp_dataops_cli.py create volume --name=team1 --size=500GB
Creating volume 'team1'.
Volume created successfully.

Step 3 – Configure the system to mount the top-level volume at startup

After the top-level volume has been created, you need to configure the system to automatically mount it at system startup. This will ensure that the top-level volume is always mounted within your development environment.

First, run the following command to retrieve the NFS mount target for the top level volume.

$ netapp_dataops_cli.py list vols
Volume Name    Size     Type     NFS Mount Target     Local Mountpoint    FlexCache    Clone    Source Volume    Source Snapshot
-------------  -------  -------  -------------------  ------------------  -----------  -------  ---------------  -----------------
team1          500.0GB  flexvol  10.61.188.90:/team1                      no           no                                         

Once you have the NFS mount target, you can configure the system to mount the volume at system startup by adding a line resembling the following to /etc/fstab. Replace the NFS mount target with your NFS mount target, and replace /mnt/team1 with your desired mountpoint.

Note: You may not have access to modify the /etc/fstab file. In that case, an administrator will need to perform this step for you.

$ sudo vi /etc/fstab

10.61.188.90:/team1 /mnt/team1 nfs defaults 0 0

Now, the system will automatically mount the top-level volume at system startup. However, this will not take effect until the next system restart. To mount the volume during the current session, run the following command.

Note: If you do not have superuser privileges, an administrator will need to perform this step for you.

$ sudo mount -a

Step 4 – Create your first workspace volume

Now that you have your top-level volume, you can create your first workspace volume. We call this a workspace volume because it will eventually contain artifacts/files that you work with. If you are a developer, this might be your codebase or a combination of your codebase and some prebuilt artifacts. If you are a data scientist, this will likely be your training or analytics dataset(s).

To create the workspace volume, run the following command.

netapp_dataops_cli.py create volume --name=<workspace_volume_name> --size=<workspace_volume_size> --junction=/<top-level_volume_name>/<workspace_volume_name> --uid=<unix_filesystem_user_id_to_apply_to_volume> --gid=<unix_filesystem_group_id_to_apply_to_volume> --permissions=<unix_filesystem_permissions_to_apply_to_volume>

The example that follows shows the creation of a workspace volume named “ws1” of size 500 GB. Note the usage of the top-level volume that was created in step 2 as part of the junction argument.

$ netapp_dataops_cli.py create volume --name=ws1 --size=500GB --junction=/team1/ws1 --uid=1000 --gid=1000 --permissions=0755
Creating volume 'ws1'.
Volume created successfully.

Step 5 – Confirm automatic mounting

Now comes the exiting part. Because of the junction path that you specified in step 4, your new workspace volume should have been automatically mounted “underneath” the top-level volume that was already mounted. To confirm that the workspace volume was automatically mounted, run the following command.

ls -lah <top-level_volume_mountpoint>

The example that follows uses the top-level volume mountpoint that we specified in the example in step 3.

$ ls -lah /mnt/team1/
total 16K
drwxrwxrwx 3 root root 4.0K Nov 5 21:02 .
drwxr-xr-x 7 root root 4.0K Nov 5 20:58 ..
drwxrwxrwx 2 root root 4.0K Nov 5 20:24 .snapshot
drwxr-xr-x 2 ai ai 4.0K Nov 5 21:02 ws1

Your workspace volume should show up as a directory “underneath” the top-level volume mount. You can access it from here without needing to run a mount command.

Step 6 – Populate workspace volume

At this point, you will want to populate your workspace volume with something useful so that you can take advantage of NetApp’s volume cloning technology and the wonders of automatic mounting using junction paths.

If you are a developer, you can clone or sync your codebase down to your workspace volume. This will enable you to rapidly clone your codebase on-demand. If you think that you would benefit from being able to clone certain built artifacts, you may also want to build these and store them on your workspace volume. If you are a data scientist, you can store your training and/or analytics dataset(s) on your workspace volume. In a nutshell, store anything that you would benefit from being able to rapidly clone.

Step 7 – Create your first clone

Now that your workspace volume has been populated with something useful, you can rapidly clone it on demand. To clone your workspace volume, run the following command. This operation should take no longer than a few seconds to complete regardless of the size of the volume or the amount of data stored on it. Yes, you read that right, you can create a full copy of your workspace in just a few seconds.

netapp_dataops_cli.py clone volume --name=<clone_volume_name> --source-volume=<source_workspace_volume_name> --junction=/<top-level_volume_name>/<clone_volume_name> --uid=<unix_filesystem_user_id_to_apply_to_volume> --gid=<unix_filesystem_group_id_to_apply_to_volume>

The example that follows shows the creation of a new workspace volume (the new clone) named “ws2”. Note the usage of the top-level volume that was created in step 2 as part of the junction argument.

$ netapp_dataops_cli.py clone volume --name=ws2 --source-volume=ws1 --junction=/team1/ws2 --uid=1000 --gid=1000
Creating clone volume 'ws2' from source volume 'ws1'.
Clone volume created successfully.

Because of the junction path that you specified, the new clone should have been automatically mounted “underneath” the top-level volume that was already mounted, just like the workspace volume that you previously created. To confirm that the new clone was automatically mounted, run the same ls command that you ran in step 5.

$ ls -lah /mnt/team1/
total 20K
drwxrwxrwx 4 root root 4.0K Nov 5 21:22 .
drwxr-xr-x 7 root root 4.0K Nov 5 20:58 ..
drwxrwxrwx 2 root root 4.0K Nov 5 20:24 .snapshot
drwxr-xr-x 2 ai ai 4.0K Nov 5 21:02 ws1
drwxr-xr-x 2 ai ai 4.0K Nov 5 21:02 ws2

Follow the NetApp DataOps Toolkit on GitHub

Now you know how to automatically mount new workspace volumes using the NetApp DataOps Toolkit and junction paths. For more information on the DataOps Toolkit and to stay up to date on new releases, check out and follow the DataOps Toolkit GitHub repo.

Mike Oglesby on GithubMike Oglesby on Linkedin
Mike Oglesby
Technical Marketing Engineer at NetApp
Mike is a Technical Marketing Engineer at NetApp focused on MLOps and Data Pipeline solutions. He architects and validates full-stack AI/ML/DL data and experiment management solutions that span a hybrid cloud. Mike has a DevOps background and a strong knowledge of DevOps processes and tools. Prior to joining NetApp, Mike worked on a line of business application development team at a large global financial services company. Outside of work, Mike loves to travel. One of his passions is experiencing other places and cultures through their food.

Pin It on Pinterest