WORM S3 Buckets with StorageGRID Webscale (Updated July 2017)

This is an update to a post from November 2016.

A while ago, NetApp released version 10.4 of StorageGRID Webscale, our massively-scalable, S3-compliant Object Store. With this release, we enhanced our support for S3 Bucket Policies, which now simplify the creation of Write-Once-Read-Many buckets (WORM). Those WORM buckets allow the creation of new objects, but do not allow overwrites, alteration, or deletion of existing content. The beauty of this solution is that it is purely utilizing standard S3 features without relying on any vendor-specific WORM implementation.

Here’s a quick rundown on how to configure this:

Step 1 – Create a versioned bucket

Create a new bucket called “wormbucket” from our master tenant:

If you haven’t setup proper SSL certificates in StorageGRID, you can use the “–no-verify-ssl” flag to disable SSL verification. Obviously, this is not recommended in production!

Step 2 – Create a new storage tenant

Next, we’ll create a new WORM storage tenant in StorageGRID Webscale, either via the UI or the RESTful API. What we need to remember is the new Tenant ID, in this case “75992376408157494073”. Otherwise we can also retrieve the Tenant ID by just listing the buckets of the tenant:

Step 3 – Apply Bucket Policy

Next, we need to create our Bucket Policy and apply it to our bucket through the master tenant:

In this case, we’ll allow all bucket and object operations for our tenant, but we explicitly deny the following ones in the “Deny” block:

  • Overwriting existing objects (this includes changing metadata via the COPY directive)
  • Deleting existing objects (including older versions)
  • Changing the bucket policies

Let’s apply the policy to the bucket:

Step 4 – Test it!

Let’s check if our WORM Tenant works by putting some objects into the bucket. Note that we now access the bucket with our newly created “worm_tenant” profile:

We can create new objects, great. Let’s try to overwrite i:

Looks good, we’re not able to overwrite the object. Let’s try deleting it:

Doesn’t work, excellent. Overwriting the object metadata also does not work:

Let’s try to be mean and overwrite the Bucket Policy:

No change. In all cases we get an “access denied” message, exactly what we were shooting for! Obviously our main tenant still has full control over the bucket, and can also delete objects.

Summary

With StorageGRID Webscale 10.4, it is very simple to create a WORM-like bucket in S3. The beauty of this solution is, that it doesn’t require any vendor specific WORM implementation because it relies purely on standard S3 features.

Clemens Siebler on GithubClemens Siebler on LinkedinClemens Siebler on Twitter
Clemens Siebler
Manager Solution Architects EMEA
Clemens is leading a technical team of Solution Architects in EMEA. In his current role, he and his team are evangelizing upcoming market trends like Containers, Object Storage, OpenStack, and NFV. His current passion is enabling customers to transition their large scale workloads to Object Storage. Before, he worked as a Software Engineer on NetApp’s software products, where he published multiple patents on plug-in frameworks.

Leave a Reply