WORM S3 Buckets with StorageGRID Webscale

NetApp recently released version 10.3 of StorageGRID Webscale, our massively-scalable Object Store. One of the release highlights for me was definitely the support of S3 Versioning. This feature allows you to create multiple S3 objects with the same key in one bucket. By using a combination of versioning and S3 Bucket Polices, we can now create Write-Once-Read-Many buckets (WORM). Those WORM buckets allow the creation of new objects, but do not allow overwrites or deletion of existing content. The beauty of this solution is that it is purely utilizing standard S3 features without relying on any vendor-specific WORM implementation.

To set this up, let’s get started:

Step 1 – Create a versioned bucket

Create a new bucket called “worm-bucket” from our master tenant:

If you haven’t setup proper SSL certificates in StorageGRID, you can use the “–no-verify-ssl” flag to disable SSL checkin. Obviously, this is not recommended in production!

Next, let’s enable bucket versioning:

Make sure it actually worked:

Step 2 – Create a new storage tenant

Next, we’ll create a new WORM storage tenant in StorageGRID Webscale, either via the UI or via its RESTful API. What we need to remember is its Tenant ID, in this case “75992376408157494073”.

Step 3 – Apply Bucket Policy

Next, we need to create our Bucket Policy that we’ll apply to our WORM bucket/tenant:

In this case, we’ll allow most object/bucket operations, except those that could delete something or change the bucket policy. If required, it can be made even more restrictive.

Step 4 – Test it!

Let’s check if our WORM Tenant can see the bucket, note that we now access with our newly created “worm_tenant” profile:

Great, the bucket is visible in the other tenant. Let’s create two objects with the same name:

Two new Version IDs, excellent! Now, let’s try to delete our objects or delete a specific version:

No chance, in both cases we get an “access denied” message, exactly what we’re shooting for! Obviously it is to note that our main tenant still has full control over the bucket, and can also delete objects.

Summary

With StorageGRID Webscale 10.3, it is quite simple to setup a WORM-like bucket in S3. The beauty of this solution is, that it doesn’t require any vendor specific WORM implementation because it relies purely on standard S3 features.

If you have any questions please join us in the Slack channels or send an email to opensource@netapp.com! Or, you can reach out to us on the developer community, developer.netapp.com. We love hearing from you and learning about your challenges!

Clemens Siebler on GithubClemens Siebler on LinkedinClemens Siebler on Twitter
Clemens Siebler
Manager Solution Architects EMEA
Clemens is leading a technical team of Solution Architects in EMEA. In his current role, he and his team are evangelizing upcoming market trends like Containers, Object Storage, OpenStack, and NFV. His current passion is enabling customers to transition their large scale workloads to Object Storage. Before, he worked as a Software Engineer on NetApp’s software products, where he published multiple patents on plug-in frameworks.

Leave a Reply