NetApp recently released version 10.3 of StorageGRID Webscale, our massively-scalable Object Store. One of the release highlights for me was definitely the support of S3 Versioning. This feature allows you to create multiple S3 objects with the same key in one bucket. By using a combination of versioning and S3 Bucket Polices, we can now create Write-Once-Read-Many buckets (WORM). Those WORM buckets allow the creation of new objects, but do not allow overwrites or deletion of existing content. The beauty of this solution is that it is purely utilizing standard S3 features without relying on any vendor-specific WORM implementation.

To set this up, let’s get started:

Step 1 – Create a versioned bucket

Create a new bucket called “worm-bucket” from our master tenant:

$ aws s3api create-bucket --bucket worm-bucket --endpoint https://s3.mycompany.com:8082 --profile master_tenant
{
    "Location": "/worm-bucket"
}

If you haven’t setup proper SSL certificates in StorageGRID, you can use the “–no-verify-ssl” flag to disable SSL checkin. Obviously, this is not recommended in production!

Next, let’s enable bucket versioning:

$ aws s3api put-bucket-versioning --bucket worm-bucket --versioning-configuration Status=Enabled --endpoint https://s3.mycompany.com:8082 --profile master_tenant

Make sure it actually worked:

$ aws s3api get-bucket-versioning --bucket worm-bucket --endpoint https://s3.mycompany.com:8082 --profile master_tenant
{
    "Status": "Enabled"
}

Step 2 – Create a new storage tenant

Next, we’ll create a new WORM storage tenant in StorageGRID Webscale, either via the UI or via its RESTful API. What we need to remember is its Tenant ID, in this case “75992376408157494073”.

Step 3 – Apply Bucket Policy

Next, we need to create our Bucket Policy that we’ll apply to our WORM bucket/tenant:

$ cat no_delete.json
{  
   "Statement":[  
      {  
         "Effect":"Allow",
         "Principal":{  
            "SGWS":"75992376408157494073"
         },
         "Action":[  
            "s3:PutObject",
            "s3:ListMultipartUploadParts",
            "s3:GetObjectVersion",
            "s3:GetObject",
            "s3:AbortMultipartUpload",
            "s3:ListAllMyBuckets",
            "s3:ListBucket",
            "s3:ListBucketMultipartUploads",
            "s3:ListBucketVersions"
         ],
         "Resource":[  
            "urn:sgws:s3:::worm-bucket",
            "urn:sgws:s3:::worm-bucket/*"
         ]
      }
   ]
}

In this case, we’ll allow most object/bucket operations, except those that could delete something or change the bucket policy. If required, it can be made even more restrictive.

$ aws s3api put-bucket-policy --bucket worm-bucket --policy file://no_delete.json --endpoint https://s3.mycompany.com:8082 --profile master_tenant

Step 4 – Test it!

Let’s check if our WORM Tenant can see the bucket, note that we now access with our newly created “worm_tenant” profile:

$ aws s3api list-buckets --endpoint https://s3.mycompany.com:8082 --profile worm_tenant
{
    "Owner": {
        "DisplayName": "worm-tenant",
        "ID": "75992376408157494073"
    },
    "Buckets": []
}

Great, the bucket is visible in the other tenant. Let’s create two objects with the same name:

$ aws s3api put-object --bucket worm-bucket --key object1 --body test.txt --endpoint https://s3.mycompany.com:8082 --profile worm_tenant
{
    "VersionId": "QjU5NUIzM0EtQjU3RS0xMUU2LTgwMDAtMDAwMDAwQzU1QUIy",
    "ETag": "\"7e102cef040e24d7e5fea56d22a170b9\""
}
$ aws s3api put-object --bucket worm-bucket --key object1 --body test.txt --endpoint https://s3.mycompany.com:8082 --profile worm_tenant
{
    "VersionId": "QkFGQTI4ODgtQjU3RS0xMUU2LTgwMDAtMDAwMDAwQzU1QUIy",
    "ETag": "\"7e102cef040e24d7e5fea56d22a170b9\""
}

Two new Version IDs, excellent! Now, let’s try to delete our objects or delete a specific version:

$ aws s3api delete-object --bucket worm-bucket --key object1 --endpoint https://s3.mycompany.com:8082 --profile worm_tenant
A client error (AccessDenied) occurred when calling the DeleteObject operation: Access authorization failed.

$ aws s3api delete-object --bucket worm-bucket --key object1 --version-id QkFGQTI4ODgtQjU3RS0xMUU2LTgwMDAtMDAwMDAwQzU1QUIy --endpoint https://s3.mycompany.com:8082 --profile worm_tenant
A client error (AccessDenied) occurred when calling the DeleteObject operation: Access authorization failed.

No chance, in both cases we get an “access denied” message, exactly what we’re shooting for! Obviously it is to note that our main tenant still has full control over the bucket, and can also delete objects.

Summary

With StorageGRID Webscale 10.3, it is quite simple to setup a WORM-like bucket in S3. The beauty of this solution is, that it doesn’t require any vendor specific WORM implementation because it relies purely on standard S3 features.

If you have any questions please join us in the Slack channels or send an email to opensource@netapp.com! Or, you can reach out to us on the developer community, developer.netapp.com. We love hearing from you and learning about your challenges!