NetApp’s StorageGRID object store is very easy to use, but it can be challenging to get all of the right infrastructure together to deploy it, especially if you just want to try it out. But under the covers it’s built and deployed as container images, and recent Kubernetes-friendly enhancements make it possible to use those to … Read more
StorageGRID Webscale 11 now supports native integration into the AWS cloud. This includes cross-region replication to AWS S3 using CloudMirror, triggering of notifications to Simple Notification Service (via SNS), as well as metadata streaming into Elasticsearch for search capability. Introduction With Amazon’s Simple Storage Service (S3) being “the internet’s hard drive”, many younger companies already … Read more
Introduction This quick example shows how we can update the bucket access policy in StorageGRID in order to allow anonymous access. This allows access to the bucket without S3 credentials, e.g. through a browser. Instructions In this example, we utilize s3cmd to connect to StorageGRID Webscale. In order to get s3cmd talking to StorageGRID, update … Read more
StorageGRID Webscale provides highly scalable S3-compatible object storage. One system can be composed of hundreds of StorageGRID Webscale nodes deployed across Docker, VMWare, engineered systems (appliances), and even in the cloud. StorageGRID Webscale 11.0 makes managing heterogenous nodes easier by introducing a new metrics database engine that collects system and services level attributes and makes … Read more
tl;dr Performing metadata search on billions of objects is now possible with StorageGRID Webscale by streaming object metadata into Elasticsearch. Introduction One advantage of using object over block and file storage is that data can be enriched with metadata and tags. These are little key-value pairs that are attached via the S3 API. Consequently, you … Read more
The Docker Registry acts as the repository for image data for most deployments. Whether you’re using the fully supported Docker Trusted Registry, or the open source version deployed by itself or in combination with something like Red Hat’s OpenShift, the preferred storage for the images is object based. While the registry can be configured to … Read more
This is an update to a post from November 2016. A while ago, NetApp released version 10.4 of StorageGRID Webscale, our massively-scalable, S3-compliant Object Store. With this release, we enhanced our support for S3 Bucket Policies, which now simplify the creation of Write-Once-Read-Many buckets (WORM). Those WORM buckets allow the creation of new objects, but do not allow overwrites, … Read more
Introduction In this post we will show how to deploy a “stateless” Apache Spark cluster on Kubernetes. Spark is a fast analytics engine designed for large-scale data processing. Furthermore, we will then run analytics queries against data sitting in S3, in our case StorageGRID Webscale. We use S3 as the data source/target because it is an elegant … Read more
One of the great things about StorageGRID Webscale is that it offers comprehensive S3 support, which is 100% compatible with the AWS S3 SDKs. This makes application code portable, as it does not rely on proprietary object storage APIs. And that applies to the command line too. In this post, I’ll show you how to access StorageGRID’s S3 … Read more
NetApp recently released version 10.3 of StorageGRID Webscale, our massively-scalable Object Store. One of the release highlights for me was definitely the support of S3 Versioning. This feature allows you to create multiple S3 objects with the same key in one bucket. By using a combination of versioning and S3 Bucket Polices, we can now create Write-Once-Read-Many buckets … Read more