StorageGRID Webscale 11 now supports native integration into the AWS cloud. This includes cross-region replication to AWS S3 using CloudMirror, triggering of notifications to Simple Notification Service (via SNS), as well as metadata streaming into Elasticsearch for search capability. Introduction With Amazon’s Simple Storage Service (S3) being “the internet’s hard drive”, many younger companies already … Read more
tl;dr Performing metadata search on billions of objects is now possible with StorageGRID Webscale by streaming object metadata into Elasticsearch. Introduction One advantage of using object over block and file storage is that data can be enriched with metadata and tags. These are little key-value pairs that are attached via the S3 API. Consequently, you … Read more
Introduction In this post we will show how to deploy a “stateless” Apache Spark cluster on Kubernetes. Spark is a fast analytics engine designed for large-scale data processing. Furthermore, we will then run analytics queries against data sitting in S3, in our case StorageGRID Webscale. We use S3 as the data source/target because it is an elegant … Read more