Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someonePin on Pinterest
Read on Mobile

S3 and EBS: A Cloud Storage Story

cloud storage

AWS is a big company with many different offerings, services and products. AWS S3 is storage for the internet. It is an object based storage system that allows users to store unlimited objects on a pay- as- you- go model. EBS, the SAN offering from AWS, provides persistent storage that is ideal for file systems. Most of these products may appear similar because they all deal with storage of some sort. As a result, the internet is filled with debates about what service to use when. Take a look at this example in Stack Overflow and in Quora.

Look at S3— you can move petabytes and zetabytes of data to allow users to use S3 as a bulk repository, data lake for analytics, and long term storage. EBS is basically a network-based storage service that stores data in a persistent manner. You can detach an EBS volume and attach it to any machine, but you will need an EC2 instance to access the data. You can configure any file system and mount any of the available EC2 instance type on it.

But when should you use one or the other? This article addresses this question. But first, let’s begin with an overview of the two products’ features.

When should you prefer Object Store over a File System or Block Storage?

Comparison of Features

S3 EBS
Storage size and limitations No limit on the number of objects.
Individual S3 objects: A maximum of 5 TB.
Single PUT upload max size: 5GB (using multipart)**multipart upload is suggested by Amazon for any object with size over 100 MB
Maximum size of 16 TB. There is a limit of 40 EBS volumes for a single Linux instance and a limit of 26 EBS volumes for a single Windows instance.
No limitation on size. Data stored based on IOPS
Data Uploading Multipart upload capability for objects larger than 100 megabytes Provisioned IOPS for faster read/write IO operations
Performance
  • Highly scalable managed service.
  • Supports 100 PUT/LIST/DELETE requests per second by default.
  • Can automatically scale till 300 PUT/LIST/DELETE requests per second or more than 800 GET requests per second.
  • Manually scale the size of the volumes.
  • Uses provisioned IOPS for increased performance.
  • Baseline performance of 3 IOPS per GB for General Purpose volumes.
Data Stored Data stored stays in the region. Replicas are made within the region. Data stored stays in the Availability Zone. Replicas are made within the AZ.
Data Access Data can be accessed over internet from anywhere using console, CLI, REST or SOAP APIs. Can be accessed only by EC2 instance.
File Permissions/File System
  • Does not have a mountable file system.
  • Folder permissions do not pass to the folders by default, unlike the traditional file system.
  • Supports File Systems (e.g., ext3, ext4).
Supported Encryption Mechanisms Server-Side Encryption

  • Server-Side Encryption with Amazon S3-Managed Keys (SSE-S3)
  • Server-Side Encryption with AWS KMS-Managed Keys (SSE-KMS)
  • Server-Side Encryption with Customer-Provided Keys (SSE-C)

Client Side Encryption

  • Using an AWS KMS–Managed Customer Master Key (CMK)
  • Using a Client-Side Master Key
Using an AWS KMS–Managed Customer Master Key (CMK) – AES 256-bit Encryption standards
Access Control
  • Using Bucket Policies and User Policies
  • Managing Access with ACLs
  • Pre-signed URLs
  • Same as for EC2 Instance.
Availability 99.99% available. 99.99% available.
Withstand AZ Failure Can withstand up to two concurrent AZ failures. Cannot withstand AZ failure without point-in time EBS Snapshots.
Durability Eleven 9s of durability

(99.999999999%).

20 times more reliable than normal hard disk.
Consistency Amazon S3 offers read-after-write consistency for PUTS of new objects and eventual consistency for overwrite PUTS and DELETES.
This means that when an Object is uploaded for the first time it may not be accessible for a few milli seconds until data is replicated across all S3 clusters in the same region.
EBS is a file system that offers persistence storage. It follow a strict consistency model, meaning that any write to the file system is instantly visible.
Pricing
  • Per GB of storage used per month
  • Number of requests made (e.g., POST, GET)
  • S3 Inventory
  • S3 Analytics
  • Storage Class Analysis
  • S3 Object Tagging
  • Data Transfer per GB out of S3
  • Amazon S3 Transfer Acceleration
  • Per GB of storage allocated per month
  • Provisioned IOPS
  • EBS Snapshots

While the best fit for EBS  is as a File System and it offers an option for backup using snapshot, the S3 offers some additional features. Those features include, versioning (for Backup & DR), lifecycle policies (archiving, deletion), static website hosting, and requester pays bucket (for cost optimization), to name just a few. The Amazon S3 transfer, event notification, and cross-region replication capabilities are also worth noting.

Read more…

Business Challenge:We've curated the most common business challenges Selecting the right storage format for business needs
Stage:We've split the research process into 3 tasks Explore Solutions and Requirements Building

Latest Additions

Get our Newsletter

Curated research and insights straight into your inbox.

(twice monthly)

We will collect, use and protect your data in accordance with our privacy policy