The computing world has changed dramatically in recent years. The widespread adoption of mobile and web services has created an explosion of unstructured data. Massive amounts of pictures, music, and video files are created and consumed all over the world on a daily basis. This content needs to be distributed across geographies and its availability needs to be maintained. Although file and block storage systems have been used for decades, this explosion of unstructured data, the proliferation of globally distributed cloud based architectures, and the need for a more simplistic and low cost storage solutions, gave rise to a new type of storage called object storage. In this blog post we will discuss the differences between file storage, block storage, and object storage in order to understand when and how you should use this new form of storage.
Block Storage Characteristics
Block storage solutions have been around for many years and are considered “tried and tested” solutions that offer high performance, security and availability. Block storage systems manage the data in blocks, which are stored on disks and accessed via low-level storage protocols, such as SCSI commands. When appropriate blocks are combined, it creates a file. The direct access to the data reduces the overhead by minimizing abstraction layers. Higher level tasks such as multi-user access, sharing, locking and security are usually handled by the operating system. There is no storage-side metadata associated with the block, except for the address, and even that, arguably, is not metadata about the block. In other words, the block is simply a chunk of data that has no description, no association and no owner. Block storage is considered as the best solution for performance sensitive, transactional, and data-base oriented applications. As such, it is mostly used locally inside a local network. Adding any distance between the application and storage would severely harm its performance.
File System Characteristics
Files in file systems are organized in folders as a hierarchy of directories, subdirectories and files that use naming conventions based on characteristics such as extensions, categories or applications. The file system stores files using relatively simple metadata, such as file name, creation date, creator, file type, most recent change and last access. When the number of files is relatively small, locating a file is manageable However, as the number of files grows into the billions, it becomes much more difficult. File system scalability is generally supported by scale-out NAS systems that, similar to object storage, scales horizontally by adding nodes. But because they’re based on hierarchical file structures with a limited namespace, they’re more restricted than the nearly infinitely scalable flat structure of object storage systems.
What is Object Storage?
Object storage manages data as objects, Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. Object storage can be implemented at multiple levels, including the device level (object storage device), the system level, and the interface level.
Highly granular metadata
Unlike a file system, object storage stored files as objects in different locations, and each object has a unique identifier and a large amount of metadata. Although the amount of metadata varies, it is significantly greater than the metadata that is managed in file systems. Object metadata frequently includes a summary of the content in the file, key words, key points, comments, locations of associated objects, data protection policies, security, access, geographic locations and more. This means that unlike block storage and file systems, object storage protects, manages, manipulates and keep objects on a much finer level of granularity. An object is not limited to any type or amount of metadata (although S3 limits user metadata to 2KB).You can assign metadata such as the application association, the importance of an application, the data protection level that you want to assign to an object, replication instructions to another site or sites, when to move this object to a different tier of storage or to a different geography, and when to delete this object. This type of metadata goes way beyond the access control lists used in file systems and of course well beyond block storage metadata.
