We all love choice in our lives; in many ways it allows us to express our uniqueness, whether that’s in the clothes we wear, the cars we drive or the style of music we like to listen to. Too much choice, however makes decision-making difficult and sometimes it seems easier to do nothing and follow the previous path. In the storage world, block and file protocols have been dominant for many years and are probably the most well known formats for storing and retrieving data. In recent years there has been a new kid on the block (no pun intended) – object storage. So, how do these three protocols differ and how should we use them in the context of cloud and software defined storage solutions? Read on to find out more about choosing your cloud storage protocol.
Block Storage
Block Storage: Block-based storage has been around since the invention of the SCSI protocol and today is still one of the most popular ways to connect internal disks and external storage systems. The “unit of consumption” is the LUN or volume, which as it’s name suggests is logical or virtual in nature, especially in shared storage systems. Data is written to LUNs in fixed sized blocks and provides another name for this storage format – FBA or Fixed Block Architecture. A block can be as small as 4KB or as large as a megabyte and provides a high level of granularity on reads and writes. However, block-based devices have no inherent awareness of data content; they simply read and write data in fixed units, albeit achieving this at high levels of performance and with low latency, especially in systems using NAND flash storage.
File Storage
File Storage: Data is typically stored on a LUN using some kind of structure such as a file system. The file system provides a hierarchical ordering of directories and files that permit data to be accessed in a nice human-readable format. File-based protocols such as NFS and SMB/CIFS extend the idea of the file system across the network, allowing remote hosts to access the data in a hierarchical structured format. They bring additional levels of security (through metadata and other file attributes like ACLs) and data integrity through locking mechanisms that can be distributed across a wide area network. Initially file-based systems were used for storing human readable documents (text, Word files, spreadsheets) but have been extended to be used for virtualisation and databases.
Object Storage
Object Storage: An object store is arguably the purest form of data storage, storing data as binary objects that can range from a few kilobytes to gigabytes in size. There’s typically no hierarchical structure in object stores, with each piece of data referenced by an object ID, a system generated string of characters and numbers that requires some external process to keep track of object IDs and what the content is actually for.
Object stores provide rich (and usually extensible) metadata that can be used to implement efficient searching of stored content. Data is written to and from an object store over proprietary or web-based and RESTful protocols like HTTP, with Amazon Web Services’ S3 API rapidly becoming the de facto standard. The downside for object stores is that they are usually not as high performance as file & block systems, mainly because most assume an object to be immutable (unchangeable) and so any updates require rewriting the object in its entirety.
Cloud Storage Protocols
All three forms of storage are available in public and private cloud deployments. Due to the increased latency incurred from accessing data across the Internet (or private networks), remotely accessing block storage is not a practical solution; attempting to fix the latency issue using techniques such as caching or parallelising the I/O results in increased risks to data integrity. As a result, block-based systems are usually accessed by servers that are themselves cloud-based.
Both file and object protocols work well in the cloud (depending of course on the application). File access can usually tolerate greater latency issues through the use of caching and because most updates are made infrequently. This isn’t the case of course where file-based systems are used for server or desktop virtualisation.
Object stores are great for cloud deployments and work well due to their native HTTP support. To date we’ve seen these platforms used as archives and for backup, primarily because of the performance characteristics of object stores.
Choosing a Storage Strategy
File and object storage can work well when accessed in the public cloud across the network. Both protocols have features to minimise latency and offer advanced security and credentials management that simply don’t exist to the same degree in block-based systems. However, when compute is run from the public cloud (accessing storage in the same location), the practical benefits of block-based access make it once again a viable solution. Block-based storage shouldn’t be discounted when looking at public cloud solutions.
What about SDS?
The great thing about using Software Defined Storage is the ability to abstract the physical resources away from the functional requirements. To that end, SDS can be used to deliver all three forms of storage protocol, leaving the customer to match their application requirements to the best solution.
Today Zadara’s VPSA allows customers to access file and block services as either a public or private cloud solution. In the public cloud, latency is as low as 1-2ms, making block-based deployments a practical solution. In addition, customers receive enterprise-class features, including snapshots, replication and quality of service. Best of all, these features are delivered with hourly billing, allowing the customer to be in full control of their costs.
All of these features are the benefit of being software defined. The future for storage is in the evolution of software and continued abstraction from the hardware.Block
