Phase 1: Cloud Fundamentals

Block storage (EBS, Persistent Disks) & IOPS tuning

Beginner ~3 min read
Think of it this way A friendly analogy. Read this if the technical version feels dense. Show Hide

Imagine a super busy restaurant kitchen. Not just any kitchen, but a cloud kitchen, where lots of amazing chefs (we call them "servers" in the computer world) are cooking up all sorts of delicious digital meals for people. Now, each chef needs ingredients, right? Things like fresh veggies, sauces, and recipe books. Block storage is like having your very own mini-fridge and pantry built right into your cooking station. It's packed with all the special ingredients and recipes you need, and nobody else can take from it or put things in it. This personal pantry is super important because it's always right there, ready for you, much like a super fast hard drive connected directly to your computer.

Why is having your own dedicated pantry so great? Well, if you had to share one giant pantry with everyone in the kitchen, you might have to wait in line or walk across the whole room just to get a tomato or a recipe card. That would slow you down! With your own block storage pantry, everything is instant. You can quickly grab exactly what you need to cook a new app, store important customer information, or even build a huge game world, all without any delays from other chefs. It's like having the fastest, most organized personal workspace for your digital projects.

Now, let's talk about how speedy your personal pantry is. We measure this with something called IOPS, which stands for Input/Output Operations Per Second. Think of it as how many times per second you, the chef, can open your fridge, grab an ingredient, maybe chop it, and put something else back. If you're making a simple snack, you might not need super high IOPS. But what if you're making a thousand fancy dishes every minute for a huge party? You'd need to be incredibly quick, grabbing and putting back hundreds or even thousands of ingredients every single second! That's high IOPS.

So, when you're building a massive online store, or a game that thousands of players are enjoying all at once, your digital chef (server) needs to get and save information super fast, just like our busy chef needs to move ingredients quickly. By choosing block storage with high IOPS, you're giving your digital projects the power to handle tons of activity smoothly, making sure everything runs without a hitch and your users have the best experience. It means you can build incredibly responsive and powerful applications that can keep up with huge demands.

Block storage is a fundamental concept in cloud computing, acting much like a traditional hard drive or SSD directly attached to a single server. In cloud environments, these are virtual disks often referred to as AWS Elastic Block Store (EBS) or Google Persistent Disks. When you launch a virtual machine (like an EC2 instance in AWS or a Compute Engine VM in GCP), you typically attach one or more block storage volumes to it. This storage is unformatted, raw disk space, ready for you to install an operating system, databases, or application files that require high performance and low-latency access. Unlike network-attached file shares, block storage is dedicated to a single compute instance at a time, making it ideal for workloads that need direct, fast disk access.

The performance of block storage is critical for many applications, and a key metric to understand is IOPS (Input/Output Operations Per Second). IOPS measures how many read and write requests a storage volume can handle each second. Imagine a database: every time it retrieves or stores a piece of data, that's an I/O operation. A database processing thousands of transactions per second needs storage with very high IOPS. Cloud providers offer different types of block storage, each with varying IOPS capabilities and costs. For example, SSD-backed volumes generally offer much higher IOPS than HDD-backed volumes, making them suitable for databases and transactional systems, while HDDs might be sufficient for less demanding tasks like log storage or backups.

"IOPS tuning" refers to the process of selecting and configuring your block storage to meet the specific performance demands of your application. Cloud platforms allow you to provision block storage with certain characteristics. With types like AWS EBS gp3 or io2, you can explicitly specify the desired IOPS and throughput (how much data can be transferred per second, measured in MB/s), rather than having it scale automatically with volume size as in older types like gp2. This independent tuning means you can provision a smaller volume for storage capacity while still getting high performance, optimizing costs. Monitoring your application's disk usage patterns and performance metrics (like latency and queue depth) is essential to determine if you need to adjust your block storage type or provisioned IOPS to avoid performance bottlenecks or unnecessary costs.

Key Takeaways

  • Block storage (EBS, Persistent Disks) provides dedicated, high-performance disk space directly attached to a single virtual server.
  • IOPS (Input/Output Operations Per Second) is a crucial metric measuring how many read/write operations your storage can handle per second, directly impacting application performance.
  • Cloud providers offer different block storage types (e.g., SSD vs. HDD) with varying IOPS capabilities and costs, allowing you to choose based on your workload.
  • IOPS tuning involves selecting the right block storage type and provisioning specific IOPS/throughput to meet application demands efficiently, balancing cost and performance.

Code Example

bash
aws ec2 create-volume \
    --availability-zone us-east-1a \
    --size 100 \
    --volume-type gp3 \
    --iops 3000 \
    --throughput 125 \
    --tag-specifications 'ResourceType=volume,Tags=[{Key=Name,Value=MyHighPerformanceDataVolume}]'

How this code works

This code creates a new block storage volume in AWS, specifically an EBS volume, designed for high performance. It's essentially like provisioning a dedicated virtual hard drive that can be attached to a virtual server (an EC2 instance). The aws ec2 create-volume command initiates this process. The availability-zone specifies the physical location within a region where the volume will reside, which is crucial for attaching it to an EC2 instance located in the same zone. The size parameter sets the storage capacity to 100 GB for this volume.

The volume-type gp3 is chosen because it allows for independent tuning of performance characteristics. This command explicitly sets the iops (input/output operations per second) to 3000 and throughput to 125 MB/s. A subtle but important aspect for beginners is understanding that for gp3, iops and throughput can be specified separately, regardless of the volume's size. This differs significantly from older volume types, where iops often scaled automatically with the volume's size, meaning a larger volume was needed to achieve higher iops. Finally, tag-specifications adds a human-readable name, MyHighPerformanceDataVolume, making the volume easier to identify and manage within the AWS console.

Related Articles

Hand-picked reads to go deeper on storage types.

Back to Cloud Architect Roadmap