Object Storage — S3 vs GCS vs Azure Blob

Object storage is the cloud's answer to "I need to store files at arbitrary scale." Unlike a file system with directories and hierarchies, object storage is flat: every object lives in a bucket (or container) identified by a key. You can't append to an object or modify it in place — you replace the whole thing. In exchange, you get practically unlimited capacity, extreme durability (11 nines on most providers), and an HTTP API that works from anywhere. If you've read the Blob Storage system design lesson, this is where that architecture meets the real products.

Storage Classes and Lifecycle Policies

Not all data is accessed equally. Providers offer storage classes (or tiers) at different price points based on access frequency.

Hot storage (S3 Standard, GCS Standard, Azure Hot) is for frequently accessed data. Highest storage cost, lowest retrieval cost. Warm tiers (S3 Infrequent Access, GCS Nearline, Azure Cool) cut storage costs 40-50% but charge per-retrieval fees and have minimum storage durations (30 days). Cold/archive tiers (S3 Glacier, GCS Archive, Azure Archive) are dirt cheap for storage but retrieval takes minutes to hours and costs significantly more.

Lifecycle policies automatically transition objects between tiers based on age. A common pattern: keep objects in Standard for 30 days, move to Infrequent Access at 30 days, move to Glacier at 90 days, delete at 365 days. Set this up early — retroactively tiering terabytes of data is painful.

Consistency, Durability, and Availability

All three major providers now offer strong read-after-write consistency. When a PUT returns success, any subsequent GET will return the new data. This wasn't always the case — S3 only achieved this in December 2020. Before that, you'd occasionally read stale data, which caused extremely subtle bugs.

Durability (will my data survive?) is 99.999999999% (11 nines) for all three. This means if you store 10 million objects, you can statistically expect to lose one every 10,000 years. Availability (can I access my data right now?) is lower — typically 99.9% to 99.99% depending on the tier. Durability is about replication across hardware; availability is about the service being up.

Versioning and Immutability

Versioning keeps every version of every object. Delete an object and it gets a delete marker — the data is still there. This protects against accidental deletion and overwrites, but your storage costs grow because you're keeping every version. Combine with lifecycle policies to expire old versions.

Object lock / WORM (Write Once Read Many) prevents objects from being deleted or overwritten for a retention period. Required for regulatory compliance in finance and healthcare. Azure calls this immutable blob storage. GCS has retention policies and bucket lock.

Soft delete (available on GCS and Azure, S3 has it as of 2024) adds a recovery window for deleted objects even without versioning enabled.

Security: The Public Bucket Problem

Misconfigured public buckets are the single most famous class of cloud breach. Capital One, Twitch, US military, and dozens of companies have leaked data through S3 buckets that were accidentally public. The root cause is usually a combination of overly permissive bucket policies, legacy ACLs, and no guardrails.

S3 Block Public Access is an account-level and bucket-level setting that overrides any policy or ACL that would make a bucket public. Enable it account-wide and only disable it on specific buckets that genuinely need to be public (like static website hosting).

The access control model has layers: IAM policies (who can call S3 APIs), bucket policies (resource-based, attached to the bucket), and ACLs (legacy, per-object). Best practice: use IAM policies and bucket policies. Disable ACLs entirely — they're a legacy mechanism that creates confusion.

Pre-signed URLs grant time-limited access to a specific object without making the bucket public. Your backend generates a URL signed with its credentials, and the client uses that URL to upload or download directly. This is how you handle user file uploads without proxying through your servers.

Encryption at rest is available (and often default) on all providers. S3 uses SSE-S3 (provider-managed keys) by default; you can upgrade to SSE-KMS (you control the key in KMS) for audit trails and key rotation control. GCS and Azure offer similar options.

AWS Implementation (S3)

S3 is the original and most feature-rich object storage service. Bucket names are globally unique. Key features: S3 Event Notifications (trigger Lambda on upload), S3 Select (query CSV/JSON in place), Transfer Acceleration (upload via CloudFront edge locations), and multi-part uploads for large files.

# Create a bucket with versioning and block public access
aws s3api create-bucket --bucket my-app-data --region us-east-1
aws s3api put-bucket-versioning --bucket my-app-data \
  --versioning-configuration Status=Enabled
aws s3api put-public-access-block --bucket my-app-data \
  --public-access-block-configuration \
  BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

GCP Implementation (GCS)

GCS integrates tightly with BigQuery (query GCS data directly) and Pub/Sub (event notifications). GCS uses a simpler permission model — uniform bucket-level access instead of per-object ACLs. Dual-region and multi-region buckets provide geo-redundancy without manual replication setup.

Azure Implementation (Blob Storage)

Azure Blob Storage organizes objects into containers within a storage account. It supports three blob types: block blobs (most common), append blobs (for logs), and page blobs (for VM disks). Azure's immutable storage has legal hold and time-based retention policies for compliance.

Side-by-Side Comparison

Aspect	AWS (S3)	GCP (GCS)	Azure (Blob Storage)
Hot tier	Standard	Standard	Hot
Warm tier	Standard-IA, One Zone-IA	Nearline (30-day min)	Cool (30-day min)
Archive tier	Glacier, Deep Archive	Coldline, Archive	Archive
Consistency	Strong read-after-write	Strong	Strong
Durability	11 nines	11 nines	11 nines (LRS: 11, GRS: 16)
Namespace	Globally unique bucket names	Globally unique bucket names	Account + container
Event triggers	S3 Events, EventBridge	Pub/Sub notifications	Event Grid
Replication	Cross-Region Replication (manual)	Dual-region / multi-region (built-in)	GRS, GZRS (built-in)

The Egress Trap

Here's the pricing insight that catches everyone: storage is cheap, egress is where the bill comes from. Storing 1 TB in S3 Standard costs roughly $23/month. Transferring that 1 TB out to the internet costs roughly $90. Transfer it out every day and you're looking at $2,700/month in egress alone.

This is by design — cloud providers want data to come in (ingress is free) and stay in. Moving data between services in the same region is free or cheap. Moving it out to the internet or to another provider is expensive. This is the economic lock-in mechanism, and it's more effective than any technical lock-in.

Plan for egress costs. Use CDNs to cache frequently accessed content. Consider CloudFlare R2 (S3-compatible, zero egress fees) for egress-heavy workloads.

Key Takeaways

• Object storage gives you practically unlimited capacity with 11-nines durability — but it's not a file system; you can't modify objects in place
• Lifecycle policies are essential — set them up at bucket creation time to automatically tier data from hot to cold to archive
• Public bucket misconfigurations are the most common cloud security incident; enable Block Public Access account-wide and disable ACLs
• Pre-signed URLs are the right way to let users upload/download without making buckets public or proxying through your backend
• Storage is cheap, egress is expensive — this is the cloud provider's economic lock-in mechanism and the leading cause of surprise bills
• GCP's built-in multi-region buckets and Azure's GRS are simpler than AWS's manual cross-region replication for geo-redundancy

Next, we'll look at managed databases — the service that saves you from running your own Postgres on a VM at 3am.