Horizontal vs Vertical Scaling

Your app is getting more traffic. You have two options: make your existing server more powerful, or add more servers. Each approach has fundamentally different tradeoffs.

Vertical Scaling (Scale Up)

Vertical scaling means replacing your current server with a more powerful one — more CPU cores, more RAM, faster NVMe storage. You are making the machine itself bigger rather than adding more machines. The analogy is upgrading from a bicycle to a sports car: same number of vehicles, dramatically more horsepower.

The appeal is simplicity. Your application code does not change. Your database connection strings do not change. You resize the instance in your cloud console, wait for a reboot, and you are done. This is particularly valuable for stateful systems like databases where distributing the load would require significant architectural changes.

The limitations are real though. Every cloud provider has a ceiling — AWS's largest general-purpose EC2 instance tops out at 192 vCPUs and 1.5 TB of RAM. At some point, no bigger machine exists to buy. Beyond the ceiling, large instances are disproportionately expensive, you still have a single point of failure, and most cloud resizes require downtime or at minimum a reboot window.

Horizontal Scaling (Scale Out)

Horizontal scaling means adding more servers and distributing the load across all of them. Instead of one fast chef trying to cook every meal in your restaurant, you hire ten chefs working in parallel. Each one handles a fraction of the total load.

The advantages are compelling at scale. Theoretically there is no upper limit — you can keep adding instances. At high volumes it is cheaper than an equivalent single machine. When one server crashes, the others keep serving traffic. No single point of failure means the system degrades gracefully rather than going completely dark.

The cost is complexity. You need a load balancer in front of your servers to distribute incoming requests. Your infrastructure management grows — deployments, health checks, and configuration now apply to a fleet of machines. And critically, your application must be designed to run statelessly, which is a hard architectural requirement.

The Stateless Requirement

This is the constraint that catches teams off guard. If your application stores session data in memory on the server — the user's login state, their cart, their in-progress form — horizontal scaling breaks that immediately. A user logs in and their session lands on Server 1. Their next request gets routed to Server 3. Server 3 has no idea who they are. They are logged out.

The solution is to move all state out of the application server's memory and into a shared external store. There are three common approaches:

# Option 1: Centralized session store
User logs in → session saved to Redis → all servers read from Redis

# Option 2: JWT tokens
User logs in → server issues signed JWT → client sends JWT on every request
→ any server can verify and decode it, no shared state needed

# Option 3: Sticky sessions (not recommended)
Load balancer always routes the same user to the same server
→ defeats the purpose of horizontal scaling, creates uneven load

JWT tokens and Redis-backed sessions are the standard patterns. Sticky sessions paper over the problem without solving it and tend to create hot spots under uneven traffic.

Database Scaling

Application servers are relatively easy to scale horizontally because they are, or can be made, stateless. Databases are harder precisely because they are the state.

Vertical scaling is the default first move for databases. Bump the RDS instance from db.r6g.large to db.r6g.4xlarge and you buy yourself a significant amount of headroom with zero application changes. Most systems run on vertically scaled databases for years before hitting the ceiling.

When vertical scaling is no longer enough, horizontal database scaling requires sharding — splitting your data across multiple database instances based on a partition key — or read replicas that offload read traffic from the primary. Both approaches add substantial complexity. That is the subject of the next lesson.

Auto Scaling

The cloud makes horizontal scaling operational rather than manual. You define the behavior once and the platform handles the rest.

AWS Auto Scaling Groups let you define a minimum and maximum number of EC2 instances and a scaling policy. GCP's Managed Instance Groups work on the same principle. When CPU utilization crosses 70% for five minutes, a new instance launches and the load balancer adds it to the pool. When traffic drops at 3am, instances terminate and you stop paying for them.

# Simplified AWS Auto Scaling policy
AutoScalingGroup:
  MinSize: 2
  MaxSize: 20
  DesiredCapacity: 4
  ScalingPolicy:
    MetricType: ASGAverageCPUUtilization
    TargetValue: 70.0

You can also scale on custom metrics — queue depth, requests per second, latency percentiles. This is how large systems absorb sudden spikes without human intervention.

When To Use Each

The short version: scale up first, scale out when you need to.

Vertical scaling is the right starting point for most systems. It is operationally simple, requires no code changes, and handles a lot of growth. It is particularly the right choice for stateful systems — primary databases, caches, and any service that would require significant rearchitecting to distribute.

Horizontal scaling is the right model for stateless application servers, APIs, web servers, and background job workers. Netflix runs tens of thousands of instances of its microservices horizontally across multiple AWS regions. Each service is stateless, so any instance can handle any request. But Netflix still scales its primary data stores vertically first, and turns to sharding only when vertical scaling runs out of road.

The pattern most production systems follow: stateless app servers scale out (horizontal), primary databases scale up (vertical) until they must be sharded.

Key Takeaways

• Vertical scaling means a bigger machine — simple, no code changes, but has a hard ceiling and remains a single point of failure.
• Horizontal scaling means more machines — resilient, cost-effective at scale, but requires stateless application design and a load balancer.
• Stateless design is not optional for horizontal scaling — session state must live in a shared external store like Redis, not in application memory.
• Databases are harder to scale horizontally than application servers and typically get vertical scaling first.
• Cloud auto scaling groups let you define scaling rules once and have the platform add or remove instances automatically based on load.
• The common production pattern is horizontal for stateless app tier, vertical for the database tier until sharding becomes necessary.

In the next lesson we look at database sharding — how to split your data across multiple database instances when a single machine is no longer enough.