Kubernetes — EKS vs GKE vs AKS

Kubernetes is a system for running and managing containerized workloads at scale. You describe what you want (desired state), and Kubernetes continuously works to make reality match. It won the orchestration war decisively — Docker Swarm and Mesos are footnotes now. But Kubernetes carries a real complexity tax. Most teams adopt it before they need it, spend months learning its abstractions, and would have been better served by a managed container platform. That said, when you genuinely need it — complex microservice topologies, stateful workloads, multi-tenant clusters, custom scheduling — nothing else comes close.

The Declarative Model

Kubernetes is built on a declarative reconciliation loop. You submit a manifest saying "I want 3 replicas of this container, with these resource limits, behind this load balancer." The system stores that as desired state in etcd, and controllers continuously compare desired state to actual state, taking corrective action. If a node dies and takes a pod with it, the controller notices the replica count dropped and schedules a replacement. You don't script recovery — the system converges.

Control Plane

The control plane runs the brain of the cluster. API Server is the front door — every interaction (kubectl, CI/CD, controllers) goes through it as REST calls. etcd is the distributed key-value store holding all cluster state. Scheduler assigns unscheduled pods to nodes based on resource requests, affinity rules, and constraints. Controller Manager runs the reconciliation loops — the Deployment controller, the ReplicaSet controller, the Job controller, and dozens more.

In managed offerings, the provider runs the control plane for you. You never SSH into an etcd node or patch the API server. This is the primary value proposition of managed Kubernetes.

Nodes and the Data Plane

Nodes are the machines (VMs or bare metal) that actually run your workloads. Each node runs: kubelet (the agent that receives pod specs from the API server and ensures containers are running), kube-proxy (handles network rules for Service traffic routing), and a container runtime (containerd, typically).

Core Objects

Pod: the smallest deployable unit — one or more containers sharing a network namespace and storage volumes. You almost never create pods directly.

ReplicaSet: ensures N copies of a pod are running. You almost never create these directly either.

Deployment: the object you actually use. It manages ReplicaSets and provides rolling updates, rollbacks, and scaling. When you update the container image, the Deployment creates a new ReplicaSet, scales it up, and scales the old one down.

Service: a stable network endpoint for a set of pods. Pods come and go; the Service's cluster IP stays constant. Types: ClusterIP (internal), NodePort (external via node ports), LoadBalancer (provisions a cloud load balancer).

Ingress: HTTP/HTTPS routing rules — host-based and path-based routing to Services. Requires an Ingress controller (NGINX, Traefik, cloud-native options).

ConfigMap / Secret: configuration data and sensitive values, injected as environment variables or mounted files.

Namespace: logical isolation within a cluster — separate environments, teams, or applications.

StatefulSet: for workloads needing stable network identity and persistent storage (databases, message brokers). Pods get ordered names (mysql-0, mysql-1) and persistent volume claims that survive rescheduling.

DaemonSet: runs one pod per node — log collectors, monitoring agents, network plugins.

Job / CronJob: run-to-completion workloads, one-time or on a schedule.

Scheduling: Requests, Limits, and Why Limits Hurt

Every container should declare requests (guaranteed resources) and limits (maximum resources). The scheduler uses requests to decide which node has room. Limits are enforced at runtime by the kernel.

Here's the nuance most teams learn the hard way: CPU limits cause throttling. If your container hits its CPU limit, the kernel throttles it — your code runs slower even if the node has idle CPU. Many teams now set CPU requests but omit CPU limits entirely, letting containers burst. Memory limits are different — exceeding them triggers an OOMKill, and you want that guardrail.

QoS classes derive from how you set requests and limits: Guaranteed (requests = limits for all resources), Burstable (requests < limits), BestEffort (no requests or limits set). Under memory pressure, Kubernetes evicts BestEffort pods first, then Burstable, then Guaranteed.

resources:
  requests:
    cpu: "250m"     # guaranteed 0.25 vCPU
    memory: "256Mi" # guaranteed 256MB
  limits:
    # cpu: omitted — allow bursting
    memory: "512Mi" # hard ceiling, OOMKilled above this

Autoscaling Three Ways

HPA (Horizontal Pod Autoscaler): scales the number of pod replicas based on CPU, memory, or custom metrics. The most common and well-understood scaling mechanism.

Cluster Autoscaler / Karpenter: scales the number of nodes. Cluster Autoscaler watches for unschedulable pods and adds nodes from predefined node groups. Karpenter (AWS-originated, now CNCF) is smarter — it looks at pending pod requirements and provisions right-sized nodes directly, without predefined groups. Karpenter is faster and more cost-efficient.

VPA (Vertical Pod Autoscaler): adjusts resource requests per pod based on observed usage. Useful for right-sizing but currently requires pod restarts to apply changes, making it tricky for production workloads.

AWS Implementation (EKS)

EKS charges $0.10/hour for the control plane ($73/month) plus your node costs. Nodes can be self-managed EC2, managed node groups (AWS handles updates), or Fargate profiles (serverless — no nodes to manage, per-pod billing). Karpenter originated at AWS and integrates deeply with EC2 instance types, spot instances, and Graviton. EKS supports managed add-ons (CoreDNS, kube-proxy, VPC CNI) and integrates with ALB Ingress Controller for native load balancing.

GCP Implementation (GKE)

GKE is widely considered the best managed Kubernetes offering. The control plane is free for Autopilot clusters and Standard clusters alike. GKE Autopilot is the headline feature — Google manages everything, including nodes. You submit pods, and Autopilot provisions right-sized nodes, enforces security best practices, and bills per pod resource request. Standard mode gives you node pool control for workloads that need GPUs, specific machine types, or custom configurations. GKE's release channels provide automatic control plane and node upgrades.

Azure Implementation (AKS)

AKS doesn't charge for the control plane (free tier) or offers an uptime SLA for a fee. Nodes run as Azure VM Scale Sets. AKS integrates tightly with Azure AD for RBAC, Azure CNI for networking, and Azure Monitor for observability. AKS supports virtual nodes (backed by Container Instances) for burst capacity and KEDA for event-driven autoscaling. The Azure-specific Ingress controller integrates with Application Gateway.

Side-by-Side Comparison

Aspect	EKS (AWS)	GKE (GCP)	AKS (Azure)
Control plane cost	$0.10/hr ($73/mo)	Free	Free (basic), paid SLA tier
Serverless nodes	Fargate profiles	Autopilot	Virtual nodes (ACI)
Node autoscaler	Karpenter / Cluster Autoscaler	GKE Autopilot / Cluster Autoscaler	Cluster Autoscaler / KEDA
Default CNI	VPC CNI (pod IPs from VPC)	Calico / GKE Dataplane V2	Azure CNI / kubenet
Ingress	ALB Controller	GKE Ingress / Gateway API	App Gateway Ingress
Upgrades	Manual or managed	Release channels (auto)	Manual or auto
Best for	AWS-heavy orgs, Karpenter	Best UX, Autopilot, GCP workloads	Azure AD/Microsoft shops

The Honest Take

Kubernetes is powerful, but ask yourself: do I need custom scheduling, service mesh, network policies, or 50+ microservices? If the answer is no, Cloud Run, ECS Fargate, or Container Apps will serve you better with a fraction of the operational overhead. Kubernetes clusters need care — upgrades, security patches, RBAC policies, monitoring, cost optimization. That's a part-time job for a small team and a full-time job for a large one.

Key Takeaways

• Kubernetes continuously reconciles desired state with actual state — you declare what you want, controllers make it happen
• Set CPU requests but consider omitting CPU limits to avoid throttling; always set memory limits to prevent runaway processes
• HPA scales pods, Cluster Autoscaler/Karpenter scales nodes, VPA right-sizes resources — use them together
• GKE Autopilot is the most hands-off managed experience; EKS offers the deepest AWS integration; AKS wins for Microsoft-centric organizations
• Most teams adopt Kubernetes before they need it — managed container platforms handle the majority of workloads with far less operational burden
• If you do adopt Kubernetes, use a managed offering — self-hosting control planes is a full-time job you don't want

Next, we'll explore the other end of the abstraction spectrum — serverless functions, where you don't even think about containers.