Kubernetes Architecture Explained: Pods, Nodes, and Clusters

From the outside, Kubernetes looks simple: write a manifest, apply it, watch your app run. What’s actually happening underneath involves six distinct components making decisions in a specific sequence, a distributed key-value store that is your single point of failure if not replicated correctly, and a scheduler that can place your workloads badly if you give it nothing to work with. Understanding that machinery is the difference between a cluster that runs itself and one that needs constant attention. This guide walks through every layer.

What Is Kubernetes Cluster Architecture?

Kubernetes cluster architecture consists of two layers: a control plane that stores desired state and makes scheduling decisions, and worker nodes that run application containers.

The control plane does not execute workloads. It watches, decides, and corrects. Worker nodes do not make decisions. They execute what the control plane instructs and report back. This separation keeps the system predictable at scale.

The architecture is consistent regardless of where you run: AWS (EKS), Azure (AKS), Google Cloud (GKE), on-premises hardware, or a local development machine. Managed services abstract the control plane, but the underlying model is identical.

Kubernetes Control Plane Components

The control plane is the decision-making layer of every Kubernetes cluster. It has four components.

What Does the Kubernetes API Server Do?

The API Server (kube-apiserver) is the single entry point for all cluster communication, from kubectl, CI/CD pipelines, and internal components. It validates requests, authenticates users, and writes state changes to etcd.

If the API Server goes down, running pods continue because the kubelet operates independently once it has a pod spec. However, all deployment, scaling, and modification operations stop until the API Server recovers.

What Is etcd in Kubernetes?

etcd is a distributed key-value store and the single source of truth for cluster state. It stores pod specs, service configurations, Secrets, ConfigMaps, and node registrations.

Losing etcd without a backup means losing cluster state, not running workloads. Pods already scheduled continue running until they need to be rescheduled, at which point the kubelet cannot reconcile against a missing store. Production setups replicate etcd across three or five nodes to maintain quorum and run regular automated backups.

How Does the Kubernetes Scheduler Work?

The kube-scheduler watches for pods in a Pending state with no node assignment. It evaluates available worker nodes based on resource capacity, affinity and anti-affinity rules, taints, tolerations, and topology spread constraints, then assigns the pod by writing a node binding to etcd via the API Server.

The scheduler makes placement decisions only. Starting the container is the kubelet’s job, not the scheduler’s.

What Is the Kubernetes Controller Manager?

The kube-controller-manager runs background reconciliation loops. Each controller watches one aspect of cluster state and corrects drift from the desired configuration:

• ReplicaSet controller maintains the correct number of pod replicas.
• Node controller monitors node health and marks unresponsive nodes as NotReady.
• Job controller manages batch workloads to completion.

These loops are what make Kubernetes self-healing. No human needs to detect a failed pod; the relevant controller creates a replacement automatically.

Also Read – 7 Common Kubernetes Myths Leaders Should Stop Believing

Kubernetes Control Plane Node vs Worker Node

A control plane node (the term “master node” was deprecated in Kubernetes 1.20) runs the API Server, etcd, scheduler, and controller manager. A worker node runs application workloads.

In production these roles run on dedicated machines. Mixing them causes resource contention: application traffic spikes can starve the control plane of CPU; heavy scheduling cycles can degrade application pods. Separation prevents this.

Worker node failure: affected pods are rescheduled on other available nodes. Running pods on healthy nodes are unaffected.

Control plane node failure without redundancy: all management operations stop, deployments, scaling, and self-healing. Already-running pods continue until they require rescheduling.

Kubernetes Worker Node Components

Every worker node runs three components.

What Is a Kubelet in Kubernetes?

The kubelet is an agent on each node. It watches the API Server for pod specs assigned to its node, instructs the container runtime to start or stop containers, restarts crashed containers per the pod’s restartPolicy, and reports node and pod status back to the control plane.

Resource usage metrics originate at the kubelet. These flow through the metrics-server to power the Horizontal Pod Autoscaler (HPA) and Vertical Pod Autoscaler (VPA), and feed back into the scheduler’s placement decisions.

What Does Kube-Proxy Do?

Kube-proxy manages iptables or IPVS rules on each node to route traffic to the correct pods based on Service definitions. When a Service is created, kube-proxy ensures requests reach a healthy pod regardless of which node it is running on.

Note: modern CNI plugins such as Cilium can replace kube-proxy entirely using eBPF-based networking, offering better performance and observability at scale.

What Is the Container Runtime in Kubernetes?

The container runtime pulls images and starts containers. Kubernetes supports any runtime implementing the Container Runtime Interface (CRI). containerd and CRI-O are the standard choices. Docker Engine was removed as a supported runtime in Kubernetes 1.24 (dockershim removal); Docker-built images remain fully compatible, but the runtime layer is no longer Docker.

The runtime has no awareness of pods, services, or any Kubernetes abstraction. It executes what the kubelet instructs.

Also Read – How Kubernetes Can Be Used for the E-commerce Industry

What Is a Pod in Kubernetes?

A Kubernetes pod is the smallest deployable unit. It wraps one or more containers that share the same network namespace, IP address, and storage volumes. Kubernetes schedules and manages pods, not individual containers.

One container per pod is the standard pattern. Multi-container pods are used when containers must share the same lifecycle and local network, for example, an application container with a sidecar for log forwarding or a service mesh proxy.

Pods are ephemeral. When a pod fails, Kubernetes does not repair it. The controller managing that pod (Deployment, StatefulSet, DaemonSet) creates a replacement, and the scheduler places it on an available node. Bare pods, created without a controller, are not rescheduled if they fail.

Each pod gets its own cluster-internal IP. Containers within the same pod communicate over localhost. Pod-to-pod communication across nodes uses Services.

How Kubernetes Architecture Works: The Request Flow

Here is the exact sequence when you deploy an application to a Kubernetes cluster:

1. You submit a manifest to the API Server via kubectl or a CI/CD pipeline.
2. The API Server validates and authenticates the request, writes desired state to etcd.
3. The Deployment controller detects the new resource and creates a ReplicaSet, which creates pod objects.
4. The scheduler detects unscheduled pods, selects a node, and writes the node binding to etcd via the API Server.
5. The kubelet on the assigned node detects its pod spec and instructs the container runtime to pull the image and start the container.
6. Kube-proxy (or the CNI plugin) updates network rules so Service traffic reaches the new pods.

Controllers continuously reconcile actual state against desired state. Failed pods are replaced. Offline nodes trigger pod rescheduling. The system corrects itself without manual intervention.

Also Read – Kubernetes for SaaS Startups: A Smarter Way to Scale, Secure, and Optimize Costs

Why Architecture Decisions Compound in Production

Early Kubernetes architecture choices determine long-term reliability.

High availability requires three or five control plane nodes with replicated etcd. A single control plane node means one failure stops all cluster management.

Resource planning requires defined requests and limits on every pod. Without them, the scheduler cannot make accurate placement decisions, and node resources are contested unpredictably during traffic spikes.

Scaling strategy ties to how HPA, VPA, and the Cluster Autoscaler interact. HPA scales pod replicas based on metrics. VPA adjusts resource requests based on observed usage. Cluster Autoscaler adds or removes nodes based on pending pods and underutilisation. These three need to be configured to complement each other.

Build Your Kubernetes Architecture with Ksolves

Ksolves delivers Kubernetes engagements across the full stack: cluster architecture design, control plane hardening, multi-cloud and on-premises deployments, workload migration, autoscaling configuration, and ongoing managed operations. Every engagement runs on an AI First, Always approach, every consultant uses AI daily across code generation, configuration review, testing, and documentation. For your project, that translates to delivery in roughly half the standard timeline, a lower total cost of ownership, and fewer issues after go-live. Whether you are starting from scratch or inheriting a cluster that has grown beyond its original design, the Ksolves Kubernetes team has handled that environment before.

Planning a Kubernetes deployment or optimising an existing cluster?

Connect with the Ksolves Kubernetes team