What Happens When You Create a Pod in Kubernetes?

So, like when you type: kubectl apply -f pod.yaml A second later, Kubernetes might respond with: pod/nginx created It is tempting to think that Kubernetes simply read your YAML and immediately started an NGINX container somewhere in the cluster. That's not what happened. Behind that single command, several Kubernetes components have already started working together. The API server receives your request, authenticates you, checks your permissions, runs admission logic, and stores the desired state. The scheduler then finds a suitable node, the kubelet on that node takes responsibility for the Pod, and the container runtime eventually creates the actual container. For the CKA, this sequence is worth understanding because Kubernetes troubleshooting becomes much easier once you know which component is responsible for which part of the journey. If a Pod is Pending, you should be thinking about scheduling. If it is stuck at ContainerCreating, you start looking toward the kubelet, runtime, networking, or storage. If it is in ImagePullBackOff, you don't waste time debugging the scheduler because scheduling has already happened. Let's walk through the entire journey as if we're standing in front of a whiteboard. The Story Begins: kubectl to kube-apiserver Let's start with a simple Pod definition: apiVersion: v1 kind: Pod metadata: name: nginx spec: containers: - name: nginx image: nginx:1.27 ports: - containerPort: 80 Now you run: kubectl apply -f pod.yaml he first thing to understand is that kubectl does not create the container. It doesn't connect directly to containerd, it doesn't SSH into a worker node, and it doesn't write anything directly into etcd. kubectl is primarily a client for the Kubernetes API. It takes the configuration you provide, constructs an API request, and sends that request to the kube-apiserver. You can think of the API server as the front door of the Kubernetes cluster. Almost every major Kubernetes component communicates through this API: users, controllers, the scheduler, kubelets, and many other components. So when you execute kubectl apply, your request first enters the control plane through the API server. At this point, Kubernetes hasn't created a container and hasn't even selected a worker node. You've simply submitted a request saying, "This is the state I want Kubernetes to maintain." The journey starts roughly like this: kubectl | v kube-apiserver From here, the API server has several questions to answer before it accepts your request. Step 1: Authentication — "Who Are You?" Before Kubernetes can decide what you're allowed to do, it needs to know who is making the request. This is the job of authentication. When you use kubectl, your client configuration normally contains credentials or instructions for obtaining credentials. Depending on the cluster, Kubernetes might use client certificates, bearer tokens, OIDC, cloud-provider identity mechanisms, or another authentication method. You can inspect your current Kubernetes configuration with: kubectl config view The important thing for the CKA is to keep authentication and authorization separate in your head. Authentication establishes your identity; it doesn't determine what you're allowed to do. If Kubernetes determines that the request came from a user called naveen, it has answered the first question: Who are you? It still needs to answer the second question: Are you allowed to create this Pod? Step 2: Authorization — Are You Allowed to Do This? Now the API server checks authorization. Kubernetes needs to determine whether the authenticated identity has permission to perform the requested operation on the requested resource in the requested namespace. In most Kubernetes environments, this is handled using RBAC — Role-Based Access Control. For example, a user might be allowed to read Pods: get pods *list pods watch pods but not create or delete them: create pods delete pods You can test permissions directly with kubectl: kubectl auth can-i create pods You can also check against a specific namespace: kubectl auth can-i create pods -n default If Kubernetes responds with: yes the request can continue. If it responds with: no the API server rejects the operation. This is one of those small distinctions that frequently appears in interviews and CKA questions: authentication answers Who are you? while authorization answers What are you allowed to do? Step 3: Admission — Should We Allow This Request? Passing authentication and authorization doesn't automatically mean the Pod will be accepted. The request now goes through the admission control stage. Admission controllers get an opportunity to inspect the API request after authentication and authorization but before the object is persisted. There are two important concepts here: mutating admission and validating admission. A mutating admission controller can modify the object before it is stored. For example, an admission webhook might inject configuration, add labels, modify security settings, or inject a sidecar container. A validating admission controller doesn't modify the object; instead, it checks whether the object satisfies a particular policy and can reject it if it doesn't. Custom admission webhooks are particularly common in production Kubernetes environments. A MutatingAdmissionWebhook might modify your Pod, while a ValidatingAdmissionWebhook might reject it because the image comes from an unapproved registry or because the workload violates an organizational security policy. The request now looks something like this: kubectl | v kube-apiserver | +--> Authentication | +--> Authorization / RBAC | +--> Admission | +--> Mutating | +--> Validating | v accepted This stage is also useful when troubleshooting. If kubectl apply immediately returns an error saying that the request was forbidden or rejected by a policy, you don't need to start inspecting kubelet logs on worker nodes. The request never got that far. The State Is Written: kube-apiserver to etcd Assume our Pod passes authentication, authorization, admission, and API validation. The API server now has an accepted Pod object that needs to become part of the cluster's state. This is where etcd enters the picture. etcd is the distributed key-value store used by Kubernetes to persist cluster state. For our Pod, the API server stores the Pod's desired configuration there. But notice something important: the Pod has been accepted and stored, but it is not necessarily running yet. At this point Kubernetes knows what you want, but nobody has necessarily decided where the Pod should run. You can think of etcd as the cluster's persistent source of truth. If Kubernetes needs to know about an object such as a Pod, Deployment, Service, ConfigMap, Secret, or many other API resources, the API server manages that state. The Pod object can exist in the cluster even while there is no running container behind it. This distinction is fundamental to understanding Kubernetes: creating the Kubernetes object and starting the workload are different events. Conceptually, the state currently looks like this: Pod: nginx Image: nginx:1.27 Desired state: exists Node: not assigned yet Container: not running yet The Pod is now waiting for the next component in the chain. Why Doesn't Everything Talk Directly to etcd? One important Kubernetes architecture rule is that normal Kubernetes components don't simply connect directly to etcd whenever they want cluster information. The kube-apiserver acts as the central API gateway between Kubernetes components and the cluster's persistent state. The simplified communication pattern looks like this: kube-scheduler ---> kube-apiserver ---> etcd kubelet ---------> kube-apiserver kubectl ----------> kube-apiserver controllers ------> kube-apiserver Instead of every component having its own direct database connection, the API server provides a consistent interface for reading and modifying Kubernetes objects. This also gives Kubernetes a central place for authentication, authorization, admission, validation, auditing, concurrency handling, and API semantics. For the CKA, remember this simple mental model: etcd stores the cluster state. The API server is the gateway to that state. That single distinction prevents a lot of architectural conf The Matchmaker: kube-scheduler Finds a Node Our Pod now exists in the cluster, but it still has a problem: where should it run? This is the responsibility of the kube-scheduler. The scheduler watches the Kubernetes API for newly created Pods that don't yet have a node assigned. Kubernetes is largely event-driven. The scheduler doesn't repeatedly ask the API server, "Do you have a Pod for me?" Instead, it maintains a watch and reacts when relevant objects change. When our nginx Pod appears without a node assignment, the scheduler sees it and begins the scheduling process. The flow now looks roughly like: Pod created | v kube-apiserver | v etcd | v kube-scheduler | v Find a suitable node The scheduler's job is not to start the container. Its job is to make a scheduling decision. Step 4: Filtering — Which Nodes Can Run the Pod? Imagine our cluster contains three worker nodes: worker-1 worker-2 worker-3 The scheduler can't simply choose the first node it sees. It evaluates the Pod's requirements against the available nodes. The first major phase is filtering, where Kubernetes eliminates nodes that cannot satisfy the Pod's requirements. A node might be filtered out because it doesn't have enough CPU or memory, because the Pod's nodeSelector doesn't match, because node affinity rules don't match, because a taint isn't tolerated, or because topology and other scheduling constraints cannot be satisfied. For example: worker-1 Not enough memory ❌ worker-2 Taint doesn't match ❌ worker-3 Suitable ✅ This gives us a useful CKA mental model: Filtering = eliminate unsuitable nodes Scoring = rank suitable nodes You may also encounter older Kubernetes documentation that uses terms such as predicates and priorities. Modern Kubernetes scheduling uses the scheduling framework with filtering and scoring plugins, but understanding the older terminology is still useful when reading older tutorials or answering interview questions. Step 5: Scoring — Which Suitable Node Is Better? What happens if several nodes pass the filtering stage? Kubernetes still needs to choose the best candidate. That's where scoring comes in. The scheduler evaluates the remaining nodes using its scheduling plugins and gives them scores based on the scheduling rules and preferences. You can influence scheduling with mechanisms such as nodeSelector, node affinity, taints and tolerations, topology constraints, and other scheduling configuration. For example: spec: nodeSelector: disktype: ssd Now the scheduler knows that the Pod should only be considered for nodes carrying the disktype=ssd label. You can inspect node labels with: kubectl get nodes --show-labels And when a Pod is stuck in Pending, one of your first troubleshooting commands should be: kubectl describe pod nginx Pay particular attention to the Events section. Kubernetes will often tell you exactly why a Pod couldn't be scheduled, such as insufficient CPU, an untolerated taint, an affinity mismatch, or another scheduling constraint. The Scheduler Doesn't Start the Container This is a point worth emphasizing because it is a common misunderstanding when people are new to Kubernetes. The scheduler doesn't SSH into a worker node, execute docker run, or directly invoke containerd. It makes a decision and records that decision through the Kubernetes API. Conceptually, the scheduler says: This Pod should run on worker-3. It then updates the Pod's assignment through the API server: kube-scheduler | | Bind Pod to worker-3 v kube-apiserver | v Pod object updated Now you can see the assigned node with: kubectl get pod nginx -o wide You might see: NAME READY STATUS RESTARTS AGE IP NODE nginx 1/1 Running 0 30s 10.244.1.10 worker-3 The scheduler has completed its part of the story. The responsibility now moves to the worker node. Handing Off to the Node: The kubelet Takes Over Every Kubernetes worker node runs a kubelet. If the scheduler is the component deciding where the Pod belongs, the kubelet is the component on the node that works to make that decision become reality. The kubelet watches the API server for Pods assigned to its node. When it sees that nginx has been assigned to worker-3, it begins reconciling the desired state with the actual state of the node. The desired state says, "There should be an nginx Pod running here." The actual state says, "There isn't one yet." The kubelet's job is to close that gap. This reconciliation model is one of the most important ideas in Kubernetes: Desired state: nginx Pod should be running Actual state: nginx Pod doesn't exist ↓ kubelet reconciles ↓ nginx Pod gets created This is why Kubernetes isn't simply a collection of commands that run once. Components continuously watch state and take action when reality doesn't match the desired configuration. The Engine Room: CRI and containerd The kubelet doesn't normally implement all of the low-level container operations itself. Instead, it communicates with the container runtime through the Container Runtime Interface (CRI). A common runtime you'll encounter is containerd. The simplified relationship is: kubelet | | CRI v containerd | v container runtime | v containers The kubelet asks the runtime to create the necessary Pod sandbox and containers. The runtime is then responsible for the lower-level container lifecycle operations. In a CKA environment, understanding this boundary is useful because it tells you where to look when Kubernetes knows about a Pod but the runtime isn't successfully creating the containers. On a node where you have access, you can inspect the CRI runtime using commands such as: crictl info You can list running and stopped containers with: crictl ps -a And you can inspect Pod sandboxes with: crictl pods If something looks wrong at the node level, these commands can reveal information that isn't always obvious from kubectl get pods. CNI Enters the Picture: Giving the Pod a Network A container also needs networking, and this is where CNI — Container Network Interface comes into the picture. Kubernetes itself defines the networking expectations, but the actual networking implementation is provided by a CNI plugin. Depending on your cluster, that could be Cilium, Calico, or another networking implementation. The simplified flow looks like this: kubelet | v container runtime | v CNI plugin | +--> create/configure network namespace | +--> assign Pod IP | +--> create network interfaces | +--> connect Pod to node network A common Linux networking pattern uses a virtual Ethernet pair: Pod network namespace | veth | veth | Node network namespace The exact implementation depends on the CNI plugin, but the general idea remains the same: the Pod needs its network namespace configured, an IP address assigned, and connectivity established with the rest of the cluster. This is why a Pod can successfully pass scheduling and still have networking problems. If the CNI layer is broken, the Pod might be assigned to a node but fail during network setup. For troubleshooting, remember that the Pod exists does not automatically mean the Pod has working networking. The Engine Starts: Image Pulling, Storage, and Containers Now the runtime needs the image requested by our Pod: nginx:1.27 If the image isn't already available on the node, the runtime pulls it from the configured container registry. The simplified process looks like: containerd | v Container Registry | v nginx:1.27 If the image is successfully pulled, container creation can continue. If the image cannot be pulled because the name is wrong, the registry is unavailable, credentials are missing, or networking is broken, the Pod may enter states such as: ErrImagePull or: ImagePullBackOff This is where kubectl describe becomes extremely useful: kubectl describe pod nginx Look at the Events section. You might see: Failed to pull image nginx:1.27 or: Back-off pulling image nginx:1.27 At this point, you know that scheduling probably isn't your problem. The scheduler already selected the node. Your investigation has moved further down the chain toward the kubelet and container runtime. Where Storage Fits: CSI Now suppose the Pod also requests persistent storage. Kubernetes may need to work with CSI — Container Storage Interface components to make that storage available to the Pod. The simplified relationship looks like: kubelet | v CSI components | v Storage system Depending on the storage configuration, Kubernetes may need to identify the volume, attach it to the node, mount it, and make the resulting filesystem available to the Pod. This gives you another useful troubleshooting boundary. A Pod can be successfully scheduled to a node but still fail to start because its required volume cannot be attached or mounted. When that happens, you might investigate the Pod's events, PVC/PV status, StorageClass configuration, and CSI components. Again, don't treat: Scheduled as equivalent to: Running Scheduling only tells you that Kubernetes has selected a node. There is still plenty of work left before the application can actually start. Init Containers Run Before the Application If the Pod contains init containers, those containers must complete successfully before the main application containers are started. For example: spec: initContainers: - name: setup image: busybox command: - sh - -c - echo "Preparing..." containers: - name: nginx image: nginx:1.27 The simplified sequence becomes: Pod sandbox | v Init container | v Init container completes | v Main application container starts If the init container fails repeatedly, the main application container doesn't move forward normally. This can make a Pod appear stuck even though the image and main application configuration look perfectly fine. You can inspect the Pod: kubectl describe pod nginx And view logs from a specific init container: kubectl logs nginx -c setup When troubleshooting a Pod that isn't progressing, always check whether init containers are part of the Pod specification. Finally: The Main Application Container Starts After the image is available, networking has been configured, required storage has been prepared, and any init containers have completed, the container runtime can finally start the main application container. At this point, the journey that started with one kubectl command has crossed almost the entire Kubernetes architecture: kubectl | v kube-apiserver | +--> Authentication | +--> Authorization / RBAC | +--> Admission | v etcd | v kube-scheduler | +--> Filter | +--> Score | +--> Bind | v kube-apiserver | v kubelet | v CRI | v containerd | +--> Pull image | +--> Configure networking through CNI | +--> Prepare storage through CSI | +--> Run init containers | v Main application container The application process is now running inside its container. Kubernetes may report the Pod as: Running But there's still another distinction worth understanding: a running container isn't necessarily a ready application. Running Doesn't Always Mean Ready Imagine your application starts its process successfully but needs another 20 seconds to initialize. The container itself is technically running, but the application isn't ready to accept traffic yet. That's where a readiness probe becomes useful: readinessProbe: httpGet: path: / port: 80 Kubernetes can use the result of that probe to determine whether the application should receive traffic through a Service. A liveness probe answers a different question. It helps Kubernetes determine whether the application is still functioning and whether the container should be restarted. So you should remember another important distinction: Container Running ≠ Application Ready This becomes especially important when you're debugging Deployments and Services. You can have Pods that are technically running but still have zero ready endpoints because their readiness checks are failing. The Complete Journey on One Whiteboard If I were explaining this during a CKA study session, this is the diagram I'd want on the board: CONTROL PLANE ┌──────────────────────────────────────────────┐ │ │ │ kubectl │ │ │ │ │ ▼ │ │ kube-apiserver │ │ │ │ │ ├── Authentication │ │ ├── Authorization / RBAC │ │ ├── Admission Controllers / Webhooks │ │ │ │ │ ▼ │ │ etcd │ │ ▲ │ │ │ │ │ kube-scheduler │ │ │ │ │ ├── Filter nodes │ │ ├── Score nodes │ │ └── Bind Pod │ │ │ └──────────────────────┬───────────────────────┘ │ │ API ▼ WORKER NODE ┌──────────────────────────────────────────────┐ │ │ │ kubelet │ │ │ │ │ ▼ │ │ CRI │ │ │ │ │ ▼ │ │ containerd │ │ │ │ │ ├── Pull image │ │ ├── Create Pod sandbox │ │ ├── Start init containers │ │ └── Start application container │ │ │ │ ├──────────► CNI │ │ │ └── Pod networking │ │ │ │ │ └──────────► CSI │ │ └── Storage │ │ │ └──────────────────────────────────────────────┘ The important thing to notice is that there isn't one Kubernetes component sitting in the middle doing everything. The API server handles the API interaction and persistence path, the scheduler makes the placement decision, and the kubelet takes responsibility for making the Pod actually exist on the selected node. The container runtime handles container lifecycle operations, while CNI and CSI handle networking and storage concerns. Once you understand those boundaries, Kubernetes becomes much easier to reason about. Instead of asking, Why isn't my Pod working?, you can ask a much better question: Which stage of the Pod lifecycle has failed? Mentor's Closing: Follow the Handoff This is the mental model I want you to carry into the CKA exam and into production troubleshooting. When a Pod fails, don't immediately start throwing random kubectl commands at it. Walk through the same path that the Pod walked during creation and identify the first stage where reality diverged from the expected state. If kubectl apply itself fails, investigate the API layer first: authentication, RBAC authorization, admission, validation, or resource-related restrictions. If the Pod object exists but remains Pending, investigate scheduling using kubectl describe pod and look at the Events section. If the Pod has a node but remains stuck in ContainerCreating, start thinking about the kubelet, CRI, container runtime, CNI networking, image pulling, or volume mounting. If you see ImagePullBackOff, think about the image name, registry access, credentials, or network connectivity. If you see CrashLoopBackOff, the container is starting and then repeatedly exiting, so inspect the application logs and previous container logs: kubectl logs kubectl logs --previous If networking isn't working, investigate the CNI, DNS, NetworkPolicies, Services, or the application itself. If storage isn't mounting, investigate the PVC, PV, StorageClass, CSI components, and volume events. And if the Pod is running but not receiving traffic, check readiness probes and the Service's endpoints. The entire lifecycle can therefore be reduced to one mental model: kubectl ↓ API Server ↓ Authentication ↓ Authorization / RBAC ↓ Admission ↓ etcd ↓ Scheduler ↓ Filter + Score ↓ Bind to Node ↓ kubelet ↓ CRI / containerd ↓ CNI + CSI ↓ Init Containers ↓ Application Container ↓ Readiness ↓ Running + Ready Once this flow becomes second nature, Kubernetes troubleshooting stops feeling like a collection of unrelated commands. Every command has a purpose because you know which component you're investigating and what should have happened before you got there. That's the real advantage of understanding Kubernetes internals. When a Pod gets stuck, you're no longer asking What command should I try next? You're asking the much more useful question: Which handoff failed? And that is the mental model that makes you considerably better at both the CKA exam and real-world Kubernetes troubleshooting.