Skip to content

Kubernetes API

Assumes: you have already seen a Pod and a Deployment - if not, read Pods and Deployments first.

The Kubernetes API is the control surface of the cluster.

Whether changes come from kubectl, CI/CD, operators, or controllers, they flow through the API server.

The short version

If you only want the beginner-level mental model, it is three sentences:

  • Everything in Kubernetes is an object with a spec (what you want) and a status (what is actually true).
  • You change objects through the API server, usually with kubectl - you never start or stop anything directly.
  • Controllers watch those objects and make status catch up to spec.

That is enough to read manifests and reason about cluster behavior. The rest of this page is how the API server implements that, which matters when you operate clusters or debug rejected requests. Sections marked "Deeper" are safe to skip on a first pass.

API Server and etcd

A core design rule is that etcd is accessed through the API server, not directly by normal components.

  • etcd stores cluster state.
  • API server validates and persists state transitions.
  • Controllers and kubelets watch API resources and react.
graph TD
    User[User or CI] -->|kubectl / API client| API[API Server]
    Controllers[Controllers] <-->|watch / patch| API
    Kubelet[Kubelet] <-->|status / pod updates| API
    API <-->|read and write| Etcd[(etcd)]

Watch and the control loop

Controllers don't poll the API server. They use the Watch mechanism: a long-lived HTTP connection that streams change events (ADDED, MODIFIED, DELETED) as objects are updated.

This is how every controller works - the Deployment controller watches for Deployments, the ReplicaSet controller watches ReplicaSets, the kubelet watches Pod objects assigned to its node. When you kubectl apply, the event reaches all relevant watchers almost instantly via this stream.

The watch mechanism keeps Kubernetes responsive and efficient. The API server buffers events from etcd and fans them out to watchers without each component independently polling.

In practice, controllers use a list-then-watch pattern (wrapped in a client library construct called an informer): list all objects once to build a local cache, then watch for changes to keep it current. If the watch connection drops, the client re-lists and resumes. This is why controllers survive restarts cleanly - their view of the world is rebuilt from the API, not from remembered events.

Deeper: how the API server handles concurrent writes

Optimistic concurrency: resourceVersion

The API server never locks objects. Instead, every object carries a resourceVersion that changes on each write. When two clients update the same object, the second write fails with a 409 Conflict if its resourceVersion is stale - the client is expected to re-read and retry.

This matters in real life:

  • Controllers are written as retry loops, so conflicts are normal and harmless.
  • If you script against the API, handle 409s by re-fetching, not by retrying the same payload.
  • kubectl apply and server-side apply manage this for you; raw kubectl replace can lose races.

The design choice is deliberate: in a system where dozens of controllers write concurrently, optimistic concurrency with retries scales far better than locks.

Object Anatomy: spec and status

Most Kubernetes resources separate intent from observation.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
status:
  replicas: 2
  • spec: desired state declared by users or automation.
  • status: current state reported by controllers.
  • reconciliation: controllers close the gap.

Request Pipeline

A create or update request passes through these stages:

flowchart LR
    REQ[Request] --> AUTHN[Authentication\nwho are you?]
    AUTHN --> AUTHZ[Authorization\nare you allowed?]
    AUTHZ --> MUT[Mutating admission\nchange the object]
    MUT --> VAL2[Schema validation]
    VAL2 --> VALW[Validating admission\naccept or reject]
    VALW --> ETCD[(Persist to etcd)]
    ETCD --> WATCH[Watch events\nfan out]
  1. Authentication: who is calling. Failure returns 401.
  2. Authorization: what they are allowed to do. Failure returns 403.
  3. Mutating admission: the object may be modified before it is stored.
  4. Schema validation: the (possibly mutated) object is checked against its API schema.
  5. Validating admission: the object may be rejected, but not changed.
  6. Persistence: the accepted object is written to etcd, and watch events fan out to listeners.

The practical takeaway for reading errors: a 401 is identity, a 403 is permissions (RBAC), and anything naming a quota or a policy is admission.

Deeper: what runs at each stage
  • Authentication accepts client certificates, bearer tokens, service account JWTs, or OIDC identities, depending on how the cluster is configured.
  • Authorization may run several modes in sequence - RBAC (Role-Based Access Control) is the usual one, alongside Node, Webhook, and the legacy ABAC.
  • Mutating admission is where built-in controllers and mutating webhooks inject sidecars, apply LimitRange defaults, or set a service account.
  • Validating admission is where validating webhooks and built-in controllers like ResourceQuota and PodSecurity reject non-compliant objects.

The ordering answers a common puzzle: why do policy engines see objects that already have defaults filled in? Because mutation completes before validation begins. It also explains a real failure mode - a misbehaving mutating webhook can silently rewrite objects before any validation sees them, which is why webhook failurePolicy and scoping deserve careful review.

API Groups and Versions

Kubernetes APIs are grouped and versioned for compatibility.

  • Alpha (v1alpha1): experimental and can change or be removed.
  • Beta (v1beta1): maturing, but still not guaranteed long-term stable.
  • Stable (v1): production-ready API contract.

Version behavior is governed by the Kubernetes deprecation policy. Do not assume beta APIs are always enabled in every environment.

Common paths:

Path Group Example resources
/api/v1 core Pods, Services, ConfigMaps, Secrets
/apis/apps/v1 apps Deployments, StatefulSets, DaemonSets
/apis/batch/v1 batch Jobs, CronJobs
/apis/networking.k8s.io/v1 networking Ingress, NetworkPolicy

Useful discovery commands:

kubectl api-resources
kubectl api-versions
kubectl explain deployment.spec

Declarative vs Imperative

Imperative commands are useful for quick actions:

kubectl scale deployment web --replicas=5

Declarative workflows are preferred for production:

kubectl apply -f deployment.yaml

Declarative config is repeatable, reviewable, and CI-friendly.

Troubleshooting API Interactions

To inspect client-level API calls:

kubectl get pods -v=6

For server-side behavior, use audit logs and events:

kubectl get events -A --sort-by=.metadata.creationTimestamp

Certification notes

  • CKA scenarios frequently test the request pipeline indirectly: a 403 means authorization (RBAC), a rejection message naming a quota or policy means admission. Reading the error tells you which stage failed.
  • kubectl explain <resource> --recursive is available in the exam and is faster than searching docs for field names.
  • Know the difference between kubectl apply (declarative, three-way merge) and kubectl create/replace (imperative) - CKAD tasks often specify one.

Summary

  • The API server is the authoritative control interface.
  • Kubernetes objects model desired and observed state.
  • Controllers continuously reconcile state via list-and-watch, not polling.
  • Writes use optimistic concurrency (resourceVersion + retry), never locks.
  • API version choice and deprecation awareness are operationally important.
  • Declarative API usage should be the default for production systems.

Check Yourself

A Deployment's spec.replicas says 3 and its status.replicas says 2. Is something broken?

Not necessarily. spec is intent and status is observation, and the gap between them is normal during a rollout, a scale-up, or an image pull. It is only a problem if the gap persists - then kubectl describe and events tell you which stage is stuck, usually scheduling or image pull.

Your kubectl apply fails with 403. Which stage of the request pipeline rejected it, and which stage would a quota rejection come from instead?

403 is authorization - RBAC decided your identity may not perform that verb on that resource. A quota rejection comes later, from validating admission, and the message names the ResourceQuota. Reading which one you got tells you whether to fix permissions or capacity.

A controller pod is killed and rescheduled. Why does it not miss the changes that happened while it was down?

Controllers use list-then-watch: on startup the client lists all relevant objects to rebuild its cache, then watches for changes. Its view of the world comes from the API server's current state, not from remembered events, so a gap in the event stream costs nothing.

Why does a mutating webhook see a different object than a validating webhook does?

Because mutation runs first. Mutating admission can rewrite the object - inject a sidecar, fill in LimitRange defaults - and only then does schema validation and validating admission run against the rewritten version. That is why policy engines see defaults they never set, and why a broken mutating webhook is dangerous.

You are told to use kubectl apply rather than kubectl replace in CI. What concretely goes wrong with replace?

replace sends a whole object built from your local file and can clobber fields another controller wrote in the meantime, or lose the race outright. apply performs a merge that respects fields it does not own, and server-side apply tracks ownership explicitly.

Read first:

Go deeper:

Beginner track - step 11 of 12. Next: Security Primer. Back to the track overview.