Health Probes¶
Assumes: you know what a Pod and a Deployment are - see Pods and Deployments.
Probes tell Kubernetes when a container is ready for traffic and when it should be restarted.
Well-designed probes reduce outages during deploys, restarts, and dependency failures.
Probe types¶
- Startup probe: gate for slow-starting applications
- Readiness probe: controls service endpoint inclusion
- Liveness probe: restarts containers that are stuck or unhealthy
The consequences differ sharply, and confusing them causes outages:
| Readiness fails | Liveness fails | |
|---|---|---|
| Action | pod removed from Service endpoints | container restarted |
| Recoverable by itself? | yes - passes again, traffic returns | no - restart is forced |
| Right response to | temporary overload, dependency down | deadlock, unrecoverable state |
Never check dependencies in a liveness probe
The classic self-inflicted outage: a liveness probe that checks a downstream dependency (database, cache). The dependency blips, every replica's liveness probe fails simultaneously, and Kubernetes restarts your entire healthy fleet at once - turning a partial degradation into a full outage, in a loop, until the dependency recovers. Liveness must test only "is this process irrecoverably stuck," never "are my dependencies happy." Dependency awareness belongs in readiness, where the response (stop sending traffic) is proportionate.
How probes interact¶
sequenceDiagram
participant K as kubelet
participant SP as startupProbe
participant RP as readinessProbe
participant LP as livenessProbe
participant SVC as Service endpoint
K->>SP: poll every periodSeconds
SP-->>K: success
Note over SP: startup probe stops
K->>RP: poll every periodSeconds
K->>LP: poll every periodSeconds
RP-->>K: success → pod added to Service
Note over SVC: pod now receiving traffic
LP-->>K: failure × failureThreshold
K->>K: restart container
While the startup probe is active, readiness and liveness probes are paused. This prevents premature restarts during slow initialization.
Recommended usage model¶
- Use startup probes for apps with non-trivial boot time. Give
failureThreshold × periodSecondsenough runway - for a 2-minute boot, usefailureThreshold: 24, periodSeconds: 5(2-minute window). - Use readiness probes for dependency-aware traffic gating. Only report ready when all dependencies (DB connections, cache warmup) are confirmed.
- Use liveness probes for deadlock or permanent failure detection. Keep them simple - a failed HTTP 200 from
/healthzis enough.
Example configuration¶
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 2
selector:
matchLabels:
app: web
template:
metadata:
labels:
app: web
spec:
containers:
- name: web
image: ghcr.io/example/web:v3.0.0
ports:
- containerPort: 8080
startupProbe:
httpGet:
path: /startup
port: 8080
periodSeconds: 5
failureThreshold: 24
readinessProbe:
httpGet:
path: /ready
port: 8080
periodSeconds: 5
timeoutSeconds: 2
failureThreshold: 3
livenessProbe:
httpGet:
path: /healthz
port: 8080
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
Probe mechanism options¶
- HTTP GET: best default for web APIs
- TCP socket: useful for non-HTTP services
- Exec: last resort for process-level checks
- gRPC health check: preferred for gRPC services that implement the protocol
Common probe mistakes¶
- liveness checks that depend on external services
- aggressive timeouts that fail under normal load spikes
- missing startup probes for applications with long initialization
- readiness endpoints that report healthy before dependencies are actually ready
Troubleshooting¶
kubectl describe pod <pod-name>
kubectl get events -A --sort-by=.metadata.creationTimestamp
kubectl logs <pod-name> --previous
Probe failures appear clearly in pod events. Start there before changing YAML.
Certification notes¶
- CKAD loves probes. Know all three types, all four mechanisms (httpGet, tcpSocket, exec, grpc), and the timing fields (
initialDelaySeconds,periodSeconds,timeoutSeconds,failureThreshold,successThreshold). - A pod
Runningwith0/1 READYand restarts climbing = liveness failures;0/1 READYwith zero restarts = readiness failures. That single distinction answers many scenario questions.
Summary¶
Startup, readiness, and liveness probes serve different goals. When tuned correctly, they protect reliability during rollouts and failures.
Check Yourself¶
Your liveness probe hits /health, which checks the database connection. The database restarts for 90 seconds. What happens to your 20 replicas?
All 20 fail liveness at roughly the same moment and get restarted, then fail again on startup while the database is still down - a self-inflicted full outage on top of a partial one. The dependency check belongs in readiness, where the consequence is "stop sending traffic" rather than "kill the process."
A pod shows Running, 0/1 READY, and RESTARTS 0. Which probe is failing?
Readiness. Liveness failures force restarts, so a restart count of zero rules them out. The pod is up and being deliberately withheld from Service endpoints.
Your app takes two minutes to warm a cache at boot. Why is a long initialDelaySeconds on liveness a worse answer than a startup probe?
initialDelaySeconds delays detection forever, not just at boot: after the delay elapses, a hung process is still only caught at the normal cadence, and you have made every restart two minutes slower to detect. A startup probe gives slow initialization its own generous budget, and once it passes, liveness runs at a tight interval for the life of the container.
What happens to readiness and liveness while a startup probe is still running?
Both are paused. That is the whole point: nothing can restart the container or route traffic to it until startup succeeds, so slow initialization cannot be mistaken for a hang.
Related Concepts¶
- Learning Paths - where this page sits in your track
- Pods and Deployments
- Troubleshooting
- Resource Requests and Limits
Beginner track - step 4 of 12. Next: Networking Concepts. Back to the track overview.