Envoy¶

Who this page is for: in plain English, Envoy is a proxy - a program that sits in the request path and forwards traffic, applying routing, retries, TLS, and metrics along the way. You rarely install it directly; you get it as the engine inside a mesh or gateway. This page is production depth, aimed at readers already comfortable with Pods, Services, and Deployments. New to Kubernetes? Start with the Beginner track.

Envoy is a CNCF graduated, high-performance L4/L7 proxy originally built at Lyft and now the most widely deployed data plane in the cloud-native ecosystem. It doesn't run as a single named product - it runs as a piece inside other things: it's the sidecar proxy in Istio, the data plane in most Gateway API implementations, and the proxy engine under a long list of API gateways and ingress controllers.

The reason Envoy ended up everywhere is its API-driven, dynamically configurable architecture. Older proxies (nginx, HAProxy) are configured with a file and reloaded on change - disruptive at scale and slow to propagate. Envoy was built from day one to be driven by a control plane over a gRPC streaming API, so a fleet of thousands of proxies can have routing, cluster membership, and TLS config pushed to them continuously, without a process restart and without dropping connections. That single design decision is what made service meshes, in their modern form, practical.

Architecture: listeners, filters, routes, clusters, endpoints¶

Envoy's config model is a small number of composable resource types, and understanding how they relate is most of understanding Envoy.

flowchart LR
    subgraph Downstream
        CLIENT[Client connection]
    end
    CLIENT --> L["Listener\n(bind address:port)"]
    L --> FC["Filter chain\n(TLS termination, HTTP\nconnection manager, ...)"]
    FC --> RT["Route configuration\n(match host/path/header\n→ pick a cluster)"]
    RT --> CL["Cluster\n(logical upstream service,\nLB policy, health checks,\ncircuit breaking)"]
    CL --> EP1["Endpoint\n(pod IP:port)"]
    CL --> EP2["Endpoint\n(pod IP:port)"]
    CL --> EP3["Endpoint\n(pod IP:port)"]

• Listener - binds to an IP:port and accepts connections. A single Envoy process typically has multiple listeners (e.g. inbound and outbound in a sidecar).
• Filter chain - a listener's connections flow through a chain of network filters, most commonly the HTTP connection manager, which itself contains a chain of HTTP filters (router, rate limiter, auth, CORS, ext_authz, etc). This is a second, HTTP-level version of the same "chain of small, composable pieces" idea CoreDNS uses for DNS.
• Route configuration - matches an incoming request (host, path, headers) and decides which cluster handles it, plus request-level behavior: retries, timeouts, header manipulation, traffic splitting by weight.
• Cluster - a logical upstream group: load balancing policy (round robin, least request, ring hash, maglev), health checking, circuit breaking thresholds, TLS settings for the upstream connection, outlier detection.
• Endpoint - the actual IP:port members of a cluster. In Kubernetes, these are pod IPs, and they're the part of the config that changes most often as pods come and go.

Everything downstream of "which cluster" is essentially unrelated to "which endpoints are in that cluster right now" - which is exactly why Envoy split these into separate, independently-updatable APIs instead of one big config blob.

xDS: the discovery service APIs¶

Rather than rewriting a whole config file on every change, Envoy's control plane pushes each resource type through its own streaming gRPC API, collectively called xDS (x Discovery Service):

In practice these are usually aggregated into a single bidirectional gRPC stream, ADS (Aggregated Discovery Service), so the control plane can guarantee ordering (clusters before the endpoints that reference them, for instance) over one connection instead of five independent streams that could race.

flowchart TD
    CP["Control plane\n(e.g. istiod)"] -->|ADS: LDS+RDS+CDS+EDS+SDS\nover one gRPC stream| E1[Envoy instance 1]
    CP -->|ADS| E2[Envoy instance 2]
    CP -->|ADS| E3[Envoy instance N]
    API["kube-apiserver\n(Services, EndpointSlices,\nGateway API resources)"] -->|watched by| CP

This is why a mesh can push a new routing rule to thousands of proxies and have it take effect within seconds, with zero connection drops and no process restart - the control plane is just streaming a new version of one resource type down an already-open connection.

Static vs. dynamic configuration¶

Envoy can run entirely from a static file with no control plane at all - useful for a standalone gateway, testing, or understanding the resource model before layering xDS on top:

# envoy.yaml -- minimal static config: proxy / on :10000 to a backend cluster
static_resources:
  listeners:
    - name: listener_0
      address:
        socket_address: { address: 0.0.0.0, port_value: 10000 }
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: backend
                      domains: ["*"]
                      routes:
                        - match: { prefix: "/" }
                          route: { cluster: backend_service }
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.Router
  clusters:
    - name: backend_service
      type: STRICT_DNS
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: backend_service
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address: { address: backend.default.svc.cluster.local, port_value: 8080 }

That's the same listener → filter chain → route → cluster shape as the diagram above, just written by hand instead of pushed by a control plane. In production mesh and gateway deployments, almost none of this is hand-authored - a control plane (istiod, Envoy Gateway's controller, Contour) translates Kubernetes-native resources (Gateway API HTTPRoute, Istio VirtualService, Ingress) into xDS resources and streams them in. That translation layer - Kubernetes CRDs in, xDS out - is the actual product most "Envoy-based" projects are selling; Envoy itself is the shared, undifferentiated engine underneath.

L7 traffic management capabilities¶

Envoy's HTTP filter chain is where most of its value over a simple L4 proxy comes from:

• HTTP/2 and gRPC - native first-class support, including gRPC-to-JSON transcoding via a filter, which is why Envoy shows up so often in front of gRPC services.
• Retries - configurable per-route (retry_on, number of attempts, per-try timeout), with retry budgets to prevent retry storms from amplifying an outage.
• Circuit breaking - caps on concurrent connections, pending requests, and retries per cluster; once a cap is hit, Envoy fails fast instead of queuing.
• Outlier detection - passive health checking: hosts returning consecutive errors get ejected from the load-balancing pool for a backoff period, independent of active health checks.
• Rate limiting - local (in-process token bucket) or global (calls out to an external rate-limit service per request).
• Traffic splitting / weighted routing - route a percentage of traffic to different clusters, the mechanism under canary and blue/green rollouts in a mesh.
• Fault injection - deliberately inject delay or aborts on a percentage of requests, used for chaos/resilience testing.

All of this is request-aware behavior a plain L4 load balancer (like kube-proxy's iptables/IPVS rules) cannot do, since kube-proxy never looks past the IP/port tuple - it's the whole reason an L7-aware proxy layer exists on top of basic Service networking.

Observability Envoy exposes natively¶

Envoy instruments itself heavily without any application code changes, which is a large part of why service meshes can offer "automatic observability":

• Stats - a large built-in set of counters, gauges, and histograms per listener, cluster, and HTTP filter (request counts, upstream/downstream latency, connection pool state), scrapeable in Prometheus format, typically on a dedicated admin/metrics port separate from traffic ports.
• Access logs - structured, configurable-format logs of every request, including upstream cluster chosen, response code, and latency breakdown (time in queue vs. time waiting on upstream).
• Distributed tracing propagation - Envoy generates and forwards trace headers (B3, W3C Trace Context) and reports spans to a tracing collector; it can originate spans for traffic passing through it, but stitching a full end-to-end trace still requires the application to forward the incoming trace headers on any calls it makes - Envoy can't do that part for you.

This native instrumentation is why istio_requests_total and friends exist "for free" the moment sidecars are injected - see Istio's observability section for how that surfaces in a mesh specifically.

Hot restart and drain¶

Even with dynamic xDS config, Envoy occasionally needs a full binary restart (a version upgrade, for instance). Hot restart lets a new Envoy process take over from an old one without dropping in-flight connections: the new process starts, passes listen sockets from the old process via a shared memory/domain-socket handoff, and the old process drains - stops accepting new connections while letting existing ones finish - before exiting.

The same drain concept applies to ordinary config changes and pod termination: when a listener is removed or the process receives a shutdown signal, Envoy stops routing new connections to it while allowing in-flight requests to complete within a configurable drain window, rather than cutting them off. In Kubernetes, this is what coordinates with a pod's preStop hook and terminationGracePeriodSeconds to achieve a genuinely zero-downtime rolling update through a sidecar.

Where you'll encounter Envoy in the Kubernetes ecosystem¶

The pattern across all of them: Envoy is the shared, high-performance data plane; the differentiation is entirely in the control plane that decides what to program it with. This is directly analogous to how multiple GitOps tools can all drive the same underlying kubectl apply - the proxy execution engine is commodity, the control logic on top is the product.

Reaching into generated config: EnvoyFilter¶

Most of the time you configure Envoy indirectly through Kubernetes-native resources (Gateway API HTTPRoute, Istio VirtualService/DestinationRule) and never touch xDS resources directly. But control planes also expose an escape hatch for the cases their higher-level API doesn't cover. Istio's is EnvoyFilter, which patches the xDS config istiod would otherwise generate:

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: add-local-ratelimit
  namespace: my-app
spec:
  workloadSelector:
    labels:
      app: checkout
  configPatches:
    - applyTo: HTTP_FILTER
      match:
        context: SIDECAR_INBOUND
        listener:
          filterChain:
            filter:
              name: envoy.filters.network.http_connection_manager
      patch:
        operation: INSERT_BEFORE
        value:
          name: envoy.filters.http.local_ratelimit
          typed_config:
            "@type": type.googleapis.com/udpa.type.v1.TypedStruct
            type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
            value:
              stat_prefix: http_local_rate_limiter
              token_bucket:
                max_tokens: 100
                tokens_per_fill: 100
                fill_interval: 60s
              filter_enabled:
                runtime_key: local_rate_limit_enabled
                default_value: { numerator: 100, denominator: HUNDRED }

This inserts a filter directly into the HTTP filter chain by matching on context (which listener direction) and applyTo (which xDS resource type to patch - HTTP_FILTER, CLUSTER, ROUTE_CONFIGURATION, NETWORK_FILTER, and others). EnvoyFilter is powerful but bypasses the abstraction Istio's higher-level APIs provide - it patches whatever istiod happens to generate today, so it's the resource most likely to break across an Istio version upgrade. Reach for it only when VirtualService/DestinationRule/Telemetry genuinely can't express what you need.

Health checking: active vs. passive¶

Envoy supports two independent, complementary mechanisms for keeping bad endpoints out of the load-balancing pool:

• Active health checking - Envoy itself periodically sends HTTP/TCP/gRPC health check requests to each endpoint in a cluster, on a configurable interval, and removes non-responders. This is Envoy proactively probing, independent of real traffic.
• Passive health checking (outlier detection) - Envoy watches the real traffic it's already sending and ejects an endpoint that returns consecutive errors, without sending any extra probe traffic. This is what Istio's DestinationRule.trafficPolicy.outlierDetection configures under the hood.

In a Kubernetes context, active health checking is often redundant with kubelet readiness probes - the endpoint wouldn't be in an EndpointSlice at all if it weren't Ready. Passive/outlier detection carries more of its own weight in a mesh, since it reacts to error conditions kubelet readiness probes may not catch (a dependency-specific failure, a resource-exhaustion condition that doesn't fail the probe itself).

TLS: termination and origination¶

Envoy handles TLS on both sides of a proxied connection, and the two directions are configured independently:

• TLS termination (downstream) - Envoy decrypts inbound TLS from the client, typically to route based on decrypted HTTP content (path, headers) that wouldn't be visible in ciphertext. Certificates come from a static file, or dynamically from SDS so they can rotate without a restart - this is exactly how Istio delivers and rotates short-lived mTLS certificates to every sidecar.
• TLS origination (upstream) - Envoy initiates a new TLS connection to the upstream cluster, independent of whatever the downstream connection used. This is the mechanism behind mesh mTLS: the application inside the pod sends plaintext to its local sidecar, and the sidecar originates mTLS to the destination sidecar, which then terminates it and forwards plaintext to the destination application. Neither application container ever touches a TLS certificate directly.

Common mistakes¶

• Treating Envoy config as something you hand-write in production - outside of small standalone gateways, config should come from a control plane (Istio, Envoy Gateway, Contour) driven by Kubernetes-native resources, not static YAML maintained by hand.
• Assuming Linkerd sidecars are Envoy - they aren't. Linkerd uses its own purpose-built lightweight Rust proxy specifically to avoid Envoy's resource footprint; don't generalize "sidecar = Envoy" across every mesh.
• Ignoring circuit breaker and connection pool limits - default cluster limits are conservative; under real load without tuning, you can hit upstream connection pool exhausted errors that look like an application outage but are actually an untuned Envoy cluster.
• Not budgeting sidecar resource overhead - each Envoy sidecar has real memory and CPU cost; at high pod-per-node density this adds up and needs to be in cluster capacity planning, not treated as free.
• Debugging by reading application logs only - Envoy's own access logs and stats (/stats, /clusters on the admin port, or istioctl proxy-config in Istio) usually show the actual routing/cluster decision faster than guessing from app-side symptoms.
• Forgetting the drain window on fast rolling updates - terminating pods too aggressively (short terminationGracePeriodSeconds, no preStop sleep) can cut off in-flight requests even though Envoy supports graceful draining; the surrounding pod lifecycle has to cooperate.

Source Links¶

• Envoy documentation
• envoyproxy/envoy on GitHub and its releases
• Envoy release and support lifecycle
• xDS API overview (LDS/RDS/CDS/EDS)
• HTTP connection manager and filters
• Circuit breaking and outlier detection
• Admin interface (/stats, /clusters, /config_dump)
• Envoy Gateway
• CNCF project page: Envoy

Related Concepts¶

• Istio - the service mesh built on Envoy sidecars and (for waypoints) ambient mode
• Cilium - an alternative/complementary data plane, eBPF-based rather than proxy-based
• Gateway API - the CRD surface that Envoy Gateway and Contour implement
• Ingress - the older API several Envoy-based controllers also support
• Services - the L4 layer Envoy sits above
• Prometheus - where Envoy's native stats typically get scraped
• Operators and CRDs - the pattern behind translating Gateway API/VirtualService into xDS
• Troubleshooting - general debugging once Envoy's own stats point at the problem