Traffic on Envoy Gateway

Backend Routing

Mon, 01 Jan 0001 00:00:00 +0000

Envoy Gateway supports routing to native K8s resources such as Service and ServiceImport. The Backend API is a custom Envoy Gateway extension resource that can used in Gateway-API BackendObjectReference.

Motivation

The Backend API was added to support several use cases:

Allowing users to integrate Envoy with services (Ext Auth, Rate Limit, ALS, …) using Unix Domain Sockets, which are currently not supported by K8s.
Simplify routing to cluster-external backends, which currently requires users to maintain both K8s Service and EndpointSlice resources.

Warning

Similar to the K8s EndpointSlice API, the Backend API can be misused to allow traffic to be sent to otherwise restricted destinations, as described in CVE-2021-25740. A Backend resource can be used to:

Circuit Breakers

Mon, 01 Jan 0001 00:00:00 +0000

Envoy circuit breakers can be used to fail quickly and apply back-pressure in response to upstream service degradation.

Envoy Gateway supports the following circuit breaker thresholds:

Concurrent Connections: limit the connections that Envoy can establish to the upstream service. When this threshold is met, new connections will not be established, and some requests will be queued until an existing connection becomes available.
Concurrent Requests: limit on concurrent requests in-flight from Envoy to the upstream service. When this threshold is met, requests will be queued.
Pending Requests: limit the pending request queue size. When this threshold is met, overflowing requests will be terminated with a 503 status code.

Envoy’s circuit breakers are distributed: counters are not synchronized across different Envoy processes. The default Envoy and Envoy Gateway circuit breaker threshold values (1024) may be too strict for high-throughput systems.

Client Traffic Policy

Mon, 01 Jan 0001 00:00:00 +0000

This task explains the usage of the ClientTrafficPolicy API.

Introduction

The ClientTrafficPolicy API allows system administrators to configure the behavior for how the Envoy Proxy server behaves with downstream clients.

Motivation

This API was added as a new policy attachment resource that can be applied to Gateway resources and it is meant to hold settings for configuring behavior of the connection between the downstream client and Envoy Proxy listener. It is the counterpart to the BackendTrafficPolicy API resource.

Connection Limit

Mon, 01 Jan 0001 00:00:00 +0000

The connection limit features allows users to limit the number of concurrently active TCP connections on a Gateway or a Listener. When the connection limit is reached, new connections are closed immediately by Envoy proxy. It’s possible to configure a delay for connection rejection.

Users may want to limit the number of connections for several reasons:

Protect resources like CPU and Memory.
Ensure that different listeners can receive a fair share of global resources.
Protect from malicious activity like DoS attacks.

Envoy Gateway introduces a new CRD called Client Traffic Policy that allows the user to describe their desired connection limit settings. This instantiated resource can be linked to a Gateway.

Direct Response

Mon, 01 Jan 0001 00:00:00 +0000

Direct responses are valuable in cases where you want the gateway itself to handle certain requests without forwarding them to backend services. This task shows you how to configure them.

Installation

Follow the steps from the Quickstart to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Testing Direct Response

Note: the size of the response body (whether provided in-line or via a reference) cannot exceed 4096 bytes.

Failover

Mon, 01 Jan 0001 00:00:00 +0000

Active-passive failover in an API gateway setup is like having a backup plan in place to keep things running smoothly if something goes wrong. Here’s why it’s valuable:

Staying Online: When the main (or “active”) backend has issues or goes offline, the fallback (or “passive”) backend is ready to step in instantly. This helps keep your API accessible and your services running, so users don’t even notice any interruptions.
Automatic Switch Over: If a problem occurs, the system can automatically switch traffic over to the fallback backend. This avoids needing someone to jump in and fix things manually, which could take time and might even lead to mistakes.

Fault Injection

Mon, 01 Jan 0001 00:00:00 +0000

Envoy fault injection can be used to inject delays and abort requests to mimic failure scenarios such as service failures and overloads.

Envoy Gateway supports the following fault scenarios:

delay fault: inject a custom fixed delay into the request with a certain probability to simulate delay failures.
abort fault: inject a custom response code into the response with a certain probability to simulate abort failures.

Envoy Gateway introduces a new CRD called BackendTrafficPolicy that allows the user to describe their desired fault scenarios. This instantiated resource can be linked to a Gateway, HTTPRoute or GRPCRoute resource.

Gateway Address

Mon, 01 Jan 0001 00:00:00 +0000

The Gateway API provides an optional Addresses field through which Envoy Gateway can set addresses for Envoy Proxy Service. Depending on the Service Type, the addresses of gateway can be used as:

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions

Install the Gateway API CRDs and Envoy Gateway using Helm:

Gateway API Support

Mon, 01 Jan 0001 00:00:00 +0000

As mentioned in the system design document, Envoy Gateway’s managed data plane is configured dynamically through Kubernetes resources, primarily Gateway API objects. Envoy Gateway supports configuration using the following Gateway API resources.

GatewayClass

A GatewayClass represents a “class” of gateways, i.e. which Gateways should be managed by Envoy Gateway. Envoy Gateway supports managing a single GatewayClass resource that matches its configured controllerName and follows Gateway API guidelines for resolving conflicts when multiple GatewayClasses exist with a matching controllerName.

Global Rate Limit

Mon, 01 Jan 0001 00:00:00 +0000

Rate limit is a feature that allows the user to limit the number of incoming requests to a predefined value based on attributes within the traffic flow.

Here are some reasons why you may want to implement Rate limits

To prevent malicious activity such as DDoS attacks.
To prevent applications and its resources (such as a database) from getting overloaded.
To create API limits based on user entitlements.

Envoy Gateway supports two types of rate limiting: Global rate limiting and Local rate limiting.

GRPC Routing

Mon, 01 Jan 0001 00:00:00 +0000

The GRPCRoute resource allows users to configure gRPC routing by matching HTTP/2 traffic and forwarding it to backend gRPC servers. To learn more about gRPC routing, refer to the Gateway API documentation.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions

Install the Gateway API CRDs and Envoy Gateway using Helm:

HTTP Redirects

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource can issue redirects to clients or rewrite paths sent upstream using filters. Note that HTTPRoute rules cannot use both filter types at once. Currently, Envoy Gateway only supports core HTTPRoute filters which consist of RequestRedirect and RequestHeaderModifier at the time of this writing. To learn more about HTTP routing, refer to the Gateway API documentation.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

HTTP Request Headers

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource can modify the headers of a request before forwarding it to the upstream service. HTTPRoute rules cannot use both filter types at once. Currently, Envoy Gateway only supports core HTTPRoute filters which consist of RequestRedirect and RequestHeaderModifier at the time of this writing. To learn more about HTTP routing, refer to the Gateway API documentation.

A RequestHeaderModifier filter instructs Gateways to modify the headers in requests that match the rule before forwarding the request upstream. Note that the RequestHeaderModifier filter will only modify headers before the request is sent from Envoy to the upstream service and will not affect response headers returned to the downstream client.

HTTP Response Headers

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource can modify the headers of a response before responding it to the downstream service. To learn more about HTTP routing, refer to the Gateway API documentation.

A ResponseHeaderModifier filter instructs Gateways to modify the headers in responses that match the rule before responding to the downstream. Note that the ResponseHeaderModifier filter will only modify headers before the response is returned from Envoy to the downstream client and will not affect request headers forwarding to the upstream service.

HTTP Routing

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource allows users to configure HTTP routing by matching HTTP traffic and forwarding it to Kubernetes backends. Currently, the only supported backend supported by Envoy Gateway is a Service resource. This task shows how to route traffic based on host, header, and path fields and forward the traffic to different Kubernetes Services. To learn more about HTTP routing, refer to the Gateway API documentation.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

HTTP Timeouts

Mon, 01 Jan 0001 00:00:00 +0000

The default request timeout is set to 15 seconds in Envoy Proxy. The HTTPRouteTimeouts resource allows users to configure request timeouts for an HTTPRouteRule. This task shows you how to configure timeouts.

The HTTPRouteTimeouts supports two kinds of timeouts:

request: Request specifies the maximum duration for a gateway to respond to an HTTP request.
backendRequest: BackendRequest specifies a timeout for an individual request from the gateway to a backend.

Note: The Request duration must be >= BackendRequest duration

HTTP URL Rewrite

Mon, 01 Jan 0001 00:00:00 +0000

HTTPURLRewriteFilter defines a filter that modifies a request during forwarding. At most one of these filters may be used on a Route rule. This MUST NOT be used on the same Route rule as a HTTPRequestRedirect filter.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions

Install the Gateway API CRDs and Envoy Gateway using Helm:

HTTP3

Mon, 01 Jan 0001 00:00:00 +0000

This task will help you get started using HTTP3 using EG. This task uses a self-signed CA, so it should be used for testing and demonstration purposes only.

Prerequisites

OpenSSL to generate TLS assets.

Installation

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Expand for instructions

Install the Gateway API CRDs and Envoy Gateway using Helm:

HTTPRoute Request Mirroring

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource allows one or more backendRefs to be provided. Requests will be routed to these upstreams. It is possible to divide the traffic between these backends using Traffic Splitting, but it is also possible to mirror requests to another Service instead. Request mirroring is accomplished using Gateway API’s HTTPRequestMirrorFilter on the HTTPRoute.

When requests are made to a HTTPRoute that uses a HTTPRequestMirrorFilter, the response will never come from the backendRef defined in the filter. Responses from the mirror backendRef are always ignored.

HTTPRoute Traffic Splitting

Mon, 01 Jan 0001 00:00:00 +0000

The HTTPRoute resource allows one or more backendRefs to be provided. Requests will be routed to these upstreams if they match the rules of the HTTPRoute. If an invalid backendRef is configured, then HTTP responses will be returned with status code 500 for all requests that would have been sent to that backend.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Load Balancing

Mon, 01 Jan 0001 00:00:00 +0000

Envoy load balancing is a way of distributing traffic between multiple hosts within a single upstream cluster in order to effectively make use of available resources.

Envoy Gateway supports the following load balancing policies:

Round Robin: a simple policy in which each available upstream host is selected in round robin order.
Random: load balancer selects a random available host.
Least Request: load balancer uses different algorithms depending on whether hosts have the same or different weights.
Consistent Hash: load balancer implements consistent hashing to upstream hosts.

Envoy Gateway introduces a new CRD called BackendTrafficPolicy that allows the user to describe their desired load balancing polices. This instantiated resource can be linked to a Gateway, HTTPRoute or GRPCRoute resource. If loadBalancer is not specified in BackendTrafficPolicy, the default load balancing policy is Least Request.

Local Rate Limit

Mon, 01 Jan 0001 00:00:00 +0000

Rate limit is a feature that allows the user to limit the number of incoming requests to a predefined value based on attributes within the traffic flow.

Here are some reasons why you may want to implement Rate limits

To prevent malicious activity such as DDoS attacks.
To prevent applications and its resources (such as a database) from getting overloaded.
To create API limits based on user entitlements.

Envoy Gateway supports two types of rate limiting: Global rate limiting and Local rate limiting.

Multicluster Service Routing

Mon, 01 Jan 0001 00:00:00 +0000

The Multicluster Service API ServiceImport object can be used as part of the GatewayAPI backendRef for configuring routes. For more information about multicluster service API follow sig documentation.

We will use Submariner project for setting up the multicluster environment for exporting the service to be routed from peer clusters.

Setting KIND clusters and installing Submariner.

We will be using KIND clusters to demonstrate this example.

git clone https://github.com/submariner-io/submariner-operator
cd submariner-operator
make clusters

Note: remain in submariner-operator directory for the rest of the steps in this section

Response Compression

Mon, 01 Jan 0001 00:00:00 +0000

Response Compression allows you to compress the response from the backend before sending it to the client. This can be useful for scenarios where the backend sends large responses that can be compressed to reduce the network bandwidth. However, this comes with a trade-off of increased CPU usage on the Envoy side to compress the response.

Installation

Follow the steps from the Quickstart to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Response Override

Mon, 01 Jan 0001 00:00:00 +0000

Response Override allows you to override the response from the backend with a custom one. This can be useful for scenarios such as returning a custom 404 page when the requested resource is not found or a custom 500 error message when the backend is failing.

Installation

Follow the steps from the Quickstart to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Retry

Mon, 01 Jan 0001 00:00:00 +0000

Starting from v1.3, Envoy Gateway supports HTTPRoute Retries(GEP-1731), this setting in the core Gateway API takes precedence over the BackendTrafficPolicy configuration.

A retry setting specifies the maximum number of times an Envoy proxy attempts to connect to a service if the initial call fails. Retries can enhance service availability and application performance by making sure that calls don’t fail permanently because of transient problems such as a temporarily overloaded service or network. The interval between retries prevents the called service from being overwhelmed with requests.

Routing outside Kubernetes

Mon, 01 Jan 0001 00:00:00 +0000

Routing to endpoints outside the Kubernetes cluster where Envoy Gateway and its corresponding Envoy Proxy fleet is running is a common use case. This can be achieved by:

defining FQDN addresses in a EndpointSlice (covered in this document)
defining a Backend resource, as described in the Backend Task.

Installation

Follow the steps from the Quickstart to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

Session Persistence

Mon, 01 Jan 0001 00:00:00 +0000

Session Persistence allows client requests to be consistently routed to the same backend service instance. This is useful in many scenarios, such as when an application needs to maintain state across multiple requests. In Envoy Gateway, session persistence can be enabled by configuring HTTPRoute.

Envoy Gateway supports following session persistence types:

Cookie-based Session Persistence: Session persistence is achieved based on specific cookie information in the request.
Header-based Session Persistence: Session persistence is achieved based on specific header information in the request.

Prerequisites

Install Envoy Gateway

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

TCP Routing

Mon, 01 Jan 0001 00:00:00 +0000

TCPRoute provides a way to route TCP requests. When combined with a Gateway listener, it can be used to forward connections on the port specified by the listener to a set of backends specified by the TCPRoute. To learn more about HTTP routing, refer to the Gateway API documentation.

Prerequisites

Follow the steps below to install Envoy Gateway and the example manifest. Before proceeding, you should be able to query the example backend using HTTP.

UDP Routing

Mon, 01 Jan 0001 00:00:00 +0000

The UDPRoute resource allows users to configure UDP routing by matching UDP traffic and forwarding it to Kubernetes backends. This task will use CoreDNS example to walk you through the steps required to configure UDPRoute on Envoy Gateway.

Note: UDPRoute allows Envoy Gateway to operate as a non-transparent proxy between a UDP client and server. The lack of transparency means that the upstream server will see the source IP and port of the Gateway instead of the client. For additional information, refer to Envoy’s UDP proxy documentation.