Design on Envoy Gateway

Goals

Mon, 01 Jan 0001 00:00:00 +0000

The high-level goal of the Envoy Gateway project is to attract more users to Envoy by lowering barriers to adoption through expressive, extensible, role-oriented APIs that support a multitude of ingress and L7/L4 traffic routing use cases; and provide a common foundation for vendors to build value-added products without having to re-engineer fundamental interactions.

Objectives

Expressive API

The Envoy Gateway project will expose a simple and expressive API, with defaults set for many capabilities.

System Design

Mon, 01 Jan 0001 00:00:00 +0000

Goals

Define the system components needed to satisfy the requirements of Envoy Gateway.

Non-Goals

Create a detailed design and interface specification for each system component.

Terminology

Control Plane- A collection of inter-related software components for providing application gateway and routing functionality. The control plane is implemented by Envoy Gateway and provides services for managing the data plane. These services are detailed in the components section.
Data Plane- Provides intelligent application-level traffic routing and is implemented as one or more Envoy proxies.

Architecture

Watching Components Design

Mon, 01 Jan 0001 00:00:00 +0000

Envoy Gateway is made up of several components that communicate in-process. Some of them (namely Providers) watch external resources, and “publish” what they see for other components to consume; others watch what another publishes and act on it (such as the resource translator watches what the providers publish, and then publishes its own results that are watched by another component). Some of these internally published results are consumed by multiple components.

Gateway API Translator Design

Mon, 01 Jan 0001 00:00:00 +0000

The Gateway API translates external resources, e.g. GatewayClass, from the configured Provider to the Intermediate Representation (IR).

Assumptions

Initially target core conformance features only, to be followed by extended conformance features.

Inputs and Outputs

The main inputs to the Gateway API translator are:

GatewayClass, Gateway, HTTPRoute, TLSRoute, Service, ReferenceGrant, Namespace, and Secret resources.

Note: ReferenceGrant is not fully implemented as of v0.2.

The outputs of the Gateway API translator are:

Control Plane Observability: Metrics

Tue, 10 Oct 2023 00:00:00 +0000

This document aims to cover all aspects of envoy gateway control plane metrics observability.

Note

Data plane observability (while important) is outside of scope for this document. For data plane observability, refer to here.

Current State

At present, the Envoy Gateway control plane provides logs and controller-runtime metrics, without traces. Logs are managed through our proprietary library (internal/logging, a shim to zap) and are written to /dev/stdout.

Backend

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces the Backend API allowing system administrators to represent backends without the use of a K8s Service resource.

Common use cases for non-Service backends in the K8s and Envoy ecosystem include:

Cluster-external endpoints, which are currently second-class citizens in Gateway-API (supported using Services and FQDN endpoints).
Host-local endpoints, such as sidecars or daemons that listen on unix domain sockets or envoy internal listeners, that cannot be represented by a K8s service at all.

Several projects currently support backends that are not registered in the infrastructure-specific service registry.

BackendTrafficPolicy

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces the BackendTrafficPolicy API allowing users to configure the behavior for how the Envoy Proxy server communicates with upstream backend services/endpoints.

Goals

Add an API definition to hold settings for configuring behavior of the connection between the backend services and Envoy Proxy listener.

Non Goals

Define the API configuration fields in this API.

Implementation

BackendTrafficPolicy is an implied hierarchy type API that can be used to extend Gateway API. It can target either a Gateway, or an xRoute (HTTPRoute/GRPCRoute/etc.). When targeting a Gateway, it will apply the configured settings within ght BackendTrafficPolicy to all children xRoute resources of that Gateway. If a BackendTrafficPolicy targets an xRoute and a different BackendTrafficPolicy targets the Gateway that route belongs to, then the configuration from the policy that is targeting the xRoute resource will win in a conflict.

Bootstrap Design

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Issue 31 specifies the need for allowing advanced users to specify their custom Envoy Bootstrap configuration rather than using the default Bootstrap configuration defined in Envoy Gateway. This allows advanced users to extend Envoy Gateway and support their custom use cases such setting up tracing and stats configuration that is not supported by Envoy Gateway.

Goals

Define an API field to allow a user to specify a custom Bootstrap
Provide tooling to allow the user to generate the default Bootstrap configuration as well as validate their custom Bootstrap.

Non Goals

Allow user to configure only a section of the Bootstrap

API

Leverage the existing EnvoyProxy resource which can be attached to the GatewayClass using the parametersRef field, and define a Bootstrap field within the resource. If this field is set, the value is used as the Bootstrap configuration for all managed Envoy Proxies created by Envoy Gateway.

ClientTrafficPolicy

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces the ClientTrafficPolicy API allowing system administrators to configure the behavior for how the Envoy Proxy server behaves with downstream clients.

Goals

Add an API definition to hold settings for configuring behavior of the connection between the downstream client and Envoy Proxy listener.

Non Goals

Define the API configuration fields in this API.

Implementation

ClientTrafficPolicy is a Direct Policy Attachment type API that can be used to extend Gateway API to define configuration that affect the connection between the downstream client and Envoy Proxy listener.

Configuration API Design

Mon, 01 Jan 0001 00:00:00 +0000

Motivation

Issue 51 specifies the need to design an API for configuring Envoy Gateway. The control plane is configured statically at startup and the data plane is configured dynamically through Kubernetes resources, primarily Gateway API objects. Refer to the Envoy Gateway design doc for additional details regarding Envoy Gateway terminology and configuration.

Goals

Define an initial API to configure Envoy Gateway at startup.
Define an initial API for configuring the managed data plane, e.g. Envoy proxies.

Non-Goals

Implementation of the configuration APIs.
Define the status subresource of the configuration APIs.
Define a complete set of APIs for configuring Envoy Gateway. As stated in the Goals, this document defines the initial configuration APIs.
Define an API for deploying/provisioning/operating Envoy Gateway. If needed, a future Envoy Gateway operator would be responsible for designing and implementing this type of API.
Specify tooling for managing the API, e.g. generate protos, CRDs, controller RBAC, etc.

Control Plane API

The EnvoyGateway API defines the control plane configuration, e.g. Envoy Gateway. Key points of this API are:

Data Plane Observability: Accesslog

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Envoy supports extensible accesslog to different sinks, File, gRPC etc.

Envoy supports customizable access log formats using predefined fields as well as arbitrary HTTP request and response headers.

Envoy supports several built-in access log filters and extension filters that are registered at runtime.

Envoy Gateway leverages Gateway API for configuring managed Envoy proxies. Gateway API defines core, extended, and implementation-specific API support levels for implementers such as Envoy Gateway to expose features. Since accesslog is not covered by Core or Extended APIs, EG should provide an easy to config access log formats and sinks per EnvoyProxy.

Data Plane Observability: Metrics

Mon, 01 Jan 0001 00:00:00 +0000

This document aims to cover all aspects of envoy gateway data plane metrics observability.

Note

Control plane observability (while important) is outside of scope for this document. For control plane observability, refer to here.

Overview

Envoy provide robust platform for metrics, Envoy support three different kinds of stats: counter, gauges, histograms.

Envoy enables prometheus format output via the /stats/prometheus admin endpoint.

Envoy support different kinds of sinks, but EG will only support Open Telemetry sink.

Data Plane Observability: Tracing

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Envoy supports extensible tracing to different sinks, Zipkin, OpenTelemetry etc. Overview of Envoy tracing can be found here.

Envoy Gateway leverages Gateway API for configuring managed Envoy proxies. Gateway API defines core, extended, and implementation-specific API support levels for implementers such as Envoy Gateway to expose features. Since tracing is not covered by Core or Extended APIs, EG should provide an easy to config tracing per EnvoyProxy.

Only OpenTelemetry sink can be configured currently, you can use OpenTelemetry Collector to export to other tracing backends.

Debug support in Envoy Gateway

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Envoy Gateway exposes endpoints at localhost:19000/debug/pprof to run Golang profiles to aid in live debugging.

The endpoints are equivalent to those found in the http/pprof package. /debug/pprof/ returns an HTML page listing the available profiles.

Goals

Add admin server to Envoy Gateway control plane, separated with admin server.
Add pprof support to Envoy Gateway control plane.
Define an API to allow Envoy Gateway to custom admin server configuration.
Define an API to allow Envoy Gateway to open envoy gateway config dump in logs.

The following are the different types of profiles end-user can run:

egctl Design

Mon, 01 Jan 0001 00:00:00 +0000

Motivation

EG should provide a command line tool with following capabilities:

Collect configuration from envoy proxy and gateway
Analyse system configuration to diagnose any issues in envoy gateway

This tool is named egctl.

Syntax

Use the following syntax to run egctl commands from your terminal window:

egctl [command] [entity] [name] [flags]

where command, name, and flags are:

command: Specifies the operation that you want to perform on one or more resources, for example config, version.

Envoy Gateway Extensions Design

Mon, 01 Jan 0001 00:00:00 +0000

As outlined in the official goals for the Envoy Gateway project, one of the main goals is to “provide a common foundation for vendors to build value-added products without having to re-engineer fundamental interactions”. Development of the Envoy Gateway project has been focused on developing the core features for the project and Kubernetes Gateway API conformance. This system focuses on the “common foundation for vendors” component by introducing a way for vendors to extend Envoy Gateway.

EnvoyExtensionPolicy

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces the EnvoyExtensionPolicy API allowing system administrators to configure traffic processing extensibility policies, based on existing Network and HTTP Envoy proxy extension points.

Envoy Gateway already provides two methods of control plane extensibility that can be used to achieve this functionality:

Envoy Patch Policy can be used to patch Listener filters and HTTP Connection Manager filters.
Envoy Extension Manager can be used to programmatically mutate Listener filters and HTTP Connection Manager filters.

These approaches require a high level of Envoy and Envoy Gateway expertise and may create a significant operational burden for users (see Alternatives for more details). For this reason, this document proposes to support Envoy data plane extensibility options as first class citizens of Envoy Gateway.

EnvoyPatchPolicy

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design introduces the EnvoyPatchPolicy API allowing users to modify the generated Envoy xDS Configuration that Envoy Gateway generates before sending it to Envoy Proxy.

Envoy Gateway allows users to configure networking and security intent using the upstream Gateway API as well as implementation specific Extension APIs defined in this project to provide a more batteries included experience for application developers.

These APIs are an abstracted version of the underlying Envoy xDS API to provide a better user experience for the application developer, exposing and setting only a subset of the fields for a specific feature, sometimes in a opinionated way (e.g RateLimit)
These APIs do not expose all the features capabilities that Envoy has either because these features are desired but the API is not defined yet or the project cannot support such an extensive list of features. To alleviate this problem, and provide an interim solution for a small section of advanced users who are well versed in Envoy xDS API and its capabilities, this API is being introduced.

Goals

Add an API allowing users to modify the generated xDS Configuration

Non Goals

Support multiple patch mechanisims

Implementation

EnvoyPatchPolicy is a Direct Policy Attachment type API that can be used to extend Gateway API Modifications to the generated xDS configuration can be provided as a JSON Patch which is defined in RFC 6902. This patching mechanism has been adopted in Kubernetes as well as Kustomize to update resource objects.

Fuzzing

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces Fuzz Testing in Envoy Gateway. Its goal is to detect unexpected crashes, memory leaks, and undefined behaviors in the translation of Gateway API resources and configuration parsing, areas that may not be fully covered by unit tests.

Additionally, OSS-Fuzz provides a continuous fuzzing infrastructure for popular open-source projects. By writing fuzz tests, we can leverage OSS-Fuzz to continuously fuzz Envoy Gateway against a wide range of inputs, improving its resilience and reliability.

Metadata in XDS resources

Mon, 01 Jan 0001 00:00:00 +0000

Overview

In Envoy, static metadata can be configured on various resources: listener, virtual host, route and cluster.

Static metadata can be used for various purposes:

Observability: enrichment of access logs and traces with metadata formatters and custom tags.
Processing: provide configuration context to filters in a certain scope (e.g. vhost, route, etc.).

This document describes how Envoy Gateway manages static metadata for various XDS resource such as listeners, virtual hosts, routes, clusters and endpoints.

Configuration

Envoy Gateway propagates certain attributes of Gateway-API resources to XDS resources. Attributes include:

Rate Limit Design

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Rate limit is a feature that allows the user to limit the number of incoming requests to a predefined value based on attributes within the traffic flow.

Here are some reasons why a user may want to implement Rate limits

To prevent malicious activity such as DDoS attacks.
To prevent applications and its resources (such as a database) from getting overloaded.
To create API limits based on user entitlements.

Scope Types

The rate limit type here describes the scope of rate limits.

Running Envoy Gateway locally

Mon, 01 Jan 0001 00:00:00 +0000

Overview

Today, Envoy Gateway runs only on Kubernetes. This is an ideal solution when the applications are running in Kubernetes. However there might be cases when the applications are running on the host which would require Envoy Gateway to run locally.

Goals

Define an API to allow Envoy Gateway to retrieve configuration while running locally.
Define an API to allow Envoy Gateway to deploy the managed Envoy Proxy fleet on the host machine.

Non Goals

Support multiple ways to retrieve configuration while running locally.
Support multiple ways to deploy the Envoy Proxy fleet locally on the host.

API

The provider field within the EnvoyGateway configuration only supports Kubernetes today which provides two features - the ability to retrieve resources from the Kubernetes API Server as well as deploy the managed Envoy Proxy fleet on Kubernetes.
This document proposes adding a new top level provider type called Custom with two fields called resource and infrastructure to allow the user to configure the sub providers for providing resource configuration and an infrastructure to deploy the Envoy Proxy data plane in.
A File resource provider will be introduced to enable retrieving configuration locally by reading from the configuration from a file.
A Host infrastructure provider will be introduced to allow Envoy Gateway to spawn a Envoy Proxy child process on the host.

Here is an example configuration

SecurityPolicy

Mon, 01 Jan 0001 00:00:00 +0000

Overview

This design document introduces the SecurityPolicy API allowing system administrators to configure authentication and authorization policies to the traffic entering the gateway.

Goals

Add an API definition to hold settings for configuring authentication and authorization rules on the traffic entering the gateway.

Non Goals

Define the API configuration fields in this API.

Implementation

SecurityPolicy is a Policy Attachment type API that can be used to extend Gateway API to define authentication and authorization rules.

TCP and UDP Proxy Design

Mon, 01 Jan 0001 00:00:00 +0000

Even though most of the use cases for Envoy Gateway are at Layer-7, Envoy Gateway can also work at Layer-4 to proxy TCP and UDP traffic. This document will explore the options we have when operating Envoy Gateway at Layer-4 and explain the design decision.

Envoy can work as a non-transparent proxy or a transparent proxy for both TCP and UDP , so ideally, Envoy Gateway should also be able to work in these two modes:

Wasm OCI Image Support

Mon, 01 Jan 0001 00:00:00 +0000

Motivation

Envoy Gateway (EG) should support Wasm OCI image as a remote wasm code source. This feature will allow users to deploy Wasm modules from OCI registries, such as Docker Hub, Google Container Registry, and Amazon Elastic Container Registry, to Envoy proxies managed by EG. Deploying Wasm modules from OCI registries has several benefits:

Versioning: Users can use the tag feature of the OCI image to manage the version of the Wasm module.
Security: Users can use private registries to store the Wasm module.
Distribution: Users can use the existing distribution mechanism of the OCI registry to distribute the Wasm module.

Goals

Define the system components needed to support Wasm OCI images as remote Wasm code sources.