Featured image of post Dabbling with Envoy configurations - Part I

Dabbling with Envoy configurations - Part I

Exploring the different options that envoy provides and how it forms the basics of service meshes

Types of envoy configurations

  1. Static configuration

  2. Dynamic configuration. This can be done in two ways:

    • From Filesystem
    • From Control plane

Static configuration

To start envoy in static configuration we need the following:

  1. listeners
  2. clusters
  3. static_reources
  4. (Optional) admin section

static_resources

Contain everything that is configured statically when envoy starts. Can contain the following:

  • []listeners
  • []clusters
  • []secrets

listeners

Lets configure an example listener on port 10000. Here all paths are matched and routed to service_envoyproxy_io cluster

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
listeners:
- name: listener_0
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 10000
  filter_chains:
  - filters:
    - name: envoy.filters.network.http_connection_manager
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
        stat_prefix: ingress_http
        access_log:
        - name: envoy.access_loggers.stdout
          typed_config:
            "@type": type.googleapis.com/envoy.extensions.access_loggers.stream.v3.StdoutAccessLog
        http_filters:
        - name: envoy.filters.http.router
        route_config:
          name: local_route
          virtual_hosts:
          - name: local_service
            domains: ["*"]
            routes:
            - match:
                prefix: "/"
              route:
                host_rewrite_literal: www.envoyproxy.io
                cluster: service_envoyproxy_io

cluster

The service_envoyproxy_io cluster proxies over TLS to https://www.envoyproxy.io

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
clusters:
- name: service_envoyproxy_io
  type: LOGICAL_DNS
  # Comment out the following line to test on v6 networks
  dns_lookup_family: V4_ONLY
  load_assignment:
    cluster_name: service_envoyproxy_io
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: www.envoyproxy.io
              port_value: 443
  transport_socket:
    name: envoy.transport_sockets.tls
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
      sni: www.envoyproxy.io

Testing this static configuration

If now we start envoy with this configuration using command envoy -c <config_name>.yaml and try querying the localhost:10000 port, we should get the envoyproxy homepage.

1
curl -v localhost:10000

Dynamic Configuration from filesystem

In this setup Envoy will automatically update its configuration whenever the files are changed on the filesystem. The following sections are a must for dynamic configuration:

  1. node
  2. dynamic_resources

node

node needs a cluster and an id

1
2
3
node:
  cluster: test-cluster
  id: test-id

dynamic_resources

Specifies where to load dynamic configuration from

1
2
3
4
5
dynamic_resources:
  cds_config:
    path: ./cds.yaml
  lds_config:
    path: ./lds_yaml

listener resources

The linked lds_config should be an implementation of a Listener Discovery Service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
resources:
- "@type": type.googleapis.com/envoy.config.listener.v3.Listener
  name: listener_0
  address:
    socket_address:
      address: 0.0.0.0
      port_value: 10000
  filter_chains:
  - filters:
    - name: envoy.http_connection_manager
      typed_config:
        "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
        stat_prefix: ingress_http
        http_filters:
        - name: envoy.router
        route_config:
          name: local_route
          virtual_hosts:
          - name: local_service
            domains:
            - "*"
            routes:
            - match:
                prefix: "/"
              route:
                host_rewrite_literal: www.envoyproxy.io
                cluster: example_proxy_cluster

cluster resources

The linked cds_config should be an implementation of a Cluster Discovery Service

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
resources:
- "@type": type.googleapis.com/envoy.config.cluster.v3.Cluster
  name: example_proxy_cluster
  type: STRICT_DNS
  connect_timeout: 3s
  typed_extension_protocol_options:
    envoy.extensions.upstreams.http.v3.HttpProtocolOptions:
      "@type": type.googleapis.com/envoy.extensions.upstreams.http.v3.HttpProtocolOptions
      explicit_http_config:
        http2_protocol_options: {}
  load_assignment:
    cluster_name: example_proxy_cluster
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: www.envoyproxy.io
              port_value: 443
  transport_socket:
    name: envoy.transport_sockets.tls
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
      sni: www.envoyproxy.io

Dynamically editing the configuration

Let’s try editing this config to start proxying to google.com instead of envoyproxy.io
In the lds.yaml file change the following:

1
2
3
4
5
6
7
            routes:
            - match:
                prefix: "/"
              route:
-               host_rewrite_literal: www.envoyproxy.io
+               host_rewrite_literal: www.google.com
                cluster: example_proxy_cluster

As soon as we do this write in the file, the LDS config in the envoy will update and will show in the logs:

1
lds: add/update listener 'listener_0'

We need to update the cds.yaml config as well:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
  load_assignment:
    cluster_name: example_proxy_cluster
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
-              address: www.envoyproxy.io
+              address: www.google.com
              port_value: 443
  transport_socket:
    name: envoy.transport_sockets.tls
    typed_config:
      "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.UpstreamTlsContext
-      sni: www.envoyproxy.io
+      sni: www.google.com

We should see the similar update in envoy’s logs about the CDS config update

1
cds: added/updated 1 cluster(s), skipped 0 unmodified cluster(s)

Hence we were able to reload the envoy configuration dynamically without restarting the server itself.

comments powered by Disqus
Built with Hugo
Theme Stack designed by Jimmy