Traffic tapping¶

Envoy currently provides two experimental extensions that can tap traffic:

HTTP tap filter. See the linked filter documentation for more information.

Tap transport socket extension that can intercept traffic and write to a protobuf trace file. The remainder of this document describes the configuration of the tap transport socket.

Tap transport socket configuration¶

Attention

The tap transport socket is experimental and is currently under active development. There is currently a very limited set of match conditions, output configuration, output sinks, etc. Capabilities will be expanded over time and the configuration structures are likely to change.

Tapping can be configured on Listener and Cluster transport sockets, providing the ability to interpose on downstream and upstream L4 connections respectively.

To configure traffic tapping, add an envoy.transport_sockets.tap transport socket configuration to the listener or cluster. For a plain text socket this might look like:

transport_socket:
  name: envoy.transport_sockets.tap
  config:
    common_config:
      static_config:
        match_config:
          any_match: true
        output_config:
          sinks:
            - format: PROTO_BINARY
              file_per_tap:
                path_prefix: /some/tap/path
    transport_socket:
      name: raw_buffer

For a TLS socket, this will be:

transport_socket:
  name: envoy.transport_sockets.tap
  config:
    common_config:
      static_config:
        match_config:
          any_match: true
        output_config:
          sinks:
            - format: PROTO_BINARY
              file_per_tap:
                path_prefix: /some/tap/path
    transport_socket:
      name: ssl
      config: <TLS context>

where the TLS context configuration replaces any existing downstream or upstream TLS configuration on the listener or cluster, respectively.

Each unique socket instance will generate a trace file prefixed with path_prefix. E.g. /some/tap/path_0.pb.

Buffered data limits¶

For buffered socket taps, Envoy will limit the amount of body data that is tapped to avoid OOM situations. The default limit is 1KiB for both received and transmitted data. This is configurable via the max_buffered_rx_bytes and max_buffered_tx_bytes settings. When a buffered socket tap is truncated, the trace will indicate truncation via the read_truncated and write_truncated fields as well as the body truncated field.

Streaming¶

The tap transport socket supports both buffered and streaming, controlled by the streaming setting. When buffering, SocketBufferedTrace messages are emitted. When streaming, a series of SocketStreamedTraceSegment are emitted.

See the HTTP tap filter streaming documentation for more information. Most of the concepts overlap between the HTTP filter and the transport socket.

PCAP generation¶

The generated trace file can be converted to libpcap format, suitable for analysis with tools such as Wireshark with the tap2pcap utility, e.g.:

bazel run @envoy_api//tools:tap2pcap /some/tap/path_0.pb path_0.pcap
tshark -r path_0.pcap -d "tcp.port==10000,http2" -P
 0.000000    127.0.0.1 → 127.0.0.1    HTTP2 157 Magic, SETTINGS, WINDOW_UPDATE, HEADERS
 0.013713    127.0.0.1 → 127.0.0.1    HTTP2 91 SETTINGS, SETTINGS, WINDOW_UPDATE
 0.013820    127.0.0.1 → 127.0.0.1    HTTP2 63 SETTINGS
 0.128649    127.0.0.1 → 127.0.0.1    HTTP2 5586 HEADERS
 0.130006    127.0.0.1 → 127.0.0.1    HTTP2 7573 DATA
 0.131044    127.0.0.1 → 127.0.0.1    HTTP2 3152 DATA, DATA