Skip to content

docs: L7 logs restructure and eBPF collector#2724

Merged
ctauchen merged 9 commits into
tigera:mainfrom
lucastigera:core-12726-l7-obs-bpf
Jul 10, 2026
Merged

docs: L7 logs restructure and eBPF collector#2724
ctauchen merged 9 commits into
tigera:mainfrom
lucastigera:core-12726-l7-obs-bpf

Conversation

@lucastigera

@lucastigera lucastigera commented May 12, 2026

Copy link
Copy Markdown
Contributor

Summary

Restructures the L7 logs docs section to introduce the new eBPF-based collector alongside the existing Envoy collector, with a shared overview and per-collector enable pages.

The Istio Waypoint collector will land in a follow-up PR — its placeholder page is set to draft: true so it's visible during local development but excluded from production builds.

L7 logs section, after this PR

Sidebar entry File What it is
Overview overview.mdx New. Big picture, value, the available collectors, and a comparison table to help pick one. Carries the shared "before you begin" log-storage note.
Enable eBPF collector enable-ebpf.mdx New. Enable via L7ObservabilityEnabled on FelixConfiguration. Limitations: kernel 5.17+, HTTP/1.0–1.1 only, TLS metadata only.
Enable Waypoint collector enable-waypoint.mdx Placeholder, dev-only. Marked draft: true so it's hidden in production builds. Content lands in the Waypoint PR.
Enable Envoy collector configure.mdx Existing page; sidebar label updated. Still labelled deprecated and being phased out.
Data types datatypes.mdx Refreshed schema reference. New Populated by column. Adds protocol, protocol_version, collector_type, collector_name, and tls_* fields. HTTP-only / TLS-only fields tagged inline (e.g. _HTTP only._ URL that the request was made against.).
Aggregation aggregation.mdx New. Documents the L7LogsFileAggregation* fields (including the new L7LogsFileAggregationTLSSNI) and includes an interactive aggregation playground — a React component (_includes/components/L7AggregationSandbox) where readers toggle aggregator checkboxes and watch sample events regroup with two-tone highlight (green = added / count up, red = removed / count down) and per-group expand/collapse.

Other changes

  • dashboards.mdx, get-started-cem.mdx, kibana.mdx: external "L7 logs" links repointed from configure.mdx (Envoy-only) to the new overview.mdx.
  • alp.mdx: left pointing at configure.mdx#kibana per maintainer note (page will be deprecated).
  • Vale lint clean on the new pages (data plane, Waypoint, plain text).

Open items / known gaps

  • L7ObservabilityEnabled and L7LogsFileAggregationTLSSNI won't render in the Felix Configuration tables until _includes/components/FelixConfig/config-params.json is regenerated from upstream calico-private. Field-level anchors (#L7ObservabilityEnabled, #L7LogsFileAggregationTLSSNI) will activate at that time.
  • Versioned-tree mirroring intentionally not done in this PR — changes land in the next tree (calico-enterprise/) only.

Test plan

  • Local Docusaurus dev build (yarn start-next) compiles and renders all six pages.
  • Sidebar order verified in .docusaurus route data.
  • Aggregation playground toggles cleanly, columns hold their widths, group expand/collapse works.
  • No remaining bare references to renamed files (configure-bpf.mdx, configure-istio-waypoint.mdx, datatypes-wip.mdx, datatypes-legacy.mdx).
  • Vale CI lint passing.
  • Production build (yarn build-next) excludes enable-waypoint.mdx — to be verified by reviewers.

🤖 Generated with Claude Code

New L7 observability pages for the eBPF-based collector (tigera/calico-private#11749):
- configure-bpf.mdx: enable via L7ObservabilityEnabled on FelixConfiguration; comparison with the Istio Waypoint collector
- configure-istio-waypoint.mdx: placeholder
- datatypes-wip.mdx: redraft of the L7 schema with a Populated by column and new protocol / collector_name / tls_* fields
- aggregation.mdx: aggregation key fields with an interactive playground component
- sidebar entries for all four pages

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 12, 2026 23:59
@netlify

netlify Bot commented May 12, 2026

Copy link
Copy Markdown

Deploy Preview for calico-docs-preview-next ready!

Name Link
🔨 Latest commit 0478ad3
🔍 Latest deploy log https://app.netlify.com/projects/calico-docs-preview-next/deploys/6a4fd8964e78b00007fec3b5
😎 Deploy Preview https://deploy-preview-2724--calico-docs-preview-next.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@netlify

netlify Bot commented May 12, 2026

Copy link
Copy Markdown

Deploy Preview for tigera failed. Why did it fail? →

Built without sensitive environment variables

Name Link
🔨 Latest commit 0478ad3
🔍 Latest deploy log https://app.netlify.com/projects/tigera/deploys/6a4fd8969ad1c00008da4670

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR (marked WIP) introduces draft Calico Enterprise documentation for a new eBPF-based L7 observability collector, including a new L7 log aggregation page with an interactive sandbox component, and wires the new pages into the Enterprise sidebar navigation.

Changes:

  • Adds new draft L7 documentation pages for eBPF configuration, Istio Waypoint (placeholder), aggregation, and an expanded (WIP) L7 schema reference.
  • Introduces a new React component (L7AggregationSandbox) used to provide an interactive aggregation-key playground in docs.
  • Updates the Calico Enterprise sidebar to surface the new L7 pages.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
sidebars-calico-enterprise.js Adds the new L7 doc pages to the Enterprise “L7 logs” sidebar category.
calico-enterprise/observability/elastic/l7/configure-bpf.mdx Draft guide for enabling/viewing eBPF-based L7 logs, including a comparison table and limitations.
calico-enterprise/observability/elastic/l7/configure-istio-waypoint.mdx Placeholder page scaffold for Istio Ambient/Waypoint-based L7 collection.
calico-enterprise/observability/elastic/l7/aggregation.mdx Draft documentation describing FelixConfiguration aggregation fields and embedding an interactive sandbox.
calico-enterprise/observability/elastic/l7/datatypes-wip.mdx Draft schema reference adding eBPF/TLS fields and a “Populated by” column.
calico-enterprise/_includes/components/L7AggregationSandbox/index.js New interactive React component that demonstrates how aggregation settings merge L7 log rows and update count.
Comments suppressed due to low confidence (4)

calico-enterprise/observability/elastic/l7/configure-bpf.mdx:70

  • Same issue as above: L7LogsFileDirectory is a FelixConfiguration (resource) field and should link to reference/resources/felixconfig.mdx#l7LogsFileDirectory (note the lowercase l in the anchor) rather than the Felix config file/env-var page.
The eBPF collector writes L7 events as JSON to a log file on each node. By default the file is at `/var/log/calico/l7logs/l7.log`. The directory is controlled by the [L7LogsFileDirectory](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileDirectory) field on `FelixConfiguration`.

calico-enterprise/observability/elastic/l7/aggregation.mdx:25

  • The per-field links here target .../component-resources/node/felix/configuration.mdx#L7Logs..., but the generated anchors for FelixConfiguration fields use the YAML field names (for example #l7LogsFileAggregationSourceInfo) and live on reference/resources/felixconfig.mdx. Please update the doc path and anchor casing for these aggregation field links so they resolve.
| [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. |
| [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. |
| [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. |
| [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. |
| [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. |

calico-enterprise/observability/elastic/l7/aggregation.mdx:28

  • These links have the same issue as above (wrong target doc + uppercase anchors). Also, L7LogsFileAggregationTLSSNI does not appear in the current config-params.json data set, so the anchor will remain broken until the FelixConfig JSON is regenerated to include it.
| [L7LogsFileAggregationResponseCode](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationResponseCode) | Response code appears on each row. | Requests are merged across response codes. |
| [L7LogsFileAggregationHTTPHeaderInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPHeaderInfo) | User agent and request type appear on each row. | Requests are merged across user agents and types. |
| [L7LogsFileAggregationTLSSNI](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTLSSNI) | TLS Server Name Indication appears on each TLS row. | TLS flows to the same destination are merged regardless of SNI. |

calico-enterprise/observability/elastic/l7/aggregation.mdx:35

  • The second table here also starts rows with ||, which will render as an extra empty column. Use a single leading | for Markdown tables.
| Field | Description |
| --- | --- |
| [L7LogsFileAggregationNumURLPath](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationNumURLPath) | Maximum number of URL path components retained. Default `5`. A negative value disables this truncation. |
| [L7LogsFileAggregationURLCharLimit](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationURLCharLimit) | Maximum URL length in characters; longer URLs are sliced. Default `250`. |

| Additional infrastructure | Waypoint Envoy pods per namespace/service | None — runs inside Felix |
| L7 coverage | Only traffic routed through a waypoint | All TCP traffic on the node |
| Network path overhead | Two extra proxy hops | None |
| Plaintext access | Decrypted at the waypoint (mTLS terminated) | Captured at the kernel TCP layer, before TLS encryption |

## Configure L7 logs

Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any dataplane.
Comment on lines +17 to +35
The following `FelixConfiguration` fields control the L7 log aggregation key. For accepted values and defaults, follow the link to the [Felix Configuration reference](../../../reference/component-resources/node/felix/configuration.mdx#l7-logs).

| Field | Effect when included | Effect when excluded |
| --- | --- | --- |
| [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. |
| [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. |
| [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. |
| [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. |
| [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. |
| [L7LogsFileAggregationResponseCode](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationResponseCode) | Response code appears on each row. | Requests are merged across response codes. |
| [L7LogsFileAggregationHTTPHeaderInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPHeaderInfo) | User agent and request type appear on each row. | Requests are merged across user agents and types. |
| [L7LogsFileAggregationTLSSNI](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTLSSNI) | TLS Server Name Indication appears on each TLS row. | TLS flows to the same destination are merged regardless of SNI. |

Two additional fields control how the URL is truncated before it enters the aggregation key, rather than whether the URL participates at all:

| Field | Description |
| --- | --- |
| [L7LogsFileAggregationNumURLPath](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationNumURLPath) | Maximum number of URL path components retained. Default `5`. A negative value disables this truncation. |
| [L7LogsFileAggregationURLCharLimit](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationURLCharLimit) | Maximum URL length in characters; longer URLs are sliced. Default `250`. |
Comment thread sidebars-calico-enterprise.js Outdated
Comment on lines +408 to +411
'observability/elastic/l7/configure-istio-waypoint',
'observability/elastic/l7/aggregation',
'observability/elastic/l7/datatypes',
'observability/elastic/l7/datatypes-wip',
Comment on lines +29 to +35
| Aspect | Istio Waypoint collector | eBPF L7 collector |
| --- | --- | --- |
| Additional infrastructure | Waypoint Envoy pods per namespace/service | None — runs inside Felix |
| L7 coverage | Only traffic routed through a waypoint | All TCP traffic on the node |
| Network path overhead | Two extra proxy hops | None |
| Plaintext access | Decrypted at the waypoint (mTLS terminated) | Captured at the kernel TCP layer, before TLS encryption |
| Protocol coverage | HTTP/1, HTTP/2, gRPC | HTTP/1.0, HTTP/1.1, TLS handshake metadata |
Comment on lines +19 to +25
| Field | Effect when included | Effect when excluded |
| --- | --- | --- |
| [L7LogsFileAggregationSourceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationSourceInfo) | Source aggregated name, namespace, and type appear on each row. | Requests are merged across sources. |
| [L7LogsFileAggregationDestinationInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationDestinationInfo) | Destination aggregated name, namespace, and type appear on each row. | Requests are merged across destinations. |
| [L7LogsFileAggregationServiceInfo](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationServiceInfo) | Destination service name, namespace, and port appear on each row. | Requests are merged across services. |
| [L7LogsFileAggregationHTTPMethod](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationHTTPMethod) | HTTP method appears on each row. | Requests are merged across methods. |
| [L7LogsFileAggregationTrimURL](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileAggregationTrimURL) | URL (or a trimmed prefix) appears on each row. Intermediate values trim the query string, then the path, before excluding the URL entirely. | Requests are merged across URLs. |
Comment on lines +29 to +33
| Name | Datatype | Populated by | Description |
| ------------------------ | -------- | ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `host` | keyword | All | Name of the node that collected the L7 log entry. |
| `start_time` | date | All | Start time of log collection in Unix timestamp format. |
| `end_time` | date | All | End time of log collection in Unix timestamp format. |
lucastigera and others added 4 commits May 13, 2026 12:27
Aggregation sandbox redesign:
- Group rows by aggregation bucket; expand/collapse to reveal merged events
- Two-tone highlight (green for added/count up, red for removed/count down)
- Color legend above the table; intro/footer condensed
- More diverse sample data spanning cart, payments, and TLS scenarios

Pages:
- Promote the WIP data types page to datatypes.mdx; move the original to datatypes-legacy.mdx
- Add collector_type and protocol_version fields
- Sidebar reordered with "+ " prefix on the new pages and shorter labels (Configure with eBPF, Configure with Istio Waypoint, Aggregation, Data types)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- dataplane / dataplanes → data plane / data planes
- waypoint → Waypoint (Calico term rule)
- Plaintext / plaintext → Plain text / plain text
- Rephrase HTTP pipelining bullet to drop dictionary-rejected words

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add Overview page (overview.mdx) with big picture, value, collectors,
  and a comparison table — moves the shared framing out of per-collector pages
- Rename configure-bpf.mdx → enable-ebpf.mdx, refocus on enablement steps
- Rename configure-istio-waypoint.mdx → enable-waypoint.mdx (placeholder)
- Drop datatypes-legacy.mdx (no inbound references)
- Sidebar labels: Overview, Enable eBPF/Waypoint/Envoy collector, Data types, Aggregation
- Datatypes page: HTTP-only / TLS-only fields tagged inline in the description
- Repoint external "L7 logs" links (dashboards, get-started-cem, kibana) to overview.mdx

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Istio Waypoint collector will land in a separate PR. For now:
- enable-waypoint.mdx: add draft: true; Docusaurus excludes it from production builds and auto-filters the sidebar entry
- overview.mdx, datatypes.mdx: remove inline Waypoint references; leave TODO notes in single-line JSX comments so the next-PR author can find the spots
- overview.mdx: drop the Istio Waypoint column from the comparison table for this PR
- datatypes.mdx: change "Populated by" from "eBPF, Waypoint" to "eBPF"

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@lucastigera lucastigera marked this pull request as ready for review May 22, 2026 18:05
@lucastigera lucastigera requested a review from a team as a code owner May 22, 2026 18:05
@lucastigera lucastigera changed the title WIP: docs for eBPF L7 observability L7 Logs - eBPF collector May 22, 2026
@lucastigera lucastigera changed the title L7 Logs - eBPF collector docs: L7 logs restructure and eBPF collector May 22, 2026
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

# L7 logs

## Big picture

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rm

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean removing the entire Big Picture session, or just the title?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the title, let me know if that is what you meant.


L7 logs complement L3/4 flow logs, which expose only the source/destination pair without request-level detail.

## Collectors

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth saying what a collector actually is.


## Collectors

$[prodname] supports multiple L7 collectors. They all feed the same aggregation and reporting pipeline; each entry on the log file is tagged with `collector_type` and `collector_name` so consumers can filter or join sources as needed.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not clear what the distinction is between type and name. Are you saying we can have an epbf type called A and another ebpf type called B?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could have multiple collectors under the same type. For example, our current eBPF collector is attached to the TCP layer, so collector_name = ebpf_tcp. We could also have an eBPF collector attached through uprobes, in which case it would be collector_name = ebpf_uprobe.

That said, I don't think we need to call that out here since the distinction isn't really clear in this version. I removed collector_name to avoid confusion.


Multiple collectors can coexist in the same cluster, with each L7 log entry tagged by `collector_type` and `collector_name` so consumers can filter or join sources as needed.

:::note

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should not be buried in a note. Probably best as a prerequisite to each 'enable' procedure.

For clarity, consider working through an example. "For example, if you want to maintain a 7-day retention period for a three-node cluster, you need to add at least 21 GB to your existing log storage."

@@ -0,0 +1,42 @@
---

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The file name is misleading. Perhaps enable-ebpf-collector or simlar.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for waypoint.

Comment on lines +26 to +30
To disable the collector, set the field back to `false`. Felix tears down its BPF programs and cleans up pinned maps; no node reboot is required.

```bash
kubectl patch felixconfiguration default --type=merge -p '{"spec":{"l7ObservabilityEnabled":false}}'
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an expectation that it wouldn't be cleaned up? I'm wondering if we can remove all of this.

@lucastigera lucastigera Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is your question whether the disable path is clear enough that it doesn’t need to be documented? If so, I agree - we are just flipping back the field. I removed the disable block.


## Enable the collector

Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any data plane.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a single step procedure, use an unordered list:

Suggested change
Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any data plane.
* Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any data plane.

```bash
tail -f /var/log/calico/l7logs/l7.log
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Additional information
* L7 log data types

Comment on lines +32 to +40
## View L7 logs

The eBPF collector writes L7 events as JSON to a log file on each node. By default the file is at `/var/log/calico/l7logs/l7.log`. The directory is controlled by the [L7LogsFileDirectory](../../../reference/component-resources/node/felix/configuration.mdx#L7LogsFileDirectory) field on `FelixConfiguration`.

To tail the log on a node:

```bash
tail -f /var/log/calico/l7logs/l7.log
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can all move to a single line after the enable step.

This could also act as a verification step, come to think of it.

  1. To verify that L7 logs are being collected, look for output in the default log file location on each node:


import L7AggregationKeyDemo from '../../../_includes/components/L7AggregationKeyDemo';

# L7 log aggregation

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the new component is handy for illustrating how aggregation works. At this stage, I have two thoughts.

  1. To what extent is this L7-specific? I get that the fields may be different, but it seems that the main idea here is to help people understand aggregation generally, not L7 aggregation. If that's the case, perhaps it would be better to use a tool like this in a general doc about aggregation?

  2. For me, it would be useful to see all parts of this dynamic in action. 1) a full list of all the communications 2) the configuration (as a code block, rendered dynamically according to the checkboxes, and 3) the actual flow logs (what the log would actually look like, not the isolated terms as cells in a table.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean moving this page to observability/elastic/aggregation.mdx and making it more generic?

If so, I created a new page there with a different interaction, based on your suggestion for something we could test.

WDYT?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's just what I meant. I think it works better there and in that format. If you're happy with that, we can leave it there and remove this page.

- Rename enable-ebpf.mdx -> enable-ebpf-collector.mdx and
  enable-waypoint.mdx -> enable-waypoint-collector.mdx; update sidebar
  and all internal links.
- Retitle collector pages to "Collect L7 logs using the ... collector".
- Restructure enable-ebpf-collector: drop "Before you begin", split into
  Limitations + Prerequisites (kernel + log-storage with worked example),
  convert the enable step to a list, fold log viewing into a Verify step,
  add an Additional information section, and remove the disable block.
- Overview: drop the "Big picture" heading, define what a collector is,
  move the storage note into the enable procedure, drop collector_name.
- get-started-cem: "Url" -> "URL" (Vale.Terms).
- Add a generic "Log aggregation" page plus AggregationKeyDemo component.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexh-tigera

Copy link
Copy Markdown
Member

@lucastigera WDYT about adding a link to observability/elastic/l7/configure#view-l7-logs-in-the-web-console in the enable-ebpf-collector.mdx file?

Comment on lines +24 to +26
```bash
kubectl patch felixconfiguration default --type=merge -p '{"spec":{"l7ObservabilityEnabled":true}}'
```

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be indented to stay in line with the list item above.

* Linux kernel 5.17 or later on every node.
* Additional log storage for the L7 logs. L7 logs require a minimum of 1 additional GB of log storage per node, per one-day retention period. Adjust your [Log Storage](../../../operations/logstorage/adjust-log-storage-size.mdx) before enabling the collector. For example, to maintain a 7-day retention period on a three-node cluster, add at least 21 GB to your existing log storage.

## Enable the collector

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can combine 'enable' and 'verify' under the enable section. Two steps, where step 2 verifies that step 1 did its job.


## Enable the collector

* Enable the eBPF L7 collector by setting the [L7ObservabilityEnabled](../../../reference/component-resources/node/felix/configuration.mdx#L7ObservabilityEnabled) field to `true` on the default `FelixConfiguration`. The setting is independent of `bpfEnabled` — the collector works with any data plane.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

independent of bpfEnabled

Consider making the implied meaning here explicit. People might not know that particular key. What you mean, I think, is that you can use the eBPF collector even if you're CE installation runs on nftables or whatever.

Come to think of it, that would be valuable information for a person who is evaluating the L7 logs feature. Some might not get as far as the enablement step before they give up, thinking that it's only for people using eBPF as a data plane. Consider making this clear in the L7 overview and intro to this page.


# L7 log data types

## Big picture

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this subhead. A small intro and table is okay.


import L7AggregationKeyDemo from '../../../_includes/components/L7AggregationKeyDemo';

# L7 log aggregation

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's just what I meant. I think it works better there and in that format. If you're happy with that, we can leave it there and remove this page.

@lucastigera

Copy link
Copy Markdown
Contributor Author

@lucastigera WDYT about adding a link to observability/elastic/l7/configure#view-l7-logs-in-the-web-console in the enable-ebpf-collector.mdx file?

Nice catch @alexh-tigera . There are three things holding me back from adding that link. Let me know if you agree.

  1. That page has a deprecation notice at the top. I could add a Service Graph section to the eBPF collector page instead, but then the other two points come into play.
  2. The Service Graph page only covers HTTP metrics, not TLS. So linking to it would only be partially correct!? hehe
  3. The goal of this first iteration of the eBPF collector is to collect and expose the data in the L7 log file. Plumbing that data all the way to the Manager UI was an extra step to prepare for the next phase, which is to build a visual interface for users to interact with. I think we need more time to figure out the best way to present that information.

Those are the reasons I only pointed users to the L7 log file in the Verify the collector section of the eBPF collector page.

- enable-ebpf-collector: merge Enable + Verify into one section with two
  numbered steps (step 2 verifies step 1); indent code blocks under their
  list items; make the bpfEnabled independence explicit (works with any
  data plane) in the intro and enable step; note that logs are flushed on
  L7LogsFlushInterval (5m default) so output is not immediate.
- overview: spell out that the eBPF collector does not require the eBPF
  data plane; point the aggregation link at the general Log aggregation
  page.
- datatypes: drop the "Big picture" subhead (intro + table is enough).
- Remove the L7-specific aggregation page and L7AggregationKeyDemo
  component; the demo now lives in the general Log aggregation doc. Drop
  the sidebar entry.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@alexh-tigera

Copy link
Copy Markdown
Member

I think we need more time to figure out the best way to present that information.

Good point. Let's leave it for now.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ctauchen ctauchen merged commit 9ef0c35 into tigera:main Jul 10, 2026
7 of 11 checks passed
@lucastigera lucastigera deleted the core-12726-l7-obs-bpf branch July 10, 2026 16:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants