diff --git a/.agents/skills/bpf-tutorial-writing-style/SKILL.md b/.agents/skills/bpf-tutorial-writing-style/SKILL.md index 71cc61fe..fee850b1 100644 --- a/.agents/skills/bpf-tutorial-writing-style/SKILL.md +++ b/.agents/skills/bpf-tutorial-writing-style/SKILL.md @@ -5,102 +5,61 @@ description: Style checklist for English and Chinese bpf-developer-tutorial READ # BPF Tutorial Writing Style +This checklist is reference material, not a mandatory second pass after `$write-bpf-production-tutorial`. When Claude is the delegated writer, do not use this skill to review or rewrite Claude's result, do not send it to Claude as an additional checklist, and do not ask Claude for another revision. Codex must make local word-choice and punctuation edits after the delegated writing pass while preserving sentence meaning and paragraph structure. + Read the complete guidelines first: - [Advanced tutorial guideline](https://github.com/eunomia-bpf/bpf-developer-tutorial/blob/main/scripts/guideline_advance.md) for tutorials 40+ - [Basic tutorial guideline](https://github.com/eunomia-bpf/bpf-developer-tutorial/blob/main/scripts/guideline_basic.md) for tutorials 0-39 Then read the English and Chinese versions of `47-cuda-events`, `48-energy`, and `49-hid` as style references. Learn their teaching rhythm and level of detail without copying their topic or wording. -## Document Structure (strictly follow guideline) - -For advanced tutorials (40+), follow this exact section order: - -1. **Title**: `# eBPF 教程：[Topic Description]` or `# eBPF Tutorial by Example: [Topic Description]` - -2. **Introduction**: Brief intro with a concrete problem scenario. Highlight significance and what readers will learn. Link to complete source once here. - -3. **Background / Why This Approach**: - - Explain WHY this approach is needed - - List traditional/alternative approaches and their limitations (e.g., killing process, firewall rules, user-space tools) - - Explain what eBPF/the new kernel feature enables that traditional approaches cannot - - State when the feature entered Linux (kernel version + commit if relevant) - -4. **High-Level Mechanism**: - - Explain HOW the eBPF feature/tool works at a high level BEFORE showing code - - Describe the overall flow: what happens in kernel, what happens in user space, how they interact - - Use diagrams for complex flows with branches, waits, ownership transfers, or 3+ dependent states - -5. **Code Implementation**: - - First introduce the overall processing logic - - Show complete kernel/BPF source and the core headers needed to understand it - - Include the complete user-space loader when it is concise; when it is long, use focused excerpts that cover the normal user workflow and link to the lesson directory for the full file - - After each code block, explain the key parts with paragraph style (not bullet lists) - - Focus on logic and advanced features, not basic syntax +## Let the topic choose the structure -6. **Additional Concepts** (if needed): Deeper explanation of specific features, edge cases, or semantics +Follow `scripts/guideline_advance.md` without turning its suggestions into ten mandatory headings. Use a direct title, a short opening, the background the example actually needs, a high-level flow, complete source with explanation, compilation and execution, a compact ending, and primary references. Choose section names that sound natural for the topic. -7. **Compilation and Execution** (AFTER code, not before): - - Build commands - - Run commands with examples - - Expected output with explanation - - Environment requirements table (kernel version, config, privileges, architecture) - -8. **Summary**: Key points, scope boundaries, future extensions - -9. **Call to Action**: Repository and website invitation (as blockquote) - -10. **References**: Links to kernel commits, upstream selftests, documentation +Introduce information at the point where the reader needs it. Compilation follows the code discussion. Extra concept sections, alternative approaches, requirements tables, and diagrams are useful when they clarify a real decision or a flow with several dependent states. ## Tell one useful story -- Open with a concrete situation that the example can reproduce and the question it answers. -- Near the first mention of eBPF, use one natural sentence to say what eBPF is and why it fits this problem. +- Position the lesson at the narrowest useful capability that the executable really provides. Keep the title, opening, CLI, normal execution path, output, and test focused on that same capability. For example, a tool that learns one domain's A record and checks one TCP port is a minimal DNS-derived IP allowlist rather than a general DNS-aware firewall. +- Open with the concrete question the tool answers. A short factual setup is often enough; avoid invented stories. +- Near the first mention of eBPF, use one natural sentence to say that it runs verified programs at kernel hooks and can send selected state to user space. - Introduce the relevant kernel subsystem and new feature when the running example needs them. State when the feature entered Linux and what it enabled. - Follow one packet, event, task, or device interaction through kernel space and user space. Explain what happens, why it happens, and what the next step enables. -- Present the whole flow before detailed code. A small diagram helps when the reader must track a branch, wait, retry, ownership transfer, or at least three dependent states. - -## Explain the "Why" thoroughly - -This is critical. The Background section must explain: -- What problem you're solving -- What traditional approaches exist (killing process, firewall rules, user-space tools like `ss --kill`, sampling, etc.) -- Why each traditional approach doesn't work well (race conditions, incomplete coverage, performance overhead, etc.) -- What the eBPF approach enables that wasn't possible before - -Example pattern for the "Why" section: -> **杀掉进程**是最直接的想法，但一个进程往往维护着多条连接，杀进程会中断所有业务流量。 -> **防火墙规则**可以阻止新连接，但对已建立的连接无能为力。 -> **用户态工具**如 `ss --kill` 依赖 `/proc/net/tcp` 遍历和注入 RST 报文，但这种方式有竞态问题。 -> **内核态方案**才能真正解决这个问题。BPF 迭代器可以在持有适当锁的情况下遍历内核的套接字表... +- Present the whole flow before detailed code. Add a diagram only when it makes a branch, wait, retry, ownership transfer, or multi-stage path materially easier to follow. ## Sound like a tutorial - Use familiar words, direct verbs, and connected paragraphs. Attraction comes from the problem and mechanism rather than promotional language. -- Prefer positive descriptions of what the example does. Put remaining limits and safety boundaries in one short paragraph near the end. +- Prefer positive descriptions of what the example does. Use negative constructions sparingly. Put the most relevant scope note in one short paragraph near the end. - Use prose for the main explanation and lists for genuinely parallel items. - Keep each paragraph focused on one job. Connect facts through cause, sequence, or contrast instead of listing them like a specification. - Write Chinese naturally from the same facts instead of translating English sentence by sentence. Use restrained punctuation and spaces between Chinese text and Latin letters or numbers. +- Do not use em dashes, doubled Chinese em dashes, or en dashes as sentence punctuation in either language. Use commas, semicolons, colons, parentheses, or separate sentences instead. Codex checks every target README pair after Claude returns. - Keep technical names, code identifiers, commands, numbers, output, versions, and references exact. +- Match operational terms to behavior. Call a tool a profiler when it collects a profile such as sampled call paths or a distribution, an index when it maintains a searchable or ordered index, and a policy tool when it actually enforces a decision. +- For a stateful policy, teach the complete trust chain: which event creates candidate state, which fields correlate later events, which checks admit state, how time or deletion retires it, and which hook makes the final decision. Give poisoning, stale-state, and bypass resistance the space required to explain the mechanism. Use this Chinese paragraph only as a voice reference for connected rhythm, restrained symbols, and technical density: -> libbpf 是一个 C/C++ 的 eBPF 用户态加载和控制库，随着内核一起分发，几乎已经成为 eBPF 用户态事实上的 API 标准，libbpf 也支持 CO-RE(Compile Once – Run Everywhere) 的解决方案，即预编译的 bpf 代码可以在不同内核版本上正常工作，而无需为每个特定内核重新编译。 +> libbpf 是一个 C/C++ 的 eBPF 用户态加载和控制库，随着内核一起分发，几乎已经成为 eBPF 用户态事实上的 API 标准，libbpf 也支持 CO-RE(Compile Once, Run Everywhere) 的解决方案，即预编译的 bpf 代码可以在不同内核版本上正常工作，而无需为每个特定内核重新编译。 -## Teach from complete source +## Teach from the code that matters -- Introduce one component, show its complete kernel/BPF source in an ordinary Markdown fence, then explain the important logic before moving to the next component. -- Show core headers completely. Include a complete user-space file when it remains readable; otherwise use focused excerpts that preserve the public control flow without making the tutorial a source dump. -- Keep every complete source block byte-exact. Preserve comments and commands in excerpts, and link once to the lesson directory for omitted user-space details. +- Introduce one component, show the code needed to understand it in an ordinary Markdown fence, then explain the important logic before moving to the next component. +- Show the core kernel-side eBPF program in full when its complete control flow is the lesson. Large or secondary kernel files may use focused excerpts when the omitted parts do not carry the mechanism being taught. +- Keep the user-space discussion compact. Show the loader, configuration, event loop, and cleanup functions only when they help explain the interaction with BPF. A complete user-space loader is optional, and omitting boilerplate is not a tutorial failure. +- Keep every included code excerpt faithful to the repository source. Link to the tutorial directory once so readers can inspect the complete implementation. - Use neither `
` nor HTML synchronization markers. -- Link to the complete lesson once through its GitHub directory. Avoid an opening catalog of individual files. -- Every Markdown link uses an absolute `https://github.com/...` target. Omit a link when no stable GitHub target exists. +- Avoid an opening catalog of individual files. +- Every Markdown link uses a stable absolute `https://` target. GitHub, kernel.org, and authoritative documentation sites are all valid; relative links are prohibited. - Public prose contains no local path, shared test repository, VM name, copy route, cache, prompt, model, agent, or trace detail. ## Finish the lesson - Show copyable build and run commands, representative real output, and what that output proves. - State kernel, configuration, privilege, architecture, and hardware requirements that affect the example (use a table). -- End with a compact scope boundary, summary, repository invitation, and primary references. +- End with a compact summary, repository invitation, and primary references. - Keep the English and Chinese files aligned on structure, facts, source, commands, output, limits, and references while allowing each language to sound natural. -The final read should answer: what problem is solved, how one event moves through the system, which eBPF mechanism makes it possible, which code matters, how to run it, what success looks like, and where the example stops. +The final read should answer: what problem is solved, how one event moves through the system, which eBPF mechanism makes it possible, which code matters, how to run it, what success looks like, and where the example stops. It does not need to reproduce every line of user-space implementation. diff --git a/.agents/skills/find-bpf-tutorial-topic/SKILL.md b/.agents/skills/find-bpf-tutorial-topic/SKILL.md index e54c5137..5e405c08 100644 --- a/.agents/skills/find-bpf-tutorial-topic/SKILL.md +++ b/.agents/skills/find-bpf-tutorial-topic/SKILL.md @@ -55,6 +55,8 @@ For every serious candidate, write seven short fields: Defer an idea when these fields remain vague. A helper name alone is not a tutorial topic. +Before marking a feature-driven candidate ready, verify the complete API path against the target kernel. Check the intended BPF program type, map fields, helpers or kfunc sets, sleepability, locking, and verifier restrictions together; a selftest using another program type is not evidence that the proposed hook can use the same API. Build and load the smallest representative probe in KVM when static inspection leaves doubt. If the target program type cannot use the feature, choose a scenario where it can instead of routing data through user space solely to make the API appear in the example. + ## Apply the practicality gate before scoring Walk through the proposed public workflow as an operator, separately from the deterministic test harness: @@ -98,6 +100,7 @@ A candidate can become `ready` only when: - the repository does not already teach the same flow; - the example can fit one coherent tutorial; - primary sources establish the technical claims; +- the intended program type can load every required map and API on the target kernel; - required kernel, architecture, privilege, and hardware conditions are known; - the likely maintenance burden is proportionate to its value. diff --git a/.agents/skills/test-bpf-tutorial-kvm/scripts/run-in-kvm.sh b/.agents/skills/test-bpf-tutorial-kvm/scripts/run-in-kvm.sh index fa51db87..ddf411f2 100755 --- a/.agents/skills/test-bpf-tutorial-kvm/scripts/run-in-kvm.sh +++ b/.agents/skills/test-bpf-tutorial-kvm/scripts/run-in-kvm.sh @@ -146,13 +146,15 @@ fi die "/dev/kvm is not available with read/write permission; refusing a TCG fallback" benchmark_root=$(realpath -e -- "$benchmark_root") -kernel_source=$benchmark_root/vendor/linux-framework kernel_build=$benchmark_root/vendor/build/x86/linux +kernel_source_link=$kernel_build/source kernel_image=$kernel_build/arch/x86/boot/bzImage kernel_config=$kernel_build/.config kernel_release_file=$kernel_build/include/config/kernel.release -[[ -d $kernel_source ]] || die "kernel source directory is missing: $kernel_source" +[[ -e $kernel_source_link ]] || die "kernel build source link is missing: $kernel_source_link" +kernel_source=$(realpath -e -- "$kernel_source_link") +[[ -d $kernel_source ]] || die "kernel build source directory is missing: $kernel_source" [[ -s $kernel_image ]] || die "built benchmark kernel is missing: $kernel_image" [[ -r $kernel_config ]] || die "kernel config is missing: $kernel_config" [[ -r $kernel_release_file ]] || die "kernel release file is missing: $kernel_release_file" @@ -191,6 +193,7 @@ printf '%s\n' \ 'KVM preflight: OK' \ "benchmark_root=$benchmark_root" \ "kernel_image=$kernel_image" \ + "kernel_source=$kernel_source" \ "kernel_release=$kernel_release" \ "kernel_sha256=$kernel_sha256" \ "kernel_config_sha256=$config_sha256" \ diff --git a/.agents/skills/write-bpf-production-tutorial/SKILL.md b/.agents/skills/write-bpf-production-tutorial/SKILL.md index 8e212337..072ac56d 100644 --- a/.agents/skills/write-bpf-production-tutorial/SKILL.md +++ b/.agents/skills/write-bpf-production-tutorial/SKILL.md @@ -1,96 +1,36 @@ --- name: write-bpf-production-tutorial -description: Design, write, or rewrite one practical bilingual bpf-developer-tutorial lesson with pinned Claude Opus 4.5, then verify the public workflow and prose against the implementation. Use when creating README.md and README.zh.md, checking whether a tutorial tool has a realistic CLI and lifecycle, preserving complete kernel source, testing the example, or preparing tutorial changes for review. +description: Write or rewrite bilingual bpf-developer-tutorial lessons by assigning each tutorial's English and Chinese README pair to its own pinned Claude Opus 4.5 process. Use when creating or revising README.md and README.zh.md while requiring Claude to edit one tutorial paragraph by paragraph from the reader's perspective, allowing Codex only local word and punctuation edits plus the smallest evidence-backed factual corrections, and forbidding an additional prose review or rewrite request. --- # Write a BPF Tutorial -Use `$bpf-tutorial-writing-style` for the finished prose. Keep this workflow small: prepare reliable source material, let one pinned writer complete both languages, then check what it actually wrote. +Keep one writer for the complete reader-facing text. One Claude Opus process performs one free, paragraph-by-paragraph pass over one tutorial's English and Chinese README pair. Codex coordinates the invocation and performs integrity and technical-fidelity checks without becoming a second prose writer or reviewer. -## 1. Prepare the lesson +## 1. Give Opus the complete task once -Read these inputs before writing: +Use the exact model ID `claude-opus-4-5-20251101`. Stop when that model is unavailable instead of substituting another model. -- `scripts/guideline_advance.md` for tutorials 40+, `scripts/guideline_basic.md` for tutorials 0-39; -- both README files from `src/47-cuda-events`, `src/48-energy`, and `src/49-hid` as style references; -- the lesson's implementation, headers, Makefile, fixtures, and tests; -- the current README pair when revising an existing lesson; -- primary upstream sources for versions and feature semantics. +Run one non-interactive invocation from the repository root for one tutorial directory. Name that tutorial's English and Chinese README pair, its implementation, headers, Makefile, tests, the applicable `scripts/guideline_advance.md` or `scripts/guideline_basic.md`, and existing tutorials as general references. Ask Claude to read those sources, keep technical claims grounded in them, revise every paragraph freely from the reader's perspective, improve readability, adjust content or structure wherever useful, finish both files before returning, and ask no questions. -Collect the facts the reader needs: the problem, why traditional approaches fail, the kernel/user-space flow, feature versions, requirements, intended public commands, real output, concurrency or admission behavior, cleanup, limits, and references. Keep every claim grounded in the code, tests, captured output, or a primary source. Preserve an existing draft before a from-scratch rewrite. +Start a separate Claude process for every additional tutorial. Never batch README pairs from different tutorial directories into one process. -Build and run the example when the environment supports it. Use `$test-bpf-tutorial-kvm` for kernel features that need the repository's KVM environment. Runtime details support the tutorial; local workspace paths, VM names, shared repositories, caches, prompts, and agent traces stay private. +Keep the prompt to that request. Do not add a paragraph plan, fact inventory, style checklist, defect list, review rubric, acceptance criteria, or instructions for a later revision. Give Claude permission to read the repository and write only the target README files. Claude does not commit or push. -## 2. Pass the practical-design gate +## 2. Preserve single-writer ownership -Validate the tool as an operator before writing prose. Keep its public workflow separate from the deterministic fixture: +Treat Claude as the sole author of sentences, headings, paragraph order, explanations, and translations. After Claude returns, Codex must make a local word-choice and punctuation pass. Except for the smallest evidence-backed factual corrections allowed by section 3, keep those edits within the existing sentence meaning and paragraph structure. Codex must not add or remove information, rewrite sentences, reorder paragraphs, change headings, shorten or expand explanations, or alter translations substantively. -- State the operational question, the exact command a reader would run, the independent workload or target, the useful output, and how the tool stops. -- Classify the lifecycle. A scan or atomic control action may be one-shot; a tracer or monitor must attach before the workload and remain active until a signal, duration, or real completion condition. -- Do not turn a blocked toy child or `/bin/true` fixture into the public CLI merely because it removes a test race. Launch-scoped tracing is valid only when it is the intended real workflow. -- Emit and test an explicit readiness signal before starting an independent workload. Do not use a fixed sleep as proof that setup or attachment completed. -- Check target scope and filters, concurrent state, admission bounds, drop and failure counters, exit status, normal cleanup, signal cleanup, and destructive-action safety in proportion to the lesson. -- For asynchronous work, stop admission first, wait for completed work rather than merely entered callbacks, drain output, report stable health, and only then destroy resources. -- Execute the documented command as written. The test must prove the real lifecycle plus one relevant failure or cleanup path, not only that the BPF program loaded. -- Describe the current public workflow directly. Do not narrate removed flags, old child-command modes, or other migration history unless backward compatibility is itself the lesson. -- Keep repository tests out of the reader-facing tutorial path. Use them as private validation evidence, but teach the normal command, independent workload, useful tool output, and shutdown sequence. -- Show only output emitted by the documented tool in public examples. Never include harness lines such as `TEST-*` or `PASS`, fixture setup, test assertions, or local test-infrastructure provenance. +Reader-facing prose must not use em dashes, doubled Chinese em dashes, or en dashes as sentence punctuation. After every Claude writing pass, run `rg -n '[—–]'` on the target README pair and inspect every match outside source code. Replace prose matches with commas, semicolons, colons, parentheses, or separate sentences. Hyphens that belong to code identifiers, command options, URLs, or established technical names remain unchanged. -If the implementation only demonstrates a helper but is awkward or misleading as a tool, revise the implementation and test before asking the writer to explain it. Do not let polished prose overclaim an impractical design. +Use exactly one Claude writing pass per tutorial. Do not ask Claude to review its result, respond to a defect list, polish selected paragraphs, or rewrite the tutorial again. Do not invoke another model, subagent, or independent reviewer for the prose. When the result needs another writing pass, report that fact to the user and wait for an explicit request. -## 3. Structure requirements +## 3. Check integrity and technical fidelity -For advanced tutorials (40+), follow this exact section order as defined in `$bpf-tutorial-writing-style`: +Check that the intended files exist, remain nonempty, stay within the requested file scope, contain no merge markers, and expose no local paths, usernames, secrets, prompts, or private infrastructure. The mandatory dash scan in the previous section is part of Codex's punctuation pass. Do not count lines or code fences as a quality proxy. Do not require the README to contain a byte-exact copy of every source file, and do not treat an omitted complete user-space loader as a failure. -1. **Title + Introduction**: Concrete problem scenario, link to source -2. **Background / Why This Approach**: Explain traditional approaches and their limitations, then what eBPF enables -3. **High-Level Mechanism**: How the feature works before showing code -4. **Code Implementation**: Complete kernel/BPF source and core headers, plus a complete user-space loader when concise or focused user-space excerpts when long, then paragraph explanations -5. **Additional Concepts** (if needed) -6. **Compilation and Execution**: AFTER code analysis, not before -7. **Summary + Call to Action** -8. **References** +Verify every included source excerpt, command, output sample, version, requirement, and behavior claim against the implementation, tests, captured evidence, or a primary source. Use `sync-source-blocks.py` for blocks intended to reproduce complete repository files when the script supports the lesson, but do not compare focused excerpts against a required inventory of every source file. Run non-rewriting repository documentation validation. When technical commands, runtime behavior, requirements, or output claims changed, run the smallest relevant build or test, or identify exact current validation evidence for the unchanged implementation. -The "Background / Why" section is critical. It must explain: -- What traditional approaches exist -- Why each doesn't work well -- What the eBPF approach enables +These are content-fidelity checks, not a second prose-writing pass. Codex may make the smallest factual correction needed to match the implementation or evidence, but must not ask another model, subagent, or independent reviewer to rewrite or polish content-only changes. Preserve prompts, responses, drafts, and failed runs; never delete real conversation or agent history. -## 4. Let Opus write - -Claude Opus writes all reader-facing tutorial prose. Use the exact model ID `claude-opus-4-5-20251101`. Stop if that model is unavailable instead of substituting another model. - -Use one non-interactive invocation for the complete English and Chinese pair. The prompt stays short and names only: - -- the two target README files; -- `scripts/guideline_advance.md` or `scripts/guideline_basic.md`; -- `$bpf-tutorial-writing-style`; -- `src/47-cuda-events`, `src/48-energy`, and `src/49-hid` as style references; -- the request to read the implementation and write both files completely before returning. - -Do not paste a second checklist, paragraph plan, fact inventory, or review rubric into the prompt. Add a technical fact only when it is unavailable in the repository. - -Run Claude from the repository root with the pinned model and permission to read the repository and write the two README files. Opus does not commit or push. - -## 5. Check the result - -Inspect both files and the diff instead of trusting the model's final message. Confirm that: - -- both languages are complete and tell the same technical story; -- the documented command, target, lifecycle, signals, concurrency bounds, cleanup, and health output match the practical design gate; -- the opening reads like a tutorial rather than an abstract or feature list; -- there is a "Why" section explaining traditional approaches and their limitations; -- the high-level mechanism is explained BEFORE code sections; -- compilation/execution is AFTER code analysis; -- every kernel/BPF source and core header appears once in a complete ordinary Markdown fence; a concise user-space loader should also be complete, while a long loader may use focused excerpts that cover the normal public control flow; -- code, commands, output, versions, requirements, cleanup, and limits agree with the repository; -- sample output comes from the normal public workflow and contains no test-harness or migration/deprecation narration; -- the opening source link points only to the lesson directory; -- every published link is an absolute `https://github.com/...` URL; -- no local infrastructure, prompt, model, agent, or trace detail appears in public text. - -Check the required complete-source inventory with the bundled `sync-source-blocks.py`: always include kernel/BPF sources and core headers, and include user-space files only when the README presents them as complete. Then run `git diff --check`, the lesson build, its tests, the exact documented public command, and the relevant runtime test. Treat a functional run as a functional run rather than a benchmark. - -Read the finished pair once as an intermediate eBPF developer. If a concrete problem remains, give Opus a short defect list in the same session and let it revise the whole pair before returning. Keep prompts, responses, partial drafts, and failed runs; never delete real conversation or agent history. - -Stop after the reviewed local result unless the user asks to commit, push, or update a PR. +Stop with the local result unless the user explicitly asks to commit, push, or update a PR. diff --git a/.agents/skills/write-bpf-production-tutorial/agents/openai.yaml b/.agents/skills/write-bpf-production-tutorial/agents/openai.yaml index ad3f247f..1888061a 100644 --- a/.agents/skills/write-bpf-production-tutorial/agents/openai.yaml +++ b/.agents/skills/write-bpf-production-tutorial/agents/openai.yaml @@ -1,4 +1,4 @@ interface: display_name: "BPF Tutorial Writer" - short_description: "Design and test practical bilingual eBPF lessons" - default_prompt: "Use $write-bpf-production-tutorial to turn this eBPF example into a practical, tested bilingual tutorial." + short_description: "Let one Opus process revise one bilingual lesson" + default_prompt: "Use $write-bpf-production-tutorial to let one pinned Claude Opus process revise this tutorial's English and Chinese README pair paragraph by paragraph from the reader's perspective." diff --git a/.github/workflows/test-libbpf.yml b/.github/workflows/test-libbpf.yml index 2c77e5c9..b1eaae25 100644 --- a/.github/workflows/test-libbpf.yml +++ b/.github/workflows/test-libbpf.yml @@ -197,6 +197,22 @@ jobs: run: | make -C src/54-exec-image-inspector + - name: test 55 DNS-derived allowlist + run: | + make -C src/55-dns-egress + + - name: test 56 TC flow index + run: | + make -C src/56-tc-flow-index + + - name: test 57 OOM reclaim profiler + run: | + make -C src/57-oom-watch + + - name: test 60 AF_XDP dump + run: | + make -C src/60-afxdp-dump + - name: test features bpf_token run: | make -C src/features/bpf_token diff --git a/README.md b/README.md index 4504b160..bd53ce24 100644 --- a/README.md +++ b/README.md @@ -78,6 +78,8 @@ Networking: - [lesson 46-xdp-test](src/46-xdp-test/README.md) Building a High-Performance XDP Packet Generator - [lesson 50-tcx](src/50-tcx/README.md) Composable Traffic Control with TCX Links - [lesson 53-egress-pacer](src/53-egress-pacer/README.md) Building an Egress Pacer with BPF Qdisc +- [lesson 56-tc-flow-index](src/56-tc-flow-index/README.md) Building a Top-Flow Monitor with Dual Rbtree Indexing +- [lesson 60-afxdp-dump](src/60-afxdp-dump/README.md) High-Performance UDP Receiving with AF_XDP Tracing: @@ -90,6 +92,7 @@ Tracing: - [lesson 40-mysql](src/40-mysql/README.md) Using eBPF to Trace MySQL Queries - [lesson 48-energy](src/48-energy/README.md) Energy Monitoring for Process-Level Power Analysis - [lesson 52-fsession-latency](src/52-fsession-latency/README.md) Tracing Slow vfs_read Calls with fsession +- [lesson 57-oom-watch](src/57-oom-watch/README.md) Profiling Memory Reclaim Before an OOM Kill Security: @@ -102,6 +105,7 @@ Security: - [lesson 34-syscall](src/34-syscall/README.md) Modifying System Call Arguments with eBPF - [lesson 51-tcp-quarantine](src/51-tcp-quarantine/README.md) Precisely Isolating Established TCP Connections - [lesson 54-exec-image-inspector](src/54-exec-image-inspector/README.md) Inspecting the Executable Image After exec +- [lesson 55-dns-egress](src/55-dns-egress/README.md) Building a DNS-Derived IP Allowlist with cgroup BPF Features: diff --git a/README.zh.md b/README.zh.md index ca7a886e..31d34b60 100644 --- a/README.zh.md +++ b/README.zh.md @@ -77,6 +77,8 @@ GPU: - [lesson 46-xdp-test](src/46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](src/50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 - [lesson 53-egress-pacer](src/53-egress-pacer/README.zh.md) eBPF 教程：用 BPF Qdisc 实现出口限速 +- [lesson 56-tc-flow-index](src/56-tc-flow-index/README.zh.md) eBPF 实战教程：用双红黑树索引构建 Top-Flow 监控器 +- [lesson 60-afxdp-dump](src/60-afxdp-dump/README.zh.md) eBPF 实战教程：用 AF_XDP 高性能接收 UDP 报文 追踪: @@ -89,6 +91,7 @@ GPU: - [lesson 40-mysql](src/40-mysql/README.zh.md) 使用 eBPF 跟踪 MySQL 查询 - [lesson 48-energy](src/48-energy/README.zh.md) eBPF 教程：进程级能源监控与功耗分析 - [lesson 52-fsession-latency](src/52-fsession-latency/README.zh.md) eBPF 教程：使用 fsession 追踪慢速 vfs_read 调用 +- [lesson 57-oom-watch](src/57-oom-watch/README.zh.md) eBPF 实战教程：在 OOM Kill 之前分析内存回收 安全: @@ -101,6 +104,7 @@ GPU: - [lesson 34-syscall](src/34-syscall/README.zh.md) eBPF 开发实践：使用 eBPF 修改系统调用参数 - [lesson 51-tcp-quarantine](src/51-tcp-quarantine/README.zh.md) eBPF 教程：精准隔离已建立的 TCP 连接 - [lesson 54-exec-image-inspector](src/54-exec-image-inspector/README.zh.md) eBPF 教程：检查 exec 后实际安装的可执行镜像 +- [lesson 55-dns-egress](src/55-dns-egress/README.zh.md) eBPF 实战教程：用 cgroup BPF 构建基于 DNS 的 IP 允许列表 特性: diff --git a/TUTORIAL_IDEAS.md b/TUTORIAL_IDEAS.md index 9edbc077..2816f4f9 100644 --- a/TUTORIAL_IDEAS.md +++ b/TUTORIAL_IDEAS.md @@ -51,13 +51,13 @@ This lesson teaches one reusable architecture: BPF captures stable kernel facts, | 90 | ready | Container-aware service dependency map | Trace `connect`, accept/state, and cgroup identity for three local services; emit named edges, latency, and failures | Combines concepts nearest to tutorials 13, 14, and `src/cgroup`, then adds two-sided workload identity and topology | [Coroot node agent](https://github.com/coroot/coroot-node-agent), [Inspektor Gadget](https://github.com/inspektor-gadget/inspektor-gadget), [Hubble](https://github.com/cilium/hubble) | | 88 | ready | Signed BPF program verification | Create a key and certificate, trust it through the kernel keyring, load a signed program, then show unknown-key and tampered-object failures | Adds BPF deployment integrity and supply-chain verification | [Linux 6.18 BPF merge](https://github.com/torvalds/linux/commit/ae28ed4578e6d5a481e39c5a9827f27048661fdd), [bpftool](https://github.com/libbpf/bpftool) | | 87 | ready | io_uring request latency and async-punt diagnosis | Correlate submit, io-wq execution, and completion; report latency by opcode, batching, and punt ratio | Observes the io_uring lifecycle rather than block-device access alone | [uringscope](https://github.com/rch0wdhury/uringscope), [liburing](https://github.com/axboe/liburing) | -| 85 | research | OOM pre-kill profile and cgroup attribution | Trigger a bounded cgroup OOM, identify the victim and memory limit, and preserve a useful pre-kill profile | Complements allocation leak tracking with evidence from the failure moment | [OOMProf](https://github.com/parca-dev/oomprof), [Coroot node agent](https://github.com/coroot/coroot-node-agent) | +| 85 | covered | OOM pre-kill profile and cgroup attribution | Profile memcg reclaim latency and kernel stacks, then correlate the accumulated profile with the selected victim and its exit | Complements allocation leak tracking with evidence from the failure moment | [OOMProf](https://github.com/parca-dev/oomprof), [Coroot node agent](https://github.com/coroot/coroot-node-agent) | | 83 | research | AI-agent runtime activity audit | Run a fixture in one cgroup and correlate its process tree, file writes, network connections, and tool subprocesses into a timeline | Applies stable tracing hooks to a new security and provenance scenario | [AgentSight](https://github.com/agent-sight/agentsight), [Tetragon](https://github.com/cilium/tetragon), [Tracee](https://github.com/aquasecurity/tracee), [Falco](https://github.com/falcosecurity/falco) | | 82 | ready | `sk_lookup` transparent local service router | Route a virtual TCP or UDP address to one of two existing sockets with `bpf_sk_assign()` and demonstrate failover | Introduces a program type and socket-selection point absent from current examples | [Linux sk_lookup selftest](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/prog_tests/sk_lookup.c) | | 81 | research | LLM and MCP call timeline | Trace one local OpenAI-compatible streaming request and one MCP stdio tool call; report model, tool, TTFT, and total latency | Turns TLS and I/O tracing into an end-to-end agent scenario | [agtap](https://github.com/zhebrak/agtap), [OpenTelemetry eBPF Instrumentation](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation) | | 80 | research | Container-aware runtime policy | Attach a BPF LSM policy to selected workload identity and audit or deny one file or network action | Moves beyond a host-wide LSM example into workload identity and policy | [Tetragon](https://github.com/cilium/tetragon), [Tracee](https://github.com/aquasecurity/tracee), [Falco](https://github.com/falcosecurity/falco) | | 79 | ready | Sidecarless service traffic split | Use cgroup socket hooks to translate one service address to two local backends and expose the chosen backend in a map | Teaches socket-level service routing rather than the existing sockhash fast path | [Cilium](https://github.com/cilium/cilium), [Kmesh](https://github.com/kmesh-net/kmesh) | -| 78 | ready | DNS-aware egress policy with BPF netfilter or cgroup hooks | Permit one resolver or domain-derived address set, count policy decisions, and show one accepted and one rejected query | Adds a concrete egress-control scenario and a new attachment choice | [Cilium](https://github.com/cilium/cilium), [Linux BPF selftests](https://github.com/torvalds/linux/tree/master/tools/testing/selftests/bpf) | +| 78 | covered | DNS-derived IP allowlist with cgroup BPF | Correlate one domain's queries and responses, preserve A records for their TTL, and enforce the resulting IP allowlist at `connect4` | Adds a concrete egress-control scenario and a new attachment choice | [Cilium](https://github.com/cilium/cilium), [Linux BPF selftests](https://github.com/torvalds/linux/tree/master/tools/testing/selftests/bpf) | | 77 | research | OCI-packaged eBPF Gadget with workload enrichment | Package one small tracer as an OCI artifact, run it on a local container, and enrich kernel IDs with container names | Teaches portable distribution and enrichment instead of only local loading | [Inspektor Gadget](https://github.com/inspektor-gadget/inspektor-gadget), [bpfman](https://github.com/bpfman/bpfman) | | 76 | ready | XDP multi-buffer parsing with `bpf_xdp_pull_data()` | Place an L4 header across fragments, show the initial boundary, pull data, reacquire pointers, and parse it | Adds non-linear packet handling to the XDP series | [Linux XDP pull-data selftest](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/test_xdp_pull_data.c) | | 75 | research | Zero-code trace export to OpenTelemetry | Trace a small HTTP client/server pair without SDK changes and emit one OTLP span with process and network attributes | Connects BPF events to an open telemetry model | [OpenTelemetry eBPF Instrumentation](https://github.com/open-telemetry/opentelemetry-ebpf-instrumentation), [Odigos](https://github.com/odigos-io/odigos) | @@ -70,7 +70,7 @@ This lesson teaches one reusable architecture: BPF captures stable kernel facts, | 67 | deferred | Cross-language continuous profiling | Collect one mixed native and managed-runtime stack and symbolize it through build identity | Goes beyond stack sampling into unwinding and symbolization architecture | [OpenTelemetry eBPF Profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler), [Parca](https://github.com/parca-dev/parca), [Pyroscope](https://github.com/grafana/pyroscope) | | 67 | research | BPF-controlled io_uring loop with `io_uring_bpf_ops` | Let a BPF struct_ops callback inspect completions and submit one follow-up operation without returning to the ordinary userspace loop | Demonstrates a new Linux 7.1 control surface | [Linux io_uring BPF source](https://github.com/torvalds/linux/blob/master/io_uring/bpf-ops.c), [Linux io_uring BPF selftests](https://github.com/torvalds/linux/tree/master/tools/testing/selftests/bpf) | | 65 | deferred | GPU causal stall diagnosis | Correlate CUDA synchronization latency with CPU scheduling and block-I/O events in one controlled workload | Extends CUDA event tracing from isolated calls to a causal chain | [OpenTelemetry eBPF Profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler) | -| 64 | research | BPF object reference counting and rbtree traversal | Maintain ordered objects with explicit ownership, lookup, acquire, and release paths | Extends graph-object coverage beyond the qdisc list example | [Linux rbtree selftest](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/rbtree_search.c) | +| 64 | covered | BPF object reference counting and rbtree traversal | Index TC egress flows in identity and traffic-ordered rbtrees while both trees own the same refcounted BPF object | Extends graph-object coverage beyond the qdisc list example | [Linux rbtree selftest](https://github.com/torvalds/linux/blob/master/tools/testing/selftests/bpf/progs/rbtree_search.c) | | 63 | deferred | NCCL collective straggler timeline | Correlate collective duration, rank, CPU preemption, and retransmits across a two-rank fixture | Adds distributed GPU communication behavior | [OpenTelemetry eBPF Profiler](https://github.com/open-telemetry/opentelemetry-ebpf-profiler), [NCCL](https://github.com/NVIDIA/nccl) | | 58 | maintenance | Ring-buffer overwrite mode | Add a bounded producer/slow-consumer demonstration to the existing ring-buffer material | The mechanism is too small for a separate numbered lesson | [Linux BPF selftests](https://github.com/torvalds/linux/tree/master/tools/testing/selftests/bpf) | @@ -101,6 +101,10 @@ Keep these entries to prevent repeated proposals. | covered | `fsession` latency tracing | Active tutorial 52 | | covered | BPF qdisc with graph objects and list ownership | Active tutorial 53 | | covered | BPF task work and file-backed dynptr inspection after exec | Active tutorial 54 | +| covered | DNS-derived IP allowlist with query correlation and TTL enforcement | Active tutorial 55 | +| covered | BPF object, refcount, and dual-rbtree TC flow index | Active tutorial 56 | +| covered | Memcg reclaim latency and stack profiler correlated with OOM victims | Active tutorial 57 | +| covered | Basic single-buffer AF_XDP receive path and frame recycling | Active tutorial 60; multi-buffer and RX metadata remain a separate candidate | ## Decision history @@ -108,3 +112,4 @@ Keep these entries to prevent repeated proposals. - 2026-07-20: Selected the container-aware service dependency map as the highest overall value. It combines a common question, stable hooks, a clear visual result, broad reuse, and deterministic KVM execution. - 2026-07-20: Kept LLM/MCP tracing as the most timely candidate while scoring its protocol and runtime maintenance cost explicitly. - 2026-07-20: Removed Ingero as a primary source after its GitHub repository returned 404 during link validation; retained the GPU candidates with currently auditable OpenTelemetry profiler and NCCL sources. +- 2026-07-22: Marked the DNS-derived allowlist, OOM reclaim profiler, and BPF object/refcount/rbtree topics as covered by active tutorials 55–57. Recorded tutorial 60 as the basic AF_XDP prerequisite while keeping multi-buffer and RX metadata as a separate candidate. diff --git a/scripts/guideline_advance.md b/scripts/guideline_advance.md index 02ad761b..c0f55e1a 100644 --- a/scripts/guideline_advance.md +++ b/scripts/guideline_advance.md @@ -28,7 +28,7 @@ Provide an overview of the specific eBPF programs, tools, or features you'll dis Dive into the kernel-mode eBPF code and user-space code, focusing on high-level concepts rather than basic syntax. -Always include the full code as it is first. then break down the key parts. +Show the core kernel-mode eBPF program in full when it is central to the lesson, then break down the key parts. For user-space code, include only the functions and excerpts needed to understand loading, configuration, event handling, and cleanup. Link to the tutorial directory for the complete implementation. Try to avoid using too much list, make it more like a story. @@ -43,7 +43,7 @@ Follow the steps: Do not make them a list, make them some paragraphs, you can also quote some code snippets to explain the key parts of the code if needed, focus on the logic and features used in advanced eBPF development. Don't make it too long, but make sure it is informative enough and you explain everything a advanced eBPF developer wants to know. -3. Then briefly explain the user-space code +3. Then briefly explain the user-space code, using focused excerpts when they make the control flow easier to follow Aim to help readers grasp how the code works without getting bogged down in basic details. @@ -76,6 +76,7 @@ You should include the important references and resources that used in the tutor - **Focus on Advanced Concepts:** Assume readers have basic eBPF knowledge; skip elementary explanations. - **Engagement:** Encourage readers to think critically and engage with the material. - **Consistency:** Keep a consistent style and formatting throughout. -- **Code Formatting:** Ensure code snippets are well-formatted and highlight key parts. Do not change or simplify any of the code and commands, keep them as they are. +- **Punctuation:** Do not use em dashes, doubled Chinese em dashes, or en dashes as sentence punctuation. Use commas, semicolons, colons, parentheses, or separate sentences. +- **Code Formatting:** Ensure code snippets are well-formatted and highlight key parts. Keep every included excerpt and command faithful to the implementation. The tutorial does not need to embed the complete user-space loader. - **Proofreading:** Double-check for errors and ensure technical accuracy. - **Accessibility:** Make the content valuable for readers with advanced expertise, avoiding unnecessary simplifications. diff --git a/scripts/test_generate_toc.py b/scripts/test_generate_toc.py index 46093362..7b0c8ca5 100644 --- a/scripts/test_generate_toc.py +++ b/scripts/test_generate_toc.py @@ -16,6 +16,7 @@ "src/40-mysql", "src/48-energy", "src/52-fsession-latency", + "src/57-oom-watch", } diff --git a/src/55-dns-egress/.config b/src/55-dns-egress/.config new file mode 100644 index 00000000..f24197e5 --- /dev/null +++ b/src/55-dns-egress/.config @@ -0,0 +1,10 @@ +level=Depth +type=Security +kernel_min=5.12 +kernel_min_basis=required-feature +architectures=x86_64 +btf=required +kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_CGROUP_BPF=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_INET=y +hardware=none +root=required +test_status=ci-build diff --git a/src/55-dns-egress/.gitignore b/src/55-dns-egress/.gitignore new file mode 100644 index 00000000..84cad9ba --- /dev/null +++ b/src/55-dns-egress/.gitignore @@ -0,0 +1,2 @@ +.output/ +dns_egress diff --git a/src/55-dns-egress/Makefile b/src/55-dns-egress/Makefile new file mode 100644 index 00000000..7042a969 --- /dev/null +++ b/src/55-dns-egress/Makefile @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +OUTPUT := .output +CLANG ?= clang +LIBBPF_SRC := $(abspath ../third_party/libbpf/src) +BPFTOOL_SRC := $(abspath ../third_party/bpftool/src) +LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a) +BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool) +BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool +ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \ + | sed 's/arm.*/arm/' \ + | sed 's/aarch64/arm64/' \ + | sed 's/ppc64le/powerpc/' \ + | sed 's/mips.*/mips/' \ + | sed 's/riscv64/riscv/' \ + | sed 's/loongarch64/loongarch/') +VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h +INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I. +CFLAGS := -g -Wall -Wextra -D_GNU_SOURCE +ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS) +APPS = dns_egress + +CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - &1 \ + | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') + +ifeq ($(V),1) + Q = + msg = +else + Q = @ + msg = @printf ' %-8s %s%s\n' "$(1)" \ + "$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \ + "$(if $(3), $(3))"; + MAKEFLAGS += --no-print-directory +endif + +define allow-override + $(if $(or $(findstring environment,$(origin $(1))),\ + $(findstring command line,$(origin $(1)))),,\ + $(eval $(1) = $(2))) +endef + +$(call allow-override,CC,$(CROSS_COMPILE)cc) + +.PHONY: all clean test +all: $(APPS) + +clean: + $(call msg,CLEAN) + $(Q)rm -rf $(OUTPUT) $(APPS) + +test: all + @if [ "$$(id -u)" -ne 0 ]; then \ + echo "test requires root; run inside the documented KVM guest" >&2; \ + exit 1; \ + fi + python3 tests/test_dns_egress.py ./dns_egress + +$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT): + $(call msg,MKDIR,$@) + $(Q)mkdir -p $@ + +$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf + $(call msg,LIB,$@) + $(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \ + OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \ + INCLUDEDIR= LIBDIR= UAPIDIR= install + +$(BPFTOOL): | $(BPFTOOL_OUTPUT) + $(call msg,BPFTOOL,$@) + $(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap + +$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL) + $(call msg,BPF,$@) + $(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \ + $(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \ + -c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + $(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + +$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL) + $(call msg,GEN-SKEL,$@) + $(Q)$(BPFTOOL) gen skeleton $< > $@ + +$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h + +$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT) + $(call msg,CC,$@) + $(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@ + +$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT) + $(call msg,BINARY,$@) + $(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@ + +.DELETE_ON_ERROR: +.SECONDARY: diff --git a/src/55-dns-egress/README.md b/src/55-dns-egress/README.md new file mode 100644 index 00000000..c2d51ae5 --- /dev/null +++ b/src/55-dns-egress/README.md @@ -0,0 +1,1161 @@ +# eBPF Tutorial: Building a DNS-Derived IP Allowlist with cgroup BPF + +Suppose you want a containerized service to communicate only with `api.example.com`. The obvious approach is an IP allowlist, but DNS answers change constantly and TTLs expire. By the time your application calls `connect()`, the kernel sees only a raw IP address with no trace of the domain that produced it. The missing piece is a short-lived correlation between the DNS lookup and the subsequent TCP connection. + +This tutorial builds that correlation. We watch DNS traffic for a specific domain, extract IP addresses only from valid query-response pairs, and permit connections until the TTL expires. The result is a minimal but complete policy tool that demonstrates how cgroup BPF programs can share state across multiple kernel hooks. + +> Complete source code: + +## The Problem: DNS and Connect Live in Different Worlds + +When your application resolves `api.example.com`, the DNS response contains an IP address and a TTL. But this information lives only in userspace: the resolver library caches it, your application calls `connect()` with the IP, and the kernel has no idea where that IP came from. This disconnect creates a fundamental security gap. + +Consider a container that should only talk to your backend API. Traditional firewalls can block destination IPs, but they cannot enforce a policy like "only connect to IPs that came from resolving api.example.com within the last 60 seconds." Kernel-level networking and application-level DNS resolution operate in complete isolation from each other. + +cgroup BPF bridges this gap. By attaching programs to both packet hooks and socket-address hooks on the same cgroup, we can observe DNS traffic and later enforce connection policy based on what we learned. The key insight: cgroup BPF programs share state through maps, enabling a trust chain from DNS query through response to eventual connection. + +## Architecture Overview + +eBPF lets us run verified programs at multiple kernel hooks and share state between them through maps. cgroup BPF ties these hooks to a workload: packet programs inspect traffic entering and leaving a cgroup, while socket-address programs can accept or reject connections before they are established. + +Our tool attaches three programs to the same cgroup: + +1. **`cgroup_skb/egress`** watches outgoing DNS queries and records which responses we expect +2. **`cgroup_skb/ingress`** validates incoming DNS responses and extracts IP addresses only from replies that match a pending query +3. **`cgroup/connect4`** makes the final decision, allowing connections only to IPs learned from valid DNS responses and only while their TTL remains valid + +Here is how a successful resolution flows through the system. Your application sends an A query for `lab.test`. The egress hook saves a correlation key containing four fields: resolver address, client address, client UDP port, and DNS transaction ID. This record expires after five seconds. When the reply arrives, the ingress hook reconstructs the same key, verifies the response, and extracts the IP address along with its TTL. A subsequent `connect()` to that address succeeds, but only while the TTL is valid. + +The pending-query record establishes the trust boundary. An unsolicited response finds no matching key in the map. A response with the wrong transaction ID looks for a different key and finds nothing. Neither can pollute the allowlist. Even if an entry lingers in the LRU map, the connect hook re-checks the expiration timestamp, so addresses naturally stop working when their DNS TTL runs out. + +## Data Structures + +Before looking at the code, let's examine the data structures that enable this design. The shared header defines DNS protocol layouts and the events we report to userspace: + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __DNS_EGRESS_H +#define __DNS_EGRESS_H + +#define DNS_QNAME_MAX 64 + +enum dns_egress_event_type { + DNS_LEARNED = 1, + DNS_ALLOWED = 2, + DNS_DENIED = 3, + DNS_EXPIRED = 4, +}; + +struct dns_header { + unsigned short id; + unsigned short flags; + unsigned short questions; + unsigned short answers; + unsigned short authorities; + unsigned short additionals; +} __attribute__((packed)); + +struct dns_question { + unsigned short type; + unsigned short class; +} __attribute__((packed)); + +struct dns_a_answer { + unsigned short name; + unsigned short type; + unsigned short class; + unsigned int ttl; + unsigned short address_length; + unsigned int address; +} __attribute__((packed)); + +struct dns_egress_event { + unsigned long long timestamp_ns; + unsigned long long expires_ns; + unsigned int type; + unsigned int pid; + unsigned int ip4; + unsigned int ttl_seconds; +}; + +#endif /* __DNS_EGRESS_H */ +``` + +The protocol structs use the `packed` attribute because they map directly to bytes on the wire, with no padding allowed. `dns_header` corresponds to the 12-byte DNS header that starts every query and response. `dns_question` follows the question name, which uses length-prefixed labels. `dns_a_answer` expects the common compressed format where the name pointer is `0xc00c`, pointing back to the question section. + +Each event sent to userspace carries both the DNS TTL in seconds and the absolute expiration timestamp in nanoseconds. Userspace prints the human-readable TTL for logging, while the BPF programs use the monotonic timestamp for decisions. This separation keeps kernel logic simple: no time format conversions in BPF code. + +## BPF Programs: Complete Implementation + +Here is the complete kernel-side implementation. It is longer than some of our examples, but each piece has a well-defined responsibility: + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "dns_egress.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define IPPROTO_UDP 17 +#define IPPROTO_TCP 6 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define DNS_QUERY_LIFETIME_NS (5ULL * 1000000000ULL) + +const volatile __u32 target_tgid; +const volatile __u32 dns_server_ip; +const volatile __u16 dns_server_port = 53; +const volatile __u16 protected_tcp_port = 443; +const volatile __u32 configured_qname_length; +const volatile unsigned char configured_qname[DNS_QNAME_MAX]; + +struct dns_state { + __u64 expires_ns; + __u32 ttl_seconds; + __u32 pad; + __u64 expired_reported; +}; + +struct dns_query_key { + __u32 server_ip; + __u32 client_ip; + __u16 client_port; + __u16 transaction_id; +}; + +struct dns_query_state { + __u64 expires_ns; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, struct dns_query_key); + __type(value, struct dns_query_state); +} pending_queries SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct dns_state); +} allowed_ips SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +static __always_inline void emit_event(__u32 type, __u32 ip4, + __u32 ttl_seconds, __u64 expires_ns) +{ + struct dns_egress_event *event; + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return; + event->timestamp_ns = bpf_ktime_get_ns(); + event->expires_ns = expires_ns; + event->type = type; + event->pid = bpf_get_current_pid_tgid() >> 32; + event->ip4 = ip4; + event->ttl_seconds = ttl_seconds; + bpf_ringbuf_submit(event, 0); +} + +static __noinline bool matches_qname(struct __sk_buff *skb, __u32 offset) +{ + unsigned char byte; + + if (!configured_qname_length || configured_qname_length > DNS_QNAME_MAX) + return false; +#pragma clang loop unroll(disable) + for (int i = 0; i < DNS_QNAME_MAX; i++) { + if (i >= configured_qname_length) + break; + if (bpf_skb_load_bytes(skb, offset + i, &byte, sizeof(byte)) || + byte != configured_qname[i]) + return false; + } + return true; +} + +static __always_inline bool parse_query_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.daddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.dest) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + key->server_ip = ip.daddr; + key->client_ip = ip.saddr; + key->client_port = udp.source; + return true; +} + +static __always_inline bool parse_dns_query(struct __sk_buff *skb, + __u32 dns_offset, + struct dns_query_key *key) +{ + struct dns_question question; + struct dns_header header; + __u16 flags; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + flags = bpf_ntohs(header.flags); + if ((flags & 0xf800) || bpf_ntohs(header.questions) != 1) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + key->transaction_id = header.id; + return true; +} + +SEC("cgroup_skb/egress") +int record_dns_query(struct __sk_buff *skb) +{ + struct dns_query_state state = { + .expires_ns = bpf_ktime_get_ns() + DNS_QUERY_LIFETIME_NS, + }; + struct dns_query_key key = {}; + __u32 dns_offset; + + if (!parse_query_transport(skb, &dns_offset, &key) || + !parse_dns_query(skb, dns_offset, &key)) + return 1; + bpf_map_update_elem(&pending_queries, &key, &state, BPF_ANY); + return 1; +} + +static __always_inline bool parse_response_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *query_key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.saddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.source) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + query_key->server_ip = ip.saddr; + query_key->client_ip = ip.daddr; + query_key->client_port = udp.dest; + return true; +} + +static __always_inline bool pending_query_is_live( + struct dns_query_key *query_key) +{ + struct dns_query_state *query; + + query = bpf_map_lookup_elem(&pending_queries, query_key); + if (!query) + return false; + if (bpf_ktime_get_ns() >= query->expires_ns) { + bpf_map_delete_elem(&pending_queries, query_key); + return false; + } + return true; +} + +static __always_inline bool parse_response_question( + struct __sk_buff *skb, __u32 dns_offset, + struct dns_query_key *query_key, __u32 *answer_offset) +{ + struct dns_question question; + struct dns_header header; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + query_key->transaction_id = header.id; + if (!pending_query_is_live(query_key)) + return false; + if ((bpf_ntohs(header.flags) & 0xf80f) != 0x8000 || + bpf_ntohs(header.questions) != 1 || !bpf_ntohs(header.answers)) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + *answer_offset = dns_offset + sizeof(question); + return true; +} + +static __always_inline bool parse_direct_a_answer(struct __sk_buff *skb, + __u32 answer_offset, + __u32 *key, __u32 *ttl) +{ + struct dns_a_answer answer; + + if (bpf_skb_load_bytes(skb, answer_offset, &answer, sizeof(answer)) || + bpf_ntohs(answer.name) != 0xc00c || bpf_ntohs(answer.type) != 1 || + bpf_ntohs(answer.class) != 1 || + bpf_ntohs(answer.address_length) != 4) + return false; + *key = answer.address; + *ttl = bpf_ntohl(answer.ttl); + return *ttl && *ttl <= 86400; +} + +SEC("cgroup_skb/ingress") +int learn_dns_answer(struct __sk_buff *skb) +{ + struct dns_query_key query_key = {}; + struct dns_state state = {}; + __u64 ttl_ns, expires; + __u32 dns_offset, answer_offset; + __u32 key, ttl; + + if (!parse_response_transport(skb, &dns_offset, &query_key) || + !parse_response_question(skb, dns_offset, &query_key, + &answer_offset) || + !parse_direct_a_answer(skb, answer_offset, &key, &ttl)) + return 1; + bpf_map_delete_elem(&pending_queries, &query_key); + ttl_ns = (__u64)ttl * 1000000000ULL; + expires = bpf_ktime_get_ns() + ttl_ns; + state.expires_ns = expires; + state.ttl_seconds = ttl; + if (bpf_map_update_elem(&allowed_ips, &key, &state, BPF_ANY)) + return 1; + emit_event(DNS_LEARNED, key, ttl, expires); + return 1; +} + +SEC("cgroup/connect4") +int enforce_dns_policy(struct bpf_sock_addr *ctx) +{ + struct dns_state *state; + __u64 expires = 0; + __u32 ip4; + __u32 ttl = 0; + + if ((target_tgid && + (__u32)(bpf_get_current_pid_tgid() >> 32) != target_tgid) || + ctx->protocol != IPPROTO_TCP || + bpf_ntohs((__u16)ctx->user_port) != protected_tcp_port) + return 1; + + ip4 = ctx->user_ip4; + state = bpf_map_lookup_elem(&allowed_ips, &ip4); + if (state) { + expires = state->expires_ns; + ttl = state->ttl_seconds; + if (bpf_ktime_get_ns() < expires) { + emit_event(DNS_ALLOWED, ip4, ttl, expires); + return 1; + } + if (__sync_val_compare_and_swap(&state->expired_reported, 0, 1) == 0) + emit_event(DNS_EXPIRED, ip4, ttl, expires); + } + emit_event(DNS_DENIED, ip4, ttl, expires); + return 0; +} +``` + +### Map Design + +The program uses three BPF maps, each serving a distinct role in the trust chain. + +**`pending_queries`** is an LRU hash that holds correlation state for DNS queries. The key combines four fields: server IP, client IP, client port, and transaction ID. Together, they uniquely identify a query-response pair. The value stores only the expiration timestamp, which is all we need for timing validation. LRU eviction keeps memory bounded even under heavy DNS traffic. + +**`allowed_ips`** is also an LRU hash, keyed simply by IPv4 address. The value contains the expiration timestamp, the original TTL in seconds (for logging), padding for alignment, and a flag indicating whether we have already reported expiration. This flag prevents duplicate "expired" events when multiple threads race on a stale entry. + +**`events`** is a 256 KB ring buffer for sending notifications to userspace. At this size, it can hold thousands of events without blocking the kernel path. Ring buffers are the modern replacement for perf buffers: more efficient and with better ordering guarantees. + +### The Egress Path: Recording Pending Queries + +When a packet leaves the cgroup, `record_dns_query` executes. The function first validates the transport layer through `parse_query_transport`, which loads the IP header, confirms IPv4 with UDP protocol, rejects fragmented packets (which would require reassembly logic we don't implement), and verifies the destination matches our configured resolver. If all checks pass, it calculates where DNS data begins and populates the correlation key. + +Next, `parse_dns_query` validates the DNS layer. It checks that the flags indicate a standard query (not a response), that exactly one question exists, and that the question name matches our configured domain. The question type and class must be A (address) and IN (internet). Only after all validation passes does it extract the transaction ID and insert the record into `pending_queries`. + +Note that `record_dns_query` always returns 1, telling the kernel to continue processing the packet normally. We are observing, not blocking. DNS queries flow through unchanged. + +### The Ingress Path: Learning from Responses + +The ingress program `learn_dns_answer` reverses the perspective. Now we examine packets arriving from the resolver, so `parse_response_transport` checks that the *source* (not destination) matches the resolver IP and port. The correlation key gets populated with the same fields, but from the response's viewpoint. + +The critical security check happens in `pending_query_is_live`. This function looks up the correlation key in `pending_queries`. If no entry exists, meaning we never saw a matching query, the response is rejected. If an entry exists but has expired, we delete it and reject the response. Only responses that match a live pending query proceed. + +After confirming we have a legitimate response, `parse_response_question` validates the DNS header. It checks that flags indicate a successful response with no errors, that exactly one question matches our domain, and that at least one answer exists. Then `parse_direct_a_answer` extracts the first A record, requiring the common `0xc00c` compressed name format, correct type and class, and a TTL between 1 and 86400 seconds. + +When validation passes, the pending query is deleted (it has been consumed), and the IP address is added to `allowed_ips` with an expiration based on the DNS TTL. An `emit_event` call sends a `DNS_LEARNED` notification to userspace. + +### The Connect Path: Enforcing Policy + +`enforce_dns_policy` attaches to `cgroup/connect4`, running before every IPv4 TCP connection attempt. The function first applies filtering: if we are targeting a specific process and this is not it, allow the connection. If it is not TCP or not the protected port, allow it. These early returns minimize overhead for irrelevant traffic. + +For connections requiring policy enforcement, we look up the destination IP in `allowed_ips`. If found and not expired, we emit `DNS_ALLOWED` and return 1 (allow). If found but expired, we use an atomic compare-and-swap on `expired_reported` to emit exactly one `DNS_EXPIRED` event even under concurrent access. This atomic operation arrived in Linux 5.12, which sets the tool's minimum kernel version. + +If the IP is not in the map or is expired, we emit `DNS_DENIED` and return 0. The kernel translates a return value of 0 into `EPERM`, and the application's `connect()` fails immediately. + +## User-Space Program + +The user-space program configures the read-only BPF data, attaches all three programs to a cgroup, and processes ring-buffer events. It also includes a self-test demo mode that exercises the complete trust chain. + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dns_egress.h" +#include "dns_egress.skel.h" + +#define DEMO_DNS_PORT 15353 +#define DEMO_TCP_PORT 19090 +#define DNS_ID 0x4b1d + +struct options { + const char *cgroup_path; + const char *domain; + const char *dns_server; + unsigned int port; + unsigned int dns_port; + unsigned int duration_seconds; + bool demo; +}; + +struct dns_runtime { + struct dns_egress_bpf *skel; + struct bpf_link *query_link; + struct bpf_link *ingress_link; + struct bpf_link *connect_link; + struct ring_buffer *ring; + int cgroup_fd; +}; + +static int event_counts[5]; +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int handle_event(void *ctx, void *data, size_t size) +{ + const struct dns_egress_event *event = data; + char address[INET_ADDRSTRLEN]; + const char *name; + + (void)ctx; + if (size != sizeof(*event) || event->type > DNS_EXPIRED) + return 0; + event_counts[event->type]++; + inet_ntop(AF_INET, &event->ip4, address, sizeof(address)); + name = event->type == DNS_LEARNED ? "learned" : + event->type == DNS_ALLOWED ? "allowed" : + event->type == DNS_DENIED ? "denied" : "expired"; + printf("event=%s pid=%u ip=%s ttl=%u\n", name, event->pid, address, + event->ttl_seconds); + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --cgroup PATH --domain NAME --dns-server IPV4 [--port PORT] [--dns-port PORT] [--duration SEC]\n" + " %s --demo\n", program, program); +} + +static int apply_option(int option, const char *program, + struct options *options) +{ + switch (option) { + case 'c': options->cgroup_path = optarg; return 0; + case 'n': options->domain = optarg; return 0; + case 'r': options->dns_server = optarg; return 0; + case 'p': + return parse_uint(optarg, 65535, &options->port) || !options->port ? + -1 : 0; + case 's': + return parse_uint(optarg, 65535, &options->dns_port) || + !options->dns_port ? -1 : 0; + case 'd': + return parse_uint(optarg, 86400, &options->duration_seconds); + case 'D': options->demo = true; return 0; + case 'h': usage(program); exit(0); + default: return -1; + } +} + +static int finish_options(struct options *options) +{ + if (!options->demo) + return options->cgroup_path && options->domain && + options->dns_server ? 0 : -1; + if (options->cgroup_path || options->domain || options->dns_server) + return -1; + options->cgroup_path = "/sys/fs/cgroup"; + options->domain = "lab.test"; + options->dns_server = "127.0.0.1"; + options->port = DEMO_TCP_PORT; + options->dns_port = DEMO_DNS_PORT; + return 0; +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "cgroup", required_argument, NULL, 'c' }, + { "domain", required_argument, NULL, 'n' }, + { "dns-server", required_argument, NULL, 'r' }, + { "port", required_argument, NULL, 'p' }, + { "dns-port", required_argument, NULL, 's' }, + { "duration", required_argument, NULL, 'd' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "c:n:r:p:s:d:Dh", long_options, + NULL)) != -1) + if (apply_option(option, argv[0], options)) + return -1; + return optind == argc ? finish_options(options) : -1; +} + +static int encode_qname(const char *domain, unsigned char output[DNS_QNAME_MAX], + unsigned int *output_length) +{ + const char *label = domain; + unsigned int used = 0; + + if (!*domain) + return -1; + while (*label) { + const char *dot = strchr(label, '.'); + size_t length = dot ? (size_t)(dot - label) : strlen(label); + + if (!length || length > 63 || used + length + 2 > DNS_QNAME_MAX) + return -1; + output[used++] = length; + memcpy(output + used, label, length); + used += length; + if (!dot) + break; + label = dot + 1; + if (!*label) + break; + } + output[used++] = 0; + *output_length = used; + return 0; +} + +static int bind_udp(struct sockaddr_in *address) +{ + socklen_t length = sizeof(*address); + int fd; + + fd = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (bind(fd, (struct sockaddr *)address, sizeof(*address)) || + getsockname(fd, (struct sockaddr *)address, &length)) { + close(fd); + return -1; + } + return fd; +} + +static int begin_dns_exchange(int server, int client, + struct sockaddr_in *server_address, + const unsigned char *qname, + unsigned int qname_length, + unsigned char message[512], + size_t *query_length, + struct sockaddr_in *client_address) +{ + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(*client_address); + ssize_t received; + + memset(message, 0, 512); + header->id = htons(DNS_ID); + header->flags = htons(0x0100); + header->questions = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + *query_length = sizeof(*header) + qname_length + sizeof(question); + if (sendto(client, message, *query_length, 0, + (struct sockaddr *)server_address, sizeof(*server_address)) != + (ssize_t)*query_length) + return -1; + received = recvfrom(server, message, 512, 0, + (struct sockaddr *)client_address, &address_length); + return received == (ssize_t)*query_length ? 0 : -1; +} + +static int send_dns_answer(int server, int client, + struct sockaddr_in *client_address, + unsigned char message[512], size_t query_length, + unsigned short transaction_id, + unsigned int ttl_seconds) +{ + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(ttl_seconds), + .address_length = htons(4), + }; + struct dns_header *header = (void *)message; + + if (query_length + sizeof(answer) > 512) + return -1; + header->id = htons(transaction_id); + header->flags = htons(0x8180); + header->answers = htons(1); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + query_length, &answer, sizeof(answer)); + if (sendto(server, message, query_length + sizeof(answer), 0, + (struct sockaddr *)client_address, sizeof(*client_address)) != + (ssize_t)(query_length + sizeof(answer))) + return -1; + return recv(client, message, 512, 0) == + (ssize_t)(query_length + sizeof(answer)) ? 0 : -1; +} + +static int send_unsolicited_dns(int server, int client, + const unsigned char *qname, + unsigned int qname_length) +{ + unsigned char message[512] = {}; + struct sockaddr_in client_address; + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(30), + .address_length = htons(4), + }; + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(client_address); + size_t message_length; + + if (getsockname(client, (struct sockaddr *)&client_address, + &address_length)) + return -1; + header->id = htons(DNS_ID + 1); + header->flags = htons(0x8180); + header->questions = htons(1); + header->answers = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + message_length = sizeof(*header) + qname_length + sizeof(question); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + message_length, &answer, sizeof(answer)); + message_length += sizeof(answer); + if (sendto(server, message, message_length, 0, + (struct sockaddr *)&client_address, address_length) != + (ssize_t)message_length) + return -1; + return recv(client, message, sizeof(message), 0) == + (ssize_t)message_length ? 0 : -1; +} + +static int create_tcp_listener(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int one = 1; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + if (bind(fd, (struct sockaddr *)&address, sizeof(address)) || + listen(fd, 4)) { + close(fd); + return -1; + } + return fd; +} + +static int connect_tcp(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int saved_errno; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (!connect(fd, (struct sockaddr *)&address, sizeof(address))) + return fd; + saved_errno = errno; + close(fd); + errno = saved_errno; + return -1; +} + +static int complete_tcp(int listener, int client) +{ + int accepted = accept4(listener, NULL, NULL, SOCK_CLOEXEC); + char byte = 'x'; + int result = 0; + + if (accepted < 0) { + close(client); + return -1; + } + if (write(accepted, &byte, 1) != 1 || read(client, &byte, 1) != 1) + result = -1; + close(accepted); + close(client); + return result; +} + +static int poll_demo_events(struct ring_buffer *ring) +{ + int result = ring_buffer__poll(ring, 100); + + return result < 0 && result != -EINTR ? -1 : 0; +} + +static int expect_blocked_connect(struct ring_buffer *ring, + unsigned int port, const char *step) +{ + int client; + + errno = 0; + client = connect_tcp(port); + if (client >= 0) { + close(client); + return -1; + } + if (errno != EPERM || poll_demo_events(ring)) + return -1; + printf("demo step=%s result=blocked\n", step); + return 0; +} + +static int expect_allowed_connect(struct ring_buffer *ring, int listener, + unsigned int port) +{ + int client = connect_tcp(port); + + if (client < 0 || complete_tcp(listener, client)) + return -1; + if (poll_demo_events(ring)) + return -1; + printf("demo step=live-answer result=allowed\n"); + return 0; +} + +static int expected_demo_events(void) +{ + return event_counts[DNS_LEARNED] == 1 && + event_counts[DNS_ALLOWED] == 1 && + event_counts[DNS_DENIED] == 4 && + event_counts[DNS_EXPIRED] == 1 ? 0 : -1; +} + +struct demo_context { + struct sockaddr_in server_address; + struct sockaddr_in response_client_address; + unsigned char dns_message[512]; + size_t query_length; + int dns_server; + int dns_client; + int listener; +}; + +static int open_demo_sockets(struct demo_context *demo, + const struct options *options) +{ + struct sockaddr_in client_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + + demo->server_address = (struct sockaddr_in) { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(options->dns_port), + }; + demo->dns_server = bind_udp(&demo->server_address); + demo->dns_client = bind_udp(&client_address); + demo->listener = create_tcp_listener(options->port); + return demo->dns_server < 0 || demo->dns_client < 0 || demo->listener < 0 ? + -1 : 0; +} + +static int test_rejected_dns_answers(struct ring_buffer *ring, + const struct options *options, + const unsigned char *qname, + unsigned int qname_length, + struct demo_context *demo) +{ + if (expect_blocked_connect(ring, options->port, "before-dns")) + return -1; + if (send_unsolicited_dns(demo->dns_server, demo->dns_client, qname, + qname_length) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, + "unsolicited-response")) + return -1; + if (begin_dns_exchange(demo->dns_server, demo->dns_client, + &demo->server_address, qname, qname_length, + demo->dns_message, &demo->query_length, + &demo->response_client_address) || + send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID + 1, 30) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "wrong-transaction-id")) + return -1; + return 0; +} + +static int test_live_and_expired_answer(struct ring_buffer *ring, + const struct options *options, + struct demo_context *demo) +{ + struct timespec wait_time = { .tv_sec = 1, .tv_nsec = 300000000 }; + + if (send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID, 1) || + poll_demo_events(ring) || + expect_allowed_connect(ring, demo->listener, options->port)) + return -1; + nanosleep(&wait_time, NULL); + if (poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "expired-answer")) + return -1; + return expected_demo_events(); +} + +static void close_demo_sockets(struct demo_context *demo) +{ + if (demo->listener >= 0) + close(demo->listener); + if (demo->dns_client >= 0) + close(demo->dns_client); + if (demo->dns_server >= 0) + close(demo->dns_server); +} + +static int run_demo(struct ring_buffer *ring, const struct options *options, + const unsigned char *qname, unsigned int qname_length) +{ + struct demo_context demo = { + .dns_server = -1, + .dns_client = -1, + .listener = -1, + }; + int err; + + err = open_demo_sockets(&demo, options); + if (!err) + err = test_rejected_dns_answers(ring, options, qname, + qname_length, &demo); + if (!err) + err = test_live_and_expired_answer(ring, options, &demo); + close_demo_sockets(&demo); + return err; +} + +static bool link_failed(struct bpf_link **link) +{ + if (!libbpf_get_error(*link)) + return false; + *link = NULL; + return true; +} + +static int prepare_runtime(struct dns_runtime *runtime, + const struct options *options, + const struct in_addr *dns_server, + const unsigned char *qname, + unsigned int qname_length) +{ + bool failed; + + runtime->cgroup_fd = open(options->cgroup_path, + O_RDONLY | O_DIRECTORY | O_CLOEXEC); + if (runtime->cgroup_fd < 0) { + fprintf(stderr, "failed to open cgroup %s: %s\n", + options->cgroup_path, strerror(errno)); + return -1; + } + runtime->skel = dns_egress_bpf__open(); + if (!runtime->skel) + return -1; + runtime->skel->rodata->target_tgid = options->demo ? getpid() : 0; + runtime->skel->rodata->dns_server_ip = dns_server->s_addr; + runtime->skel->rodata->dns_server_port = options->dns_port; + runtime->skel->rodata->protected_tcp_port = options->port; + runtime->skel->rodata->configured_qname_length = qname_length; + memcpy((void *)runtime->skel->rodata->configured_qname, qname, + qname_length); + if (dns_egress_bpf__load(runtime->skel)) { + fprintf(stderr, "failed to load DNS egress BPF programs\n"); + return -1; + } + runtime->query_link = bpf_program__attach_cgroup( + runtime->skel->progs.record_dns_query, runtime->cgroup_fd); + runtime->ingress_link = bpf_program__attach_cgroup( + runtime->skel->progs.learn_dns_answer, runtime->cgroup_fd); + runtime->connect_link = bpf_program__attach_cgroup( + runtime->skel->progs.enforce_dns_policy, runtime->cgroup_fd); + failed = link_failed(&runtime->query_link); + failed |= link_failed(&runtime->ingress_link); + failed |= link_failed(&runtime->connect_link); + if (failed) { + fprintf(stderr, "failed to attach programs to cgroup %s\n", + options->cgroup_path); + return -1; + } + runtime->ring = ring_buffer__new( + bpf_map__fd(runtime->skel->maps.events), handle_event, NULL, NULL); + return runtime->ring ? 0 : -1; +} + +static int poll_policy_events(struct ring_buffer *ring, + unsigned int duration_seconds) +{ + unsigned long long deadline = 0; + + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + if (duration_seconds) + deadline = monotonic_ns() + + (unsigned long long)duration_seconds * 1000000000ULL; + while (!stop && (!deadline || monotonic_ns() < deadline)) { + int result = ring_buffer__poll(ring, 100); + + if (result < 0 && result != -EINTR) { + fprintf(stderr, "ring buffer poll failed: %d\n", result); + return -1; + } + } + return 0; +} + +static void destroy_runtime(struct dns_runtime *runtime) +{ + ring_buffer__free(runtime->ring); + bpf_link__destroy(runtime->connect_link); + bpf_link__destroy(runtime->ingress_link); + bpf_link__destroy(runtime->query_link); + if (runtime->cgroup_fd >= 0) + close(runtime->cgroup_fd); + dns_egress_bpf__destroy(runtime->skel); +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 443, .dns_port = 53 }; + struct dns_runtime runtime = { .cgroup_fd = -1 }; + struct in_addr dns_server = {}; + unsigned char qname[DNS_QNAME_MAX] = {}; + unsigned int qname_length = 0; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options) || + encode_qname(options.domain, qname, &qname_length) || + inet_pton(AF_INET, options.dns_server, &dns_server) != 1) { + usage(argv[0]); + return 2; + } + if (prepare_runtime(&runtime, &options, &dns_server, qname, + qname_length)) + goto cleanup; + + printf("dns-egress attached cgroup=%s domain=%s resolver=%s tcp_port=%u dns_port=%u\n", + options.cgroup_path, options.domain, options.dns_server, + options.port, options.dns_port); + if (options.demo) { + if (run_demo(runtime.ring, &options, qname, qname_length)) + goto cleanup; + } else if (poll_policy_events(runtime.ring, options.duration_seconds)) + goto cleanup; + err = 0; + +cleanup: + destroy_runtime(&runtime); + return err; +} +``` + +### User-Space Control Flow + +The user-space program follows a clear initialization sequence. First, `parse_options` handles command-line arguments, validates inputs, and sets defaults. Demo mode automatically configures loopback addresses and non-standard ports to avoid conflicts with real DNS and web traffic. + +The `encode_qname` function converts a human-readable domain name like `lab.test` into DNS wire format: `\x03lab\x04test\x00`. Each label starts with a length byte followed by the label content, terminated by a zero byte. This encoding happens once at startup and is written into the BPF skeleton's `rodata` section, where the verifier treats it as a constant. + +`prepare_runtime` ties everything together. It opens the target cgroup directory, opens the BPF skeleton, configures all `rodata` values (resolver IP, ports, domain name), loads the BPF programs, and attaches each program to the cgroup. The three separate links allow independent attachment and detachment. Finally, it creates a ring buffer consumer that calls `handle_event` for each kernel notification. + +In normal mode, `poll_policy_events` loops on the ring buffer until the duration expires or a signal arrives. Each event is printed with its IP address, PID, TTL, and type. Demo mode runs `run_demo` instead, exercising the complete trust chain with synthetic DNS traffic and TCP connections. + +### Demo Mode: Validating Security Properties + +Demo mode functions as both an integration test and a demonstration of the security model. It runs entirely on loopback using non-standard ports (15353 for DNS, 19090 for TCP) to avoid interfering with real services. + +The test sequence begins by verifying that connections are blocked before any DNS traffic occurs. It then sends an unsolicited DNS response, one that arrives without a preceding query. The BPF program rejects this because `pending_queries` contains no matching entry. The test confirms the connection remains blocked. + +Next, it sends a legitimate DNS query but responds with the wrong transaction ID. The BPF program rejects this too, because the transaction ID is part of the correlation key. Again, the test confirms the connection stays blocked. + +Finally, it sends a response with the correct transaction ID and a 1-second TTL. Now the connection succeeds. After waiting 1.3 seconds (longer than the TTL), the test confirms the connection is blocked again. + +This sequence proves that the tool correctly implements query-response correlation, rejects spoofing attempts, honors TTLs, and properly expires allowlist entries. + +## Building and Running + +Build the example: + +```bash +cd src/55-dns-egress +make +``` + +Attach to a service cgroup and monitor a specific domain: + +```bash +sudo ./dns_egress \ + --cgroup /sys/fs/cgroup/my-service \ + --domain api.example.com \ + --dns-server 127.0.0.53 \ + --port 443 +``` + +The specified cgroup must contain the workload whose DNS packets and connections should share policy state. TCP port 443 and DNS port 53 are the defaults; `--dns-port` selects an alternate resolver port, and `--duration` sets a time limit. The built-in demo requires no external DNS server: + +```bash +sudo ./dns_egress --demo +``` + +Example output: + +```text +dns-egress attached cgroup=/sys/fs/cgroup domain=lab.test resolver=127.0.0.1 tcp_port=19090 dns_port=15353 +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=before-dns result=blocked +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=unsolicited-response result=blocked +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=wrong-transaction-id result=blocked +event=learned pid=1246 ip=127.0.0.1 ttl=1 +event=allowed pid=1246 ip=127.0.0.1 ttl=1 +demo step=live-answer result=allowed +event=expired pid=1246 ip=127.0.0.1 ttl=1 +event=denied pid=1246 ip=127.0.0.1 ttl=1 +demo step=expired-answer result=blocked +``` + +The first three `denied` events show that merely receiving DNS-shaped traffic or seeing the correct domain name is not enough. `learned` appears only for a properly correlated response, `allowed` covers its live TTL window, and once the TTL expires the next connect is immediately denied. + +## Requirements + +| Requirement | Details | +|---|---| +| Kernel | Linux 5.12+ (BPF atomic compare-and-exchange) | +| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_CGROUP_BPF`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_INET` | +| cgroup | cgroup v2, with workload placed below the attached directory | +| Privileges | Root, or equivalent BPF and network capabilities | +| Architecture | Tested on x86-64; no special network hardware required | + +## Scope and Limitations + +This tool deliberately implements a narrow scope: one exact domain, one resolver, one protected TCP port, IPv4 UDP DNS, and the first direct A answer. It recognizes the common `0xc00c` compressed owner name format. Supporting CNAME chains, alternate answer layouts, TCP DNS, IPv6, DoH, or DoT would require additional parsers or observation points. + +This compact scope keeps the central property visible: an IP enters the allowlist through a recent matching DNS query and exits when the DNS TTL expires. + +## Summary + +This example transforms observed DNS results into a time-bounded connection policy. The egress and ingress hooks establish a trustworthy query-response correlation, the TTL controls address lifetime, and the connect hook enforces the policy for the protected port. + +> For more eBPF tutorials, visit our repository at or our website at . + +## References + +- [BPF ring buffer](https://docs.kernel.org/bpf/ringbuf.html) +- [BPF LRU hash maps](https://docs.kernel.org/bpf/map_hash.html) +- [BPF atomic compare-and-exchange commit](https://github.com/torvalds/linux/commit/5ffa25502b5ab3d639829a2d1e316cff7f59a41e) +- [Control Group v2](https://docs.kernel.org/admin-guide/cgroup-v2.html) +- [RFC 1035: Domain Names: Implementation and Specification](https://www.rfc-editor.org/rfc/rfc1035.html) diff --git a/src/55-dns-egress/README.zh.md b/src/55-dns-egress/README.zh.md new file mode 100644 index 00000000..15161c60 --- /dev/null +++ b/src/55-dns-egress/README.zh.md @@ -0,0 +1,1161 @@ +# eBPF 实战教程：用 cgroup BPF 构建基于 DNS 的 IP 允许列表 + +假设你希望一个容器化服务只能访问 `api.example.com`，拒绝所有其他连接。最直接的方案是 IP 白名单，但 DNS 应答会变、TTL 会过期。到了 `connect()` 阶段，内核只看到一个裸 IP，完全不知道它源自哪个域名。缺少的是一条短期关联：把 DNS 查询和随后的 TCP 连接串起来。 + +本教程就来构建这条关联。我们监控特定域名的 DNS 流量，只从有效的查询与响应对中提取 IP 地址，并在 TTL 有效期内允许连接。最终是一个简洁但完整的策略工具，展示 cgroup BPF 程序如何跨多个内核 hook 共享状态。 + +> 完整源代码： + +## 问题的本质：DNS 和 connect 相互隔离 + +当应用解析 `api.example.com` 时，DNS 响应包含 IP 地址和 TTL。但这些信息只存在于用户空间：解析器库把它缓存起来，应用拿着这个 IP 调用 `connect()`，而内核完全不知道这个 IP 从何而来。这种割裂造成了根本性的安全缺口。 + +考虑一个只应该访问后端 API 的容器。传统防火墙可以按目标 IP 过滤，却无法执行“只允许连接最近 60 秒内从 api.example.com 解析出来的 IP”这类策略。内核网络栈和应用层 DNS 解析彼此孤立，互不感知。 + +cgroup BPF 弥合了这个缺口。在同一个 cgroup 上同时挂载 packet hook 和 socket-address hook，就能先观察 DNS 流量，再据此执行连接策略。关键洞察是：cgroup BPF 程序通过 map 共享状态，从而把 DNS 查询、响应和最终连接串成一条信任链。 + +## 整体架构 + +eBPF 允许在多个内核 hook 上运行经过验证的程序，并通过 map 在它们之间共享状态。cgroup BPF 让这些 hook 跟随工作负载：packet 程序检查进出 cgroup 的流量，socket-address 程序则能在连接建立之前决定放行或拦截。 + +本工具在同一个 cgroup 上挂载三个程序： + +1. **`cgroup_skb/egress`**：监控出站 DNS 查询，记录哪些响应是我们期待的 +2. **`cgroup_skb/ingress`**：验证入站 DNS 响应，只从匹配待处理查询的回复中提取 IP 地址 +3. **`cgroup/connect4`**：做最终裁决，只允许连接到从有效 DNS 响应中学到的 IP，且仅在 TTL 有效期内 + +下面跟着一次成功的解析走完整个流程。应用发送 `lab.test` 的 A 查询，egress hook 保存一个关联 key，包含四个字段：解析器地址、客户端地址、客户端 UDP 端口和 DNS 事务 ID。这条记录 5 秒后过期。响应到达后，ingress hook 用相同字段重建 key，验证响应内容，并提取 IP 地址和 TTL。之后发往这个地址的 `connect()` 可以成功，但只在 TTL 有效期内。 + +待处理查询记录就是信任边界。主动推送的伪造响应在 map 中找不到匹配的键；事务 ID 错误的响应会查找另一个键而落空。两者都无法污染允许列表。即使条目还留在 LRU map 中，connect hook 也会重新检查过期时间戳，所以地址会随 DNS TTL 自然失效。 + +## 数据结构 + +先看让这一切运转的数据结构。共享头文件定义了 DNS 协议布局和上报给用户空间的事件： + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __DNS_EGRESS_H +#define __DNS_EGRESS_H + +#define DNS_QNAME_MAX 64 + +enum dns_egress_event_type { + DNS_LEARNED = 1, + DNS_ALLOWED = 2, + DNS_DENIED = 3, + DNS_EXPIRED = 4, +}; + +struct dns_header { + unsigned short id; + unsigned short flags; + unsigned short questions; + unsigned short answers; + unsigned short authorities; + unsigned short additionals; +} __attribute__((packed)); + +struct dns_question { + unsigned short type; + unsigned short class; +} __attribute__((packed)); + +struct dns_a_answer { + unsigned short name; + unsigned short type; + unsigned short class; + unsigned int ttl; + unsigned short address_length; + unsigned int address; +} __attribute__((packed)); + +struct dns_egress_event { + unsigned long long timestamp_ns; + unsigned long long expires_ns; + unsigned int type; + unsigned int pid; + unsigned int ip4; + unsigned int ttl_seconds; +}; + +#endif /* __DNS_EGRESS_H */ +``` + +协议结构使用 `packed` 属性，因为它们直接映射网络报文字节，不允许填充。`dns_header` 对应每个查询和响应开头的 12 字节 DNS 头。`dns_question` 跟在问题名（使用长度前缀标签）后面。`dns_a_answer` 期望常见的压缩格式，其中 name 指针是 `0xc00c`，指回问题部分。 + +发给用户空间的每个事件同时携带以秒为单位的 DNS TTL 和以纳秒为单位的绝对过期时间戳。用户空间打印人类可读的 TTL 用于日志，而 BPF 程序用单调时间戳做决策。这种分离让内核逻辑保持简洁，BPF 代码中无需时间格式转换。 + +## BPF 程序：完整实现 + +下面是完整的内核态实现。虽然比某些例子长，但每个部分都有明确的职责： + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "dns_egress.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define IPPROTO_UDP 17 +#define IPPROTO_TCP 6 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define DNS_QUERY_LIFETIME_NS (5ULL * 1000000000ULL) + +const volatile __u32 target_tgid; +const volatile __u32 dns_server_ip; +const volatile __u16 dns_server_port = 53; +const volatile __u16 protected_tcp_port = 443; +const volatile __u32 configured_qname_length; +const volatile unsigned char configured_qname[DNS_QNAME_MAX]; + +struct dns_state { + __u64 expires_ns; + __u32 ttl_seconds; + __u32 pad; + __u64 expired_reported; +}; + +struct dns_query_key { + __u32 server_ip; + __u32 client_ip; + __u16 client_port; + __u16 transaction_id; +}; + +struct dns_query_state { + __u64 expires_ns; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, struct dns_query_key); + __type(value, struct dns_query_state); +} pending_queries SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct dns_state); +} allowed_ips SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +static __always_inline void emit_event(__u32 type, __u32 ip4, + __u32 ttl_seconds, __u64 expires_ns) +{ + struct dns_egress_event *event; + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return; + event->timestamp_ns = bpf_ktime_get_ns(); + event->expires_ns = expires_ns; + event->type = type; + event->pid = bpf_get_current_pid_tgid() >> 32; + event->ip4 = ip4; + event->ttl_seconds = ttl_seconds; + bpf_ringbuf_submit(event, 0); +} + +static __noinline bool matches_qname(struct __sk_buff *skb, __u32 offset) +{ + unsigned char byte; + + if (!configured_qname_length || configured_qname_length > DNS_QNAME_MAX) + return false; +#pragma clang loop unroll(disable) + for (int i = 0; i < DNS_QNAME_MAX; i++) { + if (i >= configured_qname_length) + break; + if (bpf_skb_load_bytes(skb, offset + i, &byte, sizeof(byte)) || + byte != configured_qname[i]) + return false; + } + return true; +} + +static __always_inline bool parse_query_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.daddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.dest) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + key->server_ip = ip.daddr; + key->client_ip = ip.saddr; + key->client_port = udp.source; + return true; +} + +static __always_inline bool parse_dns_query(struct __sk_buff *skb, + __u32 dns_offset, + struct dns_query_key *key) +{ + struct dns_question question; + struct dns_header header; + __u16 flags; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + flags = bpf_ntohs(header.flags); + if ((flags & 0xf800) || bpf_ntohs(header.questions) != 1) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + key->transaction_id = header.id; + return true; +} + +SEC("cgroup_skb/egress") +int record_dns_query(struct __sk_buff *skb) +{ + struct dns_query_state state = { + .expires_ns = bpf_ktime_get_ns() + DNS_QUERY_LIFETIME_NS, + }; + struct dns_query_key key = {}; + __u32 dns_offset; + + if (!parse_query_transport(skb, &dns_offset, &key) || + !parse_dns_query(skb, dns_offset, &key)) + return 1; + bpf_map_update_elem(&pending_queries, &key, &state, BPF_ANY); + return 1; +} + +static __always_inline bool parse_response_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *query_key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.saddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.source) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + query_key->server_ip = ip.saddr; + query_key->client_ip = ip.daddr; + query_key->client_port = udp.dest; + return true; +} + +static __always_inline bool pending_query_is_live( + struct dns_query_key *query_key) +{ + struct dns_query_state *query; + + query = bpf_map_lookup_elem(&pending_queries, query_key); + if (!query) + return false; + if (bpf_ktime_get_ns() >= query->expires_ns) { + bpf_map_delete_elem(&pending_queries, query_key); + return false; + } + return true; +} + +static __always_inline bool parse_response_question( + struct __sk_buff *skb, __u32 dns_offset, + struct dns_query_key *query_key, __u32 *answer_offset) +{ + struct dns_question question; + struct dns_header header; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + query_key->transaction_id = header.id; + if (!pending_query_is_live(query_key)) + return false; + if ((bpf_ntohs(header.flags) & 0xf80f) != 0x8000 || + bpf_ntohs(header.questions) != 1 || !bpf_ntohs(header.answers)) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + *answer_offset = dns_offset + sizeof(question); + return true; +} + +static __always_inline bool parse_direct_a_answer(struct __sk_buff *skb, + __u32 answer_offset, + __u32 *key, __u32 *ttl) +{ + struct dns_a_answer answer; + + if (bpf_skb_load_bytes(skb, answer_offset, &answer, sizeof(answer)) || + bpf_ntohs(answer.name) != 0xc00c || bpf_ntohs(answer.type) != 1 || + bpf_ntohs(answer.class) != 1 || + bpf_ntohs(answer.address_length) != 4) + return false; + *key = answer.address; + *ttl = bpf_ntohl(answer.ttl); + return *ttl && *ttl <= 86400; +} + +SEC("cgroup_skb/ingress") +int learn_dns_answer(struct __sk_buff *skb) +{ + struct dns_query_key query_key = {}; + struct dns_state state = {}; + __u64 ttl_ns, expires; + __u32 dns_offset, answer_offset; + __u32 key, ttl; + + if (!parse_response_transport(skb, &dns_offset, &query_key) || + !parse_response_question(skb, dns_offset, &query_key, + &answer_offset) || + !parse_direct_a_answer(skb, answer_offset, &key, &ttl)) + return 1; + bpf_map_delete_elem(&pending_queries, &query_key); + ttl_ns = (__u64)ttl * 1000000000ULL; + expires = bpf_ktime_get_ns() + ttl_ns; + state.expires_ns = expires; + state.ttl_seconds = ttl; + if (bpf_map_update_elem(&allowed_ips, &key, &state, BPF_ANY)) + return 1; + emit_event(DNS_LEARNED, key, ttl, expires); + return 1; +} + +SEC("cgroup/connect4") +int enforce_dns_policy(struct bpf_sock_addr *ctx) +{ + struct dns_state *state; + __u64 expires = 0; + __u32 ip4; + __u32 ttl = 0; + + if ((target_tgid && + (__u32)(bpf_get_current_pid_tgid() >> 32) != target_tgid) || + ctx->protocol != IPPROTO_TCP || + bpf_ntohs((__u16)ctx->user_port) != protected_tcp_port) + return 1; + + ip4 = ctx->user_ip4; + state = bpf_map_lookup_elem(&allowed_ips, &ip4); + if (state) { + expires = state->expires_ns; + ttl = state->ttl_seconds; + if (bpf_ktime_get_ns() < expires) { + emit_event(DNS_ALLOWED, ip4, ttl, expires); + return 1; + } + if (__sync_val_compare_and_swap(&state->expired_reported, 0, 1) == 0) + emit_event(DNS_EXPIRED, ip4, ttl, expires); + } + emit_event(DNS_DENIED, ip4, ttl, expires); + return 0; +} +``` + +### Map 设计 + +程序使用三个 BPF map，各自在信任链中承担不同角色。 + +**`pending_queries`** 是一个 LRU hash，保存 DNS 查询的关联状态。键由四个字段组成：服务器 IP、客户端 IP、客户端端口和事务 ID。这些字段共同唯一标识一个查询与响应对。值只存储过期时间戳，这就是时序验证所需的全部信息。LRU 淘汰机制确保即使 DNS 流量很大，内存占用也保持有界。 + +**`allowed_ips`** 也是 LRU hash，但只用 IPv4 地址作为键。值包含过期时间戳、以秒为单位的原始 TTL（用于日志）、对齐填充，以及一个标记是否已上报过期的标志位。这个标志位可以防止多线程竞争访问过期条目时产生重复的 `expired` 事件。 + +**`events`** 是一个 256 KB 的 ring buffer，用于向用户空间发送通知。这个容量可以容纳数千个事件而不会阻塞内核路径。ring buffer 是 perf buffer 的现代替代方案：效率更高，顺序保证也更清晰。 + +### 出站路径：记录待处理查询 + +当报文离开 cgroup 时，`record_dns_query` 开始执行。函数首先通过 `parse_query_transport` 验证传输层：加载 IP 头，确认这是 IPv4 UDP 报文，拒绝分片报文（需要我们没有实现的重组逻辑），并验证目标是否匹配配置的解析器。所有检查通过后，计算 DNS 数据的起始位置并填充关联键。 + +接着 `parse_dns_query` 验证 DNS 层。它检查 flags 是否表示标准查询（而非响应）、是否恰好有一个问题、以及问题名是否匹配配置的域名。问题类型和类必须是 A（地址）和 IN（互联网）。只有所有验证都通过后，才提取事务 ID 并将记录插入 `pending_queries`。 + +注意 `record_dns_query` 始终返回 1，告诉内核继续正常处理报文。我们是在观察，不是阻断。DNS 查询原样流过。 + +### 入站路径：从响应中学习 + +入站程序 `learn_dns_answer` 反转了视角。现在我们检查的是从解析器到达的报文，所以 `parse_response_transport` 检查*源*（而非目标）是否匹配解析器 IP 和端口。关联键使用相同的字段填充，但改从响应的视角取值。 + +关键的安全检查发生在 `pending_query_is_live`。这个函数在 `pending_queries` 中查找关联键。如果没有条目，说明我们从未看到匹配的查询，响应会被拒绝。如果条目存在但已过期，删除它并拒绝响应。只有匹配活跃待处理查询的响应才能继续。 + +确认响应合法后，`parse_response_question` 验证 DNS 头。它检查 flags 是否表示成功响应且无错误、是否恰好包含一个匹配我们域名的问题、以及是否至少有一个应答。然后 `parse_direct_a_answer` 提取第一个 A 记录，要求常见的 `0xc00c` 压缩 name 格式、正确的 type 和 class，以及 1 到 86400 秒之间的 TTL。 + +验证通过后，待处理查询被删除（已被消费），IP 地址以基于 DNS TTL 的过期时间添加到 `allowed_ips`。`emit_event` 调用向用户空间发送 `DNS_LEARNED` 通知。 + +### 连接路径：执行策略 + +`enforce_dns_policy` 挂载到 `cgroup/connect4`，在每个 IPv4 TCP 连接之前执行。函数首先应用过滤：如果我们针对特定进程且这不是它，放行；如果不是 TCP 或不是受保护端口，放行。这些提前返回将无关流量的开销降到最低。 + +对于需要策略执行的连接，我们在 `allowed_ips` 中查找目标 IP。如果找到且未过期，发出 `DNS_ALLOWED` 并返回 1（允许）。如果找到但已过期，使用原子 compare-and-swap 操作 `expired_reported`，确保即使并发访问也只发出一个 `DNS_EXPIRED` 事件。这个原子操作在 Linux 5.12 引入，也确定了本工具的最低内核版本。 + +如果 IP 不在 map 中或已过期，发出 `DNS_DENIED` 并返回 0。内核将返回值 0 转换为 `EPERM`，应用的 `connect()` 立即失败。 + +## 用户态程序 + +用户态程序配置 BPF 只读数据区，把三个程序挂载到 cgroup，并处理 ring buffer 事件。它还包含一个自测的 demo 模式，用于验证完整的信任链。 + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dns_egress.h" +#include "dns_egress.skel.h" + +#define DEMO_DNS_PORT 15353 +#define DEMO_TCP_PORT 19090 +#define DNS_ID 0x4b1d + +struct options { + const char *cgroup_path; + const char *domain; + const char *dns_server; + unsigned int port; + unsigned int dns_port; + unsigned int duration_seconds; + bool demo; +}; + +struct dns_runtime { + struct dns_egress_bpf *skel; + struct bpf_link *query_link; + struct bpf_link *ingress_link; + struct bpf_link *connect_link; + struct ring_buffer *ring; + int cgroup_fd; +}; + +static int event_counts[5]; +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int handle_event(void *ctx, void *data, size_t size) +{ + const struct dns_egress_event *event = data; + char address[INET_ADDRSTRLEN]; + const char *name; + + (void)ctx; + if (size != sizeof(*event) || event->type > DNS_EXPIRED) + return 0; + event_counts[event->type]++; + inet_ntop(AF_INET, &event->ip4, address, sizeof(address)); + name = event->type == DNS_LEARNED ? "learned" : + event->type == DNS_ALLOWED ? "allowed" : + event->type == DNS_DENIED ? "denied" : "expired"; + printf("event=%s pid=%u ip=%s ttl=%u\n", name, event->pid, address, + event->ttl_seconds); + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --cgroup PATH --domain NAME --dns-server IPV4 [--port PORT] [--dns-port PORT] [--duration SEC]\n" + " %s --demo\n", program, program); +} + +static int apply_option(int option, const char *program, + struct options *options) +{ + switch (option) { + case 'c': options->cgroup_path = optarg; return 0; + case 'n': options->domain = optarg; return 0; + case 'r': options->dns_server = optarg; return 0; + case 'p': + return parse_uint(optarg, 65535, &options->port) || !options->port ? + -1 : 0; + case 's': + return parse_uint(optarg, 65535, &options->dns_port) || + !options->dns_port ? -1 : 0; + case 'd': + return parse_uint(optarg, 86400, &options->duration_seconds); + case 'D': options->demo = true; return 0; + case 'h': usage(program); exit(0); + default: return -1; + } +} + +static int finish_options(struct options *options) +{ + if (!options->demo) + return options->cgroup_path && options->domain && + options->dns_server ? 0 : -1; + if (options->cgroup_path || options->domain || options->dns_server) + return -1; + options->cgroup_path = "/sys/fs/cgroup"; + options->domain = "lab.test"; + options->dns_server = "127.0.0.1"; + options->port = DEMO_TCP_PORT; + options->dns_port = DEMO_DNS_PORT; + return 0; +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "cgroup", required_argument, NULL, 'c' }, + { "domain", required_argument, NULL, 'n' }, + { "dns-server", required_argument, NULL, 'r' }, + { "port", required_argument, NULL, 'p' }, + { "dns-port", required_argument, NULL, 's' }, + { "duration", required_argument, NULL, 'd' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "c:n:r:p:s:d:Dh", long_options, + NULL)) != -1) + if (apply_option(option, argv[0], options)) + return -1; + return optind == argc ? finish_options(options) : -1; +} + +static int encode_qname(const char *domain, unsigned char output[DNS_QNAME_MAX], + unsigned int *output_length) +{ + const char *label = domain; + unsigned int used = 0; + + if (!*domain) + return -1; + while (*label) { + const char *dot = strchr(label, '.'); + size_t length = dot ? (size_t)(dot - label) : strlen(label); + + if (!length || length > 63 || used + length + 2 > DNS_QNAME_MAX) + return -1; + output[used++] = length; + memcpy(output + used, label, length); + used += length; + if (!dot) + break; + label = dot + 1; + if (!*label) + break; + } + output[used++] = 0; + *output_length = used; + return 0; +} + +static int bind_udp(struct sockaddr_in *address) +{ + socklen_t length = sizeof(*address); + int fd; + + fd = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (bind(fd, (struct sockaddr *)address, sizeof(*address)) || + getsockname(fd, (struct sockaddr *)address, &length)) { + close(fd); + return -1; + } + return fd; +} + +static int begin_dns_exchange(int server, int client, + struct sockaddr_in *server_address, + const unsigned char *qname, + unsigned int qname_length, + unsigned char message[512], + size_t *query_length, + struct sockaddr_in *client_address) +{ + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(*client_address); + ssize_t received; + + memset(message, 0, 512); + header->id = htons(DNS_ID); + header->flags = htons(0x0100); + header->questions = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + *query_length = sizeof(*header) + qname_length + sizeof(question); + if (sendto(client, message, *query_length, 0, + (struct sockaddr *)server_address, sizeof(*server_address)) != + (ssize_t)*query_length) + return -1; + received = recvfrom(server, message, 512, 0, + (struct sockaddr *)client_address, &address_length); + return received == (ssize_t)*query_length ? 0 : -1; +} + +static int send_dns_answer(int server, int client, + struct sockaddr_in *client_address, + unsigned char message[512], size_t query_length, + unsigned short transaction_id, + unsigned int ttl_seconds) +{ + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(ttl_seconds), + .address_length = htons(4), + }; + struct dns_header *header = (void *)message; + + if (query_length + sizeof(answer) > 512) + return -1; + header->id = htons(transaction_id); + header->flags = htons(0x8180); + header->answers = htons(1); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + query_length, &answer, sizeof(answer)); + if (sendto(server, message, query_length + sizeof(answer), 0, + (struct sockaddr *)client_address, sizeof(*client_address)) != + (ssize_t)(query_length + sizeof(answer))) + return -1; + return recv(client, message, 512, 0) == + (ssize_t)(query_length + sizeof(answer)) ? 0 : -1; +} + +static int send_unsolicited_dns(int server, int client, + const unsigned char *qname, + unsigned int qname_length) +{ + unsigned char message[512] = {}; + struct sockaddr_in client_address; + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(30), + .address_length = htons(4), + }; + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(client_address); + size_t message_length; + + if (getsockname(client, (struct sockaddr *)&client_address, + &address_length)) + return -1; + header->id = htons(DNS_ID + 1); + header->flags = htons(0x8180); + header->questions = htons(1); + header->answers = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + message_length = sizeof(*header) + qname_length + sizeof(question); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + message_length, &answer, sizeof(answer)); + message_length += sizeof(answer); + if (sendto(server, message, message_length, 0, + (struct sockaddr *)&client_address, address_length) != + (ssize_t)message_length) + return -1; + return recv(client, message, sizeof(message), 0) == + (ssize_t)message_length ? 0 : -1; +} + +static int create_tcp_listener(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int one = 1; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + if (bind(fd, (struct sockaddr *)&address, sizeof(address)) || + listen(fd, 4)) { + close(fd); + return -1; + } + return fd; +} + +static int connect_tcp(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int saved_errno; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (!connect(fd, (struct sockaddr *)&address, sizeof(address))) + return fd; + saved_errno = errno; + close(fd); + errno = saved_errno; + return -1; +} + +static int complete_tcp(int listener, int client) +{ + int accepted = accept4(listener, NULL, NULL, SOCK_CLOEXEC); + char byte = 'x'; + int result = 0; + + if (accepted < 0) { + close(client); + return -1; + } + if (write(accepted, &byte, 1) != 1 || read(client, &byte, 1) != 1) + result = -1; + close(accepted); + close(client); + return result; +} + +static int poll_demo_events(struct ring_buffer *ring) +{ + int result = ring_buffer__poll(ring, 100); + + return result < 0 && result != -EINTR ? -1 : 0; +} + +static int expect_blocked_connect(struct ring_buffer *ring, + unsigned int port, const char *step) +{ + int client; + + errno = 0; + client = connect_tcp(port); + if (client >= 0) { + close(client); + return -1; + } + if (errno != EPERM || poll_demo_events(ring)) + return -1; + printf("demo step=%s result=blocked\n", step); + return 0; +} + +static int expect_allowed_connect(struct ring_buffer *ring, int listener, + unsigned int port) +{ + int client = connect_tcp(port); + + if (client < 0 || complete_tcp(listener, client)) + return -1; + if (poll_demo_events(ring)) + return -1; + printf("demo step=live-answer result=allowed\n"); + return 0; +} + +static int expected_demo_events(void) +{ + return event_counts[DNS_LEARNED] == 1 && + event_counts[DNS_ALLOWED] == 1 && + event_counts[DNS_DENIED] == 4 && + event_counts[DNS_EXPIRED] == 1 ? 0 : -1; +} + +struct demo_context { + struct sockaddr_in server_address; + struct sockaddr_in response_client_address; + unsigned char dns_message[512]; + size_t query_length; + int dns_server; + int dns_client; + int listener; +}; + +static int open_demo_sockets(struct demo_context *demo, + const struct options *options) +{ + struct sockaddr_in client_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + + demo->server_address = (struct sockaddr_in) { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(options->dns_port), + }; + demo->dns_server = bind_udp(&demo->server_address); + demo->dns_client = bind_udp(&client_address); + demo->listener = create_tcp_listener(options->port); + return demo->dns_server < 0 || demo->dns_client < 0 || demo->listener < 0 ? + -1 : 0; +} + +static int test_rejected_dns_answers(struct ring_buffer *ring, + const struct options *options, + const unsigned char *qname, + unsigned int qname_length, + struct demo_context *demo) +{ + if (expect_blocked_connect(ring, options->port, "before-dns")) + return -1; + if (send_unsolicited_dns(demo->dns_server, demo->dns_client, qname, + qname_length) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, + "unsolicited-response")) + return -1; + if (begin_dns_exchange(demo->dns_server, demo->dns_client, + &demo->server_address, qname, qname_length, + demo->dns_message, &demo->query_length, + &demo->response_client_address) || + send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID + 1, 30) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "wrong-transaction-id")) + return -1; + return 0; +} + +static int test_live_and_expired_answer(struct ring_buffer *ring, + const struct options *options, + struct demo_context *demo) +{ + struct timespec wait_time = { .tv_sec = 1, .tv_nsec = 300000000 }; + + if (send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID, 1) || + poll_demo_events(ring) || + expect_allowed_connect(ring, demo->listener, options->port)) + return -1; + nanosleep(&wait_time, NULL); + if (poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "expired-answer")) + return -1; + return expected_demo_events(); +} + +static void close_demo_sockets(struct demo_context *demo) +{ + if (demo->listener >= 0) + close(demo->listener); + if (demo->dns_client >= 0) + close(demo->dns_client); + if (demo->dns_server >= 0) + close(demo->dns_server); +} + +static int run_demo(struct ring_buffer *ring, const struct options *options, + const unsigned char *qname, unsigned int qname_length) +{ + struct demo_context demo = { + .dns_server = -1, + .dns_client = -1, + .listener = -1, + }; + int err; + + err = open_demo_sockets(&demo, options); + if (!err) + err = test_rejected_dns_answers(ring, options, qname, + qname_length, &demo); + if (!err) + err = test_live_and_expired_answer(ring, options, &demo); + close_demo_sockets(&demo); + return err; +} + +static bool link_failed(struct bpf_link **link) +{ + if (!libbpf_get_error(*link)) + return false; + *link = NULL; + return true; +} + +static int prepare_runtime(struct dns_runtime *runtime, + const struct options *options, + const struct in_addr *dns_server, + const unsigned char *qname, + unsigned int qname_length) +{ + bool failed; + + runtime->cgroup_fd = open(options->cgroup_path, + O_RDONLY | O_DIRECTORY | O_CLOEXEC); + if (runtime->cgroup_fd < 0) { + fprintf(stderr, "failed to open cgroup %s: %s\n", + options->cgroup_path, strerror(errno)); + return -1; + } + runtime->skel = dns_egress_bpf__open(); + if (!runtime->skel) + return -1; + runtime->skel->rodata->target_tgid = options->demo ? getpid() : 0; + runtime->skel->rodata->dns_server_ip = dns_server->s_addr; + runtime->skel->rodata->dns_server_port = options->dns_port; + runtime->skel->rodata->protected_tcp_port = options->port; + runtime->skel->rodata->configured_qname_length = qname_length; + memcpy((void *)runtime->skel->rodata->configured_qname, qname, + qname_length); + if (dns_egress_bpf__load(runtime->skel)) { + fprintf(stderr, "failed to load DNS egress BPF programs\n"); + return -1; + } + runtime->query_link = bpf_program__attach_cgroup( + runtime->skel->progs.record_dns_query, runtime->cgroup_fd); + runtime->ingress_link = bpf_program__attach_cgroup( + runtime->skel->progs.learn_dns_answer, runtime->cgroup_fd); + runtime->connect_link = bpf_program__attach_cgroup( + runtime->skel->progs.enforce_dns_policy, runtime->cgroup_fd); + failed = link_failed(&runtime->query_link); + failed |= link_failed(&runtime->ingress_link); + failed |= link_failed(&runtime->connect_link); + if (failed) { + fprintf(stderr, "failed to attach programs to cgroup %s\n", + options->cgroup_path); + return -1; + } + runtime->ring = ring_buffer__new( + bpf_map__fd(runtime->skel->maps.events), handle_event, NULL, NULL); + return runtime->ring ? 0 : -1; +} + +static int poll_policy_events(struct ring_buffer *ring, + unsigned int duration_seconds) +{ + unsigned long long deadline = 0; + + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + if (duration_seconds) + deadline = monotonic_ns() + + (unsigned long long)duration_seconds * 1000000000ULL; + while (!stop && (!deadline || monotonic_ns() < deadline)) { + int result = ring_buffer__poll(ring, 100); + + if (result < 0 && result != -EINTR) { + fprintf(stderr, "ring buffer poll failed: %d\n", result); + return -1; + } + } + return 0; +} + +static void destroy_runtime(struct dns_runtime *runtime) +{ + ring_buffer__free(runtime->ring); + bpf_link__destroy(runtime->connect_link); + bpf_link__destroy(runtime->ingress_link); + bpf_link__destroy(runtime->query_link); + if (runtime->cgroup_fd >= 0) + close(runtime->cgroup_fd); + dns_egress_bpf__destroy(runtime->skel); +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 443, .dns_port = 53 }; + struct dns_runtime runtime = { .cgroup_fd = -1 }; + struct in_addr dns_server = {}; + unsigned char qname[DNS_QNAME_MAX] = {}; + unsigned int qname_length = 0; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options) || + encode_qname(options.domain, qname, &qname_length) || + inet_pton(AF_INET, options.dns_server, &dns_server) != 1) { + usage(argv[0]); + return 2; + } + if (prepare_runtime(&runtime, &options, &dns_server, qname, + qname_length)) + goto cleanup; + + printf("dns-egress attached cgroup=%s domain=%s resolver=%s tcp_port=%u dns_port=%u\n", + options.cgroup_path, options.domain, options.dns_server, + options.port, options.dns_port); + if (options.demo) { + if (run_demo(runtime.ring, &options, qname, qname_length)) + goto cleanup; + } else if (poll_policy_events(runtime.ring, options.duration_seconds)) + goto cleanup; + err = 0; + +cleanup: + destroy_runtime(&runtime); + return err; +} +``` + +### 用户态控制流程 + +用户态程序遵循清晰的初始化序列。首先 `parse_options` 处理命令行参数，验证输入并设置默认值。Demo 模式自动配置 loopback 地址和非标准端口，以避免与真实 DNS 和 Web 流量冲突。 + +`encode_qname` 函数将人类可读的域名如 `lab.test` 转换为 DNS 报文格式：`\x03lab\x04test\x00`。每个标签以长度字节开头，后跟标签内容，以零字节结尾。这个编码在启动时只执行一次，写入 BPF skeleton 的 `rodata` 部分，验证器将其视为常量。 + +`prepare_runtime` 把一切串联起来。它打开目标 cgroup 目录、打开 BPF skeleton、配置所有 `rodata` 值（解析器 IP、端口、域名）、加载 BPF 程序，并将每个程序挂载到 cgroup。三个独立的 link 允许独立地挂载和卸载。最后，它创建一个 ring buffer 消费者，为每个内核通知调用 `handle_event`。 + +在普通模式下，`poll_policy_events` 在 ring buffer 上循环，直到持续时间结束或收到信号。每个事件都会打印其 IP 地址、PID、TTL 和类型。Demo 模式则运行 `run_demo`，用合成的 DNS 流量和 TCP 连接验证完整的信任链。 + +### Demo 模式：验证安全属性 + +Demo 模式既是集成测试，也是安全模型的演示。它完全在 loopback 上运行，使用非标准端口（DNS 15353，TCP 19090）以避免干扰真实服务。 + +测试序列首先验证在任何 DNS 流量之前连接被阻止。然后发送一个未经请求的 DNS 响应，也就是一个没有前置查询就到达的响应。BPF 程序拒绝它，因为 `pending_queries` 中没有匹配的条目。测试确认连接仍然被阻止。 + +接下来，发送一个合法的 DNS 查询，但响应使用错误的事务 ID。BPF 程序同样拒绝它，因为事务 ID 是关联键的一部分。测试再次确认连接仍处于阻断状态。 + +最后，发送一个事务 ID 正确、TTL 为 1 秒的响应。现在连接成功了。等待 1.3 秒（超过 TTL）后，测试确认连接再次被阻止。 + +这个序列证明了工具正确实现了查询-响应关联、拒绝欺骗尝试、遵守 TTL，并正确使允许列表条目过期。 + +## 编译和运行 + +构建示例： + +```bash +cd src/55-dns-egress +make +``` + +挂载到服务 cgroup 并监控特定域名： + +```bash +sudo ./dns_egress \ + --cgroup /sys/fs/cgroup/my-service \ + --domain api.example.com \ + --dns-server 127.0.0.53 \ + --port 443 +``` + +指定的 cgroup 必须包含目标工作负载，让它的 DNS 报文和连接共享策略状态。TCP 443 和 DNS 53 是默认端口；`--dns-port` 选择其他解析器端口，`--duration` 设置时间上限。内置 demo 不需要外部 DNS 服务器： + +```bash +sudo ./dns_egress --demo +``` + +示例输出： + +```text +dns-egress attached cgroup=/sys/fs/cgroup domain=lab.test resolver=127.0.0.1 tcp_port=19090 dns_port=15353 +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=before-dns result=blocked +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=unsolicited-response result=blocked +event=denied pid=1246 ip=127.0.0.1 ttl=0 +demo step=wrong-transaction-id result=blocked +event=learned pid=1246 ip=127.0.0.1 ttl=1 +event=allowed pid=1246 ip=127.0.0.1 ttl=1 +demo step=live-answer result=allowed +event=expired pid=1246 ip=127.0.0.1 ttl=1 +event=denied pid=1246 ip=127.0.0.1 ttl=1 +demo step=expired-answer result=blocked +``` + +前三个 `denied` 事件说明，仅仅收到 DNS 格式的流量或看到正确的域名是不够的。只有正确关联的响应才产生 `learned`，`allowed` 覆盖其有效 TTL 窗口，TTL 一过期，下一次连接立即被拒绝。 + +## 环境要求 + +| 要求 | 说明 | +|---|---| +| 内核 | Linux 5.12+（BPF atomic compare-and-exchange） | +| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_CGROUP_BPF`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_INET` | +| cgroup | cgroup v2，工作负载位于挂载目录之下 | +| 权限 | root 权限，或等价的 BPF 与网络能力 | +| 架构 | x86-64 已测试；不需要特殊网卡 | + +## 范围与限制 + +本工具刻意实现了一个紧凑的范围：一个精确域名、一个解析器、一个受保护的 TCP 端口、IPv4 UDP DNS 和第一个直接 A 应答。它识别常见的 `0xc00c` 压缩 owner name 格式。支持 CNAME 链、其他应答布局、TCP DNS、IPv6、DoH 或 DoT 需要额外的解析器或新的观察点。 + +这个紧凑范围保留了核心属性的可见性：IP 通过最近一次匹配的 DNS 查询进入允许列表，在 DNS TTL 过期时退出。 + +## 总结 + +本示例把观察到的 DNS 结果转化为有时间边界的连接策略。egress 和 ingress hook 建立可信的查询与响应关联，TTL 控制地址生命周期，connect hook 对受保护端口执行策略。 + +> 更多 eBPF 教程，请访问我们的代码仓库 或网站 。 + +## 参考资料 + +- [BPF ring buffer](https://docs.kernel.org/bpf/ringbuf.html) +- [BPF LRU hash map](https://docs.kernel.org/bpf/map_hash.html) +- [BPF atomic compare-and-exchange commit](https://github.com/torvalds/linux/commit/5ffa25502b5ab3d639829a2d1e316cff7f59a41e) +- [Control Group v2](https://docs.kernel.org/admin-guide/cgroup-v2.html) +- [RFC 1035：Domain Names: Implementation and Specification](https://www.rfc-editor.org/rfc/rfc1035.html) diff --git a/src/55-dns-egress/dns_egress.bpf.c b/src/55-dns-egress/dns_egress.bpf.c new file mode 100644 index 00000000..c2baeedd --- /dev/null +++ b/src/55-dns-egress/dns_egress.bpf.c @@ -0,0 +1,289 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "dns_egress.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define IPPROTO_UDP 17 +#define IPPROTO_TCP 6 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define DNS_QUERY_LIFETIME_NS (5ULL * 1000000000ULL) + +const volatile __u32 target_tgid; +const volatile __u32 dns_server_ip; +const volatile __u16 dns_server_port = 53; +const volatile __u16 protected_tcp_port = 443; +const volatile __u32 configured_qname_length; +const volatile unsigned char configured_qname[DNS_QNAME_MAX]; + +struct dns_state { + __u64 expires_ns; + __u32 ttl_seconds; + __u32 pad; + __u64 expired_reported; +}; + +struct dns_query_key { + __u32 server_ip; + __u32 client_ip; + __u16 client_port; + __u16 transaction_id; +}; + +struct dns_query_state { + __u64 expires_ns; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, struct dns_query_key); + __type(value, struct dns_query_state); +} pending_queries SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct dns_state); +} allowed_ips SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +static __always_inline void emit_event(__u32 type, __u32 ip4, + __u32 ttl_seconds, __u64 expires_ns) +{ + struct dns_egress_event *event; + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return; + event->timestamp_ns = bpf_ktime_get_ns(); + event->expires_ns = expires_ns; + event->type = type; + event->pid = bpf_get_current_pid_tgid() >> 32; + event->ip4 = ip4; + event->ttl_seconds = ttl_seconds; + bpf_ringbuf_submit(event, 0); +} + +static __noinline bool matches_qname(struct __sk_buff *skb, __u32 offset) +{ + unsigned char byte; + + if (!configured_qname_length || configured_qname_length > DNS_QNAME_MAX) + return false; +#pragma clang loop unroll(disable) + for (int i = 0; i < DNS_QNAME_MAX; i++) { + if (i >= configured_qname_length) + break; + if (bpf_skb_load_bytes(skb, offset + i, &byte, sizeof(byte)) || + byte != configured_qname[i]) + return false; + } + return true; +} + +static __always_inline bool parse_query_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.daddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.dest) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + key->server_ip = ip.daddr; + key->client_ip = ip.saddr; + key->client_port = udp.source; + return true; +} + +static __always_inline bool parse_dns_query(struct __sk_buff *skb, + __u32 dns_offset, + struct dns_query_key *key) +{ + struct dns_question question; + struct dns_header header; + __u16 flags; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + flags = bpf_ntohs(header.flags); + if ((flags & 0xf800) || bpf_ntohs(header.questions) != 1) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + key->transaction_id = header.id; + return true; +} + +SEC("cgroup_skb/egress") +int record_dns_query(struct __sk_buff *skb) +{ + struct dns_query_state state = { + .expires_ns = bpf_ktime_get_ns() + DNS_QUERY_LIFETIME_NS, + }; + struct dns_query_key key = {}; + __u32 dns_offset; + + if (!parse_query_transport(skb, &dns_offset, &key) || + !parse_dns_query(skb, dns_offset, &key)) + return 1; + bpf_map_update_elem(&pending_queries, &key, &state, BPF_ANY); + return 1; +} + +static __always_inline bool parse_response_transport( + struct __sk_buff *skb, __u32 *dns_offset, struct dns_query_key *query_key) +{ + struct udphdr udp; + struct iphdr ip; + __u32 ip_header_len; + + if (bpf_skb_load_bytes(skb, 0, &ip, sizeof(ip))) + return false; + if (ip.version != 4 || ip.protocol != IPPROTO_UDP || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + ip.saddr != dns_server_ip) + return false; + ip_header_len = ip.ihl * 4; + if (bpf_skb_load_bytes(skb, ip_header_len, &udp, sizeof(udp)) || + bpf_ntohs(udp.source) != dns_server_port) + return false; + *dns_offset = ip_header_len + sizeof(udp); + query_key->server_ip = ip.saddr; + query_key->client_ip = ip.daddr; + query_key->client_port = udp.dest; + return true; +} + +static __always_inline bool pending_query_is_live( + struct dns_query_key *query_key) +{ + struct dns_query_state *query; + + query = bpf_map_lookup_elem(&pending_queries, query_key); + if (!query) + return false; + if (bpf_ktime_get_ns() >= query->expires_ns) { + bpf_map_delete_elem(&pending_queries, query_key); + return false; + } + return true; +} + +static __always_inline bool parse_response_question( + struct __sk_buff *skb, __u32 dns_offset, + struct dns_query_key *query_key, __u32 *answer_offset) +{ + struct dns_question question; + struct dns_header header; + + if (bpf_skb_load_bytes(skb, dns_offset, &header, sizeof(header))) + return false; + query_key->transaction_id = header.id; + if (!pending_query_is_live(query_key)) + return false; + if ((bpf_ntohs(header.flags) & 0xf80f) != 0x8000 || + bpf_ntohs(header.questions) != 1 || !bpf_ntohs(header.answers)) + return false; + if (!matches_qname(skb, dns_offset + sizeof(header))) + return false; + dns_offset += sizeof(header) + configured_qname_length; + if (bpf_skb_load_bytes(skb, dns_offset, &question, sizeof(question)) || + bpf_ntohs(question.type) != 1 || bpf_ntohs(question.class) != 1) + return false; + *answer_offset = dns_offset + sizeof(question); + return true; +} + +static __always_inline bool parse_direct_a_answer(struct __sk_buff *skb, + __u32 answer_offset, + __u32 *key, __u32 *ttl) +{ + struct dns_a_answer answer; + + if (bpf_skb_load_bytes(skb, answer_offset, &answer, sizeof(answer)) || + bpf_ntohs(answer.name) != 0xc00c || bpf_ntohs(answer.type) != 1 || + bpf_ntohs(answer.class) != 1 || + bpf_ntohs(answer.address_length) != 4) + return false; + *key = answer.address; + *ttl = bpf_ntohl(answer.ttl); + return *ttl && *ttl <= 86400; +} + +SEC("cgroup_skb/ingress") +int learn_dns_answer(struct __sk_buff *skb) +{ + struct dns_query_key query_key = {}; + struct dns_state state = {}; + __u64 ttl_ns, expires; + __u32 dns_offset, answer_offset; + __u32 key, ttl; + + if (!parse_response_transport(skb, &dns_offset, &query_key) || + !parse_response_question(skb, dns_offset, &query_key, + &answer_offset) || + !parse_direct_a_answer(skb, answer_offset, &key, &ttl)) + return 1; + bpf_map_delete_elem(&pending_queries, &query_key); + ttl_ns = (__u64)ttl * 1000000000ULL; + expires = bpf_ktime_get_ns() + ttl_ns; + state.expires_ns = expires; + state.ttl_seconds = ttl; + if (bpf_map_update_elem(&allowed_ips, &key, &state, BPF_ANY)) + return 1; + emit_event(DNS_LEARNED, key, ttl, expires); + return 1; +} + +SEC("cgroup/connect4") +int enforce_dns_policy(struct bpf_sock_addr *ctx) +{ + struct dns_state *state; + __u64 expires = 0; + __u32 ip4; + __u32 ttl = 0; + + if ((target_tgid && + (__u32)(bpf_get_current_pid_tgid() >> 32) != target_tgid) || + ctx->protocol != IPPROTO_TCP || + bpf_ntohs((__u16)ctx->user_port) != protected_tcp_port) + return 1; + + ip4 = ctx->user_ip4; + state = bpf_map_lookup_elem(&allowed_ips, &ip4); + if (state) { + expires = state->expires_ns; + ttl = state->ttl_seconds; + if (bpf_ktime_get_ns() < expires) { + emit_event(DNS_ALLOWED, ip4, ttl, expires); + return 1; + } + if (__sync_val_compare_and_swap(&state->expired_reported, 0, 1) == 0) + emit_event(DNS_EXPIRED, ip4, ttl, expires); + } + emit_event(DNS_DENIED, ip4, ttl, expires); + return 0; +} diff --git a/src/55-dns-egress/dns_egress.c b/src/55-dns-egress/dns_egress.c new file mode 100644 index 00000000..88ec79fb --- /dev/null +++ b/src/55-dns-egress/dns_egress.c @@ -0,0 +1,639 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "dns_egress.h" +#include "dns_egress.skel.h" + +#define DEMO_DNS_PORT 15353 +#define DEMO_TCP_PORT 19090 +#define DNS_ID 0x4b1d + +struct options { + const char *cgroup_path; + const char *domain; + const char *dns_server; + unsigned int port; + unsigned int dns_port; + unsigned int duration_seconds; + bool demo; +}; + +struct dns_runtime { + struct dns_egress_bpf *skel; + struct bpf_link *query_link; + struct bpf_link *ingress_link; + struct bpf_link *connect_link; + struct ring_buffer *ring; + int cgroup_fd; +}; + +static int event_counts[5]; +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int handle_event(void *ctx, void *data, size_t size) +{ + const struct dns_egress_event *event = data; + char address[INET_ADDRSTRLEN]; + const char *name; + + (void)ctx; + if (size != sizeof(*event) || event->type > DNS_EXPIRED) + return 0; + event_counts[event->type]++; + inet_ntop(AF_INET, &event->ip4, address, sizeof(address)); + name = event->type == DNS_LEARNED ? "learned" : + event->type == DNS_ALLOWED ? "allowed" : + event->type == DNS_DENIED ? "denied" : "expired"; + printf("event=%s pid=%u ip=%s ttl=%u\n", name, event->pid, address, + event->ttl_seconds); + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --cgroup PATH --domain NAME --dns-server IPV4 [--port PORT] [--dns-port PORT] [--duration SEC]\n" + " %s --demo\n", program, program); +} + +static int apply_option(int option, const char *program, + struct options *options) +{ + switch (option) { + case 'c': options->cgroup_path = optarg; return 0; + case 'n': options->domain = optarg; return 0; + case 'r': options->dns_server = optarg; return 0; + case 'p': + return parse_uint(optarg, 65535, &options->port) || !options->port ? + -1 : 0; + case 's': + return parse_uint(optarg, 65535, &options->dns_port) || + !options->dns_port ? -1 : 0; + case 'd': + return parse_uint(optarg, 86400, &options->duration_seconds); + case 'D': options->demo = true; return 0; + case 'h': usage(program); exit(0); + default: return -1; + } +} + +static int finish_options(struct options *options) +{ + if (!options->demo) + return options->cgroup_path && options->domain && + options->dns_server ? 0 : -1; + if (options->cgroup_path || options->domain || options->dns_server) + return -1; + options->cgroup_path = "/sys/fs/cgroup"; + options->domain = "lab.test"; + options->dns_server = "127.0.0.1"; + options->port = DEMO_TCP_PORT; + options->dns_port = DEMO_DNS_PORT; + return 0; +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "cgroup", required_argument, NULL, 'c' }, + { "domain", required_argument, NULL, 'n' }, + { "dns-server", required_argument, NULL, 'r' }, + { "port", required_argument, NULL, 'p' }, + { "dns-port", required_argument, NULL, 's' }, + { "duration", required_argument, NULL, 'd' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "c:n:r:p:s:d:Dh", long_options, + NULL)) != -1) + if (apply_option(option, argv[0], options)) + return -1; + return optind == argc ? finish_options(options) : -1; +} + +static int encode_qname(const char *domain, unsigned char output[DNS_QNAME_MAX], + unsigned int *output_length) +{ + const char *label = domain; + unsigned int used = 0; + + if (!*domain) + return -1; + while (*label) { + const char *dot = strchr(label, '.'); + size_t length = dot ? (size_t)(dot - label) : strlen(label); + + if (!length || length > 63 || used + length + 2 > DNS_QNAME_MAX) + return -1; + output[used++] = length; + memcpy(output + used, label, length); + used += length; + if (!dot) + break; + label = dot + 1; + if (!*label) + break; + } + output[used++] = 0; + *output_length = used; + return 0; +} + +static int bind_udp(struct sockaddr_in *address) +{ + socklen_t length = sizeof(*address); + int fd; + + fd = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (bind(fd, (struct sockaddr *)address, sizeof(*address)) || + getsockname(fd, (struct sockaddr *)address, &length)) { + close(fd); + return -1; + } + return fd; +} + +static int begin_dns_exchange(int server, int client, + struct sockaddr_in *server_address, + const unsigned char *qname, + unsigned int qname_length, + unsigned char message[512], + size_t *query_length, + struct sockaddr_in *client_address) +{ + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(*client_address); + ssize_t received; + + memset(message, 0, 512); + header->id = htons(DNS_ID); + header->flags = htons(0x0100); + header->questions = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + *query_length = sizeof(*header) + qname_length + sizeof(question); + if (sendto(client, message, *query_length, 0, + (struct sockaddr *)server_address, sizeof(*server_address)) != + (ssize_t)*query_length) + return -1; + received = recvfrom(server, message, 512, 0, + (struct sockaddr *)client_address, &address_length); + return received == (ssize_t)*query_length ? 0 : -1; +} + +static int send_dns_answer(int server, int client, + struct sockaddr_in *client_address, + unsigned char message[512], size_t query_length, + unsigned short transaction_id, + unsigned int ttl_seconds) +{ + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(ttl_seconds), + .address_length = htons(4), + }; + struct dns_header *header = (void *)message; + + if (query_length + sizeof(answer) > 512) + return -1; + header->id = htons(transaction_id); + header->flags = htons(0x8180); + header->answers = htons(1); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + query_length, &answer, sizeof(answer)); + if (sendto(server, message, query_length + sizeof(answer), 0, + (struct sockaddr *)client_address, sizeof(*client_address)) != + (ssize_t)(query_length + sizeof(answer))) + return -1; + return recv(client, message, 512, 0) == + (ssize_t)(query_length + sizeof(answer)) ? 0 : -1; +} + +static int send_unsolicited_dns(int server, int client, + const unsigned char *qname, + unsigned int qname_length) +{ + unsigned char message[512] = {}; + struct sockaddr_in client_address; + struct dns_a_answer answer = { + .name = htons(0xc00c), + .type = htons(1), + .class = htons(1), + .ttl = htonl(30), + .address_length = htons(4), + }; + struct dns_question question = { + .type = htons(1), + .class = htons(1), + }; + struct dns_header *header = (void *)message; + socklen_t address_length = sizeof(client_address); + size_t message_length; + + if (getsockname(client, (struct sockaddr *)&client_address, + &address_length)) + return -1; + header->id = htons(DNS_ID + 1); + header->flags = htons(0x8180); + header->questions = htons(1); + header->answers = htons(1); + memcpy(message + sizeof(*header), qname, qname_length); + memcpy(message + sizeof(*header) + qname_length, &question, + sizeof(question)); + message_length = sizeof(*header) + qname_length + sizeof(question); + inet_pton(AF_INET, "127.0.0.1", &answer.address); + memcpy(message + message_length, &answer, sizeof(answer)); + message_length += sizeof(answer); + if (sendto(server, message, message_length, 0, + (struct sockaddr *)&client_address, address_length) != + (ssize_t)message_length) + return -1; + return recv(client, message, sizeof(message), 0) == + (ssize_t)message_length ? 0 : -1; +} + +static int create_tcp_listener(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int one = 1; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one)); + if (bind(fd, (struct sockaddr *)&address, sizeof(address)) || + listen(fd, 4)) { + close(fd); + return -1; + } + return fd; +} + +static int connect_tcp(unsigned int port) +{ + struct sockaddr_in address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(port), + }; + int saved_errno; + int fd; + + fd = socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0); + if (fd < 0) + return -1; + if (!connect(fd, (struct sockaddr *)&address, sizeof(address))) + return fd; + saved_errno = errno; + close(fd); + errno = saved_errno; + return -1; +} + +static int complete_tcp(int listener, int client) +{ + int accepted = accept4(listener, NULL, NULL, SOCK_CLOEXEC); + char byte = 'x'; + int result = 0; + + if (accepted < 0) { + close(client); + return -1; + } + if (write(accepted, &byte, 1) != 1 || read(client, &byte, 1) != 1) + result = -1; + close(accepted); + close(client); + return result; +} + +static int poll_demo_events(struct ring_buffer *ring) +{ + int result = ring_buffer__poll(ring, 100); + + return result < 0 && result != -EINTR ? -1 : 0; +} + +static int expect_blocked_connect(struct ring_buffer *ring, + unsigned int port, const char *step) +{ + int client; + + errno = 0; + client = connect_tcp(port); + if (client >= 0) { + close(client); + return -1; + } + if (errno != EPERM || poll_demo_events(ring)) + return -1; + printf("demo step=%s result=blocked\n", step); + return 0; +} + +static int expect_allowed_connect(struct ring_buffer *ring, int listener, + unsigned int port) +{ + int client = connect_tcp(port); + + if (client < 0 || complete_tcp(listener, client)) + return -1; + if (poll_demo_events(ring)) + return -1; + printf("demo step=live-answer result=allowed\n"); + return 0; +} + +static int expected_demo_events(void) +{ + return event_counts[DNS_LEARNED] == 1 && + event_counts[DNS_ALLOWED] == 1 && + event_counts[DNS_DENIED] == 4 && + event_counts[DNS_EXPIRED] == 1 ? 0 : -1; +} + +struct demo_context { + struct sockaddr_in server_address; + struct sockaddr_in response_client_address; + unsigned char dns_message[512]; + size_t query_length; + int dns_server; + int dns_client; + int listener; +}; + +static int open_demo_sockets(struct demo_context *demo, + const struct options *options) +{ + struct sockaddr_in client_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + + demo->server_address = (struct sockaddr_in) { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + .sin_port = htons(options->dns_port), + }; + demo->dns_server = bind_udp(&demo->server_address); + demo->dns_client = bind_udp(&client_address); + demo->listener = create_tcp_listener(options->port); + return demo->dns_server < 0 || demo->dns_client < 0 || demo->listener < 0 ? + -1 : 0; +} + +static int test_rejected_dns_answers(struct ring_buffer *ring, + const struct options *options, + const unsigned char *qname, + unsigned int qname_length, + struct demo_context *demo) +{ + if (expect_blocked_connect(ring, options->port, "before-dns")) + return -1; + if (send_unsolicited_dns(demo->dns_server, demo->dns_client, qname, + qname_length) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, + "unsolicited-response")) + return -1; + if (begin_dns_exchange(demo->dns_server, demo->dns_client, + &demo->server_address, qname, qname_length, + demo->dns_message, &demo->query_length, + &demo->response_client_address) || + send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID + 1, 30) || + poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "wrong-transaction-id")) + return -1; + return 0; +} + +static int test_live_and_expired_answer(struct ring_buffer *ring, + const struct options *options, + struct demo_context *demo) +{ + struct timespec wait_time = { .tv_sec = 1, .tv_nsec = 300000000 }; + + if (send_dns_answer(demo->dns_server, demo->dns_client, + &demo->response_client_address, demo->dns_message, + demo->query_length, DNS_ID, 1) || + poll_demo_events(ring) || + expect_allowed_connect(ring, demo->listener, options->port)) + return -1; + nanosleep(&wait_time, NULL); + if (poll_demo_events(ring) || + expect_blocked_connect(ring, options->port, "expired-answer")) + return -1; + return expected_demo_events(); +} + +static void close_demo_sockets(struct demo_context *demo) +{ + if (demo->listener >= 0) + close(demo->listener); + if (demo->dns_client >= 0) + close(demo->dns_client); + if (demo->dns_server >= 0) + close(demo->dns_server); +} + +static int run_demo(struct ring_buffer *ring, const struct options *options, + const unsigned char *qname, unsigned int qname_length) +{ + struct demo_context demo = { + .dns_server = -1, + .dns_client = -1, + .listener = -1, + }; + int err; + + err = open_demo_sockets(&demo, options); + if (!err) + err = test_rejected_dns_answers(ring, options, qname, + qname_length, &demo); + if (!err) + err = test_live_and_expired_answer(ring, options, &demo); + close_demo_sockets(&demo); + return err; +} + +static bool link_failed(struct bpf_link **link) +{ + if (!libbpf_get_error(*link)) + return false; + *link = NULL; + return true; +} + +static int prepare_runtime(struct dns_runtime *runtime, + const struct options *options, + const struct in_addr *dns_server, + const unsigned char *qname, + unsigned int qname_length) +{ + bool failed; + + runtime->cgroup_fd = open(options->cgroup_path, + O_RDONLY | O_DIRECTORY | O_CLOEXEC); + if (runtime->cgroup_fd < 0) { + fprintf(stderr, "failed to open cgroup %s: %s\n", + options->cgroup_path, strerror(errno)); + return -1; + } + runtime->skel = dns_egress_bpf__open(); + if (!runtime->skel) + return -1; + runtime->skel->rodata->target_tgid = options->demo ? getpid() : 0; + runtime->skel->rodata->dns_server_ip = dns_server->s_addr; + runtime->skel->rodata->dns_server_port = options->dns_port; + runtime->skel->rodata->protected_tcp_port = options->port; + runtime->skel->rodata->configured_qname_length = qname_length; + memcpy((void *)runtime->skel->rodata->configured_qname, qname, + qname_length); + if (dns_egress_bpf__load(runtime->skel)) { + fprintf(stderr, "failed to load DNS egress BPF programs\n"); + return -1; + } + runtime->query_link = bpf_program__attach_cgroup( + runtime->skel->progs.record_dns_query, runtime->cgroup_fd); + runtime->ingress_link = bpf_program__attach_cgroup( + runtime->skel->progs.learn_dns_answer, runtime->cgroup_fd); + runtime->connect_link = bpf_program__attach_cgroup( + runtime->skel->progs.enforce_dns_policy, runtime->cgroup_fd); + failed = link_failed(&runtime->query_link); + failed |= link_failed(&runtime->ingress_link); + failed |= link_failed(&runtime->connect_link); + if (failed) { + fprintf(stderr, "failed to attach programs to cgroup %s\n", + options->cgroup_path); + return -1; + } + runtime->ring = ring_buffer__new( + bpf_map__fd(runtime->skel->maps.events), handle_event, NULL, NULL); + return runtime->ring ? 0 : -1; +} + +static int poll_policy_events(struct ring_buffer *ring, + unsigned int duration_seconds) +{ + unsigned long long deadline = 0; + + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + if (duration_seconds) + deadline = monotonic_ns() + + (unsigned long long)duration_seconds * 1000000000ULL; + while (!stop && (!deadline || monotonic_ns() < deadline)) { + int result = ring_buffer__poll(ring, 100); + + if (result < 0 && result != -EINTR) { + fprintf(stderr, "ring buffer poll failed: %d\n", result); + return -1; + } + } + return 0; +} + +static void destroy_runtime(struct dns_runtime *runtime) +{ + ring_buffer__free(runtime->ring); + bpf_link__destroy(runtime->connect_link); + bpf_link__destroy(runtime->ingress_link); + bpf_link__destroy(runtime->query_link); + if (runtime->cgroup_fd >= 0) + close(runtime->cgroup_fd); + dns_egress_bpf__destroy(runtime->skel); +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 443, .dns_port = 53 }; + struct dns_runtime runtime = { .cgroup_fd = -1 }; + struct in_addr dns_server = {}; + unsigned char qname[DNS_QNAME_MAX] = {}; + unsigned int qname_length = 0; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options) || + encode_qname(options.domain, qname, &qname_length) || + inet_pton(AF_INET, options.dns_server, &dns_server) != 1) { + usage(argv[0]); + return 2; + } + if (prepare_runtime(&runtime, &options, &dns_server, qname, + qname_length)) + goto cleanup; + + printf("dns-egress attached cgroup=%s domain=%s resolver=%s tcp_port=%u dns_port=%u\n", + options.cgroup_path, options.domain, options.dns_server, + options.port, options.dns_port); + if (options.demo) { + if (run_demo(runtime.ring, &options, qname, qname_length)) + goto cleanup; + } else if (poll_policy_events(runtime.ring, options.duration_seconds)) + goto cleanup; + err = 0; + +cleanup: + destroy_runtime(&runtime); + return err; +} diff --git a/src/55-dns-egress/dns_egress.h b/src/55-dns-egress/dns_egress.h new file mode 100644 index 00000000..3802feee --- /dev/null +++ b/src/55-dns-egress/dns_egress.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __DNS_EGRESS_H +#define __DNS_EGRESS_H + +#define DNS_QNAME_MAX 64 + +enum dns_egress_event_type { + DNS_LEARNED = 1, + DNS_ALLOWED = 2, + DNS_DENIED = 3, + DNS_EXPIRED = 4, +}; + +struct dns_header { + unsigned short id; + unsigned short flags; + unsigned short questions; + unsigned short answers; + unsigned short authorities; + unsigned short additionals; +} __attribute__((packed)); + +struct dns_question { + unsigned short type; + unsigned short class; +} __attribute__((packed)); + +struct dns_a_answer { + unsigned short name; + unsigned short type; + unsigned short class; + unsigned int ttl; + unsigned short address_length; + unsigned int address; +} __attribute__((packed)); + +struct dns_egress_event { + unsigned long long timestamp_ns; + unsigned long long expires_ns; + unsigned int type; + unsigned int pid; + unsigned int ip4; + unsigned int ttl_seconds; +}; + +#endif /* __DNS_EGRESS_H */ diff --git a/src/55-dns-egress/tests/test_dns_egress.py b/src/55-dns-egress/tests/test_dns_egress.py new file mode 100644 index 00000000..34065aef --- /dev/null +++ b/src/55-dns-egress/tests/test_dns_egress.py @@ -0,0 +1,29 @@ +#!/usr/bin/env python3 +import subprocess +import sys + + +def main(): + result = subprocess.run( + [sys.argv[1], "--demo"], + text=True, + capture_output=True, + check=False, + timeout=30, + ) + output = result.stdout + result.stderr + if result.returncode: + raise AssertionError(output) + assert "demo step=before-dns result=blocked" in output, output + assert "demo step=unsolicited-response result=blocked" in output, output + assert "demo step=wrong-transaction-id result=blocked" in output, output + assert "event=learned" in output and "ip=127.0.0.1 ttl=1" in output, output + assert "demo step=live-answer result=allowed" in output, output + assert "event=expired" in output, output + assert "demo step=expired-answer result=blocked" in output, output + assert output.count("event=denied") == 4, output + print("DNS-derived allowlist integration test: PASS") + + +if __name__ == "__main__": + main() diff --git a/src/56-tc-flow-index/.config b/src/56-tc-flow-index/.config new file mode 100644 index 00000000..940e6f6b --- /dev/null +++ b/src/56-tc-flow-index/.config @@ -0,0 +1,10 @@ +level=Depth +type=Networking +kernel_min=6.16 +kernel_min_basis=required-feature +architectures=x86_64 +btf=required +kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_NET_SCHED=y,CONFIG_NET_CLS_BPF=y +hardware=network-interface +root=required +test_status=ci-build diff --git a/src/56-tc-flow-index/.gitignore b/src/56-tc-flow-index/.gitignore new file mode 100644 index 00000000..ff08a9da --- /dev/null +++ b/src/56-tc-flow-index/.gitignore @@ -0,0 +1,2 @@ +.output/ +tc_flow_index diff --git a/src/56-tc-flow-index/Makefile b/src/56-tc-flow-index/Makefile new file mode 100644 index 00000000..7e0a404f --- /dev/null +++ b/src/56-tc-flow-index/Makefile @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +OUTPUT := .output +CLANG ?= clang +LIBBPF_SRC := $(abspath ../third_party/libbpf/src) +BPFTOOL_SRC := $(abspath ../third_party/bpftool/src) +LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a) +BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool) +BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool +ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \ + | sed 's/arm.*/arm/' \ + | sed 's/aarch64/arm64/' \ + | sed 's/ppc64le/powerpc/' \ + | sed 's/mips.*/mips/' \ + | sed 's/riscv64/riscv/' \ + | sed 's/loongarch64/loongarch/') +VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h +INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I. +CFLAGS := -g -Wall -Wextra -D_GNU_SOURCE +ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS) +APPS = tc_flow_index + +CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - &1 \ + | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') + +ifeq ($(V),1) + Q = + msg = +else + Q = @ + msg = @printf ' %-8s %s%s\n' "$(1)" \ + "$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \ + "$(if $(3), $(3))"; + MAKEFLAGS += --no-print-directory +endif + +define allow-override + $(if $(or $(findstring environment,$(origin $(1))),\ + $(findstring command line,$(origin $(1)))),,\ + $(eval $(1) = $(2))) +endef + +$(call allow-override,CC,$(CROSS_COMPILE)cc) + +.PHONY: all clean test +all: $(APPS) + +clean: + $(call msg,CLEAN) + $(Q)rm -rf $(OUTPUT) $(APPS) + +test: all + @if [ "$$(id -u)" -ne 0 ]; then \ + echo "test requires root; run inside the documented KVM guest" >&2; \ + exit 1; \ + fi + python3 tests/test_tc_flow_index.py ./tc_flow_index + +$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT): + $(call msg,MKDIR,$@) + $(Q)mkdir -p $@ + +$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf + $(call msg,LIB,$@) + $(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \ + OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \ + INCLUDEDIR= LIBDIR= UAPIDIR= install + +$(BPFTOOL): | $(BPFTOOL_OUTPUT) + $(call msg,BPFTOOL,$@) + $(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap + +$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL) + $(call msg,BPF,$@) + $(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \ + $(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \ + -c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + $(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + +$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL) + $(call msg,GEN-SKEL,$@) + $(Q)$(BPFTOOL) gen skeleton $< > $@ + +$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h + +$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT) + $(call msg,CC,$@) + $(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@ + +$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT) + $(call msg,BINARY,$@) + $(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@ + +.DELETE_ON_ERROR: +.SECONDARY: diff --git a/src/56-tc-flow-index/README.md b/src/56-tc-flow-index/README.md new file mode 100644 index 00000000..6f27720d --- /dev/null +++ b/src/56-tc-flow-index/README.md @@ -0,0 +1,834 @@ +# eBPF Tutorial: Building a Top-Flow Monitor with Dual Rbtree Indexing + +Ever tried building a network flow monitor and hit this wall? Every packet needs lightning-fast lookup to update counters, but at report time you want flows sorted by traffic volume. One index can't do both efficiently, and keeping two separate copies means counters drift apart. What if you could have both views of the same data, always in sync? + +This tutorial builds a TC egress flow monitor that solves exactly this problem. We'll index each IPv4 TCP or UDP flow in two red-black trees simultaneously: one keyed by five-tuple for fast packet lookup, another sorted by bytes for instant top-flow output. The magic ingredient is BPF refcounting, which lets both trees own the same flow record without duplication. + +> Complete source code: + +## The Dual-Index Challenge + +Network monitoring tools face a fundamental tension. When a packet arrives, you need to find the matching flow record by its five-tuple (source IP, destination IP, source port, destination port, protocol) in microseconds. But when the user asks "show me the top 10 flows," you need those same records sorted by traffic volume. + +Traditional solutions either maintain two independent data structures (which can drift out of sync) or rebuild the ranking on demand (which burns CPU and delays output). Neither feels right. + +The approach we take here is different. Each flow record lives in memory exactly once, but it participates in two different orderings through two embedded tree nodes. When you update the byte counter, both views see the change immediately because they point to the same object. This is possible thanks to three recent eBPF features: + +- **BPF object allocation** (`bpf_obj_new`): Create dynamically allocated structures in BPF programs +- **BPF refcounting** (`bpf_refcount_acquire`): Give multiple owners shared access to one object +- **Rbtree traversal** (`bpf_rbtree_root/left/right`): Search and walk trees without removing nodes + +Linux 6.4 introduced refcounting, and 6.16 completed the picture with tree traversal. Together, they enable data structures that were previously impossible in BPF. + +## How One Object Joins Two Trees + +Let's trace what happens when a new flow appears. The TC program sees a packet with five-tuple (10.0.0.1, 10.0.0.2, 50000, 80, TCP). It searches the identity tree by key and finds nothing. Time to create a new entry. + +First, `bpf_obj_new()` allocates a `flow_entry`. This structure embeds a `bpf_refcount` and two `bpf_rb_node` fields, one for each tree. Right after allocation, the object has exactly one owning reference. + +Next, `bpf_refcount_acquire()` creates a second owning reference to the same object. Now we have two references pointing to one piece of memory. The first `bpf_rbtree_add()` transfers one reference to the identity tree. The second call transfers the other to the traffic tree. At this point, both trees jointly own the flow entry, and the verifier knows no raw pointers escaped. + +When a subsequent packet for this flow arrives, we find the existing entry in the identity tree. But adding bytes changes the traffic ranking. The program removes only the traffic-tree node (which returns its owning reference), updates the counters, and reinserts at the new position. The identity node never moves, so lookup remains valid throughout. + +## Architecture Overview + +The implementation splits into four files: + +| File | Purpose | +|------|---------| +| `tc_flow_index.h` | Shared structures: five-tuple, snapshot, and cursor | +| `bpf_experimental.h` | Kfunc declarations for BPF graph APIs | +| `tc_flow_index.bpf.c` | BPF program: TC hook and snapshot syscall | +| `tc_flow_index.c` | User-space loader, demo traffic, and output | + +The BPF side handles packet parsing, tree maintenance, and cursor-based snapshot iteration. The user side attaches the TC program, optionally generates test traffic, and queries the index through `BPF_PROG_TEST_RUN`. + +## Shared Data Structures + +The header file defines the flow key, snapshot result, and traversal cursor: + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_H +#define __TC_FLOW_INDEX_H + +#define FLOW_COMM_LEN 16 + +struct flow_key { + unsigned int source_ip; + unsigned int destination_ip; + unsigned short source_port; + unsigned short destination_port; + unsigned char protocol; + unsigned char padding[3]; +}; + +struct flow_snapshot { + unsigned long long packets; + unsigned long long bytes; + unsigned long long last_seen_ns; + struct flow_key key; + unsigned int found; + char comm[FLOW_COMM_LEN]; +}; + +struct flow_cursor { + unsigned long long bytes; + unsigned long long packets; + struct flow_key key; + unsigned int valid; +}; + +#endif /* __TC_FLOW_INDEX_H */ +``` + +The `flow_key` keeps addresses and ports in network byte order, so the BPF program can copy packet fields directly without conversion. User space converts to host order only when formatting output. + +## Experimental Kfunc Declarations + +BPF graph APIs use kfuncs rather than stable UAPI helpers. This compatibility header declares the functions our program uses: + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_EXPERIMENTAL_H +#define __TC_FLOW_INDEX_EXPERIMENTAL_H + +#include + +#define __contains(name, node) \ + __attribute__((btf_decl_tag("contains:" #name ":" #node))) + +extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; +#define bpf_obj_new(type) \ + ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; +#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) + +extern void *bpf_refcount_acquire_impl(void *kptr, void *meta) __ksym; +#define bpf_refcount_acquire(kptr) \ + bpf_refcount_acquire_impl(kptr, NULL) + +extern int bpf_rbtree_add_impl(struct bpf_rb_root *root, + struct bpf_rb_node *node, + bool (*less)(struct bpf_rb_node *, + const struct bpf_rb_node *), + void *meta, __u64 off) __ksym; +#define bpf_rbtree_add(root, node, less) \ + bpf_rbtree_add_impl(root, node, less, NULL, 0) + +extern struct bpf_rb_node * +bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_first(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_root(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_left(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_right(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; + +#endif /* __TC_FLOW_INDEX_EXPERIMENTAL_H */ +``` + +The `__contains(flow_entry, by_identity)` annotation tells the verifier which containing type and member belong to each root. Combined with the neighboring spin lock, this lets the verifier enforce ownership and critical-section rules. These interfaces may evolve between kernels, which is why we note a concrete minimum version below. + +## The BPF Program + +Here's the complete BPF implementation. We'll walk through the key parts afterward. + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_experimental.h" +#include "tc_flow_index.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_TCP 6 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define INDEX_SEARCH_DEPTH 32 +#define TC_ACT_OK 0 + +const volatile __u32 maximum_entries = 4096; + +struct flow_entry { + struct bpf_refcount ref; + struct bpf_rb_node by_identity; + struct bpf_rb_node by_traffic; + struct flow_key key; + __u64 packets; + __u64 bytes; + __u64 last_seen_ns; + char comm[FLOW_COMM_LEN]; +}; + +#define private(name) \ + SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(index) struct bpf_spin_lock index_lock; +private(index) struct bpf_rb_root identity_root + __contains(flow_entry, by_identity); +private(index) struct bpf_rb_root traffic_root + __contains(flow_entry, by_traffic); + +struct flow_cursor snapshot_cursor; +struct flow_snapshot snapshot_result; +__u64 observed_packets; +__u64 indexed_flows; +__u64 dropped_new_flows; +__u64 allocation_failures; +__u64 refcount_failures; +__u64 rank_update_failures; + +static __always_inline int compare_key(const struct flow_key *left, + const struct flow_key *right) +{ + if (left->source_ip != right->source_ip) + return left->source_ip < right->source_ip ? -1 : 1; + if (left->destination_ip != right->destination_ip) + return left->destination_ip < right->destination_ip ? -1 : 1; + if (left->source_port != right->source_port) + return left->source_port < right->source_port ? -1 : 1; + if (left->destination_port != right->destination_port) + return left->destination_port < right->destination_port ? -1 : 1; + if (left->protocol != right->protocol) + return left->protocol < right->protocol ? -1 : 1; + return 0; +} + +static bool identity_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, + by_identity); + struct flow_entry *right = container_of(b, struct flow_entry, + by_identity); + + return compare_key(&left->key, &right->key) < 0; +} + +static __always_inline bool traffic_before(__u64 left_bytes, + __u64 left_packets, + const struct flow_key *left_key, + __u64 right_bytes, + __u64 right_packets, + const struct flow_key *right_key) +{ + if (left_bytes != right_bytes) + return left_bytes > right_bytes; + if (left_packets != right_packets) + return left_packets > right_packets; + return compare_key(left_key, right_key) < 0; +} + +static bool traffic_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, by_traffic); + struct flow_entry *right = container_of(b, struct flow_entry, by_traffic); + + return traffic_before(left->bytes, left->packets, &left->key, + right->bytes, right->packets, &right->key); +} + +static __always_inline struct flow_entry * +find_flow_locked(const struct flow_key *key) +{ + struct bpf_rb_node *rb = bpf_rbtree_root(&identity_root); + + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + struct flow_entry *entry = + container_of(rb, struct flow_entry, by_identity); + int comparison = compare_key(key, &entry->key); + + if (!comparison) + return entry; + if (comparison < 0) + rb = bpf_rbtree_left(&identity_root, rb); + else + rb = bpf_rbtree_right(&identity_root, rb); + } + return NULL; +} + +static __always_inline bool update_existing_locked(struct flow_entry *entry, + __u32 packet_bytes, + __u64 now_ns) +{ + struct flow_entry *owner; + struct bpf_rb_node *removed; + + removed = bpf_rbtree_remove(&traffic_root, &entry->by_traffic); + if (!removed) + return false; + owner = container_of(removed, struct flow_entry, by_traffic); + owner->packets++; + owner->bytes += packet_bytes; + owner->last_seen_ns = now_ns; + bpf_rbtree_add(&traffic_root, &owner->by_traffic, traffic_less); + return true; +} + +static __always_inline void update_flow(const struct flow_key *key, + __u32 packet_bytes, __u64 now_ns, + const char comm[FLOW_COMM_LEN]) +{ + struct flow_entry *entry, *new_entry, *traffic_owner; + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + return; + } + bpf_spin_unlock(&index_lock); + + new_entry = bpf_obj_new(typeof(*new_entry)); + if (!new_entry) { + __sync_fetch_and_add(&allocation_failures, 1); + return; + } + traffic_owner = bpf_refcount_acquire(new_entry); + if (!traffic_owner) { + __sync_fetch_and_add(&refcount_failures, 1); + bpf_obj_drop(new_entry); + return; + } + new_entry->key.source_ip = key->source_ip; + new_entry->key.destination_ip = key->destination_ip; + new_entry->key.source_port = key->source_port; + new_entry->key.destination_port = key->destination_port; + new_entry->key.protocol = key->protocol; + new_entry->packets = 1; + new_entry->bytes = packet_bytes; + new_entry->last_seen_ns = now_ns; + __builtin_memcpy(new_entry->comm, comm, sizeof(new_entry->comm)); + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + if (indexed_flows >= maximum_entries) { + dropped_new_flows++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + bpf_rbtree_add(&identity_root, &new_entry->by_identity, identity_less); + bpf_rbtree_add(&traffic_root, &traffic_owner->by_traffic, traffic_less); + indexed_flows++; + bpf_spin_unlock(&index_lock); +} + +static __always_inline bool parse_flow(struct __sk_buff *skb, + struct flow_key *key) +{ + struct ethhdr ethernet; + struct iphdr ip; + __u32 transport_offset; + __be16 ports[2]; + + if (bpf_skb_load_bytes(skb, 0, ðernet, sizeof(ethernet)) || + ethernet.h_proto != bpf_htons(ETH_P_IP)) + return false; + if (bpf_skb_load_bytes(skb, sizeof(ethernet), &ip, sizeof(ip)) || + ip.version != 4 || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + (ip.protocol != IPPROTO_TCP && ip.protocol != IPPROTO_UDP)) + return false; + transport_offset = sizeof(ethernet) + ip.ihl * 4; + if (bpf_skb_load_bytes(skb, transport_offset, ports, sizeof(ports))) + return false; + key->source_ip = ip.saddr; + key->destination_ip = ip.daddr; + key->source_port = ports[0]; + key->destination_port = ports[1]; + key->protocol = ip.protocol; + return true; +} + +SEC("tc") +int index_egress_flow(struct __sk_buff *skb) +{ + struct flow_key key = {}; + char comm[FLOW_COMM_LEN]; + + if (!parse_flow(skb, &key)) + return TC_ACT_OK; + __sync_fetch_and_add(&observed_packets, 1); + bpf_get_current_comm(comm, sizeof(comm)); + update_flow(&key, skb->len, bpf_ktime_get_ns(), comm); + return TC_ACT_OK; +} + +static __always_inline bool cursor_before_entry(const struct flow_cursor *cursor, + const struct flow_entry *entry) +{ + return traffic_before(cursor->bytes, cursor->packets, &cursor->key, + entry->bytes, entry->packets, &entry->key); +} + +SEC("syscall") +int snapshot_next(void *ctx) +{ + struct flow_entry *entry = NULL; + struct bpf_rb_node *candidate = NULL; + struct bpf_rb_node *rb; + + (void)ctx; + __builtin_memset(&snapshot_result, 0, sizeof(snapshot_result)); + bpf_spin_lock(&index_lock); + if (!snapshot_cursor.valid) { + candidate = bpf_rbtree_first(&traffic_root); + } else { + rb = bpf_rbtree_root(&traffic_root); + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + entry = container_of(rb, struct flow_entry, by_traffic); + if (cursor_before_entry(&snapshot_cursor, entry)) { + candidate = rb; + rb = bpf_rbtree_left(&traffic_root, rb); + } else { + rb = bpf_rbtree_right(&traffic_root, rb); + } + } + } + if (candidate) { + entry = container_of(candidate, struct flow_entry, by_traffic); + snapshot_result.packets = entry->packets; + snapshot_result.bytes = entry->bytes; + snapshot_result.last_seen_ns = entry->last_seen_ns; + snapshot_result.key.source_ip = entry->key.source_ip; + snapshot_result.key.destination_ip = entry->key.destination_ip; + snapshot_result.key.source_port = entry->key.source_port; + snapshot_result.key.destination_port = entry->key.destination_port; + snapshot_result.key.protocol = entry->key.protocol; + snapshot_result.found = 1; + __builtin_memcpy(snapshot_result.comm, entry->comm, + sizeof(snapshot_result.comm)); + } + bpf_spin_unlock(&index_lock); + return 0; +} +``` + +### Understanding the Flow Entry + +The `flow_entry` structure is the heart of this design. It embeds one `bpf_refcount` for ownership tracking and two `bpf_rb_node` fields for participation in both trees. The lock, `identity_root`, and `traffic_root` share a private map value, ensuring all access is serialized. + +### The Packet Path + +When `index_egress_flow` runs, `parse_flow()` extracts the five-tuple from non-fragmented IPv4 TCP and UDP packets. The main function always returns `TC_ACT_OK` because we're observing, not filtering. + +The `update_flow()` function first searches under lock. If the flow exists, it removes the traffic node, updates counters, and reinserts. If not, it allocates outside the lock, then locks again for a second lookup. This double-check handles two CPUs discovering the same new flow simultaneously: one wins the insertion race, the other updates the existing entry and drops its unused references. + +### The Snapshot Mechanism + +The `snapshot_next` syscall program reads the traffic tree under the same lock. With an empty cursor, it returns `bpf_rbtree_first()`. Subsequent calls search for the first entry after the previous position. Since the cursor and result live in BSS, user space can fetch one ranked entry at a time without receiving a per-packet event stream. + +## The User-Space Program + +The loader attaches the TC program, optionally generates demo traffic, detaches before reading, then queries through `BPF_PROG_TEST_RUN`: + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "tc_flow_index.h" +#include "tc_flow_index.skel.h" + +#define MAX_TOP 64 + +struct options { + const char *interface; + unsigned int duration_seconds; + unsigned int top; + bool demo; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || !parsed || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--duration SEC] [--top N]\n" + " %s --demo [--top N]\n", program, program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "duration", required_argument, NULL, 'd' }, + { "top", required_argument, NULL, 't' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:d:t:Dh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'd': + if (parse_uint(optarg, 86400, &options->duration_seconds)) + return -1; + break; + case 't': + if (parse_uint(optarg, MAX_TOP, &options->top)) + return -1; + break; + case 'D': options->demo = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + if (options->demo) { + if (options->interface) + return -1; + options->interface = "lo"; + } + return optind == argc && options->interface ? 0 : -1; +} + +static int send_demo_flow(unsigned int datagrams, size_t payload_size) +{ + struct sockaddr_in receiver_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + struct sockaddr_in sender_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + socklen_t address_length = sizeof(receiver_address); + char payload[1000] = {}; + int receiver = -1, sender = -1; + int err = -1; + + if (payload_size > sizeof(payload)) + return -1; + receiver = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + sender = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (receiver < 0 || sender < 0 || + bind(receiver, (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) || + getsockname(receiver, (struct sockaddr *)&receiver_address, + &address_length) || + bind(sender, (struct sockaddr *)&sender_address, + sizeof(sender_address))) + goto cleanup; + for (unsigned int i = 0; i < datagrams; i++) + if (sendto(sender, payload, payload_size, 0, + (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) != (ssize_t)payload_size) + goto cleanup; + err = 0; + +cleanup: + if (receiver >= 0) close(receiver); + if (sender >= 0) close(sender); + return err; +} + +static int run_demo_traffic(void) +{ + return send_demo_flow(2, 100) || + send_demo_flow(4, 300) || + send_demo_flow(6, 700); +} + +static int snapshot_next(struct tc_flow_index_bpf *skel, + struct flow_snapshot *result) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + + if (bpf_prog_test_run_opts(bpf_program__fd(skel->progs.snapshot_next), + &opts)) { + fprintf(stderr, "failed to query BPF flow index: %s\n", + strerror(errno)); + return -1; + } + *result = skel->bss->snapshot_result; + if (result->found) { + skel->bss->snapshot_cursor.bytes = result->bytes; + skel->bss->snapshot_cursor.packets = result->packets; + skel->bss->snapshot_cursor.key = result->key; + skel->bss->snapshot_cursor.valid = 1; + } + return 0; +} + +static int print_flows(struct tc_flow_index_bpf *skel, unsigned int top) +{ + struct flow_snapshot entry; + + memset(&skel->bss->snapshot_cursor, 0, + sizeof(skel->bss->snapshot_cursor)); + printf("\nTop egress flows, ranked in the BPF rbtree by bytes:\n"); + printf("%-21s %-21s %-5s %10s %12s %-16s\n", + "SOURCE", "DESTINATION", "PROTO", "PACKETS", "BYTES", "COMM"); + for (unsigned int i = 0; i < top; i++) { + char source_ip[INET_ADDRSTRLEN], destination_ip[INET_ADDRSTRLEN]; + char source[64], destination[64]; + + if (snapshot_next(skel, &entry)) + return -1; + if (!entry.found) + break; + inet_ntop(AF_INET, &entry.key.source_ip, source_ip, + sizeof(source_ip)); + inet_ntop(AF_INET, &entry.key.destination_ip, destination_ip, + sizeof(destination_ip)); + snprintf(source, sizeof(source), "%s:%u", source_ip, + ntohs(entry.key.source_port)); + snprintf(destination, sizeof(destination), "%s:%u", + destination_ip, ntohs(entry.key.destination_port)); + printf("%-21s %-21s %-5s %10llu %12llu %-16s\n", + source, destination, + entry.key.protocol == IPPROTO_TCP ? "TCP" : "UDP", + entry.packets, entry.bytes, entry.comm); + } + return 0; +} + +static int attach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *attach, + bool *hook_created, bool *attached) +{ + int err = bpf_tc_hook_create(hook); + + if (!err) + *hook_created = true; + else if (err != -EEXIST) { + fprintf(stderr, "failed to create clsact hook: %s\n", strerror(-err)); + return -1; + } + err = bpf_tc_attach(hook, attach); + if (err) { + fprintf(stderr, "failed to attach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = true; + return 0; +} + +static int capture_traffic(const struct options *options) +{ + unsigned long long deadline; + + if (options->demo) + return run_demo_traffic(); + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + deadline = monotonic_ns() + + (unsigned long long)options->duration_seconds * 1000000000ULL; + while (!stop && monotonic_ns() < deadline) { + struct timespec pause = { .tv_nsec = 100000000 }; + + nanosleep(&pause, NULL); + } + return 0; +} + +static int detach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *detach, bool *attached) +{ + int err = bpf_tc_detach(hook, detach); + + if (err) { + fprintf(stderr, "failed to detach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = false; + return 0; +} + +int main(int argc, char **argv) +{ + struct options options = { .duration_seconds = 10, .top = 10 }; + struct tc_flow_index_bpf *skel = NULL; + LIBBPF_OPTS(bpf_tc_hook, hook, .attach_point = BPF_TC_EGRESS); + LIBBPF_OPTS(bpf_tc_opts, attach, .handle = 1, .priority = 1); + LIBBPF_OPTS(bpf_tc_opts, detach, .handle = 1, .priority = 1); + bool hook_created = false; + bool attached = false; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + hook.ifindex = if_nametoindex(options.interface); + if (!hook.ifindex) { + fprintf(stderr, "interface does not exist: %s\n", options.interface); + return 2; + } + + skel = tc_flow_index_bpf__open_and_load(); + if (!skel) { + fprintf(stderr, "failed to load TC flow index\n"); + goto cleanup; + } + attach.prog_fd = bpf_program__fd(skel->progs.index_egress_flow); + if (attach_tc_program(&hook, &attach, &hook_created, &attached)) + goto cleanup; + printf("Indexing IPv4 TCP/UDP egress flows on %s for %u seconds.\n", + options.interface, options.demo ? 0 : options.duration_seconds); + + if (capture_traffic(&options)) { + fprintf(stderr, "failed to generate traffic\n"); + goto cleanup; + } + + if (detach_tc_program(&hook, &detach, &attached)) + goto cleanup; + if (print_flows(skel, options.top)) + goto cleanup; + printf("observed_packets=%llu indexed_flows=%llu dropped_new=%llu " + "allocation_failures=%llu refcount_failures=%llu rank_update_failures=%llu\n", + (unsigned long long)skel->bss->observed_packets, + (unsigned long long)skel->bss->indexed_flows, + (unsigned long long)skel->bss->dropped_new_flows, + (unsigned long long)skel->bss->allocation_failures, + (unsigned long long)skel->bss->refcount_failures, + (unsigned long long)skel->bss->rank_update_failures); + if (options.demo && + (skel->bss->indexed_flows != 3 || skel->bss->rank_update_failures)) + goto cleanup; + err = 0; + +cleanup: + if (attached) + bpf_tc_detach(&hook, &detach); + if (hook_created) + bpf_tc_hook_destroy(&hook); + tc_flow_index_bpf__destroy(skel); + return err; +} +``` + +Detaching before output freezes packet updates, so the cursor walks a stable traffic ordering. The demo mode creates three UDP flows over loopback: 2 packets at 100 bytes, 4 at 300 bytes, and 6 at 700 bytes. With headers, these become 284, 1368, and 4452 bytes total, producing a deterministic ranking. + +## Building and Running + +Build the tool: + +```bash +cd src/56-tc-flow-index +make +``` + +Monitor real egress traffic for 30 seconds: + +```bash +sudo ./tc_flow_index --interface eth0 --duration 30 --top 10 +``` + +Run the deterministic demo: + +```bash +sudo ./tc_flow_index --demo --top 3 +``` + +Expected demo output: + +```text +Indexing IPv4 TCP/UDP egress flows on lo for 0 seconds. + +Top egress flows, ranked in the BPF rbtree by bytes: +SOURCE DESTINATION PROTO PACKETS BYTES COMM +127.0.0.1:38918 127.0.0.1:47669 UDP 6 4452 tc_flow_index +127.0.0.1:57860 127.0.0.1:52539 UDP 4 1368 tc_flow_index +127.0.0.1:54177 127.0.0.1:45452 UDP 2 284 tc_flow_index +observed_packets=12 indexed_flows=3 dropped_new=0 allocation_failures=0 refcount_failures=0 rank_update_failures=0 +``` + +The table arrives pre-sorted by bytes. Twelve packets created three identity entries, and all ownership counters remain zero, confirming correct dual-tree management. + +## Requirements + +| Requirement | Details | +|-------------|---------| +| Kernel | Linux 6.16+ (refcounted BPF objects + rbtree search kfuncs) | +| Config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_NET_SCHED`, `CONFIG_NET_CLS_BPF` | +| Privileges | Root or CAP_BPF + CAP_NET_ADMIN | +| Interface | Any interface supporting clsact egress | +| Architecture | Tested on x86-64 | + +## Extending the Design + +This example covers non-fragmented IPv4 TCP/UDP egress with a fixed 4096-entry capacity. For production use, consider: + +- **Idle expiry**: Remove flows that haven't seen traffic for N seconds, freeing both tree nodes before dropping references +- **Ingress indexing**: Add a second TC program for incoming packets +- **IPv6 support**: Extend the flow key structure +- **Real-time streaming**: Use a ring buffer alongside the trees for per-packet events + +## Summary + +This tutorial showed how BPF object ownership enables sophisticated data structures that weren't possible before. The identity tree handles fast five-tuple lookups, the traffic tree provides instant top-flow rankings, and refcounting lets both share one object without state duplication. The technique generalizes to any scenario where you need multiple orderings over the same dataset. + +> If you'd like to dive deeper into eBPF, check out our tutorial repository at or visit our website at . + +## References + +- [BPF graph data structures](https://docs.kernel.org/bpf/graph_ds_impl.html) +- [Linux rbtree search selftest](https://github.com/torvalds/linux/blob/v6.16/tools/testing/selftests/bpf/progs/rbtree_search.c) +- [Linux refcounted graph selftests](https://github.com/torvalds/linux/tree/v6.16/tools/testing/selftests/bpf/progs) +- [libbpf TC attach implementation](https://github.com/libbpf/libbpf/blob/master/src/netlink.c) diff --git a/src/56-tc-flow-index/README.zh.md b/src/56-tc-flow-index/README.zh.md new file mode 100644 index 00000000..6c855794 --- /dev/null +++ b/src/56-tc-flow-index/README.zh.md @@ -0,0 +1,834 @@ +# eBPF 实战教程：用双红黑树索引构建 Top-Flow 监控器 + +做过流量监控的人可能都遇到过这个问题：每个报文到达时，需要按五元组快速找到对应的 flow 记录更新计数器；但输出报告时，又希望同一批数据按流量大小排序，让最繁忙的 flow 排在前面。一份索引很难同时满足两种需求，而维护两份独立副本又会让计数器逐渐失去同步。有没有办法让两种视图共享同一份数据，始终保持一致？ + +本教程构建的 TC egress 流量监控器正是为了解决这个问题。我们把每条 IPv4 TCP 或 UDP 流同时索引到两棵红黑树中：一棵按五元组键值排序，用于快速报文查找；另一棵按字节数排序，用于即时输出 top-flow 结果。核心技术是 BPF 引用计数，它让两棵树能够共同拥有同一条 flow 记录，无需复制数据。 + +> 完整源代码： + +## 双索引的挑战 + +网络监控工具面临一个根本性的矛盾。报文到达时，需要在微秒级别内按五元组（源 IP、目的 IP、源端口、目的端口、协议）找到对应的 flow 记录。但当用户查询"显示前 10 条流量最大的 flow"时，又需要这些记录按流量大小排序。 + +传统方案要么维护两份独立的数据结构（可能出现不一致），要么在查询时重新排序（消耗 CPU 并延迟输出）。两种方案都不够理想。 + +我们采用的方法完全不同。每条 flow 记录在内存中只存在一份，但通过两个内嵌的树节点参与两种不同的排序。更新字节计数器时，两种视图立即看到变化，因为它们指向同一个对象。这得益于三个最新的 eBPF 特性： + +- **BPF 对象分配**（`bpf_obj_new`）：在 BPF 程序中创建动态分配的结构体 +- **BPF 引用计数**（`bpf_refcount_acquire`）：让多个所有者共享同一个对象 +- **红黑树遍历**（`bpf_rbtree_root/left/right`）：不移除节点也能搜索和遍历树 + +Linux 6.4 引入了引用计数，6.16 又添加了树遍历功能。两者结合，使得以前在 BPF 中不可能实现的数据结构成为现实。 + +## 一个对象如何加入两棵树 + +我们来追踪一条新 flow 出现时会发生什么。TC 程序看到一个五元组为 (10.0.0.1, 10.0.0.2, 50000, 80, TCP) 的报文。它在 identity 树中按键值搜索，没有找到匹配项。是时候创建新条目了。 + +首先，`bpf_obj_new()` 分配一个 `flow_entry`。这个结构体内嵌一个 `bpf_refcount` 和两个 `bpf_rb_node` 字段，分别对应两棵树。刚分配时，对象恰好有一个 owning reference。 + +接下来，`bpf_refcount_acquire()` 为同一个对象创建第二个 owning reference。现在我们有两个引用指向同一块内存。第一次 `bpf_rbtree_add()` 把一个引用交给 identity 树。第二次调用把另一个交给 traffic 树。此时，两棵树共同拥有这个 flow entry，验证器知道没有裸指针逃逸。 + +当这条 flow 的后续报文到达时，我们在 identity 树中找到已有条目。但增加字节数会改变 traffic 排名。程序只移除 traffic 树节点（返回其 owning reference），更新计数器，然后在新位置重新插入。identity 节点始终不动，所以查找在整个过程中保持有效。 + +## 架构概览 + +实现分为四个文件： + +| 文件 | 作用 | +|------|------| +| `tc_flow_index.h` | 共享结构体：五元组、快照、游标 | +| `bpf_experimental.h` | BPF graph API 的 kfunc 声明 | +| `tc_flow_index.bpf.c` | BPF 程序：TC hook 和快照系统调用 | +| `tc_flow_index.c` | 用户态加载器、演示流量和输出 | + +BPF 端负责报文解析、树维护和基于游标的快照迭代。用户端挂载 TC 程序，可选生成测试流量，并通过 `BPF_PROG_TEST_RUN` 查询索引。 + +## 共享数据结构 + +头文件定义 flow key、快照结果和遍历游标： + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_H +#define __TC_FLOW_INDEX_H + +#define FLOW_COMM_LEN 16 + +struct flow_key { + unsigned int source_ip; + unsigned int destination_ip; + unsigned short source_port; + unsigned short destination_port; + unsigned char protocol; + unsigned char padding[3]; +}; + +struct flow_snapshot { + unsigned long long packets; + unsigned long long bytes; + unsigned long long last_seen_ns; + struct flow_key key; + unsigned int found; + char comm[FLOW_COMM_LEN]; +}; + +struct flow_cursor { + unsigned long long bytes; + unsigned long long packets; + struct flow_key key; + unsigned int valid; +}; + +#endif /* __TC_FLOW_INDEX_H */ +``` + +`flow_key` 中的地址和端口保留网络字节序，BPF 程序可以直接从报文复制字段。只有用户态格式化输出时才转换为主机字节序。 + +## 实验性 kfunc 声明 + +BPF graph API 使用 kfunc 而非稳定的 UAPI helper。这个兼容头文件声明了程序使用的函数： + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_EXPERIMENTAL_H +#define __TC_FLOW_INDEX_EXPERIMENTAL_H + +#include + +#define __contains(name, node) \ + __attribute__((btf_decl_tag("contains:" #name ":" #node))) + +extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; +#define bpf_obj_new(type) \ + ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; +#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) + +extern void *bpf_refcount_acquire_impl(void *kptr, void *meta) __ksym; +#define bpf_refcount_acquire(kptr) \ + bpf_refcount_acquire_impl(kptr, NULL) + +extern int bpf_rbtree_add_impl(struct bpf_rb_root *root, + struct bpf_rb_node *node, + bool (*less)(struct bpf_rb_node *, + const struct bpf_rb_node *), + void *meta, __u64 off) __ksym; +#define bpf_rbtree_add(root, node, less) \ + bpf_rbtree_add_impl(root, node, less, NULL, 0) + +extern struct bpf_rb_node * +bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_first(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_root(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_left(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_right(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; + +#endif /* __TC_FLOW_INDEX_EXPERIMENTAL_H */ +``` + +`__contains(flow_entry, by_identity)` 注解告诉验证器每个 root 对应的 containing type 和 member。结合相邻的 spin lock，验证器可以强制检查 ownership 和临界区规则。这些接口可能随内核版本演进，因此我们在下文注明了最低版本要求。 + +## BPF 程序 + +下面是完整的 BPF 实现，之后我们会逐一讲解关键部分。 + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_experimental.h" +#include "tc_flow_index.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_TCP 6 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define INDEX_SEARCH_DEPTH 32 +#define TC_ACT_OK 0 + +const volatile __u32 maximum_entries = 4096; + +struct flow_entry { + struct bpf_refcount ref; + struct bpf_rb_node by_identity; + struct bpf_rb_node by_traffic; + struct flow_key key; + __u64 packets; + __u64 bytes; + __u64 last_seen_ns; + char comm[FLOW_COMM_LEN]; +}; + +#define private(name) \ + SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(index) struct bpf_spin_lock index_lock; +private(index) struct bpf_rb_root identity_root + __contains(flow_entry, by_identity); +private(index) struct bpf_rb_root traffic_root + __contains(flow_entry, by_traffic); + +struct flow_cursor snapshot_cursor; +struct flow_snapshot snapshot_result; +__u64 observed_packets; +__u64 indexed_flows; +__u64 dropped_new_flows; +__u64 allocation_failures; +__u64 refcount_failures; +__u64 rank_update_failures; + +static __always_inline int compare_key(const struct flow_key *left, + const struct flow_key *right) +{ + if (left->source_ip != right->source_ip) + return left->source_ip < right->source_ip ? -1 : 1; + if (left->destination_ip != right->destination_ip) + return left->destination_ip < right->destination_ip ? -1 : 1; + if (left->source_port != right->source_port) + return left->source_port < right->source_port ? -1 : 1; + if (left->destination_port != right->destination_port) + return left->destination_port < right->destination_port ? -1 : 1; + if (left->protocol != right->protocol) + return left->protocol < right->protocol ? -1 : 1; + return 0; +} + +static bool identity_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, + by_identity); + struct flow_entry *right = container_of(b, struct flow_entry, + by_identity); + + return compare_key(&left->key, &right->key) < 0; +} + +static __always_inline bool traffic_before(__u64 left_bytes, + __u64 left_packets, + const struct flow_key *left_key, + __u64 right_bytes, + __u64 right_packets, + const struct flow_key *right_key) +{ + if (left_bytes != right_bytes) + return left_bytes > right_bytes; + if (left_packets != right_packets) + return left_packets > right_packets; + return compare_key(left_key, right_key) < 0; +} + +static bool traffic_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, by_traffic); + struct flow_entry *right = container_of(b, struct flow_entry, by_traffic); + + return traffic_before(left->bytes, left->packets, &left->key, + right->bytes, right->packets, &right->key); +} + +static __always_inline struct flow_entry * +find_flow_locked(const struct flow_key *key) +{ + struct bpf_rb_node *rb = bpf_rbtree_root(&identity_root); + + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + struct flow_entry *entry = + container_of(rb, struct flow_entry, by_identity); + int comparison = compare_key(key, &entry->key); + + if (!comparison) + return entry; + if (comparison < 0) + rb = bpf_rbtree_left(&identity_root, rb); + else + rb = bpf_rbtree_right(&identity_root, rb); + } + return NULL; +} + +static __always_inline bool update_existing_locked(struct flow_entry *entry, + __u32 packet_bytes, + __u64 now_ns) +{ + struct flow_entry *owner; + struct bpf_rb_node *removed; + + removed = bpf_rbtree_remove(&traffic_root, &entry->by_traffic); + if (!removed) + return false; + owner = container_of(removed, struct flow_entry, by_traffic); + owner->packets++; + owner->bytes += packet_bytes; + owner->last_seen_ns = now_ns; + bpf_rbtree_add(&traffic_root, &owner->by_traffic, traffic_less); + return true; +} + +static __always_inline void update_flow(const struct flow_key *key, + __u32 packet_bytes, __u64 now_ns, + const char comm[FLOW_COMM_LEN]) +{ + struct flow_entry *entry, *new_entry, *traffic_owner; + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + return; + } + bpf_spin_unlock(&index_lock); + + new_entry = bpf_obj_new(typeof(*new_entry)); + if (!new_entry) { + __sync_fetch_and_add(&allocation_failures, 1); + return; + } + traffic_owner = bpf_refcount_acquire(new_entry); + if (!traffic_owner) { + __sync_fetch_and_add(&refcount_failures, 1); + bpf_obj_drop(new_entry); + return; + } + new_entry->key.source_ip = key->source_ip; + new_entry->key.destination_ip = key->destination_ip; + new_entry->key.source_port = key->source_port; + new_entry->key.destination_port = key->destination_port; + new_entry->key.protocol = key->protocol; + new_entry->packets = 1; + new_entry->bytes = packet_bytes; + new_entry->last_seen_ns = now_ns; + __builtin_memcpy(new_entry->comm, comm, sizeof(new_entry->comm)); + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + if (indexed_flows >= maximum_entries) { + dropped_new_flows++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + bpf_rbtree_add(&identity_root, &new_entry->by_identity, identity_less); + bpf_rbtree_add(&traffic_root, &traffic_owner->by_traffic, traffic_less); + indexed_flows++; + bpf_spin_unlock(&index_lock); +} + +static __always_inline bool parse_flow(struct __sk_buff *skb, + struct flow_key *key) +{ + struct ethhdr ethernet; + struct iphdr ip; + __u32 transport_offset; + __be16 ports[2]; + + if (bpf_skb_load_bytes(skb, 0, ðernet, sizeof(ethernet)) || + ethernet.h_proto != bpf_htons(ETH_P_IP)) + return false; + if (bpf_skb_load_bytes(skb, sizeof(ethernet), &ip, sizeof(ip)) || + ip.version != 4 || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + (ip.protocol != IPPROTO_TCP && ip.protocol != IPPROTO_UDP)) + return false; + transport_offset = sizeof(ethernet) + ip.ihl * 4; + if (bpf_skb_load_bytes(skb, transport_offset, ports, sizeof(ports))) + return false; + key->source_ip = ip.saddr; + key->destination_ip = ip.daddr; + key->source_port = ports[0]; + key->destination_port = ports[1]; + key->protocol = ip.protocol; + return true; +} + +SEC("tc") +int index_egress_flow(struct __sk_buff *skb) +{ + struct flow_key key = {}; + char comm[FLOW_COMM_LEN]; + + if (!parse_flow(skb, &key)) + return TC_ACT_OK; + __sync_fetch_and_add(&observed_packets, 1); + bpf_get_current_comm(comm, sizeof(comm)); + update_flow(&key, skb->len, bpf_ktime_get_ns(), comm); + return TC_ACT_OK; +} + +static __always_inline bool cursor_before_entry(const struct flow_cursor *cursor, + const struct flow_entry *entry) +{ + return traffic_before(cursor->bytes, cursor->packets, &cursor->key, + entry->bytes, entry->packets, &entry->key); +} + +SEC("syscall") +int snapshot_next(void *ctx) +{ + struct flow_entry *entry = NULL; + struct bpf_rb_node *candidate = NULL; + struct bpf_rb_node *rb; + + (void)ctx; + __builtin_memset(&snapshot_result, 0, sizeof(snapshot_result)); + bpf_spin_lock(&index_lock); + if (!snapshot_cursor.valid) { + candidate = bpf_rbtree_first(&traffic_root); + } else { + rb = bpf_rbtree_root(&traffic_root); + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + entry = container_of(rb, struct flow_entry, by_traffic); + if (cursor_before_entry(&snapshot_cursor, entry)) { + candidate = rb; + rb = bpf_rbtree_left(&traffic_root, rb); + } else { + rb = bpf_rbtree_right(&traffic_root, rb); + } + } + } + if (candidate) { + entry = container_of(candidate, struct flow_entry, by_traffic); + snapshot_result.packets = entry->packets; + snapshot_result.bytes = entry->bytes; + snapshot_result.last_seen_ns = entry->last_seen_ns; + snapshot_result.key.source_ip = entry->key.source_ip; + snapshot_result.key.destination_ip = entry->key.destination_ip; + snapshot_result.key.source_port = entry->key.source_port; + snapshot_result.key.destination_port = entry->key.destination_port; + snapshot_result.key.protocol = entry->key.protocol; + snapshot_result.found = 1; + __builtin_memcpy(snapshot_result.comm, entry->comm, + sizeof(snapshot_result.comm)); + } + bpf_spin_unlock(&index_lock); + return 0; +} +``` + +### 理解 Flow Entry + +`flow_entry` 结构体是整个设计的核心。它内嵌一个 `bpf_refcount` 用于 ownership 跟踪，以及两个 `bpf_rb_node` 字段用于参与两棵树。lock、`identity_root` 和 `traffic_root` 共享一个 private map value，确保所有访问都是串行化的。 + +### 报文处理路径 + +当 `index_egress_flow` 运行时，`parse_flow()` 从非分片的 IPv4 TCP 和 UDP 报文中提取五元组。主函数始终返回 `TC_ACT_OK`，因为我们只是观察，不是过滤。 + +`update_flow()` 函数首先在锁内搜索。如果 flow 存在，它移除 traffic 节点、更新计数器并重新插入。如果不存在，则在锁外分配，然后重新加锁做第二次查找。这个 double-check 处理了两个 CPU 同时发现新 flow 的情况：一个赢得插入竞争，另一个更新已有条目并释放自己未使用的引用。 + +### 快照机制 + +`snapshot_next` syscall 程序在同一把锁下读取 traffic 树。空游标时返回 `bpf_rbtree_first()`，后续调用则搜索上一个位置之后的第一个条目。由于游标和结果都在 BSS 中，用户态可以逐个获取排序好的条目，无需接收每报文事件流。 + +## 用户态程序 + +加载器挂载 TC 程序，可选生成演示流量，读取前先解挂，然后通过 `BPF_PROG_TEST_RUN` 查询： + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "tc_flow_index.h" +#include "tc_flow_index.skel.h" + +#define MAX_TOP 64 + +struct options { + const char *interface; + unsigned int duration_seconds; + unsigned int top; + bool demo; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || !parsed || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--duration SEC] [--top N]\n" + " %s --demo [--top N]\n", program, program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "duration", required_argument, NULL, 'd' }, + { "top", required_argument, NULL, 't' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:d:t:Dh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'd': + if (parse_uint(optarg, 86400, &options->duration_seconds)) + return -1; + break; + case 't': + if (parse_uint(optarg, MAX_TOP, &options->top)) + return -1; + break; + case 'D': options->demo = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + if (options->demo) { + if (options->interface) + return -1; + options->interface = "lo"; + } + return optind == argc && options->interface ? 0 : -1; +} + +static int send_demo_flow(unsigned int datagrams, size_t payload_size) +{ + struct sockaddr_in receiver_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + struct sockaddr_in sender_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + socklen_t address_length = sizeof(receiver_address); + char payload[1000] = {}; + int receiver = -1, sender = -1; + int err = -1; + + if (payload_size > sizeof(payload)) + return -1; + receiver = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + sender = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (receiver < 0 || sender < 0 || + bind(receiver, (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) || + getsockname(receiver, (struct sockaddr *)&receiver_address, + &address_length) || + bind(sender, (struct sockaddr *)&sender_address, + sizeof(sender_address))) + goto cleanup; + for (unsigned int i = 0; i < datagrams; i++) + if (sendto(sender, payload, payload_size, 0, + (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) != (ssize_t)payload_size) + goto cleanup; + err = 0; + +cleanup: + if (receiver >= 0) close(receiver); + if (sender >= 0) close(sender); + return err; +} + +static int run_demo_traffic(void) +{ + return send_demo_flow(2, 100) || + send_demo_flow(4, 300) || + send_demo_flow(6, 700); +} + +static int snapshot_next(struct tc_flow_index_bpf *skel, + struct flow_snapshot *result) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + + if (bpf_prog_test_run_opts(bpf_program__fd(skel->progs.snapshot_next), + &opts)) { + fprintf(stderr, "failed to query BPF flow index: %s\n", + strerror(errno)); + return -1; + } + *result = skel->bss->snapshot_result; + if (result->found) { + skel->bss->snapshot_cursor.bytes = result->bytes; + skel->bss->snapshot_cursor.packets = result->packets; + skel->bss->snapshot_cursor.key = result->key; + skel->bss->snapshot_cursor.valid = 1; + } + return 0; +} + +static int print_flows(struct tc_flow_index_bpf *skel, unsigned int top) +{ + struct flow_snapshot entry; + + memset(&skel->bss->snapshot_cursor, 0, + sizeof(skel->bss->snapshot_cursor)); + printf("\nTop egress flows, ranked in the BPF rbtree by bytes:\n"); + printf("%-21s %-21s %-5s %10s %12s %-16s\n", + "SOURCE", "DESTINATION", "PROTO", "PACKETS", "BYTES", "COMM"); + for (unsigned int i = 0; i < top; i++) { + char source_ip[INET_ADDRSTRLEN], destination_ip[INET_ADDRSTRLEN]; + char source[64], destination[64]; + + if (snapshot_next(skel, &entry)) + return -1; + if (!entry.found) + break; + inet_ntop(AF_INET, &entry.key.source_ip, source_ip, + sizeof(source_ip)); + inet_ntop(AF_INET, &entry.key.destination_ip, destination_ip, + sizeof(destination_ip)); + snprintf(source, sizeof(source), "%s:%u", source_ip, + ntohs(entry.key.source_port)); + snprintf(destination, sizeof(destination), "%s:%u", + destination_ip, ntohs(entry.key.destination_port)); + printf("%-21s %-21s %-5s %10llu %12llu %-16s\n", + source, destination, + entry.key.protocol == IPPROTO_TCP ? "TCP" : "UDP", + entry.packets, entry.bytes, entry.comm); + } + return 0; +} + +static int attach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *attach, + bool *hook_created, bool *attached) +{ + int err = bpf_tc_hook_create(hook); + + if (!err) + *hook_created = true; + else if (err != -EEXIST) { + fprintf(stderr, "failed to create clsact hook: %s\n", strerror(-err)); + return -1; + } + err = bpf_tc_attach(hook, attach); + if (err) { + fprintf(stderr, "failed to attach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = true; + return 0; +} + +static int capture_traffic(const struct options *options) +{ + unsigned long long deadline; + + if (options->demo) + return run_demo_traffic(); + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + deadline = monotonic_ns() + + (unsigned long long)options->duration_seconds * 1000000000ULL; + while (!stop && monotonic_ns() < deadline) { + struct timespec pause = { .tv_nsec = 100000000 }; + + nanosleep(&pause, NULL); + } + return 0; +} + +static int detach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *detach, bool *attached) +{ + int err = bpf_tc_detach(hook, detach); + + if (err) { + fprintf(stderr, "failed to detach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = false; + return 0; +} + +int main(int argc, char **argv) +{ + struct options options = { .duration_seconds = 10, .top = 10 }; + struct tc_flow_index_bpf *skel = NULL; + LIBBPF_OPTS(bpf_tc_hook, hook, .attach_point = BPF_TC_EGRESS); + LIBBPF_OPTS(bpf_tc_opts, attach, .handle = 1, .priority = 1); + LIBBPF_OPTS(bpf_tc_opts, detach, .handle = 1, .priority = 1); + bool hook_created = false; + bool attached = false; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + hook.ifindex = if_nametoindex(options.interface); + if (!hook.ifindex) { + fprintf(stderr, "interface does not exist: %s\n", options.interface); + return 2; + } + + skel = tc_flow_index_bpf__open_and_load(); + if (!skel) { + fprintf(stderr, "failed to load TC flow index\n"); + goto cleanup; + } + attach.prog_fd = bpf_program__fd(skel->progs.index_egress_flow); + if (attach_tc_program(&hook, &attach, &hook_created, &attached)) + goto cleanup; + printf("Indexing IPv4 TCP/UDP egress flows on %s for %u seconds.\n", + options.interface, options.demo ? 0 : options.duration_seconds); + + if (capture_traffic(&options)) { + fprintf(stderr, "failed to generate traffic\n"); + goto cleanup; + } + + if (detach_tc_program(&hook, &detach, &attached)) + goto cleanup; + if (print_flows(skel, options.top)) + goto cleanup; + printf("observed_packets=%llu indexed_flows=%llu dropped_new=%llu " + "allocation_failures=%llu refcount_failures=%llu rank_update_failures=%llu\n", + (unsigned long long)skel->bss->observed_packets, + (unsigned long long)skel->bss->indexed_flows, + (unsigned long long)skel->bss->dropped_new_flows, + (unsigned long long)skel->bss->allocation_failures, + (unsigned long long)skel->bss->refcount_failures, + (unsigned long long)skel->bss->rank_update_failures); + if (options.demo && + (skel->bss->indexed_flows != 3 || skel->bss->rank_update_failures)) + goto cleanup; + err = 0; + +cleanup: + if (attached) + bpf_tc_detach(&hook, &detach); + if (hook_created) + bpf_tc_hook_destroy(&hook); + tc_flow_index_bpf__destroy(skel); + return err; +} +``` + +输出前解挂会冻结报文更新，让游标可以遍历一份稳定的 traffic 排序。演示模式创建三条 loopback 上的 UDP 流：2 个报文 100 字节、4 个报文 300 字节、6 个报文 700 字节。加上协议头后，分别是 284、1368 和 4452 字节，产生确定性的排名。 + +## 编译和运行 + +构建工具： + +```bash +cd src/56-tc-flow-index +make +``` + +监控真实 egress 流量 30 秒： + +```bash +sudo ./tc_flow_index --interface eth0 --duration 30 --top 10 +``` + +运行确定性演示： + +```bash +sudo ./tc_flow_index --demo --top 3 +``` + +预期演示输出： + +```text +Indexing IPv4 TCP/UDP egress flows on lo for 0 seconds. + +Top egress flows, ranked in the BPF rbtree by bytes: +SOURCE DESTINATION PROTO PACKETS BYTES COMM +127.0.0.1:38918 127.0.0.1:47669 UDP 6 4452 tc_flow_index +127.0.0.1:57860 127.0.0.1:52539 UDP 4 1368 tc_flow_index +127.0.0.1:54177 127.0.0.1:45452 UDP 2 284 tc_flow_index +observed_packets=12 indexed_flows=3 dropped_new=0 allocation_failures=0 refcount_failures=0 rank_update_failures=0 +``` + +表格到达用户态时已按字节数降序排列。12 个报文创建了 3 个 identity 条目，所有 ownership 计数器都是 0，确认双树管理正确无误。 + +## 环境要求 + +| 要求 | 说明 | +|------|------| +| 内核 | Linux 6.16+（需要 refcounted BPF object + rbtree search kfunc） | +| 配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_NET_SCHED`、`CONFIG_NET_CLS_BPF` | +| 权限 | root 或 CAP_BPF + CAP_NET_ADMIN | +| 网络接口 | 支持 clsact egress 的任意接口 | +| 架构 | 已在 x86-64 上测试 | + +## 扩展方向 + +本示例覆盖非分片的 IPv4 TCP/UDP egress 流量，容量固定为 4096 条。对于生产环境，可以考虑： + +- **空闲过期**：移除 N 秒内没有流量的 flow，先释放两棵树的节点再 drop reference +- **入口索引**：为入站报文添加第二个 TC 程序 +- **IPv6 支持**：扩展 flow key 结构 +- **实时流式输出**：在树之外添加 ring buffer 用于每报文事件 + +## 总结 + +本教程展示了 BPF object ownership 如何实现以前不可能的复杂数据结构。identity 树负责快速的五元组查找，traffic 树提供即时的 top-flow 排名，而引用计数让两者共享同一个对象，无需状态复制。这个技术可以推广到任何需要对同一数据集维护多种排序的场景。 + +> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 或访问我们的网站 。 + +## 参考资料 + +- [BPF graph data structures](https://docs.kernel.org/bpf/graph_ds_impl.html) +- [Linux rbtree search selftest](https://github.com/torvalds/linux/blob/v6.16/tools/testing/selftests/bpf/progs/rbtree_search.c) +- [Linux refcounted graph selftests](https://github.com/torvalds/linux/tree/v6.16/tools/testing/selftests/bpf/progs) +- [libbpf TC attach implementation](https://github.com/libbpf/libbpf/blob/master/src/netlink.c) diff --git a/src/56-tc-flow-index/bpf_experimental.h b/src/56-tc-flow-index/bpf_experimental.h new file mode 100644 index 00000000..16cd8d28 --- /dev/null +++ b/src/56-tc-flow-index/bpf_experimental.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_EXPERIMENTAL_H +#define __TC_FLOW_INDEX_EXPERIMENTAL_H + +#include + +#define __contains(name, node) \ + __attribute__((btf_decl_tag("contains:" #name ":" #node))) + +extern void *bpf_obj_new_impl(__u64 local_type_id, void *meta) __ksym; +#define bpf_obj_new(type) \ + ((type *)bpf_obj_new_impl(bpf_core_type_id_local(type), NULL)) + +extern void bpf_obj_drop_impl(void *kptr, void *meta) __ksym; +#define bpf_obj_drop(kptr) bpf_obj_drop_impl(kptr, NULL) + +extern void *bpf_refcount_acquire_impl(void *kptr, void *meta) __ksym; +#define bpf_refcount_acquire(kptr) \ + bpf_refcount_acquire_impl(kptr, NULL) + +extern int bpf_rbtree_add_impl(struct bpf_rb_root *root, + struct bpf_rb_node *node, + bool (*less)(struct bpf_rb_node *, + const struct bpf_rb_node *), + void *meta, __u64 off) __ksym; +#define bpf_rbtree_add(root, node, less) \ + bpf_rbtree_add_impl(root, node, less, NULL, 0) + +extern struct bpf_rb_node * +bpf_rbtree_remove(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_first(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_root(struct bpf_rb_root *root) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_left(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; +extern struct bpf_rb_node * +bpf_rbtree_right(struct bpf_rb_root *root, struct bpf_rb_node *node) __ksym; + +#endif /* __TC_FLOW_INDEX_EXPERIMENTAL_H */ diff --git a/src/56-tc-flow-index/tc_flow_index.bpf.c b/src/56-tc-flow-index/tc_flow_index.bpf.c new file mode 100644 index 00000000..ede33f71 --- /dev/null +++ b/src/56-tc-flow-index/tc_flow_index.bpf.c @@ -0,0 +1,286 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "bpf_experimental.h" +#include "tc_flow_index.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_TCP 6 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff +#define INDEX_SEARCH_DEPTH 32 +#define TC_ACT_OK 0 + +const volatile __u32 maximum_entries = 4096; + +struct flow_entry { + struct bpf_refcount ref; + struct bpf_rb_node by_identity; + struct bpf_rb_node by_traffic; + struct flow_key key; + __u64 packets; + __u64 bytes; + __u64 last_seen_ns; + char comm[FLOW_COMM_LEN]; +}; + +#define private(name) \ + SEC(".data." #name) __hidden __attribute__((aligned(8))) + +private(index) struct bpf_spin_lock index_lock; +private(index) struct bpf_rb_root identity_root + __contains(flow_entry, by_identity); +private(index) struct bpf_rb_root traffic_root + __contains(flow_entry, by_traffic); + +struct flow_cursor snapshot_cursor; +struct flow_snapshot snapshot_result; +__u64 observed_packets; +__u64 indexed_flows; +__u64 dropped_new_flows; +__u64 allocation_failures; +__u64 refcount_failures; +__u64 rank_update_failures; + +static __always_inline int compare_key(const struct flow_key *left, + const struct flow_key *right) +{ + if (left->source_ip != right->source_ip) + return left->source_ip < right->source_ip ? -1 : 1; + if (left->destination_ip != right->destination_ip) + return left->destination_ip < right->destination_ip ? -1 : 1; + if (left->source_port != right->source_port) + return left->source_port < right->source_port ? -1 : 1; + if (left->destination_port != right->destination_port) + return left->destination_port < right->destination_port ? -1 : 1; + if (left->protocol != right->protocol) + return left->protocol < right->protocol ? -1 : 1; + return 0; +} + +static bool identity_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, + by_identity); + struct flow_entry *right = container_of(b, struct flow_entry, + by_identity); + + return compare_key(&left->key, &right->key) < 0; +} + +static __always_inline bool traffic_before(__u64 left_bytes, + __u64 left_packets, + const struct flow_key *left_key, + __u64 right_bytes, + __u64 right_packets, + const struct flow_key *right_key) +{ + if (left_bytes != right_bytes) + return left_bytes > right_bytes; + if (left_packets != right_packets) + return left_packets > right_packets; + return compare_key(left_key, right_key) < 0; +} + +static bool traffic_less(struct bpf_rb_node *a, + const struct bpf_rb_node *b) +{ + struct flow_entry *left = container_of(a, struct flow_entry, by_traffic); + struct flow_entry *right = container_of(b, struct flow_entry, by_traffic); + + return traffic_before(left->bytes, left->packets, &left->key, + right->bytes, right->packets, &right->key); +} + +static __always_inline struct flow_entry * +find_flow_locked(const struct flow_key *key) +{ + struct bpf_rb_node *rb = bpf_rbtree_root(&identity_root); + + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + struct flow_entry *entry = + container_of(rb, struct flow_entry, by_identity); + int comparison = compare_key(key, &entry->key); + + if (!comparison) + return entry; + if (comparison < 0) + rb = bpf_rbtree_left(&identity_root, rb); + else + rb = bpf_rbtree_right(&identity_root, rb); + } + return NULL; +} + +static __always_inline bool update_existing_locked(struct flow_entry *entry, + __u32 packet_bytes, + __u64 now_ns) +{ + struct flow_entry *owner; + struct bpf_rb_node *removed; + + removed = bpf_rbtree_remove(&traffic_root, &entry->by_traffic); + if (!removed) + return false; + owner = container_of(removed, struct flow_entry, by_traffic); + owner->packets++; + owner->bytes += packet_bytes; + owner->last_seen_ns = now_ns; + bpf_rbtree_add(&traffic_root, &owner->by_traffic, traffic_less); + return true; +} + +static __always_inline void update_flow(const struct flow_key *key, + __u32 packet_bytes, __u64 now_ns, + const char comm[FLOW_COMM_LEN]) +{ + struct flow_entry *entry, *new_entry, *traffic_owner; + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + return; + } + bpf_spin_unlock(&index_lock); + + new_entry = bpf_obj_new(typeof(*new_entry)); + if (!new_entry) { + __sync_fetch_and_add(&allocation_failures, 1); + return; + } + traffic_owner = bpf_refcount_acquire(new_entry); + if (!traffic_owner) { + __sync_fetch_and_add(&refcount_failures, 1); + bpf_obj_drop(new_entry); + return; + } + new_entry->key.source_ip = key->source_ip; + new_entry->key.destination_ip = key->destination_ip; + new_entry->key.source_port = key->source_port; + new_entry->key.destination_port = key->destination_port; + new_entry->key.protocol = key->protocol; + new_entry->packets = 1; + new_entry->bytes = packet_bytes; + new_entry->last_seen_ns = now_ns; + __builtin_memcpy(new_entry->comm, comm, sizeof(new_entry->comm)); + + bpf_spin_lock(&index_lock); + entry = find_flow_locked(key); + if (entry) { + if (!update_existing_locked(entry, packet_bytes, now_ns)) + rank_update_failures++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + if (indexed_flows >= maximum_entries) { + dropped_new_flows++; + bpf_spin_unlock(&index_lock); + bpf_obj_drop(new_entry); + bpf_obj_drop(traffic_owner); + return; + } + bpf_rbtree_add(&identity_root, &new_entry->by_identity, identity_less); + bpf_rbtree_add(&traffic_root, &traffic_owner->by_traffic, traffic_less); + indexed_flows++; + bpf_spin_unlock(&index_lock); +} + +static __always_inline bool parse_flow(struct __sk_buff *skb, + struct flow_key *key) +{ + struct ethhdr ethernet; + struct iphdr ip; + __u32 transport_offset; + __be16 ports[2]; + + if (bpf_skb_load_bytes(skb, 0, ðernet, sizeof(ethernet)) || + ethernet.h_proto != bpf_htons(ETH_P_IP)) + return false; + if (bpf_skb_load_bytes(skb, sizeof(ethernet), &ip, sizeof(ip)) || + ip.version != 4 || ip.ihl < 5 || + (bpf_ntohs(ip.frag_off) & (IP_MF | IP_OFFSET)) || + (ip.protocol != IPPROTO_TCP && ip.protocol != IPPROTO_UDP)) + return false; + transport_offset = sizeof(ethernet) + ip.ihl * 4; + if (bpf_skb_load_bytes(skb, transport_offset, ports, sizeof(ports))) + return false; + key->source_ip = ip.saddr; + key->destination_ip = ip.daddr; + key->source_port = ports[0]; + key->destination_port = ports[1]; + key->protocol = ip.protocol; + return true; +} + +SEC("tc") +int index_egress_flow(struct __sk_buff *skb) +{ + struct flow_key key = {}; + char comm[FLOW_COMM_LEN]; + + if (!parse_flow(skb, &key)) + return TC_ACT_OK; + __sync_fetch_and_add(&observed_packets, 1); + bpf_get_current_comm(comm, sizeof(comm)); + update_flow(&key, skb->len, bpf_ktime_get_ns(), comm); + return TC_ACT_OK; +} + +static __always_inline bool cursor_before_entry(const struct flow_cursor *cursor, + const struct flow_entry *entry) +{ + return traffic_before(cursor->bytes, cursor->packets, &cursor->key, + entry->bytes, entry->packets, &entry->key); +} + +SEC("syscall") +int snapshot_next(void *ctx) +{ + struct flow_entry *entry = NULL; + struct bpf_rb_node *candidate = NULL; + struct bpf_rb_node *rb; + + (void)ctx; + __builtin_memset(&snapshot_result, 0, sizeof(snapshot_result)); + bpf_spin_lock(&index_lock); + if (!snapshot_cursor.valid) { + candidate = bpf_rbtree_first(&traffic_root); + } else { + rb = bpf_rbtree_root(&traffic_root); + for (int i = 0; i < INDEX_SEARCH_DEPTH && rb; i++) { + entry = container_of(rb, struct flow_entry, by_traffic); + if (cursor_before_entry(&snapshot_cursor, entry)) { + candidate = rb; + rb = bpf_rbtree_left(&traffic_root, rb); + } else { + rb = bpf_rbtree_right(&traffic_root, rb); + } + } + } + if (candidate) { + entry = container_of(candidate, struct flow_entry, by_traffic); + snapshot_result.packets = entry->packets; + snapshot_result.bytes = entry->bytes; + snapshot_result.last_seen_ns = entry->last_seen_ns; + snapshot_result.key.source_ip = entry->key.source_ip; + snapshot_result.key.destination_ip = entry->key.destination_ip; + snapshot_result.key.source_port = entry->key.source_port; + snapshot_result.key.destination_port = entry->key.destination_port; + snapshot_result.key.protocol = entry->key.protocol; + snapshot_result.found = 1; + __builtin_memcpy(snapshot_result.comm, entry->comm, + sizeof(snapshot_result.comm)); + } + bpf_spin_unlock(&index_lock); + return 0; +} diff --git a/src/56-tc-flow-index/tc_flow_index.c b/src/56-tc-flow-index/tc_flow_index.c new file mode 100644 index 00000000..b64d2ccf --- /dev/null +++ b/src/56-tc-flow-index/tc_flow_index.c @@ -0,0 +1,316 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "tc_flow_index.h" +#include "tc_flow_index.skel.h" + +#define MAX_TOP 64 + +struct options { + const char *interface; + unsigned int duration_seconds; + unsigned int top; + bool demo; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || !parsed || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--duration SEC] [--top N]\n" + " %s --demo [--top N]\n", program, program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "duration", required_argument, NULL, 'd' }, + { "top", required_argument, NULL, 't' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:d:t:Dh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'd': + if (parse_uint(optarg, 86400, &options->duration_seconds)) + return -1; + break; + case 't': + if (parse_uint(optarg, MAX_TOP, &options->top)) + return -1; + break; + case 'D': options->demo = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + if (options->demo) { + if (options->interface) + return -1; + options->interface = "lo"; + } + return optind == argc && options->interface ? 0 : -1; +} + +static int send_demo_flow(unsigned int datagrams, size_t payload_size) +{ + struct sockaddr_in receiver_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + struct sockaddr_in sender_address = { + .sin_family = AF_INET, + .sin_addr.s_addr = htonl(INADDR_LOOPBACK), + }; + socklen_t address_length = sizeof(receiver_address); + char payload[1000] = {}; + int receiver = -1, sender = -1; + int err = -1; + + if (payload_size > sizeof(payload)) + return -1; + receiver = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + sender = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0); + if (receiver < 0 || sender < 0 || + bind(receiver, (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) || + getsockname(receiver, (struct sockaddr *)&receiver_address, + &address_length) || + bind(sender, (struct sockaddr *)&sender_address, + sizeof(sender_address))) + goto cleanup; + for (unsigned int i = 0; i < datagrams; i++) + if (sendto(sender, payload, payload_size, 0, + (struct sockaddr *)&receiver_address, + sizeof(receiver_address)) != (ssize_t)payload_size) + goto cleanup; + err = 0; + +cleanup: + if (receiver >= 0) close(receiver); + if (sender >= 0) close(sender); + return err; +} + +static int run_demo_traffic(void) +{ + return send_demo_flow(2, 100) || + send_demo_flow(4, 300) || + send_demo_flow(6, 700); +} + +static int snapshot_next(struct tc_flow_index_bpf *skel, + struct flow_snapshot *result) +{ + LIBBPF_OPTS(bpf_test_run_opts, opts); + + if (bpf_prog_test_run_opts(bpf_program__fd(skel->progs.snapshot_next), + &opts)) { + fprintf(stderr, "failed to query BPF flow index: %s\n", + strerror(errno)); + return -1; + } + *result = skel->bss->snapshot_result; + if (result->found) { + skel->bss->snapshot_cursor.bytes = result->bytes; + skel->bss->snapshot_cursor.packets = result->packets; + skel->bss->snapshot_cursor.key = result->key; + skel->bss->snapshot_cursor.valid = 1; + } + return 0; +} + +static int print_flows(struct tc_flow_index_bpf *skel, unsigned int top) +{ + struct flow_snapshot entry; + + memset(&skel->bss->snapshot_cursor, 0, + sizeof(skel->bss->snapshot_cursor)); + printf("\nTop egress flows, ranked in the BPF rbtree by bytes:\n"); + printf("%-21s %-21s %-5s %10s %12s %-16s\n", + "SOURCE", "DESTINATION", "PROTO", "PACKETS", "BYTES", "COMM"); + for (unsigned int i = 0; i < top; i++) { + char source_ip[INET_ADDRSTRLEN], destination_ip[INET_ADDRSTRLEN]; + char source[64], destination[64]; + + if (snapshot_next(skel, &entry)) + return -1; + if (!entry.found) + break; + inet_ntop(AF_INET, &entry.key.source_ip, source_ip, + sizeof(source_ip)); + inet_ntop(AF_INET, &entry.key.destination_ip, destination_ip, + sizeof(destination_ip)); + snprintf(source, sizeof(source), "%s:%u", source_ip, + ntohs(entry.key.source_port)); + snprintf(destination, sizeof(destination), "%s:%u", + destination_ip, ntohs(entry.key.destination_port)); + printf("%-21s %-21s %-5s %10llu %12llu %-16s\n", + source, destination, + entry.key.protocol == IPPROTO_TCP ? "TCP" : "UDP", + entry.packets, entry.bytes, entry.comm); + } + return 0; +} + +static int attach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *attach, + bool *hook_created, bool *attached) +{ + int err = bpf_tc_hook_create(hook); + + if (!err) + *hook_created = true; + else if (err != -EEXIST) { + fprintf(stderr, "failed to create clsact hook: %s\n", strerror(-err)); + return -1; + } + err = bpf_tc_attach(hook, attach); + if (err) { + fprintf(stderr, "failed to attach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = true; + return 0; +} + +static int capture_traffic(const struct options *options) +{ + unsigned long long deadline; + + if (options->demo) + return run_demo_traffic(); + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + deadline = monotonic_ns() + + (unsigned long long)options->duration_seconds * 1000000000ULL; + while (!stop && monotonic_ns() < deadline) { + struct timespec pause = { .tv_nsec = 100000000 }; + + nanosleep(&pause, NULL); + } + return 0; +} + +static int detach_tc_program(struct bpf_tc_hook *hook, + struct bpf_tc_opts *detach, bool *attached) +{ + int err = bpf_tc_detach(hook, detach); + + if (err) { + fprintf(stderr, "failed to detach TC program: %s\n", strerror(-err)); + return -1; + } + *attached = false; + return 0; +} + +int main(int argc, char **argv) +{ + struct options options = { .duration_seconds = 10, .top = 10 }; + struct tc_flow_index_bpf *skel = NULL; + LIBBPF_OPTS(bpf_tc_hook, hook, .attach_point = BPF_TC_EGRESS); + LIBBPF_OPTS(bpf_tc_opts, attach, .handle = 1, .priority = 1); + LIBBPF_OPTS(bpf_tc_opts, detach, .handle = 1, .priority = 1); + bool hook_created = false; + bool attached = false; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + hook.ifindex = if_nametoindex(options.interface); + if (!hook.ifindex) { + fprintf(stderr, "interface does not exist: %s\n", options.interface); + return 2; + } + + skel = tc_flow_index_bpf__open_and_load(); + if (!skel) { + fprintf(stderr, "failed to load TC flow index\n"); + goto cleanup; + } + attach.prog_fd = bpf_program__fd(skel->progs.index_egress_flow); + if (attach_tc_program(&hook, &attach, &hook_created, &attached)) + goto cleanup; + printf("Indexing IPv4 TCP/UDP egress flows on %s for %u seconds.\n", + options.interface, options.demo ? 0 : options.duration_seconds); + + if (capture_traffic(&options)) { + fprintf(stderr, "failed to generate traffic\n"); + goto cleanup; + } + + if (detach_tc_program(&hook, &detach, &attached)) + goto cleanup; + if (print_flows(skel, options.top)) + goto cleanup; + printf("observed_packets=%llu indexed_flows=%llu dropped_new=%llu " + "allocation_failures=%llu refcount_failures=%llu rank_update_failures=%llu\n", + (unsigned long long)skel->bss->observed_packets, + (unsigned long long)skel->bss->indexed_flows, + (unsigned long long)skel->bss->dropped_new_flows, + (unsigned long long)skel->bss->allocation_failures, + (unsigned long long)skel->bss->refcount_failures, + (unsigned long long)skel->bss->rank_update_failures); + if (options.demo && + (skel->bss->indexed_flows != 3 || skel->bss->rank_update_failures)) + goto cleanup; + err = 0; + +cleanup: + if (attached) + bpf_tc_detach(&hook, &detach); + if (hook_created) + bpf_tc_hook_destroy(&hook); + tc_flow_index_bpf__destroy(skel); + return err; +} diff --git a/src/56-tc-flow-index/tc_flow_index.h b/src/56-tc-flow-index/tc_flow_index.h new file mode 100644 index 00000000..1fc33cae --- /dev/null +++ b/src/56-tc-flow-index/tc_flow_index.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __TC_FLOW_INDEX_H +#define __TC_FLOW_INDEX_H + +#define FLOW_COMM_LEN 16 + +struct flow_key { + unsigned int source_ip; + unsigned int destination_ip; + unsigned short source_port; + unsigned short destination_port; + unsigned char protocol; + unsigned char padding[3]; +}; + +struct flow_snapshot { + unsigned long long packets; + unsigned long long bytes; + unsigned long long last_seen_ns; + struct flow_key key; + unsigned int found; + char comm[FLOW_COMM_LEN]; +}; + +struct flow_cursor { + unsigned long long bytes; + unsigned long long packets; + struct flow_key key; + unsigned int valid; +}; + +#endif /* __TC_FLOW_INDEX_H */ diff --git a/src/56-tc-flow-index/tests/test_tc_flow_index.py b/src/56-tc-flow-index/tests/test_tc_flow_index.py new file mode 100644 index 00000000..0de5a1a9 --- /dev/null +++ b/src/56-tc-flow-index/tests/test_tc_flow_index.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 +import re +import subprocess +import sys + + +def run_demo(): + result = subprocess.run( + [sys.argv[1], "--demo", "--top", "3"], + text=True, + capture_output=True, + check=False, + timeout=30, + ) + output = result.stdout + result.stderr + if result.returncode: + raise AssertionError(output) + assert "ranked in the BPF rbtree by bytes:" in output, output + assert "indexed_flows=3" in output, output + assert "allocation_failures=0" in output, output + assert "refcount_failures=0" in output, output + assert "rank_update_failures=0" in output, output + byte_counts = [int(value) for value in re.findall(r"UDP\s+\d+\s+(\d+)", output)] + assert len(byte_counts) == 3, output + assert byte_counts == sorted(byte_counts, reverse=True), output + + +def main(): + run_demo() + run_demo() + invalid = subprocess.run( + [sys.argv[1], "--interface", "no-such-interface"], + text=True, + capture_output=True, + check=False, + timeout=5, + ) + assert invalid.returncode == 2, invalid.stdout + invalid.stderr + assert "interface does not exist" in invalid.stderr, invalid.stderr + print("TC flow index integration test: PASS") + + +if __name__ == "__main__": + main() diff --git a/src/57-oom-watch/.config b/src/57-oom-watch/.config new file mode 100644 index 00000000..61fe7f78 --- /dev/null +++ b/src/57-oom-watch/.config @@ -0,0 +1,10 @@ +level=Depth +type=Tracing +kernel_min=7.1 +kernel_min_basis=required-feature +architectures=x86_64 +btf=required +kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_BPF_EVENTS=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_MEMCG=y +hardware=none +root=required +test_status=ci-build diff --git a/src/57-oom-watch/.gitignore b/src/57-oom-watch/.gitignore new file mode 100644 index 00000000..a2940add --- /dev/null +++ b/src/57-oom-watch/.gitignore @@ -0,0 +1,2 @@ +.output/ +oom_watch diff --git a/src/57-oom-watch/Makefile b/src/57-oom-watch/Makefile new file mode 100644 index 00000000..5167f6dc --- /dev/null +++ b/src/57-oom-watch/Makefile @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +OUTPUT := .output +CLANG ?= clang +LIBBPF_SRC := $(abspath ../third_party/libbpf/src) +BPFTOOL_SRC := $(abspath ../third_party/bpftool/src) +LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a) +BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool) +BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool +ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \ + | sed 's/arm.*/arm/' \ + | sed 's/aarch64/arm64/' \ + | sed 's/ppc64le/powerpc/' \ + | sed 's/mips.*/mips/' \ + | sed 's/riscv64/riscv/' \ + | sed 's/loongarch64/loongarch/') +VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h +INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I. +CFLAGS := -g -Wall -Wextra -D_GNU_SOURCE -pthread +ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS) +APPS = oom_watch + +CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - &1 \ + | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') + +ifeq ($(V),1) + Q = + msg = +else + Q = @ + msg = @printf ' %-8s %s%s\n' "$(1)" \ + "$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \ + "$(if $(3), $(3))"; + MAKEFLAGS += --no-print-directory +endif + +define allow-override + $(if $(or $(findstring environment,$(origin $(1))),\ + $(findstring command line,$(origin $(1)))),,\ + $(eval $(1) = $(2))) +endef + +$(call allow-override,CC,$(CROSS_COMPILE)cc) + +.PHONY: all clean test +all: $(APPS) + +clean: + $(call msg,CLEAN) + $(Q)rm -rf $(OUTPUT) $(APPS) + +test: all + @if [ "$$(id -u)" -ne 0 ]; then \ + echo "test requires root; run inside the documented KVM guest" >&2; \ + exit 1; \ + fi + python3 tests/test_oom_watch.py ./oom_watch + +$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT): + $(call msg,MKDIR,$@) + $(Q)mkdir -p $@ + +$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf + $(call msg,LIB,$@) + $(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \ + OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \ + INCLUDEDIR= LIBDIR= UAPIDIR= install + +$(BPFTOOL): | $(BPFTOOL_OUTPUT) + $(call msg,BPFTOOL,$@) + $(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap + +$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL) + $(call msg,BPF,$@) + $(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \ + $(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \ + -c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + $(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + +$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL) + $(call msg,GEN-SKEL,$@) + $(Q)$(BPFTOOL) gen skeleton $< > $@ + +$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h + +$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT) + $(call msg,CC,$@) + $(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@ + +$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT) + $(call msg,BINARY,$@) + $(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@ + +.DELETE_ON_ERROR: +.SECONDARY: diff --git a/src/57-oom-watch/README.md b/src/57-oom-watch/README.md new file mode 100644 index 00000000..17904a6c --- /dev/null +++ b/src/57-oom-watch/README.md @@ -0,0 +1,525 @@ +# eBPF Tutorial: Profiling Memory Reclaim Before an OOM Kill + +Have you ever had a container or service suddenly killed by the Linux OOM killer, leaving you with nothing but a cryptic log message? The kernel tells you *which* process it chose as a victim, but says almost nothing about the memory pressure that led there. How many times did the system try to reclaim memory? How long did each attempt take? Which kernel code paths consumed all that time? + +This tutorial builds `oom-watch`, an eBPF tool that captures what happens *before* the kill. It profiles recorded memcg reclaim intervals as a latency histogram and a set of sampled kernel stacks, then attaches this accumulated profile to the OOM victim and tracks the process until it exits. + +> Complete source code: + +## The Missing Piece in OOM Debugging + +When a memory cgroup approaches its limit, any allocation might trigger reclaim. The kernel scans the cgroup for pages it can free. Sometimes a few short attempts succeed. Other times, reclaim keeps running with little progress until the OOM killer finally steps in. + +Looking only at the `oom/mark_victim` tracepoint loses all this context. You see the victim, but not the struggle that came before. Operators need answers to questions like: + +- How many reclaim cycles happened before the kill? +- Were they quick microsecond scans, or long millisecond stalls? +- Did reclaim come from allocations inside the cgroup, or from external proactive reclaim via `memory.reclaim`? +- Which kernel functions dominated the reclaim time? + +`oom-watch` answers all of these. It hooks into the kernel's vmscan tracepoints to measure matched reclaim intervals, samples kernel stacks to show where time was spent, and when OOM selects a victim, it snapshots the accumulated profile right alongside the victim information. Its drop counters expose intervals that could not be retained in the bounded state maps. + +## Why eBPF for Memory Profiling? + +Traditional monitoring approaches have serious limitations for this use case. Polling `/proc/meminfo` or cgroup stats misses short-lived reclaim events. `perf` can capture stack traces but requires careful setup and post-processing. Neither approach easily connects reclaim activity to a specific OOM event. + +eBPF changes the game. Programs run directly in the kernel, triggered by events with nanosecond precision. Maps carry state between events, letting us build histograms and correlate begin/end pairs. The ring buffer delivers events to userspace with minimal overhead. The verifier rejects unsafe programs before loading, substantially reducing the risk of kernel memory corruption compared with an unchecked kernel module. + +For `oom-watch`, we use several tracepoints: + +- `mm_vmscan_memcg_reclaim_begin`: Fires when reclaim starts for a specific memory cgroup +- `mm_vmscan_memcg_reclaim_end`: Fires when that reclaim interval completes +- `oom/mark_victim`: Fires when the OOM killer selects a victim +- `sched/sched_process_exit`: Fires when the victim process exits + +Linux 7.1 added a crucial feature: the vmscan tracepoints now include the target `mem_cgroup` being scanned. This matters because a process in one cgroup can trigger reclaim in a different cgroup through `memory.reclaim`. With the target cgroup in the tracepoint, we can correctly attribute the work. + +## Architecture Overview + +The tool has three main components working together. + +The **header file** defines shared data structures: a per-cgroup reclaim profile containing latency buckets and counters, a per-stack aggregate tracking samples and timing, and an event structure for OOM notifications. + +The **BPF program** runs in the kernel. On reclaim begin, it records the start time and optionally captures a kernel stack. On reclaim end, it computes duration, updates the histogram, and stores the stack aggregate. When OOM marks a victim, it looks up the victim's cgroup, copies the accumulated profile into an event, and sends it to userspace. It also saves victim state so it can report when the process exits. + +The **userspace program** loads kernel symbols for stack symbolization, manages the optional cgroup filter, processes events from the ring buffer, and includes a self-contained demo mode that triggers an OOM to verify everything works. + +## The Profile Data Structures + +The shared header defines what we're measuring: + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __OOM_WATCH_H +#define __OOM_WATCH_H + +#define OOM_RECLAIM_BUCKETS 20 +#define OOM_STACK_DEPTH 127 + +enum oom_watch_event_type { + OOM_VICTIM_MARKED = 1, + OOM_VICTIM_EXITED = 2, +}; + +struct reclaim_profile { + unsigned long long begin_count; + unsigned long long end_count; + unsigned long long reclaimed_pages; + unsigned long long cross_cgroup_reclaims; + unsigned long long last_reclaim_ns; + unsigned long long total_reclaim_ns; + unsigned long long maximum_reclaim_ns; + unsigned long long stack_samples; + unsigned long long stack_failures; + unsigned long long latency_slots[OOM_RECLAIM_BUCKETS]; +}; + +struct reclaim_stack_key { + unsigned long long cgroup_id; + signed int stack_id; + unsigned int padding; +}; + +struct reclaim_stack_profile { + unsigned long long samples; + unsigned long long total_ns; + unsigned long long maximum_ns; + unsigned long long reclaimed_pages; +}; + +struct oom_watch_event { + unsigned long long timestamp_ns; + unsigned long long cgroup_id; + struct reclaim_profile profile; + unsigned long long total_vm_kb; + unsigned long long anon_rss_kb; + unsigned long long file_rss_kb; + unsigned int type; + unsigned int victim_pid; + unsigned int victim_tid; + unsigned int triggering_tgid; + signed int exit_code; + char comm[16]; +}; + +#endif /* __OOM_WATCH_H */ +``` + +The `latency_slots` array holds a power-of-two histogram with 20 buckets. Bucket 0 covers 0-1 microseconds, bucket 1 covers 2-3 microseconds, then 4-7 microseconds, and so on. The final bucket catches everything over half a second. This logarithmic distribution captures both fast reclaim cycles and rare slow ones efficiently. + +When an OOM event fires, it embeds a complete copy of the cgroup's profile. The numbers printed with the victim show exactly how things stood at the moment of selection. + +## The BPF Program + +Here's the complete BPF code. We'll walk through how it works after the listing: + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "oom_watch.h" + +char LICENSE[] SEC("license") = "GPL"; + +const volatile __u64 target_cgroup_id; +const volatile __u32 sample_every = 1; + +extern struct task_struct *bpf_task_from_pid(__s32 pid) __ksym; +extern void bpf_task_release(struct task_struct *task) __ksym; + +struct active_reclaim { + __u64 started_ns; + __u64 cgroup_id; + __s32 stack_id; + __u32 padding; +}; + +struct victim_state { + __u64 cgroup_id; + __u32 triggering_tgid; + __u32 victim_tgid; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct reclaim_profile); +} profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct active_reclaim); +} active_reclaims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(max_entries, 1024); + __uint(key_size, sizeof(__u32)); + __uint(value_size, OOM_STACK_DEPTH * sizeof(__u64)); +} stack_traces SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 8192); + __type(key, struct reclaim_stack_key); + __type(value, struct reclaim_stack_profile); +} stack_profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct victim_state); +} victims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +__u64 dropped_victim_states; +__u64 dropped_reclaim_states; + +static __always_inline bool selected_cgroup(__u64 cgroup_id) +{ + return !target_cgroup_id || cgroup_id == target_cgroup_id; +} + +static __always_inline __u64 victim_task_info(__u32 victim_pid, + __u32 *victim_tgid) +{ + struct task_struct *task; + __u64 cgroup_id = 0; + + task = bpf_task_from_pid(victim_pid); + if (!task) + return 0; + cgroup_id = BPF_CORE_READ(task, cgroups, dfl_cgrp, kn, id); + *victim_tgid = BPF_CORE_READ(task, tgid); + bpf_task_release(task); + return cgroup_id; +} + +static __always_inline __u64 memcg_cgroup_id(struct mem_cgroup *memcg) +{ + struct cgroup *cgroup; + + cgroup = BPF_CORE_READ(memcg, css.cgroup); + if (!cgroup) + return 0; + return BPF_CORE_READ(cgroup, kn, id); +} + +static __always_inline struct reclaim_profile *get_profile(__u64 cgroup_id) +{ + struct reclaim_profile initial = {}; + struct reclaim_profile *profile; + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + return profile; + bpf_map_update_elem(&profiles, &cgroup_id, &initial, BPF_NOEXIST); + return bpf_map_lookup_elem(&profiles, &cgroup_id); +} + +static __always_inline __u32 latency_bucket(__u64 duration_ns) +{ + __u64 microseconds = duration_ns / 1000; + __u32 bucket = 0; + + for (int i = 0; i < OOM_RECLAIM_BUCKETS - 1; i++) { + if (microseconds < 2) + break; + microseconds >>= 1; + bucket++; + } + return bucket; +} + +static __always_inline void update_maximum(__u64 *maximum, __u64 value) +{ + __u64 previous = *maximum; + + for (int i = 0; i < 8 && previous < value; i++) { + __u64 observed = __sync_val_compare_and_swap(maximum, previous, + value); + + if (observed == previous) + break; + previous = observed; + } +} + +static __always_inline void update_stack_profile(__u64 cgroup_id, + __s32 stack_id, + __u64 duration_ns, + __u64 reclaimed) +{ + struct reclaim_stack_key key = { + .cgroup_id = cgroup_id, + .stack_id = stack_id, + }; + struct reclaim_stack_profile initial = {}; + struct reclaim_stack_profile *profile; + + profile = bpf_map_lookup_elem(&stack_profiles, &key); + if (!profile) { + bpf_map_update_elem(&stack_profiles, &key, &initial, BPF_NOEXIST); + profile = bpf_map_lookup_elem(&stack_profiles, &key); + } + if (!profile) + return; + __sync_fetch_and_add(&profile->samples, 1); + __sync_fetch_and_add(&profile->total_ns, duration_ns); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + update_maximum(&profile->maximum_ns, duration_ns); +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_begin") +int BPF_PROG(profile_reclaim_begin, gfp_t gfp_flags, int order, + struct mem_cgroup *memcg) +{ + struct active_reclaim active = { .stack_id = -1 }; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 sequence; + + (void)gfp_flags; + (void)order; + if (!selected_cgroup(cgroup_id)) + return 0; + profile = get_profile(cgroup_id); + if (!profile) + return 0; + sequence = __sync_fetch_and_add(&profile->begin_count, 1); + if (bpf_get_current_cgroup_id() != cgroup_id) + __sync_fetch_and_add(&profile->cross_cgroup_reclaims, 1); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + active.started_ns = profile->last_reclaim_ns; + active.cgroup_id = cgroup_id; + if (!sample_every || sequence % sample_every == 0) { + active.stack_id = bpf_get_stackid(ctx, &stack_traces, + BPF_F_FAST_STACK_CMP | 2); + if (active.stack_id >= 0) + __sync_fetch_and_add(&profile->stack_samples, 1); + else + __sync_fetch_and_add(&profile->stack_failures, 1); + } + if (bpf_map_update_elem(&active_reclaims, &pid_tgid, &active, BPF_ANY)) + __sync_fetch_and_add(&dropped_reclaim_states, 1); + return 0; +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_end") +int BPF_PROG(profile_reclaim_end, unsigned long reclaimed, + struct mem_cgroup *memcg) +{ + struct active_reclaim *active; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 duration_ns; + __s32 stack_id; + + if (!selected_cgroup(cgroup_id)) + return 0; + active = bpf_map_lookup_elem(&active_reclaims, &pid_tgid); + if (!active || active->cgroup_id != cgroup_id) + return 0; + duration_ns = bpf_ktime_get_ns() - active->started_ns; + stack_id = active->stack_id; + profile = get_profile(cgroup_id); + if (profile) { + __sync_fetch_and_add(&profile->end_count, 1); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + __sync_fetch_and_add(&profile->total_reclaim_ns, duration_ns); + __sync_fetch_and_add(&profile->latency_slots[latency_bucket(duration_ns)], + 1); + update_maximum(&profile->maximum_reclaim_ns, duration_ns); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + } + if (stack_id >= 0) + update_stack_profile(cgroup_id, stack_id, duration_ns, reclaimed); + bpf_map_delete_elem(&active_reclaims, &pid_tgid); + return 0; +} + +SEC("tracepoint/oom/mark_victim") +int capture_oom_victim(struct trace_event_raw_mark_victim *ctx) +{ + struct reclaim_profile *profile; + struct victim_state victim; + struct oom_watch_event *event; + __u64 cgroup_id; + __u32 victim_pid = ctx->pid; + __u32 victim_tgid = 0; + + cgroup_id = victim_task_info(victim_pid, &victim_tgid); + if (!victim_tgid || !selected_cgroup(cgroup_id)) + return 0; + victim.cgroup_id = cgroup_id; + victim.triggering_tgid = bpf_get_current_pid_tgid() >> 32; + victim.victim_tgid = victim_tgid; + if (bpf_map_update_elem(&victims, &victim_pid, &victim, BPF_ANY)) { + __sync_fetch_and_add(&dropped_victim_states, 1); + return 0; + } + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return 0; + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = cgroup_id; + event->type = OOM_VICTIM_MARKED; + event->victim_pid = victim_tgid; + event->victim_tid = victim_pid; + event->triggering_tgid = victim.triggering_tgid; + event->total_vm_kb = ctx->total_vm; + event->anon_rss_kb = ctx->anon_rss; + event->file_rss_kb = ctx->file_rss; + bpf_probe_read_kernel_str(event->comm, sizeof(event->comm), + (void *)ctx + (ctx->__data_loc_comm & 0xffff)); + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + __builtin_memcpy(&event->profile, profile, sizeof(event->profile)); + bpf_ringbuf_submit(event, 0); + return 0; +} + +SEC("tracepoint/sched/sched_process_exit") +int capture_victim_exit(void *ctx) +{ + struct victim_state *victim; + struct task_struct *task; + struct oom_watch_event *event; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u32 tid = pid_tgid; + + (void)ctx; + victim = bpf_map_lookup_elem(&victims, &tid); + if (!victim) + return 0; + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (event) { + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = victim->cgroup_id; + event->type = OOM_VICTIM_EXITED; + event->victim_pid = victim->victim_tgid; + event->victim_tid = tid; + event->triggering_tgid = victim->triggering_tgid; + task = (struct task_struct *)bpf_get_current_task_btf(); + event->exit_code = BPF_CORE_READ(task, exit_code); + bpf_get_current_comm(event->comm, sizeof(event->comm)); + bpf_ringbuf_submit(event, 0); + } + bpf_map_delete_elem(&victims, &tid); + return 0; +} +``` + +The program uses several BPF maps to maintain state. The `active_reclaims` map joins begin and end events by storing the start timestamp keyed by `pid_tgid`. The `profiles` map accumulates per-cgroup statistics. The `stack_traces` map stores deduplicated kernel stacks, while `stack_profiles` aggregates timing data for each unique stack. + +At reclaim begin, we extract the target cgroup ID from the `mem_cgroup` parameter, not from the current task's cgroup. This distinction matters for cross-cgroup reclaim. If someone calls `memory.reclaim` on a cgroup from outside, we correctly attribute the work to the target. The `cross_cgroup_reclaims` counter tracks how often this happens. + +Stack sampling uses the `--sample-every` setting. With the default value of 1, we attempt to capture a stack for each recorded begin event. Higher values reduce overhead, while every successfully matched interval still contributes to the histogram. The `BPF_F_FAST_STACK_CMP` flag speeds up stack deduplication, and the `2` in the flags skips two tracing frames to get cleaner stacks. + +The OOM handler has to resolve process identity carefully. The tracepoint gives us a thread ID, but we also need the thread group ID (PID from userspace perspective) and the cgroup. We use `bpf_task_from_pid()` to look up the task, read what we need, then release the reference. Victim state is keyed by TID because `sched_process_exit` fires in that thread's context. + +## Userspace: Symbols and Presentation + +The userspace code handles several jobs: loading kernel symbols for stack symbolization, setting up the BPF program with any cgroup filter, processing events from the ring buffer, and ranking stacks by total time spent. It also includes a demo mode that creates a memory-limited cgroup and triggers an OOM to verify the tool works. + +The full userspace code is quite long, so we'll highlight the key parts. On startup, it reads `/proc/kallsyms`, sorts symbols by address, and later uses binary search to resolve each stack frame. When symbol addresses are restricted (common on production systems), it still prints raw addresses - the measurements stay accurate. + +Stack groups are ranked by cumulative reclaim time. This ranking surfaces both frequently-called paths and rare slow intervals. After an OOM event, the top five stacks are printed with their timing statistics. + +The demo mode creates a cgroup with `memory.max=32 MiB`, no swap, and grouped OOM behavior. A worker process faults 24 MiB then pauses, letting the parent request 8 MiB through `memory.reclaim` from outside the cgroup. This exercises cross-cgroup attribution. The worker then continues faulting a 128 MiB mapping until OOM kills it. The process leader exits before the second stage, testing that we handle TGID and victim TID independently. + +## Compilation and Execution + +Build the tool: + +```bash +cd src/57-oom-watch +make +``` + +Profile a specific cgroup for 60 seconds, sampling one kernel stack for every ten reclaim intervals: + +```bash +sudo ./oom_watch \ + --cgroup /sys/fs/cgroup/my-service \ + --duration 60 \ + --sample-every 10 +``` + +Omit `--cgroup` to watch all cgroups. Omit `--duration` to run until interrupted. The built-in demo captures every reclaim stack: + +```bash +sudo ./oom_watch --demo +``` + +Here's output from a real demo run. PIDs, cgroup IDs, addresses, and timings vary between runs: + +```text +oom-watch tracing cgroup=/sys/fs/cgroup/ebpf-oom-watch-1262 cgroup_id=151 +event=oom-victim pid=1263 tid=1264 comm=oom_watch trigger_pid=1263 cgroup_id=151 anon_rss_kb=32064 file_rss_kb=0 total_vm_kb=141844 reclaim_cycles=44 cross_cgroup_reclaims=22 reclaimed_pages=40 +reclaim_profile cgroup_id=151 cycles=44 completed=44 total_ms=0.274 max_ms=0.056 reclaimed_pages=40 cross_cgroup=22 stack_samples=44 stack_failures=0 +reclaim_latency_us=0-1 count=9 +reclaim_latency_us=2-3 count=13 +reclaim_latency_us=4-7 count=14 +reclaim_latency_us=8-15 count=5 +reclaim_latency_us=16-31 count=2 +reclaim_latency_us=32-63 count=1 +reclaim_stack rank=1 samples=22 total_ms=0.181 max_ms=0.056 reclaimed_pages=40 + #0 try_to_free_mem_cgroup_pages+0x... + #1 try_charge_memcg+0x... +reclaim_stack rank=2 samples=22 total_ms=0.093 max_ms=0.012 reclaimed_pages=0 + #0 try_to_free_mem_cgroup_pages+0x... + #1 user_proactive_reclaim+0x... +event=victim-exit pid=1263 tid=1264 cgroup_id=151 exit_code=9 +demo workload signaled=1 signal=9 +demo result=matched-profile-to-victim +dropped_victim_states=0 dropped_reclaim_states=0 +``` + +Look at what this tells us: 44 reclaim cycles completed before the kill, with half triggered from outside the target cgroup. The histogram shows most cycles were fast (under 8 microseconds), but a few took longer. The two ranked stacks separate allocation-triggered reclaim from explicit `memory.reclaim` requests. The exit event confirms the victim received `SIGKILL`. + +## Requirements + +| Requirement | Details | +|---|---| +| Kernel | Linux 7.1 or newer (needs target-memcg vmscan tracepoints) | +| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_BPF_EVENTS`, `CONFIG_DEBUG_INFO_BTF`, `CONFIG_MEMCG`; `CONFIG_KALLSYMS` improves symbol output | +| cgroup | cgroup v2 with memory controller; demo mode needs cgroup admin access | +| Privileges | Root or equivalent BPF and tracing capabilities | +| Architecture | x86-64 tested; no special hardware needed | + +## Implementation Scope + +Profiles accumulate from program attach until exit, using bounded LRU maps: 4096 cgroup profiles, 4096 active intervals, 8192 stack aggregates, and 1024 unique stacks. One active interval is retained per `pid_tgid`, matching the begin/end tracing pattern. The tool captures kernel stacks rather than user stacks, and treats symbolization as presentation - restricted `kallsyms` changes names to addresses without affecting measurements. + +## Summary + +`oom-watch` turns the chaos before an OOM kill into evidence you can examine. It measures recorded memcg reclaim intervals, samples and ranks the kernel paths that consumed time, correctly attributes work to the target cgroup (even for cross-cgroup reclaim), and connects this profile to the victim selection and exit. + +The next time a container dies and someone asks "what happened?", you'll have more than a one-line kernel log to show them. + +> If you'd like to dive deeper into eBPF, check out our tutorial repository at or visit our website at . + +## References + +- [Linux vmscan tracepoints](https://github.com/torvalds/linux/blob/v7.1/include/trace/events/vmscan.h) +- [Target-memcg vmscan attribution commit](https://github.com/torvalds/linux/commit/874a0a566ede40f3d6062cae8fe1022e616edd1a) +- [Linux OOM tracepoints](https://github.com/torvalds/linux/blob/v7.1/include/trace/events/oom.h) +- [BPF kfunc documentation](https://docs.kernel.org/bpf/kfuncs.html) +- [Control Group v2 memory interface](https://docs.kernel.org/admin-guide/cgroup-v2.html) diff --git a/src/57-oom-watch/README.zh.md b/src/57-oom-watch/README.zh.md new file mode 100644 index 00000000..e35a26b5 --- /dev/null +++ b/src/57-oom-watch/README.zh.md @@ -0,0 +1,525 @@ +# eBPF 实战教程：在 OOM Kill 之前分析内存回收 + +你是否遇到过容器或服务突然被 Linux OOM killer 杀掉，只留下一行不明所以的内核日志？内核会告诉你它选择了*哪个*进程作为 victim，但对于导致这一结果的内存压力几乎只字未提。系统尝试了多少次内存回收？每次花了多久？哪些内核路径消耗了这些时间？ + +本教程构建 `oom-watch`，一个 eBPF 工具，用于捕获 OOM kill *之前*发生的事情。它将记录到的 memcg 回收间隔汇总为延迟直方图和采样的内核调用栈，然后将积累的 profile 附加到 OOM victim 上，并跟踪进程直到它退出。 + +> 完整源代码： + +## OOM 调试中缺失的一环 + +当 memory cgroup 接近其限制时，任何内存分配都可能触发回收。内核会扫描该 cgroup 中可释放的页面。有时几次短暂的尝试就能成功。有时，回收反复运行却进展甚微，直到 OOM killer 最终介入。 + +如果只观察 `oom/mark_victim` tracepoint，你会丢失所有这些上下文。你看到了 victim，却看不到之前的挣扎过程。运维人员需要这些问题的答案： + +- OOM kill 之前发生了多少次回收周期？ +- 它们是微秒级的快速扫描，还是毫秒级的长时间停顿？ +- 回收是由 cgroup 内部的分配触发的，还是通过 `memory.reclaim` 从外部主动触发的？ +- 哪些内核函数占用了回收时间？ + +`oom-watch` 可以回答所有这些问题。它通过 hook 内核的 vmscan tracepoint 来测量成功匹配的回收间隔，采样内核调用栈以展示时间花在了哪里，当 OOM 选择 victim 时，它会将累积的 profile 快照与 victim 信息一起输出。对于有界状态 map 未能保留的间隔，工具会通过 drop 计数器明确报告。 + +## 为什么用 eBPF 做内存分析？ + +传统监控方法在这个场景下有严重的局限性。轮询 `/proc/meminfo` 或 cgroup 统计会错过短暂的回收事件。`perf` 可以捕获调用栈，但需要仔细配置和后处理。两种方法都不容易将回收活动与特定的 OOM 事件关联起来。 + +eBPF 改变了这一切。程序直接在内核中运行，以纳秒级精度响应事件。Map 在事件之间传递状态，让我们可以构建直方图并关联 begin/end 对。Ring buffer 以最小的开销将事件传递给用户空间。验证器会在加载前拒绝不安全的程序，与未经检查的内核模块相比，这显著降低了破坏内核内存的风险。 + +对于 `oom-watch`，我们使用几个 tracepoint： + +- `mm_vmscan_memcg_reclaim_begin`：当特定 memory cgroup 的回收开始时触发 +- `mm_vmscan_memcg_reclaim_end`：当该回收间隔完成时触发 +- `oom/mark_victim`：当 OOM killer 选择 victim 时触发 +- `sched/sched_process_exit`：当 victim 进程退出时触发 + +Linux 7.1 添加了一个关键特性：vmscan tracepoint 现在包含被扫描的目标 `mem_cgroup`。这很重要，因为一个 cgroup 中的进程可以通过 `memory.reclaim` 触发另一个 cgroup 的回收。有了 tracepoint 中的目标 cgroup，我们可以正确归因这些工作。 + +## 架构概述 + +这个工具有三个主要组件协同工作。 + +**头文件**定义共享数据结构：包含延迟桶和计数器的 per-cgroup 回收 profile，跟踪采样和时间的 per-stack aggregate，以及 OOM 通知的事件结构。 + +**BPF 程序**在内核中运行。在回收开始时，它记录开始时间并可选地捕获内核调用栈。在回收结束时，它计算持续时间，更新直方图，并存储 stack aggregate。当 OOM 标记 victim 时，它查找 victim 的 cgroup，将累积的 profile 复制到事件中，并发送给用户空间。它还保存 victim 状态，以便在进程退出时进行报告。 + +**用户空间程序**加载内核符号用于调用栈符号化，管理可选的 cgroup 过滤器，处理来自 ring buffer 的事件，并包含一个自包含的 demo 模式，可以触发 OOM 来验证一切正常工作。 + +## Profile 数据结构 + +共享头文件定义了我们测量的内容： + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __OOM_WATCH_H +#define __OOM_WATCH_H + +#define OOM_RECLAIM_BUCKETS 20 +#define OOM_STACK_DEPTH 127 + +enum oom_watch_event_type { + OOM_VICTIM_MARKED = 1, + OOM_VICTIM_EXITED = 2, +}; + +struct reclaim_profile { + unsigned long long begin_count; + unsigned long long end_count; + unsigned long long reclaimed_pages; + unsigned long long cross_cgroup_reclaims; + unsigned long long last_reclaim_ns; + unsigned long long total_reclaim_ns; + unsigned long long maximum_reclaim_ns; + unsigned long long stack_samples; + unsigned long long stack_failures; + unsigned long long latency_slots[OOM_RECLAIM_BUCKETS]; +}; + +struct reclaim_stack_key { + unsigned long long cgroup_id; + signed int stack_id; + unsigned int padding; +}; + +struct reclaim_stack_profile { + unsigned long long samples; + unsigned long long total_ns; + unsigned long long maximum_ns; + unsigned long long reclaimed_pages; +}; + +struct oom_watch_event { + unsigned long long timestamp_ns; + unsigned long long cgroup_id; + struct reclaim_profile profile; + unsigned long long total_vm_kb; + unsigned long long anon_rss_kb; + unsigned long long file_rss_kb; + unsigned int type; + unsigned int victim_pid; + unsigned int victim_tid; + unsigned int triggering_tgid; + signed int exit_code; + char comm[16]; +}; + +#endif /* __OOM_WATCH_H */ +``` + +`latency_slots` 数组是一个 20 个桶的 2 次幂直方图。桶 0 覆盖 0-1 微秒，桶 1 覆盖 2-3 微秒，然后是 4-7 微秒，依此类推。最后一个桶捕获超过半秒的所有内容。这种对数分布可以高效地捕获快速回收周期和罕见的慢速周期。 + +当 OOM 事件触发时，它会嵌入 cgroup profile 的完整副本。与 victim 一起打印的数字准确显示了选择时的状态。 + +## BPF 程序 + +下面是完整的 BPF 代码。我们将在代码之后解释它的工作原理： + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "oom_watch.h" + +char LICENSE[] SEC("license") = "GPL"; + +const volatile __u64 target_cgroup_id; +const volatile __u32 sample_every = 1; + +extern struct task_struct *bpf_task_from_pid(__s32 pid) __ksym; +extern void bpf_task_release(struct task_struct *task) __ksym; + +struct active_reclaim { + __u64 started_ns; + __u64 cgroup_id; + __s32 stack_id; + __u32 padding; +}; + +struct victim_state { + __u64 cgroup_id; + __u32 triggering_tgid; + __u32 victim_tgid; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct reclaim_profile); +} profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct active_reclaim); +} active_reclaims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(max_entries, 1024); + __uint(key_size, sizeof(__u32)); + __uint(value_size, OOM_STACK_DEPTH * sizeof(__u64)); +} stack_traces SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 8192); + __type(key, struct reclaim_stack_key); + __type(value, struct reclaim_stack_profile); +} stack_profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct victim_state); +} victims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +__u64 dropped_victim_states; +__u64 dropped_reclaim_states; + +static __always_inline bool selected_cgroup(__u64 cgroup_id) +{ + return !target_cgroup_id || cgroup_id == target_cgroup_id; +} + +static __always_inline __u64 victim_task_info(__u32 victim_pid, + __u32 *victim_tgid) +{ + struct task_struct *task; + __u64 cgroup_id = 0; + + task = bpf_task_from_pid(victim_pid); + if (!task) + return 0; + cgroup_id = BPF_CORE_READ(task, cgroups, dfl_cgrp, kn, id); + *victim_tgid = BPF_CORE_READ(task, tgid); + bpf_task_release(task); + return cgroup_id; +} + +static __always_inline __u64 memcg_cgroup_id(struct mem_cgroup *memcg) +{ + struct cgroup *cgroup; + + cgroup = BPF_CORE_READ(memcg, css.cgroup); + if (!cgroup) + return 0; + return BPF_CORE_READ(cgroup, kn, id); +} + +static __always_inline struct reclaim_profile *get_profile(__u64 cgroup_id) +{ + struct reclaim_profile initial = {}; + struct reclaim_profile *profile; + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + return profile; + bpf_map_update_elem(&profiles, &cgroup_id, &initial, BPF_NOEXIST); + return bpf_map_lookup_elem(&profiles, &cgroup_id); +} + +static __always_inline __u32 latency_bucket(__u64 duration_ns) +{ + __u64 microseconds = duration_ns / 1000; + __u32 bucket = 0; + + for (int i = 0; i < OOM_RECLAIM_BUCKETS - 1; i++) { + if (microseconds < 2) + break; + microseconds >>= 1; + bucket++; + } + return bucket; +} + +static __always_inline void update_maximum(__u64 *maximum, __u64 value) +{ + __u64 previous = *maximum; + + for (int i = 0; i < 8 && previous < value; i++) { + __u64 observed = __sync_val_compare_and_swap(maximum, previous, + value); + + if (observed == previous) + break; + previous = observed; + } +} + +static __always_inline void update_stack_profile(__u64 cgroup_id, + __s32 stack_id, + __u64 duration_ns, + __u64 reclaimed) +{ + struct reclaim_stack_key key = { + .cgroup_id = cgroup_id, + .stack_id = stack_id, + }; + struct reclaim_stack_profile initial = {}; + struct reclaim_stack_profile *profile; + + profile = bpf_map_lookup_elem(&stack_profiles, &key); + if (!profile) { + bpf_map_update_elem(&stack_profiles, &key, &initial, BPF_NOEXIST); + profile = bpf_map_lookup_elem(&stack_profiles, &key); + } + if (!profile) + return; + __sync_fetch_and_add(&profile->samples, 1); + __sync_fetch_and_add(&profile->total_ns, duration_ns); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + update_maximum(&profile->maximum_ns, duration_ns); +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_begin") +int BPF_PROG(profile_reclaim_begin, gfp_t gfp_flags, int order, + struct mem_cgroup *memcg) +{ + struct active_reclaim active = { .stack_id = -1 }; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 sequence; + + (void)gfp_flags; + (void)order; + if (!selected_cgroup(cgroup_id)) + return 0; + profile = get_profile(cgroup_id); + if (!profile) + return 0; + sequence = __sync_fetch_and_add(&profile->begin_count, 1); + if (bpf_get_current_cgroup_id() != cgroup_id) + __sync_fetch_and_add(&profile->cross_cgroup_reclaims, 1); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + active.started_ns = profile->last_reclaim_ns; + active.cgroup_id = cgroup_id; + if (!sample_every || sequence % sample_every == 0) { + active.stack_id = bpf_get_stackid(ctx, &stack_traces, + BPF_F_FAST_STACK_CMP | 2); + if (active.stack_id >= 0) + __sync_fetch_and_add(&profile->stack_samples, 1); + else + __sync_fetch_and_add(&profile->stack_failures, 1); + } + if (bpf_map_update_elem(&active_reclaims, &pid_tgid, &active, BPF_ANY)) + __sync_fetch_and_add(&dropped_reclaim_states, 1); + return 0; +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_end") +int BPF_PROG(profile_reclaim_end, unsigned long reclaimed, + struct mem_cgroup *memcg) +{ + struct active_reclaim *active; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 duration_ns; + __s32 stack_id; + + if (!selected_cgroup(cgroup_id)) + return 0; + active = bpf_map_lookup_elem(&active_reclaims, &pid_tgid); + if (!active || active->cgroup_id != cgroup_id) + return 0; + duration_ns = bpf_ktime_get_ns() - active->started_ns; + stack_id = active->stack_id; + profile = get_profile(cgroup_id); + if (profile) { + __sync_fetch_and_add(&profile->end_count, 1); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + __sync_fetch_and_add(&profile->total_reclaim_ns, duration_ns); + __sync_fetch_and_add(&profile->latency_slots[latency_bucket(duration_ns)], + 1); + update_maximum(&profile->maximum_reclaim_ns, duration_ns); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + } + if (stack_id >= 0) + update_stack_profile(cgroup_id, stack_id, duration_ns, reclaimed); + bpf_map_delete_elem(&active_reclaims, &pid_tgid); + return 0; +} + +SEC("tracepoint/oom/mark_victim") +int capture_oom_victim(struct trace_event_raw_mark_victim *ctx) +{ + struct reclaim_profile *profile; + struct victim_state victim; + struct oom_watch_event *event; + __u64 cgroup_id; + __u32 victim_pid = ctx->pid; + __u32 victim_tgid = 0; + + cgroup_id = victim_task_info(victim_pid, &victim_tgid); + if (!victim_tgid || !selected_cgroup(cgroup_id)) + return 0; + victim.cgroup_id = cgroup_id; + victim.triggering_tgid = bpf_get_current_pid_tgid() >> 32; + victim.victim_tgid = victim_tgid; + if (bpf_map_update_elem(&victims, &victim_pid, &victim, BPF_ANY)) { + __sync_fetch_and_add(&dropped_victim_states, 1); + return 0; + } + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return 0; + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = cgroup_id; + event->type = OOM_VICTIM_MARKED; + event->victim_pid = victim_tgid; + event->victim_tid = victim_pid; + event->triggering_tgid = victim.triggering_tgid; + event->total_vm_kb = ctx->total_vm; + event->anon_rss_kb = ctx->anon_rss; + event->file_rss_kb = ctx->file_rss; + bpf_probe_read_kernel_str(event->comm, sizeof(event->comm), + (void *)ctx + (ctx->__data_loc_comm & 0xffff)); + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + __builtin_memcpy(&event->profile, profile, sizeof(event->profile)); + bpf_ringbuf_submit(event, 0); + return 0; +} + +SEC("tracepoint/sched/sched_process_exit") +int capture_victim_exit(void *ctx) +{ + struct victim_state *victim; + struct task_struct *task; + struct oom_watch_event *event; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u32 tid = pid_tgid; + + (void)ctx; + victim = bpf_map_lookup_elem(&victims, &tid); + if (!victim) + return 0; + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (event) { + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = victim->cgroup_id; + event->type = OOM_VICTIM_EXITED; + event->victim_pid = victim->victim_tgid; + event->victim_tid = tid; + event->triggering_tgid = victim->triggering_tgid; + task = (struct task_struct *)bpf_get_current_task_btf(); + event->exit_code = BPF_CORE_READ(task, exit_code); + bpf_get_current_comm(event->comm, sizeof(event->comm)); + bpf_ringbuf_submit(event, 0); + } + bpf_map_delete_elem(&victims, &tid); + return 0; +} +``` + +程序使用多个 BPF map 来维护状态。`active_reclaims` map 通过以 `pid_tgid` 为 key 存储开始时间戳来关联 begin 和 end 事件。`profiles` map 累积每个 cgroup 的统计信息。`stack_traces` map 存储去重后的内核调用栈，而 `stack_profiles` 为每个唯一调用栈聚合时间数据。 + +在回收开始时，我们从 `mem_cgroup` 参数而不是当前 task 的 cgroup 中提取目标 cgroup ID。这个区别对于跨 cgroup 回收很重要。如果有人从外部对某个 cgroup 调用 `memory.reclaim`，我们可以正确地将工作归因到目标。`cross_cgroup_reclaims` 计数器跟踪这种情况发生的频率。 + +调用栈采样使用 `--sample-every` 设置。默认值为 1 时，我们会为每个记录到的 begin 事件尝试捕获调用栈。更大的值可以减少开销，而每个成功匹配的间隔仍会计入直方图。`BPF_F_FAST_STACK_CMP` 标志加速调用栈去重，标志中的 `2` 跳过两个 tracing 帧以获得更清晰的调用栈。 + +OOM 处理程序必须仔细解析进程身份。tracepoint 给我们一个线程 ID，但我们还需要线程组 ID（用户空间视角的 PID）和 cgroup。我们使用 `bpf_task_from_pid()` 查找 task，读取所需信息，然后释放引用。victim 状态以 TID 为 key，因为 `sched_process_exit` 在该线程的上下文中触发。 + +## 用户空间：符号化和展示 + +用户空间代码负责几项工作：加载内核符号用于调用栈符号化，使用任何 cgroup 过滤器设置 BPF 程序，处理来自 ring buffer 的事件，以及按总耗时对调用栈进行排名。它还包含一个 demo 模式，可以创建一个内存受限的 cgroup 并触发 OOM 来验证工具是否正常工作。 + +完整的用户空间代码相当长，所以我们重点介绍关键部分。启动时，它读取 `/proc/kallsyms`，按地址排序符号，然后使用二分查找解析每个栈帧。当符号地址受限时（在生产系统上很常见），它仍然打印原始地址，测量结果保持准确。 + +调用栈组按累计回收时间排名。这种排名可以展示频繁调用的路径和罕见的慢速间隔。在 OOM 事件之后，打印前五个调用栈及其时间统计。 + +demo 模式创建一个 `memory.max=32 MiB`、无 swap、启用分组 OOM 行为的 cgroup。一个 worker 进程首先 fault 24 MiB 然后暂停，让父进程从 cgroup 外部通过 `memory.reclaim` 请求 8 MiB。这会测试跨 cgroup 归因。然后 worker 继续 fault 一个 128 MiB 的映射直到 OOM 杀死它。进程 leader 在第二阶段之前退出，测试我们是否正确地独立处理 TGID 和 victim TID。 + +## 编译和运行 + +构建工具： + +```bash +cd src/57-oom-watch +make +``` + +分析一个特定的 cgroup 60 秒，每十个回收间隔采样一次内核调用栈： + +```bash +sudo ./oom_watch \ + --cgroup /sys/fs/cgroup/my-service \ + --duration 60 \ + --sample-every 10 +``` + +省略 `--cgroup` 可以观察所有 cgroup。省略 `--duration` 可以持续运行直到中断。内置 demo 会捕获每次回收调用栈： + +```bash +sudo ./oom_watch --demo +``` + +下面是一次真实 demo 运行的输出。PID、cgroup ID、地址和时间在不同运行之间会有变化： + +```text +oom-watch tracing cgroup=/sys/fs/cgroup/ebpf-oom-watch-1262 cgroup_id=151 +event=oom-victim pid=1263 tid=1264 comm=oom_watch trigger_pid=1263 cgroup_id=151 anon_rss_kb=32064 file_rss_kb=0 total_vm_kb=141844 reclaim_cycles=44 cross_cgroup_reclaims=22 reclaimed_pages=40 +reclaim_profile cgroup_id=151 cycles=44 completed=44 total_ms=0.274 max_ms=0.056 reclaimed_pages=40 cross_cgroup=22 stack_samples=44 stack_failures=0 +reclaim_latency_us=0-1 count=9 +reclaim_latency_us=2-3 count=13 +reclaim_latency_us=4-7 count=14 +reclaim_latency_us=8-15 count=5 +reclaim_latency_us=16-31 count=2 +reclaim_latency_us=32-63 count=1 +reclaim_stack rank=1 samples=22 total_ms=0.181 max_ms=0.056 reclaimed_pages=40 + #0 try_to_free_mem_cgroup_pages+0x... + #1 try_charge_memcg+0x... +reclaim_stack rank=2 samples=22 total_ms=0.093 max_ms=0.012 reclaimed_pages=0 + #0 try_to_free_mem_cgroup_pages+0x... + #1 user_proactive_reclaim+0x... +event=victim-exit pid=1263 tid=1264 cgroup_id=151 exit_code=9 +demo workload signaled=1 signal=9 +demo result=matched-profile-to-victim +dropped_victim_states=0 dropped_reclaim_states=0 +``` + +看看这告诉我们什么：在 kill 之前完成了 44 个回收周期，其中一半是从目标 cgroup 外部触发的。直方图显示大多数周期很快（不到 8 微秒），但有几个花了更长时间。两个排名的调用栈将分配触发的回收与显式的 `memory.reclaim` 请求区分开来。exit 事件确认 victim 收到了 `SIGKILL`。 + +## 环境要求 + +| 要求 | 说明 | +|---|---| +| 内核 | Linux 7.1 或更高版本（需要 target-memcg vmscan tracepoint） | +| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_BPF_EVENTS`、`CONFIG_DEBUG_INFO_BTF`、`CONFIG_MEMCG`；`CONFIG_KALLSYMS` 可以改善符号输出 | +| cgroup | 带 memory controller 的 cgroup v2；demo 模式需要 cgroup 管理权限 | +| 权限 | root 或等效的 BPF 和 tracing capability | +| 架构 | 已在 x86-64 上测试；不需要特殊硬件 | + +## 实现范围 + +Profile 从程序附加开始累积直到退出，使用有界的 LRU map：4096 个 cgroup profile、4096 个活动间隔、8192 个调用栈聚合和 1024 个唯一调用栈。每个 `pid_tgid` 保留一个活动间隔，与 begin/end 跟踪模式相匹配。工具捕获内核调用栈而不是用户调用栈，并将符号化视为展示环节：受限的 `kallsyms` 会将名称变为地址，但不影响测量结果。 + +## 总结 + +`oom-watch` 将 OOM kill 之前的混乱变成可以检查的证据。它测量记录到的 memcg 回收间隔，采样并排名消耗时间的内核路径，正确归因工作到目标 cgroup（即使对于跨 cgroup 回收），并将此 profile 与 victim 选择和退出关联起来。 + +下次容器死掉有人问"发生了什么？"时，你将不仅仅有一行内核日志可以展示。 + +> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 或访问我们的网站 。 + +## 参考资料 + +- [Linux vmscan tracepoint](https://github.com/torvalds/linux/blob/v7.1/include/trace/events/vmscan.h) +- [target-memcg vmscan attribution commit](https://github.com/torvalds/linux/commit/874a0a566ede40f3d6062cae8fe1022e616edd1a) +- [Linux OOM tracepoint](https://github.com/torvalds/linux/blob/v7.1/include/trace/events/oom.h) +- [BPF kfunc 文档](https://docs.kernel.org/bpf/kfuncs.html) +- [Control Group v2 memory interface](https://docs.kernel.org/admin-guide/cgroup-v2.html) diff --git a/src/57-oom-watch/oom_watch.bpf.c b/src/57-oom-watch/oom_watch.bpf.c new file mode 100644 index 00000000..d1c5d34b --- /dev/null +++ b/src/57-oom-watch/oom_watch.bpf.c @@ -0,0 +1,310 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include +#include "oom_watch.h" + +char LICENSE[] SEC("license") = "GPL"; + +const volatile __u64 target_cgroup_id; +const volatile __u32 sample_every = 1; + +extern struct task_struct *bpf_task_from_pid(__s32 pid) __ksym; +extern void bpf_task_release(struct task_struct *task) __ksym; + +struct active_reclaim { + __u64 started_ns; + __u64 cgroup_id; + __s32 stack_id; + __u32 padding; +}; + +struct victim_state { + __u64 cgroup_id; + __u32 triggering_tgid; + __u32 victim_tgid; +}; + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct reclaim_profile); +} profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 4096); + __type(key, __u64); + __type(value, struct active_reclaim); +} active_reclaims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_STACK_TRACE); + __uint(max_entries, 1024); + __uint(key_size, sizeof(__u32)); + __uint(value_size, OOM_STACK_DEPTH * sizeof(__u64)); +} stack_traces SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_LRU_HASH); + __uint(max_entries, 8192); + __type(key, struct reclaim_stack_key); + __type(value, struct reclaim_stack_profile); +} stack_profiles SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_HASH); + __uint(max_entries, 1024); + __type(key, __u32); + __type(value, struct victim_state); +} victims SEC(".maps"); + +struct { + __uint(type, BPF_MAP_TYPE_RINGBUF); + __uint(max_entries, 256 * 1024); +} events SEC(".maps"); + +__u64 dropped_victim_states; +__u64 dropped_reclaim_states; + +static __always_inline bool selected_cgroup(__u64 cgroup_id) +{ + return !target_cgroup_id || cgroup_id == target_cgroup_id; +} + +static __always_inline __u64 victim_task_info(__u32 victim_pid, + __u32 *victim_tgid) +{ + struct task_struct *task; + __u64 cgroup_id = 0; + + task = bpf_task_from_pid(victim_pid); + if (!task) + return 0; + cgroup_id = BPF_CORE_READ(task, cgroups, dfl_cgrp, kn, id); + *victim_tgid = BPF_CORE_READ(task, tgid); + bpf_task_release(task); + return cgroup_id; +} + +static __always_inline __u64 memcg_cgroup_id(struct mem_cgroup *memcg) +{ + struct cgroup *cgroup; + + cgroup = BPF_CORE_READ(memcg, css.cgroup); + if (!cgroup) + return 0; + return BPF_CORE_READ(cgroup, kn, id); +} + +static __always_inline struct reclaim_profile *get_profile(__u64 cgroup_id) +{ + struct reclaim_profile initial = {}; + struct reclaim_profile *profile; + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + return profile; + bpf_map_update_elem(&profiles, &cgroup_id, &initial, BPF_NOEXIST); + return bpf_map_lookup_elem(&profiles, &cgroup_id); +} + +static __always_inline __u32 latency_bucket(__u64 duration_ns) +{ + __u64 microseconds = duration_ns / 1000; + __u32 bucket = 0; + + for (int i = 0; i < OOM_RECLAIM_BUCKETS - 1; i++) { + if (microseconds < 2) + break; + microseconds >>= 1; + bucket++; + } + return bucket; +} + +static __always_inline void update_maximum(__u64 *maximum, __u64 value) +{ + __u64 previous = *maximum; + + for (int i = 0; i < 8 && previous < value; i++) { + __u64 observed = __sync_val_compare_and_swap(maximum, previous, + value); + + if (observed == previous) + break; + previous = observed; + } +} + +static __always_inline void update_stack_profile(__u64 cgroup_id, + __s32 stack_id, + __u64 duration_ns, + __u64 reclaimed) +{ + struct reclaim_stack_key key = { + .cgroup_id = cgroup_id, + .stack_id = stack_id, + }; + struct reclaim_stack_profile initial = {}; + struct reclaim_stack_profile *profile; + + profile = bpf_map_lookup_elem(&stack_profiles, &key); + if (!profile) { + bpf_map_update_elem(&stack_profiles, &key, &initial, BPF_NOEXIST); + profile = bpf_map_lookup_elem(&stack_profiles, &key); + } + if (!profile) + return; + __sync_fetch_and_add(&profile->samples, 1); + __sync_fetch_and_add(&profile->total_ns, duration_ns); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + update_maximum(&profile->maximum_ns, duration_ns); +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_begin") +int BPF_PROG(profile_reclaim_begin, gfp_t gfp_flags, int order, + struct mem_cgroup *memcg) +{ + struct active_reclaim active = { .stack_id = -1 }; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 sequence; + + (void)gfp_flags; + (void)order; + if (!selected_cgroup(cgroup_id)) + return 0; + profile = get_profile(cgroup_id); + if (!profile) + return 0; + sequence = __sync_fetch_and_add(&profile->begin_count, 1); + if (bpf_get_current_cgroup_id() != cgroup_id) + __sync_fetch_and_add(&profile->cross_cgroup_reclaims, 1); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + active.started_ns = profile->last_reclaim_ns; + active.cgroup_id = cgroup_id; + if (!sample_every || sequence % sample_every == 0) { + active.stack_id = bpf_get_stackid(ctx, &stack_traces, + BPF_F_FAST_STACK_CMP | 2); + if (active.stack_id >= 0) + __sync_fetch_and_add(&profile->stack_samples, 1); + else + __sync_fetch_and_add(&profile->stack_failures, 1); + } + if (bpf_map_update_elem(&active_reclaims, &pid_tgid, &active, BPF_ANY)) + __sync_fetch_and_add(&dropped_reclaim_states, 1); + return 0; +} + +SEC("tp_btf/mm_vmscan_memcg_reclaim_end") +int BPF_PROG(profile_reclaim_end, unsigned long reclaimed, + struct mem_cgroup *memcg) +{ + struct active_reclaim *active; + struct reclaim_profile *profile; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u64 cgroup_id = memcg_cgroup_id(memcg); + __u64 duration_ns; + __s32 stack_id; + + if (!selected_cgroup(cgroup_id)) + return 0; + active = bpf_map_lookup_elem(&active_reclaims, &pid_tgid); + if (!active || active->cgroup_id != cgroup_id) + return 0; + duration_ns = bpf_ktime_get_ns() - active->started_ns; + stack_id = active->stack_id; + profile = get_profile(cgroup_id); + if (profile) { + __sync_fetch_and_add(&profile->end_count, 1); + __sync_fetch_and_add(&profile->reclaimed_pages, reclaimed); + __sync_fetch_and_add(&profile->total_reclaim_ns, duration_ns); + __sync_fetch_and_add(&profile->latency_slots[latency_bucket(duration_ns)], + 1); + update_maximum(&profile->maximum_reclaim_ns, duration_ns); + profile->last_reclaim_ns = bpf_ktime_get_ns(); + } + if (stack_id >= 0) + update_stack_profile(cgroup_id, stack_id, duration_ns, reclaimed); + bpf_map_delete_elem(&active_reclaims, &pid_tgid); + return 0; +} + +SEC("tracepoint/oom/mark_victim") +int capture_oom_victim(struct trace_event_raw_mark_victim *ctx) +{ + struct reclaim_profile *profile; + struct victim_state victim; + struct oom_watch_event *event; + __u64 cgroup_id; + __u32 victim_pid = ctx->pid; + __u32 victim_tgid = 0; + + cgroup_id = victim_task_info(victim_pid, &victim_tgid); + if (!victim_tgid || !selected_cgroup(cgroup_id)) + return 0; + victim.cgroup_id = cgroup_id; + victim.triggering_tgid = bpf_get_current_pid_tgid() >> 32; + victim.victim_tgid = victim_tgid; + if (bpf_map_update_elem(&victims, &victim_pid, &victim, BPF_ANY)) { + __sync_fetch_and_add(&dropped_victim_states, 1); + return 0; + } + + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (!event) + return 0; + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = cgroup_id; + event->type = OOM_VICTIM_MARKED; + event->victim_pid = victim_tgid; + event->victim_tid = victim_pid; + event->triggering_tgid = victim.triggering_tgid; + event->total_vm_kb = ctx->total_vm; + event->anon_rss_kb = ctx->anon_rss; + event->file_rss_kb = ctx->file_rss; + bpf_probe_read_kernel_str(event->comm, sizeof(event->comm), + (void *)ctx + (ctx->__data_loc_comm & 0xffff)); + + profile = bpf_map_lookup_elem(&profiles, &cgroup_id); + if (profile) + __builtin_memcpy(&event->profile, profile, sizeof(event->profile)); + bpf_ringbuf_submit(event, 0); + return 0; +} + +SEC("tracepoint/sched/sched_process_exit") +int capture_victim_exit(void *ctx) +{ + struct victim_state *victim; + struct task_struct *task; + struct oom_watch_event *event; + __u64 pid_tgid = bpf_get_current_pid_tgid(); + __u32 tid = pid_tgid; + + (void)ctx; + victim = bpf_map_lookup_elem(&victims, &tid); + if (!victim) + return 0; + event = bpf_ringbuf_reserve(&events, sizeof(*event), 0); + if (event) { + __builtin_memset(event, 0, sizeof(*event)); + event->timestamp_ns = bpf_ktime_get_ns(); + event->cgroup_id = victim->cgroup_id; + event->type = OOM_VICTIM_EXITED; + event->victim_pid = victim->victim_tgid; + event->victim_tid = tid; + event->triggering_tgid = victim->triggering_tgid; + task = (struct task_struct *)bpf_get_current_task_btf(); + event->exit_code = BPF_CORE_READ(task, exit_code); + bpf_get_current_comm(event->comm, sizeof(event->comm)); + bpf_ringbuf_submit(event, 0); + } + bpf_map_delete_elem(&victims, &tid); + return 0; +} diff --git a/src/57-oom-watch/oom_watch.c b/src/57-oom-watch/oom_watch.c new file mode 100644 index 00000000..a33d7765 --- /dev/null +++ b/src/57-oom-watch/oom_watch.c @@ -0,0 +1,739 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "oom_watch.h" +#include "oom_watch.skel.h" + +struct options { + const char *cgroup_path; + unsigned int duration_seconds; + unsigned int sample_every; + bool demo; +}; + +static volatile sig_atomic_t stop; +static int victim_events; +static int exit_events; +static unsigned long long observed_cgroup_id; +static unsigned int observed_victim_pid; +static unsigned int observed_victim_tid; +static unsigned long long observed_reclaims; +static unsigned long long observed_cross_cgroup_reclaims; +static unsigned long long observed_stack_samples; + +struct kernel_symbol { + unsigned long long address; + char *name; +}; + +struct kernel_symbols { + struct kernel_symbol *items; + size_t count; + size_t capacity; +}; + +struct runtime_context { + int profiles_fd; + int stack_profiles_fd; + int stack_traces_fd; + struct kernel_symbols symbols; +}; + +struct oom_runtime { + struct oom_watch_bpf *skel; + struct ring_buffer *ring; + struct runtime_context context; +}; + +struct selected_cgroup { + char demo_path[256]; + const char *path; + struct stat metadata; + bool demo_created; + bool memory_enabled_by_demo; +}; + +struct demo_process { + pid_t child; + int ready_pipe[2]; + int continue_pipe[2]; + int status; +}; + +struct ranked_stack { + struct reclaim_stack_key key; + struct reclaim_stack_profile profile; +}; + +struct allocation_context { + int ready_fd; + int continue_fd; +}; + +static struct allocation_context allocation_context; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static unsigned long long monotonic_ns(void) +{ + struct timespec now; + + clock_gettime(CLOCK_MONOTONIC, &now); + return (unsigned long long)now.tv_sec * 1000000000ULL + now.tv_nsec; +} + +static int write_text(const char *path, const char *text) +{ + int fd = open(path, O_WRONLY | O_CLOEXEC); + ssize_t length = strlen(text); + int err = 0; + + if (fd < 0) + return -1; + if (write(fd, text, length) != length) + err = -1; + close(fd); + return err; +} + +static int memory_controller_enabled(bool *enabled) +{ + char controllers[4096]; + ssize_t length; + int fd; + + fd = open("/sys/fs/cgroup/cgroup.subtree_control", + O_RDONLY | O_CLOEXEC); + if (fd < 0) + return -1; + length = read(fd, controllers, sizeof(controllers) - 1); + close(fd); + if (length < 0) + return -1; + controllers[length] = '\0'; + *enabled = strstr(controllers, "memory") != NULL; + return 0; +} + +static int compare_symbols(const void *left, const void *right) +{ + const struct kernel_symbol *a = left; + const struct kernel_symbol *b = right; + + return a->address < b->address ? -1 : a->address > b->address ? 1 : 0; +} + +static int load_kernel_symbols(struct kernel_symbols *symbols) +{ + char name[256]; + char type; + unsigned long long address; + FILE *file = fopen("/proc/kallsyms", "r"); + + if (!file) + return -1; + while (fscanf(file, "%llx %c %255s%*[^\n]\n", &address, &type, + name) == 3) { + struct kernel_symbol *item; + + (void)type; + if (symbols->count == symbols->capacity) { + size_t capacity = symbols->capacity ? symbols->capacity * 2 : 4096; + void *items = realloc(symbols->items, + capacity * sizeof(*symbols->items)); + + if (!items) + goto error; + symbols->items = items; + symbols->capacity = capacity; + } + item = &symbols->items[symbols->count++]; + item->address = address; + item->name = strdup(name); + if (!item->name) + goto error; + } + fclose(file); + qsort(symbols->items, symbols->count, sizeof(*symbols->items), + compare_symbols); + return symbols->count ? 0 : -1; + +error: + fclose(file); + return -1; +} + +static void free_kernel_symbols(struct kernel_symbols *symbols) +{ + for (size_t i = 0; i < symbols->count; i++) + free(symbols->items[i].name); + free(symbols->items); +} + +static const struct kernel_symbol *find_kernel_symbol( + const struct kernel_symbols *symbols, unsigned long long address) +{ + size_t low = 0, high = symbols->count; + + while (low < high) { + size_t middle = low + (high - low) / 2; + + if (symbols->items[middle].address <= address) + low = middle + 1; + else + high = middle; + } + return low ? &symbols->items[low - 1] : NULL; +} + +static void insert_ranked_stack(struct ranked_stack top[5], size_t *count, + const struct reclaim_stack_key *key, + const struct reclaim_stack_profile *profile) +{ + size_t position = 0; + + while (position < *count && + top[position].profile.total_ns >= profile->total_ns) + position++; + if (position >= 5) + return; + if (*count < 5) + (*count)++; + for (size_t i = *count - 1; i > position; i--) + top[i] = top[i - 1]; + top[position].key = *key; + top[position].profile = *profile; +} + +static void print_reclaim_stacks(struct runtime_context *runtime, + __u64 cgroup_id) +{ + struct ranked_stack top[5] = {}; + struct reclaim_stack_key previous, next; + bool have_previous = false; + size_t count = 0; + + while (!bpf_map_get_next_key(runtime->stack_profiles_fd, + have_previous ? &previous : NULL, &next)) { + struct reclaim_stack_profile profile; + + if (next.cgroup_id == cgroup_id && + !bpf_map_lookup_elem(runtime->stack_profiles_fd, &next, + &profile)) + insert_ranked_stack(top, &count, &next, &profile); + previous = next; + have_previous = true; + } + + for (size_t rank = 0; rank < count; rank++) { + unsigned long long addresses[OOM_STACK_DEPTH] = {}; + + printf("reclaim_stack rank=%zu samples=%llu total_ms=%.3f " + "max_ms=%.3f reclaimed_pages=%llu\n", + rank + 1, top[rank].profile.samples, + top[rank].profile.total_ns / 1000000.0, + top[rank].profile.maximum_ns / 1000000.0, + top[rank].profile.reclaimed_pages); + if (bpf_map_lookup_elem(runtime->stack_traces_fd, + &top[rank].key.stack_id, addresses)) + continue; + for (size_t frame = 0; frame < OOM_STACK_DEPTH && addresses[frame]; + frame++) { + const struct kernel_symbol *symbol = + find_kernel_symbol(&runtime->symbols, addresses[frame]); + + if (symbol && symbol->address) + printf(" #%zu %s+0x%llx\n", frame, symbol->name, + addresses[frame] - symbol->address); + else + printf(" #%zu 0x%llx\n", frame, addresses[frame]); + } + } +} + +static void print_reclaim_profile(struct runtime_context *runtime, + __u64 cgroup_id, + const struct reclaim_profile *profile) +{ + printf("reclaim_profile cgroup_id=%llu cycles=%llu completed=%llu " + "total_ms=%.3f max_ms=%.3f reclaimed_pages=%llu " + "cross_cgroup=%llu stack_samples=%llu stack_failures=%llu\n", + (unsigned long long)cgroup_id, profile->begin_count, + profile->end_count, profile->total_reclaim_ns / 1000000.0, + profile->maximum_reclaim_ns / 1000000.0, + profile->reclaimed_pages, profile->cross_cgroup_reclaims, + profile->stack_samples, profile->stack_failures); + for (unsigned int bucket = 0; bucket < OOM_RECLAIM_BUCKETS; bucket++) { + unsigned long long low, high; + + if (!profile->latency_slots[bucket]) + continue; + low = bucket ? 1ULL << bucket : 0; + high = (1ULL << (bucket + 1)) - 1; + if (bucket == OOM_RECLAIM_BUCKETS - 1) + printf("reclaim_latency_us=>=%llu count=%llu\n", low, + profile->latency_slots[bucket]); + else + printf("reclaim_latency_us=%llu-%llu count=%llu\n", low, + high, profile->latency_slots[bucket]); + } + print_reclaim_stacks(runtime, cgroup_id); +} + +static void print_live_profiles(struct runtime_context *runtime) +{ + __u64 previous, next; + bool have_previous = false; + + while (!bpf_map_get_next_key(runtime->profiles_fd, + have_previous ? &previous : NULL, &next)) { + struct reclaim_profile profile; + + if (!bpf_map_lookup_elem(runtime->profiles_fd, &next, &profile)) + print_reclaim_profile(runtime, next, &profile); + previous = next; + have_previous = true; + } +} + +static int handle_event(void *ctx, void *data, size_t size) +{ + const struct oom_watch_event *event = data; + struct runtime_context *runtime = ctx; + + if (size != sizeof(*event)) + return 0; + if (event->type == OOM_VICTIM_MARKED) { + victim_events++; + observed_cgroup_id = event->cgroup_id; + observed_victim_pid = event->victim_pid; + observed_reclaims = event->profile.begin_count; + observed_victim_tid = event->victim_tid; + observed_cross_cgroup_reclaims = + event->profile.cross_cgroup_reclaims; + observed_stack_samples = event->profile.stack_samples; + printf("event=oom-victim pid=%u tid=%u comm=%s trigger_pid=%u cgroup_id=%llu " + "anon_rss_kb=%llu file_rss_kb=%llu total_vm_kb=%llu " + "reclaim_cycles=%llu cross_cgroup_reclaims=%llu " + "reclaimed_pages=%llu\n", + event->victim_pid, event->victim_tid, event->comm, + event->triggering_tgid, + (unsigned long long)event->cgroup_id, + (unsigned long long)event->anon_rss_kb, + (unsigned long long)event->file_rss_kb, + (unsigned long long)event->total_vm_kb, + (unsigned long long)event->profile.begin_count, + (unsigned long long)event->profile.cross_cgroup_reclaims, + (unsigned long long)event->profile.reclaimed_pages); + print_reclaim_profile(runtime, event->cgroup_id, &event->profile); + } else if (event->type == OOM_VICTIM_EXITED) { + exit_events++; + printf("event=victim-exit pid=%u tid=%u cgroup_id=%llu exit_code=%d\n", + event->victim_pid, event->victim_tid, + (unsigned long long)event->cgroup_id, event->exit_code); + } + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || !parsed || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s [--cgroup PATH] [--duration SEC] [--sample-every N]\n" + " %s --demo [--sample-every N]\n", program, program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "cgroup", required_argument, NULL, 'c' }, + { "duration", required_argument, NULL, 'd' }, + { "sample-every", required_argument, NULL, 's' }, + { "demo", no_argument, NULL, 'D' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "c:d:s:Dh", long_options, + NULL)) != -1) { + switch (option) { + case 'c': options->cgroup_path = optarg; break; + case 'd': + if (parse_uint(optarg, 86400, &options->duration_seconds)) + return -1; + break; + case 's': + if (parse_uint(optarg, 1000000, &options->sample_every)) + return -1; + break; + case 'D': options->demo = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + return optind == argc && !(options->demo && options->cgroup_path) ? 0 : -1; +} + +static void *allocation_worker(void *argument) +{ + struct allocation_context *context = argument; + size_t first_stage = 24 * 1024 * 1024; + size_t length = 128 * 1024 * 1024; + unsigned char *memory; + char byte = 'x'; + + memory = mmap(NULL, length, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); + if (memory == MAP_FAILED) + _exit(4); + for (size_t offset = 0; offset < first_stage; offset += 4096) + memory[offset] = 0xa5; + if (write(context->ready_fd, &byte, 1) != 1 || + read(context->continue_fd, &byte, 1) != 1) + _exit(5); + for (size_t offset = first_stage; offset < length; offset += 4096) + memory[offset] = 0xa5; + _exit(6); +} + +static void allocate_until_killed(const char *cgroup_path, int ready_fd, + int continue_fd) +{ + char procs_path[512]; + char pid_text[32]; + pthread_t worker; + + snprintf(procs_path, sizeof(procs_path), "%s/cgroup.procs", cgroup_path); + snprintf(pid_text, sizeof(pid_text), "%d", getpid()); + if (write_text(procs_path, pid_text)) + _exit(3); + allocation_context.ready_fd = ready_fd; + allocation_context.continue_fd = continue_fd; + if (pthread_create(&worker, NULL, allocation_worker, + &allocation_context)) + _exit(4); + pthread_detach(worker); + pthread_exit(NULL); +} + +static int configure_demo_cgroup(const char *path, bool *created, + bool *enabled_by_demo) +{ + bool memory_enabled; + char file[512]; + + if (memory_controller_enabled(&memory_enabled)) + return -1; + if (!memory_enabled) { + if (write_text("/sys/fs/cgroup/cgroup.subtree_control", "+memory")) + return -1; + *enabled_by_demo = true; + } + if (mkdir(path, 0755)) + return -1; + *created = true; + snprintf(file, sizeof(file), "%s/memory.max", path); + if (write_text(file, "33554432")) + return -1; + snprintf(file, sizeof(file), "%s/memory.swap.max", path); + if (write_text(file, "0")) + return -1; + snprintf(file, sizeof(file), "%s/memory.oom.group", path); + return write_text(file, "1"); +} + +static int trigger_cross_cgroup_reclaim(const char *cgroup_path) +{ + char reclaim_path[512]; + + snprintf(reclaim_path, sizeof(reclaim_path), "%s/memory.reclaim", + cgroup_path); + if (!write_text(reclaim_path, "8388608") || errno == EAGAIN) + return 0; + return -1; +} + +static int select_cgroup(const struct options *options, + struct selected_cgroup *selected) +{ + if (options->demo) { + snprintf(selected->demo_path, sizeof(selected->demo_path), + "/sys/fs/cgroup/ebpf-oom-watch-%d", getpid()); + if (configure_demo_cgroup(selected->demo_path, + &selected->demo_created, + &selected->memory_enabled_by_demo)) { + fprintf(stderr, "failed to configure demo memory cgroup: %s\n", + strerror(errno)); + return -1; + } + selected->path = selected->demo_path; + } else { + selected->path = options->cgroup_path; + } + if (!selected->path) + return 0; + if (!stat(selected->path, &selected->metadata)) + return 0; + fprintf(stderr, "failed to stat cgroup %s: %s\n", selected->path, + strerror(errno)); + return -1; +} + +static void cleanup_selected_cgroup(struct selected_cgroup *selected, + int *result) +{ + if (selected->demo_created && rmdir(selected->demo_path) && !*result) + *result = 1; + if (selected->memory_enabled_by_demo && + write_text("/sys/fs/cgroup/cgroup.subtree_control", "-memory") && + !*result) + *result = 1; +} + +static int prepare_runtime(struct oom_runtime *runtime, + const struct options *options, + const struct selected_cgroup *selected) +{ + runtime->skel = oom_watch_bpf__open(); + if (!runtime->skel) + return -1; + runtime->skel->rodata->target_cgroup_id = + selected->path ? selected->metadata.st_ino : 0; + runtime->skel->rodata->sample_every = options->sample_every; + if (oom_watch_bpf__load(runtime->skel) || + oom_watch_bpf__attach(runtime->skel)) { + fprintf(stderr, "failed to load and attach OOM watcher\n"); + return -1; + } + runtime->context.profiles_fd = + bpf_map__fd(runtime->skel->maps.profiles); + runtime->context.stack_profiles_fd = + bpf_map__fd(runtime->skel->maps.stack_profiles); + runtime->context.stack_traces_fd = + bpf_map__fd(runtime->skel->maps.stack_traces); + if (load_kernel_symbols(&runtime->context.symbols)) + fprintf(stderr, "warning: kernel symbols unavailable; printing raw stack addresses\n"); + runtime->ring = ring_buffer__new( + bpf_map__fd(runtime->skel->maps.events), handle_event, + &runtime->context, NULL); + return runtime->ring ? 0 : -1; +} + +static void destroy_runtime(struct oom_runtime *runtime) +{ + ring_buffer__free(runtime->ring); + free_kernel_symbols(&runtime->context.symbols); + oom_watch_bpf__destroy(runtime->skel); +} + +static void init_demo_process(struct demo_process *demo) +{ + memset(demo, 0, sizeof(*demo)); + demo->child = -1; + demo->ready_pipe[0] = -1; + demo->ready_pipe[1] = -1; + demo->continue_pipe[0] = -1; + demo->continue_pipe[1] = -1; +} + +static void close_demo_pipe(int *fd) +{ + if (*fd >= 0) + close(*fd); + *fd = -1; +} + +static void cleanup_demo_process(struct demo_process *demo) +{ + if (demo->child > 0) { + kill(demo->child, SIGKILL); + waitpid(demo->child, NULL, 0); + } + close_demo_pipe(&demo->ready_pipe[0]); + close_demo_pipe(&demo->ready_pipe[1]); + close_demo_pipe(&demo->continue_pipe[0]); + close_demo_pipe(&demo->continue_pipe[1]); +} + +static int start_demo_process(struct demo_process *demo, + const char *cgroup_path) +{ + struct pollfd ready = { .events = POLLIN }; + struct timespec leader_exit_delay = { .tv_nsec = 100000000 }; + char byte = 'x'; + + if (pipe(demo->ready_pipe) || pipe(demo->continue_pipe)) + return -1; + demo->child = fork(); + if (demo->child < 0) + return -1; + if (!demo->child) { + close(demo->ready_pipe[0]); + close(demo->continue_pipe[1]); + allocate_until_killed(cgroup_path, demo->ready_pipe[1], + demo->continue_pipe[0]); + } + close_demo_pipe(&demo->ready_pipe[1]); + close_demo_pipe(&demo->continue_pipe[0]); + ready.fd = demo->ready_pipe[0]; + if (poll(&ready, 1, 5000) != 1 || + read(demo->ready_pipe[0], &byte, 1) != 1 || + trigger_cross_cgroup_reclaim(cgroup_path)) + return -1; + nanosleep(&leader_exit_delay, NULL); + if (write(demo->continue_pipe[1], &byte, 1) != 1) + return -1; + close_demo_pipe(&demo->ready_pipe[0]); + close_demo_pipe(&demo->continue_pipe[1]); + return 0; +} + +static int collect_demo_events(struct demo_process *demo, + struct ring_buffer *ring) +{ + for (int i = 0; i < 200; i++) { + pid_t waited; + + ring_buffer__poll(ring, 50); + waited = waitpid(demo->child, &demo->status, WNOHANG); + if (waited == demo->child) { + demo->child = -1; + break; + } + } + for (int i = 0; i < 10 && exit_events < 1; i++) + ring_buffer__poll(ring, 50); + return demo->child < 0 ? 0 : -1; +} + +static bool valid_demo_observation(const struct demo_process *demo, + unsigned long long cgroup_id, + const struct oom_watch_bpf *skel) +{ + return WIFSIGNALED(demo->status) && + WTERMSIG(demo->status) == SIGKILL && victim_events == 1 && + exit_events == 1 && observed_cgroup_id == cgroup_id && + observed_victim_pid && observed_victim_tid && + observed_victim_pid != observed_victim_tid && observed_reclaims && + observed_cross_cgroup_reclaims && observed_stack_samples && + !skel->bss->dropped_victim_states && + !skel->bss->dropped_reclaim_states; +} + +static int run_demo(struct oom_runtime *runtime, + const struct selected_cgroup *selected) +{ + struct demo_process demo; + int result = -1; + + init_demo_process(&demo); + if (start_demo_process(&demo, selected->path) || + collect_demo_events(&demo, runtime->ring)) + goto cleanup; + printf("demo workload signaled=%d signal=%d\n", + WIFSIGNALED(demo.status), + WIFSIGNALED(demo.status) ? WTERMSIG(demo.status) : 0); + if (!valid_demo_observation(&demo, selected->metadata.st_ino, + runtime->skel)) + goto cleanup; + printf("demo result=matched-profile-to-victim\n"); + result = 0; + +cleanup: + cleanup_demo_process(&demo); + return result; +} + +static int watch_profiles(struct oom_runtime *runtime, + unsigned int duration_seconds) +{ + unsigned long long deadline = 0; + + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + if (duration_seconds) + deadline = monotonic_ns() + + (unsigned long long)duration_seconds * 1000000000ULL; + while (!stop && (!deadline || monotonic_ns() < deadline)) { + int result = ring_buffer__poll(runtime->ring, 100); + + if (result < 0 && result != -EINTR) { + fprintf(stderr, "ring buffer poll failed: %d\n", result); + return -1; + } + } + print_live_profiles(&runtime->context); + return 0; +} + +int main(int argc, char **argv) +{ + struct options options = { .sample_every = 1 }; + struct selected_cgroup selected = {}; + struct oom_runtime runtime = {}; + int err = 1; + + setvbuf(stdout, NULL, _IONBF, 0); + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + if (select_cgroup(&options, &selected) || + prepare_runtime(&runtime, &options, &selected)) + goto cleanup; + + if (selected.path) + printf("oom-watch tracing cgroup=%s cgroup_id=%llu\n", + selected.path, + (unsigned long long)selected.metadata.st_ino); + else + printf("oom-watch tracing all cgroups\n"); + + if ((options.demo && run_demo(&runtime, &selected)) || + (!options.demo && watch_profiles(&runtime, + options.duration_seconds))) + goto cleanup; + printf("dropped_victim_states=%llu dropped_reclaim_states=%llu\n", + (unsigned long long)runtime.skel->bss->dropped_victim_states, + (unsigned long long)runtime.skel->bss->dropped_reclaim_states); + err = 0; + +cleanup: + destroy_runtime(&runtime); + cleanup_selected_cgroup(&selected, &err); + return err; +} diff --git a/src/57-oom-watch/oom_watch.h b/src/57-oom-watch/oom_watch.h new file mode 100644 index 00000000..f2a71f98 --- /dev/null +++ b/src/57-oom-watch/oom_watch.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __OOM_WATCH_H +#define __OOM_WATCH_H + +#define OOM_RECLAIM_BUCKETS 20 +#define OOM_STACK_DEPTH 127 + +enum oom_watch_event_type { + OOM_VICTIM_MARKED = 1, + OOM_VICTIM_EXITED = 2, +}; + +struct reclaim_profile { + unsigned long long begin_count; + unsigned long long end_count; + unsigned long long reclaimed_pages; + unsigned long long cross_cgroup_reclaims; + unsigned long long last_reclaim_ns; + unsigned long long total_reclaim_ns; + unsigned long long maximum_reclaim_ns; + unsigned long long stack_samples; + unsigned long long stack_failures; + unsigned long long latency_slots[OOM_RECLAIM_BUCKETS]; +}; + +struct reclaim_stack_key { + unsigned long long cgroup_id; + signed int stack_id; + unsigned int padding; +}; + +struct reclaim_stack_profile { + unsigned long long samples; + unsigned long long total_ns; + unsigned long long maximum_ns; + unsigned long long reclaimed_pages; +}; + +struct oom_watch_event { + unsigned long long timestamp_ns; + unsigned long long cgroup_id; + struct reclaim_profile profile; + unsigned long long total_vm_kb; + unsigned long long anon_rss_kb; + unsigned long long file_rss_kb; + unsigned int type; + unsigned int victim_pid; + unsigned int victim_tid; + unsigned int triggering_tgid; + signed int exit_code; + char comm[16]; +}; + +#endif /* __OOM_WATCH_H */ diff --git a/src/57-oom-watch/tests/test_oom_watch.py b/src/57-oom-watch/tests/test_oom_watch.py new file mode 100644 index 00000000..3f198738 --- /dev/null +++ b/src/57-oom-watch/tests/test_oom_watch.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 +import re +import subprocess +import sys +from pathlib import Path + + +def main(): + subtree_control = Path("/sys/fs/cgroup/cgroup.subtree_control") + before = subtree_control.read_text() + result = subprocess.run( + [sys.argv[1], "--demo"], + text=True, + capture_output=True, + check=False, + timeout=30, + ) + output = result.stdout + result.stderr + after = subtree_control.read_text() + if result.returncode: + raise AssertionError(output) + assert before == after, (before, after, output) + assert "event=oom-victim" in output, output + match = re.search(r"event=oom-victim pid=(\d+) tid=(\d+)", output) + assert match and match.group(1) != match.group(2), output + assert "reclaim_cycles=" in output, output + assert "cross_cgroup_reclaims=" in output, output + assert "reclaim_profile cgroup_id=" in output, output + assert "reclaim_latency_us=" in output, output + assert "reclaim_stack rank=1" in output, output + assert "try_to_free_mem_cgroup_pages" in output, output + assert "event=victim-exit" in output, output + assert "demo workload signaled=1 signal=9" in output, output + assert "demo result=matched-profile-to-victim" in output, output + assert "dropped_victim_states=0 dropped_reclaim_states=0" in output, output + print("OOM watch integration test: PASS") + + +if __name__ == "__main__": + main() diff --git a/src/60-afxdp-dump/.config b/src/60-afxdp-dump/.config new file mode 100644 index 00000000..e5d9cf41 --- /dev/null +++ b/src/60-afxdp-dump/.config @@ -0,0 +1,10 @@ +level=Depth +type=Networking +kernel_min=5.7 +kernel_min_basis=required-feature +architectures=x86_64 +btf=required +kernel_config=CONFIG_BPF=y,CONFIG_BPF_SYSCALL=y,CONFIG_BPF_JIT=y,CONFIG_DEBUG_INFO_BTF=y,CONFIG_XDP_SOCKETS=y +hardware=network-interface +root=required +test_status=ci-build diff --git a/src/60-afxdp-dump/.gitignore b/src/60-afxdp-dump/.gitignore new file mode 100644 index 00000000..6d312feb --- /dev/null +++ b/src/60-afxdp-dump/.gitignore @@ -0,0 +1,2 @@ +.output/ +afxdp_dump diff --git a/src/60-afxdp-dump/Makefile b/src/60-afxdp-dump/Makefile new file mode 100644 index 00000000..7379603d --- /dev/null +++ b/src/60-afxdp-dump/Makefile @@ -0,0 +1,94 @@ +# SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +OUTPUT := .output +CLANG ?= clang +LIBBPF_SRC := $(abspath ../third_party/libbpf/src) +BPFTOOL_SRC := $(abspath ../third_party/bpftool/src) +LIBBPF_OBJ := $(abspath $(OUTPUT)/libbpf.a) +BPFTOOL_OUTPUT ?= $(abspath $(OUTPUT)/bpftool) +BPFTOOL ?= $(BPFTOOL_OUTPUT)/bootstrap/bpftool +ARCH ?= $(shell uname -m | sed 's/x86_64/x86/' \ + | sed 's/arm.*/arm/' \ + | sed 's/aarch64/arm64/' \ + | sed 's/ppc64le/powerpc/' \ + | sed 's/mips.*/mips/' \ + | sed 's/riscv64/riscv/' \ + | sed 's/loongarch64/loongarch/') +VMLINUX := ../third_party/vmlinux/$(ARCH)/vmlinux.h +INCLUDES := -I$(OUTPUT) -I../third_party/libbpf/include/uapi -I$(dir $(VMLINUX)) -I. +CFLAGS := -g -Wall -Wextra -D_GNU_SOURCE +ALL_LDFLAGS := $(LDFLAGS) $(EXTRA_LDFLAGS) +APPS = afxdp_dump + +CLANG_BPF_SYS_INCLUDES ?= $(shell $(CLANG) -v -E - &1 \ + | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') + +ifeq ($(V),1) + Q = + msg = +else + Q = @ + msg = @printf ' %-8s %s%s\n' "$(1)" \ + "$(patsubst $(abspath $(OUTPUT))/%,%,$(2))" \ + "$(if $(3), $(3))"; + MAKEFLAGS += --no-print-directory +endif + +define allow-override + $(if $(or $(findstring environment,$(origin $(1))),\ + $(findstring command line,$(origin $(1)))),,\ + $(eval $(1) = $(2))) +endef + +$(call allow-override,CC,$(CROSS_COMPILE)cc) + +.PHONY: all clean test +all: $(APPS) + +clean: + $(call msg,CLEAN) + $(Q)rm -rf $(OUTPUT) $(APPS) + +test: all + @if [ "$$(id -u)" -ne 0 ]; then \ + echo "test requires root; run inside the documented KVM guest" >&2; \ + exit 1; \ + fi + python3 tests/test_afxdp_dump.py ./afxdp_dump + +$(OUTPUT) $(OUTPUT)/libbpf $(BPFTOOL_OUTPUT): + $(call msg,MKDIR,$@) + $(Q)mkdir -p $@ + +$(LIBBPF_OBJ): $(wildcard $(LIBBPF_SRC)/*.[ch] $(LIBBPF_SRC)/Makefile) | $(OUTPUT)/libbpf + $(call msg,LIB,$@) + $(Q)$(MAKE) -C $(LIBBPF_SRC) BUILD_STATIC_ONLY=1 \ + OBJDIR=$(dir $@)/libbpf DESTDIR=$(dir $@) \ + INCLUDEDIR= LIBDIR= UAPIDIR= install + +$(BPFTOOL): | $(BPFTOOL_OUTPUT) + $(call msg,BPFTOOL,$@) + $(Q)$(MAKE) ARCH= CROSS_COMPILE= OUTPUT=$(BPFTOOL_OUTPUT)/ -C $(BPFTOOL_SRC) bootstrap + +$(OUTPUT)/%.bpf.o: %.bpf.c $(LIBBPF_OBJ) $(wildcard %.h) $(VMLINUX) | $(OUTPUT) $(BPFTOOL) + $(call msg,BPF,$@) + $(Q)$(CLANG) -g -O2 -target bpf -D__TARGET_ARCH_$(ARCH) \ + $(INCLUDES) $(CLANG_BPF_SYS_INCLUDES) \ + -c $(filter %.c,$^) -o $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + $(Q)$(BPFTOOL) gen object $@ $(patsubst %.bpf.o,%.tmp.bpf.o,$@) + +$(OUTPUT)/%.skel.h: $(OUTPUT)/%.bpf.o | $(OUTPUT) $(BPFTOOL) + $(call msg,GEN-SKEL,$@) + $(Q)$(BPFTOOL) gen skeleton $< > $@ + +$(patsubst %,$(OUTPUT)/%.o,$(APPS)): %.o: %.skel.h + +$(OUTPUT)/%.o: %.c $(wildcard %.h) | $(OUTPUT) + $(call msg,CC,$@) + $(Q)$(CC) $(CFLAGS) $(INCLUDES) -c $(filter %.c,$^) -o $@ + +$(APPS): %: $(OUTPUT)/%.o $(LIBBPF_OBJ) | $(OUTPUT) + $(call msg,BINARY,$@) + $(Q)$(CC) $(CFLAGS) $^ $(ALL_LDFLAGS) -lelf -lz -o $@ + +.DELETE_ON_ERROR: +.SECONDARY: diff --git a/src/60-afxdp-dump/README.md b/src/60-afxdp-dump/README.md new file mode 100644 index 00000000..268f4719 --- /dev/null +++ b/src/60-afxdp-dump/README.md @@ -0,0 +1,728 @@ +# eBPF Tutorial: High-Performance UDP Receiving with AF_XDP + +Have you ever wondered how high-performance network applications receive millions of packets per second without drowning in kernel overhead? The key is bypassing most of the network stack. AF_XDP lets you select packets at the driver boundary, copy them directly into your application's memory, and process them without a system call for every packet. + +This tutorial builds an exclusive packet receiver from scratch using the raw AF_XDP interface. No helper libraries, no magic abstractions. You'll see exactly how UMEM registration, ring buffers, and XDP redirection work together. The result is `afxdp-dump`, a tool that redirects IPv4 UDP packets for a specific port away from the normal socket stack, prints a payload preview, and properly recycles every frame to keep receiving indefinitely. + +> Complete source code: + +## Why AF_XDP? + +Traditional packet capture with `libpcap` or raw sockets has a fundamental problem: every packet crosses the kernel-userspace boundary through expensive system calls. When you're capturing 10 Gbps of traffic, this overhead becomes the bottleneck, not your processing code. + +AF_XDP solves this by establishing shared memory between kernel and userspace. The kernel writes packets directly into memory your application can read. You communicate through lock-free ring buffers instead of system calls. A single `poll()` can wake you for hundreds of packets. This architecture enables packet rates of millions per second on commodity hardware. + +Understanding AF_XDP also teaches patterns that appear throughout high-performance systems: shared memory, lock-free data structures, and explicit ownership transfer. + +## The AF_XDP Architecture + +AF_XDP works through four components that must coordinate precisely: + +**UMEM (User Memory)** is a region of memory you allocate that both kernel and userspace can access. You divide it into fixed-size frames, typically 4096 bytes each. Every packet the kernel delivers arrives in one of these frames. + +**The Fill Ring** is how you tell the kernel which frames are available for incoming packets. You post frame addresses here. The kernel consumes these addresses when it needs somewhere to put a packet. + +**The RX Ring** is where the kernel tells you about received packets. Each entry contains a frame address and packet length. When you see an entry here, you own that frame until you return it. + +**XSKMAP** is a BPF map that connects XDP programs to AF_XDP sockets. The XDP program decides which packets to redirect, looks up the socket for the current RX queue in this map, and calls `bpf_redirect_map()` to deliver the packet. + +The flow works like this: you post 64 frame addresses to the Fill Ring. A UDP packet arrives. Your XDP program checks the destination port, looks up the socket in XSKMAP, and redirects. The kernel copies the packet into one of your frames and publishes a descriptor on the RX Ring. You read the descriptor, process the packet, and post the frame address back to the Fill Ring. The cycle continues indefinitely. + +This ownership model is critical. A frame starts with you. You lend it to the kernel via the Fill Ring. The kernel borrows it to receive a packet. You reclaim it from the RX Ring. You must return it to the Fill Ring or you'll run out of frames after 64 packets. Our tool proves this works by successfully capturing 65 packets, which requires at least one frame to complete the full ownership cycle. + +## The XDP Program: Filtering and Redirecting + +The kernel-side BPF program is compact because it only handles filtering and redirection. All the complexity of buffer management lives in userspace. + +First, we define a shared header that sets the XSKMAP capacity. Queue IDs are map keys, so this example can address queues 0 through 63: + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __AFXDP_DUMP_H +#define __AFXDP_DUMP_H + +#define AFXDP_MAX_QUEUES 64 + +#endif /* __AFXDP_DUMP_H */ +``` + +Now the XDP program itself: + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "afxdp_dump.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff + +const volatile __u16 capture_port = 8080; + +struct { + __uint(type, BPF_MAP_TYPE_XSKMAP); + __uint(max_entries, AFXDP_MAX_QUEUES); + __type(key, __u32); + __type(value, __u32); +} xsk_map SEC(".maps"); + +__u64 redirected_packets; + +SEC("xdp") +int redirect_udp(struct xdp_md *ctx) +{ + void *data = (void *)(long)ctx->data; + void *data_end = (void *)(long)ctx->data_end; + struct ethhdr *ethernet = data; + struct iphdr *ip; + struct udphdr *udp; + __u32 queue = ctx->rx_queue_index; + __u32 ip_header_length; + __u32 ip_length; + __u32 udp_length; + + if ((void *)(ethernet + 1) > data_end || + ethernet->h_proto != bpf_htons(ETH_P_IP)) + return XDP_PASS; + ip = (void *)(ethernet + 1); + if ((void *)(ip + 1) > data_end || ip->version != 4 || + ip->protocol != IPPROTO_UDP || + ip->ihl < 5 || (bpf_ntohs(ip->frag_off) & (IP_MF | IP_OFFSET))) + return XDP_PASS; + ip_header_length = ip->ihl * 4; + ip_length = bpf_ntohs(ip->tot_len); + udp = (void *)ip + ip_header_length; + if (ip_length < ip_header_length + sizeof(*udp) || + (void *)ip + ip_length > data_end || + (void *)(udp + 1) > data_end) + return XDP_PASS; + udp_length = bpf_ntohs(udp->len); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp->dest != bpf_htons(capture_port)) + return XDP_PASS; + if (!bpf_map_lookup_elem(&xsk_map, &queue)) + return XDP_PASS; + __sync_fetch_and_add(&redirected_packets, 1); + return bpf_redirect_map(&xsk_map, queue, XDP_PASS); +} +``` + +The program parses packets layer by layer, validating boundaries at each step. This careful bounds checking is required by the BPF verifier and also ensures we don't misinterpret truncated or malformed packets. + +The parsing starts with Ethernet, checking that there's room for the header and that the EtherType indicates IPv4. Then it validates the IPv4 header: correct version, UDP protocol, minimum header length, and no fragmentation (fragmented packets would need reassembly, which is beyond our scope). The program computes the actual IP header length from the IHL field and uses it to locate the UDP header. + +The UDP validation ensures the length field is sane and that the destination port matches our target. Only then does the program check if there's actually a socket registered for this queue. This ordering is deliberate: most packets will fail earlier checks, so we avoid the map lookup cost for packets we won't capture anyway. + +The XSKMAP lookup is also a safety check. If userspace hasn't registered a socket for this queue, the lookup returns NULL and we pass the packet to the normal stack. When everything checks out, `bpf_redirect_map()` sends the packet to AF_XDP. The second argument is the queue index, which becomes the map key. The third argument is the fallback action if something goes wrong. + +One important detail: once a packet is redirected, it's consumed by AF_XDP. The regular socket stack will never see it. This is expected for this exclusive receiver, but it means you need to be careful about what you redirect. + +## The Userspace Application + +The userspace code handles everything AF_XDP needs: memory allocation, ring setup, socket binding, XDP loading, and the receive loop. Here's the complete implementation: + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#define _FILE_OFFSET_BITS 64 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "afxdp_dump.h" +#include "afxdp_dump.skel.h" + +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif +#ifndef AF_XDP +#define AF_XDP 44 +#endif +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +#define FRAME_SIZE 4096 +#define FRAME_COUNT 64 +#define RING_SIZE 64 +#define UMEM_SIZE ((size_t)FRAME_SIZE * FRAME_COUNT) +#define PAYLOAD_PREVIEW 32 +#define IPV4_MORE_FRAGMENTS 0x2000 +#define IPV4_FRAGMENT_OFFSET 0x1fff + +struct options { + const char *interface; + unsigned int queue; + unsigned int port; + unsigned int count; + bool skb_mode; +}; + +struct mapped_ring { + void *mapping; + size_t mapping_size; + unsigned int *producer; + unsigned int *consumer; + unsigned int *flags; + void *descriptors; + unsigned int mask; +}; + +struct xsk_state { + int fd; + void *umem; + struct mapped_ring fill; + struct mapped_ring completion; + struct mapped_ring rx; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static void map_ring_fields(struct mapped_ring *ring, void *mapping, + size_t mapping_size, + const struct xdp_ring_offset *offset) +{ + ring->mapping = mapping; + ring->mapping_size = mapping_size; + ring->producer = mapping + offset->producer; + ring->consumer = mapping + offset->consumer; + ring->flags = mapping + offset->flags; + ring->descriptors = mapping + offset->desc; + ring->mask = RING_SIZE - 1; +} + +static int map_xsk_ring(struct mapped_ring *ring, int fd, + const struct xdp_ring_offset *offset, + off_t page_offset, size_t descriptor_size) +{ + size_t size = offset->desc + RING_SIZE * descriptor_size; + void *mapping; + + mapping = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, page_offset); + if (mapping == MAP_FAILED) + return -1; + map_ring_fields(ring, mapping, size, offset); + return 0; +} + +static void close_xsk(struct xsk_state *xsk) +{ + struct mapped_ring *rings[] = { + &xsk->rx, &xsk->completion, &xsk->fill, + }; + + for (size_t i = 0; i < sizeof(rings) / sizeof(rings[0]); i++) + if (rings[i]->mapping && rings[i]->mapping != MAP_FAILED) + munmap(rings[i]->mapping, rings[i]->mapping_size); + if (xsk->fd >= 0) + close(xsk->fd); + if (xsk->umem && xsk->umem != MAP_FAILED) + munmap(xsk->umem, UMEM_SIZE); + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; +} + +static int open_xsk(struct xsk_state *xsk, int ifindex, unsigned int queue) +{ + struct xdp_umem_reg registration = { + .len = UMEM_SIZE, + .chunk_size = FRAME_SIZE, + }; + struct sockaddr_xdp address = { + .sxdp_family = PF_XDP, + .sxdp_ifindex = ifindex, + .sxdp_queue_id = queue, + .sxdp_flags = XDP_COPY, + }; + struct xdp_mmap_offsets offsets; + socklen_t offsets_size = sizeof(offsets); + unsigned int ring_size = RING_SIZE; + unsigned long long *fill_addresses; + + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; + xsk->umem = mmap(NULL, UMEM_SIZE, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0); + if (xsk->umem == MAP_FAILED) + return -1; + xsk->fd = socket(AF_XDP, SOCK_RAW | SOCK_CLOEXEC, 0); + if (xsk->fd < 0) + return -1; + registration.addr = (uintptr_t)xsk->umem; + if (setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_REG, ®istration, + sizeof(registration)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_FILL_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_COMPLETION_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_RX_RING, &ring_size, + sizeof(ring_size)) || + getsockopt(xsk->fd, SOL_XDP, XDP_MMAP_OFFSETS, &offsets, + &offsets_size)) + return -1; + + if (map_xsk_ring(&xsk->fill, xsk->fd, &offsets.fr, + XDP_UMEM_PGOFF_FILL_RING, sizeof(unsigned long long)) || + map_xsk_ring(&xsk->completion, xsk->fd, &offsets.cr, + XDP_UMEM_PGOFF_COMPLETION_RING, + sizeof(unsigned long long)) || + map_xsk_ring(&xsk->rx, xsk->fd, &offsets.rx, + XDP_PGOFF_RX_RING, sizeof(struct xdp_desc)) || + bind(xsk->fd, (struct sockaddr *)&address, sizeof(address))) + return -1; + + fill_addresses = xsk->fill.descriptors; + for (unsigned int i = 0; i < FRAME_COUNT; i++) + fill_addresses[i & xsk->fill.mask] = + (unsigned long long)i * FRAME_SIZE; + __atomic_store_n(xsk->fill.producer, FRAME_COUNT, __ATOMIC_RELEASE); + return 0; +} + +static unsigned long long data_address(unsigned long long address) +{ + return (address & XSK_UNALIGNED_BUF_ADDR_MASK) + + (address >> XSK_UNALIGNED_BUF_OFFSET_SHIFT); +} + +static int recycle_frame(struct xsk_state *xsk, unsigned long long address) +{ + unsigned long long *fill_addresses = xsk->fill.descriptors; + unsigned int producer, consumer; + + producer = __atomic_load_n(xsk->fill.producer, __ATOMIC_RELAXED); + consumer = __atomic_load_n(xsk->fill.consumer, __ATOMIC_ACQUIRE); + if (producer - consumer == RING_SIZE) + return -1; + fill_addresses[producer & xsk->fill.mask] = address; + __atomic_store_n(xsk->fill.producer, producer + 1, __ATOMIC_RELEASE); + return 0; +} + +static const struct iphdr *parse_ipv4(const void *packet, unsigned int length, + unsigned int *ip_header_length, + unsigned int *ip_length) +{ + const struct ethhdr *ethernet = packet; + const struct iphdr *ip; + unsigned int available_ip; + + if (length < sizeof(*ethernet) + sizeof(*ip) || + ethernet->h_proto != htons(ETH_P_IP)) + return NULL; + ip = packet + sizeof(*ethernet); + *ip_header_length = ip->ihl * 4; + if (ip->version != 4 || ip->protocol != IPPROTO_UDP || + (ntohs(ip->frag_off) & + (IPV4_MORE_FRAGMENTS | IPV4_FRAGMENT_OFFSET)) || + *ip_header_length < sizeof(*ip) || + length < sizeof(*ethernet) + *ip_header_length) + return NULL; + *ip_length = ntohs(ip->tot_len); + available_ip = length - sizeof(*ethernet); + if (*ip_length < *ip_header_length + sizeof(struct udphdr) || + *ip_length > available_ip) + return NULL; + return ip; +} + +static const struct udphdr *parse_udp(const void *packet, + unsigned int length, + unsigned int ip_header_length, + unsigned int ip_length, + const unsigned char **payload, + unsigned int *payload_length) +{ + const struct udphdr *udp = packet + sizeof(struct ethhdr) + + ip_header_length; + unsigned int available_payload; + unsigned int udp_length; + + *payload = (const unsigned char *)(udp + 1); + udp_length = ntohs(udp->len); + available_payload = length - (*payload - (const unsigned char *)packet); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp_length - sizeof(*udp) > available_payload) + return NULL; + *payload_length = udp_length - sizeof(*udp); + return udp; +} + +static void dump_packet(const void *packet, unsigned int length, + unsigned int packet_number) +{ + const struct iphdr *ip; + const struct udphdr *udp; + const unsigned char *payload; + char source[INET_ADDRSTRLEN], destination[INET_ADDRSTRLEN]; + char preview[PAYLOAD_PREVIEW + 1]; + unsigned int ip_header_length, ip_length, payload_length, preview_length; + + ip = parse_ipv4(packet, length, &ip_header_length, &ip_length); + if (!ip) + return; + udp = parse_udp(packet, length, ip_header_length, ip_length, &payload, + &payload_length); + if (!udp) + return; + preview_length = payload_length < PAYLOAD_PREVIEW ? + payload_length : PAYLOAD_PREVIEW; + for (unsigned int i = 0; i < preview_length; i++) + preview[i] = isprint(payload[i]) ? payload[i] : '.'; + preview[preview_length] = '\0'; + inet_ntop(AF_INET, &ip->saddr, source, sizeof(source)); + inet_ntop(AF_INET, &ip->daddr, destination, sizeof(destination)); + printf("packet=%u %s:%u -> %s:%u bytes=%u payload=\"%s\"\n", + packet_number, source, ntohs(udp->source), destination, + ntohs(udp->dest), length, preview); +} + +static int receive_packets(struct xsk_state *xsk, unsigned int count) +{ + struct pollfd poll_fd = { .fd = xsk->fd, .events = POLLIN }; + unsigned int received = 0; + + while (!stop && (!count || received < count)) { + unsigned int consumer, producer; + int poll_result = poll(&poll_fd, 1, 250); + + if (poll_result < 0) { + if (errno == EINTR) + continue; + return -1; + } + if (!poll_result) + continue; + consumer = __atomic_load_n(xsk->rx.consumer, __ATOMIC_RELAXED); + producer = __atomic_load_n(xsk->rx.producer, __ATOMIC_ACQUIRE); + while (consumer != producer && (!count || received < count)) { + struct xdp_desc *descriptors = xsk->rx.descriptors; + struct xdp_desc descriptor = + descriptors[consumer & xsk->rx.mask]; + unsigned long long address = data_address(descriptor.addr); + + if (descriptor.options & XDP_PKT_CONTD) { + errno = EMSGSIZE; + return -1; + } + if (address + descriptor.len > UMEM_SIZE) { + errno = EFAULT; + return -1; + } + received++; + dump_packet(xsk->umem + address, descriptor.len, received); + consumer++; + __atomic_store_n(xsk->rx.consumer, consumer, + __ATOMIC_RELEASE); + if (recycle_frame(xsk, descriptor.addr)) { + errno = ENOBUFS; + return -1; + } + } + } + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--queue N] [--port PORT] [--count N] [--skb-mode]\n", + program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "queue", required_argument, NULL, 'q' }, + { "port", required_argument, NULL, 'p' }, + { "count", required_argument, NULL, 'n' }, + { "skb-mode", no_argument, NULL, 'S' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:q:p:n:Sh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'q': + if (parse_uint(optarg, AFXDP_MAX_QUEUES - 1, + &options->queue)) + return -1; + break; + case 'p': + if (parse_uint(optarg, 65535, &options->port) || + !options->port) + return -1; + break; + case 'n': + if (parse_uint(optarg, 1000000, &options->count)) + return -1; + break; + case 'S': options->skb_mode = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + return optind == argc && options->interface ? 0 : -1; +} + +static int attach_xdp(int ifindex, int program_fd, bool skb_mode, + unsigned int *attached_flags) +{ + unsigned int flags = XDP_FLAGS_UPDATE_IF_NOEXIST | + (skb_mode ? XDP_FLAGS_SKB_MODE : XDP_FLAGS_DRV_MODE); + int err; + + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + if (err && !skb_mode && (err == -EOPNOTSUPP || err == -EINVAL)) { + flags = XDP_FLAGS_UPDATE_IF_NOEXIST | XDP_FLAGS_SKB_MODE; + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + } + if (!err) + *attached_flags = flags; + return err; +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 8080 }; + struct afxdp_dump_bpf *skel = NULL; + struct xsk_state xsk; + unsigned int attached_flags = 0; + int ifindex; + int err = 1; + bool attached = false; + + setvbuf(stdout, NULL, _IONBF, 0); + memset(&xsk, 0, sizeof(xsk)); + xsk.fd = -1; + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + ifindex = if_nametoindex(options.interface); + if (!ifindex) { + fprintf(stderr, "unknown interface: %s\n", options.interface); + return 2; + } + if (open_xsk(&xsk, ifindex, options.queue)) { + fprintf(stderr, "failed to create AF_XDP socket: %s\n", + strerror(errno)); + goto cleanup; + } + skel = afxdp_dump_bpf__open(); + if (!skel) + goto cleanup; + skel->rodata->capture_port = options.port; + if (afxdp_dump_bpf__load(skel)) { + fprintf(stderr, "failed to load XDP program\n"); + goto cleanup; + } + if (bpf_map__update_elem(skel->maps.xsk_map, &options.queue, + sizeof(options.queue), &xsk.fd, sizeof(xsk.fd), + BPF_ANY)) { + fprintf(stderr, "failed to populate XSK map: %s\n", strerror(errno)); + goto cleanup; + } + { + int attach_error = attach_xdp(ifindex, + bpf_program__fd(skel->progs.redirect_udp), + options.skb_mode, &attached_flags); + + if (attach_error) { + fprintf(stderr, "failed to attach XDP program: %s\n", + strerror(-attach_error)); + goto cleanup; + } + } + attached = true; + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + printf("afxdp-dump ready interface=%s queue=%u port=%u mode=%s count=%u\n", + options.interface, options.queue, options.port, + attached_flags & XDP_FLAGS_SKB_MODE ? "skb" : "driver", + options.count); + + if (receive_packets(&xsk, options.count)) { + fprintf(stderr, "AF_XDP receive failed: %s\n", strerror(errno)); + goto cleanup; + } + printf("redirected=%llu\n", + (unsigned long long)skel->bss->redirected_packets); + err = 0; + +cleanup: + if (attached) { + LIBBPF_OPTS(bpf_xdp_attach_opts, detach_options, + .old_prog_fd = bpf_program__fd( + skel->progs.redirect_udp)); + int detach_error = bpf_xdp_detach( + ifindex, attached_flags & XDP_FLAGS_MODES, + &detach_options); + + if (detach_error) { + fprintf(stderr, "failed to detach XDP program: %s\n", + strerror(-detach_error)); + err = 1; + } + } + afxdp_dump_bpf__destroy(skel); + close_xsk(&xsk); + return err; +} +``` + +### Understanding UMEM and Ring Setup + +The `open_xsk()` function creates the AF_XDP socket and all its supporting infrastructure. It starts by mapping 256 KiB of anonymous memory for UMEM, divided into 64 frames of 4096 bytes each. This memory will be shared with the kernel after registration. + +After creating the socket with `AF_XDP`, it registers the UMEM with `XDP_UMEM_REG`, telling the kernel where our packet buffers live. Then it requests fill, completion, and RX rings of 64 entries each with `setsockopt`. The `XDP_MMAP_OFFSETS` getsockopt reveals where each ring's producer index, consumer index, flags, and descriptor array are located within pages that can be mmap'd. + +The ring mapping is tricky because the kernel lays out each ring at a fixed page offset. The Fill Ring lives at `XDP_UMEM_PGOFF_FILL_RING`, the Completion Ring at `XDP_UMEM_PGOFF_COMPLETION_RING`, and the RX Ring at `XDP_PGOFF_RX_RING`. Each ring contains pointers to the producer and consumer indices, which are the synchronization points between kernel and userspace. + +The final step before binding is posting all 64 frame addresses to the Fill Ring. We write each address into the descriptor array and then publish the producer index with release semantics. This tells the kernel it has 64 frames available for receiving packets. + +### Memory Ordering in Ring Operations + +Ring indices are shared between kernel and userspace, which makes memory ordering critical. The pattern is consistent throughout: a producer writes descriptors before publishing its index with release semantics, and a consumer acquires the producer index before reading descriptors. + +In `recycle_frame()`, we read our own producer index (which only we modify) with relaxed ordering, but we acquire the consumer index because the kernel writes it. If there's room in the ring, we write the address and publish with release. The kernel will eventually acquire our producer update and see the address we wrote. + +The same pattern appears in the receive loop. We acquire the producer index written by the kernel, read descriptors, and publish our consumer update with release. This ensures the kernel knows we're done with those frames before it reuses them. + +### Copy Mode vs Zero-Copy + +The `XDP_COPY` flag in the socket address tells AF_XDP to copy packets into UMEM rather than doing true zero-copy. Copy mode works on any interface without driver support, making it the right choice for a learning example. The kernel allocates its own memory for incoming packets and copies them into our UMEM frames. + +The XDP program attach mode is different. We first try native driver mode (`XDP_FLAGS_DRV_MODE`), which runs the XDP program in the driver before SKB allocation. If that fails because the driver doesn't support XDP, we fall back to generic SKB mode (`XDP_FLAGS_SKB_MODE`), which runs after the SKB is created but still lets us redirect to AF_XDP. The `--skb-mode` flag forces generic mode directly. + +### The Receive Loop + +The receive loop polls for packets, processes them in batches, and recycles frames immediately. The 250ms timeout ensures we can respond to signals even when no packets arrive. + +When poll indicates data, we check the RX ring for new descriptors. For each one, we compute the actual data address (the descriptor address might be encoded with offset information), verify it's within UMEM bounds, and check that this isn't a multi-buffer packet (which would have `XDP_PKT_CONTD` set). Multi-buffer support would require accumulating fragments, which is beyond this example's scope. + +After printing the packet, we immediately return the frame to the Fill Ring via `recycle_frame()`. This is critical: without recycling, we'd run out of frames after 64 packets. The tool proves recycling works by successfully capturing 65 packets. + +### Cleanup and Safe Detach + +On exit, we use compare-and-detach to remove only our XDP program. The `old_prog_fd` option tells `bpf_xdp_detach` to only detach if the currently attached program matches ours. This prevents accidentally detaching someone else's XDP program if they attached one while we were running. + +## Compilation and Execution + +Build the executable: + +```bash +cd src/60-afxdp-dump +make +``` + +Capture five UDP packets arriving on queue 0 with destination port 8080: + +```bash +sudo ./afxdp_dump --interface eth0 --queue 0 --port 8080 --count 5 +``` + +Use `--count 0` to run until interrupted with Ctrl+C. Add `--skb-mode` to force generic XDP mode. The selected queue must actually receive the traffic; on multi-queue NICs, this depends on RSS (receive-side scaling) configuration. + +Send test traffic from another machine or terminal: + +```bash +echo "hello-afxdp" | nc -u target-ip 8080 +``` + +A longer run with `--count 65` demonstrates that frame recycling works: + +```text +afxdp-dump ready interface=eth0 queue=0 port=8080 mode=driver count=65 +packet=1 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp" +packet=65 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp" +redirected=65 +``` + +Packet 65 proves that at least one frame completed the full ownership cycle: posted to Fill, used for receive, consumed from RX, and posted back to Fill. The `redirected=65` counter comes from the BPF program and matches our receive count. + +## Requirements + +| Requirement | Details | +|---|---| +| Kernel | Linux 5.7 or newer. AF_XDP arrived in 4.18, XSKMAP lookup from XDP in 5.3, and safe expected-FD detach in 5.7 | +| Kernel config | `CONFIG_BPF`, `CONFIG_BPF_SYSCALL`, `CONFIG_BPF_JIT`, `CONFIG_XDP_SOCKETS`, `CONFIG_DEBUG_INFO_BTF` | +| Privileges | Root or equivalent BPF and network-admin capabilities | +| Interface | Any interface with the selected RX queue. Native XDP is optional since generic mode works everywhere | +| Architecture | x86-64 is tested. Copy mode works without driver zero-copy support | + +## What's Next + +This example is deliberately minimal: receive-only, single-queue, single-buffer packets, copy mode. Real production AF_XDP applications extend this foundation in several directions: + +**TX rings** let you send packets with the same zero-overhead model. You'd add a TX ring, post frame addresses with packet data, and poll for completion notifications. + +**Zero-copy mode** eliminates the copy into UMEM. The driver uses your UMEM directly, but this requires driver support and careful buffer alignment. + +**Multi-buffer packets** handle jumbo frames or when UMEM frames are smaller than the MTU. You'd accumulate fragments marked with `XDP_PKT_CONTD` until the final fragment. + +**Shared UMEM** lets multiple sockets share the same memory, useful for load-balancing across queues or between RX and TX paths. + +## Summary + +AF_XDP lets packet reception bypass the normal network stack while retaining eBPF's safety checks. The XDP program selects traffic at the driver boundary, the XSKMAP routes packets to your socket, and lock-free ring buffers transfer data without system calls. This example showed the complete receive contract: post frames to Fill, receive descriptors on RX, process packets, recycle frames back to Fill. + +Understanding this flow is valuable beyond packet reception. The patterns here, shared memory between kernel and userspace, explicit ownership transfer, lock-free synchronization, appear throughout high-performance systems from databases to GPU drivers. + +> If you'd like to dive deeper into eBPF, check out our tutorial repository at or visit our website at . + +## References + +- [Linux AF_XDP Documentation](https://docs.kernel.org/networking/af_xdp.html) +- [AF_XDP Introduction Commit](https://github.com/torvalds/linux/commit/c0c77d8fb787cfe0c3fca689c2a30d1dad4eaba7) +- [XSKMAP Lookup from XDP](https://github.com/torvalds/linux/commit/fada7fdc83c0) +- [Expected-Program FD for XDP Detach](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9) +- [libxdp Library](https://github.com/xdp-project/xdp-tools) - Higher-level AF_XDP helpers if you want to skip the raw ABI diff --git a/src/60-afxdp-dump/README.zh.md b/src/60-afxdp-dump/README.zh.md new file mode 100644 index 00000000..c21a4a22 --- /dev/null +++ b/src/60-afxdp-dump/README.zh.md @@ -0,0 +1,728 @@ +# eBPF 实战教程：用 AF_XDP 高性能接收 UDP 报文 + +你有没有好奇过高性能网络应用是如何每秒接收数百万个报文而不被内核开销拖垮的？关键在于绕过大部分网络协议栈。AF_XDP 让你在网卡驱动入口选择报文，直接复制到应用程序的内存空间，处理时不需要为每个报文做一次系统调用。 + +本教程从零开始，使用原始的 AF_XDP 接口构建一个独占式报文接收器。没有辅助库，没有魔法抽象。你会清楚看到 UMEM 注册、ring buffer 和 XDP 重定向是如何协同工作的。最终成果是 `afxdp-dump`，它把指定端口的 IPv4 UDP 报文从普通 socket 协议栈重定向到用户态，打印 payload 预览，并正确回收每个 frame 以持续接收。 + +> 完整源代码： + +## 为什么选择 AF_XDP？ + +传统的 `libpcap` 或 raw socket 抓包有一个根本问题：每个报文都要通过昂贵的系统调用跨越内核与用户态的边界。当你抓取 10 Gbps 流量时，这些开销会成为瓶颈，而不是你的处理代码。 + +AF_XDP 通过在内核和用户态之间建立共享内存来解决这个问题。内核把报文直接写入你的应用程序可以读取的内存，双方通过无锁的 ring buffer 通信，而不是系统调用。一次 `poll()` 就能唤醒你处理数百个报文。这种架构在普通硬件上就能达到每秒数百万报文的接收速率。 + +理解 AF_XDP 也能让你掌握高性能系统中反复出现的模式：共享内存、无锁数据结构和显式的所有权转移。 + +## AF_XDP 架构 + +AF_XDP 通过四个必须精确协调的组件工作： + +**UMEM（User Memory）** 是你分配的、内核和用户态都能访问的内存区域。你把它划分为固定大小的 frame，通常每个 4096 字节。内核送达的每个报文都会放进这些 frame 之一。 + +**Fill Ring** 是你告诉内核哪些 frame 可用于接收报文的方式。你把 frame 地址发布到这里，内核需要存放报文时就会消费这些地址。 + +**RX Ring** 是内核通知你已接收报文的地方。每个 entry 包含 frame 地址和报文长度。当你在这里看到 entry 时，你就拥有那个 frame 的所有权，直到你归还它。 + +**XSKMAP** 是一个 BPF map，用于连接 XDP 程序和 AF_XDP socket。XDP 程序决定重定向哪些报文，在这个 map 中查找当前 RX queue 对应的 socket，然后调用 `bpf_redirect_map()` 来投递报文。 + +流程是这样的：你向 Fill Ring 发布 64 个 frame 地址。一个 UDP 报文到达。你的 XDP 程序检查目的端口，在 XSKMAP 中查找 socket，然后重定向。内核把报文复制到你的某个 frame 中，并在 RX Ring 上发布一个 descriptor。你读取 descriptor、处理报文，再把这个 frame 地址放回 Fill Ring。循环继续。 + +这个所有权模型至关重要。一个 frame 开始时属于你。你通过 Fill Ring 借给内核。内核借用它来接收报文。你从 RX Ring 回收它。你必须把它归还到 Fill Ring，否则 64 个报文之后就会耗尽 frame。我们的工具通过成功捕获 65 个报文来证明这个机制有效，这需要至少一个 frame 完成完整的所有权周期。 + +## XDP 程序：过滤和重定向 + +内核侧的 BPF 程序很精简，因为它只负责过滤和重定向。buffer 管理的所有复杂性都在用户态。 + +首先，我们定义一个共享头文件来设置 XSKMAP 容量。queue ID 是 map 的 key，所以这个例子可以寻址 0 到 63 号队列： + +```c +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __AFXDP_DUMP_H +#define __AFXDP_DUMP_H + +#define AFXDP_MAX_QUEUES 64 + +#endif /* __AFXDP_DUMP_H */ +``` + +现在是 XDP 程序本身： + +```c +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "afxdp_dump.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff + +const volatile __u16 capture_port = 8080; + +struct { + __uint(type, BPF_MAP_TYPE_XSKMAP); + __uint(max_entries, AFXDP_MAX_QUEUES); + __type(key, __u32); + __type(value, __u32); +} xsk_map SEC(".maps"); + +__u64 redirected_packets; + +SEC("xdp") +int redirect_udp(struct xdp_md *ctx) +{ + void *data = (void *)(long)ctx->data; + void *data_end = (void *)(long)ctx->data_end; + struct ethhdr *ethernet = data; + struct iphdr *ip; + struct udphdr *udp; + __u32 queue = ctx->rx_queue_index; + __u32 ip_header_length; + __u32 ip_length; + __u32 udp_length; + + if ((void *)(ethernet + 1) > data_end || + ethernet->h_proto != bpf_htons(ETH_P_IP)) + return XDP_PASS; + ip = (void *)(ethernet + 1); + if ((void *)(ip + 1) > data_end || ip->version != 4 || + ip->protocol != IPPROTO_UDP || + ip->ihl < 5 || (bpf_ntohs(ip->frag_off) & (IP_MF | IP_OFFSET))) + return XDP_PASS; + ip_header_length = ip->ihl * 4; + ip_length = bpf_ntohs(ip->tot_len); + udp = (void *)ip + ip_header_length; + if (ip_length < ip_header_length + sizeof(*udp) || + (void *)ip + ip_length > data_end || + (void *)(udp + 1) > data_end) + return XDP_PASS; + udp_length = bpf_ntohs(udp->len); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp->dest != bpf_htons(capture_port)) + return XDP_PASS; + if (!bpf_map_lookup_elem(&xsk_map, &queue)) + return XDP_PASS; + __sync_fetch_and_add(&redirected_packets, 1); + return bpf_redirect_map(&xsk_map, queue, XDP_PASS); +} +``` + +程序逐层解析报文，在每一步都验证边界。这种仔细的边界检查是 BPF 验证器的要求，同时也确保我们不会误解截断或畸形的报文。 + +解析从 Ethernet 开始，检查是否有足够空间容纳头部，以及 EtherType 是否表示 IPv4。然后验证 IPv4 头部：正确的版本、UDP 协议、最小头部长度、没有分片（分片报文需要重组，超出了我们的范围）。程序从 IHL 字段计算实际的 IP 头部长度，并用它来定位 UDP 头部。 + +UDP 验证确保长度字段合理，目的端口与目标匹配。只有这时程序才检查这个 queue 是否真的注册了 socket。这个顺序是刻意的：大多数报文会在更早的检查中失败，所以对不会捕获的报文我们避免了 map lookup 的开销。 + +XSKMAP lookup 也是一个安全检查。如果用户态没有为这个 queue 注册 socket，lookup 返回 NULL，我们就把报文传递给普通协议栈。当所有检查都通过时，`bpf_redirect_map()` 把报文发送到 AF_XDP。第二个参数是 queue index，会成为 map 的 key。第三个参数是出错时的回退 action。 + +一个重要细节：一旦报文被重定向，它就被 AF_XDP 消费了。普通的 socket 协议栈永远不会看到它。这符合这个独占式接收器的设计，但也意味着你需要小心选择重定向哪些报文。 + +## 用户态应用程序 + +用户态代码处理 AF_XDP 需要的一切：内存分配、ring 设置、socket 绑定、XDP 加载和接收循环。以下是完整实现： + +```c +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#define _FILE_OFFSET_BITS 64 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "afxdp_dump.h" +#include "afxdp_dump.skel.h" + +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif +#ifndef AF_XDP +#define AF_XDP 44 +#endif +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +#define FRAME_SIZE 4096 +#define FRAME_COUNT 64 +#define RING_SIZE 64 +#define UMEM_SIZE ((size_t)FRAME_SIZE * FRAME_COUNT) +#define PAYLOAD_PREVIEW 32 +#define IPV4_MORE_FRAGMENTS 0x2000 +#define IPV4_FRAGMENT_OFFSET 0x1fff + +struct options { + const char *interface; + unsigned int queue; + unsigned int port; + unsigned int count; + bool skb_mode; +}; + +struct mapped_ring { + void *mapping; + size_t mapping_size; + unsigned int *producer; + unsigned int *consumer; + unsigned int *flags; + void *descriptors; + unsigned int mask; +}; + +struct xsk_state { + int fd; + void *umem; + struct mapped_ring fill; + struct mapped_ring completion; + struct mapped_ring rx; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static void map_ring_fields(struct mapped_ring *ring, void *mapping, + size_t mapping_size, + const struct xdp_ring_offset *offset) +{ + ring->mapping = mapping; + ring->mapping_size = mapping_size; + ring->producer = mapping + offset->producer; + ring->consumer = mapping + offset->consumer; + ring->flags = mapping + offset->flags; + ring->descriptors = mapping + offset->desc; + ring->mask = RING_SIZE - 1; +} + +static int map_xsk_ring(struct mapped_ring *ring, int fd, + const struct xdp_ring_offset *offset, + off_t page_offset, size_t descriptor_size) +{ + size_t size = offset->desc + RING_SIZE * descriptor_size; + void *mapping; + + mapping = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, page_offset); + if (mapping == MAP_FAILED) + return -1; + map_ring_fields(ring, mapping, size, offset); + return 0; +} + +static void close_xsk(struct xsk_state *xsk) +{ + struct mapped_ring *rings[] = { + &xsk->rx, &xsk->completion, &xsk->fill, + }; + + for (size_t i = 0; i < sizeof(rings) / sizeof(rings[0]); i++) + if (rings[i]->mapping && rings[i]->mapping != MAP_FAILED) + munmap(rings[i]->mapping, rings[i]->mapping_size); + if (xsk->fd >= 0) + close(xsk->fd); + if (xsk->umem && xsk->umem != MAP_FAILED) + munmap(xsk->umem, UMEM_SIZE); + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; +} + +static int open_xsk(struct xsk_state *xsk, int ifindex, unsigned int queue) +{ + struct xdp_umem_reg registration = { + .len = UMEM_SIZE, + .chunk_size = FRAME_SIZE, + }; + struct sockaddr_xdp address = { + .sxdp_family = PF_XDP, + .sxdp_ifindex = ifindex, + .sxdp_queue_id = queue, + .sxdp_flags = XDP_COPY, + }; + struct xdp_mmap_offsets offsets; + socklen_t offsets_size = sizeof(offsets); + unsigned int ring_size = RING_SIZE; + unsigned long long *fill_addresses; + + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; + xsk->umem = mmap(NULL, UMEM_SIZE, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0); + if (xsk->umem == MAP_FAILED) + return -1; + xsk->fd = socket(AF_XDP, SOCK_RAW | SOCK_CLOEXEC, 0); + if (xsk->fd < 0) + return -1; + registration.addr = (uintptr_t)xsk->umem; + if (setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_REG, ®istration, + sizeof(registration)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_FILL_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_COMPLETION_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_RX_RING, &ring_size, + sizeof(ring_size)) || + getsockopt(xsk->fd, SOL_XDP, XDP_MMAP_OFFSETS, &offsets, + &offsets_size)) + return -1; + + if (map_xsk_ring(&xsk->fill, xsk->fd, &offsets.fr, + XDP_UMEM_PGOFF_FILL_RING, sizeof(unsigned long long)) || + map_xsk_ring(&xsk->completion, xsk->fd, &offsets.cr, + XDP_UMEM_PGOFF_COMPLETION_RING, + sizeof(unsigned long long)) || + map_xsk_ring(&xsk->rx, xsk->fd, &offsets.rx, + XDP_PGOFF_RX_RING, sizeof(struct xdp_desc)) || + bind(xsk->fd, (struct sockaddr *)&address, sizeof(address))) + return -1; + + fill_addresses = xsk->fill.descriptors; + for (unsigned int i = 0; i < FRAME_COUNT; i++) + fill_addresses[i & xsk->fill.mask] = + (unsigned long long)i * FRAME_SIZE; + __atomic_store_n(xsk->fill.producer, FRAME_COUNT, __ATOMIC_RELEASE); + return 0; +} + +static unsigned long long data_address(unsigned long long address) +{ + return (address & XSK_UNALIGNED_BUF_ADDR_MASK) + + (address >> XSK_UNALIGNED_BUF_OFFSET_SHIFT); +} + +static int recycle_frame(struct xsk_state *xsk, unsigned long long address) +{ + unsigned long long *fill_addresses = xsk->fill.descriptors; + unsigned int producer, consumer; + + producer = __atomic_load_n(xsk->fill.producer, __ATOMIC_RELAXED); + consumer = __atomic_load_n(xsk->fill.consumer, __ATOMIC_ACQUIRE); + if (producer - consumer == RING_SIZE) + return -1; + fill_addresses[producer & xsk->fill.mask] = address; + __atomic_store_n(xsk->fill.producer, producer + 1, __ATOMIC_RELEASE); + return 0; +} + +static const struct iphdr *parse_ipv4(const void *packet, unsigned int length, + unsigned int *ip_header_length, + unsigned int *ip_length) +{ + const struct ethhdr *ethernet = packet; + const struct iphdr *ip; + unsigned int available_ip; + + if (length < sizeof(*ethernet) + sizeof(*ip) || + ethernet->h_proto != htons(ETH_P_IP)) + return NULL; + ip = packet + sizeof(*ethernet); + *ip_header_length = ip->ihl * 4; + if (ip->version != 4 || ip->protocol != IPPROTO_UDP || + (ntohs(ip->frag_off) & + (IPV4_MORE_FRAGMENTS | IPV4_FRAGMENT_OFFSET)) || + *ip_header_length < sizeof(*ip) || + length < sizeof(*ethernet) + *ip_header_length) + return NULL; + *ip_length = ntohs(ip->tot_len); + available_ip = length - sizeof(*ethernet); + if (*ip_length < *ip_header_length + sizeof(struct udphdr) || + *ip_length > available_ip) + return NULL; + return ip; +} + +static const struct udphdr *parse_udp(const void *packet, + unsigned int length, + unsigned int ip_header_length, + unsigned int ip_length, + const unsigned char **payload, + unsigned int *payload_length) +{ + const struct udphdr *udp = packet + sizeof(struct ethhdr) + + ip_header_length; + unsigned int available_payload; + unsigned int udp_length; + + *payload = (const unsigned char *)(udp + 1); + udp_length = ntohs(udp->len); + available_payload = length - (*payload - (const unsigned char *)packet); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp_length - sizeof(*udp) > available_payload) + return NULL; + *payload_length = udp_length - sizeof(*udp); + return udp; +} + +static void dump_packet(const void *packet, unsigned int length, + unsigned int packet_number) +{ + const struct iphdr *ip; + const struct udphdr *udp; + const unsigned char *payload; + char source[INET_ADDRSTRLEN], destination[INET_ADDRSTRLEN]; + char preview[PAYLOAD_PREVIEW + 1]; + unsigned int ip_header_length, ip_length, payload_length, preview_length; + + ip = parse_ipv4(packet, length, &ip_header_length, &ip_length); + if (!ip) + return; + udp = parse_udp(packet, length, ip_header_length, ip_length, &payload, + &payload_length); + if (!udp) + return; + preview_length = payload_length < PAYLOAD_PREVIEW ? + payload_length : PAYLOAD_PREVIEW; + for (unsigned int i = 0; i < preview_length; i++) + preview[i] = isprint(payload[i]) ? payload[i] : '.'; + preview[preview_length] = '\0'; + inet_ntop(AF_INET, &ip->saddr, source, sizeof(source)); + inet_ntop(AF_INET, &ip->daddr, destination, sizeof(destination)); + printf("packet=%u %s:%u -> %s:%u bytes=%u payload=\"%s\"\n", + packet_number, source, ntohs(udp->source), destination, + ntohs(udp->dest), length, preview); +} + +static int receive_packets(struct xsk_state *xsk, unsigned int count) +{ + struct pollfd poll_fd = { .fd = xsk->fd, .events = POLLIN }; + unsigned int received = 0; + + while (!stop && (!count || received < count)) { + unsigned int consumer, producer; + int poll_result = poll(&poll_fd, 1, 250); + + if (poll_result < 0) { + if (errno == EINTR) + continue; + return -1; + } + if (!poll_result) + continue; + consumer = __atomic_load_n(xsk->rx.consumer, __ATOMIC_RELAXED); + producer = __atomic_load_n(xsk->rx.producer, __ATOMIC_ACQUIRE); + while (consumer != producer && (!count || received < count)) { + struct xdp_desc *descriptors = xsk->rx.descriptors; + struct xdp_desc descriptor = + descriptors[consumer & xsk->rx.mask]; + unsigned long long address = data_address(descriptor.addr); + + if (descriptor.options & XDP_PKT_CONTD) { + errno = EMSGSIZE; + return -1; + } + if (address + descriptor.len > UMEM_SIZE) { + errno = EFAULT; + return -1; + } + received++; + dump_packet(xsk->umem + address, descriptor.len, received); + consumer++; + __atomic_store_n(xsk->rx.consumer, consumer, + __ATOMIC_RELEASE); + if (recycle_frame(xsk, descriptor.addr)) { + errno = ENOBUFS; + return -1; + } + } + } + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--queue N] [--port PORT] [--count N] [--skb-mode]\n", + program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "queue", required_argument, NULL, 'q' }, + { "port", required_argument, NULL, 'p' }, + { "count", required_argument, NULL, 'n' }, + { "skb-mode", no_argument, NULL, 'S' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:q:p:n:Sh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'q': + if (parse_uint(optarg, AFXDP_MAX_QUEUES - 1, + &options->queue)) + return -1; + break; + case 'p': + if (parse_uint(optarg, 65535, &options->port) || + !options->port) + return -1; + break; + case 'n': + if (parse_uint(optarg, 1000000, &options->count)) + return -1; + break; + case 'S': options->skb_mode = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + return optind == argc && options->interface ? 0 : -1; +} + +static int attach_xdp(int ifindex, int program_fd, bool skb_mode, + unsigned int *attached_flags) +{ + unsigned int flags = XDP_FLAGS_UPDATE_IF_NOEXIST | + (skb_mode ? XDP_FLAGS_SKB_MODE : XDP_FLAGS_DRV_MODE); + int err; + + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + if (err && !skb_mode && (err == -EOPNOTSUPP || err == -EINVAL)) { + flags = XDP_FLAGS_UPDATE_IF_NOEXIST | XDP_FLAGS_SKB_MODE; + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + } + if (!err) + *attached_flags = flags; + return err; +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 8080 }; + struct afxdp_dump_bpf *skel = NULL; + struct xsk_state xsk; + unsigned int attached_flags = 0; + int ifindex; + int err = 1; + bool attached = false; + + setvbuf(stdout, NULL, _IONBF, 0); + memset(&xsk, 0, sizeof(xsk)); + xsk.fd = -1; + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + ifindex = if_nametoindex(options.interface); + if (!ifindex) { + fprintf(stderr, "unknown interface: %s\n", options.interface); + return 2; + } + if (open_xsk(&xsk, ifindex, options.queue)) { + fprintf(stderr, "failed to create AF_XDP socket: %s\n", + strerror(errno)); + goto cleanup; + } + skel = afxdp_dump_bpf__open(); + if (!skel) + goto cleanup; + skel->rodata->capture_port = options.port; + if (afxdp_dump_bpf__load(skel)) { + fprintf(stderr, "failed to load XDP program\n"); + goto cleanup; + } + if (bpf_map__update_elem(skel->maps.xsk_map, &options.queue, + sizeof(options.queue), &xsk.fd, sizeof(xsk.fd), + BPF_ANY)) { + fprintf(stderr, "failed to populate XSK map: %s\n", strerror(errno)); + goto cleanup; + } + { + int attach_error = attach_xdp(ifindex, + bpf_program__fd(skel->progs.redirect_udp), + options.skb_mode, &attached_flags); + + if (attach_error) { + fprintf(stderr, "failed to attach XDP program: %s\n", + strerror(-attach_error)); + goto cleanup; + } + } + attached = true; + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + printf("afxdp-dump ready interface=%s queue=%u port=%u mode=%s count=%u\n", + options.interface, options.queue, options.port, + attached_flags & XDP_FLAGS_SKB_MODE ? "skb" : "driver", + options.count); + + if (receive_packets(&xsk, options.count)) { + fprintf(stderr, "AF_XDP receive failed: %s\n", strerror(errno)); + goto cleanup; + } + printf("redirected=%llu\n", + (unsigned long long)skel->bss->redirected_packets); + err = 0; + +cleanup: + if (attached) { + LIBBPF_OPTS(bpf_xdp_attach_opts, detach_options, + .old_prog_fd = bpf_program__fd( + skel->progs.redirect_udp)); + int detach_error = bpf_xdp_detach( + ifindex, attached_flags & XDP_FLAGS_MODES, + &detach_options); + + if (detach_error) { + fprintf(stderr, "failed to detach XDP program: %s\n", + strerror(-detach_error)); + err = 1; + } + } + afxdp_dump_bpf__destroy(skel); + close_xsk(&xsk); + return err; +} +``` + +### 理解 UMEM 和 Ring 设置 + +`open_xsk()` 函数创建 AF_XDP socket 及其所有支撑基础设施。它首先映射 256 KiB 的匿名内存作为 UMEM，划分为 64 个 4096 字节的 frame。这块内存在注册后会与内核共享。 + +用 `AF_XDP` 创建 socket 后，它通过 `XDP_UMEM_REG` 注册 UMEM，告诉内核我们的报文 buffer 在哪里。然后用 `setsockopt` 请求各 64 个 entry 的 fill、completion 和 RX ring。`XDP_MMAP_OFFSETS` getsockopt 揭示了每个 ring 的 producer index、consumer index、flags 和 descriptor 数组在可 mmap 的页面中的位置。 + +Ring 映射有些技巧，因为内核把每个 ring 放在固定的页面偏移。Fill Ring 在 `XDP_UMEM_PGOFF_FILL_RING`，Completion Ring 在 `XDP_UMEM_PGOFF_COMPLETION_RING`，RX Ring 在 `XDP_PGOFF_RX_RING`。每个 ring 包含指向 producer 和 consumer index 的指针，它们是内核和用户态之间的同步点。 + +bind 之前的最后一步是把所有 64 个 frame 地址发布到 Fill Ring。我们把每个地址写入 descriptor 数组，然后用 release 语义发布 producer index。这告诉内核有 64 个 frame 可用于接收报文。 + +### Ring 操作中的内存顺序 + +Ring index 由内核和用户态共享，这使得内存顺序至关重要。模式是一致的：producer 在用 release 语义发布 index 之前先写 descriptor，consumer 在读取 descriptor 之前先 acquire producer index。 + +在 `recycle_frame()` 中，我们用 relaxed 顺序读取自己的 producer index（只有我们修改它），但要 acquire consumer index，因为内核会写它。如果 ring 有空间，我们写入地址并用 release 发布。内核最终会 acquire 我们的 producer 更新并看到我们写的地址。 + +同样的模式出现在接收循环中。我们 acquire 内核写的 producer index，读取 descriptor，用 release 发布我们的 consumer 更新。这确保内核知道我们已经完成这些 frame，然后才能复用它们。 + +### Copy Mode 与 Zero-Copy + +Socket 地址中的 `XDP_COPY` 标志告诉 AF_XDP 把报文复制到 UMEM，而不是真正的 zero-copy。Copy mode 在任何接口上都能工作，不需要驱动支持，是学习示例的正确选择。内核为传入报文分配自己的内存，然后复制到我们的 UMEM frame。 + +XDP 程序的 attach mode 是另一回事。我们首先尝试 native driver mode（`XDP_FLAGS_DRV_MODE`），它在 SKB 分配之前在驱动中运行 XDP 程序。如果因为驱动不支持 XDP 而失败，我们回退到 generic SKB mode（`XDP_FLAGS_SKB_MODE`），它在 SKB 创建之后运行，但仍然让我们重定向到 AF_XDP。`--skb-mode` 标志直接强制使用 generic mode。 + +### 接收循环 + +接收循环轮询报文，批量处理它们，并立即回收 frame。250ms 超时确保即使没有报文到达，我们也能响应信号。 + +当 poll 指示有数据时，我们检查 RX ring 是否有新 descriptor。对于每一个，我们计算实际的数据地址（descriptor 地址可能编码了 offset 信息），验证它在 UMEM 边界内，并检查这不是一个 multi-buffer 报文（那会设置 `XDP_PKT_CONTD`）。Multi-buffer 支持需要累积分片，超出了这个例子的范围。 + +打印报文后，我们立即通过 `recycle_frame()` 把 frame 归还到 Fill Ring。这至关重要：不回收的话，64 个报文之后就会耗尽 frame。工具通过成功捕获 65 个报文来证明回收有效。 + +### 清理和安全卸载 + +退出时，我们使用 compare-and-detach 只移除我们自己的 XDP 程序。`old_prog_fd` 选项告诉 `bpf_xdp_detach` 只在当前挂载的程序与我们的匹配时才卸载。这防止了在我们运行期间如果有人挂载了其他 XDP 程序时，意外卸载别人的程序。 + +## 编译和运行 + +构建可执行文件： + +```bash +cd src/60-afxdp-dump +make +``` + +捕获 queue 0 上发往 UDP 8080 端口的 5 个报文： + +```bash +sudo ./afxdp_dump --interface eth0 --queue 0 --port 8080 --count 5 +``` + +使用 `--count 0` 持续运行直到 Ctrl+C 中断。添加 `--skb-mode` 强制使用 generic XDP mode。选中的 queue 必须实际接收到流量；在多队列网卡上，这取决于 RSS（receive-side scaling）配置。 + +从另一台机器或另一个终端发送测试流量： + +```bash +echo "hello-afxdp" | nc -u target-ip 8080 +``` + +使用 `--count 65` 运行更长时间可以证明 frame 回收有效： + +```text +afxdp-dump ready interface=eth0 queue=0 port=8080 mode=driver count=65 +packet=1 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp" +packet=65 10.77.0.1:60414 -> 10.77.0.2:8080 bytes=53 payload="hello-afxdp" +redirected=65 +``` + +Packet 65 证明至少有一个 frame 完成了完整的所有权周期：发布到 Fill，用于接收，从 RX 消费，再发布回 Fill。`redirected=65` 计数器来自 BPF 程序，与我们的接收计数一致。 + +## 环境要求 + +| 要求 | 说明 | +|---|---| +| 内核 | Linux 5.7 或更高版本。AF_XDP 在 4.18 引入，XDP 对 XSKMAP 的 lookup 在 5.3 引入，安全的 expected-FD detach 在 5.7 引入 | +| 内核配置 | `CONFIG_BPF`、`CONFIG_BPF_SYSCALL`、`CONFIG_BPF_JIT`、`CONFIG_XDP_SOCKETS`、`CONFIG_DEBUG_INFO_BTF` | +| 权限 | root 或等价的 BPF 与网络管理 capability | +| 网络接口 | 任何包含所选 RX queue 的接口。native XDP 可选，generic mode 处处可用 | +| 架构 | x86-64 已测试。copy mode 不需要驱动 zero-copy 支持 | + +## 后续扩展 + +这个例子有意做得最小化：只接收、单队列、single-buffer 报文、copy mode。真正的生产 AF_XDP 应用会在这个基础上向多个方向扩展： + +**TX ring** 让你用同样的零开销模型发送报文。你需要添加 TX ring，发布带有报文数据的 frame 地址，然后轮询 completion 通知。 + +**Zero-copy mode** 消除了复制到 UMEM 的过程。驱动直接使用你的 UMEM，但这需要驱动支持和仔细的 buffer 对齐。 + +**Multi-buffer 报文** 处理巨型帧或 UMEM frame 小于 MTU 的情况。你需要累积标记了 `XDP_PKT_CONTD` 的分片，直到最后一个分片。 + +**Shared UMEM** 让多个 socket 共享同一块内存，对于跨 queue 负载均衡或 RX 和 TX 路径之间共享很有用。 + +## 总结 + +AF_XDP 让报文接收绕过普通网络协议栈，同时保留 eBPF 的安全检查。XDP 程序在驱动边界选择流量，XSKMAP 把报文路由到你的 socket，无锁的 ring buffer 不用系统调用就能传输数据。这个例子展示了完整的接收契约：向 Fill 发布 frame，在 RX 上接收 descriptor，处理报文，把 frame 回收到 Fill。 + +理解这个流程的价值超越了报文接收本身。这里的模式包括内核与用户态之间的共享内存、显式的所有权转移和无锁同步，这些模式在从数据库到 GPU 驱动的各种高性能系统中反复出现。 + +> 如果你想深入了解 eBPF，请查看我们的教程代码仓库 或访问我们的网站 。 + +## 参考资料 + +- [Linux AF_XDP 文档](https://docs.kernel.org/networking/af_xdp.html) +- [AF_XDP 引入 commit](https://github.com/torvalds/linux/commit/c0c77d8fb787cfe0c3fca689c2a30d1dad4eaba7) +- [XDP 支持 XSKMAP lookup 的 commit](https://github.com/torvalds/linux/commit/fada7fdc83c0) +- [XDP detach 的 expected-program FD](https://github.com/torvalds/linux/commit/92234c8f15c8d96ad7e52afdc5994cba6be68eb9) +- [libxdp 库](https://github.com/xdp-project/xdp-tools) - 如果想跳过原始 ABI，这里有更高层的 AF_XDP 辅助函数 diff --git a/src/60-afxdp-dump/afxdp_dump.bpf.c b/src/60-afxdp-dump/afxdp_dump.bpf.c new file mode 100644 index 00000000..1ad73086 --- /dev/null +++ b/src/60-afxdp-dump/afxdp_dump.bpf.c @@ -0,0 +1,62 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "vmlinux.h" +#include +#include +#include "afxdp_dump.h" + +char LICENSE[] SEC("license") = "GPL"; + +#define ETH_P_IP 0x0800 +#define IPPROTO_UDP 17 +#define IP_MF 0x2000 +#define IP_OFFSET 0x1fff + +const volatile __u16 capture_port = 8080; + +struct { + __uint(type, BPF_MAP_TYPE_XSKMAP); + __uint(max_entries, AFXDP_MAX_QUEUES); + __type(key, __u32); + __type(value, __u32); +} xsk_map SEC(".maps"); + +__u64 redirected_packets; + +SEC("xdp") +int redirect_udp(struct xdp_md *ctx) +{ + void *data = (void *)(long)ctx->data; + void *data_end = (void *)(long)ctx->data_end; + struct ethhdr *ethernet = data; + struct iphdr *ip; + struct udphdr *udp; + __u32 queue = ctx->rx_queue_index; + __u32 ip_header_length; + __u32 ip_length; + __u32 udp_length; + + if ((void *)(ethernet + 1) > data_end || + ethernet->h_proto != bpf_htons(ETH_P_IP)) + return XDP_PASS; + ip = (void *)(ethernet + 1); + if ((void *)(ip + 1) > data_end || ip->version != 4 || + ip->protocol != IPPROTO_UDP || + ip->ihl < 5 || (bpf_ntohs(ip->frag_off) & (IP_MF | IP_OFFSET))) + return XDP_PASS; + ip_header_length = ip->ihl * 4; + ip_length = bpf_ntohs(ip->tot_len); + udp = (void *)ip + ip_header_length; + if (ip_length < ip_header_length + sizeof(*udp) || + (void *)ip + ip_length > data_end || + (void *)(udp + 1) > data_end) + return XDP_PASS; + udp_length = bpf_ntohs(udp->len); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp->dest != bpf_htons(capture_port)) + return XDP_PASS; + if (!bpf_map_lookup_elem(&xsk_map, &queue)) + return XDP_PASS; + __sync_fetch_and_add(&redirected_packets, 1); + return bpf_redirect_map(&xsk_map, queue, XDP_PASS); +} diff --git a/src/60-afxdp-dump/afxdp_dump.c b/src/60-afxdp-dump/afxdp_dump.c new file mode 100644 index 00000000..f4b1427c --- /dev/null +++ b/src/60-afxdp-dump/afxdp_dump.c @@ -0,0 +1,489 @@ +// SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) +#define _FILE_OFFSET_BITS 64 +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include "afxdp_dump.h" +#include "afxdp_dump.skel.h" + +#ifndef SOL_XDP +#define SOL_XDP 283 +#endif +#ifndef AF_XDP +#define AF_XDP 44 +#endif +#ifndef PF_XDP +#define PF_XDP AF_XDP +#endif + +#define FRAME_SIZE 4096 +#define FRAME_COUNT 64 +#define RING_SIZE 64 +#define UMEM_SIZE ((size_t)FRAME_SIZE * FRAME_COUNT) +#define PAYLOAD_PREVIEW 32 +#define IPV4_MORE_FRAGMENTS 0x2000 +#define IPV4_FRAGMENT_OFFSET 0x1fff + +struct options { + const char *interface; + unsigned int queue; + unsigned int port; + unsigned int count; + bool skb_mode; +}; + +struct mapped_ring { + void *mapping; + size_t mapping_size; + unsigned int *producer; + unsigned int *consumer; + unsigned int *flags; + void *descriptors; + unsigned int mask; +}; + +struct xsk_state { + int fd; + void *umem; + struct mapped_ring fill; + struct mapped_ring completion; + struct mapped_ring rx; +}; + +static volatile sig_atomic_t stop; + +static void handle_signal(int signal_number) +{ + (void)signal_number; + stop = 1; +} + +static void map_ring_fields(struct mapped_ring *ring, void *mapping, + size_t mapping_size, + const struct xdp_ring_offset *offset) +{ + ring->mapping = mapping; + ring->mapping_size = mapping_size; + ring->producer = mapping + offset->producer; + ring->consumer = mapping + offset->consumer; + ring->flags = mapping + offset->flags; + ring->descriptors = mapping + offset->desc; + ring->mask = RING_SIZE - 1; +} + +static int map_xsk_ring(struct mapped_ring *ring, int fd, + const struct xdp_ring_offset *offset, + off_t page_offset, size_t descriptor_size) +{ + size_t size = offset->desc + RING_SIZE * descriptor_size; + void *mapping; + + mapping = mmap(NULL, size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_POPULATE, fd, page_offset); + if (mapping == MAP_FAILED) + return -1; + map_ring_fields(ring, mapping, size, offset); + return 0; +} + +static void close_xsk(struct xsk_state *xsk) +{ + struct mapped_ring *rings[] = { + &xsk->rx, &xsk->completion, &xsk->fill, + }; + + for (size_t i = 0; i < sizeof(rings) / sizeof(rings[0]); i++) + if (rings[i]->mapping && rings[i]->mapping != MAP_FAILED) + munmap(rings[i]->mapping, rings[i]->mapping_size); + if (xsk->fd >= 0) + close(xsk->fd); + if (xsk->umem && xsk->umem != MAP_FAILED) + munmap(xsk->umem, UMEM_SIZE); + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; +} + +static int open_xsk(struct xsk_state *xsk, int ifindex, unsigned int queue) +{ + struct xdp_umem_reg registration = { + .len = UMEM_SIZE, + .chunk_size = FRAME_SIZE, + }; + struct sockaddr_xdp address = { + .sxdp_family = PF_XDP, + .sxdp_ifindex = ifindex, + .sxdp_queue_id = queue, + .sxdp_flags = XDP_COPY, + }; + struct xdp_mmap_offsets offsets; + socklen_t offsets_size = sizeof(offsets); + unsigned int ring_size = RING_SIZE; + unsigned long long *fill_addresses; + + memset(xsk, 0, sizeof(*xsk)); + xsk->fd = -1; + xsk->umem = mmap(NULL, UMEM_SIZE, PROT_READ | PROT_WRITE, + MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0); + if (xsk->umem == MAP_FAILED) + return -1; + xsk->fd = socket(AF_XDP, SOCK_RAW | SOCK_CLOEXEC, 0); + if (xsk->fd < 0) + return -1; + registration.addr = (uintptr_t)xsk->umem; + if (setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_REG, ®istration, + sizeof(registration)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_FILL_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_UMEM_COMPLETION_RING, &ring_size, + sizeof(ring_size)) || + setsockopt(xsk->fd, SOL_XDP, XDP_RX_RING, &ring_size, + sizeof(ring_size)) || + getsockopt(xsk->fd, SOL_XDP, XDP_MMAP_OFFSETS, &offsets, + &offsets_size)) + return -1; + + if (map_xsk_ring(&xsk->fill, xsk->fd, &offsets.fr, + XDP_UMEM_PGOFF_FILL_RING, sizeof(unsigned long long)) || + map_xsk_ring(&xsk->completion, xsk->fd, &offsets.cr, + XDP_UMEM_PGOFF_COMPLETION_RING, + sizeof(unsigned long long)) || + map_xsk_ring(&xsk->rx, xsk->fd, &offsets.rx, + XDP_PGOFF_RX_RING, sizeof(struct xdp_desc)) || + bind(xsk->fd, (struct sockaddr *)&address, sizeof(address))) + return -1; + + fill_addresses = xsk->fill.descriptors; + for (unsigned int i = 0; i < FRAME_COUNT; i++) + fill_addresses[i & xsk->fill.mask] = + (unsigned long long)i * FRAME_SIZE; + __atomic_store_n(xsk->fill.producer, FRAME_COUNT, __ATOMIC_RELEASE); + return 0; +} + +static unsigned long long data_address(unsigned long long address) +{ + return (address & XSK_UNALIGNED_BUF_ADDR_MASK) + + (address >> XSK_UNALIGNED_BUF_OFFSET_SHIFT); +} + +static int recycle_frame(struct xsk_state *xsk, unsigned long long address) +{ + unsigned long long *fill_addresses = xsk->fill.descriptors; + unsigned int producer, consumer; + + producer = __atomic_load_n(xsk->fill.producer, __ATOMIC_RELAXED); + consumer = __atomic_load_n(xsk->fill.consumer, __ATOMIC_ACQUIRE); + if (producer - consumer == RING_SIZE) + return -1; + fill_addresses[producer & xsk->fill.mask] = address; + __atomic_store_n(xsk->fill.producer, producer + 1, __ATOMIC_RELEASE); + return 0; +} + +static const struct iphdr *parse_ipv4(const void *packet, unsigned int length, + unsigned int *ip_header_length, + unsigned int *ip_length) +{ + const struct ethhdr *ethernet = packet; + const struct iphdr *ip; + unsigned int available_ip; + + if (length < sizeof(*ethernet) + sizeof(*ip) || + ethernet->h_proto != htons(ETH_P_IP)) + return NULL; + ip = packet + sizeof(*ethernet); + *ip_header_length = ip->ihl * 4; + if (ip->version != 4 || ip->protocol != IPPROTO_UDP || + (ntohs(ip->frag_off) & + (IPV4_MORE_FRAGMENTS | IPV4_FRAGMENT_OFFSET)) || + *ip_header_length < sizeof(*ip) || + length < sizeof(*ethernet) + *ip_header_length) + return NULL; + *ip_length = ntohs(ip->tot_len); + available_ip = length - sizeof(*ethernet); + if (*ip_length < *ip_header_length + sizeof(struct udphdr) || + *ip_length > available_ip) + return NULL; + return ip; +} + +static const struct udphdr *parse_udp(const void *packet, + unsigned int length, + unsigned int ip_header_length, + unsigned int ip_length, + const unsigned char **payload, + unsigned int *payload_length) +{ + const struct udphdr *udp = packet + sizeof(struct ethhdr) + + ip_header_length; + unsigned int available_payload; + unsigned int udp_length; + + *payload = (const unsigned char *)(udp + 1); + udp_length = ntohs(udp->len); + available_payload = length - (*payload - (const unsigned char *)packet); + if (udp_length < sizeof(*udp) || + udp_length > ip_length - ip_header_length || + udp_length - sizeof(*udp) > available_payload) + return NULL; + *payload_length = udp_length - sizeof(*udp); + return udp; +} + +static void dump_packet(const void *packet, unsigned int length, + unsigned int packet_number) +{ + const struct iphdr *ip; + const struct udphdr *udp; + const unsigned char *payload; + char source[INET_ADDRSTRLEN], destination[INET_ADDRSTRLEN]; + char preview[PAYLOAD_PREVIEW + 1]; + unsigned int ip_header_length, ip_length, payload_length, preview_length; + + ip = parse_ipv4(packet, length, &ip_header_length, &ip_length); + if (!ip) + return; + udp = parse_udp(packet, length, ip_header_length, ip_length, &payload, + &payload_length); + if (!udp) + return; + preview_length = payload_length < PAYLOAD_PREVIEW ? + payload_length : PAYLOAD_PREVIEW; + for (unsigned int i = 0; i < preview_length; i++) + preview[i] = isprint(payload[i]) ? payload[i] : '.'; + preview[preview_length] = '\0'; + inet_ntop(AF_INET, &ip->saddr, source, sizeof(source)); + inet_ntop(AF_INET, &ip->daddr, destination, sizeof(destination)); + printf("packet=%u %s:%u -> %s:%u bytes=%u payload=\"%s\"\n", + packet_number, source, ntohs(udp->source), destination, + ntohs(udp->dest), length, preview); +} + +static int receive_packets(struct xsk_state *xsk, unsigned int count) +{ + struct pollfd poll_fd = { .fd = xsk->fd, .events = POLLIN }; + unsigned int received = 0; + + while (!stop && (!count || received < count)) { + unsigned int consumer, producer; + int poll_result = poll(&poll_fd, 1, 250); + + if (poll_result < 0) { + if (errno == EINTR) + continue; + return -1; + } + if (!poll_result) + continue; + consumer = __atomic_load_n(xsk->rx.consumer, __ATOMIC_RELAXED); + producer = __atomic_load_n(xsk->rx.producer, __ATOMIC_ACQUIRE); + while (consumer != producer && (!count || received < count)) { + struct xdp_desc *descriptors = xsk->rx.descriptors; + struct xdp_desc descriptor = + descriptors[consumer & xsk->rx.mask]; + unsigned long long address = data_address(descriptor.addr); + + if (descriptor.options & XDP_PKT_CONTD) { + errno = EMSGSIZE; + return -1; + } + if (address + descriptor.len > UMEM_SIZE) { + errno = EFAULT; + return -1; + } + received++; + dump_packet(xsk->umem + address, descriptor.len, received); + consumer++; + __atomic_store_n(xsk->rx.consumer, consumer, + __ATOMIC_RELEASE); + if (recycle_frame(xsk, descriptor.addr)) { + errno = ENOBUFS; + return -1; + } + } + } + return 0; +} + +static int parse_uint(const char *text, unsigned int maximum, + unsigned int *value) +{ + char *end = NULL; + unsigned long parsed; + + errno = 0; + parsed = strtoul(text, &end, 10); + if (errno || !*text || *end || parsed > maximum) + return -1; + *value = parsed; + return 0; +} + +static void usage(const char *program) +{ + printf("Usage: %s --interface IFACE [--queue N] [--port PORT] [--count N] [--skb-mode]\n", + program); +} + +static int parse_options(int argc, char **argv, struct options *options) +{ + static const struct option long_options[] = { + { "interface", required_argument, NULL, 'i' }, + { "queue", required_argument, NULL, 'q' }, + { "port", required_argument, NULL, 'p' }, + { "count", required_argument, NULL, 'n' }, + { "skb-mode", no_argument, NULL, 'S' }, + { "help", no_argument, NULL, 'h' }, + {}, + }; + int option; + + while ((option = getopt_long(argc, argv, "i:q:p:n:Sh", long_options, + NULL)) != -1) { + switch (option) { + case 'i': options->interface = optarg; break; + case 'q': + if (parse_uint(optarg, AFXDP_MAX_QUEUES - 1, + &options->queue)) + return -1; + break; + case 'p': + if (parse_uint(optarg, 65535, &options->port) || + !options->port) + return -1; + break; + case 'n': + if (parse_uint(optarg, 1000000, &options->count)) + return -1; + break; + case 'S': options->skb_mode = true; break; + case 'h': usage(argv[0]); exit(0); + default: return -1; + } + } + return optind == argc && options->interface ? 0 : -1; +} + +static int attach_xdp(int ifindex, int program_fd, bool skb_mode, + unsigned int *attached_flags) +{ + unsigned int flags = XDP_FLAGS_UPDATE_IF_NOEXIST | + (skb_mode ? XDP_FLAGS_SKB_MODE : XDP_FLAGS_DRV_MODE); + int err; + + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + if (err && !skb_mode && (err == -EOPNOTSUPP || err == -EINVAL)) { + flags = XDP_FLAGS_UPDATE_IF_NOEXIST | XDP_FLAGS_SKB_MODE; + err = bpf_xdp_attach(ifindex, program_fd, flags, NULL); + } + if (!err) + *attached_flags = flags; + return err; +} + +int main(int argc, char **argv) +{ + struct options options = { .port = 8080 }; + struct afxdp_dump_bpf *skel = NULL; + struct xsk_state xsk; + unsigned int attached_flags = 0; + int ifindex; + int err = 1; + bool attached = false; + + setvbuf(stdout, NULL, _IONBF, 0); + memset(&xsk, 0, sizeof(xsk)); + xsk.fd = -1; + if (parse_options(argc, argv, &options)) { + usage(argv[0]); + return 2; + } + ifindex = if_nametoindex(options.interface); + if (!ifindex) { + fprintf(stderr, "unknown interface: %s\n", options.interface); + return 2; + } + if (open_xsk(&xsk, ifindex, options.queue)) { + fprintf(stderr, "failed to create AF_XDP socket: %s\n", + strerror(errno)); + goto cleanup; + } + skel = afxdp_dump_bpf__open(); + if (!skel) + goto cleanup; + skel->rodata->capture_port = options.port; + if (afxdp_dump_bpf__load(skel)) { + fprintf(stderr, "failed to load XDP program\n"); + goto cleanup; + } + if (bpf_map__update_elem(skel->maps.xsk_map, &options.queue, + sizeof(options.queue), &xsk.fd, sizeof(xsk.fd), + BPF_ANY)) { + fprintf(stderr, "failed to populate XSK map: %s\n", strerror(errno)); + goto cleanup; + } + { + int attach_error = attach_xdp(ifindex, + bpf_program__fd(skel->progs.redirect_udp), + options.skb_mode, &attached_flags); + + if (attach_error) { + fprintf(stderr, "failed to attach XDP program: %s\n", + strerror(-attach_error)); + goto cleanup; + } + } + attached = true; + signal(SIGINT, handle_signal); + signal(SIGTERM, handle_signal); + printf("afxdp-dump ready interface=%s queue=%u port=%u mode=%s count=%u\n", + options.interface, options.queue, options.port, + attached_flags & XDP_FLAGS_SKB_MODE ? "skb" : "driver", + options.count); + + if (receive_packets(&xsk, options.count)) { + fprintf(stderr, "AF_XDP receive failed: %s\n", strerror(errno)); + goto cleanup; + } + printf("redirected=%llu\n", + (unsigned long long)skel->bss->redirected_packets); + err = 0; + +cleanup: + if (attached) { + LIBBPF_OPTS(bpf_xdp_attach_opts, detach_options, + .old_prog_fd = bpf_program__fd( + skel->progs.redirect_udp)); + int detach_error = bpf_xdp_detach( + ifindex, attached_flags & XDP_FLAGS_MODES, + &detach_options); + + if (detach_error) { + fprintf(stderr, "failed to detach XDP program: %s\n", + strerror(-detach_error)); + err = 1; + } + } + afxdp_dump_bpf__destroy(skel); + close_xsk(&xsk); + return err; +} diff --git a/src/60-afxdp-dump/afxdp_dump.h b/src/60-afxdp-dump/afxdp_dump.h new file mode 100644 index 00000000..d5a20a6a --- /dev/null +++ b/src/60-afxdp-dump/afxdp_dump.h @@ -0,0 +1,7 @@ +/* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ +#ifndef __AFXDP_DUMP_H +#define __AFXDP_DUMP_H + +#define AFXDP_MAX_QUEUES 64 + +#endif /* __AFXDP_DUMP_H */ diff --git a/src/60-afxdp-dump/tests/test_afxdp_dump.py b/src/60-afxdp-dump/tests/test_afxdp_dump.py new file mode 100644 index 00000000..a9c8f98b --- /dev/null +++ b/src/60-afxdp-dump/tests/test_afxdp_dump.py @@ -0,0 +1,118 @@ +#!/usr/bin/env python3 +import os +import pathlib +import select +import socket +import subprocess +import sys + + +SUFFIX = os.getpid() % 10000 +RX = f"axdp{SUFFIX}r" +TX = f"axdp{SUFFIX}t" +NETNS = f"afxdp-tx-ns-{os.getpid()}" + + +def run(*args, check=True): + return subprocess.run(args, text=True, capture_output=True, check=check) + + +def cleanup(): + run("ip", "link", "del", RX, check=False) + run("ip", "netns", "del", NETNS, check=False) + + +def setup(): + cleanup() + run("ip", "netns", "add", NETNS) + run("ip", "link", "add", RX, "type", "veth", "peer", "name", TX) + run("ip", "link", "set", TX, "netns", NETNS) + run("ip", "addr", "add", "10.77.0.2/24", "dev", RX) + run("ip", "link", "set", RX, "up") + run( + "ip", "netns", "exec", NETNS, + "ip", "addr", "add", "10.77.0.1/24", "dev", TX, + ) + run("ip", "netns", "exec", NETNS, "ip", "link", "set", TX, "up") + run("ip", "netns", "exec", NETNS, "ip", "link", "set", "lo", "up") + + +def send_packets(count): + code = ( + "import socket,time; " + "s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM); " + f"[(s.sendto(b'hello-afxdp',('10.77.0.2',8080)),time.sleep(0.005)) " + f"for _ in range({count})]; s.close()" + ) + run("ip", "netns", "exec", NETNS, "python3", "-c", code) + + +def verify_nonmatching_passes(): + receiver = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + try: + receiver.bind(("10.77.0.2", 8081)) + receiver.settimeout(3) + code = ( + "import socket; " + "s=socket.socket(socket.AF_INET,socket.SOCK_DGRAM); " + "s.sendto(b'pass-afxdp',('10.77.0.2',8081)); s.close()" + ) + run("ip", "netns", "exec", NETNS, "python3", "-c", code) + payload, _ = receiver.recvfrom(64) + assert payload == b"pass-afxdp", payload + finally: + receiver.close() + + +def main(): + binary = str(pathlib.Path(sys.argv[1]).resolve()) + process = None + try: + setup() + command = [ + binary, + "--interface", RX, + "--queue", "0", + "--port", "8080", + "--count", "65", + ] + sys.argv[2:] + process = subprocess.Popen( + command, + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + ) + readable, _, _ = select.select([process.stdout], [], [], 5) + if not readable: + process.kill() + stdout, stderr = process.communicate(timeout=5) + raise AssertionError("timed out waiting for ready line\n" + stdout + stderr) + ready = process.stdout.readline() + if "afxdp-dump ready" not in ready: + stdout, stderr = process.communicate(timeout=5) + raise AssertionError(ready + stdout + stderr) + verify_nonmatching_passes() + send_packets(65) + stdout, stderr = process.communicate(timeout=20) + output = ready + stdout + stderr + if process.returncode: + raise AssertionError(output) + assert "10.77.0.1:" in output and "-> 10.77.0.2:8080" in output, output + assert 'payload="hello-afxdp"' in output, output + assert "packet=65 " in output and "redirected=65" in output, output + lines = output.splitlines() + print(ready, end="") + print(next(line for line in lines if line.startswith("packet=1 "))) + print(next(line for line in lines if line.startswith("packet=65 "))) + print(next(line for line in lines if line == "redirected=65")) + print("nonmatching-pass=verified") + print("AF_XDP dump integration test: PASS") + finally: + if process and process.poll() is None: + process.kill() + process.wait() + cleanup() + + +if __name__ == "__main__": + main() diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 96c9e73e..e110c203 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -69,6 +69,8 @@ Networking: - [lesson 46-xdp-test](46-xdp-test/README.md) Building a High-Performance XDP Packet Generator - [lesson 50-tcx](50-tcx/README.md) Composable Traffic Control with TCX Links - [lesson 53-egress-pacer](53-egress-pacer/README.md) Building an Egress Pacer with BPF Qdisc +- [lesson 56-tc-flow-index](56-tc-flow-index/README.md) Building a Top-Flow Monitor with Dual Rbtree Indexing +- [lesson 60-afxdp-dump](60-afxdp-dump/README.md) High-Performance UDP Receiving with AF_XDP Tracing: @@ -81,6 +83,7 @@ Tracing: - [lesson 40-mysql](40-mysql/README.md) Using eBPF to Trace MySQL Queries - [lesson 48-energy](48-energy/README.md) Energy Monitoring for Process-Level Power Analysis - [lesson 52-fsession-latency](52-fsession-latency/README.md) Tracing Slow vfs_read Calls with fsession +- [lesson 57-oom-watch](57-oom-watch/README.md) Profiling Memory Reclaim Before an OOM Kill Security: @@ -93,6 +96,7 @@ Security: - [lesson 34-syscall](34-syscall/README.md) Modifying System Call Arguments with eBPF - [lesson 51-tcp-quarantine](51-tcp-quarantine/README.md) Precisely Isolating Established TCP Connections - [lesson 54-exec-image-inspector](54-exec-image-inspector/README.md) Inspecting the Executable Image After exec +- [lesson 55-dns-egress](55-dns-egress/README.md) Building a DNS-Derived IP Allowlist with cgroup BPF Features: diff --git a/src/SUMMARY.zh.md b/src/SUMMARY.zh.md index 46490ec4..5e483f51 100644 --- a/src/SUMMARY.zh.md +++ b/src/SUMMARY.zh.md @@ -69,6 +69,8 @@ GPU: - [lesson 46-xdp-test](46-xdp-test/README.zh.md) eBPF 实例教程：构建高性能 XDP 数据包生成器 - [lesson 50-tcx](50-tcx/README.zh.md) eBPF 入门实践教程第五十篇：使用 TCX Link 实现可组合的流量控制 - [lesson 53-egress-pacer](53-egress-pacer/README.zh.md) eBPF 教程：用 BPF Qdisc 实现出口限速 +- [lesson 56-tc-flow-index](56-tc-flow-index/README.zh.md) eBPF 实战教程：用双红黑树索引构建 Top-Flow 监控器 +- [lesson 60-afxdp-dump](60-afxdp-dump/README.zh.md) eBPF 实战教程：用 AF_XDP 高性能接收 UDP 报文 追踪: @@ -81,6 +83,7 @@ GPU: - [lesson 40-mysql](40-mysql/README.zh.md) 使用 eBPF 跟踪 MySQL 查询 - [lesson 48-energy](48-energy/README.zh.md) eBPF 教程：进程级能源监控与功耗分析 - [lesson 52-fsession-latency](52-fsession-latency/README.zh.md) eBPF 教程：使用 fsession 追踪慢速 vfs_read 调用 +- [lesson 57-oom-watch](57-oom-watch/README.zh.md) eBPF 实战教程：在 OOM Kill 之前分析内存回收 安全: @@ -93,6 +96,7 @@ GPU: - [lesson 34-syscall](34-syscall/README.zh.md) eBPF 开发实践：使用 eBPF 修改系统调用参数 - [lesson 51-tcp-quarantine](51-tcp-quarantine/README.zh.md) eBPF 教程：精准隔离已建立的 TCP 连接 - [lesson 54-exec-image-inspector](54-exec-image-inspector/README.zh.md) eBPF 教程：检查 exec 后实际安装的可执行镜像 +- [lesson 55-dns-egress](55-dns-egress/README.zh.md) eBPF 实战教程：用 cgroup BPF 构建基于 DNS 的 IP 允许列表 特性: diff --git a/src/compatibility.md b/src/compatibility.md index 07f6aa09..6570ec38 100644 --- a/src/compatibility.md +++ b/src/compatibility.md @@ -61,6 +61,10 @@ This table is generated from each tutorial's `.config` metadata. `Minimum kernel | [eBPF Tutorial: Tracing Slow vfs_read Calls with fsession](52-fsession-latency/README.md) | 7.0 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y | None | Required | CI build | | [eBPF Tutorial: Building an Egress Pacer with BPF Qdisc](53-egress-pacer/README.md) | 6.16 | Tutorial docs | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | Network interface | Required | CI build | | [eBPF Tutorial: Inspecting the Executable Image After exec](54-exec-image-inspector/README.md) | 6.19 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_LSM=y, CONFIG_SECURITY=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | +| [eBPF Tutorial: Building a DNS-Derived IP Allowlist with cgroup BPF](55-dns-egress/README.md) | 5.12 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_CGROUP_BPF=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_INET=y | None | Required | CI build | +| [eBPF Tutorial: Building a Top-Flow Monitor with Dual Rbtree Indexing](56-tc-flow-index/README.md) | 6.16 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_CLS_BPF=y | Network interface | Required | CI build | +| [eBPF Tutorial: Profiling Memory Reclaim Before an OOM Kill](57-oom-watch/README.md) | 7.1 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_MEMCG=y | None | Required | CI build | +| [eBPF Tutorial: High-Performance UDP Receiving with AF_XDP](60-afxdp-dump/README.md) | 5.7 | Required feature | x86_64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_XDP_SOCKETS=y | Network interface | Required | CI build | | [eBPF Tutorial: cgroup-based Policy Control](cgroup/README.md) | 5.8 | Required feature | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | | [eBPF Tutorial by Example: BPF Arena for Zero-Copy Shared Memory](features/bpf_arena/README.md) | 6.9 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | | [eBPF Tutorial: BPF Iterators for Kernel Data Export](features/bpf_iters/README.md) | 5.8 | Tutorial docs | x86_64, arm64 | Required | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | None | Required | CI build | diff --git a/src/compatibility.zh.md b/src/compatibility.zh.md index 4d6353f8..eaf59c89 100644 --- a/src/compatibility.zh.md +++ b/src/compatibility.zh.md @@ -61,6 +61,10 @@ | [eBPF 教程：使用 fsession 追踪慢速 vfs_read 调用](52-fsession-latency/README.zh.md) | 7.0 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS=y | 无 | 需要 | CI 构建测试 | | [eBPF 教程：用 BPF Qdisc 实现出口限速](53-egress-pacer/README.zh.md) | 6.16 | 教程文档 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_SCH_BPF=y | 网络接口 | 需要 | CI 构建测试 | | [eBPF 教程：检查 exec 后实际安装的可执行镜像](54-exec-image-inspector/README.zh.md) | 6.19 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_LSM=y, CONFIG_SECURITY=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 | +| [eBPF 实战教程：用 cgroup BPF 构建基于 DNS 的 IP 允许列表](55-dns-egress/README.zh.md) | 5.12 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_CGROUP_BPF=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_INET=y | 无 | 需要 | CI 构建测试 | +| [eBPF 实战教程：用双红黑树索引构建 Top-Flow 监控器](56-tc-flow-index/README.zh.md) | 6.16 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_NET_SCHED=y, CONFIG_NET_CLS_BPF=y | 网络接口 | 需要 | CI 构建测试 | +| [eBPF 实战教程：在 OOM Kill 之前分析内存回收](57-oom-watch/README.zh.md) | 7.1 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_BPF_EVENTS=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_MEMCG=y | 无 | 需要 | CI 构建测试 | +| [eBPF 实战教程：用 AF_XDP 高性能接收 UDP 报文](60-afxdp-dump/README.zh.md) | 5.7 | 必需特性 | x86_64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y, CONFIG_XDP_SOCKETS=y | 网络接口 | 需要 | CI 构建测试 | | [eBPF 实例教程：基于 cgroup 的策略控制](cgroup/README.zh.md) | 5.8 | 必需特性 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_CGROUPS=y, CONFIG_CGROUP_BPF=y, CONFIG_CGROUP_DEVICE=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 | | [eBPF 实例教程：BPF Arena 零拷贝共享内存](features/bpf_arena/README.zh.md) | 6.9 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_BPF_JIT=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 | | [eBPF 教程：BPF 迭代器用于内核数据导出](features/bpf_iters/README.zh.md) | 5.8 | 教程文档 | x86_64, arm64 | 需要 | CONFIG_BPF=y, CONFIG_BPF_SYSCALL=y, CONFIG_DEBUG_INFO_BTF=y | 无 | 需要 | CI 构建测试 |