nfs-foresight

eBPF-based large-directory GETATTR/LOOKUP storm detector and reactive prewarmer for NFS clients. It tracks open NFS directories, flags "large" ones by entry count, counts per-directory getattr/lookup cache misses (server round-trips) in a 30-second sliding window, and — when active (see Activation) — reactively prewarms a directory on a miss storm with a paced getdents64+statx scan so per-file GETATTR/LOOKUP storms are served from cache as batched READDIRPLUS instead.

It runs as a systemd daemon (off by default) that attaches its probes only while an NFS mount is present and reads its thresholds from a live-reloaded config file.

---

Requirements

• Runtime (customer host): Linux with BTF (/sys/kernel/btf/vmlinux), bpffs at /sys/fs/bpf, systemd. Validated on SLES 15 SP5, kernel 5.14.21-150500.55.177-default.
• Build host: clang (>= 15), bpftool, libbpf (headers + static libbpf.a recommended), libelf, zlib, and kernel BTF. Build on SLES 15 SP5 so the userspace binary's glibc matches the target (the BPF object is CO-RE and relocates at load; glibc does not).

Build

make                      # builds foresight.bpf.o, skeleton, and the binary
make dist                 # + smoke check (--version, ldd) and nfs-foresight-<ver>.tar.gz
# override the static lib path if needed:
make LIBBPF_A=/usr/lib64/libbpf.a

make dist produces nfs-foresight-<version>.tar.gz containing the binary, systemd unit, default config, and the install scripts.

Install (on the customer host)

tar xzf nfs-foresight-<version>.tar.gz
cd nfs-foresight-<version>
sudo ./install.sh         # preflight + install the unit (off by default; not started)
sudo ./uninstall.sh       # remove (config in /etc/nfs-foresight is left in place)

install.sh preflights root, systemd, kernel BTF, bpffs, and that the binary runs (glibc check). It installs the unit without enabling or starting it, and preserves an existing /etc/nfs-foresight/foresight.conf and /etc/nfs-foresight/enabled on upgrade.

Activation

Two independent switches, both off by default:

• Run the daemon — systemctl start nfs-foresight starts it in observer mode: it attaches its probes, tracks large dirs, and counts miss storms, but does not prewarm. (Not running = fully passive.) Use systemctl enable to also start at boot.

• Enable prewarm — the trigger file /etc/nfs-foresight/enabled is the sole authority for observer vs active, applied live via inotify (no restart):

  echo 1 > /etc/nfs-foresight/enabled   # ACTIVE: reactively prewarm on storms
  echo 0 > /etc/nfs-foresight/enabled   # OBSERVER: detect + log only (default)

  Absent, empty, or invalid content ⇒ observer (fail-safe). 1/on/true (case-insensitive) ⇒ active. This is the live kill switch for prewarming.

Configuration

File: /etc/nfs-foresight/foresight.conf (key = value, # comments). Edits apply live on save via inotify — no restart. Invalid/out-of-range values are rejected and the previous value is kept (a WARN is logged); a missing file falls back to built-in defaults.

Key	Meaning	Range	Default
large_dir_threshold	directory entry count to classify "large"	1 .. 100000000	1000
storm_miss_threshold	foreground getattr+lookup misses per fixed 30s window, per dir, that trigger a prewarm	5 .. 1000000	20
allow_mounts	track only NFS mounts under these mountpoint prefixes	comma list	(unset)
deny_mounts	track all NFS mounts except these mountpoint prefixes	comma list	(unset)

Notes:

• Observer vs active is not a config key — it is the /etc/nfs-foresight/enabled trigger file (see Activation). A legacy mode key in the config is ignored (with a one-time deprecation warning).
• allow_mounts and deny_mounts are mutually exclusive; if both are set, allow_mounts wins. Unset ⇒ all NFS mounts are in scope.
• Directories on out-of-scope mounts are not tracked, counted, or prewarmed at all (mount-level granularity; matching is by mountpoint prefix).
• storm_miss_threshold is misses per 30-second window (the window is fixed), not per second. The floor of 5 prevents a mistuned value from stampeding the server.

Prewarm safety (active mode)

Prewarming is bounded: at most 8 concurrent scanners, 30 spawns/minute, and a 120-second per-scan deadline (overrunning scans are killed). Scans are paced (50 ms between batches) to let the kernel self-serve READDIRPLUS rather than bursting RPCs.

Raising large_dir_threshold

On an increase, previously-large directories are forced to re-walk and their miss rings are dropped, so directories that fall below the new bar stop being prewarmed. (Lowering self-heals as directories are re-walked.)

Operations

systemctl status nfs-foresight
journalctl -u nfs-foresight -f      # storms, prewarm decisions, applied config
nfs-foresight --status              # effective live config + tracked-dir counts
nfs-foresight --version

The daemon attaches its probes when an in-scope NFS mount appears and detaches after ~120 s with no NFS mounts (hysteresis absorbs autofs churn). While idle it is event-driven and near-zero cost.

Supported kernel & degradation

CO-RE relocates the BPF object to the running kernel's types, but the tool is validated only against 5.14.21-150500.55.177-default. One probe target, nfs_lookup_revalidate_dentry, is a static symbol that a maintenance kernel may inline; the daemon detects its absence and disables just that probe (cold nfs_lookup still covers the dominant case) rather than failing to start. After a kernel update, re-verify with nfs-foresight --status and the journal.

Known overheads & assumptions

• While attached, filldir64 is hooked, which fires per directory entry for all filesystems (not just NFS); the handler is minimal but non-zero. This is why the daemon detaches when no NFS is mounted.
• Path resolution and prewarm scanning use the daemon's mount namespace; a containerized/namespaced NFS client is not supported.