From 4beef18d68f0556a7acd5317ab291047ed501105 Mon Sep 17 00:00:00 2001 From: Tim Bai Date: Fri, 21 Aug 2026 15:27:33 -0400 Subject: [PATCH] in_tail: restrict file_cache_advise eviction to the consumed range Since #8422, when file_cache_advise is enabled (the default on Linux), in_tail calls posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED) over the whole file on every read cycle. This has two unintended side effects on files that are being actively appended to by a writer: 1. Write amplification: POSIX_FADV_DONTNEED starts writeback of dirty pages. The last page of an actively written log file is almost always dirty, so the same tail page is flushed to disk on every read cycle instead of once by the kernel flusher. 2. Cold reads for other consumers: the advice also evicts pages that other readers still need. For example, kubelet (Kubernetes) compresses rotated container log files; with the cache evicted it re-reads every rotated file from disk. Steady-state extra reads equal roughly 1x the log production rate. Measured on GKE (3 nodes, ~2.6 MB/s of logs per node, 10 min windows, whole-file advise vs no advise): disk reads 2.55 MB/s -> 0.00 MB/s, disk write IOPS 110 -> 76, at identical log throughput. The fix is to cap the advised range at the last page boundary below the consumed offset: posix_fadvise(fd, 0, aligned_end, POSIX_FADV_DONTNEED). The still-appended (dirty) tail page is never advised, so no page is flushed to disk more than once, and data that has not been read yet is never evicted. The advice is still issued on every read cycle so that pages which were dirty (and therefore not droppable) when first advised are evicted by a later call once their writeback completes; consumed pages are never re-dirtied by the writer, so the repetition causes no additional writes. This keeps the feature's purpose: consumed log data does not accumulate in the page cache. Also fix the error check: posix_fadvise() returns the error number and does not set errno, so the previous `== -1` check plus flb_errno() could never report a real failure. Signed-off-by: Tim Bai --- plugins/in_tail/tail_file.c | 42 ++++++++++++++++++++++++++++++++++--- 1 file changed, 39 insertions(+), 3 deletions(-) diff --git a/plugins/in_tail/tail_file.c b/plugins/in_tail/tail_file.c index 131893e6518..390d8cf34d7 100644 --- a/plugins/in_tail/tail_file.c +++ b/plugins/in_tail/tail_file.c @@ -1907,6 +1907,11 @@ int flb_tail_file_chunk(struct flb_tail_file *file) char *tmp; int ret; int lines; +#ifdef __linux__ + int64_t advise_end; + long page_size; + int advise_ret; +#endif struct flb_tail_config *ctx; /* Check if we the engine issued a pause */ @@ -2001,9 +2006,40 @@ int flb_tail_file_chunk(struct flb_tail_file *file) #ifdef __linux__ if (ctx->file_cache_advise) { - if (posix_fadvise(file->fd, 0, 0, POSIX_FADV_DONTNEED) == -1) { - flb_errno(); - flb_plg_error(ctx->ins, "error during posix_fadvise"); + /* + * Advise only the page-aligned byte range that has already been + * consumed (up to file->offset). + * + * Advising the whole file would also hit the last page, which the + * writer is still appending to: POSIX_FADV_DONTNEED starts + * writeback of dirty pages, so the same tail page would be flushed + * to disk over and over (write amplification). It would also evict + * pages that other readers of the file (e.g. log rotation + * compressors) have not consumed yet, turning their cache hits + * into physical reads. + * + * The advice is repeated over the whole consumed range on every + * cycle on purpose: the kernel cannot drop pages that are still + * dirty when advised, it only starts their writeback, so they can + * only be evicted by a later call once they are clean. Consumed + * pages are never re-dirtied (the writer only appends), so each + * page is still written back at most once, and re-advising a + * mostly evicted range is cheap. + */ + page_size = sysconf(_SC_PAGESIZE); + if (page_size > 0) { + advise_end = file->offset & ~((int64_t) page_size - 1); + if (advise_end > 0) { + advise_ret = posix_fadvise(file->fd, 0, (off_t) advise_end, + POSIX_FADV_DONTNEED); + if (advise_ret != 0) { + /* posix_fadvise() returns the error number, it does + * not set errno */ + flb_plg_error(ctx->ins, + "posix_fadvise error=%i file=%s", + advise_ret, file->name); + } + } } } #endif