Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 74 additions & 7 deletions pipeline/outputs/file.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,16 @@ The _File_ output plugin lets you write the data received through the input plug

| Key | Description | Default |
| :--- | :--- | :--- |
| `fallback_file` | Static filename used when a `fallback` action is applied. Required whenever `path` or `file` uses a record accessor. See [Dynamic destinations](#dynamic-destinations). | _none_ |
| `enable_strftime` | Enable [`strftime`](#strftime-placeholders) placeholders in `path` and `file`. Disabled by default so that literal percent (`%`) characters in existing filenames are preserved. | `false` |
| `fallback_file` | Static filename used when a `fallback` action is applied. Required whenever `path` or `file` uses a record accessor or a `strftime` placeholder. See [Dynamic destinations](#dynamic-destinations). | _none_ |
| `fallback_path` | Static directory path used together with `fallback_file` when a `fallback` action is applied. | _none_ |
| `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions. | _none_ |
| `file` | Set filename to store the records. If not set, the filename will be the `tag` associated with the records. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions. When `enable_strftime` is `true`, `strftime` placeholders are expanded using the UTC event timestamp. | _none_ |
| `format` | The [format](#format) of the file content. | _none_ |
| `max_dynamic_files` | Maximum number of distinct destinations that record accessor expressions can resolve to. Set to `0` for unlimited. | `1024` |
| `max_dynamic_files` | Maximum number of distinct destinations that record accessor expressions and `strftime` placeholders can resolve to. Set to `0` for unlimited. | `1024` |
| `mkdir` | Recursively create output directory if it doesn't exist. Permissions set to `0755`. | `false` |
| `on_limit_reached` | Action to take for a record whose destination would exceed `max_dynamic_files`. Accepted values: `error`, `drop`, `fallback`. | `error` |
| `on_missing_field` | Action to take for a record whose record accessor field is missing, or resolves to an unsafe value. Accepted values: `error`, `drop`, `fallback`. | `error` |
| `path` | Directory path to store files. If not set, Fluent Bit will write the files in its own working directory. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions, which must follow a static prefix. | _none_ |
| `path` | Directory path to store files. If not set, Fluent Bit will write the files in its own working directory. Supports [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions, which must follow a static prefix. When `enable_strftime` is `true`, `strftime` placeholders are expanded using the UTC event timestamp. | _none_ |
| `rotate` | Enable size-based [log rotation](#log-rotation). When enabled, files that exceed `rotate_max_size` are rotated and optionally compressed. | `false` |
| `rotate_gzip` | Compress rotated files using gzip. Only applies when `rotate` is enabled. | `true` |
| `rotate_max_files` | Maximum number of rotated files to retain per output file. Oldest files are deleted first. Must be `1` or greater. Only applies when `rotate` is enabled. | `7` |
Expand Down Expand Up @@ -183,17 +184,41 @@ pipeline:

Dynamic destinations are available in Fluent Bit version 5.1 and greater.

The `path` and `file` parameters accept [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions, which lets one output instance write records to different files based on the content of each record. Fluent Bit treats a destination as dynamic when `path` or `file` contains a `$` character.
The `path` and `file` parameters accept [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) expressions, which lets one output instance write records to different files based on the content of each record. They also accept [`strftime` placeholders](#strftime-placeholders), which route records by the time at which each event occurred. Fluent Bit treats a destination as dynamic when `path` or `file` contains a `$` character, or when `enable_strftime` is `true` and `path` or `file` contains a percent (`%`) character.

Two rules apply when you configure a dynamic destination:

- A dynamic `path` must begin with a static prefix. Fluent Bit rejects a `path` that starts with a record accessor, so that generated paths always stay under a directory you chose.
- You must set `fallback_file`. Fluent Bit fails to start if a record accessor is used without it, because there would be nowhere to write records whose destination can't be resolved.
- A `path` that uses a record accessor must begin with a static prefix. Fluent Bit rejects a `path` that starts with a record accessor, so that generated paths always stay under a directory you chose. This check doesn't apply to `strftime` placeholders. See [`strftime` placeholders](#strftime-placeholders).
- You must set `fallback_file`. Fluent Bit fails to start if a record accessor or a `strftime` placeholder is used without it, because there would be nowhere to write records whose destination can't be resolved.

Fluent Bit rejects a resolved destination as unsafe when the filename is empty, is `.` or `..`, or contains a path separator or one of the characters `:`, `*`, `?`, `"`, `<`, `>`, or `|`. Path components of `.` and `..` are also rejected. This keeps a record value from redirecting output outside of the configured directory.

Generated directories are only created if you also enable `mkdir`.

Dynamic destinations apply to log records only. Metrics records are always written to `fallback_path` and `fallback_file`.

### `strftime` placeholders

Support for `strftime` placeholders is available in Fluent Bit version 5.1.1 and greater.

Set `enable_strftime` to `true` to expand [`strftime`](https://man7.org/linux/man-pages/man3/strftime.3.html) placeholders such as `%Y`, `%m`, and `%d` in `path` and `file`. Fluent Bit formats each placeholder using the timestamp of the event being written, expressed in UTC. Because the value comes from the event itself and not from the clock at flush time, records that arrive late are still written to the file for the period in which they occurred.

This parameter is disabled by default. When `enable_strftime` is `false`, a percent character in `path` or `file` is written literally, which preserves existing filenames that contain one.
Comment thread
eschabell marked this conversation as resolved.

When `enable_strftime` is `true`, escape every literal percent character in `path` and `file` as `%%`. A `%` that isn't escaped is consumed as the start of a `strftime` conversion specifier and is replaced with a time value.

Fluent Bit treats any `path` or `file` containing a percent character as a dynamic destination when `enable_strftime` is `true`, so `fallback_file` is required even when the value contains only escaped `%%` sequences and always resolves to the same destination.

You can combine placeholders with record accessors in the same value. Fluent Bit expands the `strftime` placeholders first, then resolves the record accessors against the result:

```text
path: /var/log/fluent-bit/%Y/%m/%d/$namespace
```

Most rules described in [Dynamic destinations](#dynamic-destinations) apply to placeholders the same way they apply to record accessors. `fallback_file` is required, resolved destinations are rejected when unsafe, and each distinct destination counts toward `max_dynamic_files`. Because a new destination is created for every time period you include, choose the smallest unit your retention needs. An hourly layout such as `%Y%m%d%H` reaches the default `max_dynamic_files` limit of `1024` after about 42 days of continuous operation.

The static prefix requirement is the exception. Fluent Bit enforces it for record accessors only, so a `path` can begin with a placeholder. A `path` of `%Y/%m/%d` resolves to a relative directory such as `2026/08/17`, located inside the Fluent Bit working directory. Begin the value with a static prefix, as in `/var/log/fluent-bit/%Y/%m/%d`, to keep output under a directory you chose.

### Actions for unresolved destinations

Two settings control what happens to a record whose destination Fluent Bit can't use. `on_missing_field` applies when the record accessor field is missing or resolves to an unsafe value, and `on_limit_reached` applies when writing to a new destination would exceed `max_dynamic_files`. Both accept the same actions:
Expand Down Expand Up @@ -248,6 +273,48 @@ pipeline:
{% endtab %}
{% endtabs %}

### Time-based destination example

The following configuration writes each record to a daily directory derived from the UTC event timestamp. Records whose destination can't be resolved go to `/var/log/fluent-bit/unrouted.log`:

{% tabs %}
{% tab title="fluent-bit.yaml" %}

```yaml
pipeline:
outputs:
- name: file
match: '*'
path: /var/log/fluent-bit/%Y/%m/%d
file: app.log
enable_strftime: true
mkdir: true
on_missing_field: fallback
on_limit_reached: fallback
fallback_path: /var/log/fluent-bit
fallback_file: unrouted.log
```

{% endtab %}
{% tab title="fluent-bit.conf" %}

```text
[OUTPUT]
Name file
Match *
Path /var/log/fluent-bit/%Y/%m/%d
File app.log
Enable_Strftime true
Mkdir true
On_Missing_Field fallback
On_Limit_Reached fallback
Fallback_Path /var/log/fluent-bit
Fallback_File unrouted.log
```

{% endtab %}
{% endtabs %}

Comment thread
eschabell marked this conversation as resolved.
## Get started

You can run the plugin from the command line or through the configuration file.
Expand Down
Loading