From 601489e112865568012cd57a29f7d932f3cdc957 Mon Sep 17 00:00:00 2001 From: "Eric D. Schabell" Date: Thu, 27 Aug 2026 13:15:23 +0200 Subject: [PATCH] docs: pipeline: outputs: splunk: document time_key and time_key_format Document the record-derived event time options added to the Splunk output plugin in fluent-bit 36b233dcc. - Add a time_key row to the event and index parameters table, covering the plain key name and record accessor forms, and linking to the new Record-derived event time section - Add a time_key_format row for the strptime(3) format used when the key holds a string, including the %L descriptor for fractional seconds - Add a Record-derived event time section explaining that Fluent Bit parses the value itself and sends the result in the HTTP Event Collector envelope, unlike auto_extract_timestamp which delegates parsing to the indexer - Document the value types parsed without a format: integer, float, numeric string, and an event time from an earlier pipeline stage - Note that a plain key name is treated as a top-level key, so aggregator_time and $aggregator_time are equivalent, and that nested access needs the record accessor form - Note that an invalid time_key_format prevents the output from starting, since the format is prepared once at initialization - Document the fallback path: a missing key logs at debug, a value that cannot be parsed logs a warning, and the event timestamp is used in both cases - Document the interaction with the two neighbouring options: both are ignored under splunk_send_raw, which warns rather than failing so an unrelated pattern cannot stop a raw mode output from starting, and time_key has no effect under auto_extract_timestamp because the time field is omitted from the envelope entirely - Add a YAML and classic configuration example in the page's tab style - Scope time_key_format startup failure to envelope mod Signed-off-by: Eric D. Schabell --- pipeline/outputs/splunk.md | 57 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) diff --git a/pipeline/outputs/splunk.md b/pipeline/outputs/splunk.md index cfdd9a0ad..88aa0e33c 100644 --- a/pipeline/outputs/splunk.md +++ b/pipeline/outputs/splunk.md @@ -43,6 +43,8 @@ Content and Splunk metadata (fields) handling configuration properties: | `event_sourcetype` | Set the `sourcetype` value to assign to the event data. | _none_ | | `event_sourcetype_key` | Set a record key that will populate `sourcetype`. If the key is found, it will have precedence over the value set in `event_sourcetype`. | _none_ | | `splunk_send_raw` | When enabled, the record keys and values are set in the top level of the map instead of under the event key. See [Sending Raw Events](#sending-raw-events) to configure this option. | `off` | +| `time_key` | Set a record key whose value populates the top level `time` field of the HTTP Event Collector payload instead of the Fluent Bit event timestamp. Accepts a plain key name or a [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) pattern. See [Record-derived event time](#record-derived-event-time). Supported in v5.1.2 or later. | _none_ | +| `time_key_format` | Set the `strptime(3)` compatible format used to parse the value of `time_key` when that value is a string, for example `%Y-%m-%dT%H:%M:%S.%LZ`. The `%L` field descriptor is supported for fractional seconds. Requires `time_key`. Supported in v5.1.2 or later. | _none_ | ### TLS / SSL @@ -235,6 +237,61 @@ If `splunk_send_raw` is also enabled, Fluent Bit doesn't generate a `time` field Splunk must be able to find a timestamp in the event data. If it can't, it assigns the time at which the event was indexed. For the timestamp formats and the extraction rules Splunk applies, see [Configure timestamp recognition](https://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps). +### Record-derived event time + +Record-derived event time is available in Fluent Bit version 5.1.2 and greater. + +Set `time_key` to the name of a record key that holds the event time when you want Splunk to receive that value in the `time` field of the event envelope instead of the timestamp Fluent Bit assigned. Unlike [automatic timestamp extraction](#automatic-timestamp-extraction), which delegates parsing to the HTTP Event Collector, Fluent Bit parses the value itself and sends the result in the envelope. + +{% tabs %} +{% tab title="fluent-bit.yaml" %} + +```yaml +pipeline: + outputs: + - name: splunk + match: '*' + host: 127.0.0.1 + port: 8088 + splunk_token: 00000000-0000-0000-0000-000000000000 + time_key: aggregator_time + time_key_format: '%Y-%m-%dT%H:%M:%S.%LZ' +``` + +{% endtab %} +{% tab title="fluent-bit.conf" %} + +```text +[OUTPUT] + Name splunk + Match * + Host 127.0.0.1 + Port 8088 + Splunk_Token 00000000-0000-0000-0000-000000000000 + Time_Key aggregator_time + Time_Key_Format %Y-%m-%dT%H:%M:%S.%LZ +``` + +{% endtab %} +{% endtabs %} + +`time_key` accepts a plain key name or a [record accessor](../../administration/configuring-fluent-bit/classic-mode/record-accessor.md) pattern. A plain key name is treated as a top level key, so `aggregator_time` and `$aggregator_time` are equivalent. Use the record accessor form to reach a nested key, for example `$meta['ts']`. + +The following value types are parsed without setting `time_key_format`: + +- An integer or float holding a Unix timestamp, with an optional fractional part. +- A string holding a numeric Unix timestamp. +- An event time value produced by an earlier stage of the pipeline. + +Set `time_key_format` for any other string format. When `time_key` is populating the event envelope, Fluent Bit prepares the format once at startup, so an invalid format prevents the output from starting. When `time_key` isn't in use, the format is never prepared and Fluent Bit logs a warning that the option has no effect. + +If the key isn't present in the record, or its value can't be parsed as a timestamp, Fluent Bit uses the event timestamp instead and continues. A missing key is reported at the `debug` log level, and a value that can't be parsed is reported as a warning. Raise the log level to confirm which records fell back. + +These options apply only to the event envelope: + +- When `splunk_send_raw` is enabled there is no envelope to populate, so both options are ignored and neither is validated. Fluent Bit logs warnings at startup rather than failing, which means neither an unrelated `time_key` pattern nor an invalid `time_key_format` can stop a raw mode output from starting. +- When `auto_extract_timestamp` is enabled the `time` field is omitted from the envelope entirely, so `time_key` has no effect. Use one option or the other. + ## Splunk metric index With Splunk version 8.0 and later, you can use the Fluent Bit Splunk output plugin to send data to metric indices. This lets you perform visualizations, metric queries, and analysis with other metrics you might be collecting. This is based off of Splunk 8.0 support of multi metric support using single JSON payload, more details can be found in [Splunk metrics documentation](https://docs.splunk.com/Documentation/Splunk/9.4.2/Metrics/GetMetricsInOther#The_multiple-metric_JSON_format)