Add Native Syslog support to CloudWatch Agent

[kwarenycia]

Description of the issue

The current CloudWatch Agent has no support for native syslog support. This means customers who have Syslog sources and using the CWA must use some other agent, some other bridging application, or write syslog to disk and have the CWA read those files. This adds complexity to customer configurations that want to use CWA or alternatively customers will use other agents.

Description of changes

This change adds native Syslog support to the CloudWatch Agent. Customers are able to define multiple Syslog listeners using tcp/udp and add various filters and routing rules to ensure log entries land on the correct log groups. These changes support both Syslog format RFCs, 5424 as well as legacy RFC 3164. Additionally, support for server and client side TLS support is included as part of the listener.

The changes are broken down in the following areas:

1. Schema Validation — JSON schema for syslog under logs_collected supporting both single-object and array (multi-section) forms. Listeners are transport-only config; routing/filters/delivery are at the
   section level.
2. Pipeline Translator — Creates one input pipeline (all receivers → global filter → routing connector) plus one output pipeline per routing rule and one default pipeline per section.
3. Syslog Receiver Translator — Wraps upstream syslogreceiver, configures TCP/UDP + TLS + protocol format + on_error: drop for unparseable messages.
4. Routing Connector Integration — Generates OTTL conditions from user match rules (hostname glob → IsMatch(attributes["hostname"], "web-*"), facility → attributes["facility"] == 4, app_name →
   IsMatch(attributes["appname"], "...")).
5. Filter Processor Integration — Global filters in input pipeline, per-rule filters in output pipelines. Exclude → IsMatch(body, "..."), include → not IsMatch(body, "...").
6. Dual Delivery Modes — "PutLogEvents" (default, raw syslog line preserved) and "OTLP", however OTLP is currently disabled for now.
7. CWL Provisioner Extension — Per-pipeline provisioner instances inject x-aws-log-group/x-aws-log-stream headers at request time (bypasses configopaque.String nil serialization issue in OTLP mode).
8. Component Registration — Registers routingconnector factory in default components.
9. Multi-Section Support — syslog accepts an array of independent section objects, each with its own listeners, routing, filters, and delivery config. Section-indexed naming (syslog_0_, syslog_1_) prevents
   collisions.

Example Config:

{
    "agent": {
        "region": "eu-west-1"
    },
    "logs": {
        "logs_collected": {
            "syslog": {
                "listeners": [{
                        "listen_address": "tcp://0.0.0.0:514"
                    },
                    {
                        "listen_address": "udp://0.0.0.0:514",
                        "protocol": "rfc3164"
                    },
                    {
                        "listen_address": "tcp://0.0.0.0:6514",
                        "tls": {
                            "cert_file": "/etc/tls/cert.pem",
                            "key_file": "/etc/tls/key.pem",
                            "ca_file": "/etc/tls/ca.crt"
                        }
                    }
                ],
                "log_group_name": "/syslog/default",
                "log_stream_name": "{hostname}",
                "retention_in_days": 7,
                "filters": [{
                    "type": "exclude",
                    "expression": "health[-_]?check"
                }],
                "routing": [{
                        "match": {
                            "hostname": "web-*"
                        },
                        "log_group_name": "/syslog/web",
                        "log_stream_name": "{hostname}/{app_name}",
                        "filters": [{
                            "type": "include",
                            "expression": "error|warn"
                        }]
                    },
                    {
                        "match": {
                            "facility": 4
                        },
                        "log_group_name": "/syslog/auth",
                        "retention_in_days": 365
                    },
                    {
                        "match": {
                            "app_name": "cron"
                        },
                        "log_group_name": "/syslog/cron"
                    }
                ]
            }
        }
    }
}

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

translator/translate/otel/pipeline/syslog

translator_test.go

Test	Description
TestNewTranslatorsNilConf	Nil config returns empty map
TestNewTranslatorsNoSyslogKey	Missing syslog key returns empty map
TestNewTranslatorsSingleListenerNoRules	Single listener with no routing produces input + default pipelines
TestNewTranslatorsWithRoutingRules	Creates input + rule + default pipelines
TestInputPipelineTranslate	Input pipeline wires receivers → filter → routing connector
TestOutputPipelineTranslate	Output pipeline wires routing connector → filter → batch → exporter
TestDeriveReceiverName	Address converted to safe component name
TestBuildOTTLCondition	Generates correct OTTL from hostname/facility/app_name match rules
TestIsGlobPattern	Detects glob characters in patterns
TestToFilters	Extracts filter type and expression from config
TestNewTranslatorsWithTLS	TLS config passed to receiver
TestFilterProcessorTranslator_ID	Returns correct filter processor component ID
TestFilterProcessorTranslator_Translate	Generates OTTL drop conditions from include/exclude filters
TestRoutingConnectorTranslator_ID	Returns correct routing connector component ID
TestRoutingConnectorTranslator_Translate	Generates routing table with OTTL conditions and default pipeline
TestSigV4AuthTranslator_ID	Returns correct sigv4auth extension ID
TestSigV4AuthTranslator_Translate	Produces sigv4auth config with region and role ARN
TestSigV4AuthTranslator_NoRoleARN	Omits role ARN when not configured
TestBuildFilterConditions	Combines exclude/include into correct OTTL drop conditions
TestNewTranslatorsMultiSection	Array form creates independent pipeline groups per section
TestNewTranslatorsArrayFormSingleSection	Single-element array treated as one section
TestNewTranslatorsDuplicateListenerAcrossSections	Duplicate listen address across sections rejected
TestNewTranslatorsUniqueListenersAcrossSections	Unique addresses across sections accepted
TestProvisionerTranslator_ID	Returns correct provisioner extension ID
TestProvisionerTranslator_Translate	Produces provisioner config with log group/stream/retention
TestProvisionerTranslator_ZeroRetention	Zero retention omitted from provisioner config

exporter_test.go

Test	Description
TestCWLExporterTranslator_ID	Returns correct CWL exporter component ID
TestCWLExporterTranslator_Translate	Produces CWL exporter config with raw_log, log group/stream
TestOTLPExporterTranslator_ID	Returns correct OTLP exporter component ID
TestOTLPExporterTranslator_Translate	Produces OTLP exporter config with endpoint and auth
TestOTLPExporterTranslator_EndpointOverride	Respects endpoint_override setting
TestNewExporterTranslator_Dispatch	Dispatches to CWL or OTLP translator based on delivery_mode

translator/translate/otel/receiver/syslog

Test	Description
TestTranslatorID	Returns correct component ID
TestTranslateTCP	Produces TCP listener config
TestTranslateUDP	Produces UDP listener config
TestTranslateInvalidAddress	Rejects malformed listen address
TestTranslateUnsupportedProtocol	Rejects non-tcp/udp protocol
TestTranslateRFC3164	Sets rfc3164 syslog format
TestTranslateDefaultProtocol	Defaults to rfc5424 format
TestTranslateTCPWithTLS	Adds TLS config to TCP
TestTranslateTCPWithClientCAFile	Adds mTLS client CA
TestTranslateUDPIgnoresTLS	TLS silently ignored for UDP
TestParseListenAddress	Splits protocol from host:port

translator/config — Schema Validation

Test	Description
TestSyslogSchema_SingleListener	Single listener object validates
TestSyslogSchema_MultipleListeners	Array of listeners validates
TestSyslogSchema_WithRouting	Routing rules validate
TestSyslogSchema_WithFilters	Include/exclude filters validate
TestSyslogSchema_WithTLS	TLS block validates
TestSyslogSchema_WithProtocol	rfc3164 protocol validates
TestSyslogSchema_WithRetention	Valid retention_in_days validates
TestSyslogSchema_FullConfig	Complete multi-listener config validates
TestSyslogSchema_InvalidTopLevelKey	Unknown key under logs rejected
TestSyslogSchema_MissingListenAddress	Required listen_address enforced
TestSyslogSchema_MissingLogGroupName	Required log_group_name enforced
TestSyslogSchema_InvalidProtocol	Unknown protocol enum rejected
TestSyslogSchema_InvalidTLSMinVersion	Bad TLS version rejected
TestSyslogSchema_InvalidRetention	Non-standard retention value rejected
TestSyslogSchema_InvalidFacility	Facility >23 rejected
TestSyslogSchema_RoutingMissingMatch	Routing without match rejected
TestSyslogSchema_RoutingMissingLogGroup	Routing without log_group rejected
TestSyslogSchema_RoutingEmptyMatch	Empty match object rejected
TestSyslogSchema_UnknownListenerField	Extra listener field rejected
TestSyslogSchema_UnknownTLSField	Extra TLS field rejected
TestSyslogSchema_WithClientCAFile	client_ca_file validates
TestSyslogSchema_UnknownMatchField	Extra match field rejected
TestSyslogSchema_ArrayForm_SingleSection	Single-element array validates
TestSyslogSchema_ArrayForm_MultipleSections	Multi-section array validates
TestSyslogSchema_ArrayForm_EmptyArray	Empty array rejected
TestSyslogSchema_ArrayForm_InvalidSection	Invalid section in array rejected

translator/tocwconfig — End-to-End

Test	Description
TestSyslogConfig	Full JSON-to-YAML translation matches golden file

Requirements

Before commiting your code, please do the following steps.

1. Run make fmt and make fmt-sh
2. Run make lint

---

Integration Tests

To run integration tests against this PR, add the ready for testing label.

[github-actions]

This PR was marked stale due to lack of activity.

[github-actions]

This PR was marked stale due to lack of activity.

[github-actions]

This PR was marked stale due to lack of activity.

[github-actions]

This PR was marked stale due to lack of activity.

[movence]

.

[movence]

We detect glob syntax (*, ?, [) here but then emit the value straight into IsMatch(attributes["hostname"], "web-*"). OTTL's IsMatch compiles that as an RE2 regular expression, not a shell glob — so web-* matches "we" followed by zero-or-more "b"s, and something like host.domain-* treats the . as "any character." Customers writing what looks like a glob will get surprising routing.

Two ways to go: translate the glob into an anchored regex (web-* → ^web-.*$, escaping the non-glob metacharacters), or drop the glob detection and document these fields as regex. Either is fine — just want the behavior to match what a user would expect from the config.

[movence]

escapeOTTL escapes backslashes and double-quotes, which stops a value from breaking out of the string literal — but any other regex metacharacters in a hostname, app-name, or filter expression still flow through into IsMatch. Since these values end up in a regex, I'd escape the full metacharacter set (or regexp.QuoteMeta the user input before interpolating) so a stray character can't silently change what a rule matches. This overlaps with the glob comment above — handling it once at the point we build the condition string can cover both.

[movence]

splitHostPort strips the brackets off an IPv6 host, but this reassembly puts it back as proto://host:port without them — so tcp://[::1]:514 comes out as tcp://::1:514, which is ambiguous/invalid for net.Listen. net.JoinHostPort(host, port) on the way out handles the bracketing correctly. Worth updating the translator_test.go case too — it currently asserts the unbracketed form, so it's locking in the behavior.

[movence]

This uniqueness check compares the listen_address strings before resolveListenAddress applies defaulting, so two entries that resolve to the same bind slip past it — e.g. tcp:// vs tcp://127.0.0.1:5514, or tcp://0.0.0.0:514 vs tcp://:514. They'd only collide later at bind time. Resolving each address first and keying seen on the canonical form would catch these at config time with the nice error message you already have here.

[movence]

syslogListenerDefinition has no required array, so a {"listeners": [{}]} entry passes schema validation and then fails later at translate time on the empty address. Adding "required": ["listen_address"] here (or applying the same defaulting the top-level shorthand uses) would surface it up front with a clearer message.

[movence]

RoundTrip sets headers on the incoming *http.Request. The http.RoundTripper contract says the transport shouldn't modify the request, and this can race under retries / HTTP-2 where the same request may be in flight more than once. Cloning first (req = req.Clone(req.Context())) and setting headers on the copy keeps it safe. I know this path is inactive today, but it'll bite once OTLP mode is on.

[movence]

This Config has no Region or LogsProvisionTimeout, but Start() calls cfg.Validate() first, which returns "region is required" — so require.NoError on the Start call below would fail as written. Same setup in TestRoundTripper_MissingAdditionalAuth. Setting Region + a positive LogsProvisionTimeout on the config should fix both. Separately, metadata.yaml doesn't define a tests.config, which the generated lifecycle test needs to build a valid config.