Kafka Connect: Include connector name in coordinator and committer thread names#16355
Open
soumyajit-sahu wants to merge 1 commit into
Open
Conversation
soumyajit-sahu
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When multiple Iceberg sink connectors run in the same Kafka Connect worker process, all
coordinator threads share the same name
iceberg-coordand all committer threads sharenames like
iceberg-committer-0,iceberg-committer-1, etc.This makes log-based debugging difficult: when a coordinator fails to commit or a committer
thread throws an error, the log output gives no indication of which connector is responsible.
Proposed Change
Include the connector name in the thread names:
CoordinatorThread: rename fromiceberg-coord→iceberg-coord-<connectorName>ThreadPoolExecutorinsideCoordinator: rename fromiceberg-committer-%d→iceberg-committer-<connectorName>-%dImpact
With this change, operators running multiple connectors on the same worker can immediately
identify which connector's coordinator or committer thread is present in a log line, without
needing additional context or MDC instrumentation.
Closes #16354