feat: Add support for reading OTel ProcessContext from profiled process memory

[nsavoire]

Implement reading of OpenTelemetry ProcessContext (OTEP #4719) from profiled processes via shared memory regions. The SDK publishes resource attributes into a named memory mapping ([anon:OTEL_CTX] or /memfd:OTEL_CTX) with a 32-byte header containing signature, version, payload size, timestamp, and a pointer to a protobuf-encoded payload.

Changes:

• Define ProcessContext protobuf message (resource + extra_attributes)
• Add processcontext.go to read and validate the shared memory header, deserialize the protobuf payload, and handle concurrent update retries
• Recognize OTEL_CTX mappings in parseMappings and store ProcessContextInfo in ProcessMeta
• Integrate ProcessContext reading into processmanager's synchronizeMappings
• Add unit tests with mock reader and integration tests using real named anonymous and memfd mappings

This PR only addresses reading process context but does yet propagate it to profile.
The goal is to gather early feedback.

Implementation notes:

• Currently process context read is attempted upon each call to synchronizeMappings. If process context is published later on, then it might be missed if no other call to synchronizeMappings occurs. A possibility would be to hook prctl calls to trigger process context reads.
• If process context read fails because a concurrent update is detected, then read is retried up to 3 times. If it still fails, for now there is no mechanism to schedule a future read attempt. Note that implementing a prctl hook would also solve this by triggering an update later on.

Remaining work:

• Merge resource attributes between process context / process env variables and containerID
• Split profiles by resource attributes
• Implement reading thread context on top of process context

[ivoanjo]

Did a quick pass! Exciting to see this here!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5bb52d716f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[christos68k]

Did one pass, have a few more things to look at / test

[ivoanjo]

👍 This looks great! I like that although the spec itself has some complex + subtle details, the implementation ends up looking quite sane and mostly straightforward

[rogercoll]

Overall lgtm, I will do another review once the ProcessContextInfo is forwarded down the pipeline

[rogercoll]

Does this need to be exported?

[nsavoire]

Read() takes maxRetries as a parameter (per @christos68k 's suggestion), so this provides a default value for external callers (e.g. processmanager/processinfo.go). Alternatively, I could use the zero value to indicate "use the default" and keep the constant unexported. What do you prefer?

[rogercoll]

IMO since no caller needs a different value, just drop the parameter and use a non exported constant internally (e.g defaultMaxRetries). If we need/expect other callers to set a different value for the retries, wdyt if we use the Options pattern instead? (or we could add the Options later as being non api breaking) E.g:

  type options struct {
      maxRetries int
  }
  type Option func(*options)
  func WithMaxRetries(n int) Option {
      return func(o *options) { o.maxRetries = n }
  }
  func Read(addr libpf.Address, rm remotememory.RemoteMemory, lastPublishedAtNs
  uint64, opts ...Option) (Info, error) {

cc @christos68k

[nsavoire]

Have you a preference @christos68k ?

[christos68k]

I'd go for the simpler option of using 0 as default, allowing for caller overrides, unless we know we'll have more than one options in the future.

[nsavoire]

Just noticed that I called the parameter maxRetries but it is implemented in fact as maxAttempts.
I renamed it, and when passed 0, it uses the default number of read attempts.

[ivoanjo]

(I meant to approve on my previous pass but pressed comment instead of approve)

[rogercoll]

Overall the code LGTM, but the ProcessContextInfo it's never consumed downstream. This makes the processcontext feature effectively a no-op in the current changeset. Is there a follow-up PR planned to wire ProcessContextInfo through the tracer, reporter and pdata path?

[nsavoire]

Overall the code LGTM, but the ProcessContextInfo it's never consumed downstream. This makes the processcontext feature effectively a no-op in the current changeset. Is there a follow-up PR planned to wire ProcessContextInfo through the tracer, reporter and pdata path?

Indeed this PR addresses only detecting/reading process context. I have a follow-up PR planned once this PR is merged (here in draft state).

[nsavoire]

Thanks for the feedback ! I think I have addressed most of the comments, let me know what I can do to progress on this PR.

Thanks for doing the work @nsavoire. I think the PR is in a good state right now. Some notes:

• As we discussed in the SIG meeting, I don't want to merge it until the OTEP is merged first (so I'll wait until that happens before I approve this)
• It'd be nice to get an approval from @ivoanjo and also @florianl
• Some changes will be needed once Feat: filter unneeded mappings in SynchronizeProcess #1267 is merged (which will probably happen before this PR is merged)

Hi @christos68k, some of the blockers you mentioned should now be resolved:

• The OTEP has been merged
• Feat: filter unneeded mappings in SynchronizeProcess #1267 has been merged as well (I've adapted the PR accordingly)
• Got an approval from @ivoanjo, waiting for approval from @florianl

Let me know if there's anything else needed from my side to move this forward !

[christos68k]

Final batch of nits

[christos68k]

Suggested change

	"fmt"
	"fmt"
	"regexp"

[christos68k]

LGTM

We'll push for open-telemetry/opentelemetry-proto#783 merge today (so I won't merge this right now PR still needs one more approval from an approver/maintainer).

[florianl]

It seems like we don't get open-telemetry/opentelemetry-proto#783 merged soonish. So I have just minor nits left and I think we should merge and continue from here.