[Playground] OTLP-Mapper - instana to otlp span mapper

[abhilash-sivan]

UI: https://instana.io/s/B2R0pfHbSpaV5jXmQNDd7w

This is just a playground where we convert Instana spans to Otel spans and send them to the Otel backend.

INSTANA_OTLP_FORMAT=true SERVICE_NAME=test-nodejs-service INSTANA_DEBUG=true node index.js

• handle multiple keys in span.data
• approve mapping design
• approve port problem
• approve agent announcement cycle disable use case
• test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js. We have to add metrics mapping. Discussed/Agreed with community.
• figure out: node.http.server - what is the official otel name/value so that the backend is labeling everything correctly in the UI? (prio 1)
• Performance check classes vs functions

https://instana.io/s/Ehe2Zhb7QyKyxrmQg0fK8Q The BE uses path_tpl or url values ot show in the UI, so we mapped the otel span name to the same data from Instana span.

After conversion using the script:

1. Instana kafka trace to otlp

{
  "attributes": [
    {
      "key": "messaging.system",
      "value": {
        "stringValue": "kafka"
      }
    },
    {
      "key": "messaging.destination.name",
      "value": {
        "stringValue": "order-events"
      }
    },
    {
      "key": "messaging.operation.type",
      "value": {
        "stringValue": "send"
      }
    },
    {
      "key": "messaging.kafka.destination.partition",
      "value": {
        "intValue": 0
      }
    },
    {
      "key": "messaging.kafka.message.offset",
      "value": {
        "intValue": 12345
      }
    },
    {
      "key": "messaging.kafka.message.key",
      "value": {
        "stringValue": "order-123"
      }
    },
    {
      "key": "service",
      "value": {
        "stringValue": "order-service"
      }
    }
  ],
  "events": [],
  "links": [],
  "resource": {
    "attributes": [
      {
        "key": "service.name",
        "value": {
          "stringValue": "order-service"
        }
      },
      {
        "key": "telemetry.sdk.language",
        "value": {
          "stringValue": "nodejs"
        }
      },
      {
        "key": "telemetry.sdk.name",
        "value": {
          "stringValue": "@instana/collector"
        }
      },
      {
        "key": "telemetry.sdk.version",
        "value": {
          "stringValue": "3.0.0"
        }
      }
    ]
  },
  "traceId": "00000000000000000000xyz789abc123",
  "spanId": "000000000span002",
  "parentSpanId": "0000000parent002",
  "startTimeUnixNano": "1716019300000000000",
  "endTimeUnixNano": "1716019300050000000",
  "status": {
    "code": 1
  },
  "name": "send order-events"
}

[abhilash-sivan]

This is where the conversion is happening.

This seems to be the optimal place because if we convert early, then in some places we talk Instana format and it could raise unexpected issues. So before transmission seems simple and maintanable

[aryamohanan]

Should we do this here or just after the current BE transformation,
https://github.com/instana/nodejs/pull/2534/changes?utm_source=chatgpt.com#diff-462ace70d7ac6eb4cd6f82756059c88629bfad1463fde0498d89106d7ccc8104L193

Right now we’re applying the transformation after collecting all spans. IMO it would be cleaner to do it before calling transmitSpans, so the transmit layer only deals with already-transformed spans.

TBD

[abhilash-sivan]

Yeah, I agree. This would be a more cleaner way

// Transform internal span data format into external (backend) readable format.
span = applySpanTransformation(span);

Just after this transformation, we can call the Otel transformation.

[abhilash-sivan]

One thing we have to consider is that currently all the spans intended for transmission — i.e., the array of spans — are converted together. If we do it in the above-mentioned area, we will have to process the spans individually.

[abhilash-sivan]

Note: TBD — Transform each span to OTLP format immediately in addSpan and store them in OTLP format in the buffer if batching is enabled. This would require changes to the batching logic.

[kirrg001]

How do you want to organize the agent ports etc?

OTLP mode is on -> port will change.
There is something missing.

[kirrg001]

Also:
How can we skip the agent announcement cycle when connecting to an otel collector later?
Did you draw that path as well to confirm the architecture?

[abhilash-sivan]

No, I haven’t worked much on the architecture yet. I can work on the dynamic port connections part, and I’ll also think through the agent announcement cycle and how we can skip it when connecting to an OTel collector

[kirrg001]

test: send traces via otlp port and metrics via standard port - Idenfies 2 diff apps 1. Otel and 2. Node.js

This needs to be communicated in the next WG. Traces & Metrics mapping is required.

[abhilash-sivan]

Not verified rabbitmq per se. It's just to show how we differentiate children under messaging or any other group.

[kirrg001]

Why do we need that here? If we have already a custom kafka mapping object?

[abhilash-sivan]

messaging.system is an additional attribute that we append to the span. For Kafka, the system name also happens to be kafka, so it may seem redundant at first glance.

However, additionalAttributes is intended for constant or computed attributes that do not directly map from a single field inside span.data.kafka.

messaging.system falls into that category since it is effectively a constant defined per transformer, rather than something extracted from the incoming span payload itself.

I have changed this to: 'messaging.system': this.systemName

which will fetch the appropriate value in this case

[kirrg001]

Override? Where was it set in the first place?

[kirrg001]

Can't we just re-use getTransformer?
Just with a different level?

[kirrg001]

This does not belong here or?