Docs » Drill down into service performance with indexed tags and MetricSets in Splunk APM » Add span tags to services in Splunk APM

Add span tags to services in Splunk APM 🔗

Important

The original µAPM product, released in 2019, is now called µAPM Previous Generation (µAPM PG). Wherever you see the reference, µAPM now refers to the product released on March 31, 2020.

If you’re using µAPM Previous Generation (µAPM PG), see Overview of SignalFx Microservices APM Previous Generation (µAPM PG).

There are a few ways to define span tags, depending on your Splunk APM deployment architecture. You can define span tags when you’re instrumenting code, when you’re configuring a Smart Agent, or when you’re configuring an OpenTelemetry Collector.

You can also define some span tags at the instrumentation or Smart Agent level, and centrally manage other span tags with an OpenTelemetry Collector.

Add span tags at the instrumentation level 🔗

How you add span tags to spans depends on your code’s language. Some languages support adding span tags with environment variables or system properties while others require that you specify span tags in your code.

For more information about adding span tags at the instrumentation level, see the instrumentation library for each supported language on GitHub:

Add span tags at the Smart Agent level 🔗

Include span tags in settings for the signalfx-forwarder monitor in your Smart Agent configuration YAML file. You can add span tags as defaultSpanTags which adds span tags to any spans that don’t already have the tags or as extraSpanTags which overrides an existing span tag value.

The settings look like this in the Smart Agent configuration YAML file:

monitors:
  - type: signalfx-forwarder
    listenAddress: 0.0.0.0:9080
    # Add a tag to spans missing it.
    defaultSpanTags:
     SPAN_TAG_KEY: "SPAN_TAG_VALUE"
    # Override an existing tag for a span.
    extraSpanTags:
     SPAN_TAG_KEY: "SPAN_TAG_VALUE"

For more information about configuring a Smart Agent, see Deploy a SignalFx Smart Agent for Splunk APM.

Add span tags at the OpenTelemetry Collector level 🔗

Include span tags in settings for the batch processor in your OpenTelemetry Collector configuration YAML file. You can add span tags as attributes/newenvironment which adds span tags to any spans that don’t already have the tags or attributes/copyfromexistingkey which overrides an existing span tag value.

The settings look like this in the OpenTelemetry Collector configuration YAML file:

processors:
  # Override an existing tag for a span.
  attributes/copyfromexistingkey:
    actions:
    - key: SPAN_TAG_KEY
      from_attribute: "SPAN_TAG_VALUE"
      action: upsert
  # Add a tag to spans missing it.
  attributes/newenvironment:
    actions:
    - key: SPAN_TAG_KEY
      value: "SPAN_TAG_VALUE"
      action: insert

If you include the batch processor, you have to add the processor to your pipelines:

service:
  pipelines:
    traces:
      processors: [batch]

For more information about configuring an OpenTelemetry Collector, see Deploy an OpenTelemetry Collector for Splunk APM.