Docs » Get started with Splunk APM » Splunk APM architecture overview

Splunk APM architecture overview 🔗

Important

The original µAPM product, released in 2019, is now called µAPM Previous Generation (µAPM PG). Wherever you see the reference, µAPM now refers to the product released on March 31, 2020.

If you’re using µAPM Previous Generation (µAPM PG), see Overview of SignalFx Microservices APM Previous Generation (µAPM PG).

Note

SignalFx Smart Agent is deprecated. For details, see the Deprecation Notice.

Set up your environment to send trace data from an application to Splunk APM. The best way to get trace data in is to deploy the Smart Agent to process and manage trace data an application exports before being routed to APM. Optionally, configure the Smart Agent to send data to an OpenTelemetry Collector to centrally manage trace context and trace volume.

Deploy the Smart Agent in the same instance that is running the application you want to instrument. Configure the application to send data to the Smart Agent. If you deploy an OpenTelemetry Collector, you can send trace data from multiple Smart Agents to an OpenTelemetry Collector. Use an OpenTelemetry Collector to centrally add or modify span tags, including overriding existing span tags or obfuscating trace attributes, and to monitor trace volume from instrumented applications.

If you can’t deploy the Smart Agent in the same instance that runs an application, you may be able to deploy an OpenTelemetry Collector to process and manage trace data from the application. For example, with a serverless application running in a VPC, you can deploy an OpenTelementry Collector in the VPC. Just keep in mind that without the Smart Agent you can’t enable host correlation.

When possible, use the Splunk APM (SAPM) span format to send traces to APM. You can also use Jaeger or Zipkin formats, particularly if you have existing custom instrumentation that already exports data in these formats. For more information about compatible span formats, see Compatible span formats for Splunk APM.

To configure the Smart Agent for APM, see Deploy a SignalFx Smart Agent for Splunk APM.

Optionally, to configure an OpenTelemetry Collector for APM, see Deploy an OpenTelemetry Collector for Splunk APM.

Deploy the Smart Agent for Splunk APM 🔗

This is the most common deployment architecture.

Configure an application you want to instrument to send data to the Smart Agent. The Smart Agent then exports trace data directly to an ingest endpoint. the Smart Agent enables host correlation and can include span tags to trace data that’s specific to the instrumented application.

The Smart Agent uses two components to receive, process, and export data:

Component Description
signalfx-forwarder monitor Receive trace data from an instrumented application. Optionally, the monitor can add span tags that provide additional context to traces and spans. By default, the Smart Agent listen address is localhost:9080/v1/trace".
SAPM writer Exports data to APM. By default, the writer exports trace data to "https://ingest.REALM.signalfx.com/v2/trace".

This diagram illustrates a deployment with the Smart Agent running in the same instance as an instrumented application that sends data to APM:

../../_images/apm-architecture-sa.png

Deploy the Smart Agent and OpenTelemetry Collector for Splunk APM 🔗

The most customizable deployment architecture for APM is when the Smart Agent runs in the same instance of an instrumented application and includes an OpenTelemetry Collector.

The Smart Agent enables host correlation and can include span tags to trace data that’s specific to the instrumented application. The OpenTelemetry Collector allows you to centrally manage span tags for trace data it receives from multiple Smart Agents.

Configure an application to send traces to the Smart Agent when you configure instrumentation. Exactly how you do this depends on the following:

  • The programming language of the code you’re instrumenting.
  • The way you configured instrumentation. You can use an instrumentation library or configure instrumentation manually.

If you use an instrumentation library, you can set the SIGNALFX_ENDPOINT_URL environment variable to point to the Smart Agent before running your application.

The Smart Agent uses two components to receive, process, and export data:

Component Description
signalfx-forwarder monitor Receives trace data from an instrumented application. Optionally, the monitor can add span tags that provide additional context to traces and spans. By default, the Smart Agent listen address is localhost:9080/v1/trace".
SAPM writer Exports data to an OpenTelemetry Collector. By default, the writer exports trace data to "0.0.0.0:7276/v2/trace".

An OpenTelemetry Collector uses three components to receive, process, and export data:

Component Description
SAPM receiver Receives trace data from the Smart Agent.
attributes processor Adds span tags that provide additional context to traces and spans.
SAPM exporter Passes an access token to your organization and sends trace data to APM. By default, the exporter sends trace data to "https://ingest.REALM.signalfx.com/v2/trace".

This diagram illustrates a deployment with the Smart Agent running in the same instance as an instrumented application that sends data to an OpenTelemetry Collector for additional span tag processing before sending everything to APM:

../../_images/apm-architecture-sa-otel.png