Docs » µAPM Instrumentation Guide » Instrumentation Overview

Instrumentation Overview

Instrumentation is the process through which your applications’ code is extended to capture and report trace spans for the operations of interest in the path of handling a particular user request or transaction. Depending on which language(s) your applications are written in, you may benefit from automatic instrumentation (“auto-instrumentation”) capabilities. Additionally, or in lieu of, you can manually instrument the areas of your code and business logic that matter to you the most.

Automatic instrumentation


Automatic instrumentation is currently in beta for all supported languages.


Automatic instrumentation, also known as auto-instrumentation, is the process through which a run-time automated process can automatically identify which frameworks and libraries are in use within your application and instrument those libraries to capture tracing instrumentation – all without having to make any code change.

SignalFx Microservices APM offers automatic instrumentation (currently in beta) for the following programming languages:

Note that it is possible to extend the scope and coverage of the traces and spans captured by auto-instrumentation using manual, custom instrumentation. All auto-instrumentation techniques listed above will automatically provide a pre-configured, OpenTracing-compatible tracer that you can re-use in other parts of your application where you want to add instrumentation.

Manual instrumentation

Manual, or custom, instrumentation, consists in modifying your application code to selectively capture trace spans and relevant span metadata tags around the blocks of code that you want to instrument, trace, and gain visibility on. This is typically a low-to-medium effort by software developers, akin to capturing custom metrics albeit with a different set of caveats and pitfalls to pay attention to.

To avoid vendor lock-in, including lock-in to a particular open-source tracing solution, we recommend that custom instrumentation be implemented using the OpenTracing APIs. OpenTracing is a CNCF project and is rapidly becoming one of the de-facto standards for code instrumentation, offering clean APIs in a growing spectrum of programming languages. We then recommend the use of the corresponding Jaeger tracer client library that implements the OpenTracing APIs and will be responsible for reporting spans to SignalFx.

For more information on how to instrument in your language, read one of our language-specific guides below or check out our tracing examples.

Planning instrumentation for distributed tracing

For customers who have not yet instrumented their applications, our default recommendation is to use OpenTracing to instrument and to use Jaeger tracer libraries to implement the OpenTracing APIs and report spans to SignalFx (via the Smart Agent and Smart Gateway). OpenTracing has a rich ecosystem of instrumentations for most often used libraries and frameworks in a growing spectrum of programming languages. Both OpenTracing and Jaeger are active CNCF projects with a rapidly growing community behind them.

RPC layer(s) and service/web framework(s) are the best places to start when thinking about how to go about instrumenting your application. Both of these will likely have a large coverage area and touch a significant number of transaction paths to give you baseline tracing coverage and visualize end-to-end traces and dependencies between your services.

Next, you should identify services critical to your business and look for areas not covered by RPC or service/web frameworks. Within these services, identify high value transactions and critical paths and start by instrumenting these.

The information listed below can provide more suggestions and food for thought.

Instrumentation guidelines and limitations

Span names

  • Span name must be present and be a valid dimension value
  • Span local endpoint service name must be present and be a valid dimension value
  • Span names must be low-cardinality; variable components should be recorded as tags