Troubleshoot Database Query Performance π
Follow the steps in the following sections to troubleshoot Database Query Performance issues:
New query data not processed because cardinality limits have been exceeded
Database issues are not originating with problematic queries
Database Query Performance is not enabled π
If you open Database Query Performance and notice the feature is not enabled, go to MetricSets Configuration and check if indexing for Database Query Performance tags is ACTIVE
. If itβs not, see Enable Database Query Performance for instructions to enable indexing for Database Query Performance tags.
No supported databases available π
If you open Database Query Performance and notice there are no supported databases available, check the following:
Ensure you have one or more of the types of SQL databases listed in Supported databases in your system.
Try adjusting the Environment, Workflow, Service, or Tag filters in the filter bar to bring databases into view.
Ensure you are using the most recent instrumentation library version to make sure the database-related span tags are being ingested. If necessary, upgrade the instrumentation library youβre using.
New query data not processed because cardinality limits have been exceeded π
Enabling Database Query Performance turns on indexing for a set of 5 database-related span tags. These tags count toward cardinality limits for indexing span tags.
When the limit is exceeded across all indexed tags in your account, Splunk APM temporarily pauses indexing all Database Query Performance tags to allow for transient spikes in cardinality and manage billing in your account. After 15 minutes, Splunk APM attempts to restart indexing Database Query Performance tags automatically.
If youβre having persistent cardinality issues, try turning off indexing for other high-cardinality span tags to free up cardinality for Database Query Performance. See Limits on indexing span tags to learn more about managing cardinality.
Database issues are not originating with problematic queries π
If you are seeing database latency, but the query itself is not causing the issue, the problem might be a slow connection or a bottleneck waiting for a queue of database queries. To identify these kinds of issues, you need to instrument the database itself. For instructions on instrumenting a SQL database, see the following links:
You can also use the Related Content tiles to pivot to Infrastructure Monitoring or Log Observer and further analyze database issues. See Related Content in Splunk Observability Cloud to learn more about Related Content.
Disable database query normalization π
By default, Splunk APM instrumentation sanitizes database queries to remove or mask sensible data, such as secrets or personal identifiable information (PII).
If you want to disable database query normalization, follow these steps:
Java: Set the
otel.instrumentation.common.db-statement-sanitizer.enabled
property tofalse
.Ruby: Set the
db_statement
setting to:include
.
Caution
Disabling normalization might result in personal identifiable information (PII) and secrets appearing in Observability Cloud.
Learn more π
See the following links for more information about Database Query Performance:
For an overview of Database Query Performance, see Monitor Database Query Performance.
To enable Database Query Performance, see Enable Database Query Performance.
For a detailed use case using Database Query Performance, see Scenario: Jax identifies slow database queries using Database Query Performance.
For reference material about Database Query Performance, see Reference for Database Query Performance.