The docs have moved!
Documentation on this site may be out of date. For the latest content, see the Splunk Observability Cloud documentation.
MongoDB 🔗
DESCRIPTION 🔗
This integration primarily consists of the Smart Agent monitor collectd/mongodb.
Below is an overview of that monitor.
Smart Agent Monitor 🔗
Monitors an instance of MongoDB using the collectd MongoDB Python plugin. Requires MongoDB 2.6 or later.
This monitor captures the following metrics about MongoDB generally:
- memory
- network input/output bytes count
- heap usage
- db connections
- operations count
- active client connections
- queued operations
The plugin also captures the following DB-specific metrics:
- db size
- db counters
Documentation for MongoDB can be found here.
Creating a MongoDB user for collectd 🔗
If you’re monitoring a secured MongoDB deployment, it is a good practice to create a MongoDB user with minimal read-only roles, as follows:
db.createUser( {
user: "collectd",
pwd: "collectd",
roles: [ { role: "readAnyDatabase", db: "admin" }, { role: "clusterMonitor", db: "admin" } ]
});
INSTALLATION 🔗
This integration is part of the SignalFx Smart Agent
as the collectd/mongodb monitor. You should first deploy the Smart Agent to the
same host as the service you want to monitor, and then continue with the
configuration instructions below.
CONFIGURATION 🔗
To activate this monitor in the Smart Agent, add the following to your agent config:
monitors: # All monitor config goes under this key
- type: collectd/mongodb
... # Additional config
For a list of monitor options that are common to all monitors, see Common Configuration.
| Config option | Required | Type | Description |
|---|---|---|---|
pythonBinary |
no | string |
Path to a python binary that should be used to execute the Python code. If not set, a built-in runtime will be used. Can include arguments to the binary as well. |
host |
yes | string |
Host name/IP address of the Mongo instance |
port |
yes | integer |
Port of the Mongo instance (default: 27017) |
databases |
yes | list of strings |
Name(s) of database(s) that you would like metrics from. Note: the first database in this list must be "admin", as it is used to perform a serverStatus() command. |
username |
no | string |
The MongoDB user to connect as |
password |
no | string |
The password of the above user |
useTLS |
no | bool |
If true, will connect to Mongo using TLS (default: false) |
caCerts |
no | string |
Path to a CA cert that will be used to verify the certificate that Mongo presents (not needed if not using TLS or if Mongo's cert is signed by a globally trusted issuer already installed in the default location on your OS) |
tlsClientCert |
no | string |
Path to a client certificate (not needed unless your Mongo instance requires x509 client verification) |
tlsClientKey |
no | string |
Path to a client certificate key (not needed unless your Mongo instance requires x509 client verification, or if your client cert above has the key included) |
tlsClientKeyPassPhrase |
no | string |
Passphrase for the TLSClientKey above |
sendCollectionMetrics |
no | bool |
Whether to send collection level metrics or not (default: false) |
sendCollectionTopMetrics |
no | bool |
Whether to send collection level top (timing) metrics or not (default: false) |
USAGE 🔗
Below are screen captures of dashboards created for this plugin by SignalFx, illustrating the metrics emitted by this plugin.
For general reference on how to monitor MongoDB performance, see Analyzing MongoDB Performance.
Monitoring MongoDB clusters
Writes to MongoDB require the use of the global lock. If lock utilization is high, operations can begin to slow down. This can be a symptom of database issues such as poorly configured or absent indexes, or a schema design that needs improvement. It can also indicate the failure of a disk. Monitor the number of readers and writers waiting for the lock with metric gauge.globalLock.currentQueue.total.
lock queueThis lock has little utilization and few queued readers and writers.
MongoDB flushes data changes from memory to disk on a timed interval, by default every 60 seconds. If background flushes begin taking longer than usual, it can indicate that the disk doesn’t have enough I/O capacity to handle the load (read more below). It could also reflect a large number of writes occurring at once – check counter.opcounters.insert and counter.opcounters.update.
Monitor average background flush time and the most recent background flush time with gauge.backgroundFlushing.average_ms and gauge.backgroundFlushing.last_ms respectively.
background flush timeAverage background flush time on this cluster is around 40ms, well within healthy parameters.
When analyzing the performance of a MongoDB cluster, it’s also important to verify that the load is balanced across each instance. The cluster dashboard included in this repository contains many list charts of individual MongoDB instances ordered by important metrics like requests per second (counter.network.numRequests) and number of connections to MongoDB (gauge.connections.current). This can help you compare load between instances. Load imbalance can arise in a sharded cluster if MongoDB is unable to balance chunks equally between the shards, for example if lock utilization is high.
top hosts by requests and connections
All the listed instances show about the same requests per second and number of connections. Their load is balanced.
Monitoring MongoDB hosts
On an individual instance level, it’s important to monitor system statistics like memory usage, page faults, and disk I/O utilization.
MongoDB uses memory-mapped files to store data, so it is important to compare the amount of memory that MongoDB has allocated to the amount of system memory. This plugin reports resident memory usage in gauge.mem.resident and mapped memory usage in gauge.mem.mapped. If either of these quantities exceed the amount of system memory (reported by the memory plugin for collectd), the system may be at or over capacity.
Memory statistics from MongoDB
This MongoDB instance is not using a large amount of resident memory, and has non-mapped memory available to the process (calculated as gauge.mem.virtual - gauge.mem.mapped.)
This plugin reports page faults in counter.extra_info.page_faults. Page faults indicate that reads or writes are occurring to data files that are not currently in memory. This is different from an OS page fault. Sudden increases in MongoDB page faults can indicate that a large read operation is taking place. Steadily high numbers of page faults indicate that MongoDB is reading more often from disk than is optimal.
Page fault statistics from MongoDB
This MongoDB instance has a low rate of page faults. This means that most of the data MongoDB needs to access is in memory, and doesn’t need to be fetched from disk.
You can monitor disk I/O utilization for your MongoDB host using the disk_ops.write and disk_ops.read metrics emitted by the disk plugin for collectd, which is included and enabled by default in most packages of collectd. Click here to learn more about the collectd-disk plugin.
METRICS 🔗
| Metric Name | Description | Type |
|---|---|---|
| counter.asserts.regular | The number of regular assertions raised since the MongoDB process started | cumulative |
| counter.asserts.warning | In MongoDB 3.x and earlier, the field returns the number of warnings raised since the MongoDB process started | cumulative |
| counter.backgroundFlushing.flushes | Number of times the database has been flushed | gauge |
| counter.collection.commandsCount | Number of commands issued for a collection | counter |
| counter.collection.commandsTime | Time spent in microseconds processing commands issued for a collection | counter |
| counter.collection.getmoreCount | Number of getMore requests issued for a collection | counter |
| counter.collection.getmoreTime | Time spent in microseconds processing getMore requests for a collection | counter |
| counter.collection.index.accesses.ops | Number of times an index has been used (only on Mongo 3.2+) | counter |
| counter.collection.insertCount | Number of inserts issued for a collection | counter |
| counter.collection.insertTime | Time spent in microseconds processing insert requests for a collection | counter |
| counter.collection.queriesCount | Number of queries issued for a collection | counter |
| counter.collection.queriesTime | Time spent in microseconds processing query requests for a collection | counter |
| counter.collection.readLockCount | Number of read locks issued for a collection | counter |
| counter.collection.readLockTime | Time spent in microseconds processing read locks for a collection | counter |
| counter.collection.removeCount | Number of remove requests issued for a collection | counter |
| counter.collection.removeTime | Time spent in microseconds processing remove requests for a collection | counter |
| counter.collection.totalCount | Total number of operations issued for a collection | counter |
| counter.collection.totalTime | Time spent in microseconds processing all operations for a collection | counter |
| counter.collection.updateCount | Number of update requests issued for a collection | counter |
| counter.collection.updateTime | Time spent in microseconds processing update requests for a collection | counter |
| counter.collection.writeLockCount | Number of write locks issued for a collection | counter |
| counter.collection.writeLockTime | Time spent in microseconds processing write locks for a collection | counter |
| counter.extra_info.page_faults | Mongod page faults | gauge |
| counter.lock.Database.acquireCount.intentExclusive | cumulative | |
| counter.lock.Database.acquireCount.intentShared | cumulative | |
| counter.lock.Database.acquireCount.read | cumulative | |
| counter.lock.Database.acquireCount.write | cumulative | |
| counter.lock.Global.acquireCount.intentExclusive | cumulative | |
| counter.lock.Global.acquireCount.intentShared | cumulative | |
| counter.lock.Global.acquireCount.write | cumulative | |
| counter.network.bytesIn | Network bytes received by the database server | gauge |
| counter.network.bytesOut | Network bytes sent by the database server | gauge |
| counter.network.numRequests | Requests received by the server | cumulative |
| counter.opcounters.command | Number of commands since last restart | cumulative |
| counter.opcounters.delete | Number of deletes since last restart | cumulative |
| counter.opcounters.getmore | Number of getmore operations since last restart | cumulative |
| counter.opcounters.insert | Number of inserts since last restart | cumulative |
| counter.opcounters.query | Number of queries since last restart | cumulative |
| counter.opcounters.update | Number of updates since last restart | cumulative |
| counter.opcountersRepl.command | Number of replicated commands since last restart | cumulative |
| counter.opcountersRepl.delete | Number of replicated deletes since last restart | cumulative |
| counter.opcountersRepl.getmore | Number of replicated getmore operations since last restart | cumulative |
| counter.opcountersRepl.insert | Number of replicated inserts since last restart | cumulative |
| counter.opcountersRepl.query | Number of replicated queries since last restart | cumulative |
| counter.opcountersRepl.update | Number of replicated updates since last restart | cumulative |
| gauge.backgroundFlushing.average_ms | Average time (ms) to write data to disk | gauge |
| gauge.backgroundFlushing.last_ms | Most recent time (ms) spent writing data to disk | gauge |
| gauge.collection.avgObjSize | Mean object/document size of a collection | gauge |
| gauge.collection.count | Number of objects/documents in a collection | gauge |
| gauge.collection.indexSize | Size of a particular index on a collection | gauge |
| gauge.collection.max | Maximum number of documents in a capped collection | gauge |
| gauge.collection.maxSize | Maximum disk usage of a capped collection | gauge |
| gauge.collection.size | Size of a collection in bytes, not including indexes | gauge |
| gauge.collection.storageSize | Size of the collection on disk in bytes, never decreases | gauge |
| gauge.collections | Number of collections | gauge |
| gauge.connections.available | The number of unused incoming connections available | gauge |
| gauge.connections.current | The number of incoming connections from clients to the database server | gauge |
| gauge.connections.totalCreated | Count of all incoming connections created to the server | cumulative |
| gauge.dataSize | Total size of data, in bytes | gauge |
| gauge.extra_info.heap_usage_bytes | Heap size used by the mongod process, in bytes | gauge |
| gauge.globalLock.activeClients.readers | Number of active client connections performing reads | gauge |
| gauge.globalLock.activeClients.total | Total number of active client connections | gauge |
| gauge.globalLock.activeClients.writers | Number of active client connections performing writes | gauge |
| gauge.globalLock.currentQueue.readers | Read operations currently in queue | gauge |
| gauge.globalLock.currentQueue.total | Total operations currently in queue | gauge |
| gauge.globalLock.currentQueue.writers | Write operations currently in queue | gauge |
| gauge.indexSize | Total size of indexes, in bytes | gauge |
| gauge.indexes | Number of indexes across all collections | gauge |
| gauge.mem.mapped | Mongodb mapped memory usage, in MB | gauge |
| gauge.mem.resident | Mongodb resident memory usage, in MB | gauge |
| gauge.mem.virtual | Mongodb virtual memory usage, in MB | gauge |
| gauge.numExtents | gauge | |
| gauge.objects | Number of documents across all collections | gauge |
| gauge.repl.active_nodes | Number of healthy members in a replicaset (reporting 1 for health) | gauge |
| gauge.repl.is_primary_node | Report 1 when member state of replicaset is PRIMARY and 2 else |
gauge |
| gauge.repl.max_lag | Replica lag in seconds calculated from the difference between the | |
| timestamp of the last oplog entry of primary and secondary see mongo | ||
| doc | gauge | |
| gauge.storageSize | Total bytes allocated to collections for document storage | gauge |
| gauge.tcmalloc.generic.heap_size | Heap size used by the mongod process, in bytes | gauge |
| gauge.uptime | Uptime of this server in milliseconds | counter |
counter.asserts.regular 🔗
cumulative
The number of regular assertions raised since the MongoDB process started. Check the log file for more information about these messages.
counter.asserts.warning 🔗
cumulative
In MongoDB 3.x and earlier, the field returns the number of warnings raised since the MongoDB process started. In MongodDB 4, this is always 0.
counter.backgroundFlushing.flushes 🔗
gauge
Number of times the database has been flushed. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)
counter.collection.commandsTime 🔗
counter
Time spent in microseconds processing commands issued for a collection
counter.collection.getmoreTime 🔗
counter
Time spent in microseconds processing getMore requests for a collection
counter.collection.index.accesses.ops 🔗
counter
Number of times an index has been used (only on Mongo 3.2+)
counter.collection.insertTime 🔗
counter
Time spent in microseconds processing insert requests for a collection
counter.collection.queriesTime 🔗
counter
Time spent in microseconds processing query requests for a collection
counter.collection.readLockTime 🔗
counter
Time spent in microseconds processing read locks for a collection
counter.collection.removeTime 🔗
counter
Time spent in microseconds processing remove requests for a collection
counter.collection.totalTime 🔗
counter
Time spent in microseconds processing all operations for a collection
counter.collection.updateTime 🔗
counter
Time spent in microseconds processing update requests for a collection
counter.collection.writeLockTime 🔗
counter
Time spent in microseconds processing write locks for a collection
counter.lock.Database.acquireCount.intentExclusive 🔗
cumulative
counter.lock.Database.acquireCount.read 🔗
cumulative
counter.lock.Database.acquireCount.write 🔗
cumulative
counter.lock.Global.acquireCount.intentExclusive 🔗
cumulative
counter.lock.Global.acquireCount.write 🔗
cumulative
counter.opcountersRepl.getmore 🔗
cumulative
Number of replicated getmore operations since last restart
gauge.backgroundFlushing.average_ms 🔗
gauge
Average time (ms) to write data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)
gauge.backgroundFlushing.last_ms 🔗
gauge
Most recent time (ms) spent writing data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)
gauge.connections.available 🔗
gauge
The number of unused incoming connections available. Consider this value
in combination with the value of gauge.connections.current to
understand the connection load on the database.
gauge.connections.current 🔗
gauge
The number of incoming connections from clients to the database server.
gauge.connections.totalCreated 🔗
cumulative
Count of all incoming connections created to the server. This number includes connections that have since closed.
gauge.extra_info.heap_usage_bytes 🔗
gauge
Heap size used by the mongod process, in bytes. Deprecated in mongo version > 3.3, use gauge.tcmalloc.generic.heap_size instead.
gauge.globalLock.activeClients.writers 🔗
gauge
Number of active client connections performing writes
gauge.mem.mapped 🔗
gauge
Mongodb mapped memory usage, in MB. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)
gauge.numExtents 🔗
gauge
gauge.repl.max_lag 🔗
gauge
Replica lag in seconds calculated from the difference between the timestamp of the last oplog entry of primary and secondary see mongo doc.
gauge.tcmalloc.generic.heap_size 🔗
gauge
Heap size used by the mongod process, in bytes. Same as gauge.extra_info.heap_usage_bytes but supports 64-bit values.
gauge.uptime 🔗
counter
Uptime of this server in milliseconds
Metrics that are categorized as container/host (default) are in bold and italics in the list below.
These are the metrics available for this integration.
counter.asserts.regular(cumulative)
The number of regular assertions raised since the MongoDB process started. Check the log file for more information about these messages.counter.asserts.warning(cumulative)
In MongoDB 3.x and earlier, the field returns the number of warnings raised since the MongoDB process started. In MongodDB 4, this is always 0.counter.backgroundFlushing.flushes(gauge)
Number of times the database has been flushed. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)counter.extra_info.page_faults(gauge)
Mongod page faultscounter.lock.Database.acquireCount.intentExclusive(cumulative)counter.lock.Database.acquireCount.intentShared(cumulative)counter.lock.Database.acquireCount.read(cumulative)counter.lock.Database.acquireCount.write(cumulative)counter.lock.Global.acquireCount.intentExclusive(cumulative)counter.lock.Global.acquireCount.intentShared(cumulative)counter.lock.Global.acquireCount.write(cumulative)counter.network.bytesIn(gauge)
Network bytes received by the database servercounter.network.bytesOut(gauge)
Network bytes sent by the database servercounter.network.numRequests(cumulative)
Requests received by the servercounter.opcounters.command(cumulative)
Number of commands since last restartcounter.opcounters.delete(cumulative)
Number of deletes since last restartcounter.opcounters.getmore(cumulative)
Number of getmore operations since last restartcounter.opcounters.insert(cumulative)
Number of inserts since last restartcounter.opcounters.query(cumulative)
Number of queries since last restartcounter.opcounters.update(cumulative)
Number of updates since last restartcounter.opcountersRepl.command(cumulative)
Number of replicated commands since last restartcounter.opcountersRepl.delete(cumulative)
Number of replicated deletes since last restartcounter.opcountersRepl.getmore(cumulative)
Number of replicated getmore operations since last restartcounter.opcountersRepl.insert(cumulative)
Number of replicated inserts since last restartcounter.opcountersRepl.query(cumulative)
Number of replicated queries since last restartcounter.opcountersRepl.update(cumulative)
Number of replicated updates since last restartgauge.backgroundFlushing.average_ms(gauge)
Average time (ms) to write data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)gauge.backgroundFlushing.last_ms(gauge)
Most recent time (ms) spent writing data to disk. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)gauge.collection.max(gauge)
Maximum number of documents in a capped collectiongauge.collection.maxSize(gauge)
Maximum disk usage of a capped collectiongauge.collections(gauge)
Number of collectionsgauge.connections.available(gauge)
The number of unused incoming connections available. Consider this value in combination with the value ofgauge.connections.currentto understand the connection load on the database.gauge.connections.current(gauge)
The number of incoming connections from clients to the database server.gauge.connections.totalCreated(cumulative)
Count of all incoming connections created to the server. This number includes connections that have since closed.gauge.dataSize(gauge)
Total size of data, in bytesgauge.extra_info.heap_usage_bytes(gauge)
Heap size used by the mongod process, in bytes. Deprecated in mongo version > 3.3, use gauge.tcmalloc.generic.heap_size instead.gauge.globalLock.activeClients.readers(gauge)
Number of active client connections performing readsgauge.globalLock.activeClients.total(gauge)
Total number of active client connectionsgauge.globalLock.activeClients.writers(gauge)
Number of active client connections performing writesgauge.globalLock.currentQueue.readers(gauge)
Read operations currently in queuegauge.globalLock.currentQueue.total(gauge)
Total operations currently in queuegauge.globalLock.currentQueue.writers(gauge)
Write operations currently in queuegauge.indexSize(gauge)
Total size of indexes, in bytesgauge.indexes(gauge)
Number of indexes across all collectionsgauge.mem.mapped(gauge)
Mongodb mapped memory usage, in MB. Only available when MMAPv1 is enabled. (MMAPv1 is not supported in MongoDB version > 4.2)gauge.mem.resident(gauge)
Mongodb resident memory usage, in MBgauge.mem.virtual(gauge)
Mongodb virtual memory usage, in MBgauge.numExtents(gauge)gauge.objects(gauge)
Number of documents across all collectionsgauge.repl.active_nodes(gauge)
Number of healthy members in a replicaset (reporting 1 for health).gauge.repl.is_primary_node(gauge)
Report 1 when member state of replicaset isPRIMARYand 2 else.gauge.repl.max_lag(gauge)
Replica lag in seconds calculated from the difference between the timestamp of the last oplog entry of primary and secondary see mongo doc.gauge.storageSize(gauge)
Total bytes allocated to collections for document storagegauge.tcmalloc.generic.heap_size(gauge)
Heap size used by the mongod process, in bytes. Same as gauge.extra_info.heap_usage_bytes but supports 64-bit values.gauge.uptime(counter)
Uptime of this server in milliseconds
Group collection 🔗
All of the following metrics are part of the collection metric group. All of
the non-default metrics below can be turned on by adding collection to the
monitor config option extraGroups:
counter.collection.index.accesses.ops(counter)
Number of times an index has been used (only on Mongo 3.2+)gauge.collection.avgObjSize(gauge)
Mean object/document size of a collectiongauge.collection.count(gauge)
Number of objects/documents in a collectiongauge.collection.indexSize(gauge)
Size of a particular index on a collectiongauge.collection.size(gauge)
Size of a collection in bytes, not including indexesgauge.collection.storageSize(gauge)
Size of the collection on disk in bytes, never decreases.
Group collection-top 🔗
All of the following metrics are part of the collection-top metric group. All of
the non-default metrics below can be turned on by adding collection-top to the
monitor config option extraGroups:
counter.collection.commandsCount(counter)
Number of commands issued for a collectioncounter.collection.commandsTime(counter)
Time spent in microseconds processing commands issued for a collectioncounter.collection.getmoreCount(counter)
Number of getMore requests issued for a collectioncounter.collection.getmoreTime(counter)
Time spent in microseconds processing getMore requests for a collectioncounter.collection.insertCount(counter)
Number of inserts issued for a collectioncounter.collection.insertTime(counter)
Time spent in microseconds processing insert requests for a collectioncounter.collection.queriesCount(counter)
Number of queries issued for a collectioncounter.collection.queriesTime(counter)
Time spent in microseconds processing query requests for a collectioncounter.collection.readLockCount(counter)
Number of read locks issued for a collectioncounter.collection.readLockTime(counter)
Time spent in microseconds processing read locks for a collectioncounter.collection.removeCount(counter)
Number of remove requests issued for a collectioncounter.collection.removeTime(counter)
Time spent in microseconds processing remove requests for a collectioncounter.collection.totalCount(counter)
Total number of operations issued for a collectioncounter.collection.totalTime(counter)
Time spent in microseconds processing all operations for a collectioncounter.collection.updateCount(counter)
Number of update requests issued for a collectioncounter.collection.updateTime(counter)
Time spent in microseconds processing update requests for a collectioncounter.collection.writeLockCount(counter)
Number of write locks issued for a collectioncounter.collection.writeLockTime(counter)
Time spent in microseconds processing write locks for a collection
Non-default metrics (version 4.7.0+) 🔗
The following information applies to the agent version 4.7.0+ that has
enableBuiltInFiltering: true set on the top level of the agent config.
To emit metrics that are not default, you can add those metrics in the
generic monitor-level extraMetrics config option. Metrics that are derived
from specific configuration options that do not appear in the above list of
metrics do not need to be added to extraMetrics.
To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.
Legacy non-default metrics (version < 4.7.0) 🔗
The following information only applies to agent versions prior to 4.7.0. If
you have a newer agent and have set enableBuiltInFiltering: true at the top
level of your agent config, see the section above. See upgrade instructions in
Old-style inclusion list filtering.
If you have a reference to the whitelist.json in your agent’s top-level
metricsToExclude config option, and you want to emit metrics that are not in
that allow list, then you need to add an item to the top-level
metricsToInclude config option to override that allow list (see Inclusion
filtering. Or you can just
copy the whitelist.json, modify it, and reference that in metricsToExclude.