MongoDB¶
Metadata associated with the MongoDB collectd plugin can be found here. The relevant code for the plugin can be found here.
DESCRIPTION¶
Use the mongodb collectd plugin to collect metrics from MongoDB nodes.
This plugin captures the following metrics about MongoDB generally:
- memory
- network input/output bytes count
- heap usage
- db connections
- operations count
- active client connections
- queued operations
The plugin also captures the following DB-specific metrics:
- db size
- db counters
Documentation for MongoDB can be found here.
REQUIREMENTS AND DEPENDENCIES¶
Version information¶
| Software | Version |
|---|---|
| collectd | 4.9 or later |
| Python | 2.4 or later |
| MongoDB | 2.6 or later |
| PyMongo | 3.0 or later |
| Python plugin for collectd | (included with SignalFx collectd agent) |
INSTALLATION¶
If you are using the new Smart Agent, see the docs for thecollectd/mongodbmonitor for more information. The configuration documentation below may be helpful as well, but consult the Smart Agent repo’s docs for the exact schema.
Install
pipandpymongo.RHEL/CentOS and Amazon Linux
yum install -y epel-release yum install -y python-pip sudo pip install pymongo==3.0.3
Ubuntu and Debian:
apt-get install -y python-pip python-dev build-essential sudo pip install pymongo==3.0.3
- If you want to use SSL/TLS to connect to Mongo, install the PyMongo TLSdependencies as well:
sudo pip install pymongo[tls]
Download the collectd-mongodb Python module.
Download SignalFx’s sample configuration file for this plugin to
/etc/collectd/managed_config.Modify the sample configuration file as described in Configuration, below.
Restart collectd.
CONFIGURATION¶
Using the example configuration file 10-mongodb.conf as a guide, provide values for the configuration options listed below that make sense for your environment and allow you to connect to the MongoDB instance to be monitored.
| configuration option | definition | default value |
|---|---|---|
| ModulePath | Path on disk where collectd can find this module. | “/usr/share/collectd/mongodb-collectd-plugin” |
| Host | Host IP | “127.0.0.1” |
| Port | Port number for IP connection | “27017” |
| User | Valid mongodb user | “” |
| Password | Associated password for valid user | “password” |
| Database | Name(s) of database(s) that you would like metrics from. Note: the first database in this list must be “admin”, as it is used to perform a serverStatus() command. |
“admin” “db-prod” “db-dev” |
| Interval | How frequently to send metrics in seconds | collectd Interval setting |
| SendCollectionMetrics | Whether to send collection level metrics or not | false |
| SendCollectionTopMetrics | Whether to send collection level top (timing) metrics or not | false |
| CollectionMetricsIntervalMultiplier | How frequently to send collection level metrics as a multiple of the configured plugin interval (e.g. if the Interval is 15 and the multiplier is 4, collection level metrics will be fetched every minute) | 6 |
| UseTLS | Set this to true if you want to connect to Mongo using TLS/x509/SSL. |
false |
| CACerts | Path to a CA cert that will be used to verify the certificate that Mongo presents (not needed if not using TLS or if Mongo’s cert is signed by a globally trusted issuer already installed in the default location on your OS) | “” |
| TLSClientCert | Path to a client certificate (not needed unless your Mongo instance requires x509 client verification) | “” |
| User | (same as above ``User`` field but required for TLS client auth) The username associated with the client cert (this is the subject field of the client cert, formatted according to RFC2253 and in the same order that was specified when creating the user in the Mongo ``$external`` database). You can get this value by running the following command: openssl x509 -in <pathToClient PEM> -inform PEM -subject -nameopt RFC2253. |
“” |
| TLSClientKey | Path to a client certificate key (not needed unless your Mongo instance requires x509 client verification, or if your client cert above has the key included) | “” |
| TLSClientKeyPassphrase | Passphrase for the TLSClientKey above (requires Python 2.7.9+) | “” |
Note: Monitoring multiple instances¶
Each MongoDB instance to be monitored is specified in a <Module>
block within the configuration file. By default, the sample
configuration file 10-mongodb.conf contains only one such block. To
monitor an additional instance of MongoDB, add another <Module>
block immediately below the first, and configure it according to that
instance’s parameters.
Note: Creating a MongoDB user for collectd¶
If you’re monitoring a secured MongoDB deployment, it is a good practice to create a MongoDB user with minimal read-only roles, as follows:
db.createUser( {
user: "collectd",
pwd: "collectd",
roles: [ { role: "readAnyDatabase", db: "admin" }, { role: "clusterMonitor", db: "admin" } ]
});
USAGE¶
Below are screen captures of dashboards created for this plugin by SignalFx, illustrating the metrics emitted by this plugin.
For general reference on how to monitor MongoDB performance, see Analyzing MongoDBPerformance.
Monitoring MongoDB clusters
Writes to MongoDB require the use of the global lock. If lock utilization is high, operations can begin to slow down. This can be a symptom of database issues such as poorly configured or absent indexes, or a schema design that needs improvement. It can also indicate the failure of a disk. Monitor the number of readers and writers waiting for the lock in gauge.globalLock.currentQueue.total.
This lock has little utilization and few queued readers and writers.
MongoDB flushes data changes from memory to disk on a timed interval, by default every 60 seconds. If background flushes begin taking longer than usual, it can indicate that the disk doesn’t have enough I/O capacity to handle the load (read more below). It could also reflect a large number of writes occurring at once – check counter.opcounters.insert and counter.opcounters.update.
Monitor average background flush time and the most recent background flush time in gauge.backgroundFlushing.average_ms and gauge.backgroundFlushing.last_ms respectively.
Average background flush time on this cluster is around 40ms, well within healthy parameters.
When analyzing the performance of a MongoDB cluster, it’s also important to verify that the load is balanced across each instance. The cluster dashboard included in this repository contains many list charts of individual MongoDB instances ordered by important metrics like requests per second (counter.network.numRequests) and number of connections to MongoDB (gauge.connections.current). This can help you compare load between instances. Load imbalance can arise in a sharded cluster if MongoDB is unable to balance chunks equally between the shards, for example if lock utilization is high.
All the listed instances show about the same requests per second and number of connections. Their load is balanced.
Monitoring MongoDB hosts
On an individual instance level, it’s important to monitor system statistics like memory usage, page faults, and disk I/O utilization.
MongoDB uses memory-mapped files to store data, so it is important to compare the amount of memory that MongoDB has allocated to the amount of system memory. This plugin reports resident memory usage in gauge.mem.resident and mapped memory usage in gauge.mem.mapped. If either of these quantities exceed the amount of system memory (reported by the memory plugin for collectd), the system may be at or over capacity.
This MongoDB instance is not using a large amount of resident memory, and has non-mapped memory available to the process (calculated as gauge.mem.virtual - gauge.mem.mapped.)
This plugin reports page faults in counter.extra_info.page_faults. Page faults indicate that reads or writes are occurring to data files that are not currently in memory. This is different from an OS page fault. Sudden increases in MongoDB page faults can indicate that a large read operation is taking place. Steadily high numbers of page faults indicate that MongoDB is reading more often from disk than is optimal.
This MongoDB instance has a low rate of page faults. This means that most of the data MongoDB needs to access is in memory, and doesn’t need to be fetched from disk.
You can monitor disk I/O utilization for your MongoDB host using the
disk_ops.write
and
disk_ops.read
metrics emitted by the disk plugin for collectd, which is included
and enabled by default in most packages of collectd. Click here to learn
more about the collectd-disk plugin.
METRICS¶
Below is a list of all metrics.
| Metric Name | Brief | Type |
|---|---|---|
| counter.backgroundFlushing.flushes | Number of times the database has been flushed | gauge |
| counter.collection.commandsCount | Number of commands issued for a collection | counter |
| counter.collection.commandsTime | Time spent in microseconds processing commands issued for a collection | counter |
| counter.collection.getmoreCount | Number of getMore requests issued for a collection | counter |
| counter.collection.getmoreTime | Time spent in microseconds processing getMore requests for a collection | counter |
| counter.collection.index.accesses.ops | Number of times an index has been used (only on Mongo 3.2+) | counter |
| counter.collection.insertCount | Number of inserts issued for a collection | counter |
| counter.collection.insertTime | Time spent in microseconds processing insert requests for a collection | counter |
| counter.collection.queriesCount | Number of queries issued for a collection | counter |
| counter.collection.queriesTime | Time spent in microseconds processing query requests for a collection | counter |
| counter.collection.readLockCount | Number of read locks issued for a collection | counter |
| counter.collection.readLockTime | Time spent in microseconds processing read locks for a collection | counter |
| counter.collection.removeCount | Number of remove requests issued for a collection | counter |
| counter.collection.removeTime | Time spent in microseconds processing remove requests for a collection | counter |
| counter.collection.totalCount | Total number of operations issued for a collection | counter |
| counter.collection.totalTime | Time spent in microseconds processing all operations for a collection | counter |
| counter.collection.updateCount | Number of update requests issued for a collection | counter |
| counter.collection.updateTime | Time spent in microseconds processing update requests for a collection | counter |
| counter.collection.writeLockCount | Number of write locks issued for a collection | counter |
| counter.collection.writeLockTime | Time spent in microseconds processing write locks for a collection | counter |
| counter.extra_info.page_faults | Mongod page faults | gauge |
| counter.network.bytesIn | Network bytes received by the database server | gauge |
| counter.network.bytesOut | Network bytes sent by the database server | gauge |
| counter.network.numRequests | Requests received by the server | cumulative_counter |
| counter.opcounters.delete | Number of deletes per second | cumulative counter |
| counter.opcounters.insert | Number of inserts per second | cumulative counter |
| counter.opcounters.query | Number of queries per second | cumulative counter |
| counter.opcounters.update | Number of updates per second | cumulative counter |
| dimension.plugin_instance | Port number of the MongoDB instance | |
| gauge.backgroundFlushing.average_ms | Average time (ms) to write data to disk | gauge |
| gauge.backgroundFlushing.last_ms | Most recent time (ms) spent writing data to disk | gauge |
| gauge.collection.avgObjSize | Mean object/document size of a collection | gauge |
| gauge.collection.count | Number of objects/documents in a collection | gauge |
| gauge.collection.indexSize | Size of a particular index on a collection | gauge |
| gauge.collection.max | Maximum number of documents in a capped collection | gauge |
| gauge.collection.maxSize | Maximum disk usage of a capped collection | gauge |
| gauge.collection.size | Size of a collection in bytes, not including indexes | gauge |
| gauge.collection.storageSize | Size of the collection on disk in bytes, never decreases. | gauge |
| gauge.collections | Number of collections | gauge |
| gauge.connections.available | Number of available incoming connections | gauge |
| gauge.connections.current | Number of current client connections | gauge |
| gauge.dataSize | Total size of data, in bytes | gauge |
| gauge.extra_info.heap_usage_bytes | Heap size used by the mongod process, in bytes | gauge |
| gauge.globalLock.activeClients.readers | Number of active client connections performing reads | gauge |
| gauge.globalLock.activeClients.total | Total number of active client connections | gauge |
| gauge.globalLock.activeClients.writers | Number of active client connections performing writes | gauge |
| gauge.globalLock.currentQueue.readers | Read operations currently in queue | gauge |
| gauge.globalLock.currentQueue.total | Total operations currently in queue | gauge |
| gauge.globalLock.currentQueue.writers | Write operations currently in queue | gauge |
| gauge.indexSize | Total size of indexes, in bytes | gauge |
| gauge.indexes | Number of indexes across all collections | gauge |
| gauge.mem.mapped | Mongodb mapped memory usage, in MB | gauge |
| gauge.mem.resident | Mongodb resident memory usage, in MB | gauge |
| gauge.mem.virtual | Mongodb virtual memory usage, in MB | gauge |
| gauge.objects | Number of documents across all collections | gauge |
| gauge.storageSize | Total bytes allocated to collections for document storage | gauge |
| gauge.uptime | Uptime of this server in milliseconds | counter |
counter.collection.commandsTime¶
counter
Time spent in microseconds processing commands issued for a collection
counter.collection.getmoreTime¶
counter
Time spent in microseconds processing getMore requests for a collection
counter.collection.index.accesses.ops¶
counter
Number of times an index has been used (only on Mongo 3.2+)
counter.collection.insertTime¶
counter
Time spent in microseconds processing insert requests for a collection
counter.collection.queriesTime¶
counter
Time spent in microseconds processing query requests for a collection
counter.collection.readLockTime¶
counter
Time spent in microseconds processing read locks for a collection
counter.collection.removeTime¶
counter
Time spent in microseconds processing remove requets for a collection
counter.collection.totalTime¶
counter
Time spent in microseconds processing all operations for a collection
counter.collection.updateTime¶
counter
Time spent in microseconds processing update requets for a collection
counter.collection.writeLockTime¶
counter
Time spent in microseconds processing write locks for a collection
counter.network.numRequests¶
cumulative_counter
Total number of unique requests received by the server.
dimension.plugin_instance¶
None
This dimension indicates the port number of the MongoDB instance being monitored.
gauge.backgroundFlushing.average_ms¶
gauge
Average time (ms) that the mongod process spent writing data to disk.
gauge.backgroundFlushing.last_ms¶
gauge
Time(ms) that the mongod process last spent writing data to disk.
gauge.connections.current¶
gauge
Number of current connections to the database server from clients, including shell commands.
gauge.extra_info.heap_usage_bytes¶
gauge
The total size in bytes of heap space used by the database process.
gauge.globalLock.activeClients.readers¶
gauge
Number of active client connections performing read operation.
gauge.globalLock.activeClients.writers¶
gauge
Number of active client connections performing write operation.
gauge.storageSize¶
gauge
The total amount of space in bytes allocated to collections in the database for document storage.