Docs » Integrations Guide » Integrations Reference » Redis

image0 Redis

Metadata associated with the Redis collectd plugin can be found here. The relevant code for the plugin can be found here.

DESCRIPTION

A Redis plugin for collectd using the Python plugin for collectd.

You can capture any kind of Redis metrics like:

  • Memory used
  • Commands processed per second
  • Number of connected clients and slaves
  • Number of blocked clients
  • Number of keys stored (per database)
  • Uptime
  • Changes since last save
  • Replication delay (per slave)

FEATURES

Built-in dashboards

  • Redis Instances: Overview of data from all Redis instances.

image1

  • Redis Instance: Focus on a single Redis instance.

image2

REQUIREMENTS AND DEPENDENCIES

This plugin requires:

Software Version
collectd 4.9+
Python plugin for collectd (included with SignalFx collectd agent)
Python 2.6+
redis 2.8+

INSTALLATION

If you are using the new Smart Agent, see the docs for thecollectd/redismonitor for more information. The configuration documentation below may be helpful as well, but consult the Smart Agent repo’s docs for the exact schema.

  1. Download the redis-collectd-plugin Python module.
  2. Download SignalFx’s sample configuration files for a Redis master or Redis slave to /etc/collectd/managed_config.
  3. Modify the sample configuration file as described in Configuration, below.
  4. Restart collectd.

CONFIGURATION

Using the example configuration files 10-redis_master.conf or 10-redis_slave.conf as a guide, provide values for the configuration options listed below that make sense for your environment and allow you to connect to the Redis instance to be monitored.

Configuration Option Type Definition
Instance Nodename The Module block identifies a Redis instance running in an specified host and port. The name for node is a canonical identifier which is used as plugin instance. It is limited to 64 characters in length.
Host Hostname The Host option is the hostname or IP-address where the Redis instance is running.
Port Port The Port option is the TCP port on which the Redis instance accepts connections. Either a service name of a port number may be given. Please note that numerical port numbers must be given as a string, too.
Auth Password Optionally specify a password to use for AUTH.
SendListLength (DB Index) (Key pattern) Specify a pattern of keys to lists for which to send their length as a metric. See below for more details.

Note: Monitoring multiple Redis instances on one host

You can configure this plugin to monitor multiple Redis instances on the same machine by repeating the section, as in the following example:

<Plugin python>
  ModulePath "/opt/collectd_plugins"
  Import "redis_info"

  <Module redis_info>
    Host "127.0.0.1"
    Port 9100
    Verbose true
    Instance "instance_9100"
    Redis_uptime_in_seconds "gauge"
    Redis_used_memory "bytes"
    Redis_used_memory_peak "bytes"
  </Module>

  <Module redis_info>
    Host "127.0.0.1"
    Port 9101
    Verbose true
    Instance "instance_9101"
    Redis_uptime_in_seconds "gauge"
    Redis_used_memory "bytes"
    Redis_used_memory_peak "bytes"
    Redis_master_repl_offset "gauge"
  </Module>

  <Module redis_info>
    Host "127.0.0.1"
    Port 9102
    Verbose true
    Instance "instance_9102"
    Redis_uptime_in_seconds "gauge"
    Redis_used_memory "bytes"
    Redis_used_memory_peak "bytes"
    Redis_slave_repl_offset "gauge"
  </Module>
</Plugin>

Note: value of plugin_instance

In the example above, 3 redis instances on the same host listen on different ports and Instance is used to supply a static value for the dimension plugin_instance. If Instance was not specified, the value of plugin_instance reported by collectd would contain the combination of Host and Port as follows:

  • “plugin_instance” => “127.0.0.1:9100”
  • “plugin_instance” => “127.0.0.1:9101”
  • “plugin_instance” => “127.0.0.1:9102”

USAGE

Sample of built-in dashboard in SignalFx:

image3

Monitoring length of Redis lists

To monitor the length of list keys, the key and database index must be
specified in the config file. Specify keys in the config file in the form
SendListLength $db_index "$key_name". $key_name can be a globbed pattern
(only * is supported), in which case all keys matching that glob will be
processed. Don’t forget to surround the pattern with double quotes or else
Collectd will strip out the asterisks. If any keys match the glob that are not
lists, an error will be sent to the collectd logs.
Lengths will be reported to SignalFx under the metric gauge.key_llen, a
separate time series for each list.
Warning: The KEYS command is used to match the globs so don’t try and
match something that is very big, as this command is not highly optimized and
can block other commands from executing.
Note: To avoid duplication reporting, this should only be reported in one node.
Keys can be defined in either the master or slave config.
<Plugin python>
  ModulePath "/opt/collectd_plugins"
  Import "redis_info"

  <Module redis_info>
    Host "127.0.0.1"
    Port 9100
    # Matches "mylist1", "mylist-test", etc...  Don't forget double quotes!
    SendListLength 0 "mylist*"
    SendListLength 0 "message-queue"
    Verbose true
    Instance "instance_9100"
    Redis_uptime_in_seconds "gauge"
    Redis_used_memory "bytes"
    Redis_used_memory_peak "bytes"
  </Module>

METRICS

Below is a list of all metrics.

Metric Name Brief Type
bytes.used_memory Number of bytes allocated by Redis gauge
bytes.used_memory_lua Number of bytes used by the Lua engine gauge
bytes.used_memory_peak Peak Number of bytes allocated by Redis gauge
bytes.used_memory_rss Number of bytes allocated by Redis as seen by the OS gauge
counter.commands_processed Total number of commands processed by the server cumulative counter
counter.connections_received Total number of connections accepted by the server cumulative counter
counter.evicted_keys Number of evicted keys due to maxmemory limit cumulative counter
counter.expired_keys Total number of key expiration events cumulative counter
counter.lru_clock Clock incrementing every minute, for LRU management cumulative counter
counter.rejected_connections Number of connections rejected because of maxclients limit cumulative counter
counter.total_net_input_bytes Total number of bytes inputted cumulative counter
counter.total_net_output_bytes Total number of bytes outputted cumulative counter
counter.used_cpu_sys System CPU consumed by the Redis server cumulative counter
counter.used_cpu_sys_children System CPU consumed by the background processes cumulative counter
counter.used_cpu_user User CPU consumed by the Redis server cumulative counter
counter.used_cpu_user_children User CPU consumed by the background processes cumulative counter
derive.keyspace_hits Number of successful lookup of keys in the main dictionary cumulative counter
derive.keyspace_misses Number of failed lookup of keys in the main dictionary cumulative counter
gauge.blocked_clients Number of clients pending on a blocking call gauge
gauge.changes_since_last_save Number of changes since the last dump gauge
gauge.client_biggest_input_buf Biggest input buffer among current client connections gauge
gauge.client_longest_output_list Longest output list among current client connections gauge
gauge.connected_clients Number of client connections (excluding connections from slaves) gauge
gauge.connected_slaves Number of connected slaves gauge
gauge.db0_avg_ttl The average time to live for all keys in redis gauge
gauge.db0_expires The total number of keys in redis that will expire gauge
gauge.db0_keys The total number of keys stored in redis gauge
gauge.instantaneous_ops_per_sec Number of commands processed per second gauge
gauge.key_llen Length of an list key gauge
gauge.latest_fork_usec Duration of the latest fork operation in microseconds gauge
gauge.master_last_io_seconds_ago Number of seconds since the last interaction with master gauge
gauge.master_repl_offset Master replication offset gauge
gauge.mem_fragmentation_ratio Ratio between used_memory_rss and used_memory gauge
gauge.rdb_bgsave_in_progress Flag indicating a RDB save is on-going gauge
gauge.repl_backlog_first_byte_offset Slave replication backlog offset gauge
gauge.slave_repl_offset Slave replication offset gauge
gauge.uptime_in_days Number of days up gauge
gauge.uptime_in_seconds Number of seconds up gauge

bytes.used_memory

gauge

used_memrory: total number of bytes allocated by Redis using its allocator (either standard libc, jemalloc, or an alternative allocator such as tcmalloc

Use this metric to see how much memory Redis is taking.

bytes.used_memory_lua

gauge

used_memory_lua: Number of bytes used by the Lua engine

Use this metric to see how much memory the Lua engine within Redis is taking.

bytes.used_memory_peak

gauge

used_memory_peak: Peak memory consumed by Redis (in bytes)

Use this metric to see the max memory Redis has taken since it last started.

bytes.used_memory_rss

gauge

used_memory_rss: Number of bytes that Redis allocated as seen by the operating system (a.k.a resident set size). This is the number reported by tools such as top(1) and ps(1)

Use this metric to see how much memory Resident memory Redis is taking.

counter.commands_processed

cumulative counter

commands_processed: Total number of commands processed by the server

Use this metric to see how many commands are processed by a particular server.

counter.connections_received

cumulative counter

Total number of connections accepted by the server

Use this metric to see how many connections are accepted by a particular server.

counter.evicted_keys

cumulative counter

evicted_keys: Number of evicted keys due to maxmemory limit

Use this metric to see how many keys are evicted due to hitting the maxmemory limit.

If this metric is significantly different than other servers:

  • Maxmemory limit may be too low

counter.expired_keys

cumulative counter

expired_keys: Total number of key expiration events

Use this metric to see how many keys are expired.

counter.lru_clock

cumulative counter

lru_clock: Clock incrementing every minute, for LRU management.

counter.rejected_connections

cumulative counter

rejected_connections: Number of connections rejected because of maxclients limit

Use this metric to see how many connections are rejected by a particular server.

If this metric is significantly different than other servers:

  • Macclients limit set too low

counter.total_net_input_bytes

cumulative counter

total_net_input_bytes: Total number of bytes inputted

Total number of bytes inputted.

counter.total_net_output_bytes

cumulative counter

total_net_output_bytes: Total number of bytes outputted

Total number of bytes outputted.

counter.used_cpu_sys

cumulative counter

used_cpu_sys: System CPU consumed by the Redis server

Use this metric to see how much cpu is being consumed by Redis.

counter.used_cpu_sys_children

cumulative counter

used_cpu_sys_children: System CPU consumed by the background processes

Use this metric to see how much cpu is being consumed by Redis background processes.

counter.used_cpu_user

cumulative counter

used_cpu_user: User CPU consumed by the Redis server

Use this metric to see how much user cpu is being consumed by Redis.

counter.used_cpu_user_children

cumulative counter

used_cpu_user_children: User CPU consumed by the background processes

Use this metric to see how much user cpu is being consumed by Redis background processes.

derive.keyspace_hits

cumulative counter

keyspace_hits: Number of successful lookup of keys in the main dictionary.

derive.keyspace_misses

cumulative counter

keyspaces_misses: Number of failed lookup of keys in the main dictionary.

gauge.blocked_clients

gauge

blocked_clients: Number of clients pending on a blocking call (BLPOP, BRPOP, BRPOPLPUSH)

Use this metric to see how many clients are pending on blocking calls.

gauge.changes_since_last_save

gauge

changes_since_last_save: Number of changes since the last dump

Use this metric to see any changes since the last dump.

gauge.client_biggest_input_buf

gauge

client_biggest_input_buf: biggest input buffer among current client connections

Use this metric to see which client conets contains the biggest input buffer.

gauge.client_longest_output_list

gauge

client_longest_output_list: Longest output list among current client connections

Use this metric to see which among the current client connections is the longest.

gauge.connected_clients

gauge

connected_clients: Number of client connections (excluding connections from slaves)

Use this metric to see how many clients are connected not including slaves.

gauge.connected_slaves

gauge

connected_slaves: Number of connected slaves

Use this metric to see how many slaves are connected.

If this metric is significantly different than other servers:

  • Perhaps unbalanced master/slave ratio

gauge.db0_avg_ttl

gauge

gauge.db0_avg_ttl : The average time to live for all keys in redis

gauge.db0_expires

gauge

gauge.db0_expires : The total number of keys stored in redis that wil expire

gauge.db0_keys

gauge

gauge.db0_keys : The total number of keys stored in redis

gauge.instantaneous_ops_per_sec

gauge

instantaneous_ops_per_sec: Number of commands processed per second

Use this metric to see how fast commands are being processed.

gauge.key_llen

gauge

gauge.key_llen : Length of an list key

gauge.latest_fork_usec

gauge

latest_fork_usec: Duration of the latest fork operation in microseconds

Use this metric to see how fast fork operatios are.

If this metric is significantly different than other servers:

  • This server could be overloaded

gauge.master_last_io_seconds_ago

gauge

master_last_io_seconds_ago: Number of seconds since the last interaction with master

Use this metric to see how long this slave last interacted with master.

gauge.master_repl_offset

gauge

master_repl_offset: target offset of master dataset

Use this metric to and compare with slave_repl_offset on slaves to figure out how replication is doing. if slave_repl_offset on slave host is behind its master’s master_repl_offset, you know you have a replication problem.

gauge.mem_fragmentation_ratio

gauge

mem_fragmentation_ratio: Ratio between used_memory_rss and used_memory

Ideally, the used_memory_rss value should be only slightly higher than used_memory. When rss used, a large difference means there is memory fragmentation (internal or external), which can be evaluated by checking mem_fragmentation_ratio. When used rss, it means part of Redis memory has been swapped off by the operating system: expect some significant latencies.

gauge.rdb_bgsave_in_progress

gauge

rdb_bgsave_in_progress: Flag indicating a RDB save is on-going

Use this metric to see if a save is ongoing.

gauge.repl_backlog_first_byte_offset

gauge

repl_backlog_first_byte_offset: Slave replication backlog offset

Compare with master_repl_offset to see how replication is doing.

gauge.slave_repl_offset

gauge

slave_repl_offset: How much has the slave replicated

Use this metric to and compare with master_repl_ofset on masters to figure out how replication is doing.

If this metric is significantly different than master:

  • There could be a replications problem

gauge.uptime_in_days

gauge

uptime_in_days: Number of days up

Use this metric to see how long redis has been up in days.

gauge.uptime_in_seconds

gauge

uptime_in_seconds: Number of seconds up

Use this metric to see how long redis has been up in seconds.