Docs » Integrations Guide » Integrations Reference » Docker

image0 Docker

Metadata associated with the Docker plugin for collectd can be found here. The relevant code for the plugin can be found here.

DESCRIPTION

This is the SignalFx Docker plugin. Follow these instructions to install the Docker plugin for collectd.

The docker-collectd plugin collects metrics about the Docker containers running on the system using Docker’s stats API. It reports metrics about the CPU utilization of each container, their memory consumption, and their network and disk activity.

FEATURES

Built-in dashboards

  • Docker Hosts: Overview of all data from Docker hosts.

image1

  • Docker Host: Focus on a single Docker host.

image2

  • Docker Container: Focus further on a single running Docker container.

image3

  • Docker Neighbors: (Optional Dashboard) Container resource allocation metrics.

image4

REQUIREMENTS AND DEPENDENCIES

Version information

Software Version
collectd 5.0 or later
Python 2.6 or later
Docker 1.9 or later
Python plugin for collectd (included with SignalFx collectd agent)

INSTALLATION

If you are using the new Smart Agent, see the docs for thecollectd/dockermonitor for more information. The configuration documentation below may be helpful as well, but consult the Smart Agent repo’s docs for the exact schema.

  1. Download the docker-collectd-plugin Python module.

  2. Run the following command to install the modules dependencies using pip, replacing the example path with the download location of the docker-collectd-plugin module:

    sudo pip install -r /path/to/docker-collectd-plugin/requirements.txt
    
    • On Amazon Linux: Run the following commands instead:

      yum install python26-pip
      sudo pip-2.6 install -r /path/to/docker-collectd-plugin/requirements.txt
      
  3. Download SignalFxs sample configuration file for this plugin to /etc/collectd/managed_config.

  4. Modify the configuration file to provide values that make sense for your environment, as described in Configuration below.

  5. Restart collectd.

  6. Optional metrics regarding CPU quota and CPU shares can be enabled in the plugin configuration file. To enable the optional metrics:

    • Set the CpuQuotaPercent and CpuSharesPercent configuration options to true
    • Configure the filter to emit the optional metrics. Please see the configuration details here
  7. The optional dashboard Docker Neighbors offers visualizations based on the CPU quota and CPU shares metrics. To view the dashboard:

    • Manually import the dashboard into your organization in SignalFx.
      • Download the dashboard group with the optional dashboard
      • Click to open the Actions menu, hover over Import, then select Dashboard Group
      • Specify the path to the downloaded Page_Docker.json file
      • The new dashboard group should appear under Custom Dashboard Groups

CONFIGURATION

Using the example configuration file 10-docker.conf as a guide, provide values for the configuration options listed below that make sense for your environment.

configuration option definition default value
TypesDB Path on disk where collectd can find the Types.db file included in this module. “/usr/share/collectd/docker-collectd-plugin/dockerplugin.db”
ModulePath Path on disk where collectd can find this module. “/usr/share/collectd/docker-collectd-plugin”
BaseURL URL of your Docker daemon’s remote API “unix://var/run/docker.sock”
Timeout Time in seconds that collectd will wait for a response from Docker 3
Verbose Turns on verbose log statements false
CpuQuotaPercent Turns on cpu quota metric false
CpuSharesPercent Turns on cpu shares metric false
CollectNetworkStats Turns on network stat gathering true

How to send metrics about resource allocation

If a filter has been configured, then additional resource allocation metrics can be gathered by adding the following snippet to the plugin’s filter configuration.

<Rule "CpuShares">
  <Match "regex">
    Type "^cpu.shares$"
  </Match>
  Target "return"
</Rule>
<Rule "CpuQuota">
<Match "regex">
  Type "^cpu.quota$"
</Match>
Target "return"
</Rule>
<Rule "CpuThrottlingData">
  <Match "regex">
    Type "^cpu.throttling_data$"
  </Match>
  Target "return"
</Rule>
<Rule "MemoryStats">
  <Match "regex">
    Type "^memory.stats$"
    TypeInstance "^swap"
  </Match>
  Target "return"
</Rule>

USAGE

All metrics reported by the Docker collectd plugin will contain the following dimensions:

  • host will contain the hostname (as known by collectd) of the machine reporting the metrics
  • plugin is always set to docker
  • plugin_instance will contain the name of the container the metrics are from. The container name is used because that’s usually a more stable value than the container ID, which changes on every restart.

Sample of built-in dashboard in SignalFx:

image5

METRICS

Below is a list of all metrics.

Metric Name Brief Type
blkio.io_service_bytes_recursive-major-minor.async Volume, in bytes, of asynchronous block I/O cumulative_counter
blkio.io_service_bytes_recursive-major-minor.read Volume, in bytes, of reads from block devices cumulative_counter
blkio.io_service_bytes_recursive-major-minor.sync Volume, in bytes, of synchronous block I/O cumulative_counter
blkio.io_service_bytes_recursive-major-minor.total Total volume, in bytes, of all block I/O cumulative_counter
blkio.io_service_bytes_recursive-major-minor.write Volume, in bytes, of writes to block devices cumulative_counter
blkio.io_service_bytes_recursive.async Volume, in bytes, of asynchronous block I/O cumulative_counter
blkio.io_service_bytes_recursive.read Volume, in bytes, of reads from block devices cumulative_counter
blkio.io_service_bytes_recursive.sync Volume, in bytes, of synchronous block I/O cumulative_counter
blkio.io_service_bytes_recursive.total Total volume, in bytes, of all block I/O cumulative_counter
blkio.io_service_bytes_recursive.write Volume, in bytes, of writes to block devices cumulative_counter
blkio.io_serviced_recursive-major-minor.async Number of asynchronous block I/O requests cumulative_counter
blkio.io_serviced_recursive-major-minor.read Number of reads requests from block devices cumulative_counter
blkio.io_serviced_recursive-major-minor.sync Number of synchronous block I/O requests cumulative_counter
blkio.io_serviced_recursive-major-minor.total Total number of block I/O requests cumulative_counter
blkio.io_serviced_recursive-major-minor.write Number of write requests to block devices cumulative_counter
blkio.io_serviced_recursive.async Number of asynchronous block I/O requests cumulative_counter
blkio.io_serviced_recursive.read Number of reads requests from block devices cumulative_counter
blkio.io_serviced_recursive.sync Number of synchronous block I/O requests cumulative_counter
blkio.io_serviced_recursive.total Total number of block I/O requests cumulative_counter
blkio.io_serviced_recursive.write Number of write requests to block devices cumulative_counter
cpu.percent Percentage of host CPU resources used by the container gauge
cpu.percpu.usage.cpuX Jiffies of CPU time spent by the container, per CPU core cumulative_counter
cpu.percpu.usage Jiffies of CPU time spent by the container, per CPU core cumulative_counter
cpu.throttling_data.periods Number of periods cumulative_counter
cpu.throttling_data.throttled_periods Number of periods throttled cumulative_counter
cpu.throttling_data.throttled_time Throttling time in nano seconds cumulative_counter
cpu.usage.kernelmode Jiffies of CPU time spent in kernel mode by the container cumulative_counter
cpu.usage.system Jiffies of CPU time used by the system cumulative_counter
cpu.usage.total Jiffies of CPU time used by the container cumulative_counter
cpu.usage.usermode Jiffies of CPU time spent in user mode by the container cumulative_counter
memory.stats.swap Bytes of swap memory used by container gauge
memory.usage.limit Memory usage limit of the container, in bytes gauge
memory.usage.max Maximum measured memory usage of the container, in bytes gauge
memory.usage.total Bytes of memory used by the container gauge
network.usage.rx_bytes Bytes received by the container via its network interface cumulative_counter
network.usage.rx_dropped Number of inbound network packets dropped by the container cumulative_counter
network.usage.rx_errors Errors receiving network packets cumulative_counter
network.usage.rx_packets Network packets received by the container via its network interface cumulative_counter
network.usage.tx_bytes Bytes sent by the container via its network interface cumulative_counter
network.usage.tx_dropped Number of outbound network packets dropped by the container cumulative_counter
network.usage.tx_errors Errors sending network packets cumulative_counter
network.usage.tx_packets Network packets sent by the container via its network interface cumulative_counter

blkio.io_service_bytes_recursive-major-minor.async

cumulative_counter

Superseded by blkio.io_service_bytes_recursive.async.

Tracks the volume, in bytes, of serviced asynchronous block I/O requests in that container.

blkio.io_service_bytes_recursive-major-minor.read

cumulative_counter

Superseded by blkio.io_service_bytes_recursive.read.

Tracks the volume, in bytes, read from block devices by that container.

blkio.io_service_bytes_recursive-major-minor.sync

cumulative_counter

Superseded by blkio.io_service_bytes_recursive.sync.

Tracks the volume, in bytes, of serviced synchronous block I/O requests in that container.

blkio.io_service_bytes_recursive-major-minor.total

cumulative_counter

Superseded by blkio.io_service_bytes_recursive.total.

Tracks the total volume, in bytes, of all serviced block I/O requests in that container.

blkio.io_service_bytes_recursive-major-minor.write

cumulative_counter

Superseded by blkio.io_service_bytes_recursive.write.

Tracks the volume, in bytes, written to block devices by that container.

blkio.io_service_bytes_recursive.async

cumulative_counter

Tracks the volume, in bytes, of serviced asynchronous block I/O requests
in that container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_service_bytes_recursive.read

cumulative_counter

Tracks the volume, in bytes, read from block devices by that container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_service_bytes_recursive.sync

cumulative_counter

Tracks the volume, in bytes, of serviced synchronous block I/O requests
in that container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_service_bytes_recursive.total

cumulative_counter

Tracks the total volume, in bytes, of all serviced block I/O requests in
that container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_service_bytes_recursive.write

cumulative_counter

Tracks the volume, in bytes, written to block devices by that container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_serviced_recursive-major-minor.async

cumulative_counter

Superseded by blkio.io_serviced_recursive.async.

Tracks the number of serviced asynchronous block I/O requests in that container.

blkio.io_serviced_recursive-major-minor.read

cumulative_counter

Superseded by blkio.io_serviced_recursive.read.

Tracks the number of read requests from block devices serviced in that container.

blkio.io_serviced_recursive-major-minor.sync

cumulative_counter

Superseded by blkio.io_serviced_recursive.sync.

Tracks the number of serviced synchronous block I/O requests in that container.

blkio.io_serviced_recursive-major-minor.total

cumulative_counter

Superseded by blkio.io_serviced_recursive.total.

Tracks of the total number of block I/O requests serviced in that container.

blkio.io_serviced_recursive-major-minor.write

cumulative_counter

Superseded by blkio.io_serviced_recursive.write.

Tracks the number of write requests to block devices serviced in that container.

blkio.io_serviced_recursive.async

cumulative_counter

Tracks the number of serviced asynchronous block I/O requests in that
container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_serviced_recursive.read

cumulative_counter

Tracks the number of read requests from block devices serviced in that
container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_serviced_recursive.sync

cumulative_counter

Tracks the number of serviced synchronous block I/O requests in that
container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_serviced_recursive.total

cumulative_counter

Tracks of the total number of block I/O requests serviced in that
container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

blkio.io_serviced_recursive.write

cumulative_counter

Tracks the number of write requests to block devices serviced in that
container.

This metric is reported with dimensions device_major and device_minor to indicate the major and minor device numbers respectively.

cpu.percent

gauge

Tracks CPU utilization percentage relative to the container host. This is the same measurement given by docker stats in CPU % column.

cpu.percpu.usage.cpuX

cumulative_counter

Superseded by cpu.percpu.usage.

How much CPU time is spent by the container, per CPU core. This metric is reported for each core as cpu.percpu.usage.cpuX where X is the number of the core. For example, a two-core machine would have cpu.percpu.usage.cpu0 and cpu.percpu.usage.cpu1.

cpu.percpu.usage

cumulative_counter

How much CPU time is spent by the container, per CPU core. This metric is reported with a dimension core that indicates the number of the core.

cpu.throttling_data.periods

cumulative_counter

(Optional metric) A cumulative count of the total number of periods that have passed. A period is a measurement of time during which containers are allocated their CPU quota. If the period is set to 100,000 microseconds and a docker container is quoted 50,000 microseconds, then the container is allocated 50% of 1 CPU during a period. The default period length is 100,000 microseconds or 100 milliseconds. Docker reports 0 if there is no quota constraint on the container.

cpu.throttling_data.throttled_periods

cumulative_counter

(Optional metric) A cumulative count of the number of periods the container has been throttled. If the container attempts to use more than its quota, it will be throttled. Docker reports 0 if there is no quota constraint on the container.

cpu.throttling_data.throttled_time

cumulative_counter

(Optional metric) The amount of time (in nanoseconds) that a container is throttled. A container is throttled when it tries to use more than it’s quota within a period. Docker reports 0 if there is no quota constraint on the container.

cpu.usage.kernelmode

cumulative_counter

Tracks how much CPU time, in jiffies, is spent by the container while executing kernel code.

cpu.usage.system

cumulative_counter

Tracks how much CPU time, in jiffies, the system (Docker host) has used. This metric can be used to calculate the CPU usage percentage of a given container (see `cpu.usage.total <cpu.usage.total.md>`__).

cpu.usage.total

cumulative_counter

Tracks how much CPU time, in jiffies, a container has used. To get a CPU utilization percentage, compute against cpu.usage.system:

CPU usage % = 100 * cpu.usage.total / cpu.usage.system

cpu.usage.usermode

cumulative_counter

Tracks how much CPU time, in jiffies, is spent by the container while executing user/application code (as opposed to kernel code).

memory.stats.swap

gauge

(Optional metric) The number of bytes of swap memory used by a container.

memory.usage.limit

gauge

Memory usage limit of the container, in bytes.

memory.usage.max

gauge

Maximum measured memory usage of the container, in bytes, during the lifetime of the container.

memory.usage.total

gauge

This metrics reports the current memory (RAM) usage of the container, in bytes. A memory utilization percentage can be calculated against the memory.usage.limit metric:

Memory usage % = 100 * memory.usage.total / memory.usage.limit

network.usage.rx_bytes

cumulative_counter

The amount of inbound network traffic, in bytes, received by the container via its network interface.

network.usage.rx_dropped

cumulative_counter

This metric tracks the number of inbound network packets dropped by the container on its network interface.

network.usage.rx_errors

cumulative_counter

This metric tracks the number of errors encountered by the container while attempting to receive network packets.

network.usage.rx_packets

cumulative_counter

The amount of network packets received by the container via its network interface.

network.usage.tx_bytes

cumulative_counter

The amount of outbound network traffic, in bytes, sent by the container via its network interface.

network.usage.tx_dropped

cumulative_counter

This metric tracks the number of outbound network packets dropped by the container on its network interface.

network.usage.tx_errors

cumulative_counter

This metric tracks the number of errors encountered by the container while attempting to transmit network packets.

network.usage.tx_packets

cumulative_counter

The amount of network packets sent by the container via its network interface.