Docs » Integrations Guide » Use the Smart Agent » Monitors » collectd/elasticsearch

collectd/elasticsearch 🔗

Monitor Type: collectd/elasticsearch (Source)

Accepts Endpoints: Yes

Multiple Instances Allowed: Yes

Overview 🔗

Monitors ElasticSearch instances. We strongly recommend using the elasticsearch monitor instead, as it will scale much better.

You can also view the source of the Python plugin.

Configuration 🔗

To activate this monitor in the Smart Agent, add the following to your agent config:

monitors:  # All monitor config goes under this key
 - type: collectd/elasticsearch
   ...  # Additional config

For a list of monitor options that are common to all monitors, see Common Configuration.

Config option Required Type Description
pythonBinary no string Path to a python binary that should be used to execute the Python code. If not set, a built-in runtime will be used. Can include arguments to the binary as well.
host yes string
port yes integer
additionalMetrics no list of strings AdditionalMetrics to report on
cluster no string Cluster name to which the node belongs. This is an optional config that will override the cluster name fetched from a node and will be used to populate the plugin_instance dimension
detailedMetrics no bool DetailedMetrics turns on additional metric time series (default: true)
enableClusterHealth no bool EnableClusterHealth enables reporting on the cluster health (default: true)
enableIndexStats no bool EnableIndexStats reports metrics about indexes (default: true)
indexes no list of strings Indexes to report on (default: [_all])
indexInterval no unsigned integer IndexInterval is an interval in seconds at which the plugin will report index stats. It must be greater than or equal, and divisible by the Interval configuration (default: 300)
indexStatsMasterOnly no bool IndexStatsMasterOnly sends index stats from the master only (default: false)
indexSummaryOnly no bool (default: false)
password no string Password used to access elasticsearch stats api
protocol no string Protocol used to connect: http or https
threadPools no list of strings ThreadPools to report on (default: [search index])
username no string Username used to access elasticsearch stats api
version no string

Metrics 🔗

These are the metrics available for this monitor. Metrics that are categorized as container/host (default) are in bold and italics in the list below.

  • bytes.indices.primaries.merges.total-size (gauge)
  • bytes.indices.total.merges.total-size (gauge)
  • counter.http.total_open (cumulative)
  • counter.indices.cache.field.eviction (cumulative)
  • counter.indices.cache.filter.cache-count (cumulative)
  • counter.indices.cache.filter.evictions (cumulative)
  • counter.indices.cache.filter.hit-count (cumulative)
  • counter.indices.cache.filter.miss-count (cumulative)
  • counter.indices.cache.filter.total-count (cumulative)
  • counter.indices.flush.time (cumulative)
  • counter.indices.flush.total (cumulative)
  • counter.indices.get.exists-time (cumulative)
  • counter.indices.get.exists-total (cumulative)
  • counter.indices.get.missing-time (cumulative)
  • counter.indices.get.missing-total (cumulative)
  • counter.indices.get.time (cumulative)
  • counter.indices.get.total (cumulative)
    The total number of get requests since node startup
  • counter.indices.indexing.delete-time (cumulative)
  • counter.indices.indexing.delete-total (cumulative)
  • counter.indices.indexing.index-time (cumulative)
  • counter.indices.indexing.index-total (cumulative)
    The total number of index requests since node startup
  • counter.indices.merges.time (cumulative)
  • counter.indices.merges.total (cumulative)
    Total number of merges since node startup
  • counter.indices.merges.total-size (cumulative)
  • counter.indices.primaries.fielddata.evictions (cumulative)
  • counter.indices.primaries.flush.total (cumulative)
  • counter.indices.primaries.flush.total-time (cumulative)
  • counter.indices.primaries.get.exists-time (cumulative)
  • counter.indices.primaries.get.exists-total (cumulative)
  • counter.indices.primaries.get.missing-time (cumulative)
  • counter.indices.primaries.get.missing-total (cumulative)
  • counter.indices.primaries.get.time (cumulative)
  • counter.indices.primaries.indexing.delete-time (cumulative)
  • counter.indices.primaries.indexing.delete-total (cumulative)
  • counter.indices.primaries.indexing.index-time (cumulative)
  • counter.indices.primaries.indexing.index-total (cumulative)
  • counter.indices.primaries.merges.total (cumulative)
  • counter.indices.primaries.merges.total-docs (cumulative)
  • counter.indices.primaries.merges.total-time (cumulative)
  • counter.indices.primaries.refresh.total (cumulative)
  • counter.indices.primaries.refresh.total-time (cumulative)
  • counter.indices.primaries.search.fetch-time (cumulative)
  • counter.indices.primaries.search.fetch-total (cumulative)
  • counter.indices.primaries.search.query-time (cumulative)
  • counter.indices.primaries.search.query-total (cumulative)
  • counter.indices.primaries.segments.count (cumulative)
  • counter.indices.primaries.translog.operations (cumulative)
  • counter.indices.primaries.warmer.total (cumulative)
  • counter.indices.primaries.warmer.total.primaries.warmer.total-time (cumulative)
  • counter.indices.refresh.time (cumulative)
  • counter.indices.refresh.total (cumulative)
  • counter.indices.search.fetch-time (cumulative)
  • counter.indices.search.fetch-total (cumulative)
  • counter.indices.search.query-time (cumulative)
    Total time spent in search queries (milliseconds)
  • counter.indices.search.query-total (cumulative)
    The total number of search requests since node startup
  • counter.indices.search.scroll-time (cumulative)
  • counter.indices.search.scroll.total (cumulative)
  • counter.indices.total.fielddata.evictions (cumulative)
  • counter.indices.total.flush.periodic (cumulative)
  • counter.indices.total.get.exists-time (cumulative)
  • counter.indices.total.get.exists-total (cumulative)
  • counter.indices.total.get.missing-time (cumulative)
  • counter.indices.total.get.missing-total (cumulative)
  • counter.indices.total.get.time (cumulative)
  • counter.indices.total.get.total (cumulative)
  • counter.indices.total.indexing.delete-time (cumulative)
  • counter.indices.total.indexing.delete-total (cumulative)
  • counter.indices.total.indexing.index-time (cumulative)
  • counter.indices.total.indexing.index-total (cumulative)
    The total number of index requests per cluster
  • counter.indices.total.merges.total (cumulative)
    Total number of merges per cluster
  • counter.indices.total.merges.total-docs (cumulative)
  • counter.indices.total.merges.total-time (cumulative)
  • counter.indices.total.search.fetch-total (cumulative)
  • counter.indices.total.search.query-time (cumulative)
  • counter.indices.total.search.query-total (cumulative)
    The total number of search requests per cluster
  • counter.indices.total.translog.earliest_last_modified_age (cumulative)
  • counter.indices.total.translog.uncommitted_operations (cumulative)
  • counter.indices.total.translog.uncommitted_size_in_bytes (cumulative)
  • counter.jvm.gc.count (cumulative)
  • counter.jvm.gc.old-count (cumulative)
  • counter.jvm.gc.old-time (cumulative)
  • counter.jvm.gc.time (cumulative)
    Total garbage collection time (milliseconds)
  • counter.jvm.uptime (cumulative)
  • counter.thread_pool.completed (cumulative)
  • counter.thread_pool.rejected (cumulative)
    Number of rejected thread pool requests
  • counter.transport.rx.count (cumulative)
  • counter.transport.rx.size (cumulative)
  • counter.transport.tx.count (cumulative)
  • counter.transport.tx.size (cumulative)
  • gauge.cluster.active-primary-shards (gauge)
    The number of active primary shards
  • gauge.cluster.active-shards (gauge)
    The number of active shards
  • gauge.cluster.initializing-shards (gauge)
    The number of currently initializing shards
  • gauge.cluster.number-of-data_nodes (gauge)
    The current number of data nodes in the cluster
  • gauge.cluster.number-of-nodes (gauge)
    Total number of nodes in the cluster
  • gauge.cluster.relocating-shards (gauge)
    The number of shards that are currently being relocated
  • gauge.cluster.status (gauge)
    The health status of the cluster
  • gauge.cluster.unassigned-shards (gauge)
    The number of shards that are currently unassigned
  • gauge.http.current_open (gauge)
  • gauge.indices.cache.field.size (gauge)
    Field data size (bytes)
  • gauge.indices.cache.filter.size (gauge)
    Filter cache size (bytes)
  • gauge.indices.docs.count (gauge)
    Number of documents on this node
  • gauge.indices.docs.deleted (gauge)
    Number of deleted documents on this node
  • gauge.indices.get.current (gauge)
  • gauge.indices.indexing.delete-current (gauge)
  • gauge.indices.indexing.index-current (gauge)
  • gauge.indices.merges.current (gauge)
    Number of active merges
  • gauge.indices.merges.current-docs (gauge)
  • gauge.indices.merges.current-size (gauge)
  • gauge.indices.merges.total-docs (gauge)
  • gauge.indices.primaries.completion.size (gauge)
  • gauge.indices.primaries.docs.count (gauge)
  • gauge.indices.primaries.docs.deleted (gauge)
  • gauge.indices.primaries.fielddata.memory-size (gauge)
  • gauge.indices.primaries.flush.periodic (gauge)
  • gauge.indices.primaries.get.current (gauge)
  • gauge.indices.primaries.indexing.delete-current (gauge)
  • gauge.indices.primaries.indexing.index-current (gauge)
  • gauge.indices.primaries.merges.current (gauge)
  • gauge.indices.primaries.merges.current-docs (gauge)
  • gauge.indices.primaries.merges.current-size (gauge)
  • gauge.indices.primaries.search.fetch-current (gauge)
  • gauge.indices.primaries.search.open-contexts (gauge)
  • gauge.indices.primaries.search.query-current (gauge)
  • gauge.indices.primaries.segments.index-writer-memory (gauge)
  • gauge.indices.primaries.segments.memory (gauge)
  • gauge.indices.primaries.segments.version-map-memory (gauge)
  • gauge.indices.primaries.store.size (gauge)
  • gauge.indices.primaries.translog.earliest_last_modified_age (gauge)
  • gauge.indices.primaries.translog.size (gauge)
  • gauge.indices.primaries.translog.uncommitted_operations (gauge)
  • gauge.indices.primaries.translog.uncommitted_size_in_bytes (gauge)
  • gauge.indices.primaries.warmer.current (gauge)
  • gauge.indices.search.fetch-current (gauge)
  • gauge.indices.search.open-contexts (gauge)
  • gauge.indices.search.query-current (gauge)
  • gauge.indices.search.scroll.current (gauge)
  • gauge.indices.segments.count (gauge)
    Number of segments on this node
  • gauge.indices.segments.index-writer-size (gauge)
  • gauge.indices.segments.size (gauge)
  • gauge.indices.store.size (gauge)
  • gauge.indices.total.docs.count (gauge)
    Number of documents in the cluster
  • gauge.indices.total.docs.deleted (gauge)
  • gauge.indices.total.fielddata.memory-size (gauge)
    Field data size (bytes)
  • gauge.indices.total.filter-cache.memory-size (gauge)
    Filter cache size (bytes)
  • gauge.indices.total.get.current (gauge)
  • gauge.indices.total.indexing.delete-current (gauge)
  • gauge.indices.total.indexing.index-current (gauge)
  • gauge.indices.total.merges.current (gauge)
  • gauge.indices.total.merges.current-docs (gauge)
  • gauge.indices.total.merges.current-size (gauge)
  • gauge.indices.total.search.open-contexts (gauge)
  • gauge.indices.total.search.query-current (gauge)
  • gauge.indices.total.store.size (gauge)
  • gauge.indices.translog.uncommitted_operations (gauge)
  • gauge.indices.translog.uncommitted_size_in_bytes (gauge)
  • gauge.jvm.mem.heap-committed (gauge)
    Total heap committed by the process (bytes)
  • gauge.jvm.mem.heap-used (gauge)
    Total heap used (bytes)
  • gauge.jvm.mem.non-heap-committed (gauge)
  • gauge.jvm.mem.non-heap-used (gauge)
  • gauge.jvm.mem.pools.old.max_in_bytes (gauge)
  • gauge.jvm.mem.pools.old.used_in_bytes (gauge)
  • gauge.jvm.mem.pools.young.max_in_bytes (gauge)
  • gauge.jvm.mem.pools.young.used_in_bytes (gauge)
  • gauge.jvm.threads.count (gauge)
  • gauge.jvm.threads.peak (gauge)
  • gauge.process.cpu.percent (gauge)
  • gauge.process.open_file_descriptors (gauge)
    Number of currently open file descriptors
  • gauge.thread_pool.active (gauge)
    Number of active threads
  • gauge.thread_pool.largest (gauge)
    Highest active threads in thread pool
  • gauge.thread_pool.queue (gauge)
    Number of Tasks in thread pool
  • gauge.thread_pool.threads (gauge)
    Number of Threads in thread pool
  • gauge.transport.server_open (gauge)
  • percent.jvm.mem.heap-used-percent (gauge)

Non-default metrics (version 4.7.0+) 🔗

To emit metrics that are not default, you can add those metrics in the generic monitor-level extraMetrics config option. Metrics that are derived from specific configuration options that do not appear in the above list of metrics do not need to be added to extraMetrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.