Docs » Integrations Guide » Integrations Reference » Jenkins

../../_images/integrations_jenkins.png Jenkins ๐Ÿ”—

DESCRIPTION ๐Ÿ”—

This integration primarily consists of the Smart Agent monitor collectd/jenkins. Below is an overview of that monitor.

Smart Agent Monitor ๐Ÿ”—

Monitors jenkins by using the jenkins collectd Python plugin, which collects metrics from Jenkins instances by hitting these endpoints: ../api/json (job metrics) and metrics/<MetricsKey>/.. (default and optional Codahale/Dropwizard JVM metrics).

Requires Jenkins 1.580.3 or later, as well as the Jenkins Metrics Plugin (see Setup).

INSTALLATION ๐Ÿ”—

This integration is part of the SignalFx Smart Agent as the collectd/jenkins monitor. You should first deploy the Smart Agent to the same host as the service you want to monitor, and then continue with the configuration instructions below.

CONFIGURATION ๐Ÿ”—

To activate this monitor in the Smart Agent, add the following to your agent config:

monitors:  # All monitor config goes under this key
 - type: collectd/jenkins
   ...  # Additional config

For a list of monitor options that are common to all monitors, see Common Configuration.

Config option Required Type Description
pythonBinary no string Path to a python binary that should be used to execute the Python code. If not set, a built-in runtime will be used. Can include arguments to the binary as well.
host yes string
port yes integer
metricsKey yes string Key required for collecting metrics. The access key located at Manage Jenkins > Configure System > Metrics > ADD. If empty, click Generate.
enhancedMetrics no bool Whether to enable enhanced metrics (default: false)
includeMetrics no list of strings Used to enable individual enhanced metrics when enhancedMetrics is false
username no string User with security access to jenkins
apiToken no string API Token of the user
sslKeyFile no string Path to the keyfile
sslCertificate no string Path to the certificate
sslCACerts no string Path to the ca file

USAGE ๐Ÿ”—

Interpreting Built-in dashboards ๐Ÿ”—

  • Jenkins:

    • Alive Status: Shows the number of Jenkins Masters that are alive.

      ../../_images/chart-jenkins-alive-status.png

    • Health Score: Shows the mean health score of each Jenkins instance on all hosts.

      ../../_images/chart-jenkins-health-score.png

    • Job Failure Rate: Shows the rate of jobs failed in the past day.

      ../../_images/chart-jenkins-job-failure-rate.png

    • Executor Usage: Shows the usage pattern of the executors. Gives an overview of the load on the Jenkins instances.

      ../../_images/chart-jenkins-executor-usage.png

    • Top 5 Failed Jobs: Shows the top 5 failed jobs over the past day based on the total failure count.

      ../../_images/chart-jenkins-top-5-failed-jobs.png

    • Busy Executors vs Pending Jobs: A line graph showing comparison between in-use executors and pending jobs in queue. On comparing this chart with two above, reason for job failures can be narrowed down further quickly.

      ../../_images/chart-jenkins-busy-executors-vs-pending-jobs.png

    • Average Duration - Past Day: Shows average duration of top 5 jobs that are taking the most time.

      ../../_images/chart-jenkins-average-duration-past-day.png

    • Slave Status: Shows the number of slave agents that are alive.

      ../../_images/chart-jenkins-slave-status.png

    • VM Memory Utilization: Area graph of the memory used by each Jenkins JVM.

      ../../_images/chart-jenkins-vm-memory-utilization.png

    • Heap Usage: Line graph of the utilization percentage of Heap memory by each Jenkins instance.

      ../../_images/chart-jenkins-heap-usage.png

    • Non-Heap Used: Line graph of the non-heap memory used by each Jenkins instance.

      ../../_images/chart-jenkins-non-heap-used.png

  • Jenkins Master:

    • Top 5 Failed Jobs: Shows the top 5 failed jobs over the past day based on the total failure count in an instance(s).

      ../../_images/chart-jenkins-master-top-5-failed-jobs.png

    • Health Checks: The status of each health check as reported by DropWizard Metrics. This gives a quick overview of whatโ€™s wrong with the instance.

      ../../_images/chart-jenkins-master-health-checks.png

    • Slave Status: Shows the number of slave agents of the instance(s) that are alive.

      ../../_images/chart-jenkins-master-slave-status.png

    • Busy Executors vs Pending Jobs: A line chart showing comparison between in-use executors and pending jobs in queue in an instance(s). On comparing this chart with two above, reason for job failures can be narrowed down further quickly.

      ../../_images/chart-jenkins-master-busy-executors-vs-pending-jobs.png

    • VM Memory Utilization: Area chart of the memory used by the Jenkins JVM instance(s) on a host.

      ../../_images/chart-jenkins-master-vm-memory-utilization.png

All DropWizard metrics reported by the jenkins collectd plugin will not contain any dimensions by default. Whereas, the job metrics sent will contain the following dimensions by default:

  • Job, name of the job
  • Result, the status of the job

A few other details:

  • plugin is always set to jenkins
  • plugin_instance will contain the IP address and the port of the member given in the configuration
  • To add metrics from the /metrics/<MetricsKey>/metrics endpoint, use the configuration options mentioned in configuration. If metrics are being included individually, make sure to give names that are valid. For example, vm.daemon.count or vm.terminated.count

METRICS ๐Ÿ”—

Metric Name Description Type
gauge.jenkins.job.duration Time taken to complete the job in ms gauge
gauge.jenkins.node.executor.count.value Total Number of executors in an instance gauge
gauge.jenkins.node.executor.in-use.value Total number of executors being used in an instance gauge
gauge.jenkins.node.health-check.score Mean health score of an instance gauge
gauge.jenkins.node.health.disk.space Binary value of disk space health gauge
gauge.jenkins.node.health.plugins Boolean value indicating state of plugins gauge
gauge.jenkins.node.health.temporary.space Binary value of temporary space health gauge
gauge.jenkins.node.health.thread-deadlock Boolean value indicating a deadlock gauge
gauge.jenkins.node.online.status Boolean value of instance is reachable or not gauge
gauge.jenkins.node.queue.size.value Total number pending jobs in queue gauge
gauge.jenkins.node.slave.online.status Boolean value for slave is reachable or not gauge
gauge.jenkins.node.vm.memory.heap.usage Percent utilization of the heap memory gauge
gauge.jenkins.node.vm.memory.non-heap.used Total amount of non-heap memory used gauge
gauge.jenkins.node.vm.memory.total.used Total Memory used by instance gauge

gauge.jenkins.job.duration ๐Ÿ”—

gauge

Time taken to complete the job in ms.

gauge.jenkins.node.executor.count.value ๐Ÿ”—

gauge

Total Number of executors in an instance

gauge.jenkins.node.executor.in-use.value ๐Ÿ”—

gauge

Total number of executors being used in an instance

gauge.jenkins.node.health-check.score ๐Ÿ”—

gauge

Mean health score of an instance

gauge.jenkins.node.health.disk.space ๐Ÿ”—

gauge

Binary value of disk space health

gauge.jenkins.node.health.plugins ๐Ÿ”—

gauge

Boolean value indicating state of plugins

gauge.jenkins.node.health.temporary.space ๐Ÿ”—

gauge

Binary value of temporary space health

gauge.jenkins.node.health.thread-deadlock ๐Ÿ”—

gauge

Boolean value indicating a deadlock

gauge.jenkins.node.online.status ๐Ÿ”—

gauge

Boolean value of instance is reachable or not

gauge.jenkins.node.queue.size.value ๐Ÿ”—

gauge

Total number pending jobs in queue

gauge.jenkins.node.slave.online.status ๐Ÿ”—

gauge

Boolean value for slave is reachable or not

gauge.jenkins.node.vm.memory.heap.usage ๐Ÿ”—

gauge

Percent utilization of the heap memory

gauge.jenkins.node.vm.memory.non-heap.used ๐Ÿ”—

gauge

Total amount of non-heap memory used

gauge.jenkins.node.vm.memory.total.used ๐Ÿ”—

gauge

Total Memory used by instance

These are the metrics available for this monitor. Metrics that are categorized as container/host (default) are in bold and italics in the list below.

  • gauge.jenkins.job.duration (gauge)
    Time taken to complete the job in ms.
  • gauge.jenkins.node.executor.count.value (gauge)
    Total Number of executors in an instance
  • gauge.jenkins.node.executor.in-use.value (gauge)
    Total number of executors being used in an instance
  • gauge.jenkins.node.health-check.score (gauge)
    Mean health score of an instance
  • gauge.jenkins.node.health.disk.space (gauge)
    Binary value of disk space health
  • gauge.jenkins.node.health.plugins (gauge)
    Boolean value indicating state of plugins
  • gauge.jenkins.node.health.temporary.space (gauge)
    Binary value of temporary space health
  • gauge.jenkins.node.health.thread-deadlock (gauge)
    Boolean value indicating a deadlock
  • gauge.jenkins.node.online.status (gauge)
    Boolean value of instance is reachable or not
  • gauge.jenkins.node.queue.size.value (gauge)
    Total number pending jobs in queue
  • gauge.jenkins.node.slave.online.status (gauge)
    Boolean value for slave is reachable or not
  • gauge.jenkins.node.vm.memory.heap.usage (gauge)
    Percent utilization of the heap memory
  • gauge.jenkins.node.vm.memory.non-heap.used (gauge)
    Total amount of non-heap memory used
  • gauge.jenkins.node.vm.memory.total.used (gauge)
    Total Memory used by instance

Non-default metrics (version 4.7.0+) ๐Ÿ”—

The following information applies to the agent version 4.7.0+ that has enableBuiltInFiltering: true set on the top level of the agent config.

To emit metrics that are not default, you can add those metrics in the generic monitor-level extraMetrics config option. Metrics that are derived from specific configuration options that do not appear in the above list of metrics do not need to be added to extraMetrics.

To see a list of metrics that will be emitted you can run agent-status monitors after configuring this monitor in a running agent instance.

Legacy non-default metrics (version < 4.7.0) ๐Ÿ”—

The following information only applies to agent version older than 4.7.0. If you have a newer agent and have set enableBuiltInFiltering: true at the top level of your agent config, see the section above. See upgrade instructions in Old-style whitelist filtering.

If you have a reference to the whitelist.json in your agentโ€™s top-level metricsToExclude config option, and you want to emit metrics that are not in that whitelist, then you need to add an item to the top-level metricsToInclude config option to override that whitelist (see Inclusion filtering. Or you can just copy the whitelist.json, modify it, and reference that in metricsToExclude.