Docs » Detectors and Alerts

Detectors and Alerts 🔗

SignalFx uses detectors, events, alerts, and notifications to keep you informed when certain criteria are met. For example, you might want a message sent to a Slack channel or to an email address for the Ops team when CPU Utilization has reached the 95th percentile, or when the number of concurrent users is approaching a limit that might require you to spin up an additional AWS instance.

The interaction between detectors, events, alerts, and notifications is as follows:

  • Detectors contain rules that specify:
    • When the detector will be triggered, based on conditions related to the detector’s signal
    • The severity of the alert to be generated by the detector
    • Where notifications should be sent
  • When a detector is triggered, it does the following:
    • Generates an event, which can be viewed on charts and in the Events sidebar
    • Triggers an alert, which can be viewed in a number of places throughout SignalFx (also known as Splunk Infrastructure Monitoring)
    • Sends one or more notifications, so people are informed about the alert even if they are not currently monitoring dashboards.
  • When the condition clears, the detector generates a second event and sends a second set of notifications.

In the following sections, View Alerts explains how to view alerts based on detectors that may have already been configured for your organization. Manage Notifications explains how to specify where alert notifications should be sent, and Mute Notifications explains how to temporarily mute (stop sending) notifications. View Additional Data with Events explains how you can create and view events to supplement alert information. Set Up Detectors to Trigger Alerts and Using Built-in Alert Conditions explain how to create detectors so they generate the events, alerts, and notifications that meet your monitoring requirements. zombie-alerts explains the default setting that automatically clears alerts generated by a metric that stops reporting.

If you want to determine why a detector did not trigger an alert you thought it would, or triggered an alert unexpectedly, then detectors-troubleshooting explains how to understand and troubleshoot detector settings.

After a detector is created, you can link it to a chart to display its alert status on the chart.