Docs » Detectors and Alerts » Set Up Detectors to Trigger Alerts

Set Up Detectors to Trigger Alerts

About detectors

A detector monitors signals on a plot line, as on a chart, and triggers alert events and clear events based on conditions you define in rules. Conceptually, you can think of a detector as a chart that can trigger alerts when a signal’s value crosses specified thresholds defined in alert rules.

../_images/detector.png


../_images/detector-rules-tab-legend.png

Note

If a detector contains a Signalflow tab, as shown in the illustration below, you are viewing a detector created programmatically using the SignalFx v2 API. If you are familiar with the API, you can use this screen to view and edit the detector code and make changes to the detector rules, instead of using the techniques described in this section.

../_images/v2-detector-signalflow.png

Alert rules use settings you specify for built-in alert conditions to define thresholds that will trigger alerts. When a detector determines that the conditions for a rule are met, it will trigger an alert, create an event, and send notifications (if specified). Detectors can send notifications via email, as well as via other systems, such as Hipchat or Slack, or via a webhook.

Active alerts and existing detectors can be found in tabs on the Alerts page, and events can be found in the Events sidebar, available from within any dashboard.

The main steps involved in creating a detector are choosing how to create the detector and building rules to specify when alerts should be triggered.

When you specify alert settings in a detector rule, an alert preview is displayed. The alert preview can help you adjust your settings as you create the detector, but also helps when you are editing existing detectors.

Choosing how to create the detector

There are several ways to begin the process of creating a detector.

  • Cloning an existing detector is recommended if you (or someone else in your organization) have already created useful detectors that you want to modify for your particular requirements (or if you just want to play around with to learn more about detectors!)
  • Starting from built-in templates visible in the Detector menu (bell icon) is often a good way to create detectors that will be immediately useful, based on what you are currently viewing (a chart or the Infrastructure Navigator).
  • Creating a detector from a chart is recommended if neither of the previous techniques meets your needs.
  • Creating a detector from scratch is generally recommended only if you are fairly proficient at creating charts from scratch, and none of the previous techniques meets your needs.
  • Using the SignalFx v2 API lets you programmatically create detectors, instead of creating them through the user interface.

Cloning an existing detector

You can see a list of existing detectors in the Detectors tab on the Alerts page. If you see one that represents a good starting point for a detector you want to create, you can open it and then select Clone from the detector’s Actions menu to save it as a new detector.

Once you have saved the new detector, skip to Building detector rules.

Creating a detector from a built-in template

SignalFx provides templates for our recommended detectors. These built-in templates allow you to create intelligent detectors based on SignalFx’s powerful analytics, without having to become an analytics expert. These templates are available from the Detector menu. Open the Detector menu by clicking the bell icon at the top right corner of a chart, or next to a heading in the Infrastructure Navigator.

Tip

When you open the Detector menu, you may see one or more related detectors that have already been created from built-in templates. Before creating a new detector, view this list of detectors; you may be able to simply subscribe to a related detector instead of creating a new detector.

When built-in detector templates are available and related to a chart, they will be visible in the Detector menu, usually under Create new detector ‣ From built-in template.

../_images/detector-from-template.png

You can also create a detector based on a template from the Templates tab on the Alerts page.

Once you have created the detector, skip to Building detector rules.

Creating a detector from a chart

If there is a chart that monitors a signal that you want to alert on, you can use that chart to create a detector. Creating a detector from a chart pre-selects one of the chart’s signals as the signal to be monitored.

To create the detector, open the Detector menu (bell icon) for the chart and select New detector from chart.

Tip

When you open the Detector menu, you may see one or more related detectors that have already been created from built-in templates. Before creating a new detector, view this list of detectors; you may be able to simply subscribe to a related detector instead of creating a new detector.

Once you have created the detector, skip to Building detector rules.

Creating a detector from scratch

It’s generally preferable to create a new detector using one of the previous techniques, so you have a solid starting point. The most useful and powerful detectors can be quite complex; the best way to learn how to create detectors is to see how existing ones are built.

To create a new detector from scratch, you can either click the New Detector button on the Alerts or Detectors tab on the Alerts page, or select Detector from the Create menu (plus sign) on the navigation bar.

Once you have created the detector, skip to Building detector rules.

Creating a detector using the SignalFx v2 API

Using the API to create a detector provides a number of capabilities that are not available in the UI, letting you build detectors with more advanced rules and conditions. You can view these detectors in the UI; the program text is displayed in place of the signals displayed in standard detectors. Some minor editing is possible, but in general it is recommended that you edit and test any significant changes via the API. For more information, see SignalFx v2 API.

../_images/v2-detector-signalflow.png

Building detector rules

If you are creating a new detector, the Alert Rule Builder will be displayed automatically. To view the Alert Rule Builder for an existing rule, click Edit to the right of the rule in the Alert Rules tab.

Note

If you are editing a detector that was created programmatically using the SignalFx v2 API, you won’t see the Alert Rule Builder, as the process for creating and editing detector rules is different for v2 detectors. For information on building rules for v2 detectors, see Building Rules in a v2 Detector.

  • In the Alert signal tab, you select one or more signals to monitor for unusual behavior.
  • In the Alert condition and Alert settings tabs, you specify criteria for triggering an alert.
  • In the Alert message and Alert recipients tabs, you specify who should receive notifications and add notes that will be included in the notifications.
  • In the Activate tab, you name the rule and make the detector “live.”

After you activate the detector, it will begin monitoring the signal immediately. When the signal meets the specified criteria, the detector will trigger alerts, create events, and send the specified message to the alert recipients.

Each tab is discussed below.

Alert signal

In the Alert signal tab, you define the signal to monitor by entering a metric and corresponding analytics.

If you are creating a detector from scratch, you must first specify the signal(s) you want to monitor. Specifying a signal for a detector is similar to specifying a signal in a chart in the Plot Editor tab in the Chart Builder. When you start typing, a dropdown list of metrics and events is displayed. Select the metric you want to monitor, and then add any filters or analytics.

If you want to add more signals, click Add Metric or Event or Add Formula. Note that you can add events to be displayed on the chart, but you cannot select an event as the signal to be monitored.

Note

If you are creating a detector from a chart or by cloning a detector, you may not need to add any new signals. However, if you do add new signals to the detector, the signals will not be added to the original chart or detector.

If the detector has multiple signals, you can choose whether to monitor one or more signals.

  • If you want to monitor one signal (the most common use case), click the bell icon at the far left to specify which signal will be monitored. A blue bell indicates the signal that is being monitored.

  • If you want to create compound conditions based on the values of more than one signal (for example, signal A is above x OR signal B is above y), click the multiple signals icon. Note that this option is available only if the alert condition (see next section) is Custom Threshold.

    ../_images/mult-signals.png

Continue to the next tab to select a condition for the detector’s rule.

Alert condition

In the Alert condition tab, you select the type of condition that will trigger an alert.

SignalFx provides several built-in alert conditions to make it simple for you to create robust alert conditions without needing to build advanced conditions behind the scenes.

The following table summarizes the available built-in alert conditions.

Name Description Summary (samples)
Static Threshold Alert when a signal crosses a static threshold Availability over the last day is below 99.9.
Heartbeat Check Alert when a signal has stopped reporting for some time Host-linux-001 has not reported for 15 minutes.
Resource Running Out Detect when a signal is projected to reach a specified minimum or maximum value disk_space_available is projected to decrease to zero within 24 hours. cpu.utilization is projected to reach 95 within 2 hours.
Outlier Detection Alert when the signal from one data source differs from similar data sources The number of logins in the last 10 minutes for this instance is 3 standard deviations lower than other instances in the same AWS availability zone.
Sudden Change Alert when a signal is different from its normal behavior (based on mean of preceding window or percentile of preceding window) All the values for cpu.utilization received in the last 15 minutes are at least 3 standard deviations above the mean of the preceding hour. All the values for latency received in the last 10 minutes are greater than 99% of the values of the preceding 1 hour.
Historical Anomaly Alert when a signal differs by a specified amount when compared to similar periods in the past The average number of logins in the last 2 hours is [30% higher] [3 standard deviations higher] than the average for this same two hours last week.
Custom Threshold Alert when a signal crosses another signal, or when you want to specify compound conditions using AND and OR operators. Example 1 - The value for cache_misses is above cache_hits. Example 2 - The value for cache_misses is above cache_hits OR the value for cache_misses_percent is above 10.

Notes

  • If you have chosen to monitor multiple signals, the only available alert condition is Custom Threshold.
  • If you want to create compound conditions using AND or OR operators on the Alert Settings tab, you must use the Custom Threshold condition. This limitation applies whether you are monitoring a single signal or multiple signals.

After you have selected the alert condition, continue to the next tab to specify the settings that will trigger alerts.

Alert settings

In the Alert settings tab, you specify the settings that will trigger an alert. The available settings vary depending on the alert condition you selected.

Tip

If you are using the Custom Threshold condition, you can click Add another condition to create compound conditions using AND and OR operators. For more information about compound conditions, see Custom Threshold.

In the chart, you see a preview of the alerts that would have been triggered based on the settings you selected. For more information on using the preview, see Previewing the results of a detector.

After you have specified settings for triggering alerts, continue to the next tab to create a message that will be sent when the alert is triggered.

Alert message

In the Alert message tab, you specify the severity of the alert and the information you want to include in the notification message.

Severity

Specify the importance of the alerts generated by this rule. SignalFx has 5 severity labels: Critical, Major, Minor, Warning and Info. Each severity label has a different color, and event markers appear on charts in the associated color.

You can create multiple rules to generate alerts with different severity levels for similar conditions, for example:

  • Critical alert for the alert condition Resource Running Out set to low trigger sensitivity
  • Major alert for the same condition set to medium sensitivity
  • Minor alert for same the condition set to high sensitivity

Another example might be:

  • Critical alert for the alert condition Heartbeat Check where the value for Hasn’t reported for is 60 minutes
  • Major alert for the same condition set at 30 minutes
  • Minor alert for same the condition set at 15 minutes

The easiest way to do this is to create a rule at one severity, select Clone from the rule’s Actions menu on the right side of the screen, then edit the settings and severity.

Runbook

Optional; enter the URL of a SignalFx dashboard or team landing page, wiki page, etc. to include in the notification message. Adding a runbook URL can help a recipient resolve an alert more quickly.

Tip

Optional; enter a suggested first course of action to include in the notification message, such as a command line to execute, or a note like “If you are on call, review immediately.” Alternatively, you can add more general information, such as “This is a test alert - ok to ignore.”

Message preview

Displays a default message that will be sent when an alert is triggered or cleared. To edit the subject or the content of the message, click Customize; you’ll see the code and variables used to construct the message. Available variables are shown to the right of the message area while you are editing the message.

Note that the use of variables is supported only in the message subject and body, not in the Runbook or Tip fields.

../_images/customize-message.png

You can also use Markdown in the message.

When entering a variable in the message, typing the first few letters will narrow down the list of variables shown on the right. If only one is shown, pressing Tab will add it to the message. If more than one is shown, pressing Tab will add the first one in the list to the message.

The following tables describe the variables and helper functions you can use when creating a custom message. Use triple braces where indicated so that the variable value will not get escaped.

Note

Different additional variables may be available depending on the alert condition you specify. If you change the alert condition after customizing the message, an icon on the Message preview tab is displayed.

../_images/message-tab-icon.png

This is to remind you to review the message, since some variables you used might no longer apply to the new condition you selected. The icon is removed when you navigate away from the Message preview tab.


DETECTOR AND RULE DETAILS

Variable Description
{{{ detectorName }}} The name of this detector
{{{ ruleName }}} The name of the rule that triggered the alert
{{ ruleSeverity }} The severity of this rule (Critical, Major, Minor, Warning, Info)
{{{ readableRule }}}
The readable description of this rule, e.g.
“The value of metric.name.here is above 100”
{{{ runbookUrl }}} URL of page to consult when this alert is triggered
{{{ tip }}} Plain text suggested first course of action, such as a command line to execute
{{ detectorId }} The ID of this detector (can be used to programmatically reference this detector)
{{ detectorUrl }} The URL of this detector


ALERT DETAILS

Variable Description
{{ timestamp }}
The GMT timestamp of this alert, in this format:
Fri, 13 Oct 2017 20:32:39 GMT
{{ anomalyState }} The state of this alert (OK or ANOMALOUS)
{{ anomalous }} Boolean; true indicates that the alert triggered
{{ normal }} Boolean; true indicates that the alert cleared
{{ imageUrl }} The URL for the preview image shown in the notification message
{{ incidentId }} The ID of this incident (the incidentID is the same for both the trigger and the clear alerts)


SIGNAL DETAILS

Variable Description
{{ inputs.A.value }} The value of the signal on plot line A
{{ inputs.B.value... }} (The value of other signals in the detector)
{{{ dimensions }}}
List of all dimensions for the signal being monitored, in the following format:
{sf_metric=metricName, dimensionNameA=valueA, dimensionNameB=valueB, ...}
{{ dimensions.dimensionName }} The value of the dimension “dimensionName” for the signal being monitored
{{ dimensions.dimensionName2... }} (The value of other dimensions for the signal being monitored)


ORGANIZATION DETAILS

Variable Description
{{ organizationId }} The organization ID (can be used to programmatically reference this organization)


HELPER FUNCTIONS

Option Description
{{ #if }} {{ else }} {{ /if }}
Conditional, e.g.
{{ #if anomalous }}Alert triggered at {{ timestamp }} {{ else }} Alert cleared at {{ timestamp }} {{ /if }}
{{ #notEmpty dimensions }} {{ /notEmpty }}
If there are dimensions associated with the signal, e.g.
{{ #notEmpty dimensions }} Signal details: {{{ dimensions }}} {{ /notEmpty }}

Below is an example of a default message that you can customize.

{{#if anomalous}}
   Rule "{{ruleName}}" in detector "{{detectorName}}" triggered at {{timestamp}}.
{{else}}
   Rule "{{ruleName}}" in detector "{{detectorName}}" cleared at {{timestamp}}.
{{/if}}

{{#if anomalous}}
Triggering condition: {{{readableRule}}}
{{/if}}

{{#if anomalous}}Signal value: {{inputs.A.value}}
{{else}}Current signal value: {{inputs.A.value}}
{{/if}}

{{#notEmpty dimensions}}
Signal details:
{{{dimensions}}}
{{/notEmpty}}

{{#if anomalous}}
{{#if runbookUrl}}Runbook: {{{runbookUrl}}}{{/if}}
{{#if tip}}Tip: {{{tip}}}{{/if}}
{{/if}}

The following tables describe the additional variables you can use when creating a custom message for specific alert conditions.

RESOURCE RUNNING OUT

Variable Description
{{ inputs.hours_left.value }} Number of hours left before reaching empty or capacity
{{ event_annotations.fire_forecast_ahead }} Threshold for triggering alert (number of hours)
{{ event_annotations.clear_forecast_ahead }} Threshold for clearing alert (number of hours)


OUTLIER DETECTION

Variable Description
{{ inputs.promoted_population_stream.value }} Signal being monitored
{{ inputs.fire_bot.value }} Threshold for triggering alert (when value is below threshold)
{{ inputs.clear_bot.value }} Threshold for clearing alert
{{ inputs.fire_top.value }} Threshold for triggering alert (when value is above threshold)
{{ inputs.clear_top.value }} Threshold for clearing alert


SUDDEN CHANGE

Variable Description
{{ event_annotations.current_window }} Window being tested for anomalous values
{{ inputs.recent_min.value }} Minimum value during current window
{{ inputs.recent_max.value }} Maximum value during current window
{{ inputs.f_bot.value }} Threshold for triggering alert (when value is below threshold)
{{ inputs.c_bot.value }} Threshold for clearing alert
{{ inputs.f_top.value }} Threshold for triggering alert (when value is above threshold)
{{ inputs.c_top.value }} Threshold for clearing alert


HISTORICAL ANOMALY

Variable Corresponds to
{{ event_annotations.current_window }} Window being tested for anomalous values
{{ inputs.summary.value }} Mean value during current window
{{ inputs.fire_bot.value }} Threshold for triggering alert (when value is below threshold)
{{ inputs.clear_bot.value }} Threshold for clearing alert
{{ inputs.fire_top.value }} Threshold for triggering alert (when value is above threshold)
{{ inputs.clear_top.value }} Threshold for clearing alert

After you have created an alert message, continue to the next tab to specify where alert messages will be sent.

Alert recipients

In the Alert recipients tab, you specify where notification messages should be sent when alerts are triggered or cleared. Recipients are considered subscribers to a rule.

If you have previously integrated your alerts with another system, those options will appear in the Add Recipient dropdown menu. You can also send to email addresses, webhook URLs, and teams. Notifications are also sent when the condition clears.

Adding recipients is optional, but highly recommended.

Tips

If you want to add the same subscriber(s) to each of multiple rules, you can add the subscribers to all rules at once by using the Manage subscriptions option in the Detectors tab on the Alerts page after you save the detector.

You can temporarily stop a detector from sending notifications by muting notifications.

Activate...

In the Activate tab you see a summary of the detector settings you specified. Review the summary and make any necessary changes in the associated tabs, then name the rule; by default, the rule name is the same as the detector name. The rule name is displayed on the Alerts page and in notifications.

Click Activate Alert Rule to save the detector and begin monitoring the specified signal. After you activate the detector, the Alert Rules tab of the detector is displayed, showing the signal you selected and a summary of the rule you built. At this point, you can edit the detector name (shown at upper left); the text you enter here is displayed as the detector name in the Detectors tab on the Alerts page. You can also provide additional descriptive text below the name, for example to clarify the purpose of the detector for others.

Important

If you make any changes to the detector name or description, be sure to click the green Save button. If you click the Close button without saving, your changes will be lost.

../_images/new-detector-rules-tab.png

Previewing the results of a detector

Setting up the right alerts is usually a process of trial and error. You set up an alert, get notified when it is triggered (or worse, don’t get notified when it should have triggered), adjust the trigger condition if necessary, and wait for the next alert. SignalFx provides an alert preview functionality that accelerates this trial and error process.

Note

Alert preview is not available in the UI for detectors that were created using the SignalFx v2 API. However, you can use the /signalflow/preflight call in the API to find out how many times a v2 detector would have triggered and cleared alerts during a specified time period.

How alert preview works

When you go to the Alert Settings tab while creating or editing a detector rule, SignalFx runs the settings against your data and generates a preview chart to help you decide if your settings are appropriate for your requirements. The preview indicates when alerts would have triggered during the time range specified in the detector. You can use the preview to fine-tune the settings that will trigger the alert, to assure that you will receive the alerts you expect when you expect them.

../_images/previewed-alerts.png

As is standard, data displayed in the chart may be rolled up based on the detector’s time range. The detailed view displays data at detector resolution for the selected time period on the chart; that is, datapoints are shown at the frequency with which the detector is monitoring the signal to determine if an alert should be triggered or cleared. This view lets you see exactly what data the detector is viewing for the period selected in the chart. You can click and drag the left edge of the detailed view to enlarge it and gain more visibility into the data.

You can navigate between tabs while the preview is shown. For example, you might want to go to the Alert Signal tab and look at the Data Table to review the values of your signal before, during, and after the time an alert would be triggered.

Note

A preview can be generated for a period of time up to the previous 1 week (-1w). If the time range on the detector is greater than one week (i.e. if you have specified -31d to see data for the previous month), the preview will display alert markers only for the previous 1-week period.

The following sections provide examples of how you can use the preview function to tailor detectors to your requirements.

Modify an existing detector based on past activity

If you already have detectors triggering alerts in your system, you may find that some of them are either too sensitive (they trigger alerts too often) or are not sensitive enough (they don’t trigger alerts when you want to be notified about an issue). In either case, you can use the preview option to modify a detector’s settings so it triggers the appropriate number of alerts.

To use the preview option for the detector, open the detector and set a time range of, for example, -1w to display all alerts that were triggered in the last week. (Alternately, you can set a shorter time range that you feel is more representative of the issue you are addressing). Display the Alert Rules tab, edit the alert condition, and open the Alert Settings tab. The chart display will change from showing actual past alerts (if any) to showing a preview of alerts that would have triggered based on the alert condition.

Depending on the type of alert condition, you have different options for making the detector more or less sensitive. For example, if you see more markers than you think you should (alert is too flappy), you might be able to change Trigger Sensitivity from High to Medium or Low, or you might be able to change the sensitivity from triggering immediately to triggering after a period of time. For many alert conditions, you can choose Custom for Trigger Sensitivity, which gives you more control over when alerts will be triggered.

As you make these changes, the preview is updated and the chart will display the number of alerts that would have triggered based on your new settings. When you have modified the detector to more accurately reflect the desired behavior, click Activate and then update the alert rule.

Note

When you save a detector with modified alert conditions, the status of any currently active alerts will be set to “Stopped”.

Create a new detector based on a recent alert

Suppose a signal spiked in the past couple of hours, and you are creating a new detector to monitor for that spike. You would add the signal that spiked on the Signals tab, then choose an appropriate condition on the Alert Conditions tab.

Tip

If you already have a chart that includes the signal as a plot line, you can create a detector from the chart instead of creating a new detector from scratch. Simply choose that signal when you begin creating the rule in the Alert Conditions tab.

In this example, we set the time range of the detector to -2h, named the detector and the new rule as appropriate, and chose the Sudden Change condition, which compares recent values with earlier values. Based on the trigger sensitivity (the default is Low), the preview chart will show event markers for times when the alert would have been triggered during the past week; you should see an event marker for the time the spike occurred. You might also see markers for other times the signal spiked in the past week, as well as markers indicating the alert condition cleared.

../_images/prev-2hours.png

As discussed above, you have different options for adjusting sensitivity depending on the type of alert condition you are using.

Create a new detector based on recent signal values

In this example, we are setting a condition with a static threshold. When looking at the chart, we chose a threshold value that looked about right. However, the preview indicates that we would have received 11 alerts in the previous day.

../_images/11-per-day.png

When we increase the threshold value, the number of projected alerts is reduced.

../_images/3-per-day.png

Adjust settings until you see a preview that makes sense for your data. For a static threshold condition, you can adjust the threshold value, but in addition you can adjust how quickly the alert is triggered. (For more information, see Static Threshold.) Use these settings in combination to customize the detector for your requirements.

As discussed above, you have different options for adjusting sensitivity depending on the type of alert condition you are using.

Specifying detector options

The Options tab lets you specify some of the same settings that are available in the Chart Options tab of a chart.

Show events as lines

Specifies whether vertical lines are displayed at times where event markers are shown.

Show data markers

Specifies whether small dots are displayed on the chart, indicating the times at which there are datapoints.

Max delay

SignalFx sets the Max Delay parameter based on estimates of how ‘on time’ the time series are. By default, the Auto setting lets SignalFx detect and apply a reasonable value automatically, based on how your data is coming in.

If you know that some of your data is being delayed, and you don’t want to wait for that data to arrive before your charts are updated, then you can set max delay accordingly. To ensure that charts are updated as quickly as possible (i.e. don’t wait for late data at all) select 1s for max delay.

To specify your own values, enter the number in milliseconds; e.g. enter 2000 to specify a max delay of 2 seconds. The upper limit is 15 minutes (900000 ms) but values over 5 minutes are generally not recommended. If you enter a value higher than 15 minutes, max delay will be set to 15 minutes. For more information, see Delayed datapoints.

Minimum resolution

Specifies the minimum interval for which SignalFx should roll up values to display a datapoint on a chart. For example, if you are tracking the number of support calls received per hour, you might not want to see a chart that shows datapoints representing the number of calls received every 15 minutes, even if data is available at that resolution. Setting this option to 1h ensures that the datapoints will always represent values for periods of 1h or more.

The detector is still triggered as configured, regardless of the resolution of the chart’s display.

Disable chart display sampling

In cases where a large number of time series would be displayed, e.g. if you choose a metric being reported by 500 servers, SignalFx samples a subset of those time series so the chart will render more quickly. The sampled display provides you with an approximate sense of the values in those time series. If you disable sampling, any time series data that were previously omitted will be shown. Depending on the number of time series, disabling sampling may cause the chart to render more slowly.

The detector is still triggered as configured, regardless of the number of time series displayed on the chart.

If you created this detector from a chart, anyone viewing the chart will see the new detector in the Related Detectors list for that chart.

Tip

If you want to add the events from the detector to the chart you are viewing in the Chart Builder, choose the new detector as the signal on a new plot line, then save the chart. For more information, see Seeing events on a chart.

Managing detector rules

In the Alert Rules tab of a detector, you can use the Actions menu for a rule (at far right, next to the Edit option) to perform any of the following actions.

  • Disable/enable

    If a detector has multiple rules, such as different rules for different severity levels, you may want to specify which ones to enable or disable. Disabling a rule prevents it from generating any events or sending any notifications. This option is commonly used after the detector has been activated for a while, to decrease or increase the number of alerts the detector is triggering.

  • Clone

    As with plot lines on charts, you can clone rules. This option is commonly used to create rules with slightly different settings from each other, such as specifying a different value for the Alert condition property and changing the severity level of an alert.

  • Delete

    Use this option to remove a rule from the detector.

Viewing detectors

Viewing a list of all detectors

You can see a list of existing detectors in the Detectors tab on the Alerts page. If a detector is currently muted or scheduled to be muted, a red or grey indicator (respectively) is displayed next to the detector. For more information, see Viewing muting information.

Viewing an individual detector

When you open a detector, the Alert Rules tab is opened, and you see a chart that represents values for the visible signals. The list of detector rules, and the number of currently active alerts for each rule, is shown. For information on creating rules, see Building detector rules.

../_images/detector-rules-tab-legend.png

As with charts, the resolution of data displayed is determined by the chart’s time range. The detailed view at right displays data at the detector’s resolution, i.e. the frequency at which the detector evaluates the signal. Any events that have occurred during the detector’s time range are shown below the X axis.

Note

If a detector contains a Signalflow tab, as shown in the illustration below, you are viewing a detector created programmatically using the SignalFx v2 API. If you are familiar with the API, you can use this screen to view and edit the detector code and make changes to the detector rules. For more information, see SignalFx v2 API.

../_images/v2-detector-signalflow.png