SignalFx Developers Guide

detect()

Creates a detector object.

Detector objects create events when their "on" or "off" filter matches the input stream:

  • "on": On a match, the detector issues an event with status anomalous

  • "off": On a match, the detector issues an event with status ok

When you write out these events with publish(), SignalFx can send event notifications to users.

To learn more about detect(), see the examples in the topic Detectors, Events, and Alerts.

Syntax

detect(<on_predicate>, off=[<off_predicate>, mode=[<evaluation_mode]")

Table 1. Parameter definitions
Parameter Type Description

on_predicate

string

Required: when() expression that fires an event containing the status "anomalous"

<off_predicate>

string

Optional, default=None: when() expression that fires an event with status "ok"

mode

string

Optional: Detector mode. Expected values are:

  • "paired": The detector evaluates the "on" and "off" conditions simultaneously:
    • If "on" is true and "off" is false, the detector raises an alert and issues an event with status "anomalous".
    • If "on" is false and "off" is true, the detector clears the alert and issues an event with status "ok".
  • "split": The detector evaluates "on" and "off" separately:
    • Detector evaluates the "on" condition only if there's no existing alert. If "on" is true, the detector raises an alert and sends an event with status "anomalous".
    • Detector evaluates the "off" condition only if an alert exists. If "off" is true, the detector clears the alert.

Returns a reference to a detect stream.

Examples

detect(on)

1
2
3
# send an "anomalous" event when cpu.utilization is greater than 50 for 5 minutes
# will send an "ok" event when cpu.utilization goes below 50 for 5 minutes
detect(when(data('cpu.utilization') > 50, '5m')).publish('cpu_too_high')

detect(on,off=<off_predicate>)

1
2
3
4
# send an "anomalous" event when cpu.utilization is greater than 50 for 5 minutes
# only send an "ok" event when cpu.utilization goes below 40 for 10 minutes
cpu = data('cpu.utilization')
detect(when(cpu > 50, '5m'), off=when(cpu < 40, '10m')).publish('cpu_too_high')

Detect with multiple inputs

1
2
3
4
5
# Combine multiple inputs and output with a label
cpu = data('cpu.utilization').mean(over='5m')
threads = data('server.threadCount').mean(over='10m')
# Check to see if the CPU usage is high with a low thread count for over 30s
detect(when(cpu > 50 and threads < 10, '30s')).publish('high_cpu_with_low_threads')

Detecting anomalies for multiple static thresholds

The previous examples apply a single static threshold to all the timeseries in an input stream. Some other situations may require a different metric threshold for each service or another way of determining anomalies.

You can address these situations with the const() function. For example:

  1. Create one or more constant-value timeseries that include custom dimensions that represent some threshold you want. For example, specify a variable threshold that defines two constant timeseries:

    • metric 50, dimension "service": "web"

    • metric 85, dimension "service": "workqueue"

1
2
3
4
threshold = const(timeseries=[
     {"key": {"service": "web"}, "value": 50},
     {"key": {"service": "workqueue"}, "value": 85}
     ])

You now have two timeseries (not just single-value constants) that represent a detection limit.

  1. Get the timeseries you want to monitor for anomalies, and add "service" to it as a dimension: cpu = data('cpu.utilization').promote('service')

  2. Compare the cpu timeseries` to the threshold timeseries in your detect() call: detect(when(cpu > threshold, '1m')).publish('high cpu usage')

© Copyright 2019 SignalFx.

Third-party license information