SignalFx Developers Guide

The SignalFx Data Model

The SignalFx data model is the basis of the API for sending and receiving measurements to your SignalFx instance.

The data model defines incoming data as metric time series (MTS), which are numbers and optional properties (dimensions) measured over time. In detail:

  • A metric is the name of a numeric measure that changes over time. For example, the metric nrequests is the name for the number of network requests a server makes in a period of time.

  • Each metric has a value for a time period. For example, the nrequests metric may have the value 3010 over the last millisecond.

  • A dimension is a name-value pair that specifies aspects of the metric that are either non-numeric or unchanging, or both. Dimensions help you save and then filter on metadata for a metric. Some examples of dimensions:

    • serverName (non-numeric)

    • region (non-numeric. If it applies to worldwide regions, doesn’t change)

    • nCores (numeric, but doesn’t change for a specific system)

Datapoints

A datapoint is the value of a metric at a specific point in time. Specifically, a datapoint has a metric name and metric value, and optionally one or more dimension names and values.

Metric time series

A metric time series contains all of the datapoints that SignalFx receives that have the same metric and set of dimensions. For example:

  • nrequests and dimensions serverName and region define an MTS. When SignalFx first receives a data value with this metric and set of dimensions, it starts tracking the MTS. As new data values with the same metric and set of dimensions arrive, they become part of the same MTS. Notice that the value of the dimension isn’t considered.

  • nrequests and dimensions serverName, region, and nCores define another MTS. As new data values with this metric and set of dimensions arrive, they become part of a new MTS.

Metric types

Metric types determine the default rollup that SignalFx uses when it displays a metric time series in the web UI. For example, the web UI displays the gauge metric type in summary form using averages.

Regardless of metric type, SignalFx stores the same set of rollups for all metrics.

SignalFx defines three metric types:

Gauge

Measure and report instantaneous data at a particular moment in time. Some examples are temperature, CPU utilization, memory usage, time spent processing a request, and so forth.

Counter

Incremented every time something happens. Counters measure the number of requests handled, emails sent, errors encountered, and so forth.

Cumulative Counter

Captures data that has the following pattern:

  • The absolute, instantaneous value is irrelevant

  • The value can roll over at any time, usually to resetting to 0, either when the machine or application restarts or when it reaches the maximum value representable (2^32 or 2^64)

  • The important data is how much the value changed between measurements.

One use of cumulative counters in measuring SNMP-like data. You can capture the data in raw form and send it to SignalFx without having to do any computation on your end to make it usable.

Custom properties and tags

Besides dimensions, SignalFx defines two other types of metadata for a datapoint:

  • Custom properties (Often called properties in the SignalFx web UI): Searchable key-value pairs

  • Tags: Searchable text

Custom Properties

Custom properties are key-value pairs that you add to dimensions or metric names that already exist in your SignalFx org. You mostly use them to associate metrics with the following types of metadata:

  • Classification metadata

  • Metadata you don’t know or can’t easily retrieve when you’re submitting data

  • Metadata where you don’t care about the tracking history

Tags

Tags are labels or keywords you assign to dimensions, metrics or other objects. They aren’t key-value pairs; instead, the tag itself is a single string attached to the object. You mostly use tags when you want to label multiple objects with the same string. For example, suppose you have hosts that are running multiple apps. You can create a tag for each app, then apply one or more "app" tags to each host to specify the apps that are running on that host.

You can also apply custom properties to tags. When you do this, any object that has that tag inherits the properties associated with the tag. For example, if you associate the "tier:web" custom property with the "apps-team" tag, SignalFx attaches the "tier:web" custom property to any metric or dimension that has the "apps-team" tag.

Datapoints can have dimensions, custom properties, and tags, but these types of metadata have some meaningful differences.

These differences are summarized in the following image

Differences between dimensions, properties, and tags
Differences between dimensions, properties, and tags

For more information on SignalFx metric metadata, see topic Metrics Metadata in the product documentation.

Streaming datapoints to SignalFx

You can stream datapoints to SignalFx in either JSON or ProtocolBuffer format. To learn more, see the topic Sending Data.

Creating or updating metrics and dimensions

You do not have to explicitly create metric or dimension objects. If they don’t already exist, SignalFx creates them when it receives them in a datapoint or dimension you send. Both the metrics and dimensions endpoints have "create or update" semantics; if SignalFx receives a PUT request for an object which doesn’t exist, it creates the object, otherwise it updates it.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
$ curl \
    --request PUT \
    --header "X-SF-TOKEN: <YOUR_ACCESS_TOKEN>" \
    --header "Content-Type: application/json" \
    --data \
        '{ \
            "key" : "<DIMENSION_KEY>", \
            "value": "<DIMENSION_VALUE>", \
            "tags": ["test-tag"], \
            "customProperties": { \
                "region" : "japan" \
            } \
        }' \
    https://api.<REALM>.signalfx.com/v2/dimension/<DIMENSION_KEY>/<DIMENSION_VALUE>

The response body for this request contains the following JSON:

{
    "creator" : "ByZgBHNAYAA",
    "lastUpdatedBy" : "ByZgBHNAYAA",
    "created" : 1450134495374,
    "lastUpdated" : 1450134531331,
    "customProperties" : \{
        "region" : "japan"
    \},
    "tags" : [ "test-tag" ],
    "key" : "<DIMENSION_KEY>",
    "value" : "<DIMENSION_VALUE>"
}

Searching for metrics, dimensions, or properties

The SignalFx API search syntax is similar to Elasticsearch syntax, which is a combination of key-value pairs with support for wildcards(*), NOT, AND and OR operators, and parentheses.

To limit a term query to a specific dimension or custom property, specify the dimension or property key followed by a colon (:) and the value to search for, such as region:japan. Dimensions and custom properties are treated alike when searching, so queries such as region:japan search both dimensions called "region" with the value "japan" and custom properties that have the key "region" and value "japan".

Example queries

Here are some example queries:

region:japan AND tags:"web-tier" AND NOT tags:"production"

This query selects objects which are

  • In the japan region

  • Tagged with "web-tier"

  • Not also tagged with "production"

(region:japan AND tags:"web-tier") OR region:emea

This query selects objects which are either in the japan region and tagged with "web-tier", or objects that are in the emea region.

region:*

This query selects any object that has a region property or dimension.

The following is a full example of a query that searches for all dimensions which have a custom property for "region" equal to "japan":

1
2
3
$ curl \
    --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
    https://api.<REALM>.signalfx.com/v2/dimension?query=region:japan

The result of this query is the following JSON:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
{
    "results" : [
        {
            "creator" : "AAAAAAAAAAA",
            "lastUpdatedBy" : "ByZgBHNAYAA",
            "created" : 1438217683080,
            "lastUpdated" : 1449856007264,
            "customProperties" : {
                "region" : "japan"
            },
            "tags" : [ "test-tag" ],
            "key" : "host",
            "value" : "lb-tokyo"
        }
    ],
    "count" : 1
}

AND, OR, and NOT Operators

By default, SignalFx uses the OR operator to combine terms when you make queries using the API. This means that, given the query region:japan host:server5, the result set contains objects that either have japan region or server5 host as a dimension or custom property.

To get results that contain all the terms you specify, join your terms with AND. For example, specifying a query of region:japan AND host:server5 returns results that have both the japan region and the server5 host as a dimension or custom property.

Use NOT to exclude results that contain certain terms. For example, searching region:japan AND NOT host:server5 returns results that contain the japan region, but only if they don’t include the server5 host as a dimension or custom property.

Grouping with parentheses

Use parentheses to group terms. For example, the query region:japan AND (host:server5 OR env:production) returns results that contain the japan region and either the server5 host or the production env dimension or custom property.

Wildcard Searches

Wildcards help you search for objects that have similar names or tags. A wildcard indicates to SignalFx that any character matches that part of the search string. The ? character is the single character wildcard and is a multiple character wildcard. For example, if you search for host:webServer, SignalFx returns all objects that have "webServer" as the first characters of the dimension or custom property.

Range Queries

Range queries match values that have an intrinsic semantic order, such as timestamps or customer names. SignalFx lets you use both numeric and alphanumeric range queries.

A range query contains a range term delimited by special characters, with the start point and end point of the range separated by the keyword TO.

.

Syntax

Non-exclusive numeric range query

The delimiters are square braces, and the boundary values are integers. <key>:[<low_integer> TO <high_integer>] matches key values between the lower and upper integer value, including the boundary values themselves.

Exclusive numeric range query

The delimiters are curly braces. <key>:{<low_integer> TO <high_integer>}` matches key values between the lower and upper integer value, but doesn’t include the boundary values.

Non-exclusive alphanumeric range query

The delimiters are square braces, and the boundary values are strings. <key>:[<start_string> TO <end_string>] matches key values between the starting and ending string, including the boundary values.

Exclusive alphanumeric range query

The delimiters are curly braces, and the boundary values are strings. <key>:{<start_string> TO <end_string>} matches key values between the starting and ending string, , but doesn’t include the boundary values.

Examples

  • Non-exclusive integer: For example, lastUpdated:[1420099200000 TO 1451635200000] returns results that were last updated in the year 2015 (the numbers are expressed in Unix time in milliseconds).

  • Non-exclusive alphanumeric: For example, customer:[Alice TO Charlie] returns results that contain a customer value between "Alice" and "Charlie" (inclusive).

  • Exclusive integer: For example, lastUpdated:{1420099200000 TO 1420099200002} only searches for the lastUpdated value of 1420099200001.

Property Existence Checks

You can search for results based on a property key using the exists and missing pseudofields. For example, to find results which contain a host_machine property you can search for exists:host_machine.

Filtering and Grouping Time Series

When you create charts or alerts, you can use a combination of metric names, dimensions, custom properties, and tags to find the metric time series you want.

Besides filtering using these metadata values, you can use shared dimensions or custom properties to group metric time series together.

For example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# Search for the dimension or custom property `region:japan`, but
# exclude results that contain `host:server1` or `host:server2`
# The query parameter must be URL encoded.

QUERY=""

# Execute the request to find metric time series that match the query

$ curl \
    --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
    https://api.<REALM>.signalfx.com/v2/metrictimeseries?query=region:japan+AND+NOT+%28host:server1+OR+host:server2%29

When you use the API to send datapoints to SignalFx, a single request body can contain datapoints for different time series and different metric types.

If you send a datapoint for a metric that didn’t previously exist, SignalFx adds the new metric using the metric type you specify. You can change the metric type later on using an API call or the web UI.

© Copyright 2019 SignalFx.

Third-party license information