SignalFx Developers Guide

The SignalFx Data Model

The SignalFx data model defines the way in which SignalFx collects monitoring data.

Metric

A metric is a measurable numeric system characteristic that changes over time. When you set up SignalFx to monitor a system, you tell it to collect metrics.

For example, the metric cpu.utilization is usually the level of cpu usage in a system, expressed as a percentage.

Metric type

A metric’s type controls how SignalFx displays the metric time in charts. For example, the web UI displays the gauge metric type in summary form using averages.

SignalFx has three metric types:

Gauge

Measures and report instantaneous data at a particular moment in time. Some examples are CPU utilization, memory usage, and request processing duration.

Counter

Increments every time something happens. Some examples are number of requests handled, number of emails sent, and number of errors encountered.

Cumulative Counter

Captures data that has the following pattern:

  • The absolute, instantaneous value is irrelevant

  • The value can roll over at any time, usually to resetting to 0, when the system or application restarts or when it reaches the maximum value representable (232 or 264)

  • The important information is how much the value changed between measurements.

    One use of cumulative counters in measuring SNMP-like data. You can capture the data in raw form and send it to SignalFx without having to do any computation on your end to make it usable.

Dimensions

Dimensions are key-value pairs that specify aspects of a datapoint such as its source host name or host location. Use dimensions as metadata that you search for when you analyze data.

Datapoints

Datapoints are measurements of a metric at a specific point in time that contain the following information:

  • Metric type

  • Metric name

  • Metric value

  • Dimensions

  • Timestamp

Metric time series

A metric time series (MTS) is a set of datapoints that all have the same metric and dimensions. The most common dimension is the source of the metric, such the host name or application. The output of a SignalFlow program is also an MTS.

Custom properties and tags

SignalFx provides two optional types of metadata for a dimension:

  • Custom properties (Often called properties in the SignalFx web UI): Searchable key-value pairs

  • Tags: Searchable text

Custom Properties

Custom properties are key-value pairs that you add to dimensions or tags. For example, add the custom property "use": "QA" to the dimension "host": "host1" to indicate that the server is used for QA.

You can also set custom properties for charts and detectors and use them the same as with dimensions.

Tags

Tags are labels or keywords you assign to dimensions. Tags are strings rather than key-value pairs. Use tags when you want to give the same searchable value to multiple dimensions. For example, suppose you have hosts that are running multiple apps. You can create a tag for each app, then apply one or more "app" tags to each host to specify the apps that are running on that host.

You can also apply custom properties to tags. When you do this, anything that has that tag inherits the properties associated with the tag. For example, if you associate the "tier:web" custom property with the "apps-team" tag, SignalFx attaches the "tier:web" custom property to any metric or dimension that has the "apps-team" tag.

You can also assign tags to charts and detectors and use them the same as with dimensions.

Dimensions, custom properties, and tags have some meaningful differences, which are summarized in the following table:

Table 1. Dimensions, properties, and tags
Dimensions Custom properties Tags

Have to know when sending a datapoint

Only available for dimensions and tags

Only available for dimensions

Defined at the time of sending the datapoint

Defined by an API request

Defined by an API request

Metric + dimensions uniquely define a MTS

Additional classification of a dimension or tag

Additional classification of a dimension

Applied to metrics

Applied to dimensions and tags

Applied to dimensions

Key-value pair

Key-value pair

Keyword

Can be used to query or group by

Can be used to query or group by

Can be used to query

To learn more about metrics metadata, see the topic Working with Metrics Metadata.

Dimensions, custom properties and tags criteria

General

The optional description property lets you provide a detailed description of a dimension or tag. You can use up to 1024 UTF-8 characters for description.

Dimensions

  • You can have up to 36 dimensions per MTS

  • UTF-8 string, maximum length of 128 characters (512 bytes)

  • Must start with an uppercase or lowercase letter. The rest of the name can contain letters, numbers, underscores (_) and hyphens (-).

  • Must not start with the underscore character (_)

  • Must not start with the prefix sf_, except for dimensions defined by SignalFx such as sf_hires

  • Must not start with the prefix aws_, gcp_, or azure_

Custom properties

  • You can have up to 75 custom properties per dimension.

  • Custom property names are UTF-8 strings with a maximum length of 128 characters (512 bytes).

  • Custom property values are UTF-8 strings with a maximum length of 256 characters (1024 bytes).

  • Numbers are represented as numeric strings.

Tags

  • You can have up to 50 tags per dimension.

  • You can have up to 50 tags per custom property.

  • Tags are UTF-8 strings with a maximum length of 256 UTF-8 characters/1024 bytes.

  • You can only have 50 tags per chart or detector.

Streaming datapoints to SignalFx

You can stream datapoints to SignalFx in either JSON or ProtocolBuffer format. To learn more, see the topic Sending Metrics and Events.

Creating or updating metrics and dimensions

You don’t have to create a metric. SignalFx adds the metric the first time it receives a datapoint containing the metric. SignalFx does the same for dimensions.

Creating or updating custom properties and tags

To update the tags or custom properties for a dimension, use the operation PUT https://api.{REALM}.signalfx.com/v2/<DIMENSION_KEY>/<DIMENSION_VALUE>. This operation has overwrite semantics, so you need to retrieve and store values you want to keep and then append new values to the stored ones.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
$ curl \
    --request PUT \
    --header "X-SF-TOKEN: <YOUR_ACCESS_TOKEN>" \
    --header "Content-Type: application/json" \
    --data \
        '{
            "key" : "<DIMENSION_KEY>",
            "value": "<DIMENSION_VALUE>",
            "tags": ["test-tag"],
            "customProperties": {
                "region" : "japan"
            }
        }' \
    https://api.<REALM>.signalfx.com/v2/dimension/<DIMENSION_KEY>/<DIMENSION_VALUE>

The response body for this request is similar to the following JSON:

{
    "creator" : "<CREATOR_ID>",
    "lastUpdatedBy" : "<UPDATER_ID>",
    "created" : <CREATED_TIME>,
    "lastUpdated" : <UPDATED_TIME>,
    "customProperties" : {
        "region" : "japan"
    },
    "tags" : [ "test-tag" ],
    "key" : "<DIMENSION_KEY>",
    "value" : "<DIMENSION_VALUE>"
}

Searching for metrics, dimensions, or properties

The SignalFx API search syntax is similar to Elasticsearch syntax, which is a combination of key-value pairs with support for wildcards(*), NOT, AND and OR operators, and parentheses.

To limit a term query to a specific dimension or custom property, specify the dimension or property key followed by a colon (:) and the value to search for, such as region:japan. Dimensions and custom properties are treated alike when searching, so queries such as region:japan search both dimensions called "region" with the value "japan" and custom properties that have the key "region" and value "japan".

Example queries

Here are some example queries:

region:japan AND tags:"web-tier" AND NOT tags:"production"

This query selects objects which are

  • In the eu-west-1 region

  • Tagged with "web-tier"

  • Not also tagged with "production"

(region:eu-west-1 AND tags:"web-tier") OR region:us-east-1

This query selects objects which are either in the eu-west-1 region and tagged with "web-tier", or objects that are in the us-east-1 region.

region:*

This query selects any object that has a region property or dimension.

The following is a full example of a query that searches for all dimensions which have a custom property for "region" equal to "eu-west-1".

1
2
3
4
$ curl \
    --request GET \
    --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
    https://api.<REALM>.signalfx.com/v2/dimension?query=region:eu-west-1

The result of this query is the following JSON:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  "count" : 1,
  "results" : [
    {
        "created" : <CREATED_TIME>,
        "creator" : "<CREATOR_ID>",
        "customProperties" : { },
        "description" : null,
        "key" : "region",
        "lastUpdated" : <UPDATED_TIME>,
        "lastUpdatedBy" : "<UPDATER_ID>",
        "tags" : [ ],
        "value" : "eu-west-1"
    }
  ]
}

AND, OR, and NOT Operators

By default, SignalFx uses the OR operator to combine terms when you make queries using the API. This means that, given the query region:japan host:server5, the result set contains objects that either have japan region or server5 host as a dimension or custom property.

To get results that contain all the terms you specify, join your terms with AND. For example, specifying a query of region:japan AND host:server5 returns results that have both the japan region and the server5 host as a dimension or custom property.

Use NOT to exclude results that contain certain terms. For example, searching region:japan AND NOT host:server5 returns results that contain the japan region, but only if they don’t include the server5 host as a dimension or custom property.

Grouping with parentheses

Use parentheses to group terms. For example, the query region:japan AND (host:server5 OR env:production) returns results that contain the japan region and either the server5 host or the production env dimension or custom property.

Wildcard Searches

Wildcards help you search for objects that have similar names or tags. A wildcard indicates to SignalFx that any character matches that part of the search string. The ? character is the single character wildcard and * is a multiple character wildcard. For example, if you search for host:webServer*, SignalFx returns all objects that have "webServer" as the first characters of the dimension or custom property.

Range Queries

Range queries match values that have an intrinsic semantic order, such as timestamps or customer names. SignalFx lets you use both numeric and alphanumeric range queries.

A range query contains a range term delimited by special characters, with the start point and end point of the range separated by the keyword TO.

.

Syntax

Non-exclusive numeric range query

The delimiters are square braces, and the boundary values are integers. <key>:[<low_integer> TO <high_integer>] matches key values between the lower and upper integer value, including the boundary values themselves.

Exclusive numeric range query

The delimiters are curly braces. <key>:{<low_integer> TO <high_integer>}` matches key values between the lower and upper integer value, but doesn’t include the boundary values.

Non-exclusive alphanumeric range query

The delimiters are square braces, and the boundary values are strings. <key>:[<start_string> TO <end_string>] matches key values between the starting and ending string, including the boundary values.

Exclusive alphanumeric range query

The delimiters are curly braces, and the boundary values are strings. <key>:{<start_string> TO <end_string>} matches key values between the starting and ending string, , but doesn’t include the boundary values.

Examples

  • Non-exclusive integer: For example, lastUpdated:[1420099200000 TO 1451635200000] returns results that were last updated in the year 2015 (the numbers are expressed in Unix time in milliseconds).

  • Non-exclusive alphanumeric: For example, customer:[Alice TO Charlie] returns results that contain a customer value between "Alice" and "Charlie" (inclusive).

  • Exclusive integer: For example, lastUpdated:{1420099200000 TO 1420099200002} only searches for the lastUpdated value of 1420099200001.

Property Existence Checks

You can search for results based on a property key using the exists and missing pseudofields. For example, to find results which contain a host_machine property you can search for exists:host_machine.

Filtering and Grouping Time Series

When you create charts or alerts, you can use a combination of metric names, dimensions, custom properties, and tags to find the metric time series you want.

Besides filtering using these metadata values, you can use shared dimensions or custom properties to group metric time series together.

The following example shows you how to combine metadata in a query.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
#Search for the dimension or custom property `region:eu-west-1`, but
#exclude results that contain `host:signalfx-agent` or `host:cfx-agent`
#The query parameter must be URL encoded.

#Execute the request to find metric time series that match the query

$ curl \
    --request GET
    --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
    https://api.<REALM>.signalfx.com/v2/metrictimeseries?query=region:eu-west-1+AND+NOT+%28host:signalfx-agent+OR+host:cfx-agent%29

When you use the API to send datapoints to SignalFx, a single request body can contain datapoints for different time series and different metric types.

If you send a datapoint for a metric that didn’t previously exist, SignalFx adds the new metric using the metric type you specify. You can change the metric type later on using an API call or the web UI.

© Copyright 2019 SignalFx.

Third-party license information