SignalFx

Metric Data Overview

The /datapoint API endpoint is used to send metric data (datapoints) to SignalFx. It is a versatile endpoint that supports multiple content types, as well as streaming and non-streaming requests.

Streaming datapoints to SignalFx

JSON

The simplest way to send a single JSON-encoded datapoint is by making a simple POST request to that endpoint. The query string must contain the ID of the organization you are sending the datapoint into, and the body must be a JSON document describing each measurement for each metric, by type. The metric name, dimension(s) and value(s) must be included with each measurement.

To specify the timestamp of the measurement, add a timestamp field to each JSON object being sent, in milliseconds since the Unix Epoch of 00:00:00 UTC on 1 January 1970.

IMPORTANT

Note that the timestamp must be in milliseconds, not in seconds.

Note that the /datapoint API endpoint requires that timestamps be monotonically increasing per time series; datapoints with timestamps earlier than previously sent for the same time series are rejected silently. If you don't specify a timestamp, the time on SignalFx's server clock when the datapoint is received will be used as the timestamp.

NOTE

If you want to send data that is earlier than data that has already been sent, you can use the /backfill API.

PROTOBUF

The /datapoint endpoint also supports a more bandwidth-efficient way of sending datapoints to SignalFx using a ProtocolBuffer encoding. The signalfx-protoc package contains the ProtocolBuffer definition of the DataPoint message that is expected by the API.

To send data encoded as ProtocolBuffer messages, make sure your POST request declares the application/x-protobuf content type.

METRICS, DIMENSIONS, AND METRIC TIME SERIES

When sending datapoints, you must specify a metric, to which you can add any number of dimensions (zero or more). A metric plus its dimension(s) comprise the unique identity of the metric time series for that particular series of measurements.

METRIC TYPES

The available metric types are gauge, counter and cumulative_counter. You can send data for different time series of different metric types in the same API call.

If you send a datapoint to a non-existent metric, the new metric will be automatically created using the metric type under which it was reported. It is possible to change the metric type after the metric has been created, using an API call or in the SignalFx web UI.

Metric types are used to determine the default rollup used when a metric is displayed in the SignalFx web UI. For example, gauges are summarized using averages. Note, however, that SignalFx stores the same set of multiple rollups for all metrics, irrespective of the declared metric type.

SignalFx makes extensive use of metadata to let users find, filter and aggregate metric time series and other objects. There are three types of metadata: dimensions, custom properties (often just called properties in the SignalFx web UI) and tags.

Naming and value criteria for metadata

For criteria and limitations related to specifying metadata, see the information in /datapoint.

Dimensions

Dimensions are key-value pairs included as part of a datapoint. Each combination of dimensions and metric names sent to SignalFx is used to uniquely identify a metric time series and the datapoints that comprise it.

Custom Properties

Custom properties are key-value pairs that are added to dimensions or metric names that already exist in your SignalFx org. As such, they are most often used to associate metrics with metadata that is useful for classification purposes; that is not known or convenient to send at the time of data submission; or when tracking history is not important.

Tags

Tags can be thought of as labels or keywords assigned to dimensions, metrics and other objects. They are not key-value pairs. The primary use case for tags is when there is a one-to-many relationship between the tag and the object to which it is assigned. For example, suppose you have hosts that are running multiple apps. You can create a tag for each app, then apply multiple tags to each host to specify the apps that are running on that host.

Custom properties can also be applied to tags. Doing so will cause any object having that tag to inherit the properties associated with the tag. For example, if the "tier:web" custom property is associated with the "apps-team" tag, any metric or dimension that is tagged with "apps-team" will also inherit the "tier:web" custom property.

Dimensions, custom properties, and tags can all be applied to datapoints, but there are some meaningful differences between them. These differences are summarized below.

For more information on SignalFx metric metadata, see the SignalFx web UI documentation on metrics metadata.

Creating or Updating Metrics and Dimensions

You do not have to explicitly create metric or dimension objects; if they don't already exist, they are created automatically by SignalFx when received as a part of a datapoint request. Both the metric and dimension endpoints have "create or update" semantics, so if a PUT request is received for an object which doesn't exist, it will be created, otherwise it will be updated. Here is an example request to create (or update) a dimension:

$ curl \
  --request PUT \
  --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{ "key" : "host", "value": "lb-tokyo", "tags": ["test-tag"], "customProperties": { "region" : "japan" } }' \
  https://api.signalfx.com/v2/dimension/host/lb-tokyo
 
{
  "creator" : "ByZgBHNAYAA",
  "lastUpdatedBy" : "ByZgBHNAYAA",
  "created" : 1450134495374,
  "lastUpdated" : 1450134531331,
  "customProperties" : {
    "region" : "japan"
  },
  "tags" : [ "test-tag" ],
  "key" : "host",
  "value" : "lb-tokyo"
}

Searching for Metrics, Dimensions, or Properties

Our API search syntax is similar to Elasticsearch syntax, which is a combination of key:values with support for * (wildcard), NOT, AND and OR operators, along with parentheses.

You can limit a term query to a specific dimension or custom property key by specifying the field followed by a colon (:) then the term to search for (such as region:japan). Dimensions and custom properties are treated alike when searching, so queries such as region:japan would search both dimensions called "region" with the value "japan" along with custom properties which have the key "region" and value "japan".

NOTE

Custom properties do not have to be prefixed with "customProperties". For example, if you have a custom property of region:japan, you can use it as is when searching for dimensions or metrics containing the property.

Here are some example queries:

region:japan AND tags:"web-tier" AND NOT tags:"production"
This query would select objects which are in the japan region, are tagged with "web-tier", and are not also tagged with "production".

(region:japan AND tags:"web-tier") OR region:emea
This query would select objects which are either in the japan region and are tagged with "web-tier, or objects that are in the emea region.

region:*
This query would select any object which has a region property or dimension.

Here's a full example of running a query to search for all dimensions which have a custom property for "region" equal to "japan":

$ curl --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
https://api.signalfx.com/v2/dimension?query=region:japan

{
  "results" : [ {
    "creator" : "AAAAAAAAAAA",
    "lastUpdatedBy" : "ByZgBHNAYAA",
    "created" : 1438217683080,
    "lastUpdated" : 1449856007264,
    "customProperties" : {
      "region" : "japan"
    },
    "tags" : [ "test-tag" ],
    "key" : "host",
    "value" : "lb-tokyo"
  } ],
  "count" : 1
}

AND / OR / NOT Operators

By default, terms are combined using the OR operator when making queries via the API. This means that given the query region:japan host:server5, the result set will contain objects which either have japan region and/or server5 host as a dimension or custom property.

AND is used to require two terms to be present in each result. For example, searching region:japan AND host:server5 would return results which have both the japan region and the server5 host as a dimension or custom property.

NOT is used to exclude results which contain certain terms. For example, searching region:japan AND NOT host:server5 would return results which contain the japan region so long as they don't include the server5 host as a dimension or custom property.

Grouping

Parenthesis can be used to group clauses. For example, the querying for region:japan AND (host:server5 OR env:production) would return results which contain the japan region and either the server5 host or the production env dimension or custom property.

Wildcard Searches

? can be used as a single character wildcard and * can be used as a multiple character wildcard in single terms. For example, host:webServer* will return results which have the "webServer" prefix for values of the host dimension or custom property.

Range Queries

Numeric range terms can be specified using brackets along with the TO operator. For example, lastUpdated:[1420099200000 TO 1451635200000] would return results which were last updated in the year 2015 (the numbers being milliseconds after Unix epoch).

Alphabetic range terms can be specified using brackets along with the TO operator. For example, customer:[Alice TO Charlie] will return any results which contain a customer value between Alice and Charlie (inclusive).

Exclusive range queries can be made by using curly braces instead of square brackets. For example, lastUpdated:{1420099200000 TO 1420099200002} would only search for the lastUpdated value of 1420099200001.

Property Existence Checks

You can search for results containing a given property key (or not containing) by using the _exists_ and _missing_ pseudofields. For example, to find results which contain a host_machine property you can search for _exists_:host_machine.

Filtering and Grouping Time Series

When charting or creating alerts for your data, you will be able to use a combination of metric names, dimensions, custom properties, and tags to find matching time series.

You can filter your dataset by using dimensions, custom properties, or tags. You can also use shared dimensions or custom properties to group time series data together. You can also use dimensions, custom properties, and tags to search for metric time series. Here's an example using cURL:

# The query parameter we'll be searching for is 
# "region:japan AND NOT (host:server1 OR host:server2)"
# which must be URLencoded

QUERY="region:japan+AND+NOT+%28host:server1+OR+host:server2%29"

# Execute the request to find metric time series that match the query
curl --header "X-SF-TOKEN: YOUR_ACCESS_TOKEN" \
https://api.signalfx.com/v2/metrictimeseries?query=$QUERY

The above query will find any time series which has the region:japan custom property or dimension, with the exception of time series which have "server1" and "server2" as the values for a "host" custom property or dimension. Note that when searching for time series, the query doesn't distinguish between dimensions or custom properties.

Metric Data Overview