SignalFx Developers Guide

SignalFlow Information Messages Reference

SignalFlow computations send you messages that provide additional details about the operation, including important warnings and information.

SignalFlow client libraries

SignalFx provides open-source, language-specific client libraries that encapsulate the following tasks:

  • Choosing the best connection method (HTTP or WebSocket) to use

  • Parsing and decoding the stream of the messages coming from your SignalFlow computations

  • High-level abstractions that simplify and speed up the process of building SignalFlow analytics applications

In most cases, you should use a client library rather than writing your own low-level connection, message processing, and request routines. The information in this topic is provide for developers who want to handle SignalFlow requests and responses at a low level.

You can receive messages using either the SSE (Server-Sent Event) or WebSocket protocol. The format of the message you receive depends on the protocol you use.

The following are links to the client libraries in GitHub:

Receiving messages via Server-Sent Events

If you send a computation to SignalFlow using the HTTP REST API, SignalFlow sends you response messages using the Server-Sent Event (SSE) protocol. The content-type is text/event-stream. Each SSE message has the name event, which is also the message type, and a data payload formatted in JSON. In accordance with the SSE standard, SignalFlow puts an empty new line between each event.

Receiving messages via WebSocket

If you send a computation to SignalFlow using a WebSocket connection to SignalFx, SignalFlow sends you response messages in either JSON text or binary format. The format SignalFlow uses depends on these factors:

  • Message type: The messageType SignalFlow assigns to the message. Data messages are binary-encoded, but all other messages use JSON.

  • Message compression settings: Whether you requested message compression when you connected to a channel using WebSocket.

SignalFlow identifies all WebSocket response messages with the channel value you specify when you make an execute request. SignalFlow also identifies the type of message in one of the top-level message properties. In all other respects, WebSocket response messages formatted in JSON have the same structure as SSE payloads.

Message compression

When you request message compression, SignalFlow attempts to compress response messages for the channel and client that sent the request. If the compressed message is smaller, SignalFlow sends it as a compressed binary message. The base format of this message is JSON except for data messages, which use the data message format.

Message type and structure

Information messages have the type message. They always include a timestamp in Unix time format, set to the time at which the computation produced them. Messages also contain a message property that contains the actual information from SignalFlow.

The message itself is under the message field (to prevent conflict with the envelope).

Messages always contain the properties described in the following table:

Table 1. Main message properties
Message property Description

message

property of the data unit envelope that contains the message

messageCode

Short text indicating the reason for the message

messageLevel  

message severity

logicalTimestampMs

Timestamp of the computation at the point that SignalFlow sent the message

timestampMs

Timestamp of the condition that led SignalFlow to send the message

Messages may contain the properties described in the following table:

Table 2. Optional message properties
Message property Description

numInputTimeSeries

defaults to 0

tsIds

IDs of the timeseries to which the message relates

contents

additional message payload. The format and contents depend on the value of messageCode.

Logical timestamp and message timestamp

The logical timestamp of a message is the logical time of the computation when the message was emitted. This may be different from the message’s own timestamp, which represents the timestamp of whatever the message talks about. For example, if your program uses transformations or timeshifts, the data’s timestamp may be offset from the computation’s logical timestamp.

SSE message example

The following JSON is a representation of a message received via the SSE protocol:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
event: message
data: {
data:   "logicalTimestampMs" : 1464736034000,
data:   "message" : {
data:     "blockSerialNumber" : 0,
data:     "contents" : {
data:       "resolutionMs" : 1000
data:     },
data:     "messageCode" : "JOB_RUNNING_RESOLUTION",
data:     "messageLevel" : "INFO",
data:     "numInputTimeSeries" : 0,
data:     "timestampMs" : 1464736033000
data:   }
data: }

WebSocket message example

The following JSON is a representation of the previously-mentioned message, but received via the WebSocket protocol:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
  "type" : "message",
  "channel" : "channel-1",
  "logicalTimestampMs" : 1464736034000,
  "message" : {
    "blockSerialNumber" : 0,
    "contents" : {
      "resolutionMs" : 1000
    },
    "messageCode" : "JOB_RUNNING_RESOLUTION",
    "messageLevel" : "INFO",
    "numInputTimeSeries" : 0,
    "timestampMs" : 1464736033000
  }
}

Message code reference

The following table lists each possible messageCode value, its corresponding contents values, and an explanation of the values.

Table 3. Message codes
messageCode content value Explanation

JOB_RUNNING_RESOLUTION

resolutionMs: the compute resolution, in milliseconds.

Tells you which compute resolution SignalFlow is using. You should expect to receive data for each logical timestamp in the overall computation, at intervals equal to and aligned on the resolution.

JOB_DETECTED_LAG

lagMs: the detected lag, in milliseconds

Contains the initial MaxDelay value, based on the detected lag of the input data.

FIND_MATCHED_NO_TIMESERIES

query: text of the timeseries query

Sent if the selected metric in a data(), graphite() or newrelic() function (eventually with some filters did not match any known active timeseries.

FIND_LIMITED_RESULT_SET

  • matchedSize: the number of matched timeseries

  • limitSize: the maximum number of timeseries allowed in this pipeline

Sent if the selected metric (with filters) matched too many timeseries and SignalFlow limited the matched set.
Note: When SignalFlow limits results, it uses the timeseries with the oldest creation times, so that the result set is consistent even if the computation runs again.

FIND_MATCHED_DIMENSIONS

dimensionCounts: an array of objects containing the following:

  • dimensions: a list of dimension names

  • count: the number of timeseries with exactly those dimensions

Contains each unique dimension combination set found in the input timeseries, and the number of timeseries for each set. For example, suppose your query for the metric cpu.utilization matches 3 timeseries, each with the following dimensions: host:h1, datacenter:us, host:h2, datacenter:us and host:h3, datacenter:eu, country:fr. The message contains two dimension sets: sf_metric, host, datacenter with a count of 2 and sf_metric, host, datacenter, country with a count of 1. The following JSON is a representation of the body of an SSE message:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
{
  "blockSerialNumber" : 0,
  "contents" : {
    "dimensionCounts" : [ {
      "count" : 2,
      "dimensions" : [ "sf_metric", "host", "datacenter" ]
    }, {
      "count" : 1,
      "dimensions" : [ "sf_metric", "host", "datacenter", "country" ]
    } ]
  },
  "messageCode" : "FIND_MATCHED_DIMENSIONS",
  "messageLevel" : "INFO",
  "numInputTimeSeries" : 3,
  "timestampMs" : 1464814200000
}

FETCH_NUM_TIMESERIES

(none)

Sent for each input data stream from the data(), graphite() or newrelic() functions. This message provides information regarding input timeseries used by the computation. Although the message doesn’t have a payload, it adds the numInputTimeSeries and the tsIds properties to the message.

WINDOW_RESIZED

  • requestedWidthMs: the requested window size, in milliseconds

  • actualWidthMs: the new window size, as adjusted, in milliseconds

Sent if the resolution of the computation is coarser (longer in duration) than the size of a transformation window. In this case, SignalFlow re-sizes the transformation window to match the compute resolution. For example, suppose you calculate mean(over=5s) (a transformation over 5 seconds). If the reporting frequency of the data is 10 seconds, or you ask for a minimum resolution of 10 seconds, SignalFlow automatically re-sizes the transformation window to 10 seconds.

GROUPBY_MISSING_PROPERTY

propertyNames: the names of the dimensions or metadata properties that are missing from some of the input timeseries

Sent if one or more input timeseries don’t contain a dimension or metadata property that an aggregation grouping usesgroupby aggregation. For example, suppose you calculate mean(by='datacenter'), but one or more of the input timeseries don’t contain a datacenter dimension or property. SignalFlow doesn’t consider these timeseries in the aggregation, and it sends this information to you. The message’s tsIds property is set to the IDs of the timeseries that are missing a required dimension or property.

© Copyright 2019 SignalFx.

Third-party license information