SignalFx

Getting Started With the SignalFlow API

SignalFx's SignalFlow streaming analytics service is available through an HTTP API. The set of endpoints offered by this API is available both via direct, authenticated HTTP requests and through a WebSocket connection.

Whenever possible, the WebSocket connection should be used; it supports executing and streaming multiple SignalFlow computations through a single WebSocket connection, uses a more efficient encoding protocol and generally offers lower latency. The HTTP endpoints offer the same functionality and can be easier to integrate with, but it requires one connection per stream.

The SignalFlow API (either directly via HTTP, or through the WebSocket connection) allows you to execute SignalFlow programs as running computations and receive their output. In the case of long-running computations, you will receive the real-time, streaming output of your computation. The returned stream is composed of independent messages carrying information about the execution of the computation, metadata of the output timeseries and of course, the generated data and events.

For more information on the format of those messages, see the Stream messages specification.

Authenticating

In order to use the SignalFlow API you must have an access token. For more information, see Authentication Overview.

Client libraries

In order to make integrating with the SignalFlow API easier, SignalFx provides client libraries in a variety of programming languages: Python and Javascript to start, while Java, Go and Ruby are being worked on. Those libraries completely abstract the details of integrating with the SignalFlow API and provide high-level abstractions that allow for easily using SignalFlow computations that execute on SignalFx's streaming analytics service.

Head over to SignalFx's GitHub page to find the client library for your programming language.

Executing and detaching from computations

Assuming that you are authenticated, you can execute SignalFlow computations from the text of your SignalFlow program. Executing a computation starts a job on SignalFx's analytics service that performs the computation and streams the output back to you via a channel. When using the direct HTTP API, the channel is implicit and is represented by the HTTP connection itself; when using the WebSocket connection, multiple channels are multiplexed through a single WebSocket and the name of the channel identifies what computation the messages and data come from.

To stop receiving the computation's output, simply detach from the channel. When using a direct HTTP connection, closing the connection automatically closes the implicit channel. When using the WebSocket connection, you have to send a detach request (see /signalflow/connect for the WebSocket connection protocol details). Note that disconnecting from the WebSocket also automatically closes all the channels that were established in that connection.

As long as a channel is open and its underlying connection remains established, the computation is automatically kept alive for you. If the computation ends (normally with an END_OF_CHANNEL message, or abnormally with an ABORT_CHANNEL message), the channel is automatically closed, and when using a direct HTTP connection, the server will close the connection following those messages.

"Hello, world!", from the command-line

The simplest possible example is to execute an ad-hoc SignalFlow computation using the HTTP API. In this example, we simply perform the overall average of the CPU utilization of all servers in our infrastructure and stream the result back in real-time.

$ curl --request POST \
       --header "Content-Type: text/plain" \
			 --header "X-SF-Token: YOUR_TOKEN" \
       --data "data('cpu.utilization').mean().publish()" \
       "https://stream.signalfx.com/v2/signalflow/execute"

The result will be a streaming HTTP response of Server-Sent Events (SSE). For more informations, see the /signalflow/execute endpoint documentation and the Stream messages specification.

Getting started in Python

The signalfx-python client library for Python includes a SignalFlow API client that makes integrating with the API a breeze. First, start by installing the client library:

$ pip install signalfx

Writing Python code to execute a SignalFlow computation and handle its real-time streaming output only takes a few lines of code! Here's the same example as above, in Python:

import signalfx

program = "data('cpu.utilization').mean().publish('cpu')"
flow = signalfx.SignalFx().signalflow('MY_TOKEN')
try:
    print('Executing {0} ...'.format(program))
    computation = flow.execute(program)
    for msg in computation.stream():
        if isinstance(msg, signalfx.signalflow.messages.DataMessage):
            print('{0}: {1}'.format(msg.logical_timestamp_ms, msg.data))
        if isinstance(msg, signalfx.signalflow.messages.EventMessage):
            print('{0}: {1}'.format(msg.timestamp_ms, msg.properties))
finally:
    flow.close()

For more information about the Python client library, its documentation and the format of the stream messages in Python, visit the signalfx-python GitHub repository.

Getting started in Javascript

We also provide a Node.js library (which can be browserified). It can be installed with npm:

$ npm install signalfx

In JavaScript as well, executing a SignalFlow computation and handling its output is very simple. Here's the same example as above using the signalfx-nodejs library:

var signalfx = require('signalfx')

var flow = signalfx.SignalFlow('MY_TOKEN');

var computation = flow.execute({
    program: "data('cpu.utilization').publish('cpu')",
    start: Date.now() - 60000,
    stop: Date.now() + 600000,
    resolution: 10000,
    maxDelay: 1000
  });

computation.stream(function (err, data) {
  if(err) {
    // handle error
  } else {
    console.log(data);
  }
});

Getting Started With the SignalFlow API