SignalFx Developers Guide

Developer Home

Product Docs

SignalFx

Trace Retrieval API

API for retrieving trace data from SignalFx

If you're an Enterprise customer, you can use POST /trace/outliers to find the traces that are the biggest contributors to latency.

Retrieve Trace Data

Retrieves traces specified by search criteria

post /trace

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace

Retrieves traces based on search criteria specified in the request body

header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Request Body schema: application/json

Search criteria

duration
object

Filter criteria for the duration of an operation, specified as elapsed times in microseconds

Both the gte and lte properties are optional:

  • If you don't specify them ("duration": {}), the API returns tags regardless of their duration.
  • If you specify gte but not lte, the API returns tags with no upper bound for duration and an lower bound of gte.
  • If you specify lte but not gte, the API returns tags with no lower bound for duration and an upper bound of lte.
endTime
object (Span End Time Filter)

Filter criteria for a span's end time. This property is optional, and you can specify it as "endTime": {}".

Considerations:

  • If you omit the property or specify a null value, the API doesn't filter the results for span end times.

  • If you specify the gte property but not the lte property, the API filters for spans where
    "gte value <= span end time <= latest span end time" is true.

  • If you specify the lte property but not the gte property, the API filters for spans where
    "earliest span end time <= span end time <= lte value" is true.

  • If you specify both gte and lte, the API filters for spans where
    "gte value <= span end time <= lte value" is true.

filters
object

Additional filters for the services and operations you want. Each filter is an array of tags that the API compares to the services and operations you specify in the search criteria.

The value of filters is an object containing one or more properties. Each property has this format:

  • key: A name for an array of tags
  • value: An array of tags
operations
Array of strings

List of operations for which you want traces, in the form of a JSON array. Each element contains the name of a single operation.

services
Array of strings

Names of one or more services for which you want traces, in the form of a JSON array. Each element specifies a single service name.

spans
object

Container for sets of compound, span-level search criteria, in the form of a JSON object.

startTime
object (Span Start Time Filter)

Filter criteria for a span's start time. This property is optional, and you can specify it as "startTime": {}".

Considerations:

  • If you omit the property or specify a null value, the API doesn't filter the results for span start times.
  • If you specify the gte property but not the lte property, the API filters for spans where
    "gte value <= span start time <= latest span start time" is true.
  • If you specify the lte property but not the gte property, the API filters for spans where
    "earliest span start time <= span start time <= lte value" is true.
  • If you specify both gte and lte, the API filters for spans where
    "gte value <= span start time <= lte value" is true.
status
string
Enum:"ALL_ERRORS" "ANY" "ENDPOINT_ERRORS" "TRACE_ERRORS"

String specifying the type of error tracing data to show. The valid values are:

  • ALL_ERRORS: Select traces that have errors anywhere in the trace
  • ENDPOINT_ERRORS: Select traces only if they have have
    errors in an endpoint operation between services
  • TRACE_ERRORS: Select traces only if they have errors in the initiating operation
  • ANY: Select traces with any status
time
object

Filter criteria for the duration of an operation, specified as starting and ending timestamps in microseconds

Both the gte and lte properties are optional:

  • If you don't specify either property ("time": {}), the API returns tags regardless of their start and end times.
  • If you specify gte but not lte, the API returns tags with no maximum start time and a minimum start time of gte
  • If you specify lte but not gte, the API returns tags with no minimum end time and a maximum end time of lte.

Responses

200

Success. SignalFx returns this code even if no trace data matches the search criteria.

Response Headers
Content-Type
string

Format of the response body. Always "application/json".

Response Schema: application/json
barChart
object

Bar chart buckets for the time distribution of errors in the retrieved traces

groups
Array of objects (A single trace group)

List of trace groups for the retrieved data, in the form of a JSON array.

A trace group is a set of traces that all have the same trace identity. If relevant, this includes the initiating service and HTTP method.

histogram
object
  • Histogram buckets for the overall distribution of latency values for the retrieved traces
  • Confidence values for the histogram
map
object

Descriptor, data type, edges, and nodes for the service map that represents the retrieved traces

range
object

Lower and upper timestamp boundaries for the retrieved traces, in the form of a JSON object

sampled
boolean

Indicates that the API sampled the result set from the database. A true value means that the API reached its search limit before it found all the available results. You should set more restrictive search criteria and resubmit the request.

400

Invalid request body. The JSON is not well-formed, or properties contain invalid values.

401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. The µAPM API is not implemented in your organization.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented in your organization.

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Request samples

application/json
Copy
Expand all Collapse all
{
  • "duration":
    {
    },
  • "endTime":
    {
    },
  • "filters":
    {
    },
  • "operations":
    [
    ],
  • "services":
    [
    ],
  • "spans":
    {
    },
  • "startTime":
    {
    },
  • "status": "ALL_ERRORS",
  • "time":
    {
    }
}

Response samples

application/json
Copy
Expand all Collapse all
{
  • "barChart":
    {
    },
  • "groups":
    [
    ],
  • "histogram":
    {
    },
  • "map":
    {
    },
  • "range":
    {
    },
  • "sampled": true
}

Retrieve Trace Data Operation Name

Retrieves operation names and endpoints

get /trace/operations

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/operations

Retrieves endpoints and operations for traces.

If you specify the cluster query parameter, the API only returns endpoints and operations for data transmitted through one of the Smart Gateways associated with the cluster.

If you specify the services query parameter, the API only returns endpoints and operations for spans that contain that service.

The API lets you specify multiple instances of the services parameter. Using this feature, you can use a set of services. The resulting filter is a logical OR of the individual values.

query Parameters
cluster
string

Smart Gateway cluster name.

Clusters are load-balanced sets of Smart Gateway instances that work together to handle large volumes of data.

If you don't specify a value, SignalFx returns data for all of the Smart Gateway clusters for your organization.

services
string

Service name or names to use as filter criteria for the returned data.

If this parameter is present, the API only returns operations associated with the specified services.

To specify more than one name, use multiple instances of services= separated by &. For example, to search for data that includes the db1 or db2 service name, use services=db1&services=db2.

header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Responses

200

Success

Response Headers
Content-Type
required
string

Format of response body. Always application/json.

Response Schema: application/json
endpoints
Array of strings (Endpoints for the selected operations)

List of endpoints associated with the operations that the API returned, in the form of a JSON array

operations
Array of strings (Operations for the selected traces)

List of operations returned by the API, in the form of a JSON array

400

Invalid request body, caused by invalid JSON or improper values

401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. The µAPM API is not implemented in your organization.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented in your organization.

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Response samples

application/json
Copy
Expand all Collapse all
{
  • "endpoints":
    [
    ],
  • "operations":
    [
    ]
}

Retrieve Trace Outlier Analysis

Retrieves trace outlier analysis specified by search criteria

post /trace/outliers

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/outliers

Requests a trace outlier analysis for search criteria you specify in the request body.

Note: This operation is only available to Enterprise customers.

Outlier analysis returns two types of information:

  • Over-represented traces identified by tag. The API detects tags that occur frequently for high latency traces
  • The top contributors among operations for higher latency traces. The API reports on all operations in the 90th percentile of high latency operation, up to 100 operations total.
header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Request Body schema: application/json

Search criteria for the tracing outliers you want to analyze

duration
object

Filter criteria for the duration of an operation, specified as elapsed times in microseconds

Both the gte and lte properties are optional:

  • If you don't specify them ("duration": {}), the API returns tags regardless of their duration.
  • If you specify gte but not lte, the API returns tags with no upper bound for duration and an lower bound of gte.
  • If you specify lte but not gte, the API returns tags with no lower bound for duration and an upper bound of lte.
endTime
object (Span End Time Filter)

Filter criteria for a span's end time. This property is optional, and you can specify it as "endTime": {}".

Considerations:

  • If you omit the property or specify a null value, the API doesn't filter the results for span end times.

  • If you specify the gte property but not the lte property, the API filters for spans where
    "gte value <= span end time <= latest span end time" is true.

  • If you specify the lte property but not the gte property, the API filters for spans where
    "earliest span end time <= span end time <= lte value" is true.

  • If you specify both gte and lte, the API filters for spans where
    "gte value <= span end time <= lte value" is true.

filters
object

Additional filters for the services and operations you want. Each filter is an array of tags that the API compares to the services and operations you specify in the search criteria.

The value of filters is an object containing one or more properties. Each property has this format:

  • key: A name for an array of tags
  • value: An array of tags
operations
Array of strings

List of operations for which you want traces, in the form of a JSON array. Each element contains the name of a single operation.

services
Array of strings

Names of one or more services for which you want traces, in the form of a JSON array. Each element specifies a single service name.

spans
object

Container for sets of compound, span-level search criteria, in the form of a JSON object.

startTime
object (Span Start Time Filter)

Filter criteria for a span's start time. This property is optional, and you can specify it as "startTime": {}".

Considerations:

  • If you omit the property or specify a null value, the API doesn't filter the results for span start times.
  • If you specify the gte property but not the lte property, the API filters for spans where
    "gte value <= span start time <= latest span start time" is true.
  • If you specify the lte property but not the gte property, the API filters for spans where
    "earliest span start time <= span start time <= lte value" is true.
  • If you specify both gte and lte, the API filters for spans where
    "gte value <= span start time <= lte value" is true.
status
string
Enum:"ALL_ERRORS" "ANY" "ENDPOINT_ERRORS" "TRACE_ERRORS"

String specifying the type of error tracing data to show. The valid values are:

  • ALL_ERRORS: Select traces that have errors anywhere in the trace
  • ENDPOINT_ERRORS: Select traces only if they have have
    errors in an endpoint operation between services
  • TRACE_ERRORS: Select traces only if they have errors in the initiating operation
  • ANY: Select traces with any status
time
object

Filter criteria for the duration of an operation, specified as starting and ending timestamps in microseconds

Both the gte and lte properties are optional:

  • If you don't specify either property ("time": {}), the API returns tags regardless of their start and end times.
  • If you specify gte but not lte, the API returns tags with no maximum start time and a minimum start time of gte
  • If you specify lte but not gte, the API returns tags with no minimum end time and a maximum end time of lte.
excludeAnalyzerResult
boolean
Default: false

Controls analyzer results in the response body.

If false, the response body is unaffected. If true, the response body has these values:

  • analyzeResults is null
  • processed is unaffected
  • tags array is unaffected
excludeTopTagAnalysis
boolean
Default: false

Controls top tag analysis results in the response body.

If false, the response body is unaffected.

If true, the response body has these values:

  • analyzeResults is unaffected
  • processed is unaffected
  • tags is null
histogramMax
integer <int64>

Optional: Limits the analyzed traces to those that had a duration shorter than or equal to this value in microseconds

histogramMin
integer <int64>

Optional: Limits the analyzed traces to those that had a duration longer than or equal to this value in microseconds

numBins
integer <int32>

Requests a number of bins for the latency histogram produced by the analysis

Responses

200

Successful request. This response code is returned even if the API doesn't find any matching traces.

Response Headers
Content-Type
string

Format of the response body. Always "application/json".

Response Schema: application/json
analyzeResults
object (Outlier analysis results)

Results of an outlier analysis, in the form of a JSON object. If you set "excludeAnalyzerResult": True in your request, SignalFx returns null for AnalyzeResults.

processed
integer <int64>

The number of traces processed by this request

tags
Array of objects

List of tags analyzed, in the form of a JSON array. If you set "excludeTopTagAnalysis": True in your request, SignalFx returns null for the value of tags.

400

Invalid request body, caused by invalid JSON or improper values

401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. Either the µAPM API is not implemented in your organization, or you're a non-Enterprise µAPM customer.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented or you're not an Enterprise customer

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Request samples

application/json
Copy
Expand all Collapse all
{
  • "duration":
    {
    },
  • "endTime":
    {
    },
  • "filters":
    {
    },
  • "operations":
    [
    ],
  • "services":
    [
    ],
  • "spans":
    {
    },
  • "startTime":
    {
    },
  • "status": "ALL_ERRORS",
  • "time":
    {
    },
  • "excludeAnalyzerResult": false,
  • "excludeTopTagAnalysis": false,
  • "histogramMax": 0,
  • "histogramMin": 0,
  • "numBins": 0
}

Response samples

application/json
Copy
Expand all Collapse all
{
  • "analyzeResults":
    {
    },
  • "processed": 0,
  • "tags":
    [
    ]
}

Retrieve Trace Data Service Name

Retrieves trace data specified by service names

get /trace/services

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/services

Retrieves service names for traces.

If you specify the cluster query parameter, the API returns service names only for data transmitted through one of the Smart Gateways associated with the cluster.

query Parameters
cluster
string

Smart Gateway cluster name.

Clusters are load-balanced sets of Smart Gateway instances that work together to handle large volumes of data.

If you don't specify a value, SignalFx returns data for all of the Smart Gateway clusters for your organization.

header Parameters
Content-Type
required
string

Format of the request body. Always "application/json".

X-SF-TOKEN
required
string

Authentication token

Responses

200

Success

Response Headers
Content-Type
required
string

Format of response body. Always "application/json".

Response Schema: application/json
property name *
object (Retrieved service name)

Service name recorded in the trace data, or service name inferred by the API

400

Invalid request body, caused by invalid JSON or improper values

401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. The µAPM API is not implemented in your organization.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented in your organization.

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Response samples

application/json
Copy
Expand all Collapse all
{
  • "database":
    {
    },
  • "unit27":
    {
    },
  • "unit5":
    {
    }
}

Retrieve Recent Spans

Retrieves span names for spans received in the last 48 hours Deprecated

get /trace/spannamesbyservice

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/spannamesbyservice

DEPRECATED:

Use /trace/operations or /trace/services instead.

Retrieves span operation names for spans received in the last 48 hours. The results are aggregated by service.

header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Responses

200

Dictionary of key-value pairs for each service name received in the last 48 hours, in the form of a JSON object.

Response Headers
Content-Type
string

Format of the response body. Always "application/json".

Response Schema: application/json
property name *
object
401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. The µAPM API is not implemented in your organization.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented in your organization.

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Response samples

application/json
Copy
Expand all Collapse all
{
  • "property1":
    {
    },
  • "property2":
    {
    }
}

Retrieve Span Tags

Retrieves span tags specified by search criteria

post /trace/spantags

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/spantags

Retrieves span tags based on the search criteria you specify

header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Request Body schema: application/json

Search criteria for the span tags you want to retrieve

duration
object

Filter criteria for the duration of an operation, specified as duration boundaries.

Both the gte and lte properties are optional:

  • If you don't specify them ("duration": {}), the API returns tags regardless of their duration.
  • If you specify gte but not lte, the API returns tags with no upper bound for duration and an lower bound of gte
  • If you specify lte but not gte, the API returns tags with no lower bound for duration and an upper bound of lte
initiators
Array of strings

Trace initiators for which you want span tags, in the form of a JSON array of strings. Each element is the name of the first operation recorded for a trace.

operations
Array of strings

Operations for which you want span tags, in the form of a JSON array. Each element contains an operation name.

partialUserInput
string

A string that the API tries to match to the beginning of a span tag key or value.

The API interprets the string value according to these rules:

  • If the value ends with a colon (:), the API uses the string before the colon as a key. For example, if partialUserInput is input:, then the API returns spans that have a span key of input:.
  • If the value contains a colon (:), the API uses the string before the colon as a key and the string after the colon as a partial value. For example, if partialUserInput is input: val, then the API returns spans with a key value that starts with val for the key input.
  • In all other cases, the API looks for span keys that start with the value of partialUserInput. For example, if partialUserInput is input (without a colon), then the API returns spans that have a key value staring with input, including input, input_tag, and so forth.
services
Array of strings

Service names for which you want span tags, in the form of a JSON array of strings. Each element contains a service name.

time
object

Filter criteria for the duration of an operation in terms of starting and ending timestamps specified in microseconds

Both the gte and lte properties are optional:

  • If you don't specify either property ("time": {}), the API returns tags regardless of their start and end times.
  • If you specify gte but not lte, the API returns tags with no maximum start time and a minimum start time of gte
  • If you specify lte but not gte, the API returns tags with no minimum end time and a maximum end time of lte.

Responses

200

Successful request. This response code is returned even if the API doesn't find any matching span tags.

The response body is a list of span tags for spans that match the specified search criteria, in the form of a JSON array. Each array element is a JSON object that contains a single property. The key is the span tag key, and the value is the data stored in the tag. Tag values can be any primitive type.

Response Headers
Content-Type
string

Format of the response body. Always "application/json".

Response Schema: application/json
property name *
Array of objects
400

Invalid request body, caused by invalid JSON or improper values

401

The value of X-SF-TOKEN is invalid.

Response Schema: application/json
code
integer (HTTP response code. Always `401`)

Indicates that the request contains an invalid authentication token

message
string (Error message)

API error response message

403

Invalid request. The µAPM API is not implemented in your organization.

Response Schema: application/json
code
integer (HTTP response code. Always `403`)

Indicates that the µAPM API is not implemented in your organization.

message
string (Error message)

API error response message. Always set to "HTTP 403 Forbidden"

500

Internal server error

Request samples

application/json
Copy
Expand all Collapse all
{
  • "duration":
    {
    },
  • "initiators":
    [
    ],
  • "operations":
    [
    ],
  • "partialUserInput": "string",
  • "services":
    [
    ],
  • "time":
    {
    }
}

Response samples

application/json
Copy
Expand all Collapse all
{
  • "property1":
    [
    ],
  • "property2":
    [
    ]
}

Retrieve Spans

Retrieves all the spans for a trace

get /trace/{id}

SignalFx API endpoint

Replace {REALM} with the name of your realm. For example, if your realm is eu0, use https://api.eu0.signalfx.com/v2.

To find your realm, go to your profile page in the SignalFx web UI.

If you don't include a realm and use https:/api.signalfx.com/v2, SignalFx interprets the endpoint as pointing to the us0 realm.

https://api.{realm}.signalfx.com/v2/trace/{id}

Retrieves all the spans for the trace specified in the id path parameter. This endpoint doesn't use a request body

path Parameters
id
required
string

Trace ID value of a retained trace

header Parameters
Content-Type
required
string

Format of the request body. Always application/json.

X-SF-TOKEN
required
string

Authentication token

Responses

200

List of spans for the specified trace, in the form of a JSON array

Response Headers
Content-Type
string

Format of the response body. Always "application/json".

Response Schema: application/json
Array
annotations
Array of objects (List of annotations for a span)

Array containing annotation objects for a span retrieved by the **POST** /trace/{id} operation.

debug
boolean (Debug sampling flag)

If true, indicates that debug sampling was enabled for this span

duration
integer <int64> (Span duration)

Time in microseconds between the start and end of the span

id
string (This span's ID) [ 16 .. 32 ] characters

ID of the span represented by the properties of this object, in the form of 16 to 32 hexadecimal digits.

inferredRemoteServiceName
string (Remote service name associated with this span)

If the span represents a remote service call, inferredRemoteServiceName contains the inferred name of the remote service. To learn more about inferred services, see the topic Microservices APM Inferred Services

kind
string (Span type)
Enum:"CLIENT" "SERVER" "PRODUCER" "CONSUMER"

The type of this span. The possible values are:

  • CLIENT
  • SERVER
  • PRODUCER
  • CONSUMER
localEndpoint
object (Local endpoint for a span)

Properties that describe a span's local (starting) endpoint

name
string (Operation name)

The operation name for this span

parentId
string (Parent span ID)

ID of the parent span to this span, if one exists

remoteEndpoint
object (Remote endpoint for a span)

Properties that describe a span's remote (destination) endpoint

shared
boolean (Shared span flag)

If true, indicates that this is a shared span

tags
object

Dictionary of tags associated with the retrieved spans, in the form of a JSON object. Each property has this format: