SignalFx Developers Guide

Microservices APM Inferred Services

Some remote services don’t produce spans. This can happen when:

  • You haven’t instrumented the service yet.

  • You can’t instrument the service.

Even though the remote service doesn’t have a span, SignalFx can infer a service name and service type for it. To do this, SignalFx looks at the span tags in the span that recorded the call to the remote service:

  • If you’re using auto-instrumentation, SignalFx automatically adds the tags to the span.

  • If you’re manually instrumenting the calling service, you have to add the tags yourself.

For the inferred remote service, SignalFx stores the service name and type. This information appears when you retrieve traces, spans, and service maps using the API.

The topic Inferred Services in the product documentation provides a detailed description of inferred services.

The following sections describe the information you receive for tracing data using the API.

Inferred service span properties

When you retrieve traces or service maps, you receive the following properties:

  • Retrieve Trace Data (POST https://api.{realm}.signalfx.com/v2/trace): The response has a map property that represents the service map for the retrieved traces. This property contains map.nodes, which is a dictionary of objects that describe individual services in the map. Each object contains the following:

    • "name": "<service-name>"
      This is the service name. For a service recorded in a span, SignalFX gets this value from the span. For an inferred service, SignalFx infers it from span tags.

    • "type": "<inferred-service-type>"
      For an inferred service, SignalFx infers the type from span tags. For a service recorded in a span, the value is set to "UNKNOWN".

    • "inferred": If the object represents an inferred service, this property is true; otherwise, it’s false. If the object doesn’t represent an inferred service, type is set to "UNKNOWN" and inferred is set to false.

  • Retrieve Spans (GET https://api.{realm}.signalfx.com/v2/trace/{id}): The response is an array in which each element is an object that represents a span. An object associated with an inferred service contains:

    • "inferredRemoteServiceName": "<service-name>"

      If the span isn’t associated with an inferred service, this property isn’t present in the object.

  • Retrieve Service Map Using Query (POST https://api.{realm}.signalfx.com/v2/servicemap/traces):The response has a map property that represents the service map for the retrieved traces. This property contains map.nodes, which is a dictionary of objects that describe individual services in the map. Each object contains the following:

    • "name": "<service-name>".
      This is the service name. For a service recorded in a span, SignalFX gets this value from the span. For an inferred service, SignalFx infers it from span tags.

    • "type": "<inferred-service-type>" For an inferred service, SignalFx infers the type from span tags. For a service recorded in a span, the value is set to "UNKNOWN".

    • "inferred": If the object represents an inferred service, this property is true; otherwise, it’s false.

  • Retrieve Service Map Using Trace ID (GET https://api.{realm}.signalfx.com/v2/servicemap/trace/{id}): The response is the same as Retrieve Service Map Using Query, but for a single trace. It contains the same properties and values for an inferred service.

Determining the service name

To infer the service name and type for a remote service, SignalFx has to find at least one of a specific set of tags in incoming spans.

For example, consider a span that records a call to a remote publish-subscribe service. Auto-instrumentation (or your code) sets the topic tag to "MessageService". Since topic is one of the tag names from which SignalFx can infer a remote service name, SignalFx stores the type "PUBSUB" and service name "MessageService" for the remote service.

The topic Inferred Services Instrumentation in the product documentation describes the inference process in more detail.

The following table summarizes the span tags that SignalFx uses to infer the hostname and type

Table 1. Service inference from span data
Span tags Inferred type Inferred service name

At least one of

  • peer.service

  • peer.hostname

  • peer.address

  • http.url

"HTTP"

  • peer.service is OpenTracing’s preferred way to specify a hostname.

  • peer.address must be a valid URL. For the URL http://www.example.com, SignalFx uses example as the service name.

  • http.url must be a valid URL. For the URL http://www.example.com, SignalFx uses example as the service name.

  • REQUIRED db.instance

  • db.type

"DATABASE"

  • db.instance is a logical database name, not instance name. For example, "users" rather than "users-1"

  • db.type: For example "mysql" or "redis". If present, prepended to the value of db.instance. For example, "mysql:users".

  • REQUIRED db.instance

  • REQUIRED db.type set to "memcached"

"CACHE"

  • db.instance is a logical database name, not instance name.

  • If "db.type" is not "memcached", SignalFx infers "DATABASE"

  • message_bus.destination

  • topic

  • peer.service

    To infer the host name, SignalFx needs at least one of message_bus.destination or topic.

"PUBSUB"

  • message_bus.destination is the name of destination topic or channel

  • topic is the name of destination topic or channel. For example, "logmessages"

  • peer.service is the logical service name for the topic or channel. For example, "pubsubserver". If present, prepended to the value of message_bus.destination or topic. For example, "pubsubserver:logmessages".

SignalFx returns information about inferred services in traces, spans, and service maps you retrieve using one of these operations:

  • Retrieve Trace Data: POST https://api.{REALM}.signalfx.com/v2/trace

  • Retrieve Spans: GET https://api.{REALM}.signalfx.com/v2/trace/{id}

  • Retrieve Service Map Using Query: POST https://api.{REALM}.signalfx.com/v2/servicemap/traces

  • Retrieve Service Map Using Trace ID: GET https://api.{REALM}.signalfx.com/v2/servicemap/trace/{id}

© Copyright 2019 SignalFx.

Third-party license information