ExternalService.Insights (ExternalService v3.1.0)

Copy Markdown View Source

Watches a service's telemetry and reports when its configuration is not doing what it was meant to do.

ExternalService.Insights.attach()

ExternalService.explain/1 and ExternalService.simulate/3 answer questions about a configuration from the configuration. This answers the one question they cannot: whether what is actually happening matches it.

The gap is attempt duration. Nothing in a configuration states how long a single attempt takes, so a breaker sized correctly on the day it was written becomes inert when the dependency slows down — same configuration, different behavior, and the symptom is a service failing every call with its breaker still closed. That is a bad thing to discover during an incident and a cheap thing to notice beforehand.

[ExternalService.Insights] :payments has failed 14 consecutive calls over
21.7s with its circuit breaker still closed. It tolerates 5 failures within
10s, but these are arriving about 1.5s apart, so at most 6 are ever counted
together. Try `circuit_breaker: [within: :timer.seconds(31)]`.

Attaching

Off by default, and free when it is not attached — no handlers, no storage, no cost on any call.

# in your application's start/2, after the supervision tree is up
ExternalService.Insights.attach()

attach/1 watches every service that has been started at that moment. A service started later needs its own call:

ExternalService.Insights.attach(services: [MyApp.Stripe])

Options

  • :services — which services to watch. Defaults to every started service.
  • :log — whether to log findings as they appear. Defaults to true.
  • :log_every — the minimum gap between log lines for one service, in milliseconds. Defaults to 5 minutes. A diagnostic that fires on every call is a diagnostic people detach.

Reading findings as data

report/1 returns what is being observed and what it means, so the same findings can drive a health endpoint or an assertion rather than a log line:

%ExternalService.Insights.Report{
  service: :payments,
  calls: 214,
  findings: [%{check: :inert_breaker, message: "..."}],
  ...
}

What it costs

One :atomics array per watched service — a fixed dozen integers, updated without locks — and one key in the calling process's dictionary for the duration of a call. Nothing accumulates, and no process is started.

Summary

Functions

Starts watching. See the module documentation for the options.

Stops watching, and discards what has been observed.

What has been observed for service, and what it means.

Functions

attach(opts \\ [])

@spec attach(keyword()) :: :ok

Starts watching. See the module documentation for the options.

detach()

@spec detach() :: :ok

Stops watching, and discards what has been observed.

report(service)

@spec report(ExternalService.service()) ::
  ExternalService.Insights.Report.t() | {:error, :not_attached}

What has been observed for service, and what it means.

Answers {:error, :not_attached} for a service that is not being watched.