Tuning

Each mechanism in this library is documented on its own page, and each is simple enough on its own. Choosing them together used to be the hard part: the retry settings decided how fast the circuit breaker tripped, and the breaker's window decided whether it tripped at all.

Most of that is gone in 3.0. :tolerate counts failing calls, so it no longer moves when you change :max_attempts; :within sizes itself against your retry settings; and the configuration you write is checked against itself when you compile it. What is left is one coupling worth understanding and a handful of judgement calls, which is what this guide is about.

Every number here was measured against the library rather than derived.

Ask the library first

Before reading further, ask your configuration what it does:

use ExternalService,
  circuit_breaker: [tolerate: 3],
  retry: [backoff: :exponential, base: 100, max_attempts: 5]

IO.puts ExternalService.explain(MyApp.Stripe)

MyApp.Stripe

  retry
    window       1.5s
    delays       100ms, 200ms, 400ms, 800ms
    attempts     up to 5
    time budget  none (:expiry unset)

  circuit breaker
    opens after      4 failing calls
    counting window  10s
    resets after     60s
    backend          ExternalService.CircuitBreaker.Fuse

  rate limit
    none  calls are not throttled

  concurrency
    none  calls are not limited in flight

  a fully-failing call
    spends  1.5s waiting between attempts
    plus    however long its 5 attempts take — nothing here bounds a single attempt

It takes a proposed keyword list as well as a started service, so you can try a configuration before shipping it. Anything this guide warns about, it also warns about — at compile time, on the line where the configuration is written.

What you are actually choosing

Four things, and the setting that controls each:

The first row is the one people get wrong. A longer retry window and more attempts feel like the same thing, and they are not:

• The window costs you nothing but latency on a call that was already failing.
• The attempt count also multiplies the load you put on a struggling dependency.

So when you want to ride out a longer outage, raise :base. Reach for :max_attempts only when you actually want more attempts.

What a configuration costs

Total time a call spends waiting between attempts, with exponential backoff. ExternalService.RetryOptions.window/1 answers this for any configuration; the table is here to show the shape.

Read that table down a column rather than across a row. Each extra attempt costs roughly as much as everything before it combined, so the last one or two attempts account for most of the window.

:cap is what stops that runaway, and it matters more the more attempts you have:

At the default of five attempts a cap changes nothing. Past six it is the difference between a bounded call and a minute-long one — which is why the library warns about an uncapped configuration above six attempts.

The coupling that remains

base: 100 is the usual advice for an HTTP dependency, and max_attempts: 10 sounds merely thorough. Together they are a 51-second call, and nothing about either setting hints at the other.

If you raise one, look at the table before raising the other — or ask explain/1, which does it for you.

Sizing the breaker

:tolerate is the number of failing calls you are willing to absorb; the breaker opens on the next one. It is independent of :max_attempts, so this is now one decision rather than an arithmetic problem:

circuit_breaker: [tolerate: 3]

Measured with base: 100, max_attempts: 5: the breaker opens on the 4th consecutive fully-failing call, and a single failing call takes about 1.4 seconds.

:within — the window those failures are counted over — defaults to :auto and is computed from your retry settings. Set it yourself only when you know something the configuration does not, which is usually how long your attempts take. A configuration states how long it waits between attempts and says nothing about the attempts themselves, so a dependency that hangs for ten seconds per attempt needs a wider window than :auto can know to ask for.

Three situations

A request path

You have a latency budget, and a call that fails fast is better than one that succeeds too late. Keep the retry window well inside the budget and let the breaker do the rest.

use ExternalService,
  circuit_breaker: [tolerate: 3, reset: :timer.seconds(5)],
  retry: [backoff: :exponential, base: 100, max_attempts: 4, expiry: 1_000, jitter: true]
  # :wait defaults to one window, which is what you want here

Measured: a fully-failing call takes about 670ms, and the breaker opens on the 4th consecutive one. After that, callers get CircuitBreakerOpen immediately instead of waiting 670ms to fail. :within resolves to 10 seconds.

Note what the :expiry is for. It is not bounding the backoff — 700ms of waiting is nowhere near a second. It bounds the case the attempt count cannot: a slow dependency, where four attempts at three seconds each would otherwise mean twelve seconds of call. :expiry is checked between attempts, so it stops the next one rather than interrupting the current one; see Nothing here bounds a single attempt.

A background job

Nobody is waiting, and the work has nowhere else to go — so trade latency for success. This is where the patient settings are correct rather than careless.

use ExternalService,
  circuit_breaker: [tolerate: 3, reset: :timer.seconds(30)],
  retry: [
    backoff: :exponential,
    base: 500,
    cap: :timer.seconds(5),
    max_attempts: :infinity,
    expiry: :timer.seconds(30),
    jitter: true
  ],
  rate_limit: [limit: 100, per: :timer.seconds(1), wait: :infinity]

max_attempts: :infinity with an :expiry is the useful shape: keep trying, but stop after half a minute. Unbounded on both is rejected outright — under the default melt semantics a call that never gives up never melts the breaker, so nothing would stop it.

Measured: a fully-failing call takes 30 seconds, spending its budget exactly, and the breaker opens on the 4th consecutive one — two minutes into a total outage. :within resolves to 180 seconds, which is what makes that possible: four failures two minutes apart have to be counted over a window wider than two minutes.

That number is worth pausing on. A slow service needs a wider counting window than a fast one, not a narrower one, and it is the single most common thing to get wrong by hand. Leave :within unset unless you know your attempts are slower than your backoff.

If your job runner already retries failed jobs, prefer a modest bound here and let re-enqueueing be the outer loop. Two retry layers multiply.

A Flow pipeline

The important setting is wait: :infinity, and it is the one most easily missed.

use ExternalService,
  circuit_breaker: [tolerate: 3],
  retry: [backoff: :exponential, base: 100, cap: 2_000, max_attempts: 5, jitter: true],
  rate_limit: [limit: 100, per: :timer.seconds(1), wait: :infinity]

Measured: a fully-failing call takes about 1.5 seconds and the breaker opens on the 4th consecutive one.

In a pipeline, sleeping is how back-pressure reaches upstream. A :wait budget sheds work mid-pipeline instead of pacing it, and that work usually has nowhere to go. See Flow Pipelines.

Where the rate limiter fits

:wait defaults to one window (:per), capped at 5 seconds — enough to absorb a burst, not enough to hide sustained saturation. Override it in two directions:

• :infinity for background work and pipelines, as above.
• false, or a budget well under one window, when your latency budget is tighter than the limiter's window and you would rather return 429 than wait.

Rate limiting composes with retries in a way worth noticing: a throttled call that exhausts its :wait budget returns RateLimited without melting the circuit breaker and without being retried, because the wrapped function never ran. Throttling is not failure. See Rate limiting.

A checklist

Four of the seven items this guide used to carry are now checked for you when you compile. What is left needs a human:

• [ ] Read your retry window off explain/1 — is it inside the latency budget of whoever is calling?
• [ ] Is :tolerate the number of failing calls you are actually willing to absorb?
• [ ] Do you know how long a single attempt can take? Nothing here bounds it, and :within cannot size itself against it.
• [ ] Is :wait right for the call site — default for a request path, :infinity for a pipeline or job?
• [ ] Is :jitter on, if many processes call this service?