# ExternalService Cheatsheet

Quick recipes for configuring and calling services. See the guides for full
detail.

## Define a service
{: .col-2}

### Module front door (recommended)

```elixir
defmodule MyApp.Api do
  use ExternalService,
    circuit_breaker: [tolerate: 5, within: :timer.seconds(1), reset: :timer.seconds(5)],
    rate_limit: [limit: 100, per: :timer.seconds(1), wait: :timer.seconds(1)],
    retry: [max_attempts: 5, backoff: :exponential, jitter: true]

  def fetch(id), do: call(fn -> HTTP.get("/things/#{id}") end)
end
```

Start it under a supervisor:

```elixir
children = [MyApp.Api]
Supervisor.start_link(children, strategy: :one_for_one)
```

### Functional API

```elixir
ExternalService.start(:payments,
  circuit_breaker: [tolerate: 5, within: 1_000, reset: 5_000],
  retry: [max_attempts: 3]
)

ExternalService.call(:payments, fn -> charge() end)
```

## Calling
{: .col-2}

### Synchronous

```elixir
MyApp.Api.call(fn -> work() end)
MyApp.Api.call([max_attempts: 2], fn -> work() end)  # per-call retry opts
MyApp.Api.call!(fn -> work() end)                     # raises on failure
```

### Async / parallel

```elixir
task = MyApp.Api.call_async(fn -> work() end)
Task.await(task)

ids
|> MyApp.Api.call_async_stream(fn id -> fetch(id) end)
|> Enum.to_list()
```

## Triggering retries
{: .col-2}

### Return values

```elixir
call fn ->
  case HTTP.get(url) do
    {:ok, %{status: 200} = r}            -> {:ok, r}
    {:ok, %{status: s}} when s in 500..599 -> {:retry, s}  # retry
    {:ok, %{status: 429}}                -> :retry          # retry
    other                                -> other           # success
  end
end
```

### Key rule

| Return | Effect |
| --- | --- |
| `:retry` | retry |
| `{:retry, reason}` | retry (reason recorded) |
| value matched by `:retry_on` predicate | retry (result recorded as reason) |
| anything else | success, returned as-is |
| raised exception | propagates (retried only if matched by `:retry_exceptions`) |

## Circuit breaker config
{: .col-2}

### Options

```elixir
circuit_breaker: [
  tolerate: 5,                # failed ATTEMPTS per window (default 10);
                              # retries melt too, so tolerate ≈
                              # failing calls × max_attempts
  within: :timer.seconds(1),  # window ms (default 10_000)
  reset: :timer.seconds(5)    # ms open before reset (default 60_000)
]

circuit_breaker: [tolerate: :infinity]   # no breaker at all;
                                         # never opens, holds no state
```

### Trip the cluster together

```elixir
circuit_breaker: [
  tolerate: 5,
  backend: ExternalService.CircuitBreaker.Cluster
]

# narrow the broadcast:
backend: {ExternalService.CircuitBreaker.Cluster,
          nodes: &MyApp.api_nodes/0}
```

Default breaker is node-local (each node trips on its own).

### Introspect & reset

```elixir
MyApp.Api.available?()   # breaker closed?
MyApp.Api.blown?()       # breaker open?
MyApp.Api.reset()        # force closed

ExternalService.all_available?([:a, :b])
```

### Understand a configuration

```elixir
# what will this do?
IO.puts ExternalService.explain(MyApp.Api)

# does the breaker actually open? (virtual clock, nothing sleeps)
ExternalService.simulate(MyApp.Api, :always_failing)
#=> %Simulation{opens_after: 4, worst_call: 1500, attempts: 20, ...}

# how long does a fully-failing call wait?
ExternalService.RetryOptions.window(base: 100, max_attempts: 5)   #=> 1500
```

Both take a proposed keyword list too, so you can try a configuration before
shipping it.

### Drive it directly

```elixir
# count a failure that happened outside call/3
ExternalService.CircuitBreaker.melt(:api)

# :ok | :blown | :not_started
ExternalService.CircuitBreaker.ask(:api)
```

## Rate limit config
{: .col-2}

### Options

```elixir
rate_limit: [
  limit: 100,               # calls per window
  per: :timer.seconds(1),   # window ms
  wait: :timer.seconds(1)   # :infinity | ms | false
]                           # unset: one window (:per), capped at 5s

rate_limit: [limit: :infinity, per: 1_000]   # no limiter at all
```

Burst up to `limit`, then paced at `per / limit`.

### Bound the wait

```elixir
rate_limit: [limit: 100, per: 1_000, wait: 2_000]

# budget exhausted -> function never runs:
{:error, %ExternalService.RateLimited{
  context: %{retry_after: ms}}}   # http_status 429
```

`wait: false` fails immediately. Never melts the breaker; never retried.
Use `wait: :infinity` for background work and Flow pipelines.

### Share a limit across a cluster

```elixir
defmodule MyApp.RateLimit do
  use Hammer, backend: Hammer.Redis
end

rate_limit: [
  limit: 100, per: 1_000,
  backend: {ExternalService.RateLimiter.Hammer,
            module: MyApp.RateLimit}
]
```

Default backend is node-local: N nodes ⇒ up to N × `limit`.

### Ask, and spend, directly

```elixir
ExternalService.rate_limited?(:api)      # boolean, consumes nothing

ExternalService.RateLimiter.peek(:api)   # :ok | {:wait, ms}

# spend budget for a call made some other way
ExternalService.RateLimiter.request(:api)
```

## Concurrency limit
{: .col-2}

### Options

```elixir
concurrency: [
  limit: 25,                        # calls in flight at once
  reclaim_after: :timer.seconds(30),# slot expiry; must exceed
                                    # your client timeout
  wait: 50                          # ms | false (default)
]
```

Over the limit, calls shed rather than queue. A short
`:wait` absorbs bursts; `:infinity` is rejected.

### Errors & introspection

```elixir
{:error, %ExternalService.ServiceSaturated{
  context: %{limit: l, in_flight: n}}}   # http_status 503

ExternalService.saturated?(:svc)
ExternalService.Concurrency.in_flight(:svc)
```

Saturation does not melt the breaker and is not retried.

## Retry options
{: .col-2}

### All options

```elixir
retry: [
  backoff: :exponential,   # or :linear
  base: 100,               # initial delay ms (default 10)
  factor: 1,               # :linear growth factor
  cap: :timer.seconds(2),  # max single delay
  max_attempts: 5,         # attempt count bound, the default (or :infinity)
  expiry: :timer.seconds(10), # time budget ms (or :infinity)
  jitter: true,            # ±10%, or a float proportion
  retry_on: &match?({:error, _}, &1), # predicate over the result
  retry_exceptions: []     # exception modules, or a predicate on the exception
]
```

### Recipes

```elixir
# Fast, bounded
retry: [max_attempts: 3, backoff: :linear, base: 50]

# Resilient HTTP default
retry: [backoff: :exponential, base: 100, cap: 2_000,
        max_attempts: 5, jitter: true]

# Retry a transient exception
retry: [retry_exceptions: [MyApp.TransientError]]

# Retry an exception only for some instances
retry: [retry_exceptions: &match?(%MyApp.HTTPError{status: s} when s >= 500, &1)]

# Retry on the result of an unmodified function
retry: [retry_on: &match?({:error, %{status: 500}}, &1)]

# Deliberately unbounded (opts out of the max_attempts: 5 default)
retry: [max_attempts: :infinity, cap: 30_000]
```

> #### `max_attempts: 5` is a bound, not an allowance {: .warning}
>
> With the default `base: 10` that is 150ms of waiting in total. For a real
> dependency raise `:base`, not the attempt count. `max_attempts: :infinity`
> retries forever — the circuit breaker does not reliably stop it.

## Error handling
{: .col-2}

### Returned by `call`

```elixir
case MyApp.Api.fetch(id) do
  {:ok, v} -> v
  {:error, %ExternalService.RetriesExhausted{}} -> degrade()
  {:error, %ExternalService.CircuitBreakerOpen{}} -> degrade()
  {:error, %ExternalService.RateLimited{}} -> shed()
  {:error, reason} -> {:error, reason}  # your own error
end
```

### Raised by `call!`

```elixir
rescue
  e in [ExternalService.RetriesExhausted,
        ExternalService.CircuitBreakerOpen] ->
    send_resp(conn, 503, "")
```

## Telemetry events
{: .col-2}

### Events

```text
[:external_service, :call, :start]
[:external_service, :call, :stop]
[:external_service, :call, :exception]
[:external_service, :call, :retry]
[:external_service, :circuit_breaker, :blown]
[:external_service, :rate_limit, :sleep]
[:external_service, :concurrency, :rejected]
[:external_service, :concurrency, :waited]
```

### Attach

```elixir
:telemetry.attach_many(
  "es-handler",
  [[:external_service, :call, :retry],
   [:external_service, :circuit_breaker, :blown]],
  &MyApp.Telemetry.handle/4,
  nil
)
```

## Testing
{: .col-2}

### Make a service inert

```elixir
# test.exs child spec override
{MyApp.Api,
 circuit_breaker: [tolerate: :infinity],
 rate_limit: [limit: :infinity],
 retry: [max_attempts: 1]}
```

Both `:infinity` keys remove state, so nothing leaks between tests.

### Isolate per test

```elixir
setup context do
  service = :"#{context.module}.#{context.test}"
  on_exit(fn -> ExternalService.stop(service) end)
  {:ok, service: service}
end
```

### Or reset shared state

```elixir
setup do
  MyApp.Api.reset_all()   # breaker + limiter
  :ok
end
```

`reset/0` clears only the breaker.

### Force the failure paths

```elixir
# open the breaker (tolerate + 1 melts)
Enum.each(0..tolerate, fn _ ->
  ExternalService.CircuitBreaker.melt(svc)
end)

# spend the rate limit budget
ExternalService.RateLimiter.request(svc)

# fail a fraction of calls
circuit_breaker: [fault_injection: 0.25]
```

### Keep it off the clock

```elixir
retry: [max_attempts: 3, base: 0]   # instant retries
rate_limit: [wait: false]           # shed, don't wait
```

For rate limits and concurrency, never `sleep_function: fn _ -> :ok end` — it busy-waits. (For retry backoff it is fine: those delays are a fixed sequence.)