How do you keep retry with backoff from turning into a thundering herd?

programming web dev
1

Yo folks, I’m wiring up a fetch wrapper that retries with exponential backoff + jitter, and I’m trying to keep it fast under load. My failure mode right now is when the API has a brief outage, a bunch of callers all retry around the same time and it spikes even harder.

const sleep = (ms, signal) => new Promise((res, rej) => {
  const id = setTimeout(res, ms);
  signal?.addEventListener("abort", () => {
    clearTimeout(id);
    rej(new DOMException("Aborted", "AbortError"));
  }, { once: true });
});

export async function fetchWithRetry(url, opts = {}) {
  const {
    retries = 5,
    baseDelay = 200,
    maxDelay = 5000,
    signal,
  } = opts;

  let lastErr;
  for (let attempt = 0; attempt <= retries; attempt++) {
    try {
      const res = await fetch(url, { ...opts, signal });
      if (res.ok) return res;
      if (res.status >= 400 && res.status < 500 && res.status !== 429) return res;
      throw new Error(`HTTP ${res.status}`);
    } catch (err) {
      lastErr = err;
      if (attempt === retries) break;

      const exp = Math.min(maxDelay, baseDelay * 2 ** attempt);
      const jitter = exp * (0.5 + Math.random());
      await sleep(jitter, signal);
    }
  }
  throw lastErr;
}

Algorithm-wise, what’s a solid pattern to reduce herd behavior here (like request coalescing, per-host token bucket, or a shared retry scheduler) without making the complexity explode?