Skip to content

29 — Resilience — Timeouts, Retries, Rate Limits and Pagination

Level: Intermediate · Time: ~40 min · Prerequisites: Lesson 28 — Authentication, Secrets and Local Mock Servers

Why this matters

One call to NASA POWER is a single coin toss; a year of irradiance is 365 of them, and the arithmetic is unforgiving — an endpoint that succeeds 99 times in 100 still fails somewhere in a 365-call batch with a probability above 97%. Lesson 21 gave you the primitives and Lesson 27's helper put a timeout on every call; this lesson turns those pieces into a policy you apply without thinking, so a flaky API costs you a delay rather than a night of debugging.

Every call carries a deadline

fetch has no default timeout, so a stalled connection stays open for as long as the operating system allows — minutes — and nothing rejects while your tool waits. Lesson 27's helper already passes AbortSignal.timeout(...); two additions make it the base below: the error must carry the server's delay.

// src/http.ts — the two additions Lesson 27's helper needs
export class HttpError extends Error {
  constructor(readonly status: number, message: string, readonly retryAfterSeconds?: number) {
    super(message);
    this.name = "HttpError";
  }
}

// inside request(): combine the caller's signal (shutdown) with the per-call deadline
const timeout = AbortSignal.timeout(options.timeoutMs ?? 15_000);
const signal = options.signal === undefined ? timeout : AbortSignal.any([options.signal, timeout]);
// and where a bad status is turned into an error:
const header = response.headers.get("retry-after");
const seconds = header === null ? undefined : Number(header);
throw new HttpError(response.status, `${response.status} from ${url.pathname}`,
  seconds !== undefined && Number.isFinite(seconds) ? seconds : undefined);

AbortSignal.any needs Node 20.3, and reading Retry-After here — the only place that sees the response — is what lets the policy obey the server instead of guessing.

Retry — which failures, how often, how long

Retrying everything is as wrong as retrying nothing: a 400 will be a 400 on the fourth attempt too.

Failure Retry? Why
fetch rejects with TypeError: fetch failed yes the connection never completed: DNS, reset socket, TLS
TimeoutError from AbortSignal.timeout yes the server was slow, not wrong
408, 425, 429 yes the server explicitly asked you to come back
500, 502, 503, 504 yes usually transient capacity problems
400, 401, 403, 404, 410, 501 no the request is wrong, gone, or not implemented
A response that fails schema validation no it fails identically every time — Lesson 30

[!IMPORTANT] A retry may only repeat what is safe to repeat: a POST that creates a row will happily create two, because the first attempt may have succeeded and lost its response on the way back.

// src/withRetry.ts
import { HttpError } from "./http.js";

export const defaultPolicy = { attempts: 4, baseMs: 300, maxDelayMs: 8_000, deadlineMs: 30_000 };
const delay = (ms: number): Promise<void> => new Promise((resolve) => setTimeout(resolve, ms));

export function isRetryable(error: unknown): boolean {
  if (error instanceof HttpError) {
    if (error.status === 408 || error.status === 425 || error.status === 429) return true;
    return error.status >= 500 && error.status !== 501;
  }
  if (error instanceof TypeError) return true; // undici reports a failed connection this way
  return (error as { name?: string } | null)?.name === "TimeoutError"; // the deadline fired
}

export async function withRetry<T>(operation: (attempt: number) => Promise<T>, policy = defaultPolicy): Promise<T> {
  const deadline = Date.now() + policy.deadlineMs;
  let lastError: unknown;
  for (let attempt = 1; attempt <= policy.attempts; attempt++) {
    try { return await operation(attempt); } catch (error) {
      lastError = error;
      if (!isRetryable(error) || attempt === policy.attempts) break;
      const backoff = Math.min(policy.baseMs * 2 ** (attempt - 1), policy.maxDelayMs) + Math.random() * policy.baseMs;
      const waitMs = error instanceof HttpError && error.retryAfterSeconds !== undefined
        ? error.retryAfterSeconds * 1_000 : backoff;
      if (Date.now() + waitMs >= deadline) { console.error(JSON.stringify({ event: "abandoned", attempt })); break; }
      console.error(JSON.stringify({ event: "retry", attempt, waitMs: Math.round(waitMs) }));
      await delay(waitMs);
    }
  }
  throw lastError; // attempts is at least 1, so something was always caught
}

Four things carry all of it: attempts, delay, a total deadline that stops a bad afternoon becoming a hung process, and one log line per attempt. Retry-After wins over the computed backoff, because the API knows when its window reopens and you do not, and jitter matters as much as doubling.

Rate limits, and what a retry is allowed to repeat

A 429 is not a failure to be beaten with more attempts; it is an instruction. Retry-After carries the wait in seconds, and some servers send an HTTP date instead, which Date.parse understands. The X-RateLimit-* family is advisory and spelled differently by each vendor, so treat it as guidance.

Your own behaviour is not advisory: a limiter that spaces calls out — keep nextAt, compare it with Date.now(), sleep the difference, then move nextAt forward — turns a burst into a steady trickle. Keep one per remote host, because a limiter per worker is defeated by the pool it protects.

The fix is Lesson 21's idempotency rule made concrete: generate one key before the loop — randomUUID() from node:crypto — and send it as Idempotency-Key on every attempt, so the server returns the first result for the repeats. When an API has no such header, do not retry the write: insert against a natural unique key and let the database reject the duplicate.

Pagination — fetch until there is no next page

Offset and limit (?limit=100&offset=200) is easy to read and to parallelise, but the rows shift while you walk — ingest day by day while the API also receives today's reading, and the walk returns one day twice. A cursor shape is stable but sequential. Either way: a page cap, a cursor that must advance, an empty-page stop.

// src/paginate.ts
interface ItemPage<T> { readonly items: readonly T[]; readonly nextCursor: string | null; }

export async function fetchAllPages<T>(fetchPage: (cursor: string | null) => Promise<ItemPage<T>>, maxPages = 50): Promise<T[]> {
  const items: T[] = [];
  let cursor: string | null = null;
  for (let page = 1; page <= maxPages; page++) {
    const result = await fetchPage(cursor);
    items.push(...result.items);
    if (result.items.length === 0 || result.nextCursor === null) return items;
    if (result.nextCursor === cursor) throw new Error(`cursor did not advance on page ${page}`);
    cursor = result.nextCursor;
  }
  throw new Error(`stopped after ${maxPages} pages; the API may be ignoring the cursor`);
}

A cursor that comes back unchanged is an infinite loop in disguise and deserves an exception, not fifty duplicate pages. Retry each page with withRetry: reads are idempotent.

Caching that pays for itself

Most repeated requests ask for data that has not changed, and three cheap layers remove most of them. An in-memory cache keyed on the request with a short time-to-live covers repeats inside one run. Conditional requests — send the stored ETag as If-None-Match, and a 304 Not Modified means "reuse what you have" — save a round trip's bandwidth and have no body, so reading one throws. The raw response written under data/raw/ outlives the process and becomes the fixture of Lesson 30. Never cache a write, a per-user response, or anything documented as live.

Common mistakes

Mistake What happens The fix
Calling fetch with no timeout A stalled connection hangs the tool for minutes with no error Pass AbortSignal.timeout, as the Lesson 27 helper does
Retrying every failure Four identical 400s, wasted quota, and a slow failure Retry only the statuses in the table above
Retrying a POST that creates a row Duplicate readings, because the first attempt may have succeeded Send an idempotency key, or do not retry writes
Backoff with no jitter Clients that failed together retry together and re-break the API Add Math.random() * baseMs to every delay
No total deadline, only attempts A long outage becomes a run that finishes after midnight Budget the whole operation and log the abandonment
Walking pagination with no page cap An offset walk over shifting data, or a cursor that never advances Cap the pages, require progress, stop on an empty page

Try it

  1. Add src/withRetry.ts as above and the two additions to src/http.ts, then extend the mock server of Lesson 28 so a failure can be temporary — a counter outside the handler, and a mode that fails twice:
if (mode === "flaky" && flakyCount++ < 2) { response.writeHead(503); response.end(); return; }
  1. Call it through the policy and watch the decision being made:
const mock = await startMockServer("test-token");
const fixture = await withRetry(() => request(mock.url, "/api/v1/daily?mode=flaky"));
console.log("succeeded after retries", fixture);
await mock.close();

Expected output: two lines on stderr — {"event":"retry","attempt":1,…} and …"attempt":2… — then succeeded after retries with the fixture. The waits grow and are never identical twice: that is the jitter.

  1. Prove the deadline: call ?mode=slow with { timeoutMs: 500 }, time the run with time node dist/index.js, then set attempts: 2.

Expected output: four attempts, each ending in a timeout, a wall clock bounded by deadlineMs, and a final TimeoutError; then the same failure, sooner.

Key takeaways

  • There is no default timeout on fetch: a deadline belongs on every request, and the signal must reach it.
  • Retry decisions come from the status: timeouts, 408, 425, 429 and 5xx except 501 are retryable; 400, 401, 403, 404 are not.
  • A policy is attempts, delay and a total budget plus jitter; it may only repeat what is safe to repeat, so writes need an idempotency key.
  • Retry-After and the X-RateLimit-* headers are instructions, your own shared limiter is cheaper than being throttled, and pagination needs a page cap plus a cursor that must advance.

Check yourself

  1. fetch rejects with TimeoutError on the fourth attempt of a GET. Should a fifth happen, and what else has to be true before it is useful?
  2. The API answers 429 with Retry-After: 30 while your backoff would have been 300 ms. Which wins, and what does the deadline do?
  3. Why is the idempotency key generated before the retry loop rather than inside it?
  4. Your pagination loop sees a nextCursor identical to the cursor it just used. Name two possible causes, and say why a higher page cap is the wrong response to either.

Next

Lesson 30 — Typing API Responses at the Boundary