21 — Concurrency — Parallel Work, Timeouts and Cancellation

Level: Intermediate · Time: ~40 min · Prerequisites: Lesson 20 — Error Handling Done Properly

Why this matters

A year of daily yields is 365 small requests, and fetching them one at a time at 200 ms each takes over a minute of wall-clock time for work the machine spends almost entirely waiting. That waiting is what Node was built around. But reaching for Promise.all everywhere creates failures of its own: one dead endpoint throws away fifty good results, five hundred simultaneous connections get you rate-limited or exhaust the socket pool, and a request that never answers keeps the process alive after the operator asked it to stop. Concurrency done properly means the amount of parallel work is a decision you took rather than an accident: how many at once, how long to wait, what happens to the failures, and how to stop cleanly on Ctrl+C.

Four ways to wait, and what each does with a failure

Combinator	Resolves when	If a promise rejects
`Promise.all`	every promise has fulfilled	the call rejects at once and the successful results are lost
`Promise.allSettled`	every promise has settled	never rejects: you get `{ status: "fulfilled" }` or `{ status: "rejected" }` per item
`Promise.race`	the first promise settles	rejects if the first to settle was a rejection
`Promise.any`	the first promise fulfils	rejects with an `AggregateError` only once all have failed

So all is for batches that must be complete to be useful, allSettled for partial success, race for a deadline or "first answer wins", and any for interchangeable sources such as two mirrors of the same dataset. Partial success is the interesting one, because it needs a report as well as a result:

const outcomes = await Promise.allSettled(urls.map((url) => fetchYield(url)));
const rows: YieldRow[] = [];
const failures: DomainError[] = [];
outcomes.forEach((outcome) => {
  if (outcome.status === "fulfilled") rows.push(...outcome.value);
  else failures.push(toDomainError(outcome.reason, "API_UNREACHABLE"));
});
console.log(`${rows.length} rows kept, ${failures.length} failed: ${failures.map((f) => f.code).join(", ")}`);

toDomainError from Lesson 20 turns each rejection reason into the one error type, so failures can be counted, logged and reported without losing the rows that arrived.

[!TIP] Reach for allSettled by default whenever the items are independent of each other, and reserve Promise.all for the batches where a single missing result makes the whole run worthless.

Batching, and a pool you can read

Unlimited parallelism is not free: every in-flight request holds a socket, a buffer and a slot on the server you are calling. A pool keeps a fixed number of workers alive, each pulling the next item as it finishes:

export async function mapWithLimit<T, R>(
  items: readonly T[],
  limit: number,
  worker: (item: T) => Promise<R>,
  onDone?: (done: number, total: number) => void,
): Promise<R[]> {
  const results = new Array<R>(items.length);
  const queue = items.map((item, index) => ({ item, index }));
  let done = 0;
  async function drain(): Promise<void> {
    for (let entry = queue.shift(); entry !== undefined; entry = queue.shift()) {
      results[entry.index] = await worker(entry.item);
      done += 1;
      onDone?.(done, items.length); // progress reporting, straight from the pool
    }
  }
  await Promise.all(Array.from({ length: Math.min(limit, items.length) }, () => drain()));
  return results;
}

JavaScript runs one thread, so no two workers can take the same item: each takes its entry from the queue before it awaits, and Promise.all over limit runners is the whole scheduler. Pass a callback as the fourth argument — mapWithLimit(urls, 8, (url) => fetchYield(url), (done, total) => console.error(\${done}/${total}`))` — and progress costs you nothing but a line.

Strategy	60 requests of roughly 200 ms	What it costs you
Sequential `for … await`	about 12 s	one request in flight at a time
`mapWithLimit(items, 8, …)`	about 1.6 s	the same throughput over eight sockets
`Promise.all` over all 60	about 0.2 s	a burst of 60 connections and every egg in one basket

Timeouts and cancellation

A request with no deadline can hang for minutes, and after Ctrl+C the process stays alive waiting for it. Abort signals are the mechanism, and they work because fetch accepts one:

const response = await fetch(url, { signal: AbortSignal.timeout(5_000) }); // a deadline for one call

const shutdown = new AbortController();                                     // fired from a handler
process.once("SIGINT", () => shutdown.abort());

const signal = AbortSignal.any([shutdown.signal, AbortSignal.timeout(5_000)]); // Node 20.3+
const cancelled = await fetch(url, { signal });

The signal has to be passed down into every call that does I/O — fetch, node:fs/promises, database drivers — or you cancel only your own waiting while the request keeps running in the background. That is the difference between cancelling the waiter and cancelling the work, and it is why the fetchYield of Lesson 20 should take a signal: AbortSignal parameter and hand it straight to fetch. Detect cancellation by name, never by message:

export function isCancellation(error: unknown): boolean {
  if (typeof error !== "object" || error === null) return false;
  const name = (error as { name?: unknown }).name;
  return name === "AbortError" || name === "TimeoutError";
}

AbortSignal.timeout aborts with a TimeoutError, controller.abort() with an AbortError, and the reason is readable as signal.reason. A cancelled request is usually not a failure to report: during shutdown it is exactly what you asked for, so check for it before logging anything.

Retry, with backoff and jitter

A timeout, a reset connection, a 429 or a 503 often succeeds on the second attempt, which is worth a few lines and a short wait:

function delay(ms: number): Promise<void> {
  return new Promise((resolve) => setTimeout(resolve, ms));
}
export async function withRetry<T>(operation: (attempt: number) => Promise<T>, attempts = 3, baseMs = 250): Promise<T> {
  let lastError: unknown;
  for (let attempt = 1; attempt <= attempts; attempt++) {
    try {
      return await operation(attempt);
    } catch (error) {
      lastError = error;
      if (isCancellation(error) || attempt === attempts) break;
      await delay(Math.min(baseMs * 2 ** (attempt - 1), 8_000) + Math.random() * baseMs);
    }
  }
  throw lastError; // attempts is at least 1, so something was always caught
}

The wait doubles — 250 ms, 500 ms, 1 s, 2 s — and is capped so a long outage does not become a minute-long sleep. The jitter matters as much as the doubling: without it, a thousand clients that failed together retry together and knock the API over again. What must never be retried is a non-idempotent write, because a POST that creates a row will happily create two; retry only operations where a repeat is harmless, or use an idempotency key as Lesson 29 describes.

Stopping without leaving work behind

Ctrl+C sends SIGINT, and process managers and container runtimes send SIGTERM. Both should cancel in-flight work rather than leave a half-finished batch behind:

const shutdown = new AbortController();
process.once("SIGINT", () => { console.error("cancelling in-flight work"); shutdown.abort(); });
process.once("SIGTERM", () => shutdown.abort());

const signal = AbortSignal.any([shutdown.signal, AbortSignal.timeout(30_000)]);
const rows = await mapWithLimit(urls, 8, (url) => fetchYield(url, signal), report);

The abort reaches every worker through the shared signal, mapWithLimit propagates the rejection, and the top-level handler sees a cancellation and exits without an error log. The second signal in the any is a hard deadline for the shutdown itself: if a worker ignores the abort, the process still gives up after thirty seconds instead of hanging forever.

Common mistakes

Mistake	What happens	The fix
`Promise.all` for a batch whose items may fail	One rejection discards every successful result	Use `Promise.allSettled` and report the failures separately
`Promise.all` over 500 items	A burst of connections: rate limits, resets, and one failure loses everything	Run a pool of five to ten workers with `mapWithLimit`
Creating a signal but not passing it to `fetch`	The timeout cancels your waiting only; the request keeps running	Pass `{ signal }` into every I/O call down the chain
Retrying a `POST` that creates a row	Duplicate rows, because the first attempt may have succeeded	Retry only idempotent operations, or use an idempotency key
Retrying with no cap and no jitter	Hundreds of clients retry in lockstep and wait minutes	Exponential backoff, capped, plus random jitter
Treating a cancelled request as a failure	Shutdown fills the log with errors for something you requested	Check for `AbortError` and `TimeoutError` and skip logging it

Try it

Create src/concurrency/pool.ts with mapWithLimit exactly as above, exported.
Create src/index.ts with a fake batch so the measurement needs no network — 60 tasks of 200 ms each:

import { mapWithLimit } from "./concurrency/pool.js";
const ids = Array.from({ length: 60 }, (_, id) => id);
const work = (id: number): Promise<number> => new Promise((resolve) => setTimeout(() => resolve(id), 200));
async function measure(label: string, run: () => Promise<unknown>): Promise<void> {
  const started = performance.now();
  await run();
  console.log(`${label}: ${((performance.now() - started) / 1000).toFixed(2)}s`);
}
await measure("sequential", async () => { for (const id of ids) await work(id); });
await measure("limit 8", () => mapWithLimit(ids, 8, work));
await measure("unlimited", () => Promise.all(ids.map(work)));

Compile and run with cd ~/dev/ts-zero-to-hero && npx tsc && node dist/index.js.

Expected output: three lines after roughly fourteen seconds in total — sequential: 12.0xs, limit 8: 1.6xs, unlimited: 0.2xs. The absolute numbers depend on your machine; the ratios between them are the lesson. Then pass a fourth argument to mapWithLimit and watch one progress line per finished task.

Add cancellation: give the worker a signal parameter, reject with signal.reason from an abort listener, and call process.kill(process.pid, "SIGINT") from a timer after 250 ms.

Expected output: cancelling in-flight work, then the run ending without an error trace — because a cancelled batch is not a failure.

Key takeaways

Promise.all is all-or-nothing; allSettled is how you keep the results that worked and report the ones that did not.
Never fire an unbounded batch: a pool of five to ten workers turns twelve seconds into under two while keeping the API you call happy.
A timeout is a cancellation, so it needs an AbortSignal passed all the way into fetch — otherwise you cancel the waiting, not the work.
Retry transient failures only, with exponential backoff, a cap and jitter, and never retry a write that is not idempotent.
Handle SIGINT and SIGTERM by aborting a shared controller, and treat the resulting AbortError as success, not as a failure.

Check yourself

A batch of ten URLs must all succeed before you write anything to the database. Which combinator is right, and what happens to the nine results that arrived before the failure?
Why does mapWithLimit not need a lock to stop two workers taking the same item?
You set AbortSignal.timeout(5_000) but the request is still running afterwards. What did you most likely forget, and why does it matter?
When is retrying a failed request actively harmful, and what makes it safe instead?