Scaling Node.js Backends Past the First 10,000 Users

Node.js does not fall over at scale. Services written like tutorials do. Here is what changes when real traffic arrives.

The event loop is your only thread — respect it

Everything in Node shares one event loop. One synchronous 200ms operation means every concurrent request waits 200ms.

// Blocks every request in flight
const parsed = JSON.parse(fs.readFileSync(hugeFile, "utf8"));
 
// Offload it
import { Worker } from "node:worker_threads";
const parsed = await runInWorker("./parse-report.js", hugeFile);

The usual suspects: synchronous crypto (pbkdf2Sync), large JSON.parse, regex catastrophes, and zlib sync calls. Profile with node --cpu-prof before guessing.

Your database dies before your server does

Every Node process opens its own connections. Ten serverless instances times ten pooled connections is a hundred connections — Postgres defaults to 100 total.

Queues turn spikes into schedules

Anything that does not need to happen during the request should not happen during the request:

export async function POST(request: Request) {
  const user = await createUser(await request.json());
 
  // Do not await the welcome email, the CRM sync, the analytics event
  await queue.enqueue("user.created", { userId: user.id });
 
  return Response.json({ ok: true }, { status: 201 });
}

The request does one insert and one enqueue. Everything else happens on a worker with retries, backoff, and a dead-letter queue.

Idempotency is not optional

Networks retry. Users double-click. Webhooks redeliver. Every mutation needs to be safe to run twice:

• Idempotency keys on payment and order endpoints
• INSERT ... ON CONFLICT DO NOTHING for event ingestion
• Webhook handlers that check "have I processed this delivery ID?"

Boring wins

The architecture that survives is rarely clever: a stateless Node service, a pooled Postgres, a queue for slow work, and metrics on all three. Add complexity only when a graph tells you to.