Distributed Throttle and Shared Sessions Across Instances

You set a throttle on a stitch — say two requests a second to a metered upstream, a proactive limiter that keeps you under the cap before the 429s arrive — and on your laptop it behaves exactly as declared. Then you deploy three instances behind a load balancer, or your handler runs as a serverless function that scales to a dozen concurrent isolates, and the upstream starts handing you 429s you can't explain. The math is the problem: three processes each enforcing two-per-second is six-per-second at the wall. Your real upstream rate is N times your cap, where N is however many copies of your code happen to be running right now.

The throttle isn't broken. It's doing precisely what it was told — for one process. The default state lives in memory, and memory isn't shared across instances.

What actually lives in the store

A stitch keeps a small amount of state between calls. By default it lives in an in-process store — a plain object in the heap of the one process that made the call. Two pieces of that state are the ones that go wrong when you scale out:

• Throttle counters. The rate limiter counts requests inside a time window and the number in flight, then makes the next caller wait if it would breach the cap. Those counters are the whole mechanism. If every process keeps its own, every process limits itself in isolation and the aggregate sails past the cap.
• Auth sessions. A cookieSession logs in once, captures the cookie, replays it, and re-logs-in when the wall returns. That captured cookie — and any token an oauth2 strategy caches — also lives in the per-process store. Across instances each process runs its own login: three instances, three logins against an upstream that may itself rate-limit logins, and a re-auth on one worker doesn't help the other two.

In-process is the right default: for a single long-lived process it's correct, zero-latency, and needs nothing to operate. It only becomes wrong the moment a second copy of your code shares the same upstream budget or account — and the fix isn't a different limiter, it's a different place to keep the counters.

The store is a seam

StitchAPI puts that per-process state behind one seam: a StitchStore, a small interface a seam reads from instead of the heap. Core ships no backend of its own — the default is in-process — but because the throttle and the session machinery already talk to the store through that interface, moving them off-box is a swap, not a rewrite. Attach a driver to the seam's store field and the call sites don't change at all.

The three first-party drivers are thin peer-dependency packages:

• @stitchapi/redis — for servers (and the edge via Upstash's HTTP Redis)
• @stitchapi/cloudflare-kv — for Cloudflare Workers and Pages
• @stitchapi/deno-kv — for Deno Deploy, Node, or Bun

import { fromIoredis, redisStore } from '@stitchapi/redis';
import Redis from 'ioredis';
import { seam } from 'stitchapi';

const api = seam({
    baseUrl: 'https://api.example.com',
    throttle: { rate: '2/s', concurrency: 4, pool: 'host' },
    // The one line that makes it fleet-wide:
    store: redisStore(fromIoredis(new Redis(process.env.REDIS_URL))),
});

The throttle declaration is unchanged from the single-process version — this article assumes you already know what rate and concurrency mean (if not, the throttle guide covers the semantics). The only new thing is store. With it attached, the counters increment in Redis instead of the local heap, so every instance pointed at the same Redis shares one budget. Two-per-second stays two-per-second whether one process is running or twelve.

Sessions ride along for free. Give two stitches the same key plus a shared store and they share one login — capture the cookie once, on whichever worker logs in first, and every other worker replays it instead of re-authenticating:

import { stitch } from 'stitchapi';
import { cookieSession, env } from 'stitchapi/auth';

const me = stitch({
    baseUrl: 'https://api.example.com',
    path: '/me',
    auth: cookieSession({
        login,
        cookie: '*',
        key: 'acme-session', // shared identity for the session
        loginInput: () => ({
            body: { user: env('USER')(), pass: env('PASS')() },
        }),
    }),
    store: redisStore(fromIoredis(client)), // same store across instances
});

Which driver fits which deployment

The drivers aren't interchangeable, and the line that separates them is one operation: atomic increment. A distributed throttle needs the store to increment a counter exactly once under concurrency — if two workers both read 5, both write 6, you've undercounted and the cap leaks. Shared cache and sessions don't need that; they only need get and set.

That single requirement decides the fit:

For a fleet of servers, reach for Redis: it has a real atomic INCR, so the counters increment exactly once and the limiter paces evenly across the fleet. Deno KV gives the same guarantee through a compare-and-set loop, and on Deno Deploy the same handle is replicated globally — a natural fit at the edge.

Workers KV is the honest exception. It's last-write-wins get/put with no atomic increment, so a throttle counter on it would undercount under concurrency and silently break the cap. Rather than pretend, cloudflareKvStore(...).increment(...) throws a documented error. KV stays the right backend for the read-heavy halves — cache and shared sessions — which is usually exactly what an edge handler needs. For a distributed throttle at the edge, pair KV with a Cloudflare Durable Object (the single-writer counter an atomic increment requires) or run the Redis driver over Upstash's HTTP API.

Start in-memory — reach for a shared store when you scale out

The in-process default is the floor you start from. One process, one upstream account, one rate budget: in-memory throttle, no store, nothing to operate.

Reach for a shared store the moment any of these are true:

• You run more than one instance. Two workers with independent in-memory limiters each enforce the full cap — the upstream sees double. Attach a store and both workers share one counter.
• You autoscale. Ephemeral instances spin up with no memory of what the others have already sent. The store is the one line that turns N independent limiters back into one.
• The upstream enforces per-account, not per-IP. One account, many servers — the limit applies across all of them, so the counter has to live somewhere they all reach.

A few trade-offs to account for when you cross that line: the store adds a network hop on every throttle check (latency on the hot path), and under contention the cap is "close to N/s across the fleet" rather than exact — keep a little headroom below the upstream's real ceiling. Workers KV is looser still (eventually-consistent reads, 60-second TTL floor), which is why increment throws there. And namespace staging vs. production with keyPrefix — one store, no collision.

It's a config change, not a rewrite. The call sites don't move.

Try it

npm install stitchapi@rc

Then attach a driver and your throttle and sessions go fleet-wide without touching a call site:

• Distributed stores — the Redis, Workers KV, and Deno KV drivers, with the atomic-increment trade-off in full.
• Pluggable store — the StitchStore interface the drivers implement.
• Distributed throttle and Shared sessions — the exact pacing and session-sharing semantics.

Point every instance at the same store, and the cap you wrote is the cap the upstream sees.