Rate limits priced in query cost

The problem

Shopify's GraphQL Admin API meters a 1,000-point bucket refilling at 50 points/second, and each query has its own price — 11 points for one, 900 for another. Four properties each defeat a different standard tool:

1. The unit is cost, not requests. No single requests-per-second is correct for both an 11-point query and a 900-point one.
2. Overspending answers 200 OK. The failure arrives in the body as a THROTTLED entry in errors[] — never a 429. Retry policies keyed on status see success.
3. The wait is arithmetic, not a guess. Every response carries extensions.cost.throttleStatus; the correct wait is (requestedQueryCost − currentlyAvailable) / restoreRate. A backoff curve over-waits when the bucket is full and under-waits when it is empty.
4. The bucket belongs to the shop, not to you. Another app draining it moves your headroom between two of your own requests (shopify-api-js#602), so a local ledger can never be authoritative — it must be overwritten from every response.

The common solutions

What StitchAPI does

Not retry, and not throttle — measured, both miss, and the section below says exactly how. The seam that fits is a custom surface: interpret sees every response body before the engine decides anything, and the SurfaceOutcome it returns can ask for a retry after a wait you computed.

import { graphqlSurface, verdictOf } from 'stitchapi';
import type { Surface, SurfaceOutcome } from 'stitchapi';

export function shopifyCostSurface(ledger: CostLedger): Surface {
    return {
        id: 'graphql',
        buildRequest: graphqlSurface.buildRequest,
        interpret: (res, cfg): SurfaceOutcome => {
            const failure = verdictOf(res, cfg);
            if (failure) return failure;

            // EVERY response updates the budget — successes carry throttleStatus too,
            // and the server's number is authoritative because the bucket is shared.
            const cost = costOfBody(res.body);
            if (cost) ledger.record(cost);

            // The 200-with-THROTTLED, and the wait the server's own arithmetic dictates.
            if (isThrottled(res.body) && cost)
                return {
                    ok: false,
                    retry: true,
                    message: `THROTTLED — need ${cost.requestedQueryCost}`,
                    after: deficitWaitMs(cost), // ← (requested − available) / restoreRate
                };

            return (
                graphqlSurface.interpret?.(res, cfg) ?? {
                    ok: true,
                    data: res.body,
                }
            );
        },
    };
}

Pair it with an onRequest hook that pauses while the ledger says the next query is unaffordable, and the reactive half only handles what the proactive half cannot predict — another app draining the shop.

Measured: the computed wait was honored exactly (6000 ms, succeeding on attempt 2, where the built-in curve waited 100 ms and failed all three attempts). Against a neighbour emptying the bucket before every single call, 8/8 queries succeeded, absorbing 8 throttles. The whole thing is 73 lines.

StitchAPI vs the common solution

The retry loop stays the engine's. Because the cost logic lives in interpret rather than in a wrapper around the call, timeout.total still bounds the whole thing, the circuit breaker still counts failures, and every wait shows up as a retry progress event on the event stream. The obvious alternative — wrapping the adapter — was measured going blind: a call that made 2 requests and slept 6 seconds reported attempts: 1 and emitted zero retry events.

What StitchAPI does not solve here

1. retry.on cannot see the body. The predicate receives exactly one argument, the status number. A 200-with-THROTTLED is invisible to every built-in retry policy.
2. retry.on: 200 retries your successes. The status matcher runs before interpret and cannot tell the two apart — measured 3× the requests and 3× the points for one result.
3. backoff has no function form. A wait computed from the payload cannot enter through it, and casting past the type error is not a way in either: construction throws bad backoff rather than silently degrading to the default curve, as it once did (#651, fixed by #666).
4. retry.respect reads a Retry-After header that Shopify never sends.
5. throttle.rate cannot express cost. Requests-per-interval only, minimum spacing, no burst. Approximating the 1,000-point bucket took 18 s for work the bucket absorbs instantly — and with mixed costs no single spacing is correct for both.
6. throttle.delegate is status-keyed. The escape hatch the throttle docs point at for vendor-accounted quotas does not reach a body-reported one.
7. A rejected body never reaches the caller. On a surface failure the StitchError has no body, the error event has no field for one, and .inspect().raw is null. Whatever you need from the body must be captured inside interpret or hooks.onResponse.
8. interpret is synchronous. A distributed cost ledger cannot live in the seam that otherwise solves this — single process only.

See also