Batch writes that fail one item at a time

The problem

You write in bulk — DynamoDB BatchWriteItem, Elasticsearch _bulk, SQS SendMessageBatch, Salesforce sObject Collections. One request carries 25 or 1,000 items, the API answers HTTP 200, and inside the body it says some of them didn't land.

The retry unit is smaller than the request. Every HTTP client retries by replaying the identical request, which here re-applies the writes that already succeeded. The correct behaviour is to rewrite the body to the failed subset, resend, wait longer each round, and report whatever never landed.

Three things make it worse than it sounds: backoff is mandatory (AWS is explicit — the cause is capacity, so an immediate resend throttles again), failures are not uniform (a 429 item should be resent, a 400 mapping error must not be), and the loop needs to hand back the residue — the caller needs the items, not just an error.

The common solutions

What StitchAPI does

The seam that works is Surface.interpret — which reads the 200 body and asks for another attempt with a wait that grows — paired with hooks.onRequest, the only place in the library that can change a request between attempts:

const kind: Surface = {
    id: 'batch-residue',
    interpret: (res, cfg): SurfaceOutcome => {
        // A 500 is a transport failure before it is a batch envelope — and must still
        // open the circuit. Let the declarative verdict compose first.
        const failed = verdictOf(res, cfg);
        if (failed) return failed;

        ledger.rounds += 1;
        ledger.landed.push(...landedOf(res.body));
        ledger.terminal.push(...terminalOf(res.body)); // 400s — never resent
        ledger.residue = residueOf(res.body); // 429s / UnprocessedItems

        if (ledger.residue.length === 0) return { ok: true, data: ledger };
        if (ledger.rounds >= rounds) {
            // Out of rounds. Resolve SUCCESSFULLY with the residue in the payload —
            // an error would throw the landed items away, and dropping it is the Logstash bug.
            ledger.gaveUp = true;
            return { ok: true, data: ledger };
        }
        // `message` is required on the retry arm — it becomes the `retry`
        // event's detail, so the trace says WHY the round is being re-run.
        return {
            ok: false,
            retry: true,
            message: `${ledger.residue.length} unprocessed after round ${ledger.rounds}`,
            after: backoff(ledger.rounds),
        };
    },
};

const hooks = {
    onRequest: (ctx) => {
        if (!ctx.req || ctx.attempt === 1) return;
        // Assign, never mutate: each attempt's request is a shallow clone of one baseReq,
        // so an in-place edit of `body` rewrites the caller's own array too.
        ctx.req.body = bodyOf(ledger.residue);
    },
};

Measured against a capacity-limited table: 4 rounds at t = 0, 1000, 3000, 7000 — abcdef → cdef → def → f — zero duplicate writes, every item landed, and the wait is the engine's own sleep rather than a hidden one. It survives three consecutive zero-progress rounds, which is precisely the case paginate drops.

StitchAPI vs the common solution

It is more code, not less — 50 lines against 28 for the hand-rolled while loop. The trade is what the hand-rolled loop gives up, and this was measured rather than assumed:

If you don't need any of that, the while loop is honestly the smaller answer. Reach for this when the batch call has to behave like one call to everything else in your system.

What StitchAPI does not solve here

1. There is no batch-residue concept. Nothing in the config vocabulary expresses "the retry unit is smaller than the request". Everything above is assembled.
2. retry is blind to per-item failure, and harmful when forced. The status is 200, and retry.on receives only the status. retry.on: 200 typechecks, fires — and made 10 duplicate writes on a batch that never failed at all.
3. paginate has no wait and no field to declare one. Six rounds fired at t=0. throttle paces them, but as one fixed ratio with no curve — and it paces every other call through that stitch (pool: 'host' pushes unrelated reads too).
4. A growing backoff is user code the engine can't see. A sleep in onRequest works, but 2.5 s of real waiting produced 0 throttled events and no waited in the run report.
5. SurfaceOutcome cannot see the request or the attempt number, so a surface can neither rewrite what it retries nor know it is on its last round. The count has to be kept by hand.
6. The engine owns no residue channel. Not the result, the error, .inspect(), .report(), the event stream, or a trace sink. If you don't capture it yourself, it is gone.
7. The ledger is per-stitch, not per-call. Two concurrent calls through one such stitch corrupt each other — measured: both callers told everything landed, two rows written by nobody. Build one stitch per in-flight batch, or add your own scoping.

See also