Submit, poll, download — the async job triangle

The problem

You ask for something slow — a Salesforce Bulk export, a report render, a transcode. The API answers 202 Accepted with a Location, and you come back later. That is the asynchronous request–reply pattern, and it is not one call but three endpoints with a loop between them:

1. POST /jobs → 202, Location: /jobs/{id}, often Retry-After
2. GET /jobs/{id} → repeatedly, until the state is terminal
3. GET <resultUrl> → the payload, often a pre-signed link that expires or is single-use

Each step brings its own difficulty. The next URL is in a response header, not the body. Terminal state is in-band — Salesforce runs InProgress → JobComplete | Failed, all at HTTP 200, so failure arrives as a successful response. The wait is the server's to set, via Retry-After, over minutes to hours. The budget spans the whole triangle, not one call. And a restart loses the job, which is still running server-side — resubmitting duplicates hours of work.

The common solutions

What StitchAPI does

There is no poll primitive. Polling is spelled as retry, where the failure is "not done yet" — a custom surface whose interpret reads the in-band state:

export function jobPollSurface(clock: Clock): Surface {
    return {
        id: 'job-poll',
        interpret: (res, cfg) => {
            // verdictOf FIRST, or a 404 comes back as a successful poll.
            const failed = verdictOf(res, cfg);
            if (failed) return failed;

            const state = stateOf(res.body);
            if (state === 'InProgress') {
                const after = retryAfterMs(res.headers['retry-after'], clock);
                return after === undefined
                    ? { ok: false, retry: true, message: 'InProgress' } // capped expo fallback
                    : { ok: false, retry: true, message: 'InProgress', after };
            }
            if (state === 'Failed')
                return {
                    ok: false,
                    message: `job failed: ${errorMessageOf(res.body)}`,
                };
            return { ok: true, data: res.body };
        },
    };
}

Three stitches then run under linked, which chains them into one trace, with a caller-owned AbortSignal as the operation deadline.

Measured: 1 submit → 5 polls at the server's own 300 s pacing → 1 download, 20 virtual minutes, Failed terminating on the first terminal body with 17 of 20 poll attempts unspent. linked produced one traceId across three spans, each parented to the last. A crash mid-poll resumed after 3 polls with 1 submit total.

The trade you have to make

This is the part worth knowing before you start. There are two constructions, and you can have one deadline over the whole triangle, or per-hop retry policies — not both.

The one-stitch form buys the config-level deadline and loses the per-hop split; linked keeps both of those and replaces the deadline with a signal you own.

StitchAPI vs the common solution

The hand-rolled while loop produces a byte-identical request sequence and pacing, in 49 lines against 110. What the extra lines buy was measured, not asserted: one start and one done per hop with the polls folded in as attempts: 3 rather than three unrelated calls, one traceId chaining job-submit → job-poll → job-download, and a per-hop retry policy. All of the semantics — the state machine, the pacing, the resume — are still yours either way.

If you don't need the trace or the per-hop policies, the while loop is the honest answer.

What StitchAPI does not solve here

1. No poll or until primitive. Nothing in the config vocabulary waits for a state.
2. retry.respect does not reach the body-driven path. With the server asking for 30 s, the measured gaps were 7 ms — the computed backoff. The engine reads Retry-After only on the status-driven path; a surface must pick the header up itself.
3. parseRetryAfter is not exported. The HTTP-date form has to be re-implemented by every surface author, and if you don't, the server's pacing is discarded silently.
4. No operation-scoped deadline field. linked takes a body and nothing else. The budget is an AbortSignal you build and thread through every input.signal; when it fires, the rejection carries your own abort reason.
5. No operation-level span. linked emits nothing of its own, so a mid-operation failure is attributable to the step, never to "the export failed".
6. Nothing persists the job id. The store is engine state — throttle, auth, cache. Resume is entirely yours.
7. Every failure arrives as StitchError, and only the deadline is typed underneath. The engine's live TimeoutError rides error.cause (the class is unexported — check cause?.constructor.name), so "deadline fired" is structurally separable. "Poll budget exhausted" and "the job failed" carry no cause, status is undefined on all three, and telling those two apart is still string matching.
8. paginate cannot do this — and not for the reason you'd guess. It does loop (the default items wraps a non-array body as one item, so the empty-page break never fires), but it cannot wait: measured gaps of 0, 0, 0, with no delay field. A paginated poll also cannot fail — Failed is aggregated as just another value.

See also