Errors, idempotency, pagination, and rate limits

The four mechanics every unattended integration needs to get right — and the specific ways each one is usually got wrong.

Everything here is boring until the afternoon it is not. These are the four behaviours that decide whether a retry duplicates a claim, whether a backfill silently stops at page one, and whether a burst of traffic degrades or falls over.

Errors

Every error is an RFC 7807 problem document, served as application/problem+json.

{
  "type": "https://api.warrantyplatform.com/errors/validation-error",
  "title": "Validation Error",
  "status": 422,
  "detail": "One or more fields failed validation.",
  "errors": [
    { "field": "customer_email", "message": "Invalid email address" }
  ]
}

Branch on type. It is stable across versions and is the only part that is. title and detail are written for humans and may be reworded at any time — code that matches on them will break on a copy edit.

The catalogue, and what each one actually means for your code:

/unauthorized 401 — key missing, malformed, or revoked. Not retryable.
/forbidden 403 — valid key, wrong scope. Not retryable; fix the key.
/tier-insufficient 403 — right scope, plan does not include it. Not retryable.
/not-found 404 — gone, or another tenant's. Not retryable.
/conflict 409 — optimistic lock. Retryable after re-reading the current version.
/validation-error 422 — see the errors array for the field. Not retryable unchanged.
/invalid-transition 422 — illegal claim state move. Re-read the claim's status.
/idempotency-conflict 422 — key reused with a different body. See below.
/rate-limited 429 — retryable, honouring Retry-After.
/internal-error 500 — retryable with backoff.

The useful split: 409, 429 and 5xx are worth retrying. 401, 403, 404 and the 422 family will return exactly the same answer forever, and retrying them just burns your rate limit.

Idempotency

Every mutating request accepts an Idempotency-Key header. Send one on anything that creates or transitions a record.

curl -s -X POST https://api.warrantini.com/api/v1/claims \
  -H "Authorization: Bearer $WARRANTINI_API_KEY" \
  -H "Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000" \
  -H "Content-Type: application/json" \
  -d '{ "registration_id": "reg_01hxyz", "description": "Impeller seized after 4 months." }'

Same key, same body → the cached response, without re-executing.
Same key, different body → 422 .../idempotency-conflict.
Keys expire after 24 hours. Any string up to 255 characters; UUIDv4 is the sensible default.

Generate the key once, before the first attempt, and reuse it for every retry of that same logical operation. A key generated inside the retry loop is a new key each time — which is exactly the duplicate-claim bug the header exists to prevent.

Ingestion is different, and better

Order and product ingestion do not need the header for safety: they are idempotent on external_id by nature, and re-posting upserts. The header still works there for exact response replay, but the upsert is the real guarantee — which is why re-running a backfill is safe.

Pagination

List endpoints are cursor-based. There is no offset pagination and no page numbers.

{
  "data": [],
  "pagination": { "cursor": "eyJpZCI6Im9yZF8wMXh5eiJ9", "has_more": true, "limit": 50 }
}

limit defaults to 50, maximum 100.
The cursor is opaque — do not decode it, construct it, or assume it encodes an ID.
It is stable under concurrent writes: it marks a keyset position, not a row offset, so records created mid-pagination will not shift the page under you or cause skips.
It expires after 24 hours of inactivity — a paused job cannot resume from a stale cursor the next morning.
cursor: null means this was the last page.

Loop on cursor !== null, not on has_more, and never on data.length === limit. A final page that happens to be exactly full is the classic off-by-one that silently truncates a backfill.

paginate.js

async function* allOrders(key) {
  let cursor = null;
  do {
    const url = new URL("https://api.warrantini.com/api/v1/orders");
    url.searchParams.set("limit", "100");
    if (cursor) url.searchParams.set("cursor", cursor);

    const res = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
    if (!res.ok) throw new Error(`${res.status} ${(await res.json()).type}`);

    const page = await res.json();
    yield* page.data;
    cursor = page.pagination.cursor;
  } while (cursor);
}

Results are newest-first by created_at, with id breaking ties. That ordering is not configurable today.

Rate limits

Limits are per key, per minute, and follow the merchant's plan.

Starter — 60/min
Growth — 300/min
Scale — 1,000/min

Every rate-limited response tells you what to do:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 300
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1719878400

Honour `Retry-After`. It is a server instruction, not a suggestion, and it is more accurate than any backoff curve you would invent. Add jitter on top if many workers share one key, or they will all wake at the same instant and reproduce the burst.

X-RateLimit-Remaining is on every response, not only the failures — so a long-running job can watch its own headroom and slow down before it is throttled rather than after.

Putting it together

A retry policy that covers all four, in one rule:

Generate an Idempotency-Key before the first attempt and hold it for every retry.
On 429, wait Retry-After seconds, plus jitter.
On 5xx or a network error, back off exponentially — 1s, 2s, 4s, 8s — and cap the attempts.
On 409, re-read the record and retry once with the current version.
On 4xx other than 409/429, stop. Log the type and the errors array; it will not resolve itself.

And because ingestion upserts on external_id, the worst case for an over-eager retry there is a wasted call — not a duplicate record.