429 Too Many Requests API Errors: Retry, Backoff and Prevention

November 9, 2026

Every API you call at scale will send you a 429 eventually. Handling it well is a few small habits: read the response, wait the right amount, retry a bounded number of times, and slow down before the wall next time. This guide is generic and works for any HTTP API. For platform-specific numbers, see the rate limit posts for YouTube, X, TikTok, Facebook and Spotify.

What 429 Too Many Requests means

RFC 6585 defines 429 as "the user has sent too many requests in a given amount of time". Three details matter:

• The server may send Retry-After, so you cannot count on it.
• The RFC does not say how the server counts. It could be per key, per user, per IP, per endpoint or across a whole cluster.
• A 429 must not be stored by a cache, so a CDN will not serve you a stale 429.

Read the response before you retry

The IETF's RateLimit header fields draft (revision 11, May 2026) is the likely future: RateLimit-Policy describes quotas with q (quota) and w (window in seconds), and RateLimit reports r (remaining) and t (seconds until reset).

Rate limit or quota? Not every 429 is retryable

Read the body. Quota errors need a different response: park the job until the reset and alert someone.

Exponential backoff with jitter

When there is no Retry-After, wait longer after each failure and add randomness so many clients do not retry at the same instant. AWS's analysis of backoff and jitter found "full jitter" does the least total work:

sleep = random_between(0, min(cap, base x 2 ** attempt))

Python: retry that respects Retry-After

import email.utils, random, time, requests

RETRYABLE = {429, 500, 502, 503, 504}

def retry_after_seconds(value):
    if not value:
        return None
    if value.strip().isdigit():
        return int(value)
    when = email.utils.parsedate_to_datetime(value)   # HTTP-date form
    return max(0, when.timestamp() - time.time())

def call(method, url, max_attempts=6, base=1.0, cap=60.0, **kwargs):
    for attempt in range(max_attempts):
        r = requests.request(method, url, timeout=30, **kwargs)
        if r.status_code not in RETRYABLE:
            return r
        wait = retry_after_seconds(r.headers.get("Retry-After"))
        if wait is None:
            wait = random.uniform(0, min(cap, base * 2 ** attempt))
        if wait > 300:
            break                     # a long wait means quota: stop and report
        time.sleep(wait)
    r.raise_for_status()
    return r

JavaScript: the same in Node or the browser

const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

function retryAfterMs(value) {
  if (!value) return null;
  if (/^\d+$/.test(value.trim())) return Number(value) * 1000;
  const when = Date.parse(value);
  return Number.isNaN(when) ? null : Math.max(0, when - Date.now());
}

async function call(url, options = {}, maxAttempts = 6, baseMs = 1000, capMs = 60000) {
  for (let attempt = 0; attempt < maxAttempts; attempt++) {
    const res = await fetch(url, options);
    if (!RETRYABLE.has(res.status)) return res;
    let wait = retryAfterMs(res.headers.get("retry-after"));
    if (wait === null) wait = Math.random() * Math.min(capMs, baseMs * 2 ** attempt);
    if (wait > 300000) throw new Error("long Retry-After: treat as quota");
    await sleep(wait);
  }
  throw new Error("gave up after " + maxAttempts + " attempts: " + url);
}

In the browser, the server must list Retry-After in Access-Control-Expose-Headers or res.headers.get returns null, and the backoff path takes over.

Prevent 429s: a client-side token bucket

Retries recover from 429s. A token bucket stops most of them. It holds up to capacity tokens, refills at rate per second, and each request takes one. Set the rate a little under the provider's limit.

import threading, time

class TokenBucket:
    def __init__(self, rate, capacity):
        self.rate, self.capacity = rate, capacity
        self.tokens, self.last = capacity, time.monotonic()
        self.lock = threading.Lock()

    def take(self):
        while True:
            with self.lock:
                now = time.monotonic()
                self.tokens = min(self.capacity, self.tokens + (now - self.last) * self.rate)
                self.last = now
                if self.tokens >= 1:
                    self.tokens -= 1
                    return
                need = (1 - self.tokens) / self.rate
            time.sleep(need)

bucket = TokenBucket(rate=4.5, capacity=10)   # about 270 a minute, under a 300 limit
for url in urls:
    bucket.take()
    call("GET", url)

Run one bucket per limit you know about: per key, per endpoint, per user token. If several workers share a key, keep the bucket in Redis or another shared store, or each worker will think it has the full budget.

Queue work with a concurrency cap

The most common cause of 429s is firing hundreds of requests at once with Promise.all. Cap how many are in flight:

async function mapLimit(items, limit, fn) {
  const results = new Array(items.length);
  let next = 0;
  async function worker() {
    while (next < items.length) {
      const i = next++;
      results[i] = await fn(items[i]);
    }
  }
  await Promise.all(Array.from({ length: limit }, worker));
  return results;
}

const pages = await mapLimit(urls, 5, (u) => call(u).then((r) => r.json()));

Cache and deduplicate

• Cache responses that change slowly, such as profiles and metadata, for hours. Every cache hit is a request you do not make.
• Collapse duplicate in-flight requests: if two jobs ask for the same URL at once, send one request and share the result.
• Use batch endpoints where they exist: many IDs per call.
• Prefer webhooks or streams to polling for anything real time.

More on this in API caching and scraping costs and common scraping mistakes.

Make retried writes idempotent

Retrying a GET is safe. Retrying a POST can create a duplicate post, order or charge if the first attempt actually succeeded. RFC 9110 defines PUT, DELETE and safe methods as idempotent; POST is not. For POSTs, send a unique key with each logical operation and reuse it on every retry. The IETF Idempotency-Key header draft standardizes this, and many payment and messaging APIs already accept a header like it. If an API supports neither, check whether the write happened before you retry it.

Checklist

• Read Retry-After in both forms, seconds and HTTP date.
• Fall back to exponential backoff with full jitter, capped.
• Stop after a fixed number of attempts, and treat very long waits as quota.
• Check the body for quota reasons before retrying.
• Pace with a token bucket set just under the published limit.
• Cap concurrency; never fire unbounded parallel requests.
• Cache, deduplicate and batch.
• Use idempotency keys on retried writes.
• Log every 429 with the endpoint and headers, so you can see which limit you hit.

Doing it with ScrapeIO

ScrapeIO allows 300 requests a minute per API key, with no daily or monthly cap beyond the credits you hold. A 429 from ScrapeIO costs nothing, and neither do failed calls, private or not-found answers, or cached repeats. Over the limit, ScrapeIO returns HTTP 429 with a JSON body that names the limit; the retry code above handles it, and a bucket at about 4.5 requests a second per key keeps you clear. One key covers 33 platforms, so instead of tuning a different limiter for each platform's rules, you tune one. Endpoints such as the tiktok profile API, youtube channel API and twitter search API all share the same envelope and limit. If you need more than 300 a minute, email us.

FAQ

How long does a 429 error last?

As long as the server's window: seconds for short rate limits, until a daily reset for quotas. Retry-After or a reset header tells you when one is given.

Is a 429 my fault or the server's?

It is a client error class: you sent more than the server allows. Sometimes servers also send 429 to shed load, which clears on its own.

Should I retry a POST after a 429?

Usually yes, because a 429 normally means the server did not process the request. Send an idempotency key anyway, in case it did.

What is the difference between 429 and 503?

429 means you in particular are sending too much. 503 means the service is unavailable to everyone. Both may carry Retry-After.

Can I avoid 429s by rotating API keys or IP addresses?

Don't. Spreading one workload over many keys or addresses to get around a limit usually breaks the provider's terms and gets accounts suspended. Ask the provider for a higher limit, or use a service whose limit fits the job.

Sources

Checked on 9 November 2026 from each vendor's own pages. Prices and limits change; follow the links before you decide.

• https://www.rfc-editor.org/rfc/rfc6585
• https://www.rfc-editor.org/rfc/rfc9110
• https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/
• https://aws.amazon.com/blogs/architecture/exponential-backoff-and-jitter/
• https://datatracker.ietf.org/doc/draft-ietf-httpapi-idempotency-key-header/
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429