Smooth your calls with rate limiting

The CaptainDNS public API enforces two layers of rate limiting: one per source IP ahead of authentication, and one per key token bucket whose capacity matches the plan. This guide explains each layer and how to write a client that respects the limits without losing calls.

Two protection layers

Pre-auth per-IP limit

Before the Authorization header is even parsed, an in-memory limiter bounds traffic per source IP. The default is 120 requests per minute per IP, adjustable server-side on demand. This layer protects against volumetric attacks and credential stuffing.

Exceeding it returns 429 RATE_LIMITED with Retry-After: 60. No key is verified, no credit is consumed.

Per-key token bucket

Once the key is authenticated, a token bucket persisted in Postgres handles the per-key limit. Capacity is aligned to the key's plan:

Every authenticated request consumes one token. The bucket refills continuously (fractional refill), so you do not need to wait a full minute tick to retry.

An empty bucket returns 429 RATE_LIMITED. The Retry-After header contains the number of seconds until the next token is available.

Response headers

Every response includes these standard headers:

RateLimit-Policy: "default";q=60;w=60
RateLimit: limit=60, remaining=58, reset=42
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 58
X-RateLimit-Reset: 1717200000

• RateLimit-Policy: IETF draft format describing the policy. q is the capacity, w the window in seconds.
• RateLimit: IETF draft format showing current state. remaining tokens available, reset seconds until full refill.
• X-RateLimit-Limit/Remaining/Reset: same data in X-headers format for compatibility with existing clients.

On a 429 you also get:

Retry-After: 12

The client must respect this value and wait the indicated seconds before retrying.

Recommended retry strategy

The idiomatic pattern for a well-behaved client is bounded exponential backoff:

1. On every response, read X-RateLimit-Remaining. If remaining / limit is below 20 %, slow down preemptively.
2. On 429 RATE_LIMITED, read Retry-After. Wait at least that long.
3. If consecutive 429s pile up, apply a multiplier: 2x, 4x, 8x with a 60-second ceiling.
4. Add a random jitter of +/- 20 % to avoid multiple clients retrying in lockstep.
5. Cap the total number of retries (for example 5). Beyond that, log as an error and surface in your observability.

Pseudo-code:

delay = retryAfter + random(-0.2, 0.2) * retryAfter
for attempt in 1..5:
    response = send(request)
    if response.status != 429:
        return response
    retryAfter = response.header["Retry-After"] or delay * attempt
    sleep(min(retryAfter, 60))
raise RateLimitExceeded

Batch pipeline strategies

Integrations that process large volumes in batches (for example 10,000 domains each night) benefit from different strategies:

• Client-side token bucket: mirror the server bucket locally and only fire a request when a token is available. This avoids 429s and jitter altogether.
• Parallelize based on capacity: a Pro plan has 500 tokens/min, roughly 8 req/s. Running 8 workers in parallel is more efficient than a single worker doing 500 iterations per minute with sleeps.
• Split expensive endpoints: if you run 500 page-crawl-check (10 credits) and 500 dmarc/lookup (1 credit), split them into two queues to smooth credit consumption and respect the per-key rate limit more easily.
• Maintenance windows: some batches can run off-peak to avoid contending with realtime traffic for the envelope.

What counts as a request

Every authenticated HTTP request consumes one token, even if it fails with 400 or 500. The rate limit protects the backend; input validation is applied after token consumption.

Exceptions:

• 429 RATE_LIMITED: obviously consumes no token, because the bucket is empty.
• 401 INVALID_API_KEY: does not touch the per-key bucket (the key is unknown at that stage). However, the per-IP rate limit does count.
• 403 IP_NOT_ALLOWED: consumes neither token nor credit, because the check runs before consumption.

Increase your capacity

Token bucket capacity is a plan property, not a key parameter. To increase capacity:

1. Upgrade to a higher plan from the /account/billing dashboard.
2. The change is immediate: the key inherits the new ceiling on its next request.

Enterprise customers can negotiate a custom ceiling (up to 5,000 req/min on request) with their CaptainDNS account manager.

Diagnose a 429

An unexpected 429 can stem from several causes:

• Pre-auth bucket: you are exceeding 120 req/min from a single IP. Verify that multiple services are not sharing the same egress IP (NAT, CGNAT).
• Per-key token bucket: your plan does not offer the required capacity. Upgrade or reduce concurrency.
• Spike burst: an unsmoothed batch fired 200 requests in a second. Add jitter and client-side smoothing.
• Clock drift: in rare cases where your server clock is out of NTP sync, the Retry-After header may look excessive. Sync your clock.

If the issue persists even with strict Retry-After compliance, open a support ticket with the X-Request-Id from a representative 429.

Related CaptainDNS tools

• The DNS lookup lets you manually verify a record without spending credits.
• DMARC monitoring aggregates DMARC reports without going through the public API.

Next step: read idempotency to save credits and tokens on retries, or the error codes to tell 429s apart from other rejections.