Rate limits and quotas
How request, token and daily limits work, how to read the rate-limit headers, and what to do when you hit one.
Sieve limits how fast and how much you can send, per organisation and per model. A key can be given tighter limits than its organisation, never looser. Your prepaid balance is the last limit: a request that it cannot cover is refused.
You need: an API key in TYPESAFE_API_KEY. The console's Limits page shows the exact limits for your organisation and lets you ask for an increase.
The limits
| Limit | Unit | When you go over |
|---|---|---|
| Request rate | Requests per second, per organisation (or key) and model | 429 rate_limit_error with retry-after |
| Token rate | Input tokens per second, per organisation (or key) and model | 429 rate_limit_error with retry-after |
| Daily tokens | Input tokens per UTC day, reset at 00:00 UTC | 429 daily_token_limit_exceeded |
| Ramp-up | New organisations start with a lower daily allowance that grows with steady use | 429 acceleration_limit_exceeded |
| Requests in flight | Concurrent requests, when your organisation has a cap | 429 |
| Credit | Your prepaid balance | 429 insufficient_quota, no retry-after |
| Monthly budget | Spend per UTC calendar month, per key or for all keys, when an admin sets one (Monthly budgets) | 429 key_budget_exceeded or project_budget_exceeded, no retry-after |
A single request that can never fit (for example more input tokens than a per-second burst or the daily limit) is not a 429: it is a 400 max_tokens_exceeded. Split it. Size and token limits per request are on Limits. Default numbers are not published yet; the Limits page shows yours.
Read the headers
When a limit is set for your organisation, each response carries x-ratelimit-limit-<unit>, x-ratelimit-remaining-<unit> and x-ratelimit-reset-<unit>, where <unit> is requests or input-tokens. The reset value is a duration such as 1s or 250ms. With no limit set, the headers are absent.
import json
import os
import urllib.request
BASE_URL = os.environ.get("TYPESAFE_BASE_URL", "https://api.sieve.one")
request = urllib.request.Request(
f"{BASE_URL}/v1/systemone",
data=json.dumps({
"model": "sieve-latest",
"state": "Where is my parcel?",
"questions": {"delivery": {"type": "noul", "instructions": "Is this about a delivery?"}},
}).encode(),
method="POST",
headers={"Authorization": f"Bearer {os.environ['TYPESAFE_API_KEY']}", "Content-Type": "application/json"},
)
with urllib.request.urlopen(request, timeout=60) as response:
limits = {k: v for k, v in response.headers.items() if k.lower().startswith("x-ratelimit-")}
print(limits or "No rate-limit headers: no request or token limit applies to this key.")
print("input tokens billed:", json.load(response)["usage"]["input_tokens"])Stay inside your limits
- Ask many questions about one state in one request. The state is read once, so one request with ten questions costs far fewer input tokens than ten requests with one question each, and it counts as one request against the request rate.
- Pace yourself when
x-ratelimit-remaining-requestsorx-ratelimit-remaining-input-tokensgets close to zero, instead of waiting for a 429. - Honour
retry-afteron a 429 and retry once it has passed. See Errors and retries. - Do not retry a 429 without
retry-after. It means the daily limit is used up or the balance is too low; retrying will not help until the reset or a top-up. - Use separate keys for separate applications, so you can see each one's usage on the console Home page and revoke one without stopping the others.
Next steps
Last updated