Skip to main content

Rate limiting

Rate limiting caps the requests, tokens and credits an API key can use, so a runaway agent or a misconfigured app cannot drain the workspace.

Owners and Admins set limits with an admin key. Member keys run models and agents, read their own usage, and cannot change or list keys.

How limits work​

Limits are checked on every request to a limited model. A request over a limit is rejected at once, not queued.

A limit on a model applies on every path that calls it: the SDK, the REST API, and the model's use as the LLM of an agent or one of its subagents. All limits on a key apply together, and the first one reached blocks the request.

LimitWhat it caps
request_per_minute, request_per_dayCalls per minute and per day.
token_per_minute, token_per_dayTokens per minute and per day. token_type picks input, output or total tokens.
budgetCredits the key can spend.

Set limits for one model (asset_limits) or across every model on the key (global_limits). When both exist, the stricter one applies.

Check your own usage​

Any key can read its own usage and limits.

from aixplain import Aixplain

aix = Aixplain(api_key="YOUR_MEMBER_API_KEY")

rows = aix.APIKey.get_usage_limits()
for row in rows:
    print(row)
Show output

The row with model=None is the key's global scope. None counts and limits mean no global limit is set.

Pass a model ID to see one model. An empty list means the key has no limit on that model.

MODEL_ID = "669a63646eb56306647e1091"  # GPT-4o mini; use the model ID, not the path

rows = aix.APIKey.get_usage_limits(model=MODEL_ID)
print(rows)
Show output

Create a key with limits​

Create admin keys in the aixplain app at Settings > API keys. An admin key cannot run models or agents.

from datetime import datetime
from aixplain import Aixplain
from aixplain.v2.api_key import APIKeyLimits

aix_admin = Aixplain(api_key="YOUR_ADMIN_API_KEY")

new_key = aix_admin.APIKey(
    name="Support bot",
    asset_limits=[
        APIKeyLimits(
            model="openai/gpt-4o-mini/openai",
            request_per_minute=10,
            request_per_day=500,
            token_per_minute=20000,
            token_per_day=200000,
        )
    ],
    global_limits=APIKeyLimits(request_per_minute=60, request_per_day=5000),
    budget=50,
    expires_at=datetime(2027, 1, 1),
).save()
ParameterTypeDefaultDescription
namestrrequiredLabel for the key.
asset_limitslist[APIKeyLimits][]Limits for individual models. model takes an asset path or ID.
global_limitsAPIKeyLimitsNoneLimits across all models on the key.
budgetintNoneCredits the key can spend. Raise it to top up the key.
expires_atdatetimeNoneExpiry date. Omit it for a key that does not expire.

List and update keys​

An admin key can list every key in the workspace, or find one by its secret with get_by_access_key().

page = aix_admin.APIKey.search()
for key in page.results:
    print(key.name, "admin" if key.is_admin else "member")

Change limits on a key object, then call save(). New limits apply from the next minute or day window.

from aixplain.v2.api_key import APIKeyLimits, TokenType

key = aix_admin.APIKey.get_by_access_key("MEMBER_API_KEY_TO_UPDATE")
key.budget = 100
key.asset_limits = [
    APIKeyLimits(
        model="openai/gpt-4o-mini/openai",
        token_per_minute=50000,
        token_per_day=500000,
        token_type=TokenType.OUTPUT,
    )
]
key.save()

TokenType.OUTPUT counts only output tokens. Omit token_type to count input and output tokens. To read another key's usage, call key.get_usage() on its key object; to delete it, call key.delete(), which revokes it at once.

Alert before a limit​

Poll usage and alert when a key passes a share of its limit.

def usage_alerts(rows, threshold=0.8):
    alerts = []
    for row in rows:
        label = row.model or "global"
        if row.daily_request_limit and row.daily_request_count / row.daily_request_limit >= threshold:
            alerts.append(f"{label}: {row.daily_request_count}/{row.daily_request_limit} requests today")
        if row.daily_token_limit and row.daily_token_count / row.daily_token_limit >= threshold:
            alerts.append(f"{label}: {row.daily_token_count}/{row.daily_token_limit} tokens today")
    return alerts

alerts = usage_alerts(aix.APIKey.get_usage_limits())
print(alerts or "Usage within threshold")
Show output

Handle rate-limit errors​

A rejected call raises APIError. HTTP 429 and 497 mean a per-minute limit or the key's budget was reached; 498 means a daily or monthly limit was reached.

import time
from aixplain.v2.exceptions import APIError

model = aix.Model.get("openai/gpt-4o-mini/openai")

def run_with_backoff(text, retries=3):
    for attempt in range(retries):
        try:
            return model.run(text=text)
        except APIError as exc:
            if exc.status_code not in (429, 497) or attempt == retries - 1:
                raise
            time.sleep(60)

Retry per-minute limits after a pause. Do not retry 498: the limit resets the next day or month, or when an Admin raises it.

Rate limits and credits​

Rate limitsCredits
Set byOwners and Admins, with an admin keyOwners and Admins, by buying credits
ScopeOne key, per model or across modelsThe whole workspace
BlocksRequests over the limitEvery paid request when the balance reaches 0
ResetsEvery minute or dayWhen credits are added

Use rate limits to control spend per app or environment, and credits as the hard cap for the workspace. To cap a single agent run instead, use Budgets.

Next steps​