Provider routing · UltraGPT API

Provider routing

A model can be served by several endpoints (upstream routes). GET /v1/models/{model} lists them under endpoints with their label, price, measured first-token time, throughput and status. By default they are tried by priority, load-balanced, and a request fails over to the next endpoint when one is down — before your models[] fallbacks. Failing endpoints are skipped automatically for ~30s.

Steer it with the provider object: order / only / ignore take endpoint labels; sort orders by price, throughput or latency (:floor and :nitro do the same); max_price ({ prompt, completion } in USD per 1M tokens you pay) skips pricier endpoints and returns 400 no_endpoints_match if none is left; preferred_max_latency (seconds) and preferred_min_throughput (tokens/s) move slower endpoints to the back; allow_fallbacks: false uses only the first. Names that are not one of the model's labels — and require_parameters, data_collection, quantizations, zdr — go to OpenRouter for the models it serves. The x-ultragpt-endpoint response header and the generation's provider_endpoint and route fields show what was used and why.

Python
response = client.chat.completions.create(
    model="grok-4.7",
    messages=messages,
    extra_body={
        "provider": {
            "sort": "price",               # or "throughput" / "latency"
            "order": ["primary"],          # endpoint labels from GET /v1/models/grok-4.7
            "ignore": ["backup-2"],
            "max_price": {"prompt": 3, "completion": 15},  # USD per 1M tokens you pay
            "allow_fallbacks": True,
            "data_collection": "deny",     # OpenRouter-served: only providers that don't train on data
            "zdr": True,                   # OpenRouter-served: zero data retention
        },
    },
)
# Which endpoint answered: response header x-ultragpt-endpoint,
# and GET /v1/generation?id=... -> provider_endpoint, route