Provider routing · UltraGPT API
Provider routing
A model can be served by several endpoints (upstream routes). GET /v1/models/{model} lists them under endpoints with their label, price, measured first-token time, throughput and status. By default they are tried by priority, load-balanced, and a request fails over to the next endpoint when one is down — before your models[] fallbacks. Failing endpoints are skipped automatically for ~30s.
Steer it with the provider object: order / only / ignore take endpoint labels; sort orders by price, throughput or latency (:floor and :nitro do the same); max_price ({ prompt, completion } in USD per 1M tokens you pay) skips pricier endpoints and returns 400 no_endpoints_match if none is left; preferred_max_latency (seconds) and preferred_min_throughput (tokens/s) move slower endpoints to the back; allow_fallbacks: false uses only the first. Names that are not one of the model's labels — and require_parameters, data_collection, quantizations, zdr — go to OpenRouter for the models it serves. The x-ultragpt-endpoint response header and the generation's provider_endpoint and route fields show what was used and why.
response = client.chat.completions.create(
model="grok-4.7",
messages=messages,
extra_body={
"provider": {
"sort": "price", # or "throughput" / "latency"
"order": ["primary"], # endpoint labels from GET /v1/models/grok-4.7
"ignore": ["backup-2"],
"max_price": {"prompt": 3, "completion": 15}, # USD per 1M tokens you pay
"allow_fallbacks": True,
"data_collection": "deny", # OpenRouter-served: only providers that don't train on data
"zdr": True, # OpenRouter-served: zero data retention
},
},
)
# Which endpoint answered: response header x-ultragpt-endpoint,
# and GET /v1/generation?id=... -> provider_endpoint, route