Concepts / Tools
Tool calling & structured output
One tool schema and one response format across every provider — enforced, not approximated.
Tool calling
Declare tools with JSON Schema in the OpenAI tools shape. The model returns message.tool_calls; you execute them and append role: "tool" results. Parallel calls arrive in one tool_calls list and are safe to run concurrently.
res = client.chat.completions.create(
model="anthropic/claude-sonnet-4",
messages=[{"role": "user", "content": "What is the p95 for checkout-agent today?"}],
tools=[{"type": "function", "function": {
"name": "query_metrics",
"description": "Read a metric for a project and window",
"parameters": {"type": "object", "properties": {
"project": {"type": "string"}, "metric": {"enum": ["p50", "p95", "error_rate"]},
"window": {"type": "string"}}, "required": ["project", "metric"]}}}],
tool_choice="auto",
)
msg = res.choices[0].message
for call in msg.tool_calls or []:
result = run(call.function.name, json.loads(call.function.arguments)) # your code
followup = client.chat.completions.create(model=res.model, tools=tools, messages=[
*messages, msg, {"role": "tool", "tool_call_id": call.id, "content": result}])Structured output
Pass response_format with a JSON Schema and it is forwarded to an endpoint that declares support for it; endpoints that do not are eliminated from the route. If no eligible endpoint supports it, the default compat_mode: "permissive" omits the parameter and says so in warnings[] (code: "parameter_omitted"); compat_mode: "strict" returns 422 capability_unsupported naming the parameter before any provider is contacted.
res = client.chat.completions.create(
model="openai/gpt-4o",
messages=[{"role": "user", "content": ticket_text}],
response_format={"type": "json_schema", "json_schema": {
"name": "triage", "strict": True,
"schema": {"type": "object", "properties": {
"severity": {"enum": ["sev1", "sev2", "sev3"]},
"component": {"type": "string"},
"summary": {"type": "string", "maxLength": 240}},
"required": ["severity", "component", "summary"], "additionalProperties": False}}},
extra_body={"compat_mode": "strict"}, # 422 rather than a silently omitted schema
)
# in permissive mode, check res.model_extra.get("warnings") for parameter_omitted
triage = json.loads(res.choices[0].message.content) # constrained by the provider's decodingCapability matrix
Verified, not advertised Capabilities are facts per endpoint, operation and parameter (GET /api/v1/catalog/endpoints/{id}/capabilities), not provider marketing. When endpoints of the same model differ, the request is routed only to endpoints that pass.