Concepts / Tools

# Tool calling & structured output

One tool schema and one response format across every provider — enforced, not approximated.

## Tool calling

Declare tools with JSON Schema in the OpenAI `tools` shape. The model returns `message.tool_calls`; you execute them and append `role: "tool"` results. Parallel calls arrive in one `tool_calls` list and are safe to run concurrently.

```
res = client.chat.completions.create(
    model="anthropic/claude-sonnet-4",
    messages=[{"role": "user", "content": "What is the p95 for checkout-agent today?"}],
    tools=[{"type": "function", "function": {
        "name": "query_metrics",
        "description": "Read a metric for a project and window",
        "parameters": {"type": "object", "properties": {
            "project": {"type": "string"}, "metric": {"enum": ["p50", "p95", "error_rate"]},
            "window": {"type": "string"}}, "required": ["project", "metric"]}}}],
    tool_choice="auto",
)
msg = res.choices[0].message
for call in msg.tool_calls or []:
    result = run(call.function.name, json.loads(call.function.arguments))   # your code
    followup = client.chat.completions.create(model=res.model, tools=tools, messages=[
        *messages, msg, {"role": "tool", "tool_call_id": call.id, "content": result}])
```

## Structured output

Pass `response_format` with a JSON Schema and it is forwarded to an endpoint that declares support for it; endpoints that do not are eliminated from the route. If no eligible endpoint supports it, the default `compat_mode: "permissive"` omits the parameter and says so in `warnings[]` (`code: "parameter_omitted"`); `compat_mode: "strict"` returns 422 `capability_unsupported` naming the parameter before any provider is contacted.

```
res = client.chat.completions.create(
    model="openai/gpt-4o",
    messages=[{"role": "user", "content": ticket_text}],
    response_format={"type": "json_schema", "json_schema": {
        "name": "triage", "strict": True,
        "schema": {"type": "object", "properties": {
            "severity": {"enum": ["sev1", "sev2", "sev3"]},
            "component": {"type": "string"},
            "summary": {"type": "string", "maxLength": 240}},
          "required": ["severity", "component", "summary"], "additionalProperties": False}}},
    extra_body={"compat_mode": "strict"},   # 422 rather than a silently omitted schema
)
# in permissive mode, check res.model_extra.get("warnings") for parameter_omitted
triage = json.loads(res.choices[0].message.content)   # constrained by the provider's decoding
```

## Capability matrix

| Model | Tools | Parallel | Structured | Vision | Caching |
| --- | --- | --- | --- | --- | --- |
| openai/gpt-4o | supported | supported | supported | supported | supported |
| openai/gpt-4o-mini | supported | supported | supported | supported | not supported |
| anthropic/claude-sonnet-4 | supported | supported | supported | supported | supported |
| google/gemini-2.5-pro | supported | supported | supported | supported | not supported |
| mistral/mistral-large-2 | supported | supported | supported | not supported | not supported |
| groq/llama-3.3-70b | supported | supported | supported | not supported | not supported |

Verified, not advertised Capabilities are facts per endpoint, operation and parameter (`GET /api/v1/catalog/endpoints/{id}/capabilities`), not provider marketing. When endpoints of the same model differ, the request is routed only to endpoints that pass.

Source: https://yelno.ai/docs/tools