Model APIs / Batches & files

# Batches & files

Send thousands of chat or embeddings requests as one JSON Lines file, collect the results as files, and pay each line's final price as it runs.

## Routes

The OpenAI Files and Batches APIs, unchanged: point an OpenAI SDK at the gateway and `files.create`, `batches.create`, `batches.retrieve`, `batches.cancel` and `files.content` work as they do there. Every route takes your platform API key with the `inference:invoke` scope, and you only ever see your own organization's files and batches.

| Method | Path | What it does |
| --- | --- | --- |
| POST | /v1/files | Upload a JSON Lines file (multipart: file, purpose=batch). |
| GET | /v1/files | List your files, newest first (purpose, limit, after). |
| GET | /v1/files/{file_id} | One file's metadata. |
| GET | /v1/files/{file_id}/content | The file's bytes. |
| DELETE | /v1/files/{file_id} | Delete a file. |
| POST | /v1/batches | Create a batch from an uploaded file. |
| GET | /v1/batches | List batches, newest first (limit, after). |
| GET | /v1/batches/{batch_id} | A batch's status, counts, files and cost. |
| POST | /v1/batches/{batch_id}/cancel | Stop a batch: lines not yet run are cancelled and never charged. |

## Input file

One request per line. Each line needs a `custom_id` that is unique in the file (it is how you match results), `method: "POST"`, a `url` equal to the batch's endpoint, and the request `body` you would send to that endpoint. Supported endpoints are `/v1/chat/completions` and `/v1/embeddings`. A line may name any model you can call; `stream` is ignored because each line is answered once, in the output file.

requests.jsonl

```
{"custom_id": "ticket-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Summarize ticket 1"}]}}
{"custom_id": "ticket-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "openai/gpt-4o-mini", "messages": [{"role": "user", "content": "Summarize ticket 2"}]}}
```

Files are UTF-8 JSON Lines up to 50 MB, and a batch holds up to 50,000 requests. A file that breaks a line rule gives a batch with `status: "failed"` and an `errors` list naming each line and problem (`invalid_json_line`, `duplicate_custom_id`, `mismatched_endpoint`, `invalid_method`, `missing_required_parameter`). A problem inside a body, such as a missing `messages` or a model you cannot use, fails only that line, with the same error the live request would get.

## Create and poll

Create a batch

```
import os
from openai import OpenAI

client = OpenAI(base_url="https://yelno.ai/v1", api_key=os.environ["YELNO_API_KEY"])

batch_file = client.files.create(file=open("requests.jsonl", "rb"), purpose="batch")
batch = client.batches.create(
    input_file_id=batch_file.id,
    endpoint="/v1/chat/completions",
    completion_window="24h",
)

batch = client.batches.retrieve(batch.id)          # poll until status is terminal
if batch.output_file_id:
    print(client.files.content(batch.output_file_id).text)
```

200 · the batch object

```
{
  "id": "batch_…",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "errors": null,
  "input_file_id": "file_…",
  "completion_window": "24h",
  "status": "completed",
  "output_file_id": "file_…",
  "error_file_id": null,
  "created_at": 1759600000,
  "in_progress_at": 1759600000,
  "expires_at": 1759686400,
  "completed_at": 1759600420,
  "request_counts": { "total": 2, "completed": 2, "failed": 0 },
  "metadata": {},
  "yelno": {
    "project_id": "proj_…",
    "cost": { "state": "final", "currency": "USD", "customer_nanos": 17640 }
  }
}
```

| Status | Meaning |
| --- | --- |
| in_progress | Validated; lines are being run. |
| finalizing | Every line has finished; the output and error files are being written. |
| completed | Done. Some lines may still have failed: read request_counts and the error file. |
| cancelling | Cancel requested; lines already running are finishing. |
| cancelled | Stopped. Lines that had not started are in the error file as batch_cancelled. |
| expired | The 24-hour window ended first. Lines not run are in the error file as batch_expired. |
| failed | The input file did not validate; errors lists each problem with its line number. |

`completion_window` is `24h`: lines still waiting when it ends are not run. `metadata` takes up to 16 string pairs. Cancelling a batch that already finished answers `409` `batch_not_cancellable`.

## Output and error files

When a batch finishes, `output_file_id` holds one line per successful request and `error_file_id` one line per request that failed or never ran. Either is `null` when it would be empty. Lines are in input order; match them by `custom_id`. Download them with `GET /v1/files/{file_id}/content`. They are kept for 30 days.

output file · one line

```
{"id": "breq_…", "custom_id": "ticket-1", "response": {"status_code": 200, "request_id": "req_…", "body": {"object": "chat.completion", "choices": […], "usage": {…}, "yelno": {…}}}, "error": null}
```

`response.body` is exactly what the live endpoint returns, including its `yelno` block with the route and the charge. A failed line carries its status code and the standard [error envelope](https://yelno.ai/docs/errors); upstream text never reaches you.

error file · two lines

```
{"id": "breq_…", "custom_id": "ticket-7", "response": {"status_code": 402, "request_id": "req_…", "body": {"error": {"type": "insufficient_funds", "message": "…", "code": "…", "request_id": "req_…", "param": null, "retriable": false}}}, "error": null}
{"id": "breq_…", "custom_id": "ticket-8", "response": null, "error": {"code": "batch_cancelled", "message": "This request was not run because the batch was cancelled."}}
```

## How a batch is charged

Each line is a request. Every line runs through the gateway as its own request, with its own `req_` id in your request log. A hold is placed before the line is sent, and the line is charged the model's listed final price when it finishes, exactly as if you had sent it yourself. Rate limits, budgets, routing policies and your source allow-list apply to every line. Nothing is held for the batch as a whole. A cancelled or expired line is never charged.

The batch's `yelno.cost.customer_nanos` is the sum of its lines' charges. It reads `accruing` while lines run and `final` once the batch is finished and every line has settled. If your wallet runs out mid-batch, the remaining lines fail with `402` `insufficient_funds`. Lines limited by your rate limit wait and retry automatically at no cost. A revoked or suspended key stops its batch.

## Storage and privacy

Batch files are the one place the gateway keeps prompts and responses, because a batch needs them. Uploaded files, each line's body and result, and the output and error files are encrypted at rest and bound to your organization. Delete an input file whenever you like. Output and error files are removed 30 days after the batch finishes. In the console, [Observe · Batches](https://yelno.ai/console/batches) lists your batches with progress and cost, and downloads their files. Each download is written to your audit log.

Source: https://yelno.ai/docs/batches