Fine-tuning / Guide

# Fine-tuning guide

Train a supported open model on your own data, serve it on a dedicated endpoint billed per GPU-hour while it runs, and call it with the OpenAI or Anthropic SDK.

Hosting is usually the larger cost. Training is charged once per job. The endpoint that serves your model is billed per GPU-hour while it is starting or running, whether or not it receives requests. Set auto-stop and stop endpoints you are not using: a stopped endpoint costs nothing. Final prices are on the [fine-tuning page](https://yelno.ai/fine-tuning).

## Enable fine-tuning

Fine-tuning sends your training data to Together AI, and training runs on Together AI's infrastructure, so it is off by default. An owner or security admin turns it on with the organization policy flag `allow_fine_tuning` (`PUT /api/v1/organizations/{organization_id}/policy`, permission `policy:update`). Turning it on loosens the data policy, so it asks for a recent sign-in, goes through your approval policies, and is written to the audit log.

Every fine-tuning route checks the flag and answers `403 fine_tuning_disabled` while it is off. An organization that requires zero data retention gets `403 zdr_required`, because the training provider's terms do not meet it. Project environment policies apply to every write. `GET /v1/fine_tuning` answers either way: whether fine-tuning is enabled and available for you, where the data goes, the provider's retention terms and the settings in force.

## Datasets and formats

A dataset is a UTF-8 JSON Lines file, one example per line, up to 100 MB and 200,000 lines. Upload it to `POST /v1/files` with `purpose=fine-tune`. The format is detected from the first valid line, or you can declare it with `format`. The returned `file_…` id goes straight into a job. Datasets are deleted from Together AI after the job finishes unless you set `keep_provider_files`; the copy kept here follows your organization's file retention.

Upload a dataset

```
import os
from openai import OpenAI

client = OpenAI(base_url="https://yelno.ai/v1", api_key=os.environ["YELNO_API_KEY"])

train = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")
print(train.id, train.status)              # "processed", or "error" with status_details
print(train.yelno["fine_tuning"]["report"])  # the validation report
```

Chat (SFT). One conversation per line: a `messages` array with an optional first `system` message, then `user` and `assistant` turns alternating, ending with the `assistant` answer the model should learn. Tool calls go in `tool_calls` on an assistant turn, answered by a `tool` message with the matching `tool_call_id`.

chat.jsonl · one line

```
{"messages": [{"role": "system", "content": "You are Acme's support assistant."}, {"role": "user", "content": "How do I reset my password?"}, {"role": "assistant", "content": "Open Settings → Security and choose Reset password. We email you a link that works for 30 minutes."}]}
```

Completion (SFT). One example per line as a `prompt` and the `completion` the model should produce. Both must be non-empty.

completion.jsonl · one line

```
{"prompt": "Classify the sentiment: The delivery was late again.\nSentiment:", "completion": " negative"}
```

Preference (DPO) (DPO). One comparison per line: the conversation so far in `input.messages`, the answer you prefer in `preferred_output`, and the one you don't in `non_preferred_output`. Used for DPO.

preference.jsonl · one line

```
{"input": {"messages": [{"role": "user", "content": "Summarize this ticket in one sentence: …"}]}, "preferred_output": [{"role": "assistant", "content": "Customer cannot log in after the 2.3 update; asks for a fix today."}], "non_preferred_output": [{"role": "assistant", "content": "The customer has an issue."}]}
```

In chat files, `system` may only come first, `user` and `assistant` alternate after it, and the conversation ends with `assistant`. An assistant turn with `tool_calls` may have `null` content, and a `tool` turn needs the `tool_call_id` of an earlier call. An optional `weight` of 0 or 1 on an assistant turn leaves it out of, or keeps it in, the loss.

## Validation report

Every file is validated on upload and the report comes back in the response's `yelno.fine_tuning` block. An invalid file is still stored, with `status: "error"` and `state: "invalid"`, so you can read the report again; a job refuses it with `400 invalid_training_file`. Up to 100 errors are listed, each with its line number; `errors_truncated` counts the rest.

the yelno block of a file with errors

```
"yelno": {"fine_tuning": {"id": "ftf_…", "format": "chat", "state": "invalid", "lines": 120, "bytes": 40960,
  "tokens_estimate": 53000, "sha256": "…",
  "report": {"valid": false,
    "errors": [{"line": 7, "code": "roles_not_alternating", "message": "…", "param": "messages[2].role"}],
    "errors_truncated": 0,
    "warnings": [{"line": 31, "code": "duplicate_example", "message": "…"}],
    "stats": {"examples": 120, "tokens_estimate": 53000, "tokens_min": 120, "tokens_max": 900, "tokens_mean": 441,
              "duplicates": 1, "empty_lines": 0, "examples_over_context": 0, "assistant_turns": 240, "tool_calls": 0}}}}
```

| Code | Meaning |
| --- | --- |
| invalid_json_line · not_an_object · empty_file | The line is not a JSON object, or the file has no examples. |
| missing_field · empty_content | A required field is absent or a message has no content. |
| invalid_role · system_not_first · roles_not_alternating · must_end_with_assistant | The chat turns break the order above. |
| invalid_tool_call · unknown_tool_call_id | A tool call is malformed, or a tool turn answers a call that does not exist. |
| invalid_preference · mixed_formats · too_many_lines | A DPO line is malformed, lines mix formats, or the file is over the line limit. |
| duplicate_example (warning) | The same example appears again; the warning names the first line. |
| few_examples (warning) | Fewer than 10 examples. |
| exceeds_context (warning) | Estimated tokens exceed the base model's context; checked when you create the job. |

Token counts in the report are estimates (about four characters per token, plus a few per message). The training provider's own count is what you are charged on.

## Jobs

A job names a base model, a training file, the method and the hyperparameters. `method.type` is `supervised` (SFT, the default) or `dpo`; `method.training_type` is `lora` (the default) or `full`. OpenAI's own `method` object is accepted too. `suffix` (lower-case letters, digits and hyphens, up to 40) becomes part of your model's id. Call `POST /v1/fine_tuning/estimate` with the same body first to see the estimate and the hold without starting anything.

Estimate and create a job

```
job = client.fine_tuning.jobs.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",  # a base model from GET /v1/fine_tuning/base_models
    training_file=train.id,
    suffix="support-bot",
    extra_body={
        "method": {"type": "supervised", "training_type": "lora"},
        "hyperparameters": {"n_epochs": 3},
    },
    extra_headers={"Idempotency-Key": "support-bot-v1"},
)
print(job.id, job.status, job.yelno["estimate_nanos"], job.yelno["hold_nanos"])
```

Send an `Idempotency-Key`: the same key and body return the same job instead of a second one, and the same key with a different body is `409 idempotency_key_reused`. If an approval policy covers the job you get `400 approval_required` with an `approval_request_id`; re-send with `X-Yelno-Approval` once it is approved.

| State | status | Meaning |
| --- | --- | --- |
| validating | `validating_files` | The files are checked and uploaded to the training provider; the hold is already placed. |
| queued | `queued` | Waiting for capacity at the training provider. |
| running | `running` | Training. Events, loss and checkpoints arrive; yelno.progress reports steps and epochs. |
| succeeded | `succeeded` | Done. fine_tuned_model names your new model; the job is charged for the trained tokens. |
| failed | `failed` | error says why. Charged only if the provider billed for the work; the rest of the hold is released. |
| cancelled | `cancelled` | Stopped by you (or staff). Charged for the training completed before it stopped. |

`status` uses OpenAI's words; `yelno.state` carries the platform's. Follow a job through its events (newest first; `data` carries step, total_steps, loss, epoch and checkpoint) and its checkpoints, and cancel it while it is open. Cancelling a finished job is `409 job_not_cancellable`.

Watch, list checkpoints, cancel

```
job = client.fine_tuning.jobs.retrieve(job.id)       # status, trained_tokens, yelno.progress
for e in client.fine_tuning.jobs.list_events(job.id, limit=20):
    print(e.created_at, e.level, e.message)          # e.data carries step, total_steps, loss
client.fine_tuning.jobs.checkpoints.list(job.id)     # intermediate checkpoints
client.fine_tuning.jobs.cancel(job.id)               # stop it; completed training is charged
```

## Hyperparameters

Every value is optional; the defaults below are used for what you leave out. Ranges: `n_epochs` 1–20, `learning_rate` above 0 up to 0.01, `batch_size` 1–1024 or `"max"`, `warmup_ratio` 0–1, `n_checkpoints` 1–10, `n_evals` 0–100, `lora_r` 1–256, `lora_alpha` 1–512, `lora_dropout` from 0 to below 1, `dpo_beta` above 0 up to 10. LoRA settings with a full fine-tune, or `dpo_beta` without DPO, are `400 invalid_hyperparameters`.

| Parameter | Default | What it does |
| --- | --- | --- |
| n_epochs | `1` | How many times training passes over the whole dataset. Each epoch is charged: two epochs cost about twice one. More epochs fit your data more closely; too many can memorize the examples instead of learning the pattern. 1 to 3 is typical. |
| learning_rate | `0.00001` | How big each update step is. Too high and training becomes unstable; too low and it learns slowly. The default suits most LoRA runs. |
| batch_size | `max` | Examples per update step. “max” lets the training provider pick the largest size the hardware fits, which is usually fastest. |
| warmup_ratio | `0` | The share of steps at the start during which the learning rate ramps up from zero. A small warmup (0 to 0.1) can steady early training. |
| n_checkpoints | `1` | How many intermediate copies of the model to save during training. Useful when a later epoch overfits and an earlier one is better. |
| n_evals | `0` | How many times to measure loss on the validation file during training. Each evaluation runs over the validation tokens and is charged like training tokens. |
| train_on_inputs | `auto` | Whether the prompt part of each example counts toward the loss, or only the answer. “auto” lets the provider decide by format. |
| lora_r | `8` | The size of the adapter. A higher rank can learn more but trains slower and uses more memory. 8 to 64 is typical. Only with LoRA. |
| lora_alpha | `16` | Scales how strongly the adapter changes the base model. Usually about twice the rank. Only with LoRA. |
| lora_dropout | `0` | Randomly drops part of the adapter during training to reduce overfitting on small datasets. 0 turns it off. Only with LoRA. |
| lora_trainable_modules | `all-linear` | Which layers the adapter attaches to. “all-linear” attaches to every linear layer. Only with LoRA. |
| dpo_beta | `0.1` | How far preference training may move away from the base model. Lower values allow bigger changes. Only with DPO. |
| max_grad_norm | `1` | Caps the size of each update so one unusual batch cannot throw training off. The default rarely needs changing. |
| weight_decay | `0` | A small pull toward smaller weights that can reduce overfitting. 0 turns it off. |

## Evaluation

To measure how training generalizes, upload a second file in the same format and pass it as `validation_file`, with `n_evals` above 0. The validation loss is reported in the job's events beside the training loss, and the console charts both. A validation loss that rises while training loss keeps falling usually means the model is memorizing the examples: fewer epochs or an earlier checkpoint may serve better.

Evaluations are charged like training tokens: the estimate is training tokens × epochs plus validation tokens × evaluations. Loss is a training signal, not a quality measure. No accuracy or quality is promised: test the model on your own cases before you rely on it.

## Deploy an endpoint

A finished job gives you a model with an id of the form `ft:<base-slug>:<suffix>:<shortid>` (the suffix is `org` when you set none). It answers requests only while a dedicated endpoint serves it. List the hardware it can run on with `GET /v1/fine_tuning/hardware?model=`: each entry carries its final price per replica-hour. Create the endpoint with the hardware and its auto-stop settings; it starts straight away unless you send `start: false`. One live endpoint is allowed per model.

Pick hardware, deploy, stop

```
import os, requests

BASE = "https://yelno.ai/v1"
H = {"Authorization": f"Bearer {os.environ['YELNO_API_KEY']}"}
MODEL = "ft:llama-3-1-8b:support-bot:a1b2c3d4"

hardware = requests.get(f"{BASE}/fine_tuning/hardware", params={"model": MODEL}, headers=H).json()["data"]
for hw in hardware:
    print(hw["id"], hw["hardware"], hw["price_per_hour_nanos"] / 1e9, "USD per hour")  # display only

endpoint = requests.post(f"{BASE}/fine_tuning/endpoints", headers=H, json={
    "model": MODEL,
    "hardware": hardware[0]["id"],
    "auto_stop_idle_minutes": 15,
    "max_hours": 8,
    "budget": 40_000_000_000,   # $40.00, in nanos
}).json()
print(endpoint["state"], endpoint["price_per_hour_nanos"])

# ... later
requests.post(f"{BASE}/fine_tuning/endpoints/{endpoint['id']}/stop", headers=H)
```

| Field | Meaning |
| --- | --- |
| hardware | An fth_… id (or the provider's hardware name) from the hardware list. |
| min_replicas · max_replicas | Default 1. The hourly price is the hardware's price × max_replicas, fixed when the endpoint starts. |
| auto_stop_idle_minutes | Stop after this many minutes without a request. Default 15. |
| max_hours | Stop when a run reaches this many hours. Default none. |
| budget | Stop when the endpoint's billed plus accrued cost reaches this amount, in nanos. Default none. |
| start | Default true. false creates the endpoint stopped. |

Starting needs at least one hour of the endpoint's hourly price available in your wallet, or `402 insufficient_funds`; an approval policy can also cover it. Change auto-stop settings with `PATCH /v1/fine_tuning/endpoints/{id}`, and use `/start`, `/stop` and `DELETE` to run, pause and remove it. Archiving a model is refused while its endpoint is starting or running.

## Call your model

Call it like any other model, with your organization's API key. Only your organization can: to anyone else the id does not exist. It is listed by `GET /v1/models` with `owned_by` set to your organization, and never appears in the public catalog. With the OpenAI SDK, set `base_url` to `https://yelno.ai/v1`:

OpenAI SDK

```
import os
from openai import OpenAI

client = OpenAI(base_url="https://yelno.ai/v1", api_key=os.environ["YELNO_API_KEY"])
resp = client.chat.completions.create(
    model="ft:llama-3-1-8b:support-bot:a1b2c3d4",
    messages=[{"role": "user", "content": "How do I reset my password?"}],
)
print(resp.choices[0].message.content)
```

With the Anthropic SDK, set the base URL to `https://yelno.ai` (it calls `/v1/messages`):

Anthropic SDK

```
import os
import anthropic

client = anthropic.Anthropic(base_url="https://yelno.ai", api_key=os.environ["YELNO_API_KEY"])
msg = client.messages.create(
    model="ft:llama-3-1-8b:support-bot:a1b2c3d4",
    max_tokens=512,
    messages=[{"role": "user", "content": "How do I reset my password?"}],
)
print(msg.content[0].text)
```

No per-token charge is added: the endpoint's hourly price covers the requests it serves, and the response says so. Your routing policy (provider allow-list, retention rules) still applies.

yelno.cost on a response from your model

```
"yelno": {"cost": {"state": "final", "customer_nanos": 0, "included_in_hosting": true, "endpoint_id": "fte_…",
  "note": "Per-token use of your fine-tuned model is included in the endpoint's hourly price."}}
```

When no endpoint is running for the model, the request is refused before anything is sent, so you are never charged for it:

409 · no endpoint running

```
{"error": {"type": "invalid_request", "code": "model_not_deployed", "param": "model",
  "message": "… start an endpoint: POST /v1/fine_tuning/endpoints/{id}/start"}}
```

## Costs and auto-stop

Training. The estimate is training tokens × epochs + validation tokens × evaluations, at the base model's final price per 1M training tokens for the method, with a per-job minimum charge. A hold of the estimate plus 20% is placed before anything is sent to the provider; if it cannot be placed, the answer is `402 insufficient_funds`. A successful job is charged for the tokens the provider reports as trained and the rest of the hold is released. A cancelled job is charged for the training completed. A job that fails on invalid input or a provider error is not charged.

Hosting. An endpoint is billed per minute while it is starting or running, posted every 5 minutes, at the hourly price fixed when it started. Stopping bills up to that minute and releases the hold; a stopped endpoint costs nothing and holds nothing. While it runs, the next interval is held in advance: when that hold cannot be placed, the endpoint stops with `wallet_empty`. The endpoint object shows `run_cost_nanos` (this run so far) and `total_cost_nanos`.

| stop_reason | Why it stopped |
| --- | --- |
| user | You stopped it (console, API or DELETE). |
| idle | No request for auto_stop_idle_minutes (default 15). |
| max_hours | The run reached max_hours. |
| budget | Billed plus accrued cost reached the endpoint's budget. |
| wallet_empty | The next billing interval could not be held: the wallet cannot pay for it. |
| staff | Platform staff stopped it (audited). |
| provider | The provider stopped it; observed and recorded. |
| failed | The endpoint failed to start or to keep running. |

Every stop is recorded with its reason and sent as the `fine_tuning.endpoint.stopped` webhook. Amounts in the API are integer nanos (1 USD = 1,000,000,000). Each job and each hosting interval is a line on your statement and in [budgets](https://yelno.ai/docs/limits) like any other spend.

## Webhooks

Subscribe a [webhook endpoint](https://yelno.ai/docs/webhooks) to these events, or to `fine_tuning.*` for all of them. Payloads carry ids and final amounts only, never your data.

| Event | Payload |
| --- | --- |
| fine_tuning.job.succeeded | `{job_id, fine_tuned_model, trained_tokens, charged_nanos, currency}` |
| fine_tuning.job.failed | `{job_id, error, charged_nanos}` |
| fine_tuning.job.cancelled | `{job_id, charged_nanos}` |
| fine_tuning.endpoint.running | `{endpoint_id, model, price_per_hour_nanos}` |
| fine_tuning.endpoint.stopped | `{endpoint_id, model, reason, run_cost_nanos}` |

## Troubleshooting

| You see | What to do |
| --- | --- |
| 403 fine_tuning_disabled | Fine-tuning is off for your organization. An owner or security admin turns on allow_fine_tuning in the organization policy. |
| 403 zdr_required | Your organization requires zero data retention, and Together AI's terms do not meet it. Nothing was sent. |
| 503 fine_tuning_unavailable | Fine-tuning is paused platform-wide. Running endpoints and settled charges are unaffected; try again later. |
| 400 invalid_training_file | The file has validation errors. Read its report (GET /v1/files/{id}), fix the lines it names and upload again. |
| 400 format_method_mismatch | DPO needs a preference file; SFT needs chat or completion. |
| 400 method_not_supported | The base model does not offer that method; GET /v1/fine_tuning/base_models lists what each one does. |
| 400 invalid_hyperparameters | A value is out of range, a LoRA setting was sent with a full fine-tune, or dpo_beta without DPO. |
| 402 insufficient_funds | The hold (estimate plus buffer) or one hour of hosting could not be placed. Top up and retry; nothing started. |
| 400 approval_required | An approval policy covers this job or endpoint. Once approved, re-send with X-Yelno-Approval: <approval_request_id>. |
| 409 model_not_deployed | No endpoint is running for the model. Start one, then retry the request. |
| 409 endpoint_exists | The model already has a live endpoint. Start or update that one instead. |
| 409 file_in_use | A job that has not finished uses this file. Wait for it, or cancel it, before deleting. |
| Warning exceeds_context | Some examples are longer than the base model's context. They will be cut; shorten or split them. |

Every route, with its request and response, is in the [fine-tuning API reference](https://yelno.ai/docs/fine-tuning-api).

Source: https://yelno.ai/docs/fine-tuning