Fine-tuning guide

Train a supported open model on your own data, serve it on a dedicated endpoint billed per GPU-hour while it runs, and call it with the OpenAI or Anthropic SDK.

#Enable fine-tuning

Fine-tuning sends your training data to Together AI, and training runs on Together AI's infrastructure, so it is off by default. An owner or security admin turns it on with the organization policy flag allow_fine_tuning (PUT /api/v1/organizations/{organization_id}/policy, permission policy:update). Turning it on loosens the data policy, so it asks for a recent sign-in, goes through your approval policies, and is written to the audit log.

Every fine-tuning route checks the flag and answers 403 fine_tuning_disabled while it is off. An organization that requires zero data retention gets 403 zdr_required, because the training provider's terms do not meet it. Project environment policies apply to every write. GET /v1/fine_tuning answers either way: whether fine-tuning is enabled and available for you, where the data goes, the provider's retention terms and the settings in force.

#Datasets and formats

A dataset is a UTF-8 JSON Lines file, one example per line, up to 100 MB and 200,000 lines. Upload it to POST /v1/files with purpose=fine-tune. The format is detected from the first valid line, or you can declare it with format. The returned file_… id goes straight into a job. Datasets are deleted from Together AI after the job finishes unless you set keep_provider_files; the copy kept here follows your organization's file retention.

import os
from openai import OpenAI

client = OpenAI(base_url="https://yelno.ai/v1", api_key=os.environ["YELNO_API_KEY"])

train = client.files.create(file=open("train.jsonl", "rb"), purpose="fine-tune")
print(train.id, train.status)              # "processed", or "error" with status_details
print(train.yelno["fine_tuning"]["report"])  # the validation report

{"messages": [{"role": "system", "content": "You are Acme's support assistant."}, {"role": "user", "content": "How do I reset my password?"}, {"role": "assistant", "content": "Open Settings → Security and choose Reset password. We email you a link that works for 30 minutes."}]}

{"prompt": "Classify the sentiment: The delivery was late again.\nSentiment:", "completion": " negative"}

{"input": {"messages": [{"role": "user", "content": "Summarize this ticket in one sentence: …"}]}, "preferred_output": [{"role": "assistant", "content": "Customer cannot log in after the 2.3 update; asks for a fix today."}], "non_preferred_output": [{"role": "assistant", "content": "The customer has an issue."}]}

In chat files, system may only come first, user and assistant alternate after it, and the conversation ends with assistant. An assistant turn with tool_calls may have null content, and a tool turn needs the tool_call_id of an earlier call. An optional weight of 0 or 1 on an assistant turn leaves it out of, or keeps it in, the loss.

#Validation report

Every file is validated on upload and the report comes back in the response's yelno.fine_tuning block. An invalid file is still stored, with status: "error" and state: "invalid", so you can read the report again; a job refuses it with 400 invalid_training_file. Up to 100 errors are listed, each with its line number; errors_truncated counts the rest.

"yelno": {"fine_tuning": {"id": "ftf_…", "format": "chat", "state": "invalid", "lines": 120, "bytes": 40960,
  "tokens_estimate": 53000, "sha256": "…",
  "report": {"valid": false,
    "errors": [{"line": 7, "code": "roles_not_alternating", "message": "…", "param": "messages[2].role"}],
    "errors_truncated": 0,
    "warnings": [{"line": 31, "code": "duplicate_example", "message": "…"}],
    "stats": {"examples": 120, "tokens_estimate": 53000, "tokens_min": 120, "tokens_max": 900, "tokens_mean": 441,
              "duplicates": 1, "empty_lines": 0, "examples_over_context": 0, "assistant_turns": 240, "tool_calls": 0}}}}

Token counts in the report are estimates (about four characters per token, plus a few per message). The training provider's own count is what you are charged on.

#Jobs

A job names a base model, a training file, the method and the hyperparameters. method.type is supervised (SFT, the default) or dpo; method.training_type is lora (the default) or full. OpenAI's own method object is accepted too. suffix (lower-case letters, digits and hyphens, up to 40) becomes part of your model's id. Call POST /v1/fine_tuning/estimate with the same body first to see the estimate and the hold without starting anything.

job = client.fine_tuning.jobs.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Reference",  # a base model from GET /v1/fine_tuning/base_models
    training_file=train.id,
    suffix="support-bot",
    extra_body={
        "method": {"type": "supervised", "training_type": "lora"},
        "hyperparameters": {"n_epochs": 3},
    },
    extra_headers={"Idempotency-Key": "support-bot-v1"},
)
print(job.id, job.status, job.yelno["estimate_nanos"], job.yelno["hold_nanos"])

Send an Idempotency-Key: the same key and body return the same job instead of a second one, and the same key with a different body is 409 idempotency_key_reused. If an approval policy covers the job you get 400 approval_required with an approval_request_id; re-send with X-Yelno-Approval once it is approved.

status uses OpenAI's words; yelno.state carries the platform's. Follow a job through its events (newest first; data carries step, total_steps, loss, epoch and checkpoint) and its checkpoints, and cancel it while it is open. Cancelling a finished job is 409 job_not_cancellable.

job = client.fine_tuning.jobs.retrieve(job.id)       # status, trained_tokens, yelno.progress
for e in client.fine_tuning.jobs.list_events(job.id, limit=20):
    print(e.created_at, e.level, e.message)          # e.data carries step, total_steps, loss
client.fine_tuning.jobs.checkpoints.list(job.id)     # intermediate checkpoints
client.fine_tuning.jobs.cancel(job.id)               # stop it; completed training is charged

#Hyperparameters

Every value is optional; the defaults below are used for what you leave out. Ranges: n_epochs 1–20, learning_rate above 0 up to 0.01, batch_size 1–1024 or "max", warmup_ratio 0–1, n_checkpoints 1–10, n_evals 0–100, lora_r 1–256, lora_alpha 1–512, lora_dropout from 0 to below 1, dpo_beta above 0 up to 10. LoRA settings with a full fine-tune, or dpo_beta without DPO, are 400 invalid_hyperparameters.

#Evaluation

To measure how training generalizes, upload a second file in the same format and pass it as validation_file, with n_evals above 0. The validation loss is reported in the job's events beside the training loss, and the console charts both. A validation loss that rises while training loss keeps falling usually means the model is memorizing the examples: fewer epochs or an earlier checkpoint may serve better.

Evaluations are charged like training tokens: the estimate is training tokens × epochs plus validation tokens × evaluations. Loss is a training signal, not a quality measure. No accuracy or quality is promised: test the model on your own cases before you rely on it.

#Deploy an endpoint

A finished job gives you a model with an id of the form ft:<base-slug>:<suffix>:<shortid> (the suffix is org when you set none). It answers requests only while a dedicated endpoint serves it. List the hardware it can run on with GET /v1/fine_tuning/hardware?model=: each entry carries its final price per replica-hour. Create the endpoint with the hardware and its auto-stop settings; it starts straight away unless you send start: false. One live endpoint is allowed per model.

import os, requests

BASE = "https://yelno.ai/v1"
H = {"Authorization": f"Bearer {os.environ['YELNO_API_KEY']}"}
MODEL = "ft:llama-3-1-8b:support-bot:a1b2c3d4"

hardware = requests.get(f"{BASE}/fine_tuning/hardware", params={"model": MODEL}, headers=H).json()["data"]
for hw in hardware:
    print(hw["id"], hw["hardware"], hw["price_per_hour_nanos"] / 1e9, "USD per hour")  # display only

endpoint = requests.post(f"{BASE}/fine_tuning/endpoints", headers=H, json={
    "model": MODEL,
    "hardware": hardware[0]["id"],
    "auto_stop_idle_minutes": 15,
    "max_hours": 8,
    "budget": 40_000_000_000,   # $40.00, in nanos
}).json()
print(endpoint["state"], endpoint["price_per_hour_nanos"])

# ... later
requests.post(f"{BASE}/fine_tuning/endpoints/{endpoint['id']}/stop", headers=H)

Starting needs at least one hour of the endpoint's hourly price available in your wallet, or 402 insufficient_funds; an approval policy can also cover it. Change auto-stop settings with PATCH /v1/fine_tuning/endpoints/{id}, and use /start, /stop and DELETE to run, pause and remove it. Archiving a model is refused while its endpoint is starting or running.

#Call your model

Call it like any other model, with your organization's API key. Only your organization can: to anyone else the id does not exist. It is listed by GET /v1/models with owned_by set to your organization, and never appears in the public catalog. With the OpenAI SDK, set base_url to https://yelno.ai/v1:

import os
from openai import OpenAI

client = OpenAI(base_url="https://yelno.ai/v1", api_key=os.environ["YELNO_API_KEY"])
resp = client.chat.completions.create(
    model="ft:llama-3-1-8b:support-bot:a1b2c3d4",
    messages=[{"role": "user", "content": "How do I reset my password?"}],
)
print(resp.choices[0].message.content)

With the Anthropic SDK, set the base URL to https://yelno.ai (it calls /v1/messages):

import os
import anthropic

client = anthropic.Anthropic(base_url="https://yelno.ai", api_key=os.environ["YELNO_API_KEY"])
msg = client.messages.create(
    model="ft:llama-3-1-8b:support-bot:a1b2c3d4",
    max_tokens=512,
    messages=[{"role": "user", "content": "How do I reset my password?"}],
)
print(msg.content[0].text)

No per-token charge is added: the endpoint's hourly price covers the requests it serves, and the response says so. Your routing policy (provider allow-list, retention rules) still applies.

"yelno": {"cost": {"state": "final", "customer_nanos": 0, "included_in_hosting": true, "endpoint_id": "fte_…",
  "note": "Per-token use of your fine-tuned model is included in the endpoint's hourly price."}}

When no endpoint is running for the model, the request is refused before anything is sent, so you are never charged for it:

{"error": {"type": "invalid_request", "code": "model_not_deployed", "param": "model",
  "message": "… start an endpoint: POST /v1/fine_tuning/endpoints/{id}/start"}}

#Costs and auto-stop

Training. The estimate is training tokens × epochs + validation tokens × evaluations, at the base model's final price per 1M training tokens for the method, with a per-job minimum charge. A hold of the estimate plus 20% is placed before anything is sent to the provider; if it cannot be placed, the answer is 402 insufficient_funds. A successful job is charged for the tokens the provider reports as trained and the rest of the hold is released. A cancelled job is charged for the training completed. A job that fails on invalid input or a provider error is not charged.

Hosting. An endpoint is billed per minute while it is starting or running, posted every 5 minutes, at the hourly price fixed when it started. Stopping bills up to that minute and releases the hold; a stopped endpoint costs nothing and holds nothing. While it runs, the next interval is held in advance: when that hold cannot be placed, the endpoint stops with wallet_empty. The endpoint object shows run_cost_nanos (this run so far) and total_cost_nanos.

Every stop is recorded with its reason and sent as the fine_tuning.endpoint.stopped webhook. Amounts in the API are integer nanos (1 USD = 1,000,000,000). Each job and each hosting interval is a line on your statement and in budgets like any other spend.

#Webhooks

Subscribe a webhook endpoint to these events, or to fine_tuning.* for all of them. Payloads carry ids and final amounts only, never your data.

#Troubleshooting

Every route, with its request and response, is in the fine-tuning API reference.