Fine-tune a model with Foundry

Foundry trains a private decision adapter (supervised LoRA) on your own labelled examples and serves it to your organisation only. You do everything on the console's Foundry page; afterwards you call the model through the normal API.

You need: the Admin role in your organisation (Members can view Foundry but not upload, train or deploy), Foundry enabled for your organisation, and at least 20 labelled examples with at least 2 labels.

How it worksCopy Anchor Link

1. Prepare the datasetCopy Anchor Link

Each line is a JSON object with exactly two string fields:

{"input": "My parcel has not arrived", "label": "delivery"}
{"input": "I was charged twice", "label": "billing"}
{"input": "Please refund my last order", "label": "refund"}

Use labels that name the decision you want (delivery, billing, refund), and give every label enough real examples: the evaluation splits need examples of each. Remove personal data you do not need; Foundry checks structure, not meaning or privacy.

This script builds the file from your reviewed records and catches the problems the uploader would refuse. It needs only Python's standard library:

import collections
import json
import sys

# Replace these with your own reviewed records: (text, label).
examples = [
    ("My parcel has not arrived", "delivery"),
    ("I was charged twice", "billing"),
    ("I was charged twice", "billing"),  # an exact repeat
    ("Please refund my last order", "refund"),
]

rows, seen, labels_by_input = [], set(), {}
for text, label in examples:
    text, label = text.strip(), label.strip()
    if not text or not label or len(label) > 128:
        sys.exit(f"Fix this record first: {text!r} -> {label!r}")
    if labels_by_input.setdefault(text, label) != label:
        sys.exit(f"Conflicting labels for {text!r}: {labels_by_input[text]!r} and {label!r}")
    if (text, label) in seen:
        continue  # drop exact duplicates (the uploader can also do this for you)
    seen.add((text, label))
    rows.append({"input": text, "label": label})

for row in rows:
    print(json.dumps(row, ensure_ascii=False))

counts = collections.Counter(row["label"] for row in rows)
print(f"{len(rows)} unique rows, labels: {dict(counts)}", file=sys.stderr)
if len(rows) < 20 or len(counts) < 2:
    print("Training needs at least 20 unique rows and at least 2 labels.", file=sys.stderr)

Run it as python prepare.py > tickets.ndjson. The Foundry page also has a local check: paste a sample or pick a file and the browser reports structure, repeats, conflicts and the label mix without uploading anything.

2. Upload and checkCopy Anchor Link

On Foundry, upload the file and choose how to treat repeats:

• Inspect keeps every line and reports duplicates.
• Deduplicate keeps one copy of each exactly repeated {input, label} row in the copy used for training.

The original file is kept unchanged and both copies are encrypted and private to your organisation. The upload shows rows, unique rows, duplicates, conflicts, labels and missing labels. Only a dataset that passes the checks in the table above can be chosen for training. Uploaded datasets are kept for 90 days by default; you can delete one at any time, which also cancels its queued training and revokes models deployed from it. See Data retention.

3. Choose a base and a presetCopy Anchor Link

Base model. Only bases your administrator approved appear; today that is the Codes family, Sieve-4B. sieve-auto is a cascade, not a base.

Preset. Each preset sets the LoRA rank, epochs and learning rate:

You also set the maximum training time (up to 3,600 seconds, the default) and a cost budget cap in micros; a job whose estimate exceeds the cap is refused. The engine fixes everything else: LoRA alpha is twice the rank, dropout 0, the query and value projections are adapted, the seed is 42, and the data is split 60/20/20 into training, validation and test sets. Related inputs (the same normalised text) stay in the same split. Out-of-range or unknown settings are refused with invalid_training_options, never silently changed.

4. Review the estimate and trainCopy Anchor Link

Select Review estimate. The server calculates the steps from your dataset size and epochs, the time from a measured reference throughput, and the cost from the configured GPU price. It is an estimate, not a quote or a reservation. Start training is enabled only once you have an estimate for the current settings.

The job moves through queued, running and evaluating; the page refreshes running jobs every five seconds and you can cancel an active job. Datasets, estimates and jobs are saved, so you can leave and come back. Each organisation can have at most four jobs queued, running or started in the last 24 hours, and four live deployments.

5. Read the resultsCopy Anchor Link

When evaluation finishes, the result shows:

• Validation accuracy and test accuracy of your adapter, each next to the baseline: the base model's accuracy on the same split.
• Per-class errors, a confusion summary and calibration error (ECE), when the training report includes them. Use them to spot labels that are under-represented or over-confident.
• Time, cost and adapter details, when reported.

Deploy gate. With the default policy, a job can be deployed only if all of these hold (an Enterprise installation can set stricter thresholds):

If the gate fails, the page names the failed check. Add representative examples for weak labels, fix conflicting labels, or try another preset, then train a new job. A passing gate shows the adapter learned your labels on held-out data; it does not prove fairness, safety or suitability for production, so also review real examples.

6. Deploy, call and revokeCopy Anchor Link

Deploy a passing job and give it a name: lowercase letters, digits and hyphens, starting with a letter, at most 48 characters (for example support-router). The model is callable only after the serving engine has loaded it and confirmed a test inference; the deployment then shows ready.

Call it with the model name ft:<name> (or its adp_… adapter ID, shown under the deployment). It works with any API key of your organisation, on System One and /v1/decisions, and it is listed in GET /v1/models by its adp_… ID. Another organisation that sends the name gets 404, exactly as for a name that does not exist; a failure never falls back to the base model silently.

curl https://api.sieve.one/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "ft:support-router",
    "state": "My parcel has not arrived",
    "questions": {
      "category": {
        "type": "choice",
        "instructions": "What is this ticket about?",
        "criteria": { "delivery": null, "billing": null, "refund": null }
      }
    }
  }'

Before the deployment is ready, or after it is revoked, the same call returns 404:

{ "detail": { "error_type": "not_found_error", "message": "Model not found: ft:support-router" } }

Revoke a deployment from the Foundry page. New requests are refused straight away and the engine unloads the adapter. To roll back, deploy an earlier passing job under a new name: a deployment is never changed in place. Each adapter is used through the normal API, so usage, limits and errors work exactly as for the public models. Custom models add a per-decision fee set in your contract.

Next stepsCopy Anchor Link