Sieve-0.8B
Model card for Sieve-0.8B, a small decision model. It returns probabilities for typed questions and does not generate text.
Hugging Face: sthanika-ai/Sieve-0.8B (Text Classification, Transformers, Apache-2.0)
API model name: sieve-0.8b (versioned ID sieve-0.8b-v1). No modelled price yet.
Sieve-0.8B answers typed questions about a text or JSON state. It returns a probability for each option. It never generates text.
Key facts
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3.5-0.8B-Base, revision dc7cdfe2 |
| Layers used | Blocks 1-16 of 24 (only these run) |
| Method | Option fork. The state is read once. Each option is scored as its own continuation. A scalar head gives a softmax over options. |
| Question types | choice, noul (a noul is scored as a 2-option choice) |
| Parameters at inference | 0.59B (full base model 0.75B) |
| Trained weights | Full fine-tune of blocks 1-16 plus the scorer head (LayerNorm, 512, 1). Token embeddings frozen. |
| Memory | About 1.2 GB in bf16 |
| Temperature | One built-in value, T = 1.1208 |
Limits
- 32,768 tokens for the state, the question and the longest option. Requests over this limit are refused, not trimmed.
- Trained on inputs up to 8,192 tokens.
- The card states no option-count limit. The API limit for
choiceis 255 options (see limits and errors).
Calibration
- The temperature changes probabilities, not the winning option.
- On the validation set (2,388 questions, accuracy 75.8%): NLL 0.645 and ECE 0.024 with T = 1.12, versus NLL 0.650 and ECE 0.029 with raw probabilities (T = 1.0).
Benchmarks (self-reported)
| Measure | Score |
|---|---|
| Decision Index Full score (leaderboard 0.3.2) | 23.1 |
| Public index (own run, 0.3 suite) | 32.26 |
| Public index (own run, 0.2.1 suite) | 31.46 |
Weak areas on the card: knowledge-heavy and multi-step tasks (MMLU-Pro, GPQA, BBH, MuSR). The card also lists zero scores on iSarcasmEval, RAGTruth, HLE, CRUXEval and a home-appliance simulator.
Training
- 190,296 questions from 155 decision tracks.
- One epoch (5,947 steps). AdamW, learning rate 5e-6, 3% warm-up, cosine decay to 10%. Batch of 32 questions. Gradient clip 1.0.
- One training run and one seed. The learning rate was chosen on a pilot.
- Training files are not published.
Intended use
Calibrated probabilities for typed questions about a text or JSON state, without generated text. Routing and classification-style decisions.
Licence
- Adapter, head and code: Apache-2.0.
- Base model: Qwen3.5 (Apache-2.0 per the card).
- Public training datasets keep their own licences.
Data retention
The model keeps nothing between calls. Through the hosted API, Sieve does not store prompts, state or answers (see data retention).
Install and call
pip install "sieve-decisions[cuda,serve] @ git+https://github.com/sthanika-ai/Sieve"
pip install --no-build-isolation causal-conv1d==1.7.0 # recommendedNeeds transformers==5.17 (for the qwen3_5 architecture).
Through System One:
{
"model": "sieve-0.8b",
"state": "I was charged twice this month.",
"questions": {
"billing": { "type": "noul", "instructions": "Is this message about billing?" }
}
}Next steps
Last updated