Sieve-4B
Model card for Sieve-4B, the hosted low-cost decision model. It returns probabilities and does not generate text.
Hugging Face: sthanika-ai/Sieve-4B (Text Classification, Apache-2.0)
API model name: sieve-4b (versioned ID sieve-4b-v1). Used first in sieve-auto (see auto cascade).
Sieve-4B answers typed questions about a text or JSON state. It returns a probability for each option. It never generates text.
Key facts
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3.5-4B, revision 851bf6e8 |
| Method | LoRA adapter (rank 32, alpha 64, dropout 0.05) on every linear layer (attention, MLP, Gated DeltaNet projections) |
| Answer head | 255-way linear head, initialised from the base model's output rows for the option codes |
| Trainable parameters | Not stated on the card |
| Question types | choice (up to 255 options), noul (asked in both option orders and averaged), score (up to 10 levels) |
| Memory | About 10 GB in bf16. Longer states need more. |
Limits
- 32,768 tokens for the state plus the longest question.
- 65,536 tokens per request.
- Up to 255
choiceoptions and 10scorelevels. - Longer requests are refused, not truncated. Training records reached 16,806 tokens.
Calibration
Temperature per question type (changes probabilities, not the winning option):
| Question type | Temperature |
|---|---|
choice | 1.161 |
noul (yes/no) | 1.440 |
score | 1.000 |
Out-of-fold ECE: choice 0.039, yes/no 0.030. The leaderboard calibration panel reports ECE 0.031.
Benchmarks (self-reported)
| Measure | Score |
|---|---|
| Decision Index Full score (leaderboard 0.3.2) | 46.11 |
| Public index (own run, 0.3 suite) | 50.06 (raw index 62.10, breadth 48.32) |
| Public index (own run, 0.2.1 suite) | 49.21 |
Area scores (Knowledge and Reasoning 33.9; Language Understanding 53.9; Retrieval and Classification 61.6; Tools and Automation 66.8; Arts and Human Taste 28.3). The card does not map these to areas, so the order is inferred and not confirmed.
Some single benchmarks are low, for example ChessBench (0.138) and HLE (0.094).
Latency (self-reported)
HTTP wall time, with three server processes sharing one A100 80GB PCIe: median 302 ms, mean 433 ms, p80 400 ms, p95 817 ms. The shared GPU makes these numbers pessimistic.
Training
- 165,832 training records from public datasets and generated decision records. Files are not published (SHA-256 hashes only).
- 1 epoch, 1,295 steps, 128 records per step, 11.6 hours on two A100 80GB PCIe GPUs.
- AdamW. Learning rate 5e-5 (adapter), 1e-4 (head). 3% warm-up, then cosine.
- Loss: cross-entropy on the target distribution, plus an option-order consistency term and a replay term.
Intended use
Calibrated probabilities for typed questions about text or JSON, without generated text.
Licence
Apache-2.0 for the adapter, head and code. Backbone: Qwen3.5 (Apache-2.0 per the card).
Data retention
The model keeps nothing between calls. Through the hosted API, Sieve does not store prompts, state or answers (see data retention).
Install and call
pip install "sieve-decisions[cuda,serve] @ git+https://github.com/sthanika-ai/Sieve"The cuda extra installs the Gated DeltaNet kernels. Load with load_sieve from the sieve package.
Through System One:
{
"model": "sieve-4b",
"state": "I was charged twice this month.",
"questions": {
"urgency": { "type": "score", "instructions": "How urgent is this?", "criteria": ["Can wait", "This week", "Today"] }
}
}Next steps
Last updated