docs
Model cards

Sieve-9B-v2

Model card for Sieve-9B-v2, a mid-size decision model with per-category calibration. It returns probabilities and does not generate text.

Hugging Face: sthanika-ai/Sieve-9B-v2 (Text Classification, Apache-2.0) API model name: sieve-9b-v2. The aliases sieve-9b and sieve-latest point to this build in the current model catalog.

Sieve-9B-v2 answers typed questions about a text or JSON state. It returns a probability for each option. It never generates text.

Key facts

ItemValue
Base modelQwen/Qwen3.5-9B, revision c2022362, text part only. The card's model tree also lists Qwen3.5-9B-Base. We are confirming which base was used.
MethodLoRA (r=32, alpha=64, dropout 0.05) on 12 projection modules (attention, MLP, linear attention) plus a pointer head trained from scratch
HeadScores the end of each option against the decision point
Trainable parametersNot stated
Question typeschoice (up to 255 options), noul, score
MemoryAbout 20 GB in bf16. Longer states need more.
LanguageEnglish only (card language tag)

Limits

  • 16,384 tokens for the state. 16,384 tokens for each question with its options.
  • Longer requests are refused, not truncated.
  • The decision engine declares larger limits: 65,536 (state) and 32,768 (question). The API follows the limits in limits and errors.
  • sieve 0.2.0 and earlier ignore the per-category temperature table. Use sieve 0.2.1 or later.

Calibration

  • Global T = 1.109.
  • 23 per-category temperatures from 1.000 to 1.338.
  • Category = question type x option-count bin (2, 3-4, 5-9, 10-25, 26+) x state kind (empty, text, JSON). A question uses its most specific match, otherwise the global T.
  • Temperatures do not change answers. temperature=1.0 gives raw probabilities.
  • Results on 43,176 calibration questions (per-category): accuracy 0.8775, NLL 0.3071, Brier 0.1676, ECE 0.0068. Raw (T = 1): ECE 0.0093.

Benchmarks (self-reported)

MeasureScore
Decision Index Full score (leaderboard 0.3.2)50.89
Public score (own run, 0.3 suite)53.35 (raw index 64.39, breadth 51.95)
  • Area scores (public suite): Knowledge and Reasoning 38.3; Language Understanding 55.9; Retrieval and Classification 64.2; Tools and Automation 69.8; Arts and Human Taste 34.0.
  • Selected: BANKING77 macro-F1 0.860; HellaSwag 0.967; GSM8K 0.687; GPQA Diamond 0.464; HLE 0.100.
  • Coverage: 140,178 scoreable requests over 43 benchmarks.
  • The Full score includes the maintainers' private tests (50%) and private new-domain tasks (30%).

Latency (self-reported)

One H100 80GB, single process, eager, 3,000-request sample of the public 0.3 suite: median 43 ms, mean 138 ms, p80 58 ms, p95 144 ms.

Training

  • 165,832 training records and 30,722 calibration records, from public datasets and generated decision records. Training files are not published (hashes in training_config.json).
  • AdamW, learning rate 5e-5 (adapter and head). 10% warm-up, OneCycle. 1 epoch, 32 records per step, bf16.
  • 5,183 steps, 5.6 hours on two H100 80GB GPUs.
  • Selected checkpoint: step 4,500 (highest calibration accuracy, NLL as tie-breaker).
  • Augmentation: permuted choice options and "none of the above". Cross-entropy loss.

Limitations

  • Requests over the token limits are refused.
  • The causal-conv1d fallback (used when the package is missing) can shift probabilities slightly.
  • The card has no separate intended-use or limitations section. The description above is taken from its model summary.

Intended use

Calibrated probabilities for typed questions about text or JSON, without generated text. Routing and classification-style decisions.

Licence

Apache-2.0 for the model and code. Training data licences: see the dataset cards (pending legal review).

Data retention

The model keeps nothing between calls. Through the hosted API, Sieve does not store prompts, state or answers (see data retention).

Install and call

pip install "sieve-decisions[cuda,serve] @ git+https://github.com/sthanika-ai/Sieve"
pip install --no-build-isolation causal-conv1d==1.7.0   # recommended

Needs sieve 0.2.1 or later. The backbone downloads at its pinned revision on first use.

Through System One: "model": "sieve-9b-v2" (or "sieve-9b"). See the quickstart.

Next steps

Model cards

One card per Sieve model, with the base model, limits, calibration, benchmarks and licence.

Pricing

Modelled per-token prices for the Sieve decision models.

Auto cascade

How sieve-auto works.

Last updated

Sieve-9B

Model card for Sieve-9B (v1), a mid-size decision model. It returns probabilities and does not generate text.

Sieve-27B

Model card for Sieve-27B (v1), a large decision model. It returns probabilities and does not generate text.

On this page

Key factsLimitsCalibrationBenchmarks (self-reported)Latency (self-reported)TrainingLimitationsIntended useLicenceData retentionInstall and callNext steps