Kev decision models cut token waste in AI agent loops

Generative AI agents incur substantial computational overhead by producing text for intermediate decisions that require no human-readable output. Developers send routing, ranking, and safety-check queries to language models, wait for token-by-token generation, then parse the results—a cycle that wastes resources. OpenAI's own researchers recently reported spending $7,000 daily on agent workloads. Kev, a fresh family of open decision models built on Qwen 3.5, eliminates this inefficiency by skipping generation entirely and completing choices in a single forward pass.

Developer Jared Palmer released the latest iteration of Kev on Sunday, offering three model sizes: 0.8 billion, 4 billion, and 9 billion parameters, all built on Qwen 3.5. The architecture is prefill-only, ingesting state, questions, and candidate options in one forward pass, then extracting decisions from a pointer head without an autoregressive decoding loop.

Kev, a new family of open decision models built on Qwen 3.5, takes a different approach and skips the generation entirely.

Decisions without generated text

Kev implements three decision primitives: Noul for binary yes/no outcomes, Choice for selecting from a set of options, and Score for ranked levels. This design mirrors TypeSafe's System One API. Developers supply the context and questions; the pointer head returns probability distributions across available candidates.

Tool routing, safety validation, escalation logic, and ranking operations can now reside in the decision layer, freeing larger reasoning models to focus on open-ended tasks that genuinely require generation.

Kev can still choose the wrong tool, but because it scores only the candidates it's given, it can't introduce an option that isn't on the list.

Batching choices, one pass

The architecture supports multiple decisions against identical context in a single forward pass, using block-causal attention masking to isolate each question while the pointer head scores its respective candidates independently.

Palmer's documentation reports the 4B model processing three questions in 277 milliseconds in bf16 on an M5, though without a controlled benchmark comparing Qwen's generation performance on identical hardware, the practical speedup remains unquantified.

As agent loops become more intricate, the capacity to evaluate multiple decisions against shared context gains value. However, bypassing generation does not automatically improve decision quality.

Calibration limits and tradeoffs

The 9-billion-parameter variant, Kev-9B, achieved 83.7% accuracy on the project's held-out test set, per Palmer's model card. Palmer acknowledges important constraints alongside this result.

The probability scores Kev produces do not always align with actual confidence levels developers should assign. Palmer observed that temperature calibration can shift when encountering unseen data distributions—a critical issue for agents relying on probability thresholds to decide whether to proceed or escalate, since even high-confidence predictions can fail.

Fine-tuning introduces capability trade-offs. Palmer's tests show performance degradation on general knowledge and arithmetic benchmarks, particularly in smaller variants. This aligns with Kev's specialized role alongside general-purpose models, though real-world performance in dynamic agent settings depends on how well it handles unfamiliar tools, options, and labels unseen during training. Agent debugging often reveals infrastructure issues rather than model deficiencies.

The decision-model approach predates Kev. TypeSafe introduced Jev earlier this month as part of its System One platform, employing the same Noul, Choice, and Score primitives. Kev implements TypeSafe's /v1/systemone request and response format, allowing applications built against that API to point to a local Kev server instead.

Open weights, open training

The critical distinction lies in licensing and transparency. Palmer released Kev under Apache 2.0 with model weights, training code, and evaluation infrastructure included, enabling developers to run and fine-tune it on their own systems. Jev's weights and training data remain proprietary, making direct performance comparisons impossible since architectural, size, and training differences cannot be isolated.

For applications requiring only a few bounded decisions, constrained decoding on an already-running model may prove simpler than introducing another model to the pipeline. Agent loops, however, execute these decisions repeatedly—routing, ranking, safety checks, tool selection, and escalation all occur before generating user-facing output. This pattern appears across model families: removing unnecessary computation when the task doesn't demand it.

When those steps only require a choice or probability, Kev can handle the decision directly while leaving open-ended reasoning and final responses to the larger generative model.