mattwood.fyi

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Links indicate relevance, not agreement. How to use this site →

How To Use Decision Models For Evals

A running theme in Matt Wood’s FYI — 7 items spanning 2026-10-01 – 2026-10-30. This page compounds: new items on this theme are added as they’re posted. Tracked since 2026-10-20.

Tensions

SPINF Serve: GPU Batch Classification Engine vs How Long Contexts Fail — SPINF's single-pass multi-question scoring is a direct engineering response to long context failure modes — by reading content once and scoring all questions together, it mitigates repeated-context inefficiency

Lines of development

How To Use Decision Models For Evals → Introducing Strands Decider 2B: Open Source Decision Model — Strands Decider 2B is an open-source decision model; this tutorial explains how to operationalize such models by tuning thresholds and interpreting scores for evaluation tasks
How To Use Decision Models For Evals → Evolution of Text Classification: From Bag-of-Words to Modern APIs — Evolution of text classification contextualizes the historical development of classification models; this guide represents the modern applied endpoint of that evolution for evaluation workflows
Strands Decider: Why Not an Encoder? → Introducing Strands Decider 2B: Open Source Decision Model — The architectural analysis directly underpins the Strands Decider 2B model — this paper explains the design rationale for choosing decoder over encoder architecture in the same model family

Items

SPINF Serve: GPU Batch Classification Engine

A tool for bulk text classification and decision inference optimized for single GPU execution, reading content once and scoring all questions in one pass with OpenAI-compatible API support.

permalink · github.com →

How To Use Decision Models For Evals

Explains how to choose appropriate decision models for evaluation tasks, tune classification thresholds beyond the default 0.5, and interpret model scores with interactive visualizations.

permalink · x.com →

JevAgentHarness: Decision Models for LLM Agent Control

A framework for benchmarking and training decision models that efficiently handle tool-selection decisions in LLM agents, reducing computational cost and latency while maintaining task success rates through adaptive confidence thresholds and training on agent-reached states.

permalink · arxiv.org →

Strands Decider: Why Not an Encoder?

Compares encoder-based and decoder-based architectures for the Strands Decider model, evaluating five different designs on accuracy, calibration, and latency to determine whether bidirectional encoders outperform causal decoders for this decision-making task.

permalink · brooker.co.za →
Clef: Open-Source Decision Models and RL Fine-Tuning

Cloudflare introduces Clef, an open-source platform featuring decision models and reinforcement learning fine-tuning capabilities for AI model optimization.

permalink · blog.cloudflare.com →

Introducing Strands Decider 2B: Open Source Decision Model

Strands releases Decider 2B, a compact open source decision model designed for AI agents to make autonomous decisions efficiently.

permalink · strandsagents.com →

Clef: Open-Source Decision Models and RL Fine-Tuning

Cloudflare introduces Clef, an open-source platform featuring decision models and reinforcement learning fine-tuning capabilities for AI applications.

permalink · blog.cloudflare.com →