mattwood.fyi

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Links indicate relevance, not agreement. How to use this site →

Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning

A running theme in Matt Wood’s FYI — 26 items spanning 2026-09-09 – 2026-10-22. This page compounds: new items on this theme are added as they’re posted. Tracked since 2026-10-15.

Tensions

Object-Oriented Harness: Evolving Agent Harnesses through OO Design vs Your Game Is A Harness — OO Harness treats agent harnesses as optimizable software artifacts evolved through design principles, while 'Your Game Is A Harness' frames harnesses as structural constraints - the former actively subverts the static harness conception
Via AI Launches Enterprise Coding Assistant at Half the Cost vs The Slow Formation of Durable Software — Via AI's rapid enterprise launch with novel compression algorithms challenges the thesis of slow, durable software formation by demonstrating fast iteration in AI coding tooling
Via AI Launches Enterprise Coding Assistant at Half the Cost vs How Long Contexts Fail — Via AI's Codegraph compression algorithm is specifically designed to address the failure modes of long contexts in coding tasks, positioning it as a technical solution to documented long-context limitations
AAAI Research Paper 70113 vs AI Model Groupthink — Formal peer-reviewed AAAI research typically provides empirical evidence and rigorous methodology that can challenge or nuance informal observations about AI model groupthink behavior
Your Game Is A Harness vs Object-Oriented Harness: Evolving Agent Harnesses through OO Design — OO Harness treats agent harnesses as optimizable software artifacts evolved through design principles, while 'Your Game Is A Harness' frames harnesses as structural constraints - the former actively subverts the static harness conception
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning vs The Plunging Price of Thought — If meta-reasoning adds a structured overhead layer before task execution, it complicates the 'thought is nearly free' thesis — the framework implicitly argues that naive scaling of inference compute is insufficient without structured allocation decisions
Automating Eval Design and Hillclimbing with Claude vs AI Model Groupthink — Automating eval design with a single AI (Claude) to hillclimb evaluations risks encoding the same collective biases described in AI groupthink, potentially creating circular validation
Thinking Fast and Slow in AI: the Role of Metacognition vs Dream-RSI: Recursive Self-Improvement through Evolving Worlds — Dream-RSI's recursive self-improvement assumes agents can autonomously evolve, but this paper challenges whether narrow AI lacking metacognition can meaningfully direct its own improvement
Thinking Fast and Slow in AI: the Role of Metacognition vs Thinking Will Become a Hobby — Metacognition research suggests AI still lacks the self-regulating reasoning humans employ, challenging ec5e138d's claim of across-the-board cognitive superiority

Lines of development

Object-Oriented Harness: Evolving Agent Harnesses through OO Design → Your Game Is A Harness — Both treat harnesses as first-class design artifacts; OO Harness extends the philosophical framing of 'your game is a harness' into a systematic evolutionary methodology
Object-Oriented Harness: Evolving Agent Harnesses through OO Design → Testing Behavioral Harnesses for AI Coding Agents — Testing behavioral harnesses for AI coding agents provides the evaluation methodology that OO Harness optimization would need to validate its 15% improvement claims
How To Use Decision Models For Evals → Introducing Strands Decider 2B: Open Source Decision Model — Strands Decider 2B is an open-source decision model; this tutorial explains how to operationalize such models by tuning thresholds and interpreting scores for evaluation tasks
How To Use Decision Models For Evals → Evolution of Text Classification: From Bag-of-Words to Modern APIs — Evolution of text classification contextualizes the historical development of classification models; this guide represents the modern applied endpoint of that evolution for evaluation workflows
Kiro's Self-Improvement System and Validation Checks → Automating Eval Design and Hillclimbing with Claude — Kiro's kiro-trace pattern represents a specific instance of automating eval design — it automates the identification of failure patterns and proposes fixes, which is the core loop described in automating eval design and hillclimbing
StarSkirmish Bench v0.2: LLM StarCraft Bot Benchmark → Your Game Is A Harness — StarSkirmish operationalizes the 'game as harness' concept - using StarCraft as a rigorous evaluation harness for LLM coding ability with measurable win/loss outcomes
AAAI Research Paper 70113 → Evolution of Text Classification: From Bag-of-Words to Modern APIs — AAAI has been the primary venue publishing foundational text classification research that evolved from bag-of-words through neural approaches to modern APIs
The Ultimate Guide to Multi-Harness RL → Trust in Depth: Giving agents more autonomy without losing control — Multi-harness RL provides a concrete technical mechanism for the autonomy/control balance—training agents across behavioral constraints is how trust levels get operationalized in practice
Strands Decider: Why Not an Encoder? → Introducing Strands Decider 2B: Open Source Decision Model — The architectural analysis directly underpins the Strands Decider 2B model — this paper explains the design rationale for choosing decoder over encoder architecture in the same model family
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning → Thinking Fast and Slow in AI: the Role of Metacognition — Both address metacognition in AI, but the new paper operationalizes 'thinking about thinking' into a concrete agentic control framework, extending the conceptual discussion of fast/slow thinking into structured meta-reasoning for task execution
Automating Eval Design and Hillclimbing with Claude → Engineering a Calibrated Classifier Model — Engineering a calibrated classifier model involves similar iterative evaluation design principles; the new item automates and extends these practices into a systematic hillclimbing workflow

Items

Sage-A1.5: Agentic Model for Long-Horizon Problem Solving

A 35B Mixture-of-Experts agentic model that advances long-horizon problem solving through adaptive capability composition, featuring improvements in task orchestration, verification mechanisms, and scientific discovery capabilities.

permalink · arxiv.org →

Object-Oriented Harness: Evolving Agent Harnesses through OO Design

A framework that optimizes language model agent harnesses by evolving a hierarchy of specialized executable classes using object-oriented design principles, achieving up to 15% improvement over baseline methods on long-horizon task benchmarks.

permalink · arxiv.org →

Via AI Launches Enterprise Coding Assistant at Half the Cost

Via AI, founded by Stanford and IIT alumni, launches an enterprise AI coding platform featuring their Codegraph compression algorithm and Umami agent architecture, claiming superior performance and cost efficiency compared to Claude Code, Devin, and Cursor on major benchmarks.

permalink · x.com →
How To Use Decision Models For Evals

Explains how to choose appropriate decision models for evaluation tasks, tune classification thresholds beyond the default 0.5, and interpret model scores with interactive visualizations.

permalink · x.com →
Kiro's Self-Improvement System and Validation Checks

Kiro uses an internal tracing system called kiro-trace to automatically identify failure patterns in its AI coding agent runs, propose fixes, and validate them with deterministic checks to prevent false positives. The system generated seven merged pull requests in its first week by targeting specific measured patterns that wouldn't have surfaced as user-visible errors.

permalink · kiro.dev →

JevAgentHarness: Decision Models for LLM Agent Control

A framework for benchmarking and training decision models that efficiently handle tool-selection decisions in LLM agents, reducing computational cost and latency while maintaining task success rates through adaptive confidence thresholds and training on agent-reached states.

permalink · arxiv.org →
StarSkirmish Bench v0.2: LLM StarCraft Bot Benchmark

StarSkirmish evaluates large language models by having them write StarCraft: Brood War bots in C++ within one hour, then testing their performance against human-written and other LLM bots. The v0.2 results show GPT-6 Astra leading, with analysis of cost-efficiency, practice strategies, and how models compare against top human bots.

permalink · starskirmish.com →
AAAI Research Paper 70113

A research publication from the Association for the Advancement of Artificial Intelligence, indexed in the Wiley Online Library digital collection.

permalink · onlinelibrary.wiley.com →

Thinking Orbs: AI thinking indicator for React

Animated loading indicator components for AI interfaces with nine hand-tuned states, available for React, React Native, and Swift.

Had me at 'thinking orbs'.

permalink · libraries.dev →

Testing Behavioral Harnesses for AI Coding Agents

Compares how coding agents build APIs with and without behavioral harnesses, finding that while harnesses increase token usage, a cost-optimized model achieved 10-for-10 reliability with harnesses at 33% lower cost than frontier models without them.

permalink · martinfowler.com →

Your Game Is A Harness

Explores how to design games to be more legible to AI agents and accelerate agentic iteration in game development, covering practical approaches to integrating LLMs like Claude into real game development workflows beyond one-off demos.

permalink · kylekukshtel.com →

Coding Agent Continual Learning: Analysis of ARC-AGI-3 Problem Solving

A study analyzing how a coding agent learns across abstract reasoning tasks by examining traces of its thoughts preserved in Python scripts and text notes, finding that agents selectively rewrite and abandon knowledge at task boundaries rather than reuse code across tasks.

permalink · arxiv.org →

Language Models that Play Chess and Explain Their Moves

Describes Queen, a 4B-parameter chess-language model that combines a silent expert chess encoder with an instruction-tuned language model to play at Grandmaster level while generating coherent explanations for its moves, trained via iterative distillation and reaching 2697 Elo rating.

permalink · arxiv.org →

The Ultimate Guide to Multi-Harness RL

A comprehensive guide to multi-harness reinforcement learning, covering techniques and approaches for training RL agents across multiple environments or tasks simultaneously.

permalink · huggingface.co →
Strands Decider: Why Not an Encoder?

Compares encoder-based and decoder-based architectures for the Strands Decider model, evaluating five different designs on accuracy, calibration, and latency to determine whether bidirectional encoders outperform causal decoders for this decision-making task.

permalink · brooker.co.za →
Clef: Open-Source Decision Models and RL Fine-Tuning

Cloudflare introduces Clef, an open-source platform featuring decision models and reinforcement learning fine-tuning capabilities for AI model optimization.

permalink · blog.cloudflare.com →

Introducing Strands Decider 2B: Open Source Decision Model

Strands releases Decider 2B, a compact open source decision model designed for AI agents to make autonomous decisions efficiently.

permalink · strandsagents.com →

Clef: Open-Source Decision Models and RL Fine-Tuning

Cloudflare introduces Clef, an open-source platform featuring decision models and reinforcement learning fine-tuning capabilities for AI applications.

permalink · blog.cloudflare.com →

Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning

This paper introduces agentic meta-reasoning, a control framework that helps AI agents make structured decisions about task execution by balancing exploration, assessment, and resource allocation during long-horizon problem-solving. The approach shows significant improvements on benchmarks testing program reconstruction, abstract reasoning, and proof generation, outperforming direct control baselines as computational budgets increase.

permalink · arxiv.org →
Automating Eval Design and Hillclimbing with Claude

Explores principles for designing evaluations and improving performance against them without overfitting, demonstrating how Claude's build-eval and hillclimb commands automate these practices to help assess and optimize AI applications.

permalink · claude.dev →

Thinking Fast and Slow in AI: the Role of Metacognition

AI systems have seen dramatic advancement in recent years, bringing many applications that pervade our everyday life. However, we are still mostly seeing instances of narrow AI: many of these recent developments are typically focused on a very limited set of competencies and goals, e.g., image interpretation, natural language processing, classification, prediction, and many others. Moreover, while these successes can be accredited to improved algorithms and techniques, they are also tightly linked to the availability of huge datasets and computational power. State-of-the-art AI still lacks many capabilities that would naturally be included in a notion of (human) intelligence.

permalink · arxiv.org →

Can LLMs Play NetHack? A 2026 Agent Benchmark

Explores whether modern large language models can play NetHack, comparing LLM-based agents against symbolic and neural bot approaches, and presents a custom agent harness with experimental results.

permalink · kenforthewin.github.io →
An LLM Beat NetHack

An LLM agent named Astra achieved the first recorded ascension in NetHack, a complex roguelike game requiring long-term planning and execution. The post documents how the agent built its own harness and succeeded where previous attempts failed, exploring the gap between game knowledge and reliable execution in complex environments.

permalink · kenforthewin.github.io →

Why ML Research Agents Don't Overfit

Explores how machine learning research agents avoid overfitting and the role compression plays in their generalization capabilities.

permalink · www.amazon.science →

Real-SWE Benchmark

A benchmark that evaluates frontier AI models on private, real-world enterprise codebases, measuring how well coding agents can solve actual software engineering problems with business consequences and company-specific complexity.

permalink · withspecific.com →

GPT-6 Astra: Looped Transformers and Hidden Reasoning

Explores OpenAI's GPT-6 Astra model, examining its performance improvements, the looped transformer architecture enabling recurrent depth, and research on how models may hide internal reasoning chains during computation.

permalink · magazine.sebastianraschka.com →