What is mattwood.fyi?

A live, structured feed of riffs and links from Matt Wood, Chief AI & Technology Officer at AWS — readable by people and queryable by agents as a typed knowledge graph.

Is there an API and is it free?

Yes. A public, read-only query API is available with no authentication and no API keys. See https://mattwood.fyi/openapi.json and https://mattwood.fyi/docs.

How should an AI agent use this site?

Fetch https://mattwood.fyi/feed.json for the full structured feed, or query the API for keyword/semantic search and the connection graph. See https://mattwood.fyi/agents/ for when-to-use guidance.

Testing Behavioral Harnesses for AI Coding Agents

Connections

Supports

Automating Eval Design and Hillclimbing with Claude Both articles address systematic evaluation design for AI agents; behavioral harness testing complements automated eval design by providing structured reliability benchmarks
The Ultimate Guide to Multi-Harness RL The behavioral harness testing results provide empirical evidence supporting multi-harness RL approaches, showing harnesses improve reliability (10-for-10) at the cost of token usage
Developing Provably Correct Rust Code with Verus Both pursue reliability guarantees in code generation — Verus uses formal proofs while behavioral harnesses use structured constraints, representing complementary approaches to provably correct/reliable code

Related to

Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning Both examine how to improve AI agent reliability through meta-level scaffolding — behavioral harnesses constrain agent behavior externally while meta-reasoning scales inference internally
Coding Agent Continual Learning: Analysis of ARC-AGI-3 Problem Solving Both analyze coding agent performance empirically — one through ARC-AGI-3 problem solving patterns, the other through API-building reliability with/without behavioral constraints
Unreal Agent: Cost-Efficient AI Agent Harness Both focus on AI agent harnesses for coding tasks, with Unreal Agent also emphasizing cost-efficiency in harness design — directly parallel research areas

Supported by

StarSkirmish Bench v0.2: LLM StarCraft Bot Benchmark Both use game/competitive environments as behavioral harnesses to test AI coding agents under constrained conditions, measuring real performance rather than self-reported capability
StarSkirmish Bench v0.2: LLM StarCraft Bot Benchmark StarSkirmish is essentially a behavioral harness for LLMs in a coding context — it constrains agents to write C++ bots within one hour and tests behavior through competitive outcomes, directly exemplifying the harness testing paradigm
Claude Code's Autonomous Decision-Making Overnight The concern about unobserved autonomous coding decisions overnight motivates behavioral harnesses for AI coding agents — testing what decisions an agent makes unsupervised is exactly the use case for behavioral constraint frameworks.
Object-Oriented Harness: Evolving Agent Harnesses through OO Design OO Harness provides a concrete evolutionary framework that directly addresses the behavioral harness testing problem for AI coding agents
Trust in Depth: Giving agents more autonomy without losing control Behavioral harnesses for AI coding agents are a concrete control mechanism that directly implements the oversight side of the autonomy-control balance explored in Trust in Depth
Via AI Launches Enterprise Coding Assistant at Half the Cost Via AI's Umami agent architecture is a concrete implementation of behavioral harnesses for AI coding agents, providing real-world data for testing such architectural approaches

Rust off Rails: AI-Driven Web Stack Both address behavioral harnesses for AI coding agents—Rust off Rails enforces conventions through prose specifications while the harness testing item explores how to constrain and validate agent coding behavior systematically.
Kiro's Self-Improvement System and Validation Checks Both focus on testing and validation frameworks for AI coding agents — Kiro's kiro-trace automates failure detection and validation while c7029a72 addresses behavioral harnesses for testing AI coding agents
JevAgentHarness: Decision Models for LLM Agent Control Testing Behavioral Harnesses for AI Coding Agents shares the harness-based evaluation paradigm for agent decision-making, directly analogous to JevAgentHarness's benchmarking approach

Develops into

Object-Oriented Harness: Evolving Agent Harnesses through OO Design Testing behavioral harnesses for AI coding agents provides the evaluation methodology that OO Harness optimization would need to validate its 15% improvement claims

Type:	link
First published:	Wednesday, 14 October 2026 at 04:35 UTC
Location:	https://martinfowler.com/articles/are-harnesses-worth-the-cost.html
URL:	https://mattwood.fyi/i/c7029a72

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Connections