What is mattwood.fyi?

A live, structured feed of riffs and links from Matt Wood, Chief AI & Technology Officer at AWS — readable by people and queryable by agents as a typed knowledge graph.

Is there an API and is it free?

Yes. A public, read-only query API is available with no authentication and no API keys. See https://mattwood.fyi/openapi.json and https://mattwood.fyi/docs.

How should an AI agent use this site?

Fetch https://mattwood.fyi/feed.json for the full structured feed, or query the API for keyword/semantic search and the connection graph. See https://mattwood.fyi/agents/ for when-to-use guidance.

StarSkirmish Bench v0.2: LLM StarCraft Bot Benchmark

Connections

Supports

Testing Behavioral Harnesses for AI Coding Agents Both use game/competitive environments as behavioral harnesses to test AI coding agents under constrained conditions, measuring real performance rather than self-reported capability
Coding Agent Continual Learning: Analysis of ARC-AGI-3 Problem Solving Both assess LLM coding and problem-solving capabilities through competitive benchmarks, with StarSkirmish using bot code generation and this analyzing ARC-AGI-3 problem solving by coding agents
Testing Behavioral Harnesses for AI Coding Agents StarSkirmish is essentially a behavioral harness for LLMs in a coding context — it constrains agents to write C++ bots within one hour and tests behavior through competitive outcomes, directly exemplifying the harness testing paradigm
Thinking Before Thinking: Scaling Agentic Inference Through Meta-Reasoning StarSkirmish's timed (1-hour) constraint implicitly tests meta-reasoning and inference scaling under pressure, providing empirical data relevant to agentic inference research
Automating Eval Design and Hillclimbing with Claude StarSkirmish's automated bot-vs-bot evaluation exemplifies the kind of automated eval design and hillclimbing methodology discussed in Claude's eval automation work
Your Game Is A Harness StarSkirmish treats the game (StarCraft) as a harness for evaluating LLM coding agents, directly embodying the 'your game is a harness' concept where game mechanics serve as evaluation infrastructure

Develops into

Your Game Is A Harness StarSkirmish operationalizes the 'game as harness' concept - using StarCraft as a rigorous evaluation harness for LLM coding ability with measurable win/loss outcomes

Related to

AAAI Research Paper 70113 AAAI research context is relevant as LLM game benchmarking is an active AI research area, and StarSkirmish-style evaluations are the kind of work appearing in AI conference venues
Language Models that Play Chess and Explain Their Moves Both benchmark LLMs on classic strategic games (StarCraft vs Chess), evaluating reasoning and decision-making capabilities through game performance rather than traditional NLP tasks
Automating Eval Design and Hillclimbing with Claude Both involve automated evaluation design for AI systems — StarSkirmish uses competitive game outcomes as evals while the other discusses automating eval design with Claude
Language Models that Play Chess and Explain Their Moves Both benchmark LLMs on strategic game tasks requiring reasoning and code generation - chess explanation vs. StarCraft bot writing - probing similar cognitive/planning capabilities
Coding Agent Continual Learning: Analysis of ARC-AGI-3 Problem Solving Both analyze LLM coding agent performance through structured benchmarks - ARC-AGI-3 for problem solving vs. StarCraft bot writing - contributing to understanding of coding agent capabilities and limits

Type:	link
First published:	Friday, 16 October 2026 at 15:59 UTC
Location:	https://starskirmish.com/bench/v0-2/
URL:	https://mattwood.fyi/i/313be558

I'm Matt Wood, and this is For Your Information. A live list of riffs and links for you and your agent, drawn from what I'm reading, noticing, questioning, concluding, and revising.

Connections