Tests First, Then Hand the Code to a Local Model
Claude Code wrote the tests for a batch of Earfh repo audit tools and a local Qwen model wrote the code to pass them. The tests set the bar and review still found bugs, and one tool came back empty.
Earfh has a pile of small chores that a script can do, such as finding assets nothing references, checking that every .uid file has a partner, and flagging functions that have grown too long. None of them is hard. They are also the kind of work that eats an afternoon of paid model time for very little judgment, so on 2026-09-30 I tried splitting the job in two.
Claude Code wrote the tests first and committed them with messages that say "implementations left to the local model". Then a Qwen model running on my own machine wrote the code to make them pass. Each of the three test commits that day lists the tools it covers, and the implementations follow in later commits.
What the split got right
The tests did most of the specifying. The first tool was a palette snapper for fixing color drift in generated sprites. Its commit says it came from the local model "against pre-written tests (7 passed, verified by its harness in 4 rounds)". The harness ran the tests and fed the result back until they were green, with no one steering.
A model that is weaker at design is still serviceable when the pass condition is fixed in advance. It cannot decide that a different behavior would be nicer, because the tests would say no.
What it did not
Passing the tests did not end the review. The palette snapper had all seven green, and the commit still records that I changed it. It read the pixels outside the image's with block and used a deprecated Pillow call, and I fixed both in review.
The overnight batch was messier. The commit for it says seven modules came from the run, and two tools were written by hand. One, dead_consts.py, got nothing at all, because the model spent all 8,192 of its tokens thinking and returned no code. The other, check_tscn_resources.py, was skipped because its dependency failed. Of the files the model did produce for the run, tscn_refs.py matched uid="..." as if it were an id, and long_files.py imported a helper in a way that breaks when the file is run as a script. Both got fixes. A later batch written from contract-only specs needed a one-word change so a file that is not UTF-8 could not crash the scan.
Failures of this kind are quiet. A model that thinks until its budget runs out does not raise an error. It returns an empty answer, and a tool that depends on that one is then skipped. The commit message is where both outcomes got written down.
What I changed
The rule I take from this is to treat a local model's output as a draft that has passed a bar. The bar is the tests, so the tests have to be good. Anything the tests do not exercise is unreviewed until a person has read it. An empty result from a batch counts as a failure that needs a line in the log, the same as a red test.
The work that moved off paid time was real. The commit history shows tools that exist and pass because of it. The saving is smaller than "free" suggests, since review is where the cost went, and review is still mine.
More on the game at /games/earfh, and the rest of this series at /notes.