← All Solo, Not Alone posts

ai-coding process

The Shotgun That Fired One Pellet

Earfh's shotgun spec called for a 25 degree cone. The build fired a 100 degree one, and the tests agreed with it, because the tests were written from the same misreading. It passed two reviews before anyone went back to the spec text.

For one task of Earfh's weapon system build, the shotgun fired a single pellet. The agent running the build had wired a stopgap into shoot() so the player script would keep compiling between two tasks, and review read it and called it "a straight, safe substitution." It ignored the weapon's own projectile_count. The next task's tests failed on it before any new code went in, and the build ledger calls it "the branch's clearest evidence that reviewing by reading is not the same as testing."

That was the easy bug. The one after it was harder to see, because the tests were in on it.

Once the shotgun fired five pellets, they fanned out across 100 degrees. With the Synergy Stack powerup raising the count to seven, 150 degrees. That is a shotgun firing past your own shoulder.

What the spec said

The weapon system design doc was clear. Line 64 reads "Shotgun | 5 shots, 25 degree cone". Line 373 says shoot() fires projectiles "whose directions span spread_degrees centered on last_direction." Both treat spread_degrees as the width of the whole cone, and the shotgun's resource file set it to 25.

The plan built from that spec treated it as the gap between neighboring pellets. Five pellets have four gaps. Four times 25 is 100.

The plan's test then asserted the outermost pellets at -50 and +50 degrees. The code matched the test. Review checked the code against the test. Every stage agreed with the stage before it, and the whole chain rested on one misreading made before a line of code existed. The test's own failure message said the outermost pellet "sits at half the total spread," which was the spec's meaning. That phrase sat next to a number that contradicted it. The controller's ledger entry puts it plainly: "I specified that fan deliberately without picturing it."

How it was caught

The controller had a doubt about the fan and raised it to the reviewer. The reviewer came back with no code defects, which was accurate, since the code did what the plan said. Then the controller stopped treating it as a tuning question and reread the spec. The ledger records the finding as "SPEC VIOLATION found, and it is mine," and explains why it survived so long: "The plan's own test assertions (-50 and +50 outermost) encoded my misreading and locked it in, which is why it passed review twice." It also concedes the field name was part of the problem. spread_degrees reads fine under either meaning.

The fix landed ten minutes after the wrong fan was committed. The step between pellets became the spread divided by one less than the pellet count, so the outermost pellets always sit at half the spread on either side. A new test pins the cone span at 25 degrees for both five and seven pellets. That test states the spec's sentence in numbers instead of restating the formula, and the re-reviewer confirmed the old formula yields 100 and 150 degrees there, far outside tolerance, so a revert fails it. As a side effect, more pellets now make the cone denser instead of wider. The powerup became an upgrade instead of a penalty.

Why this is a solo problem

With one person and an agent, a spec has one human reader, and the agent's plan is the second reading. If the plan misreads it, every downstream check inherits the misreading. Tests derived from the plan will pass. A review that compares code to plan will pass. Nobody else on the project is going to read the design doc cold and say that 25 degrees does not mean the gap.

What I take from this branch is narrow and practical. A clean review means the code is consistent with the plan. It says nothing about whether the plan is consistent with the spec. For any number the spec states outright, at least one test should assert that number directly, written from the spec's words, so the plan cannot quietly reinterpret it. And when a field name could mean two things, rename it before it ships two meanings.

The bug never reached a player. It was caught the same afternoon, on a branch, by an agent that went back to the source text. That step is the one I cannot assume will happen on its own, so it belongs in the process on purpose.

More on the game at /games/earfh, and the rest of the series at /notes.

← All Solo, Not Alone posts