Blog

Same spec, same behavior on Monday and Friday?

Illustration of a person with a clipboard beside a large compass-like mechanism whose arrow points from scattered code fragments to a tablet labelled Spec.

If the specification is the new source of truth, then it should give you the same behavior on Monday and on Friday. Does it?๐Ÿงญ

I've been following Andrey Breslav and his project CodeSpeak with great interest, and recently read about their feature that lets the tool turn spec diffs into code diffs incrementally. Very cool direction. My own intuition has been a bit different: when rebuilding gets cheap, you just regenerate the whole codebase from scratch. Two different paths, same underlying shift โ€” the spec becomes the thing that matters, and code just catches up to the paper. ๐Ÿ“

Two questions keep me up at night.

First one โ€” determinism. If two teammates build from the same unchanged spec, will they get identical behaviour? Because if the answer is "kind of, mostly, usually", then the spec isn't really the source of truth yet. The generated code is. Because only code guarantees consistent behaviour โ€” spec doesn't.

And that means we still need to verify the code, still need code reviews to understand what's actually in there. That's 75% of a developer's time โ€” even in the AI era. The human stays the bottleneck, and the whole process doesn't actually move faster. ๐Ÿข

Second one โ€” who validates whom? When the same LLM generates both the code and the tests from the same spec, how do you guard against it validating its own interpretation? Tests pass, the behavior looks right on the surface, and yet the actual intent quietly drifts. It's like asking a student to write the exam and grade it too, of course, they pass. ๐Ÿคท

I'm not saying this is unsolvable. Quite the opposite: I'm convinced it's solvable, and that's exactly what I'm working on right now. The real question is: what's the missing piece? What do we need to add to this system so that behavior stays consistent between Monday and Friday, and so that validation doesn't depend on humans doing code reviews? ๐ŸŽฏ