I’ve spent roughly 12 years as a technical writer and over 12 years in product management. Across both, my work has always centered on documentation: end-user docs after engineers ship something new, and product requirements specs that tell engineers what to build. Writing specs for AI agents felt familiar the first time I tried it. Getting a spec to actually stick across sessions took me longer to get right.
Here’s what changed my thinking: I recently caught a real gap in PR #92. The spec for this content pipeline says that once an article’s PR merges, its row in the content calendar gets updated from “Queued” to “Published.” That instruction is clear. It’s written down in MILESTONES.md.
And it still got missed, not once, but twice, for two different articles, in two different sessions.
That’s the core problem with spec writing for AI agents: a spec that states a task is not the same as a spec that verifies the task happened. If you want an AI agent to actually follow your spec across sessions, you have to build the check into the spec, not just the instruction.
Why did a clear instruction still get missed twice?
The instruction to update the content calendar wasn’t vague. It named the file, the field, and the trigger condition: when a PR merges, flip the status. By any normal reading, that’s a good spec.
What it lacked was a way for a fresh session to verify the check had actually happened. Claude Code’s own documentation on memory files describes something worth sitting with: CLAUDE.md and similar files are loaded as context, not enforced as policy. The agent reads the instruction each session, but nothing forces it to cross-reference “did I do the thing I was told to do” against the actual repo state before moving on.
That’s exactly what happened here. Article C8 published, and its calendar row stayed marked “Queued” until someone noticed later. The same thing happened again with article C2. Two separate sessions, two separate agents, the same spec, the same silent gap.
I clearly communicate expectations in every spec I write, and I still hit this. The lesson wasn’t “write a clearer instruction.” The instruction was already clear. The lesson was that a spec needs its own built-in way to catch when it’s been skipped.
What does a self-verifying spec actually look like?
A self-verifying spec doesn’t just say what to do. It says how a fresh session, one with no memory of the last one, checks that the task was actually done.
For the content calendar problem, that means the spec shouldn’t just say “update the status after merge.” It should say something closer to: “before ending a content session, diff the calendar table against the list of merged content PRs; if any published article’s row still says Queued, fix it before closing out.” That’s a check the next session can run cold, without having to trust that a previous session did its job.
This distinction matters because AI agents don’t carry the same kind of continuity a person does. A developer who forgets to update a spreadsheet might remember two days later in the shower. An AI agent starting a new session has no shower moment.
It has whatever’s written down, and nothing else. If the verification step isn’t written down, it doesn’t exist.
I’ve found real value in using AI to help poke holes in my own specs before this ever becomes a problem in production. It’s worth going a step further: have the agent draft the verification step for its own task, not just the task itself. If it can’t describe how a future session would confirm the work was done, the spec isn’t finished yet.
How does agile practice fit into spec writing for AI agents?
I agree with spec writing as a discipline, and I also believe in applying agile practices to how AI agents work, not just how human teams do. In practice this looks like research happening in parallel with drafting, agents assigned to specific slices of a problem so they can work on it together. Then, critically, agents review each other’s output instead of a single agent grading its own work.
That last part is the same principle probl.me’s own multi-agent content pipeline applies to content and code: the agent that writes something is never the only one that approves it. A Writer Agent doesn’t get to also be its own SEO Reviewer, and a Developer Agent doesn’t get to be its own Code Reviewer. It’s genuinely interesting to watch two agents push back on each other’s work and refine it further, and it catches things a single pass, human or AI, tends to miss.
Self-verification and independent review aren’t competing approaches. They’re two different layers. A self-check catches the agent skipping its own step. Only a separate reviewing agent catches the self-check itself being wrong, because a self-check can only find the kinds of mistakes the same model is capable of noticing in the first place.
A widely-read Hacker News comment on a Claude Code thread put the same idea more bluntly: orchestration setups aren’t the point, “validation is the bottleneck.” I’d narrow that slightly. Validation is one bottleneck. The other is whether the spec even wrote down what validation should check for in the first place.
Are there tools already doing this for you?
Some of it, yes. GitHub’s Spec-Kit formalizes a Spec → Plan → Tasks → Code workflow, so the spec becomes a real artifact the agent works from at each stage instead of a paragraph typed once and forgotten. AWS’s Kiro goes a step further with “Agent Hooks,” automated checks that fire when a file changes, enforcing consistency between what the spec says and what the code actually does.
Those are useful analogs, but I’d be careful not to treat tooling as a substitute for the habit. Addy Osmani’s widely-cited guide to writing specs for AI agents already recommends having the agent verify its own work against the spec, and it’s one of the most thorough references out there.
What I haven’t seen addressed as directly is the failure mode I hit. The same well-written instruction got quietly skipped more than once, in separate sessions, because the check itself was never written down as its own step. You don’t need Spec-Kit or Kiro to fix that. The verification step needs to live inside the spec you’re already writing.
Isn’t this all overkill for a quick prototype?
Not every task needs this level of structure. If you’re spinning up a one-off script or a disposable prototype, a full spec with a built-in verification step is probably more ceremony than the work deserves. The tradeoff shows up in the comparison people usually draw between “vibe coding” and spec-driven development: vibe coding feels faster because the first output arrives sooner, but that speed is front-loaded. Rework shows up later, in debugging, when nobody wrote down what “done” was supposed to look like.
The line I’d draw is durability. If a task lives inside one session and gets thrown away, skip the ceremony. But if the outcome needs to survive into a future session, a future PR, or a future person reading it, write the check. The 10 extra minutes it takes is cheaper than finding the same gap twice.
There’s a fair pushback worth naming directly: some engineers argue that persistent instruction files and drift-prevention tooling manufacture a problem to sell a solution, that good project structure should already prevent this. I’d have agreed with that before PR #92.
The instruction existed. It was well-structured, and it still failed twice. That’s not a hypothetical. It’s the exact failure mode the pushback assumes shouldn’t happen.
It’s also worth being honest about the limits here. Self-verification narrows the failure surface. It doesn’t close it. A check written by the same kind of model doing the work can miss the same category of error the model is prone to making in the first place.
That’s an argument for pairing self-checks with independent review, not an argument against writing the check at all.
What would I tell someone starting from zero?
Write your spec so the AI’s next self can verify compliance without re-reading the whole conversation. That’s the single habit that would have caught the C2 and C8 gap the first time, not the second.
Concretely: for every instruction in a spec that a future session needs to follow, ask “how would a fresh session, with zero memory of right now, confirm this actually happened?” If you can’t answer that in one or two sentences, the spec isn’t done. You’ve written a task. You haven’t written a way to catch when the task gets missed.
This isn’t a one-time fix, either. MILESTONES.md on this site has now been updated twice for the exact same class of gap. The next version of that spec should include the verification step directly, so a third instance doesn’t happen quietly a third time.
Key Takeaways
A spec for an AI agent needs to do two things: describe the task, and describe how a future session checks the task was actually completed. Writing the instruction clearly isn’t enough on its own, because an AI agent has no continuity between sessions beyond what’s written down. Self-verification narrows failures but doesn’t eliminate them, which is why independent review by a separate agent still matters alongside it.
The real cost of skipping this isn’t dramatic. It’s quiet: the same small gap, missed twice, discovered only by accident.
My 12 years writing product specs and 12 years writing documentation taught me the same lesson from two different directions: a requirement nobody checks isn’t really a requirement. That’s just as true when the reader is an AI agent as when it was an engineer.
