mattalcock.com

· AI Products

Your PRD template is wrong for agents

User stories, acceptance criteria, success metrics. Every section assumes software does the same thing every time. Agents don't.

User stories, acceptance criteria, success metrics. Every section of the PRD template we all learned assumes the same thing: if you build it right, it does the same thing every time.

Agents don't. The same input can produce a different output, the agent decides its own steps, and the most important behaviour is what it does when nothing in the spec applies. Write an agent spec with a feature template and you get a document that reads well and tells engineering almost nothing.

Where the template breaks

  • User stories describe a path. Agents choose their own path. "As a user I want to cancel my subscription" is fine for a button. For an agent, the hard part is the forty ways a person might ask, and the ten things it should check before it cancels anything.
  • Acceptance criteria assume a right answer. For most agent tasks there's a range of acceptable answers and a set of unacceptable ones. You need both, written down.
  • Success metrics measure the happy path. The metric that matters for an agent is often a failure rate: confident wrong answers, actions taken that shouldn't have been.

What to write instead: the scope doc

Before the PRD, write one page that defines the agent's world.

  1. What it can see. Which data, which tools, which history.
  2. What it can do. Every action, with the rung of autonomy each starts on (see Earned autonomy).
  3. What it must never do. The short list that matters most.
  4. When it must stop and ask. The triggers for handing back to a person.
  5. How we'll know it's good. An eval set: real examples with acceptable and unacceptable answers, run before every change.

Then the PRD gets shorter

With the scope doc in place, the PRD can focus on what a PRD is good at: the problem, the users, why now and what success looks like for the business. The behaviour lives in the scope doc and the eval set, where engineering can test against it.

← All writingRSS