What is the Data Team Test?
- Data Team Test
- The Data Team Test is twelve yes or no questions that tell you quickly whether a data team can trust its numbers. It's a Joel Test for data teams, first written in 2013 and updated in 2026 for teams where models and agents use the data too.
What are the twelve questions?
Yes or no, no partial credit.
- Is every metric that matters defined once, in code, with an owner?
- Can someone outside the data team answer a basic question without filing a ticket?
- Do you test your data like code?
- Do you find out data is broken before your stakeholders do?
- Does every experiment have its success metric and decision rule written down before it launches?
- Do you measure long-term holdouts, not just A/B test wins?
- Do data people sit in the product squads, not a service desk?
- Do you have evals for every model and prompt that ships?
- Can you trace an AI answer back to the data it used?
- Do agents get the same access controls and audit trail as people?
- Can you list the decisions your data changed last quarter?
- Can a new hire ship a useful analysis in their first week?
How do you score it?
| Score | What it means |
|---|---|
| 12 | Great |
| 11 | Fine |
| 10 or below | Something structural is wrong, usually in questions 1, 4 or 11 |
Fix the lowest-numbered "no" first. Metric definitions and data quality come before experimentation and AI.
What does it look like in practice?
Take question 1. If "active user" means three different things in three dashboards, nothing else on the list matters. Or question 11. The answer isn't the dashboards you shipped. It's the decisions your data changed.
Questions 8 to 10 are the AI-era ones. An assistant that answers from your data is only as trustworthy as the metric definitions, tests and evals underneath it.
Where does the idea come from?
Joel Spolsky published The Joel Test for software teams in 2000. I wrote a version for data teams in February 2013, while leading data analytics at NaturalMotion, and published it with the 2026 update in The Data Team Test, 13 years on. The 2013 list was about building a data team. The 2026 list is about data that everyone, and everything, can use without a data person in the room.
Related ideas
- The Product-Engineering Team Test. Twelve questions on whether a product team can turn what it learns into something shipped. In the same post.
- The Yes Machine. Confident wrong answers come from systems nobody checks. In the Yes Machine.
- Earned autonomy. How an agent earns the right to act on its own. In earned autonomy.
FAQ
What is the Data Team Test? The Data Team Test is twelve yes or no questions that tell you quickly whether a data team can trust its numbers. It's a Joel Test for data teams, first written in 2013 and updated in 2026 for teams where models and agents use the data too.
Is there a Joel Test for data teams? Yes. The Data Team Test applies Joel Spolsky's twelve yes or no format to data and analytics teams, covering metric ownership, data quality, experimentation, AI evals and the decisions data changed.
How do I use the Data Team Test? Answer each question as a team, honestly. The questions people disagree on are more useful than the score.
What changed between the 2013 and 2026 versions? The 2013 questions were about getting the data, like a warehouse that doesn't hit the live system. The 2026 questions are about trusting it, including evals, tracing AI answers to their data and access controls for agents.
To cite this, use “Data Team Test”, Matt Alcock, mattalcock.com/ideas/data-team-test.