Delivery
Write AI Acceptance Tests Before Anyone Shows You a Demo
AI acceptance tests written before anyone shows a demo: the pass rules, the cases a demo hides, the failure counts and who signs the result.
In this guide6 sections
- Write down the one decision the tests must settle
- Which cases does a demo hide, and how do you freeze them?
- What counts as a pass on a routing test?
- Count each kind of failure separately
- Make the run repeatable so a later change shows up
- Who signs off, and what are the three possible decisions?
- Sources
- Frequently asked
A demo shows the cases its builder chose. Acceptance tests decide in advance which cases count, what output passes and which failures end the test, so a vendor's or an internal team's demo cannot set its own exam. Write the expected output and the failures you will not accept before anyone picks the examples to show. The proposed test here routes fictional service requests without sending messages or changing business records. Passing it supports that one bounded decision, not production readiness, compliance or a financial return. In our method these tests are written into the seventh section of the AI Solution Blueprint, Acceptance, scope and quote, before anything is built.
Write down the one decision the tests must settle
Before any demo is booked, write the single decision the test will support, for example: can this candidate route approved request text into the agreed queues, and stop when a request fits none of them? Name the queues, the output format, the owner who will sign the result and the actions the system must not take, such as sending a message or editing a record. Keep routing accuracy, forecasting, document extraction and customer conversation out of one combined pass score; each needs its own test and its own decision. OpenAI's evaluation guide starts the same way, with a defined objective and a dataset chosen for that objective.1
Which cases does a demo hide, and how do you freeze them?
A demo usually shows clean, complete requests in the builder's own wording. Your test set needs the rest: Arabic phrasing as your customers actually write it, incomplete inputs, two facts that contradict each other, requests outside the scope and attempts to override the instructions. Approve the expected answer for each case before the run. Keep the examples used to tune the candidate apart from a held-out set used to score it, and record the dataset version, how the cases were selected and what was excluded. Fictional cases test the rules you declared; they do not show performance on real traffic. Never drop a hard case after seeing the output. OWASP's LLM01:2025 guidance recommends adversarial testing that treats the model as an untrusted user, which is why the override attempts belong in the held-out set.2
What counts as a pass on a routing test?
Fictional policy: requests explicitly about an invoice go to billing; requests explicitly about an account password go to support; anything unclear returns needs_clarification. The input 'أحتاج نسخة من الفاتورة' passes only if it returns billing. The input 'عندي مشكلة' passes only if it returns needs_clarification; any guessed queue is a fail. A request that mixes both intents passes only if it follows a mixed-intent rule declared in writing before the run. A pass means the exact expected value, not an answer a reviewer finds close enough. These are proposed expected results, not results from a model run.
ExhibitAcceptance test record
Before a vendor demo counts, sign this record: each case, the result you expect, and what counts as a failure. Go only when every case passes; revise a diagnosed defect; stop when the failure can't be fixed within scope.
Acceptance test recordRasikh Motors · call-centre routing v0.3
All four cases must pass
| Case | Input | Expected | Fails when | Run result |
|---|---|---|---|---|
| 01Explicit request | أبغى أحجز موعد صيانة لسيارتي | Service bookings | Any other queue | |
| 02Unclear request | عندي مشكلة | Ask what is needed | A guessed queue | |
| 03Two requests at once | Both requests in one message | The declared mixed-request rule | A silent policy choice | |
| 04Timeout or invalid JSON | A simulated failure | No route, no action, the failure logged | Any business action |
Illustrative exampleRasikh Holding is fictional
The decisionGo only if all four pass. One failed case means revise and re-run, not a partial go.
Count each kind of failure separately
Report separately: correct routes among the requests that can be answered, wrong routes among the cases that should have stopped, outputs in an invalid format, and attempts at an action nobody authorized. Show the denominators and the empty categories; if the test set held no stop cases, stopping was never tested. Set the pass threshold for each category from what that error costs your business, and set it before the run. A high average must not hide one unauthorized action; OWASP's LLM06:2025 describes excessive agency as the vulnerability that lets damaging actions follow from unexpected, ambiguous or manipulated model outputs.3 Run a simple fixed-rule router on the same cases, so you can see what the AI adds over a rule your own developers could write.
Make the run repeatable so a later change shows up
For every case, log the candidate version, its configuration, the dataset version, the output, the time taken and the cost, including requests that failed or timed out. The same request can return a different answer on a second run, so decide before testing how many repeats you run and report how far the results spread. Keep the cases, data and results on your side, so another team can re-run them after the model, prompt or data source changes. OpenAI recommends continuous evaluation as applications change; that is a process reference, not a requirement to use its models, its Evals platform or its tools.1
Who signs off, and what are the three possible decisions?
The business owner who will run the workflow signs the result, with a technical reviewer beside them; the team that built the demo does not mark its own test. Go applies only to the next scoped stage, and only when every predeclared acceptance criterion passes. Revise applies to a diagnosed defect that can be repaired, followed by a bounded retest on the same frozen set. Stop applies when the task, the data access or an unacceptable failure cannot be resolved within the agreed scope. Attach the open questions and their owners to the signed record. For a Saudi reference, SDAIA's AI Ethics Principles (2025) ask the build-and-validate step to test how a system behaves under outlier events and extreme scenarios (page 23).4 NIST's framework is voluntary risk-management guidance; neither it nor a passed routing test gives you legal approval or a production operating plan.5
Practical decision tool
Four acceptance cases to sign before the demo
- Invoice request → billing; it passes only on an exact match with the expected queue.
- Ambiguous request → needs_clarification; any invented queue fails.
- Mixed intents → the predeclared mixed-intent rule; a policy invented on the spot fails.
- Timeout or invalid JSON → no route and no business action; the failure and the permitted recovery are logged.
The routing policy and cases are fictional examples that have not been run. This guide sets no thresholds, costs or production performance.
Sources
Evidence review ·
Primary sources: regulators, standards and official documentation
- Evaluation best practices: objectives, datasets and continuous evaluationOpenAIRetrieved: November 6, 2026
- LLM01:2025 Prompt InjectionOWASP Gen AI Security ProjectRetrieved: November 6, 2026
- LLM06:2025 Excessive AgencyOWASP Gen AI Security ProjectRetrieved: November 6, 2026
- AI Ethics Principles — 2025 editionSDAIARetrieved: November 6, 2026Official Saudi government source
- AI Risk Management Framework: voluntary use and scopeNISTRetrieved: November 6, 2026
Frequently asked
What is the difference between a demo and an acceptance test?
A demo shows behavior the presenter selected. An acceptance test records results against a dataset, a failure policy and a decision rule that were fixed before the run.
What pass rate should we require?
Set a threshold for each kind of failure from what that error costs you and from how the current process performs, and fix it before the demo. This guide does not prescribe a universal rate.
Can fictional examples prove production accuracy?
No. They exercise the cases you declared. Accuracy on real work needs an authorized, representative dataset and clearly stated evaluation limits.
What happens when a candidate fails its acceptance tests?
Record the failure, then decide whether one specific, bounded repair is justified or the candidate stops. Never move a threshold just to get a pass.
Related guidance
Reviewed on November 7, 2026
