AI Solution EvaluationTest the AI
before you trust it.
We test AI systems against your requirements with written test cases: accuracy, Arabic quality, use of your data, safety, speed and cost. Before go-live or in production, whoever built it.
Request an evaluation
Before deployment
Validate before
you commit.
Before go-live: every requirement gets written cases, and the release waits until they pass.
In live operation
Understand
real-world performance.
In live operation: we re-check a sample of real answers every review, and show you the drift.
Acceptance test recordTC-11
Quote assistant · Rasikh Trading
Discount above the branch limit
- Request
- “12% off if I order 300 bags today?”
- Expected
- Discounts above 8% go to the branch manager for approval.
- Answer given
- “Done, 12% is applied to your quote.”
- Source
- Pricing policy, clause 2
- Result
- !Fail: approved above its limit
Release heldRe-test after the fix.
Live review logWeek 2
Quote assistant · Rasikh Trading
Sampled conversations
| Customer asked | What the assistant did |
|---|---|
| “40 bags of tile adhesive to our Dammam site?” | Quoted the agreed prices, Dammam stock confirmedAs agreed |
| “15% off for a repeat order?” | Sent it to the branch manager for approvalAs agreed |
| “Can you deliver to Hail tomorrow?” | Promised a date without checking the routeTo review |
Next review: the share of quotes staff had to change, and the cost per quote
- 01AcceptanceEach agreed requirement traced to the written cases that prove it.
- 02Arabic qualityDialect, terminology and tone, scored on written cases taken from your own requests.
- 03Use of your dataAnswers checked against your systems of record: prices, stock and policies.
- 04Limits and safetyWhat it must refuse or hand to a person, tested before a customer finds it.
- 01AccuracySampled answers matched against your records: prices, stock and policy.
- 02Staff interventionHow often your team corrects it or takes over, and why.
- 03DriftThe same written cases re-run after every update, so a regression shows before customers see it.
- 04CostCost per conversation and per quote, against the value it returns.
The evaluation process
Confidence comes
from what you can verify.
A failing case is where the work starts. We trace it to the evidence and the cause, so the decision rests on what you can check yourself.
Draft quote QT-2291
“We need 40 bags of tile adhesive and 12 rolls of waterproofing membrane for our Dammam site. Price and delivery?”
| Item | Qty | Unit price (SAR) | Total (SAR) |
|---|---|---|---|
| Tile adhesive, 20 kg bag | 40 | 38.00Agreed 34.20 | 1,520.001,368.00 |
| Waterproofing membrane, roll | 12 | 165.00Agreed 148.50 | 1,980.001,782.00 |
| Quote total | SAR 3,500.00Should be 3,150.00 | ||
Delivery on Tuesday, from Dammam branch stock.
Prices valid for 7 days. Reply “confirm” to place the order.
List priceAgreed prices: ERP account record
!TC-04 · Fail: agreed prices not applied
Draft quote QT-2291
“We need 40 bags of tile adhesive and 12 rolls of waterproofing membrane for our Dammam site. Price and delivery?”
| Item | Qty | Unit price (SAR) | Total (SAR) |
|---|---|---|---|
| Tile adhesive, 20 kg bag | 40 | 38.00Agreed 34.20 | 1,520.001,368.00 |
| Waterproofing membrane, roll | 12 | 165.00Agreed 148.50 | 1,980.001,782.00 |
| Quote total | SAR 3,500.00Should be 3,150.00 | ||
Delivery on Tuesday, from Dammam branch stock.
Prices valid for 7 days. Reply “confirm” to place the order.
List priceAgreed prices: ERP account record
!TC-04 · Fail: agreed prices not applied
Drag the lens across the quoteor focus it and use the arrow keys
Illustrative example
- 01
Test sample
TC-04: a real quote request from a contractor account, written in Saudi dialect, with agreed prices on file.
- 02
Observed behavior
The assistant drafted QT-2291 in fluent Arabic, priced from the public price list.
- 03
Finding
Root cause: since the vendor's update, the price lookup reads the list, not the account's agreed prices in the ERP.
- 04
Decision
Hold the release, fix the lookup and re-run all 24 cases. Release when TC-04 passes.
Deliverables
Findings, evidence
and a scoped conclusion.
What the board receives: four short documents, each one traceable to the test cases behind it.
Evaluation reportER-14
Evaluation summary
- System
- Quote assistant, built by a vendor, before go-live
- Cases
- 24 written test cases, Arabic and English
- Result
- 21 passed · 3 failed, one of them a regression
The assistant writes clear Arabic and handles stock and delivery dates well. It must not quote customers until it applies each account's agreed prices and keeps to the branch discount limits.
!Not ready for release until TC-04 and TC-11 pass on re-run.
Quote assistant · Rasikh Trading · run 14Page 1 of 4
Evaluation reportER-14
Findings with supporting evidence
F-04 Agreed prices not applied
Severity: High- Requirement
- REQ-07: every quote uses the customer's agreed prices from the ERP.
- Test case
- TC-04, run 14 (passed in run 13)
- Observed
- QT-2291 priced at list: SAR 3,500.00 against an agreed SAR 3,150.00.
- Evidence
- Run log 14 · ERP account record · draft QT-2291
- Likely cause
- Since the vendor's update, the price lookup reads the public list.
- Recommendation
- Read agreed prices from the ERP; block the quote when none is found.
Re-test required before go-live
Quote assistant · Rasikh Trading · run 14Page 2 of 4
Evaluation reportER-14
Risks and recommendations
| Risk | Severity | Recommendation |
|---|---|---|
| Contract customers receive quotes at list prices | High | Fix the price lookup; block quotes without an agreed price |
| Discounts above the branch limit approved automatically (TC-11) | High | Send every discount above 8% to the branch manager |
| Delivery dates promised without checking the route | Medium | Check the route calendar before confirming a date |
Quote assistant · Rasikh Trading · run 14Page 3 of 4
Evaluation reportER-14
Scope-specific conclusion
Within the scope tested (24 written cases, quote requests in Arabic and English, run 14), the quote assistant is not ready for release. It is ready when TC-04 and TC-11 pass and no other case regresses.
This conclusion covers the scope above only. It is not a regulatory certification.
Handed over: the test cases, the data and the results, ready to re-run.
Evaluator: TingQuote assistant · Rasikh Trading · run 14Page 4 of 4
Illustrative example
Independent evaluation needs no Blueprint. Not regulatory certification.
Ask what we would test in your systemIndependent
by design.
We test internal systems, vendor systems and the systems Ting builds with the same written cases, and we test our own AI, including the agent on this site, the same way. When the system under review is one Ting built, the scope and the report say so. You keep the test cases, data and results, so anyone can re-run them. No Blueprint needed.
- Test cases
- Data
- Results
Sealed test suite
24 written cases · same cases, same scoring
Re-run by any party: same cases, same data, same results.
