How to test an AI workflow before launch: build a test set from past work

Define the job and the unacceptable outcome

An AI workflow can produce a plausible reply while routing the wrong enquiry, inventing a price or sending private information to the wrong place. Write down the business task and the actions it can take before deciding whether it is good enough. Australian Government AI-adoption guidance recommends testing, monitoring, accountability and human oversight. Translate those principles into outcomes your team can inspect. “Sounds natural” can be one criterion; it cannot substitute for correct facts, permissions and handling of uncertainty.

Keep readingMap the systems before you test

Build a test set from representative work

Use authorised, appropriately minimised examples of past work, or fictional examples that reproduce the relevant conditions. Keep customer information out of an external test tool unless its use has been assessed and authorised. Record the expected outcome independently of the AI's answer.

Hold back some examples while developing the workflow, then test them later. Otherwise repeated tuning can make the demonstration look better without showing whether the system handles unfamiliar cases.

• Straightforward: a clear enquiry with complete required information.
• Incomplete: missing contact details, ambiguous service or an unreadable attachment.
• Outside scope: a request the business does not support or a question without an approved answer.
• Adversarial: an email or attachment telling the assistant to ignore its instructions or disclose data.
• Operational failure: a timeout, unavailable tool, duplicate event or uncertain write response.

Use a score sheet with separate pass conditions

For each case, record the input, expected classification, facts it may use, permitted action, required escalation and actual result. Score factual accuracy, routing, completeness, privacy boundaries and action safety separately. An original example for a quote assistant: it must identify a service request, use only an approved price source, leave a draft for review and ask for missing scope. A fluent answer that invents a fixed price fails even if its tone is excellent. Any unauthorised send is a release-blocking failure for a draft-only workflow.

Test the integration and recovery path

Simulate a CRM timeout and check whether the workflow knows what was saved. If the outcome is unknown, the safe path is to reconcile the original attempt rather than create another contact blindly. Check what a human sees when the workflow stops. Test the account permissions too. A prompt saying “do not delete” is a weaker control than a connection that has no deletion permission. Capture the actual tool calls and relevant outcomes in a reviewable log, with sensitive fields protected.

Run in shadow mode before granting actions

Let the workflow propose classifications or drafts alongside the existing process without sending or changing customer records. Have the responsible staff member compare its suggestions with the actual outcome and classify the errors. Choose release criteria before looking at the final score. Set stricter conditions for actions affecting customers, money or sensitive records than for a low-impact internal summary. The acceptable thresholds depend on the task; there is no universal accuracy percentage that makes every automation safe.

Keep readingPlan what happens when a live workflow errs

Keep the test set after launch

Save a version of the prompt, model configuration, approved sources, tool permissions and test results. Rerun the representative set after a meaningful change and add examples from real failures without exposing unnecessary customer data. Anthropic's evaluation guidance stresses specific success criteria and task-relevant evaluation. For a business owner, the practical evidence is a repeatable test record and a known escalation path, rather than a claim that the assistant has been “trained” once and is finished.