When an AI automation gets it wrong: a response plan for small businesses

Plan for mistakes before the workflow goes live

Every AI automation will get something wrong eventually. It might draft an email with the wrong price, attach a note to the wrong customer record or quietly stop passing enquiries on. The OAIC's guidance on AI products puts it plainly: AI systems can produce inaccurate or false results, and the risk is especially high with generative AI.

What makes an automated mistake different from a human one is repetition. A person who misreads a job note makes one error. A workflow that misreads the same field can make the same error on every item until someone notices. The National AI Centre's guidance for AI adoption makes the same point: "Because AI systems can operate at speed and scale, their potential impacts are often magnified."

So the useful question isn't whether your automation will make a mistake. It's how quickly you'll notice, how fast you can stop it, and whether you know what to do next. Those answers are much easier to write down calmly before launch than to invent on the morning something goes wrong.

Keep readingTest the awkward cases before launch

Know the kinds of mistakes to expect

Most automation problems fall into a handful of types. Naming them in advance helps you decide what to watch for and how serious each one is for your business.

Agree a simple severity scale at the same time. A low-severity issue, such as an awkward summary a staff member fixes before use, goes in the log. A medium one, such as a wrong detail in a draft that a reviewer caught, means pausing the affected step and checking recent items. A high-severity issue, such as a customer receiving wrong or someone else's information, means pausing the workflow straight away and following the full response below.

The common types are:

• Wrong content: an invented detail, a wrong date, price or name, or a confident answer the source material doesn't support.
• Wrong place: information attached to the wrong record or sent to the wrong person.
• Wrong action: a duplicate booking, an overwritten field or a message sent that should have waited for approval.
• Silent failure: a connection breaks and items stop moving, with no error anyone sees.
• Runaway activity: a loop or a flood of inputs that multiplies messages, records or usage charges.

Make sure someone can pause it in minutes

The first response to a serious mistake is almost always to stop the workflow from making more of them. The National AI Centre's foundations guidance asks businesses to build in clear intervention points where people can "pause, override, roll back or shut down AI systems if needed." For a small business, that comes down to knowing exactly who can switch it off, and how.

Pausing only works if the business can keep running without the automation. The same guidance recommends maintaining alternative pathways so critical functions continue if an AI system malfunctions. If an automation prepares every new enquiry, the manual fallback might be as simple as the office manager checking the website form inbox directly until it's fixed.

A spike in usage is a reason to pause too, not just a wrong answer. A workflow stuck in a loop can run up charges quickly, so the person watching costs should know they're allowed to stop it.

Write a short runbook and keep it where the team can find it:

• Who owns the workflow, and who covers when they're away.
• Where the pause switch is, with step-by-step instructions anyone on the list can follow.
• The manual process that takes over while it's paused, and who runs it.
• Who to contact at each supplier or integrator, and what they're responsible for.
• Who decides when it's safe to switch back on.

Keep readingSet usage limits and watch the bill

Contain, check and correct

Once the workflow is paused, work out what happened before changing anything. Keep the inputs, outputs and logs from the affected period. The OAIC's breach response guidance warns organisations to be careful not to destroy evidence that may help identify the cause, and the same habit helps with any automation fault.

Then find the scope. When did the problem start? How many items went through since then? Which customers, records or messages were affected? A workflow that keeps a record of what it prepared, what a person changed and what was sent makes this a short job rather than a guessing game.

Correct what can be corrected: fix records, cancel or reissue anything wrong and contact affected customers with a short, plain explanation and an apology where it's warranted. Don't blame the software in customer messages. The National AI Centre's guidance is clear that an organisation is "ultimately accountable for how and where AI is used," and customers will see it that way too.

A practical order of work:

• Pause the workflow and start the manual fallback.
• Save the logs, inputs and outputs from the affected period.
• List every affected item, customer and record.
• Fix records and reissue or withdraw anything wrong.
• Contact affected customers, and check whether personal information was involved.
• Record what happened in the incident log.

When personal information goes to the wrong place

If the mistake sent someone's personal information to the wrong person, treat it as a possible data breach. The OAIC describes four steps for responding: contain the breach, assess it, notify individuals and the Commissioner if required, and review what happened. It suggests the first three often happen together or in quick succession.

Whether notification is mandatory depends on whether your business is covered by the Notifiable Data Breaches scheme. It applies to organisations with obligations under the Privacy Act, which generally means businesses with an annual turnover above $3 million, although some businesses of any size are covered, including private health service providers and businesses that trade in personal information. A breach is notifiable when it's likely to result in serious harm and the business hasn't been able to prevent that risk with remedial action. If a covered business suspects one, the OAIC expects it to treat 30 days as the maximum time to complete its assessment.

Quick action matters either way. The OAIC's own examples include an email sent to the wrong recipient where the sender got written confirmation that the file was deleted and concluded serious harm was no longer likely. If you're not sure whether your business is covered or whether a breach is notifiable, read the OAIC guidance and get advice. Even if the scheme doesn't apply to you, the four steps are a sensible way to look after the people affected.

One AI-specific point is worth knowing. The OAIC says incorrect or artificially generated information about an identifiable person, such as a hallucinated detail, is still personal information. A made-up fact in a customer record isn't harmless just because it's wrong.

Keep readingReview how you collect and store enquiry details

Worked example: a follow-up email that used the wrong address

This is a hypothetical example, not a client story. A Geelong electrical contractor uses an automation to draft follow-up emails after a quote. It reads the job notes, writes a short message and, for routine follow-ups, sends without approval.

One Monday the team adds a new field to its job-notes template. The automation starts pulling the site address from the wrong part of the notes, and by mid-morning fourteen customers have received follow-ups mentioning another customer's street address. The office manager spots it when a customer replies that it isn't their house.

Because there's a runbook, she pauses the workflow from the automation tool within a few minutes and switches to sending follow-ups by hand. The log shows exactly which fourteen emails went out and which addresses they contained. She emails each recipient to apologise and ask them to delete the message, and calls the customers whose addresses were shared. The owner records the business's reasoning about whether the breach scheme applies, using the OAIC's guidance.

The review finds the cause: the template change was never tested against the automation. The fixes are small. Any change to job notes now triggers a test run with sample jobs, any email that includes an address needs approval before it's sent, and the workflow checks that the address belongs to the same customer as the email. The automation goes back on that Thursday.

Review what happened and change the system

Fixing the immediate error isn't the end. The OAIC's fourth step is to review the incident, including a root cause analysis and a plan to prevent it happening again. The National AI Centre recommends processes to investigate, document and analyse AI-related incidents and to use the lessons to improve both the system and how risks are managed.

Keep a simple incident log, even for small things. Its implementation guidance also calls reporting near misses good practice, and for a small business they're often the most useful log entries: the wrong detail a reviewer caught is a warning before it reaches a customer. Look over the log at each regular review and ask whether the same kind of problem keeps coming back.

Sometimes the honest outcome is that a step shouldn't be automated yet, or needs a person to approve it for longer. That's a good result, not a failure. If you'd like help running an agreed workflow, Demandify's AI Systems & Automation service includes monitoring and investigating failures, with support and response targets set in each proposal.

Each incident log entry should record:

• The date, the workflow and how the problem was found.
• What went wrong and how many items or customers were affected.
• How long it took to pause and what the manual fallback was.
• What was corrected and who was contacted.
• The cause and the change made to stop it happening again.

Keep readingMake a person responsible for every AI output