A customer asks whether an extra service is included. Your AI assistant says yes and links to a company policy. The account manager confirms it. Then someone discovers the policy was replaced.
In this hypothetical scenario, the AI did not invent a document. It found a genuine one that was wrong for the situation. The result could be an unpriced commitment, a disputed quotation or an unhappy customer.
For Australian businesses, the question is not just, “Does our AI provide references?” It is, “Who decides which information it is allowed to rely on?”
The source is real. The advice is wrong.
Many business AI tools use retrieval-augmented generation, or RAG. The application searches connected documents, supplies relevant passages to the model and generates an answer, often with citations.
The gap: matching a question is not the same as establishing which policy applies.
- Old versions can look relevant. A retired policy may use almost identical language to its replacement.
- Drafts can sound authoritative. A clear explanation is not evidence of business approval.
- A citation proves less than it appears. It identifies a source, not necessarily the right version or terms for that customer.
A more fluent model will not automatically fix the source-selection problem.
You updated the policy. Did your AI catch up?
Your assistant may search an index containing extracted passages rather than read the original document afresh. Updating a shared-drive file does not, by itself, prove those passages have changed.
Ask your provider to demonstrate:
- How approved replacements reach the retrieval system, and how long that takes.
- Whether withdrawn content becomes ineligible, including indexed copies.
- Whether access changes carry through, so confidential documents remain restricted.
Connector capabilities vary. An archive folder, a renamed file or a prompt saying “use the latest policy” is not proof that the control works.
The newest policy may still be wrong for this customer
New terms may apply to new customers while an existing customer remains on an earlier signed agreement. “Most recently uploaded” is not a reliable rule for deciding contractual applicability.
- Commercial owners should define applicable terms and exceptions, with legal advice where needed.
- Document owners should maintain approval status, version and effective dates.
- Technical teams should enforce supported retrieval rules and preserve access controls.
Where the application cannot establish applicability, escalation may be safer than a confident answer. Generating advice does not itself authorise changing an agreement.
Before your AI makes a promise, test the awkward questions
Do not approve a rollout because straightforward demonstrations look convincing. Test the cases most likely to create a customer dispute:
- Two policies give conflicting answers.
- A withdrawn document remains in the search results.
- An older agreement still governs an existing customer.
- The question lacks enough customer context to choose correctly.
Have the business owner define the expected answer or escalation independently. Then verify both the retrieved evidence and the final response. A high search score is not proof that a policy applies.
Retain enough protected diagnostic information to identify the source version used. A link to a document that later changes may not explain an earlier answer. Avoid unnecessarily logging sensitive customer material.
Customers already received the wrong advice? Fix more than the document.
Pause the affected answer category or require review. Identify responses that relied on questionable material, preserve relevant versions and have the accountable owner assess corrections. Updating the knowledge base does not repair quotations or messages already sent.
The National AI Centre highlights timeliness, traceability and ownership. The ACCC identifies AI-related consumer risks. Specific legal consequences depend on the circumstances, not simply on whether AI was involved.
How Xpansion can support your next step
Xpansion Technologies can help review AI retrieval, document-update flows, access controls and escalation tests. Your authorised commercial owners should still approve which terms apply.
The takeaway: a credible reference is a starting point. Before customers rely on the answer, establish who owns its authority and how the system checks it.
References
- National AI Centre: Data quality checklist
- ACCC: AI developments and consumer risks, December 2025
- Microsoft Learn: RAG and indexes, technical vendor guidance
Sources checked 3 November 2026. Scenario illustrative. General technical and business-risk information, not legal advice; implementations and obligations vary.
Leave a comment