Imagine your team uses an AI coding tool to build a customer portal. Customers can sign in, download invoices and check job progress. The demonstration looks excellent. But has anyone proved customer data separation, or only shown that the screens work?
In this hypothetical example, a customer opens a saved invoice link belonging to another account. The portal checks that they are logged in, but not whether they may read that particular invoice. A working login has become the wrong reassurance.
A successful demo is not a security boundary
AI-assisted development can make a useful prototype much faster. It does not remove the need to specify and verify security requirements. Canada’s Cyber Centre warns that AI-generated code can contain vulnerabilities and that a model’s explanation or self-review is a weak control on its own.
The commercial decision is not simply whether to use AI. It is what evidence you require before an AI-built application handles real customer records. A prototype using synthetic information and a live portal storing confidential quotes need different levels of oversight.
Customer data separation must happen on the server
Authentication establishes who signed in. Authorisation decides what that person may do to a particular record. OWASP describes a failure to enforce that second decision as broken object-level authorisation.
A portal can hide another customer’s invoice in its menu while its underlying application programming interface still returns the file when directly requested. A browser-supplied record identifier is not evidence of permission. Replacing simple numbers with hard-to-guess identifiers helps reduce guessing, but does not replace access checks.
The server must check the authenticated user’s permitted relationship to the requested record and action. Reading an invoice, changing an address and exporting documents may require different permissions. A blanket check that someone is a valid customer is insufficient. The same problem can exist in human-written software; AI is not the only cause.
If that boundary fails, one customer could receive another’s pricing, addresses or commercially sensitive documents. Where write operations are also exposed, unauthorised changes may follow. Those are potential consequences, not a report of an incident at a particular business.
Ask for evidence from two test customers
Before launch, the business owner should agree the access rules with the developer. Then have a suitably qualified reviewer test those rules in an authorised staging environment using synthetic records and separate customer accounts.
- Prove the permitted journey. Each test customer can reach their own intended records and complete the actions they genuinely need.
- Prove the denied journey. One account cannot retrieve, alter or export the other account’s records through direct requests, saved links or alternative download routes.
- Test the exceptions. Check expired invitations, removed users, shared customer accounts and legitimate staff access against the agreed rules.
- Keep the result reproducible. Record the tested software version, expected permissions and actual outcomes. Repeat these tests after relevant changes.
Use only systems you own or have permission to test. Automated scans and an AI-generated assurance are not substitutes for checking the application’s actual customer relationships. These tests are useful evidence, not a guarantee that every vulnerability has been found.
Keep the speed. Assign the responsibility.
The business owner defines which records may be shared. The developer implements the server-side checks. An appropriately skilled reviewer challenges the result. The release owner decides whether the evidence supports using real data, and retains a rollback and incident-response path.
Australian government software-development guidance explicitly covers AI-assisted development and supports documented requirements, security responsibilities and segregated environments. It is engineering guidance here, not a claim that every ISM control is a universal legal requirement for every Australian small business.
The useful acceptance question is: Can you demonstrate that the wrong customer is refused, not just that the right customer gets in?
How Xpansion can support your next step
Xpansion Technologies can help scope AI-powered applications and automation, map customer access requirements and plan a controlled pilot. Agree the security review and acceptance evidence before a successful demonstration becomes a live customer service.
Sources
- ASD Information Security Manual: Guidelines for software development, September 2026 edition
- Canadian Centre for Cyber Security: Managing security risks in vibe coding, released 29 October 2026
- OWASP API Security Top 10: Broken Object Level Authorization
- OWASP Authorization Cheat Sheet
Source scope: the Canadian publication is overseas government guidance; OWASP is a technical security project, not an Australian regulator. This is an educational article with a hypothetical scenario, not a breaking-news report.
