Remote SupportDeepfake CEO and CFO fraud
The deepfake CEO fraud pattern is now a proven, documented loss vector rather than a theoretical risk, because the British engineering firm Arup confirmed in May 2024 that a Hong Kong employee was duped into sending HK 200 million dollars, about 20 million pounds, to criminals by an AI generated video conference call where every participant looked and sounded like a real colleague, so the Australian controls below are a response to an attack that has already cost a major professional services firm eight figures. The Guardian, Arup deepfake scam.
Threat 01
Synthetic video or audio of a senior leader instructing staff to authorise a payment, change banking details or release sensitive data.
Pattern
The attack typically starts with a brief impersonated Teams or Zoom call, often outside business hours, followed up by a written instruction sent from a lookalike domain or compromised mailbox.
What it looks like
- Urgency framed around confidentiality (board deal, acquisition, audit).
- Pressure to bypass the normal finance approval workflow.
- Banking detail changes communicated outside the usual vendor portal.
- Awkward camera positioning, eye contact, or short call duration.
Controls that help
- Out-of-band verification of every payment-related instruction over a fixed value (call back on the known number, not the one provided).
- Documented two-person approval on bank detail changes for vendors and payroll.
- Staff awareness brief specifically covering deepfake patterns, refreshed annually.
AI-assisted business email compromise
AI assisted business email compromise bypasses traditional phishing filters because the Verizon 2026 DBIR found that 15 per cent of attack techniques are now being bolstered by generative AI and social engineering has emerged as the most common way attackers gain initial access into an organisation, which means the quality of writing is no longer a reliable phishing signal and the verification step is now the safety net. Verizon 2026 Data Breach Investigations Report.
The ACCC Targeting Scams Report 2025 recorded 2.18 billion dollars in total reported losses across 481,523 scam reports, with payment redirection scams accounting for 166.8 million dollars and phishing scams 97.6 million dollars, so AI assisted BEC targeting Australian finance teams is now measured in nine figure annual losses nationally. ACCC Targeting Scams Report 2025.
Threat 02
BEC drafted with generative AI that mirrors your internal tone, references real projects and clears traditional phishing filters.
Pattern
Attackers harvest publicly available signals (LinkedIn posts, vendor case studies, press releases) and use a language model to draft messages that read indistinguishably from internal correspondence.
What it looks like
- Internal-feeling language with no grammatical tells.
- Reference to real projects, clients or staff names sourced from public material.
- Reply-to address that differs subtly from the displayed sender.
- No links or attachments, just a request to update payment or contract terms.
Controls that help
- DMARC enforced at p=reject across all sending domains.
- Conditional access blocking legacy authentication and risky locations.
- Mailbox rules audit run quarterly to catch silent forwarding rules.
- Staff aware that quality of writing is no longer a phishing signal.
Voice cloning and vishing
Voice cloning and vishing are difficult to detect because the Verizon 2026 DBIR notes attackers are increasingly using voice and other mobile-centric techniques to catch people off guard in the middle of the workday, and a short voice sample taken from a webinar or podcast is enough to generate convincing phone instructions, so helpdesk verification scripts requiring a second factor outside the call channel are now the minimum control. Verizon 2026 DBIR.
Threat 03
A short voice sample of a director or manager (taken from a webinar, podcast or social video) used to generate convincing phone instructions.
Pattern
Most common against finance, HR and IT helpdesk. Often paired with a fake caller ID and a fabricated urgency, such as a director travelling and needing a password reset.
What it looks like
- Calls outside business hours from an unfamiliar number claiming to be a senior staff member.
- Requests for password resets, MFA bypass, payroll changes or vendor payments.
- Background noise inconsistent with the claimed location.
- Reluctance to switch to a video channel or callback verification.
Controls that help
- Voice instructions never sufficient for password resets, MFA bypass or payment release.
- Helpdesk verification scripts requiring a second factor outside the call channel.
- Director and senior staff briefed that their public voice samples are now an attack surface.
Prompt injection and data exfiltration via AI tools
Prompt injection is now a documented enterprise risk because staff routinely feed external documents into Copilot, ChatGPT or Gemini to summarise them, and an instruction hidden in an email or web page can cause the tool to leak earlier conversation context or draft phishing replies, so an acceptable use policy stating that unknown external documents are not fed into AI tools is now a baseline control rather than an optional hardening step. CISA, Careful Adoption of Agentic AI Services.
Threat 04
Attackers embed hidden instructions in documents, emails or websites that hijack a generative AI assistant when it processes the content.
Pattern
Most relevant where staff use Copilot, ChatGPT or Gemini to summarise external documents or browse web pages. The injected instruction can cause the tool to leak earlier conversation context, draft phishing replies, or fetch data the user did not ask for.
What it looks like
- Unexpected AI responses that include instructions the user did not type.
- AI assistants drafting messages or summaries with content unrelated to the prompt.
- Documents from unknown senders that staff are asked to summarise.
- AI tool behaviour changing after browsing or ingesting external content.
Controls that help
- Acceptable use policy stating that unknown external documents are not fed into AI tools.
- AI usage logged at the tenant level where possible (Copilot audit log, Workspace activity).
- Sensitivity labels applied to documents containing personal information so AI tools can be restricted accordingly.
- Staff aware that AI tools can be tricked, the same way browsers can.
AI-accelerated reconnaissance and exploitation
The OAIC received 1,205 data breach notifications in 2025, the highest since the Notifiable Data Breaches scheme began and an 8 per cent increase over 2024, with 716 of those attributable to malicious or criminal activity, so the compressed CVE to exploitation window driven by AI profiling is now landing in a regulatory environment where breaches are reported and investigated at record volume. OAIC, data breach notifications increase to all-time high in 2025.
Threat 05
Adversaries use language models to profile target organisations and to translate proof-of-concept exploit code into working attacks faster than before.
Pattern
The window between a CVE being disclosed and being actively exploited has compressed from weeks to hours for high-value targets. AI profiling lowers the skill threshold to identify staff, suppliers and weak links.
What it looks like
- Spear phishing referencing specific internal context within days of a personnel change.
- Failed login attempts targeting newly disclosed VPN, firewall or remote access products.
- Increased low-volume probing across exposed services.
- Targeted phishing of staff whose roles map to specific applications (finance, HR, IT admin).
Controls that help
- Patching SLA commitment for internet-facing services measured in days, not weeks.
- External attack surface monitoring (the same view an attacker has of your perimeter).
- Privileged access management for IT and admin accounts.
- Threat intelligence feed integrated into detection (ASD ACSC alerts, vendor advisories).
Agentic AI taking unintended actions
On May 1 2026, six national cybersecurity agencies including Australia ASD ACSC, CISA, NSA, the Canadian Centre for Cyber Security, New Zealand NCSC and the UK NCSC jointly published Careful Adoption of Agentic AI Services, the first coordinated Five Eyes guidance specifically addressing agentic AI, following testing that showed advanced AI models can autonomously reason about objectives, adapt to changing circumstances and combine multiple technical actions into sophisticated attack sequences. CISA, Careful Adoption of Agentic AI Services.
The Five Eyes guidance recommends treating threat modelling at the design stage as non-negotiable, that no agent should extend implicit trust to another agent output, that altered files, modified access controls and deleted audit trails may be the first evidence of an agentic compromise, and that SIEM tooling not yet adapted for agentic workloads must be upgraded to log every agent action including triggering prompts and complete tool call chains. Cyber.gov.au, Five Eyes cyber security agencies statement.
Threat 06
AI agents that can make decisions, interact with tools and take actions with limited human intervention can be tricked, hijacked or escalate privileges if not tightly scoped. In May 2026 ASD and its Five Eyes partners published joint guidance following testing that showed advanced AI models can autonomously reason about objectives, adapt to changing circumstances and combine multiple technical actions into sophisticated attack sequences.
Pattern
Unlike traditional AI that generates information for human review, agentic AI systems interact with enterprise systems, external data sources and tools. Each component widens the attack surface. A malicious prompt hidden in an email or web page can trick an agent into downloading malware, sending unauthorised messages or escalating privileges. Complex interactions between multiple agents can cause cascading failures across interconnected systems.
What it looks like
- AI agents performing actions outside their intended scope or objective.
- Unexpected tool usage or system access by an AI agent during normal operation.
- Agent behaviour changing after processing external content (emails, documents, web pages).
- Difficulty tracing which agent made a decision or triggered an action in a multi-agent setup.
Controls that help
- Limit agent permissions to the minimum level required to perform approved tasks (least privilege).
- Maintain human oversight and approval for high-impact or sensitive actions.
- Continuously monitor agent behaviour, decisions and tool usage with alerts for anomalous activity.
- Comprehensive logging and auditing of agent actions so they can be reviewed and reversed.
- Regular red teaming and adversarial testing before and during deployment.
- Validate third-party tools, integrations and dependencies before deployment.
- Deploy progressively: start with low-risk use cases, increase autonomy only as assurance matures.
- Isolate agents and enforce strict controls over interactions between systems and environments.
How to brief your staff this quarter
The three messages in the quarterly staff brief below are the practical response to the Verizon 2026 DBIR finding that the human element was present in 62 per cent of breaches and that social engineering is now the most common initial access vector, so treating the verification step as the safety net rather than the quality of writing is the single behavioural change that moves the needle on AI augmented threats. Verizon 2026 DBIR.
We recommend a single fifteen-minute staff brief covering three messages, repeated every quarter.
Quality of writing is no longer a phishing signal.
Treat the verification step (call back, second factor, confirm with the person in person) as the safety net. Polished, internal-sounding messages are normal now.
A voice or video instruction is not authority.
Every payment, bank detail change, password reset and MFA action needs verification outside the call channel that initiated the request, every time.
AI tools can be tricked, the same way browsers can.
Unknown external documents and webpages can carry instructions that hijack the AI assistant processing them. Use AI tools only on content you trust, or that has been vetted.
Common questions
How are deepfake CEO fraud attacks actually carried out against Australian businesses?
Why does AI-assisted business email compromise bypass traditional phishing filters?
What makes voice cloning and vishing attacks difficult to detect?
What is prompt injection and how does it exploit AI tools like Copilot or ChatGPT?
How has AI compressed the window between CVE disclosure and active exploitation?
What controls should Australian businesses deploy against AI-augmented threats?
Want a tabletop walkthrough?
We run AI-themed incident simulations with leadership teams, covering deepfake CEO fraud, voice cloning vishing and AI-assisted BEC. Ninety minutes, decision by decision.