Real BytesCall us
Threat literacy brief

The AI threats Australian SMBs are actually seeing.

Adversaries are using the same generative AI tools your team uses, applied to phishing, voice impersonation, document poisoning and reconnaissance. This brief covers the five attack patterns we see most often, what each one actually looks like, and the controls that move the needle.

Threat 01

Deepfake CEO and CFO fraud

Synthetic video or audio of a senior leader instructing staff to authorise a payment, change banking details or release sensitive data.

Pattern

The attack typically starts with a brief impersonated Teams or Zoom call, often outside business hours, followed up by a written instruction sent from a lookalike domain or compromised mailbox.

What it looks like

  • Urgency framed around confidentiality (board deal, acquisition, audit).
  • Pressure to bypass the normal finance approval workflow.
  • Banking detail changes communicated outside the usual vendor portal.
  • Awkward camera positioning, eye contact, or short call duration.

Controls that help

  • Out-of-band verification of every payment-related instruction over a fixed value (call back on the known number, not the one provided).
  • Documented two-person approval on bank detail changes for vendors and payroll.
  • Staff awareness brief specifically covering deepfake patterns, refreshed annually.

Threat 02

AI-assisted business email compromise

BEC drafted with generative AI that mirrors your internal tone, references real projects and clears traditional phishing filters.

Pattern

Attackers harvest publicly available signals (LinkedIn posts, vendor case studies, press releases) and use a language model to draft messages that read indistinguishably from internal correspondence.

What it looks like

  • Internal-feeling language with no grammatical tells.
  • Reference to real projects, clients or staff names sourced from public material.
  • Reply-to address that differs subtly from the displayed sender.
  • No links or attachments, just a request to update payment or contract terms.

Controls that help

  • DMARC enforced at p=reject across all sending domains.
  • Conditional access blocking legacy authentication and risky locations.
  • Mailbox rules audit run quarterly to catch silent forwarding rules.
  • Staff aware that quality of writing is no longer a phishing signal.

Threat 03

Voice cloning and vishing

A short voice sample of a director or manager (taken from a webinar, podcast or social video) used to generate convincing phone instructions.

Pattern

Most common against finance, HR and IT helpdesk. Often paired with a fake caller ID and a fabricated urgency, such as a director travelling and needing a password reset.

What it looks like

  • Calls outside business hours from an unfamiliar number claiming to be a senior staff member.
  • Requests for password resets, MFA bypass, payroll changes or vendor payments.
  • Background noise inconsistent with the claimed location.
  • Reluctance to switch to a video channel or callback verification.

Controls that help

  • Voice instructions never sufficient for password resets, MFA bypass or payment release.
  • Helpdesk verification scripts requiring a second factor outside the call channel.
  • Director and senior staff briefed that their public voice samples are now an attack surface.

Threat 04

Prompt injection and data exfiltration via AI tools

Attackers embed hidden instructions in documents, emails or websites that hijack a generative AI assistant when it processes the content.

Pattern

Most relevant where staff use Copilot, ChatGPT or Gemini to summarise external documents or browse web pages. The injected instruction can cause the tool to leak earlier conversation context, draft phishing replies, or fetch data the user did not ask for.

What it looks like

  • Unexpected AI responses that include instructions the user did not type.
  • AI assistants drafting messages or summaries with content unrelated to the prompt.
  • Documents from unknown senders that staff are asked to summarise.
  • AI tool behaviour changing after browsing or ingesting external content.

Controls that help

  • Acceptable use policy stating that unknown external documents are not fed into AI tools.
  • AI usage logged at the tenant level where possible (Copilot audit log, Workspace activity).
  • Sensitivity labels applied to documents containing personal information so AI tools can be restricted accordingly.
  • Staff aware that AI tools can be tricked, the same way browsers can.

Threat 05

AI-accelerated reconnaissance and exploitation

Adversaries use language models to profile target organisations and to translate proof-of-concept exploit code into working attacks faster than before.

Pattern

The window between a CVE being disclosed and being actively exploited has compressed from weeks to hours for high-value targets. AI profiling lowers the skill threshold to identify staff, suppliers and weak links.

What it looks like

  • Spear phishing referencing specific internal context within days of a personnel change.
  • Failed login attempts targeting newly disclosed VPN, firewall or remote access products.
  • Increased low-volume probing across exposed services.
  • Targeted phishing of staff whose roles map to specific applications (finance, HR, IT admin).

Controls that help

  • Patching SLA commitment for internet-facing services measured in days, not weeks.
  • External attack surface monitoring (the same view an attacker has of your perimeter).
  • Privileged access management for IT and admin accounts.
  • Threat intelligence feed integrated into detection (ASD ACSC alerts, vendor advisories).

Threat 06

Agentic AI taking unintended actions

AI agents that can make decisions, interact with tools and take actions with limited human intervention can be tricked, hijacked or escalate privileges if not tightly scoped. In May 2026 ASD and its Five Eyes partners published joint guidance following testing that showed advanced AI models can autonomously reason about objectives, adapt to changing circumstances and combine multiple technical actions into sophisticated attack sequences.

Pattern

Unlike traditional AI that generates information for human review, agentic AI systems interact with enterprise systems, external data sources and tools. Each component widens the attack surface. A malicious prompt hidden in an email or web page can trick an agent into downloading malware, sending unauthorised messages or escalating privileges. Complex interactions between multiple agents can cause cascading failures across interconnected systems.

What it looks like

  • AI agents performing actions outside their intended scope or objective.
  • Unexpected tool usage or system access by an AI agent during normal operation.
  • Agent behaviour changing after processing external content (emails, documents, web pages).
  • Difficulty tracing which agent made a decision or triggered an action in a multi-agent setup.

Controls that help

  • Limit agent permissions to the minimum level required to perform approved tasks (least privilege).
  • Maintain human oversight and approval for high-impact or sensitive actions.
  • Continuously monitor agent behaviour, decisions and tool usage with alerts for anomalous activity.
  • Comprehensive logging and auditing of agent actions so they can be reviewed and reversed.
  • Regular red teaming and adversarial testing before and during deployment.
  • Validate third-party tools, integrations and dependencies before deployment.
  • Deploy progressively: start with low-risk use cases, increase autonomy only as assurance matures.
  • Isolate agents and enforce strict controls over interactions between systems and environments.

How to brief your staff this quarter.

We recommend a single fifteen-minute staff brief covering three messages, repeated every quarter:

  1. 01

    Quality of writing is no longer a phishing signal.

    Treat the verification step (call back, second factor, confirm with the person in person) as the safety net. Polished, internal-sounding messages are normal now.

  2. 02

    A voice or video instruction is not authority.

    Every payment, bank detail change, password reset and MFA action needs verification outside the call channel that initiated the request, every time.

  3. 03

    AI tools can be tricked, the same way browsers can.

    Unknown external documents and webpages can carry instructions that hijack the AI assistant processing them. Use AI tools only on content you trust, or that has been vetted.

Want a tabletop walkthrough?

We run AI-themed incident simulations with leadership teams, covering deepfake CEO fraud, voice cloning vishing and AI-assisted BEC. Ninety minutes, decision by decision.

Try the incident simulator AI Readiness Hub