Phish are now baiting your AI email assistant

You dodged the phish. You didn’t click, reply, or unpack the file. Then your AI assistant dutifully summarized your inbox—and nudged you to take action on the very message you skipped.

That’s the twist Malwarebytes flags in new reporting: phishing emails are now baiting AI assistants with hidden instructions designed to reshape summaries and push harmful “to‑dos” (Malwarebytes).

What happened

Malwarebytes describes research from Barracuda that found phishing messages written for two audiences at once: people and machines reading the same email. For humans, some included a password‑protected attachment with the unlock code right in the body. For AI, they embedded concealed instructions.

Those instructions used prompt injection—text that tells an AI system what to do rather than what to read. When the source is external content like email or a webpage, it’s often called indirect prompt injection, because the malicious prompt piggybacks on material the model is expected to process.

Barracuda’s examples included a block that told a summarizing assistant to add a fake priority task: change a vendor’s payment details. That transforms a fraudster’s request into the assistant’s recommendation, raising the chance a busy user treats it as legitimate.

Malwarebytes notes that the concealed content didn’t need to look like normal paragraphs. Barracuda spotted instructions buried in HTML comments, text hidden by styling, encoded snippets, and zero‑width characters—methods that leave the content machine‑readable while keeping it invisible to people skimming the email.

Whether those hidden parts reach the model depends on how a given assistant ingests and cleans messages. The report emphasizes that extraction and preprocessing choices can decide if invisible text becomes part of the prompt the model sees.

Developer Apollo Raines, who builds the AI code‑review tool ShipItClean, told Malwarebytes the core failure is a model that “can’t tell the difference between text it’s supposed to read and instructions it’s supposed to follow.” He argued stripping hidden content is necessary but framed the problem as an arms race.

Raines also urged that some protections must “live inside the model itself,” not just in wrappers. He listed wrappers, filters, sandboxing, and human sign‑off on payments as necessary guardrails when assistants interact with inboxes and workflows.

Malwarebytes warns that assistants which only summarize can still mislead users about which messages deserve attention and which actions are urgent. The risk compounds when assistants have permissions to act, potentially triggering unauthorized operations or disclosing sensitive data.

The advice in the report is to keep using helpful tools, but to narrow trust and access. Limit what the assistant can do, and re‑introduce human checks for money or sensitive requests that originate in email.

The background

Prompt injection has trailed generative AI since its debut, but email creates a tidy delivery channel: business processes already rely on what lands in inboxes, and many teams now let assistants triage that flow. Malwarebytes points out that attackers don’t need to win over the recipient if they can lean on the software the recipient trusts.

Malwarebytes explains that indirect prompt injection works because AI systems often treat ingested text as both context and instruction. Without strict content boundaries, hidden directives can be interpreted as system or user messages, subtly steering outputs.

The report highlights that concealment can be as simple as CSS that sets text color to match a background, or as tricky as zero‑width characters that split tokens while staying invisible on screen. Either way, the model sees text the person never noticed.

Password‑protected attachments play a parallel role aimed at the human. They can increase curiosity and bypass some scanners, while the email body handles the AI manipulation. Malwarebytes references Barracuda’s finding that these two tactics can appear together.

When assistants escalate a doctored summary to a to‑do, financial workflows are an obvious target. Malwarebytes cites the vendor‑payment example because it could divert funds without malware or account compromise—just a convincing “urgent” summary.

The piece also underscores that the cleanliness of inputs is decisive. If an assistant’s pipeline removes comments, invisible spans, and odd encodings before chunking content for the model, many injection attempts won’t survive long enough to influence a summary.

Email Vox has tracked vendors pitching broader context‑aware defenses, such as intent‑focused filtering across email and browsers. For a look at that direction, see our earlier overview of intent‑based email defenses. We’ve also covered how coalitions cut off phishing infrastructure at the source in a major takedown of cloud‑abused accounts.

What people are saying

Malwarebytes quotes Apollo Raines urging vendors to strip hidden content by default, while warning it’s “an arms race” as attackers rotate encoding tricks. He adds that guardrails should include model‑level constraints, not just pre‑ and post‑processing.

Malwarebytes’ write‑up stresses that user awareness alone won’t solve this, because the target is an AI pipeline. The defense must sit inside the tools that parse messages, decide what reaches the model, and frame what the assistant is allowed to do next.

The report’s practical checks include verifying payment changes through known channels independent of the email, and never treating a summary as verification. It also recommends limiting assistants’ permissions to the minimum needed, especially in work settings.

Human approval for high‑impact actions—like moving money or changing vendor details—is specifically called out. Malwarebytes points out approval should include checking the original request, not just rubber‑stamping the assistant’s summary.

For teams experimenting with AI‑infused inboxes, Email Vox has profiled how vendors are threading email data into chat tools. That trend raises the stakes for input hygiene; see our explainer on bringing Engage email data into AI chats for the usability upside and the caveat about still checking answers.

What happens next

Based on Malwarebytes’ account of Barracuda’s findings, expect security teams to harden ingestion paths: sanitize HTML, drop invisible spans, decode and normalize text, and quarantine password‑protected attachments pending extra checks.

Assistant builders are likely to add stricter role separation in prompts so that ingested body text can’t silently overwrite system rules. Malwarebytes relays Raines’ push for model‑level boundaries, which could include structured extraction that discards free‑form directives.

Operations teams will probably revisit permissions for AI that touches mail. Malwarebytes recommends keeping assistants on read‑only for triage, forbidding financial actions, and routing any proposed money move through a human with an out‑of‑band verification step.

Finance and procurement teams may update runbooks to include a “summary is not proof” reminder where invoice changes are concerned. Malwarebytes repeatedly emphasizes verifying via trusted contact details that predate the suspect email.

Security awareness programs can add a module that shows how invisible content works, paired with assistant behavior tests. Malwarebytes’ takeaway is not to abandon inbox helpers—but to decide, in advance, what they can’t do.

Teams looking to situate these moves within broader email risk can browse our security hub. For more on how attackers continually adapt email techniques, our reporting on North Korean reply‑bait phishing shows the social playbook evolving even without AI in the loop.