Source: Unite.AI
Audian Paxson, Principal Technical Strategist at IRONSCALES, is a cybersecurity and product strategy veteran with more than two decades of experience spanning email security, threat detection, networking, and enterprise technology. At IRONSCALES, he focuses on product launches, messaging and positioning, go-to-market strategy, and thought leadership around the use of artificial intelligence and machine learning to address emerging email security threats. His career includes senior product management, product marketing, and business development roles at Netgate, BlueCat, Alert Logic, Critical Watch, Iconix, Strix Systems, and Intel. Paxson has played key roles in launching cybersecurity and networking products, developing enterprise go-to-market strategies, and expanding companies into new markets, and is also a co-inventor on patented security technology.
IRONSCALES is an AI-powered email security company focused on protecting organizations against phishing, business email compromise, account takeover, deepfakes, and other evolving social engineering threats. Its platform combines Adaptive AI with human and community intelligence to detect, investigate, and remediate threats across Microsoft 365 and Google Workspace, while also providing capabilities for phishing simulation, security awareness training, outbound email protection, and domain security. The company has expanded its approach with specialized AI agents for red teaming, phishing security operations, and phishing simulation, designed to anticipate attacks, investigate suspicious activity, and strengthen employee defenses. IRONSCALES says its technology is used by more than 18,000 organizations and incorporates intelligence contributed by a community of more than 36,000 security professionals.
Your work in email security stretches back more than two decades, including helping develop early anti-phishing products and now focusing on how AI and machine learning can address emerging email threats. From that vantage point, what feels genuinely new about AI-driven social engineering, and what parts of the problem have remained surprisingly familiar?
Thanks for dating me! When we started fighting phishing in the early 2000s, the biggest problem was awareness, and filters could only judge a sender’s domain reputation. But the tactics haven’t changed much. Urgency, authority, a fake invoice. I recognize almost every pretext I see today. What AI changed is the craftsmanship barrier, so an unskilled attacker can now produce a flawless, personalized version of a 2004 scam at scale.
Anthropic found that Claude Mythos 5 could recognize signs of the real internet when those signals were presented independently, but largely failed to act on the same evidence while immersed in a task. What does that gap tell us about the difference between an AI model possessing knowledge and actually applying it at the right moment?
I’m an email security guy, not an AI researcher, so I read it through that lens. Mythos was told up front that it was in a simulation, and that framing won. It explained away real evidence because the evidence didn’t fit the story it was already in. That’s a pretext doing its job, and it’s why attackers work so hard to set the frame first.
There is an interesting parallel here with human phishing behavior: an employee may easily identify suspicious indicators during training but overlook those same indicators when rushed or focused on completing another task. How far do you think that comparison between human and AI judgment can reasonably be taken?
Up to a point. Both lose the plot when they’re heads-down on a task, and a message that looks like a routine business process (“the following claims need review”) gets waved through by people and models alike. The comparison breaks down on emotion. Humans get rushed, scared, or eager to please. An agent fails on its own triggers, like instructions it can’t tell apart from the data it’s reading.
Security training has traditionally focused on teaching people to recognize malicious links, unusual requests, impersonation, and other warning signs. If both humans and AI systems can ignore signals they technically understand, should security teams be thinking less about recognition and more about how decisions are made under pressure and within a particular context?
Yes. Recognition training assumes people are paying attention. Attackers design messages to land when they aren’t or cause them to forget their training. The better investment is building a pause into the moments that matter (like payments, credential resets, access changes) so neither a person nor an agent makes that call on autopilot.
As AI agents become more deeply integrated into inboxes and enterprise workflows, attackers may increasingly be able to socially engineer the AI system itself rather than only the employee. What might an attack designed specifically to manipulate an email agent or autonomous assistant look like?
It’s an email written for the agent, with the instructions hidden from the human in white-on-white text, image alt-text, or another language so English-tuned filters skip it. One scenario our research team built plants a false “fact” in the agent’s memory (“approvals from finance@ are pre-authorized”), so a spoofed finance email weeks later sails through. Another hides text inside an invoice image telling the agent to mark it low-risk and summarize it as safe to pay. The human trusts the summary.
Could attackers deliberately create long sequences of seemingly legitimate interactions that gradually push an AI system toward a bad decision, rather than relying on a single obviously malicious email or prompt? How difficult would these multi-step attacks be for today’s security tools to detect?
They already happen against people, and agents are next. Attackers send a few harmless emails (an intro, a thank-you) to clear first-time-sender flags, shift to messages relevant to the target’s role, and only then deliver the payload, often a meeting invite whose link shows scanners a clean page and real users the phishing site. Tools that judge each email in isolation miss it. No single message looks bad, so you have to score the relationship over time.
AI-generated phishing is often discussed in terms of better writing, personalization, and deepfakes. Are you more concerned about AI improving the content of social engineering attacks, or about attackers using AI to understand exactly when and how a particular target is most likely to make a mistake?
Both, and the content fight is already over. Polished, AI-written phishing has been the baseline since ChatGPT went mainstream. The bigger shift is that targeting, personalization, and timing are now automated, so someone with zero technical skill can combine breach data with password dumps like the infamous RockYou list, take over a real mailbox, and send a spoof of a routine transactional email right when the target expects one.
As attackers become better at removing obvious technical red flags and making malicious messages appear legitimate, what signals become most important for distinguishing legitimate communication from manipulation?
Context comes first. For instance, is this normal for this person or vendor, in this channel, at this time? Then urgency and repetition. Somebody with a truly urgent request picks up the phone or pings you on Teams or Slack. They don’t send the same email three times. And trust your spidey sense. My rule is “Don’t just do something, stand there,” so pause, get a second set of eyes, and verify through a different channel.
How should organizations structure human and AI oversight when both sides have potential weaknesses? Could combining human judgment with AI actually create new blind spots if each assumes the other has already validated the decision?
Give each side the job it’s best at and make the handoff explicit. AI handles volume and spots patterns across the whole environment; humans own the high-consequence calls (moving money, granting access), and the AI has to show its evidence so the reviewer can check the reasoning. The blind spot you describe is real. The fix is a hard rule… “the AI said it’s safe” never counts as a human review.
If the fundamental problem is a gap between what humans or AI systems know and what they recognize while actively performing a task, what practical changes can organizations make today to narrow that gap and make social engineering substantially harder to execute successfully?
Borrow an old control we used for writing checks, “separation of duties.” One person can create a new user but can’t activate it, a second person can activate a user but can’t create one, and a third gets notified of both. Build that same split into payments, vendor changes, and your AI agents (no agent should both request and approve an action). One fooled employee or one hijacked agent can’t finish the attack alone.
Thank you for the great interview, readers who wish to learn more should visit IRONSCALES.