Phishing Tests Don't Work. Fight Me.
security-culture hacklore human-risk
Your click rate dropped from 24% to 7%. You put that slide in the board deck. Everyone nodded. The phishing simulation vendor sent a congratulations email.
And your actual exposure to phishing-based attacks barely moved.
This is the core problem with how most organizations run phishing tests. The metric improved. The risk didn’t. Those aren’t the same thing, and treating them as equivalent is how you end up with a mature-looking program and a real gap underneath it.
What Phishing Tests Actually Measure
A phishing simulation measures whether your employees can spot a specific type of lure, run by your own IT team, at a cadence your vendor controls, with lures that look like phishing emails from 2019.
That’s useful information. It’s not a proxy for whether your organization is resilient to actual phishing attacks.
Real phishing is targeted. It knows your org chart, your vendors, your current projects. It arrives at the right moment. It’s written by someone who studied your LinkedIn page. The generic “your package is ready for pickup” simulation that gets your click rate to single digits does very little to prepare your CFO for a spoofed wire transfer request from someone who knows your CEO is traveling this week.
The simulation gap is real and most programs don’t account for it.
AI Just Made This Worse
For years, the advice was: look for bad grammar, weird spelling, generic greetings, a sender address that doesn’t quite match. Those were real tells. They worked because most phishing was mass-produced garbage.
That era is over.
AI can write a flawless email. It can match tone, use the right terminology for your industry, reference a real event from your company’s LinkedIn page, and address your CFO by name with context that makes it feel like it came from someone who knows them. It doesn’t make typos. It doesn’t write “Dear Valued Customer.” It sounds exactly like a vendor your team has worked with for two years.
The “spot the tells” skill your phishing training is building is becoming obsolete in real time. Attackers have access to the same AI tools your employees use to write better emails. The result is phishing that clears every mental checklist your users have been trained to run.
This matters for simulation programs specifically because most vendors are still using lures that look like 2019 phishing. If your employees are getting better at spotting those, you’ve trained them for a threat model that no longer represents what’s actually in their inbox. They pass your test. They fail against a well-crafted AI-generated spearphish targeting their specific role. Those are two different problems and your click rate captures neither.
The Rebound Problem
Even within the narrow thing phishing tests measure, the results don’t stick. The research on this is consistent: click rates on simulated phishing drop sharply after training, then rebound. Within three to six months, most organizations are close to baseline. You run the training again. The cycle repeats.
Knowledge doesn’t change behavior under pressure. Someone with a full inbox, four meetings before lunch, and a message that looks like it’s from their bank isn’t going to apply what they learned in a training module four months ago. The conditions that produce bad security decisions are mostly environmental. A training completion doesn’t change the environment.
The Culture Problem Nobody Talks About
Here’s the one that bothers me most. Poorly designed phishing simulations actively damage the thing you need most from employees: a willingness to report suspicious activity.
When people get shamed for clicking a simulated phish, especially with aggressive “gotcha” landing pages or immediate mandatory retraining, the lesson many of them actually learn is not “be more careful with email.” It’s “do not tell IT anything.”
If someone receives a real suspicious email and their gut reaction is to quietly close it rather than report it because they’re afraid of getting in trouble, your phishing test program just made you less secure. The detection and reporting chain is broken, and it broke because you used simulation as a performance evaluation instead of a learning tool.
The click isn’t the thing you’re protecting against. The silence afterward is.
What Actually Reduces Phishing Risk
The interventions that move the needle are control decisions, not awareness campaigns.
Phishing-resistant MFA is the most important one. Hardware keys, passkeys, FIDO2 authenticators: if your authentication requires something an attacker can’t phish, a user clicking a malicious link becomes a much smaller problem. You’ve addressed the risk at the control layer instead of trying to change behavior at the user layer. This is the shift that matters most and most organizations haven’t made it.
Email filtering, link rewriting, and attachment sandboxing reduce the volume of real threats that reach inboxes in the first place. Less exposure means less opportunity for a click. That math is simpler and more reliable than behavioral change.
Easy reporting mechanisms matter more than click rates. If reporting a suspicious email takes five clicks and a ticket number, people won’t report. If it takes one click, most will. The goal of a phishing program should be building a fast, friction-free reporting pipeline, not optimizing a simulation score.
What Phishing Tests Are Actually Good For
I’m not saying to stop running them. They are useful when used correctly.
Treat simulations as a diagnostic, not a benchmark. High click rates in specific departments or roles tell you where to focus technical controls and targeted training. Finance clicking at twice the rate of IT tells you something actionable. Reporting that number to the board as a measure of security posture does not.
Use results to find gaps in your email controls. If simulated phish are consistently getting through your filters, that’s a problem with your tooling, not your users.
Run targeted, realistic simulations occasionally rather than frequent generic ones. Spearphishing tests aimed at specific high-risk roles with contextual lures tell you more about actual resilience than monthly simulations everyone has learned to recognize.
The Honest Conversation
If you have to run phishing tests because your compliance framework or cyber insurance policy requires it, run them. Document them. Satisfy the requirement.
And then have a separate conversation about what’s actually protecting you from phishing-based compromises. That conversation involves MFA authentication strength, email filtering efficacy, and how fast your team can identify and contain a real credential theft. It doesn’t hinge on whether your click rate is below 10%.
The click rate is the metric that got sold to you. Phishing-resistant controls are the strategy.
If you ever have a phishing-related incident, the question that matters isn’t “what was our click rate?” It’s “why did clicking that link cause damage?” The answer to the second question is where your real program lives.
Questions about building a phishing defense strategy that goes beyond simulation scores? Let’s talk.
Ready to Secure Your Growth?
Whether you need an executive speaker for your next event or a fractional CISO to build your security roadmap, let's talk.
Consulting services are delivered through Vaughn Cyber Group.
