Spotting AI-Generated Scams Is The Wrong Defence

In February, a familiar voice helped move around €95 million out of an Italian private bank. The voice was a copy, and the case shows how AI-generated scams now work.

According to Italian reporting, the fraud at Fideuram, part of the Intesa Sanpaolo group, began with a WhatsApp message that appeared to come from the group’s chief executive. It asked the then chair to arrange urgent transfers through Fideuram’s treasury for a deal abroad. A phone call followed in which AI replicated the voice of a lawyer the chair knew. The money went mostly to China and Hong Kong. Part of it was recovered through international cooperation between banks, but at least €36 million, converted into cryptocurrency, is still untraced.

The target was the chair of a bank, not a novice. That is the point.

AI-generated scams target recognition

On 30 September the European Commission published a Eurobarometer survey on cyber threats at work, released as European Cybersecurity Month began. Three in four EU employees said they had encountered suspicious emails, messages or links at work. Phishing led the list, reported by 39%. AI-generated scams were reported by 15%.

The more revealing figures sit next to each other. 83% said the potential consequences of cyberattacks are serious. Only 48% said they could recognise an AI-generated fake video. And just 18% said their organisation had experienced no cyber incident at all, as far as they were aware.

The Commission calls this a gap between awareness and daily practice. That is a polite way of putting it. People know the danger is real, and fewer than half believe they could spot the fake.

What the survey does and does not tell you

The survey measures what employees say about themselves. “Could recognise a fake video” is a statement of confidence, not the result of a test. Confidence in this area is not a reliable guide, as a banker who recognised a familiar voice discovered at some cost.

So the 48% is not a floor you can build on. It is closer to a ceiling.

The 15% figure deserves the same caution. It counts the AI-generated scams that employees noticed. A scam nobody noticed does not appear in it.

Why spotting AI-generated scams is the wrong defence

The usual answer to AI-generated scams is awareness training: show people examples, teach them the tells, test them with simulated phishing. That training still has value. However, it rests on an assumption the Fideuram case breaks. It assumes that the person at the moment of decision will notice that something is off.

A good fake removes exactly that moment. The message carries the right name. A voice you know confirms it. Unusual as the request is, it arrives with a reason why the normal route will not work. Recognition is the one thing the attacker has designed out.

The defence therefore has to work even when nobody recognises anything. That means procedure, not perception.

Controls that do not depend on spotting the fake

Four rules carry the load against AI-generated scams, and none of them requires anyone to detect a forgery:

• Verify through a channel you already hold. Call back on a number from your own records, never on a number, link or contact supplied in the request.
• Two people for unusual payments. A second approver who was not part of the conversation breaks the spell that a single target is under.
• Urgency and secrecy are triggers, not reasons. A request that cannot wait for the normal check needs it more than any other.
• A messaging app is not an approval channel. Money and sensitive data move on instructions from systems with an audit trail, whoever appears to be asking.

These rules are dull. That is their strength. They do not get worse as the fakes get better.

Where training still earns its place

Training should change its target. Instead of teaching staff to be detectors, it should teach them the procedure and give them permission to use it. A simulated exercise with AI-generated scams then tests the right thing: did the employee call back on a known number, not did they spot the forgery.

Permission is harder than it sounds. In the Fideuram case the message appeared to come from the head of the whole group. Seniority is part of the attack. An employee who is asked by the chief executive to move quickly needs to know, before it happens, that pausing to verify is expected and will be backed.

That backing has to come from the top. A verification rule that the board would waive for its own convenience is not a rule.

The question for your AI lead

AI-generated scams are not a technology problem that the next detection tool will solve. They exploit trust, hierarchy and urgency, and those are governance problems.

For the AI lead, this sits on the border between security and AI governance, which is how it can end up belonging to nobody. The attackers used AI as a tool. The defence is organisational, and someone has to own it.

So the question is concrete. Is there one payment, one data transfer or one access change in your organisation that a convincing message and a familiar voice could still push through without a second check?

If the honest answer is yes, the fix is a rule, and it can be written this week. AI-generated scams will keep getting better. The rule does not need to.