Artificial intelligence threats and defending against them
An attack no longer requires skill, only machine time. A cloned voice, a convincing message and tireless persistence change the defence calculus.
The scale changes, not the technique
Executive impersonation is as old as the telephone. What is new is that it now sounds like the executive.
The attack has not changed in concept but in cost and credibility. A decent voice clone needs a few minutes of recording, and there is plenty of that at every conference and in every post. A message once recognisable by its poor language is now written flawlessly and tailored to its recipient.
On top of that comes agentic artificial intelligence: a system that does not merely answer a question but takes steps of its own, searching, writing, sending and repeating. In an attack that means tirelessness at a cost close to zero, and behaviour that shifts from one attempt to the next.
Once an attempt becomes cheap enough to be repeated a thousand times, defence can no longer rest on the tool alone. What helps most in practice is not a new device but a rule that a payment instruction is confirmed through a second channel, and a rule that applies to the management too.
Where defences give way
Five patterns we see in practice, with the Annex II measure each one touches.
These map onto measure 4, human resources security and digital identities, measure 5, cyber hygiene, measure 8, supply chain security, and measure 11, incident handling. In other words, defending against these attacks is not extra work alongside aligning with the Act but part of it.
What we do with you
Six steps that need no new tool, only clear rules and one exercise.
Reviewing the paths money takes out
We go through who may initiate a payment, who approves it, and where those two steps can collapse into one person. That is exactly where the attack leans.
A second-channel confirmation rule
A payment instruction and a change of bank details are confirmed by a call to a number from your own records. The rule has to apply to the board too, otherwise it applies to nobody, because the attack always presents itself as the exception.
Recognition training for higher-risk roles
Finance, procurement, the executive office and IT. The aim is not that an employee spots every fraud, but that they know they may stop and check without being second-guessed.
A channel for reporting suspicion
One address or one number, known to everyone, where you can report without explaining yourself and without fear of looking foolish. It is better for somebody to speak up too early than to wait until they are certain.
Reviewing suppliers with access
A list of those with access to your systems or data, with security clauses in the contract and a rule for changing bank details. An attack through a partner passes most easily where the partner is taken for granted.
Recording attempts and the duties that follow
An attempt that failed is still an incident to be recorded, and a record of a blocked attack is one of the few proofs that monitoring works. If money left or data was exposed, the reporting deadlines start running.
Why the attack is no longer recognised in advance
Three assumptions classic defences rest on, and why they no longer hold.
Frequently asked questions
The questions we are asked most often before we start.
Is there a tool that solves this?
There is no single one. What helps most is the rule that a payment instruction is confirmed through a second channel, together with training and clear accountability. Tools help, but a rule that holds without exception also holds when the tool lets something through.How convincing is a cloned voice today?
Convincing enough to pass on the phone. A decent voice clone needs a few minutes of recording, and there is plenty of that at every conference and in every post. That is why the defence does not rest on recognising a voice but on confirming through a second channel.What is agentic artificial intelligence?
A system that does not merely answer a question but takes steps of its own: searching, writing, sending and repeating. In an attack that means tirelessness at a cost close to zero.How do you defend a payment instruction?
By confirming through a second channel. The rule is set out in step 02, and it applies just as much to every change to a supplier’s bank details.Does training help if the deception is that good?
It helps, but not on its own. The aim is not that an employee spots every fraud but that they know they may stop and check without being second-guessed. That is why training always comes with a channel for reporting a suspicion.Does such an attempt carry regulatory obligations?
It does. As a rule it is an incident to be recorded, and if money left or data was exposed the reporting deadlines run: an early warning to the competent CSIRT within 24 hours, a notification within 72 hours, and for a personal data breach a notification to the supervisory authority within 72 hours.Where does an attacker get information about us?
From public sources, and mostly from those we publish ourselves: registers, websites, social media, conference posts and announcements of new contracts. A review of your own public footprint is therefore a cheap and useful first step.Are the figures on the rise in such fraud reliable?
They come from external reports rather than legislation, so we always cite them with a source and with the caveat that they are estimates. What matters for the decision is not the exact percentage but the fact that the attempt has become cheap and repeatable.Do you carry out technical resilience testing?
We review exposure and the processes around it. Penetration testing we deliver through partners, because it is a separate line of work with its own rules and its own liability.How long does it take to put these rules in place?
The second-channel confirmation rule and the reporting channel can be set up in a few days. The exercise and the supplier review take a few weeks. What takes longest is amending supplier contracts, because that requires negotiation.Where to go next
The measures where this defence actually lives.
From the blog: artificial intelligence
The three latest texts on AI threats and AI governance.
Let us see how you would hold up today.
Half an hour, no obligation. We walk through how a payment instruction moves in your organisation and where a cloned voice would get through.