Can You Still Trust What You See and Hear Online?

AI has become part of everyday life very quickly. We use it to write emails, generate images, summarize documents, translate languages and help with work. But the same technology is also making online scams much harder to recognise.

A scam email used to have obvious warning signs: spelling mistakes, strange wording or a suspicious link. Today, AI can help scammers create professional messages, realistic images, cloned voices and convincing fake videos.

The basic idea behind fraud hasn't changed. Someone still wants to convince another person to send money, reveal information or provide access to an account. What has changed is how convincing and scalable that deception can be.

The scale is already significant. Consumer losses reported to the FBI's IC3 reached $16.6 billion in 2024.

When Recognising a Voice Isn't Enough

According to the research underpinning this analysis, as little as three seconds of recorded audio can produce an 85% voice match. Human listeners correctly identified synthetic speech only 48% of the time — no better than a coin flip.

This doesn't mean distrusting every phone call. It means changing how we verify unusual requests. Calling back on a known number — or asking something only they would know — is now better practice than relying on voice alone.

Deepfakes Are More Than Fake Celebrity Videos

Deepfakes are usually associated with fake celebrity clips or misinformation, but they are also becoming a practical cybersecurity problem. Many companies now verify customers remotely using video-based KYC. Attackers are attempting to use synthetic video to imitate facial movements, blinking and head rotations to bypass these systems.

Some Fake Identities Don't Belong to Anyone

AI fraud isn't limited to impersonating real people. Criminals create synthetic identities — "Frankenstein IDs" — combining a stolen Social Security number with a fabricated name, date of birth and an AI-generated photograph. Over time, that identity builds credit before being used to obtain large loans or credit lines. Synthetic identity fraud is projected to cause at least $23 billion in losses by 2030.

AI Is Changing Phishing Too

Traditional phishing advice: check the grammar, look for unusual wording. That advice isn't useless — but it isn't enough anymore. Generative AI can produce professional, contextually appropriate messages that pass every surface check.

Business Email Compromise (BEC) is particularly important because a fraudulent payment request doesn't necessarily contain malware or a suspicious attachment. Sometimes the message itself is the attack. From a SOC perspective, this is a meaningful shift — security teams can't rely entirely on traditional indicators of compromise when the attack is based on identity, context and human behaviour.

AI Is Also Part of the Defense

Organizations are also deploying machine intelligence to identify suspicious behaviour. Behavioral profiling examines device information, typing behaviour, mouse movements and session context — asking not just "did you enter the right password?" but "does your entire behaviour pattern match the legitimate user?"

What Can We Actually Do?

When an unusual request involves money, passwords, account access or sensitive information, verify it through another channel:

The Bigger Change

The most important part of AI-enabled fraud isn't simply that criminals have another technology available. It's that the signals we've traditionally associated with authenticity are becoming weaker:

• A professional email doesn't necessarily mean the sender is legitimate.
• A familiar voice doesn't necessarily prove who's speaking.
• A realistic photograph doesn't necessarily represent a real person.
• And video increasingly needs verification rather than automatic trust.