An interesting variant on fraudulent deepfake scams.
Kenny Li of Manta fame was sucked into a scam attempt, but was able to perceive the scam before any damage was done.
Li responded to a message from a known contact, which resulted in a Telegram conversation, which resulted in a Zoom call.
“In the call, there were team members who had their cameras on, and [the] Manta founder could see their faces. He mentioned that “Everything looked very real. But I couldn’t hear them.” Then came the “Zoom update required” prompt…”
Li didn’t fall for it.
(Imagen 3)
And one more thing…
The formal announcement is embargoed until Monday, but Bredemarket has TWO openings to act as your on-demand marketing muscle for facial recognition or cybersecurity:
Adobe is implementing C2PA-compliant content credentials in AEM [Adobe Experience Manager]. These can tell you the facts about an asset (in C2PA terms, the “provenance“): whether the image of a product from a particular manufacturer was created by that manufacturer, whether a voice recording of Anil Chakravarthy was created by Anil Chakravarthy, or whether an image of a peaceful meadow was actually generated by artificial intelligence (such as Adobe Firefly).
But perhaps you would prefer to hear from someone who knows what they’re talking about.
On a webcast this morning, C. Maxine Most of The Prism Project reminded us that the “Biometric Digital Identity Deepfake and Synthetic Identity Prism Report” is scheduled for publication in May 2025, just a little over a month from now.
As with all other Prism Project publications, I expect a report that details the identity industry’s solutions to battle deepfakes and synthetic identities, and the vendors who provide them.
And the report is coming from one of the few industry researchers who knows the industry. Max doesn’t write synthetic identity reports one week and refrigerator reports the next, if you know what I mean.
At this point The Prism Project is soliciting sponsorships. Quality work doesn’t come for free, you know. If your company is interested in sponsoring the report, visit this link.
While waiting for Max, here are the Five Tops
And while you’re waiting for Max’s authoritative report on deepfakes and synthetic identity, you may want to take a look at Min’s (my) views, such as they are. Here are my current “five tops” posts on deepfakes and synthetic identity.
My own post referenced the Auriemma Group estimate of a $6 billion cost to U.S. lenders.
McKinsey preferred to use a percentage estimate of “10–15% of charge offs in a typical unsecured lending portfolio.” However, this may not be restricted to synthetic identity fraud, but may include other types of fraud.
Thomson Reuters quoted Socure’s Johnny Ayers, who estimated that “20% of credit losses stem from synthetic identity fraud.”
Oh, and a later post that I wrote quoted a $20 billion figure for synthetic identity fraud losses in 2020. Plus this is where I learned the cool acronym “SIF” to refer to synthetic identity fraud. As far as I know, there is no government agency with the acronym SIF, which would of course cause confusion. (There was a Social Innovation Fund, but that may no longer exist in 2025.)
Never Search Alone, not National Security Agency. AI image from Imagen 3.
Back to synthetic identity fraud, which reportedly resulted in between $6 billion and $20 billion in losses in 2020.
The financial toll of AI-driven fraud is staggering, with projected global losses reaching $40 billion by 2027 up from US12.3 billion in 2023 (CAGR 32%)., driven by sophisticated fraud techniques and automation, such as synthetic identities created with AI tools.
Again this includes non-synthetic fraud, but it’s a good number for the high end. While my FTC fraud post didn’t break out synthetic identity fraud figures, Plaid cited a 2023 $1.8 billion figure for the auto industry alone, and Mastercard cited a $5 billion figure.
But everyone agrees on a figure of billions and billions.
The real Carl Sagan.
The deepfake Carl Sagan.
(I had to stop writing this post for a minute because I received a phone call from “JP Morgan Chase,” but the person didn’t know who they were talking to, merely asking for the owner of the phone number. Back to fraud.)
Reducing SIF in 2025
In a 2023 post, I cataloged four ways to fight synthetic identity fraud:
Private databases.
Government documents.
Government databases.
A “who you are” test with facial recognition and liveness detection (presentation attack detection).
Ideally an identity verification solution should use multiple methods, and not just one. It doesn’t do you any good to forge a driver’s license if AAMVA doesn’t know about the license in any state or provincial database.
If you sell hammers, then hammers are the solution to everything, including tooth decay.
I previously wrote about the nearly $3 billion lost to imposter fraud in the United States in 2024.
But how do you mitigate it?
“Advanced, contextual fraud prevention” firm ThreatMark has the answer:
“As impersonation scams use a wide range of fraudulent methods, they require a comprehensive approach to detection and prevention. One of the most efficient in this regard is behavioral intelligence. Its advantages lie mainly in its ability to detect both authorized and unauthorized fraud in real time across all digital channels, based on a variety of signals.”
The post touched on many items, one of which was the relative ease in using popular voice cloning programs to create fraudulent voices. Consumer Reports determined that four popular voice cloning programs “did not have the technical mechanisms necessary to prevent cloning someone’s voice without their knowledge or to limit the AI cloning to only the user’s voice.”
Reducing voice clone fraud?
Joel R. McConvey of Biometric Update wrote a piece (“Hi mom, it’s me,” an example of a popular fraudulent voice clone) that included an update on one of the four vendors cited by Consumer Reports.
In its responses, ElevenLabs – which was implicated in the deepfake Joe Biden robocall scam of November 2023 – says it is “implementing Coalition for Content Provenance and Authenticity (C2PA) standards by embedding cryptographically-signed metadata into the audio generated on our platform,” and lists customer screening, voice CAPTCHA and its No-Go Voice technology, which blocks the voices of hundreds of public figures, as among safeguards it already deploys.
Coalition for Content Provenance and Authenticity
So what are these C2PA standards? As a curious sort (I am ex-IDEMIA, after all), I investigated.
The Coalition for Content Provenance and Authenticity (C2PA) addresses the prevalence of misleading information online through the development of technical standards for certifying the source and history (or provenance) of media content. C2PA is a Joint Development Foundation project, formed through an alliance between Adobe, Arm, Intel, Microsoft and Truepic.
There are many other organizations whose logos appear on the website, including Amazon, Google, Meta, and Open AI.
Provenance
I won’t plunge into the entire specifications, but this excerpt from the “Explainer” highlights an important word, “provenance” (the P in C2PA).
Provenance generally refers to the facts about the history of a piece of digital content assets (image, video, audio recording, document). C2PA enables the authors of provenance data to securely bind statements of provenance data to instances of content using their unique credentials. These provenance statements are called assertions by the C2PA. They may include assertions about who created the content and how, when, and where it was created. They may also include assertions about when and how it was edited throughout its life. The content author, and publisher (if authoring provenance data) always has control over whether to include provenance data as well as what assertions are included, such as whether to include identifying information (in order to allow for anonymous or pseudonymous assets). Included assertions can be removed in later edits without invalidating or removing all of the included provenance data in a process called redaction.
Providence
I would really have to get into the nitty gritty of the specifications to see exactly how ElevenLabs, or anyone else, can accurately assert that a voice recording alleged to have been made by Richard Nixon actually was made by Richard Nixon. Hint: this one wasn’t.
Incidentally, while this was obviously never spoken, and I don’t believe that Nixon ever saw it, the speech was drafted as a contingency by William Safire. And I think everyone can admit that Safire could soar as a speechwriter for Nixon, whose sense of history caused him to cast himself as an American Churchill (with 1961 to 1969 as Nixon’s “wilderness years”). Safire also wrote for Agnew, who was not known as a great strategic thinker.
And the Apollo 11 speech above is not the only contingency speech ever written. Someone should create a deepfake of this speech that was NEVER delivered by then-General Dwight D. Eisenhower after D-Day:
Our landings in the Cherbourg-Havre have failed to gain a satisfactory foothold and I have withdrawn the troops. My decision to attack at this time and place was based upon the best information available. The troops, the air and the Navy did all that bravery and devotion to duty could do. If any blame or fault attaches to the attempt it is mine alone.
In 2023, 27% of people who reported a fraud said they lost money, while in 2024, that figure jumped to 38%.
In a way this is odd, since you would think that we would better detect fraud attempts now. But I guess we don’t. (I’ll say why in a minute.)
Imposter scams
The second fraud category, after investment scams, was imposter scams.
The second highest reported loss amount came from imposter scams, with $2.95 billion reported lost. In 2024, consumers reported losing more money to scams where they paid with bank transfers or cryptocurrency than all other payment methods combined.
Deepfakes
I’ve spent…a long time in the business of determining who people are, and who people aren’t. While the FTC summary didn’t detail the methods of imposter scams, at least some of these may have used deepfakes to perpetuate the scam.
…technology that simulates human activity, such as software that creates deepfake videos and voice clones….They can use deepfakes and voice clones to facilitate imposter scams, extortion, and financial fraud. And that’s very much a non-exhaustive list.
The results found that four of the six products — from ElevenLabs, Speechify, PlayHT, and Lovo — did not have the technical mechanisms necessary to prevent cloning someone’s voice without their knowledge or to limit the AI cloning to only the user’s voice.
Instead, the protection was limited to a box users had to check off, confirming they had the legal right to clone the voice.
And of course the identity/biometric vendor commuity is addressing deepfakes also. Research from iProov indicates one reason why 38% of the FTC reporters lost money to fraud:
[M]ost people can’t identify deepfakes – those incredibly realistic AI-generated videos and images often designed to impersonate people. The study tested 2,000 UK and US consumers, exposing them to a series of real and deepfake content. The results are alarming: only 0.1% of participants could accurately distinguish real from fake content across all stimuli which included images and video… in a study where participants were primed to look for deepfakes. In real-world scenarios, where people are less aware, the vulnerability to deepfakes is likely even higher.
So what’s the solution? Throw more technology at the problem? Multi factor authentication (requiring the fraudster to deepfake multiple items)? Injection attack detection? Something else?
And of course I referenced VeriDas in my February 7 post when it defined the difference between presentation attack detection and injection attack detection.
Biometric Update played up this difference:
To stay ahead of the curve, Spanish biometrics company Veridas has introduced an advanced injection attack detection capability into its system, to combat the growing threat of synthetic identities and deepfakes….
Veridas says that standard fraud detection only focuses on what it sees or hears – for example, face or voice biometrics. So-called Presentation Attack Detection (PAD) looks for fake images, videos and voices. Deepfake detection searches for the telltale artifacts that give away the work of generative AI.
Neither are monitoring where the feed comes from or whether the device is compromised.
I can revisit the arguments about whether you should get PAD and…IAD?…from the same vendor, or whether you should get best in-class solutions to address each issue separately.
When considering falsifying identity verification or authentication, it’s helpful to see how VeriDas defines two different types of falsification:
Presentation Attacks: These involve an attacker presenting falsified evidence directly to the capture device’s camera. Examples include using photocopies, screenshots, or other forms of impersonation to deceive the system.
Injection Attacks: These are more sophisticated, where the attacker introduces false evidence directly into the system without using the camera. This often involves manipulating the data capture or communication channels.
To be honest, most of my personal experience involves presentation attacks, in which the identity verification/authentication system remains secure but the information, um, presented to it is altered in some way. See my posts on Vision Transformer (ViT) Models and NIST IR 8491.
In an injection attack, the identity verification/authentication system itself is compromised. For example, instead of taking its data from the camera, data from some other source is, um, injected so that it look like it came from the camera.
Incidentally, I should tangentially note that injection attacks greatly differ from scraping attacks, in which content from legitimate blogs is stolen and injected into scummy blogs that merely rip off content from their original writers. Speaking for myself, it is clear that this repurpose is not an honorable practice.
Note that injection attacks don’t only affect identity systems, but can affect ANY computer system. SentinelOne digs into the different types of injection attacks, including manipulation of SQL queries, cross-site scripting (XSS), and other types. Here’s an example from the health world that is pertinent to Bredemarket readers:
In May 2024, Advocate Aurora Health, a healthcare system in Wisconsin and Illinois, reported a data breach exposing the personal information of 3 million patients. The breach was attributed to improper use of Meta Pixel on the websites of the provider. After the breach, Advocate Health was faced with hefty fines and legal battles resulting from the exposure of Protected Health Information(PHI).
Deepfakes utilize AI and machine learning to create lifelike videos of real people saying or doing things they never actually did. By injecting such videos into a system’s feed, fraudsters can mimic the appearance of a legitimate user, thus bypassing facial recognition security measures.
Again, this differs from someone with a mask getting in front of the system’s camera. Injections bypass the system’s camera.
Fight back, even when David Horowitz isn’t helping you
Do how do you detect that you aren’t getting data from the camera or capture device that is supposed to be providing it? Many vendors offer tactics to attack the attackers; here’s what ID R&D (part of Mitek Systems) proposes.
These steps include creating a comprehensive attack tree, implementing detectors that cover all the attack vectors, evaluating potential security loopholes, and setting up a continuous improvement process for the attack tree and associated mitigation measures.
As you can see, the tactics to fight injection attacks are far removed from the more forensic “liveness” procedures such as detecting whether a presented finger is from a living breathing human.
Presentation attack detection can only go so far.
Injection attack detection is also necessary.
So if you’re a company guarding against spoofing, you need someone who can create content, proposals, and analysis that can address both biometric and non-biometric factors.
Checking the purported identity against private databases, such as credit records.
Checking the person’s driver’s license or other government document to ensure it’s real and not a fake.
Checking the purported identity against government databases, such as driver’s license databases. (What if the person presents a real driver’s license, but that license was subsequently revoked?)
Perform a “who you are” biometric test against the purported identity.
If you conduct all four tests, then you have used multiple factors of authentication to confirm that the person is who they say they are. If the identity is synthetic, chances are the purported person will fail at least one of these tests.
Do you fight synthetic identity fraud?
If you fight synthetic identity fraud, you should let people know about your solution.
Bredemarket 