Using AI hacking tools? Then you’re ahead of the AI companies themselves, and the companies being hacked.
PYMNTS reported that Latin American banks were hacked by a group called CyberCartel. The group used AI tools, including Claude and Grok, to achieve their illicit aims.
The whole episode shows “hackers employing AI models to automate breaches into financial institutions. In this case, CyberCartel seems to have used the agents to help it steal password and consumer data, and to access customer bank accounts autonomously without much human effort.”
Because fraudsters have the same financial pressures as legitimate firms. If it’s too expensive to use U.S. humans, use offshore humans. And if that’s too expensive, use bots.
And if the tokens for the bots cost too much…well, that’s why CyberCartel was hacking banks.
Now of course Anthropic plugged CyberCartel’s access…after the fact. Because fraudsters can assemble AI hacking tools so quickly, they can often mount an attack BEFORE the target can defend against it.
The constant battle between fraudsters and the good people continues, just much more rapidly.
The Department of Defense didn’t put a human in the loop until well after an AI-generated report raised concerns. The Philadelphia Police were smarter, with a bot AND a human in the loop.
This story involves Anthropic and some testing that got out of control.
“The AI company Anthropic said its model was conducting a test involving interactions with randomly selected websites when it accessed PhillyUnsolvedMurders.com and submitted false information on an unsolved homicide. The tip claimed to come from someone with information on the case.”
The false report was filed on July 18, but Anthropic didn’t discover it until more than two months later, on September 28. This was particularly worrisome, since this could have resulted in a “confess your crime” waste of police resources (Kevin and Bean fans know).
The company instituted controls to prevent this in the future. Then Anthropic did the right thing and notified the Philadelphia Police Department on Wednesday (October 7). The two parties met the next day, and the police then investigated to see what happened to the false report.
Nothing. “Police then located the submission in the website’s tip records and confirmed the corresponding email remained in spam.”
As we all know, spam flags are often automated, occasionally letting spam get through, occasionally stopping real messages from being delivered. Check your spam folder every few days for the latter, especially if you’re expecting an important message.
But even if the false tip made it past the anti-spam bot, there were other protections.
“Regardless of who submits information or how it reaches the department, a tip is a lead to assess – not an established fact. Investigators evaluate its credibility and seek corroborating evidence. An automated submission does not bypass that process.”
After all these years, someone is finally acknowledging that an investigative lead is only a lead.
“When you asked me about my AI use before I started the work, I outlined my process for user-controlled AI. Specifically, I wrote the first draft myself, and then used generative AI to touch up bits and pieces of the text. I probably rejected half the suggestions, but there were a few good ones that I adopted.”
The customer wasn’t calmed.
“But…it’s SLOP!”
“I’m sorry my work was not to your standard,” the consultant replied. “How about if I withdraw the draft; there are portions of it that I can adapt for other work. The text returns to me, you don’t owe me anything, and you can find another writer to create your text with no assistance, no AI, no Grammarly, no Copilot-powered spellcheck.”
“You’d take the text back?” the customer protested.
“You said it was slop. So you obviously can’t use it.”
The customer became emphatic. “No, I do want it…for half price.”
“I wouldn’t want you to accept work that you think is bad,” the consultant answered, his irritation starting to show. “So I’m taking the rights back. If you publish it, I will invoice you…in full. Here’s Upwork’s URL; I’m sure you can find someone who will write the text you want.”
Yes, I’m finally to THAT issue: hidden watermarks inserted by Anthropic’s Claude (and by other AI tools via content credentials…sometimes). So let’s look at those first before looking at Claude, because that will be instructive.
Credentials from the Coalition for Content Provenance and Authenticity
Let’s go back to last November, when I noticed that LinkedIn inserted a symbol on a Google Gemini image I uploaded.
LinkedIn.
LinkedIn had detected image metadata that revealed the origin of the image.
LinkedIn.
I did not freak out over the revelation that the image in question was created by Google generative AI. I usually disclose such things anyway.
Unfortunately, this metadata is not always there, and not always revealed. If I download an image and then directly upload it to LinkedIn, the content credentials symbol appears. But if I directly upload it to Facebook or WordPress, it doesn’t. And if I upload it to WordPress then download it, the content credentials are often stripped.
Why did Claude do this anyway? According to Superpath, the European Union.
“The EU recently put the AI Act in place to regulate AI products and protect normies like you and me from potential risks of nefarious AI use. They’re mostly trying to prevent stuff like manipulation (e.g., deepfakes) and classification (e.g., facial recognition databases).
“Part of the AI Act forces LLMs like Claude to clearly label when content has been processed by AI.”
To understand what “processed” means, we need to understand the three uses of AI-generated text.
Don’t use AI-generated text at all.
Only use AI-generated text.
But the two extremes are rarely used (except for Bredebot). Instead we opt for the third choice.
Use a mixture of human-written and AI-generated text.
As you know from what I’ve shared before, I use AI for ideation (before I write a single word) and touch ups (after I have drafted it). So if the ALL CAPS material below represents AI-generated text, the true origins of my content may appear like this:
INCREASING REVENUE WITH Focused Messaging
HOW DO YOU GET PROSPECTS TO BUY? NOT WITH messaging that wanders through the plains like a drunken wildebeest.
So if some tool analyzed text that was partially written by Claude, you would get a confusing mishmash in which every fourth word was written by Claude and the other words weren’t.
Google Gemini.
Assuming you pasted the Claude output into your document. But what if you just read Claude’s suggestions…and manually typed them in yourself? No one would know.
Google Gemini.
What does this mean?
Now let’s look at the effects of Claude and other watermarks. Here’s what Superpath says:
“I am willing to put my stellar reputation on the line and confidently predict this will have little-to-no impact for B2B content folks who use (or don’t use) AI.”
Think about it.
The folks who insist that you should never ever use AI, Wikipedia, or Cliffs Notes already get bent out of shape when they see a single em dash—and they’ll continue to freak out if they discover Claude corrected the spelling of “ephemeral.”
The folks who are already freaked out over Bredebot’s 100% AI writings will continue to freak.
Everyone else won’t care. You would think that Bredemarket’s clients would want to know if I was really writing all my stuff for them, but as long as they like the output they never ask. (And I’m an opinionated writer anyway.) In fact, one client set me up with a paid Perplexity account to help in my analysis work for the client.
So I’ve dispensed with that tempest in a teapot and can move on to more important things…such as cultural appropriation in AI images.
Google Gemini. Check the woman with the mustache on the left.
Anthropic states (possibly in self-interest) that unemployment could hit 20% in five years.
One quarter of all programming jobs already vanished in the last two years.
A LinkedIn executive echoed the pessimism about the future (while LinkedIn hypes its own AI capabilities to secure the dwindling number of jobs remaining).
The Federal Reserve cited high college graduate rates of unemployment (5.8%) and underemployment (41.2%).