Sounds like AI is just greasing the wheels of a long established 'grandparent scam'... goes something like this:
1) voice one: young adult calls, sobbing 2) grandparent inquires with a name... "Ben, is that you?" 3) voice one: "Yes grandma, it's me, Ben... I'm in trouble, please don't tell mom 4) voice two: "Hello, I'm attorney..."
My grandmother fell victim to this almost 20 years ago, which only stopped when Western Union refused to let her continue sending wires... she was forced to call her daughter (at which point they just called my brother.)
Our takeaway (at the time)... the voice doesn't even need to be terribly accurate, since the original interaction is brief / somewhat inaudible over the tears. Typically just requires an older vulnerable adult, a lucky strike with the initial setup (e.g. grandparent actually has a grandkid), and a lot of high pressure / duress salesmanship.
"Greasing the wheels" seems right in principle, but possibly putting the accelerant factor a bit... mildly. Like going from burning the turkey in the oven, to deep frying and burning your whole house down.
> cybercrime losses across the United States rose 26 per cent in a single year
> The FBI was candid that even these figures understate the problem. AI attribution in the report reflects only what victims recognised and reported, and most victims of a cloned-voice call never learn that a machine was involved at all.
> INTERPOL found that AI-enhanced fraud is roughly four and a half times more profitable than its traditional equivalent, and that so-called agentic AI systems can now autonomously plan and execute entire fraud campaigns, from reconnaissance through to the ransom demand.
I told my parents that I will never ask for money, doesn't matter the situation, even with live video, it's trivial to generate live audio and video nowadays.
i agreed key phrases. id recommend it. something unrelated to the family and totally arbitrary, agreedupon only verbally. (or write it down for them if they are old and memory is an issue. you can remind them to read your note out loud.. easy).
this way, you do not footgun yourself in the event you'd ever need to ask something. Money isnt the only thing they can ask, and no one (i think) has a glass orb to tell their future and know for certain such a call would never happen. its easy to think it wont happen to you, i think that is most peoples' sentiment until it does. (having a need for help from family that is)
Key phrases make sense to put in place, but another easy safeguard is:
"Before you send anything to anyone, ever, call them back. Doesn't matter if it's me, the bank, a lawyer, whatever... tell them 'hang on I have another call coming in, let me just call you back in a few minutes, okay?'"
Not a thorough safeguard, if scammers have half a brain cell they can provision a VOIP number for such a request. They’re nothing if not accommodating.
I overheard a cab driver being scammed by someone claiming to be HMRC (UK version of the IRS), and when he asked to call them back they managed to convince him to call them back on Viber, and he was about to when I intervened and pointed out to him that 1) what I'd overheard was blatantly a scam, 2) if he was unsure, to go to their official website and find their number...
If you're going to get people to call you back, it has the problem of ensuring that they call you back on your real number - giving reasons why they have to call you back on some other number is way too easy ("I've lost my phone", "my phone is at home", and so on)
“But, Ben, you didn’t ask me for money when you called last night at 2AM. You asked for $4,000 in Fortnite gift cards to be released from Mexican prison. Thankfully 7-11 had them and was open!”
I recently did the same, and I, too, hope they got the message. We agreed that there is nothing that needs to be acted upon immediately, and that anything questionable would be discussed with the whole family first.
I remember my JROTC instructor also running into that and how she said afterwards they have a secret phrase between them two as a way of verifying it's truly them.
It's very, very hard for untrained people to be strict about verifying any secret phrase. The attacker can make all kinds of excuses, while creating urgency, and many people quickly abandon verifying the phrase. A scene in One Battle After Another comes to mind.
Oh, man. That does seem likely. What a world. I wonder if eventually there won’t be a human in the loop, just a model trained to make money with a strategy like that, automatedly selecting victims and a person to be impersonated for each. Pre-render a few videos, place multiple calls in parallel. Basically a turnkey Docker container that takes a bitcoin address as a parameter and fills it with stolen money.
Just add video and a better voip endpoint to existing setups and you're already there. A lot of scams can be (and are) run by a small handful of people now. Phishing at scale with stuff like fake tracking links etc. is already fully automated by people in a ton of ways and can definitely handle some back-and-forth over sms/whatsapp/whatever.
lucky strike is the key here they can do this with VOIP really easily to massive amounts of numbers. its staggering amounts if you look at the traffic really. worst is if they proxy that via residential proxy services which often come from end-user / individuals phones so the traffic is hard to detect for carriers etc. since it looks like a regular VOIP app connection.
> LPT: Please have a codeword or phrase that you use with your loved ones
They keep refusing ideas like these on the grounds of them being “not stupid” and “able to see through such attempts immediately, 100% of the time” and “do you think we’re stupid?”
I'm getting a lot of calls recently and don't give them more than a Hello and whatever music, radio show or Tour de France broadcast I'm listening to. Sometimes they hang in there for half a minute.
Sometimes if I’m suspicious about the number now, I just answer and say nothing. A human will get confused after 5 seconds and say “Hello?Hello??” But the very shitty bots that usually call, just wait patiently for a long time for your hello, and don’t seem at all fazed by it.
A terse, altered "Hello" is all I say. Sometimes I don't say anything. Most humans would wait a few seconds then prompt with "...Hello?", whereas bots tend to hang up after ~2s silence
I somewhere read about a service that would use AI generated voices to combat these scam calls, basically talking to the forever. Forgot the name though...
> Every article published on SmarterArticles is authored and editorially controlled by Tim Green. Artificial intelligence tools are used within a structured and supervised workflow as research and drafting instruments. All arguments, framing decisions, source selections, and final publication choices remain human-directed and under my full responsibility.
There are references at the bottom, but I would have preferred direct links or footnotes within the article. Also, direct quotes are nice. I didn’t notice any glaring AI cliches.
> Artificial intelligence tools are used within a structured and supervised workflow as research and drafting instruments.
This has become the highbrow way of admitting that AI is writing the articles: Calling it “drafting” is another way of saying that the article was written by AI and the person publishing it maybe reviewed it. Maybe they skimmed it and published it directly.
For what it’s worth the article felt obviously AI heavy to my first read.
> Calling it “drafting” is another way of saying that the article was written by AI and the person publishing it maybe reviewed it.
I don't think that's a safe assumption. You could construct an article pretty quickly if you had a topic, a few points to hit, a conclusion and a short list of links, then fed that to the machine. All the LLM would be doing is fluffing it up with worthless words, unnecessary metaphors and maybe a pop culture reference or two so it looks like what people expect from an "article."
"Rewrite this as a slate dot com article"
If it were an email newsletter instead, there wouldn't be that fluffy expectation and you could just leave the bullet points and links as they were.
> I don't think that's a safe assumption. You could construct an article pretty quickly if you had a topic, a few points to hit, a conclusion and a short list of links, then fed that to the machine. All the LLM would be doing is fluffing it up with worthless words, unnecessary metaphors and maybe a pop culture reference or two so it looks like what people expect from an "article."
We’re not disagreeing? That’s basically what I said: The AI wrote the article. Saying it drafted the article is a way of admitting it was written by AI but making it sound like it was actually a journalistic endeavor.
A good example of someone using AI as a tool and owning their output they should. The craftsman is still responsible for straightening the bent nail, even if it were the hammer that slipped.
Interesting, something about the phrasing and pacing felt like AI to me, but not a model I’m familiar with - I guess that’s why. Usually I close articles once I realize they’re AI written, but this one was mild enough that I finished the whole thing.
I suspect this is much more common than you're imagining. I think it'd be silly for publishers of any kind, really, to not use AI tools for things like fact checking and so on.
Interesting. To me it reeks of AI, and the first major tell was the first paragraph, at which point I stopped reading.
> No human had. The crying had been synthesised from a fragment of audio, and the daughter she thought she was rescuing existed only as a pattern of numbers in someone else's machine.
Yeah, I can't explain it but the headings feel very AI. And there are a bunch of things in the text that remind me of chats I have with Claude during the day job
>and it is worth being clear about why
>The emotional mechanism the scam exploits is not a gap in knowledge that a leaflet can fill; it is the love a person has for their grandchild, weaponised
>These are not the numbers of a credulous minority being separated from pocket money. They are the numbers of a generation's accumulated savings being drained
>that a fraud requiring the absolute frontier of machine learning can be perpetrated against an ordinary grandmother in her kitchen, at scale, for the price of nothing
(To be clear this isn't automatically disqualifying to me. But I am interested in LLM writing patterns and my ability to detect them. And in the case of this article I sense the kind of linguistic padding that has made Claude a little harder to work with in the more recent Opus point releases because it obscures the most important bits of information.)
That's pure generic "I'm getting paid by the word" article writing. If it reads to AI to you it's in part because AI has been trained on a billion articles like that.
AI definitely amplifies this problem, but it's not like it didn't exist before. Old people get scammed the old way all the time too. My mom calls me every once in a while asking about some freebie offer that she gets emailed from sketchy domains claiming to be spotify or something.
Not saying that "there's nothing we can do" or anything, but it does feel like this is one of those instincts that you develop growing up with the internet. Like, my first instinct reading that (and I hope getting that call) would be "what the hell is the lawyer doing at the scene". You have to treat _everything_ coming through your phone as potentially untrusted. I don't have any data on this, but it feels like my friends, and especially younger people, do that automatically.
The primary defence against all phishing is to tell yourself: nothing is ever really that urgent. Nothing is ever that good.
The problem described in the article is unsolvable, given that a mid-range desktop from a few years ago can easily clone a voice that's convincing enough and there are no guardrails to those. Some silly KYC laws might limit a highschool kid making deepfakes of his crush, but once a model exists it's trivial to spread it around, and for organized groups to get ahold of those. Similar will happen with images, it's just that nobody with any serious money bothered releasing image gen models that compete with gemini or chatgpt -- but it's just a question of time. A year or three, what difference does it really make?
As the cost goes down to near-zero you can scale it up almost infinitely, especially if the profits are high enough to get some smart people working on the problem, which going by the article is already the case ("INTERPOL's finding that AI-enhanced fraud is four and a half times more profitable than the traditional kind"; incidents rose by 26% last year). If AI does succeed on mutilating white collar work enough there will be a large supply of knowledge workers that might just join International Scam Co. rather than have their families go homeless. Drowning man clutching at straw and all.
So if technologically it's impossible to prevent and societally it's impossible to prevent (like the attorney that got pwned same as the grandma), I'm not sure if there exists an answer that isn't worse than the thing it's supposed to prevent. I suppose we'll soon be in a situation where nothing we don't directly perceive in real life is provably true. That journalism and media in general seem to be in a deep crisis of trustworthiness means that you won't even get the benefit of the chain-of-trust as a proxy for whether something is or isn't real.
Ignoring everything happening outside of your immediate surroundings is a choice, and probably even good for people's mental health, but my gut feeling is that it does make humanity as a whole dumber and disempowered. What does corruption matter if nobody cares, or even hears about it? It was AI generated by $current_enemy anyway; nothing to see here, citizen.
> The problem described in the article is unsolvable
Well, not completely unsolvable. But nobody would like the solution.
What all these scams rely on is a way to transfer money in an irrevocable fashion. Restrict that in meaningful ways and you end a lot of the abilities for these scams to operate.
You could, for example, outlaw gift cards as a start. You could force the likes of Western Union to have a holding period before releasing money. Crypto would be hard as any regulation against it is pretty easily circumvented, but you could outlaw crypto currency exchanges (I'd worry less about crypto though as it's pretty hard for grandma to reliably setup).
> (I'd worry less about crypto though as it's pretty hard for grandma to reliably setup).
Difficulty in setting up a money transfer is not a hindrance. I have heard stories of scammers walking someone through the entire process to getting a mortgage on a house on the A&E Intervention episode of Greg. If they think you have money and they think you are gullible, they will devote time and effort to getting it.
There's a reason scammers rely heavily on things like gift cards, it's because hiring mules is expensive and creates a trail police can follow back to the scammers. It requires them to be in the same locale as the person they are scamming. Mailing cash is also pretty dicey for the scammers because you have to send the mail to a valid address. That becomes something police can trace.
If you wanted to completely eliminate scams then yeah, you'd also outlaw cash.
Also education. No court or attorney is going to demand payment of any fines or bail in gift cards or send a courier to pick up cash.
High schools should teach how to spot a scam. As others have observed, this is not a new one, it's just gotten more high-tech and convincing. This is one of many practical things our schools should teach about that they just don't.
Yet all of this can be easily defeated with soft language. The basic check "what's the password/verification word" will defeat this every time. This is basically opsec that we taught my grandparents, who were in their 90s. Its doable.
Yes but that's more of a mitigation than prevention. It's an additional step, you have to remember to do it, and under the pressure of the situation you might easily forget to do it.
I don't know about that but finding excuses for the scum of this earth is certainly not a solution.
Take Europe for example: nobody dies of hunger in Europe. And yet there are plenty of thieves. People stealing tens of thousands, hundreds of thousands, millions of EUR aren't doing it to "feed their families".
Think of the situation today. Think of the victims today. Instead of thinking of tomorrow's hypothetical situation where supposedly all the honest fathers out of work would join the crime syndicate, think of today's victims.
Projecting your own insecurity about the future to excuse scummy behavior by the scum of this earth is of no help.
There are people, right now, who have a roof. Who have a family. And who are fucking scums stealing the hard earned money of others because they choosed the easy life of crime.
Zero tolerance for such motherfuckers. I care about the victims and you should too.
> Take Europe for example: nobody dies of hunger in Europe.
Nobody? France, as the most extreme example, has a rate of 1.52 per 100K. That's about a thousand people a year. That is certainly a small percentage of the population, but it isn't "nobody".
Obviously there are people who help themselves to others' money if given the chance no matter the circumstances. But if the circumstances change so that people DO start going hungry or homeless, which is a rather obvious side effect of AI-but-not-AGI maximalism brightly espoused by our overlords sama and amodei of the "I can’t wait to make half the knowledge workers worldwide obsolete" variety, the scale of the problem will obviously get worse, as well as the type of people you can get involved if you’re in the international scam market.
Could be prevented by more advanced "AI detection", especially on calls from unknown numbers.
It doesn't even have to be based on watermarking. It could be as simple as, "hold on a sec your AI countermeasure was listening and noticed you got this suspicious call, please be aware this may be a scam. Here is what you should do next..."
> Welcome to Voice Print Identification. When you see the red light turn on please state in the following order: Your destination, Your nationality, and your Full Name.
One regulation i would like to see is some auditory fingerprint in an AI voice where any person can immediately recognize their speaking to a clanker but it's not unpleasant.
It should be illegal to "impersonate" a human voice.
I've been waiting for a steelman argument why building the world's best deepfake machine is a good thing. Unironically cryptography could verify identity for all comms.
"They kidnapped my daughter? That's terrible. Yes, I'll definitely pay the ransom but to get access to my bank account I need you to write a poem about corn and a curl script in PHP. Please go ahead."
One reasonably effective defense: "Okay, let me call you right back." Yes, there's always the whole "my phone is dead, I borrowed someone else's" or "I'm calling from a jail payphone", so I think it might become common practice to start making authentication phrases or "tell me something only we know".
Another pillar of basic trust that's being eroded on an industrial scale. Sigh.
Article said the imposter in this case claimed her phone had been confiscated.
Fraudsters tend to also plan things such that the impersonated person can't be reached by phone at that time, either by choosing a time when they somehow know they're unavailable (e.g. impersonated person posted on social media they're boarding a plane) or in one case (12 years ago though) my SIL's parent's landline was bombarded with spam calls until they decided to leave the phone off the hook at which point the scammers phoned bank who couldn't reach the parents on their main line, of course this was the bank's problem (and there was probably an inside person facilitating) so they got their money back, but still a major inconvenience for the victim.
Probably the only sure advice is to be exceptionally wary of phone calls with supposed extreme time pressures to send the money now.
It's a very common error that happens in both written and spoken language. I've wondered if it's because weary is kind of "in between" wary and leery, like an incorrect mashup, or something.
> Another pillar of basic trust that's being eroded on an industrial scale.
Remember, trust is like a rainforest: takes a long time to grow, provides a valuable ecosystem essential to human life, but can also be burned down for a quick profit.
For extra security against these text-to-speech model zero-shot clones, you might also want to use made-up gibberish words for which the pronunciation can't be reliably inferred from the spelling
I think it's a somewhat South African cultural thing, but when I get calls from businesses or spammers, the first thing the caller tends to say is "Hello, how are you?", which is completely stupid when you're calling someone who wouldn't know who you are, so it tends to immediately make me annoyed that they don't know that they should have introduced themselves first.
As 99% of the time these are spam calls, I used to respond with something like "I'm fine, but who are you / do I know you?", but that was pretty much always inefficient as that might say their name (which from a spammer is useless information), maybe a sales pitch "how much do you spend on x?" or maybe something deliberately misleading about their company and saying something like <major brand name> even though they're some independent sales crowd getting commission selling contracts for them.
Eventually I found that the most effective response is "Sorry. Where are you calling from and what is this in regard to?" which I've found without fail seems to surprise, disarm them and immediately elicit whether the call is a waste of my time. At which point I either become very friendly (because it's a call I'm expecting) or I simply respond with "Sorry, not interested, goodbye." and immediately put down the phone.
I just want the disruption to be as minimal as possible and to not let myself even get an emotional reaction from it, so I don't want to get annoyed at them, never mind wasting time telling them off, besides, I suspect that my ruthlessly efficient getting rid of them without them even having a chance to try their pitch is received as a super cold shoulder, akin to being told to f-off.
me and my wife made up a word in 2024 for this. the word doesn't exist in any language. we say it to each other all the time. even if i give you the spelling for it, you will say it wrong. i recommend everyone to do something similar. i should do it with my parents too.
our family has had a special 'code word' we have had since the kids were in elementary school. If someone ever needed to pick up our kids from school (they never did) our kids were taught to ask for that word.
This is a good reminder that we should review that, since its been 10 years or so.
This. All of this is a solved problem. It's just not a thing that most families do and do regularly. Code word, insider info, etc. "Oh I am so sorry you got arrested Tommy. Before I wire the $, where did we go on vacation last year?'
The opening example is of a person listening to their daughter’s voice on an unknown number, how would calling them back help? Or am I missing something obvious?
What’s terrible is each time I am forced to call the bank, the more they try to tell me voice ID is secure and want me to provide my voice to authenticate. Never. Did ya’ll never play Uplink? With voice cloning as good as it is now, there’s no way a voice ID is secure enough for authentication.
Someone else pointed out how easy it would be to make a video of anyone having a finger cut off, or similar torture, to scare the victim into believing that the “grandchild” forgot their password and needs you to override the password protocol to save their other 9 fingers. I have to agree. That’s like 80¢ of compute to do. If that’s effective even 30% of the time, it means “just have a password” doesn’t make you safe.
> It requires, second, regulating the supply of the weapon
Heavy sigh. The “weapon” is software. It cannot be regulated unless we live in the fascist dystopia where I have to ask the governments approval to run any piece of software.
We (the people) have pushed body cams on almost all law enforcement at this point, which had a noble enough motivation behind it -- but we also have pushed for and have various public disclosure laws (also well intended!) that mean those body cams are torrents of data entering the public sphere if anyone simply asks for it.
Now in the era of AI, this means anyone in the vicinity of an officer has a voice sample in the public domain, plus potentially their image.
Complex issue. I like body cams, I like freedom of information laws, but don't love this particular outcome.
What are legitimate uses for copying someone's voice without permission? I see none. Those scientists are just helping criminals to fully automate scamming and governments to create fake videos.
Well of course as you pointed out the legitimate one would be copying voices WITH permission (yours, someone you know who gives authorization, through contracts for movies/bots etc). The model can’t differentiate between voices for which you have permission or not.
But more generally while recordings might be copyrighted, the voice itself isn’t so copying a voice isn’t a crime, at least as it currently stands. You cannot however use said voice for deceptive practices. You can however for advertisement (needs permission). And in the US you can for satire, at least in the US, withOUT permission (falls under the 1st amendment).
Maybe the voice should be copyighted or protected then, I do not want anyone use to my voice, let them use their own boring afwully sounding voice. I am sure such models are used in 99% cases for illegal purposes or creating fake news (I think Russian propaganda used it).
Sounds like AI is just greasing the wheels of a long established 'grandparent scam'... goes something like this:
1) voice one: young adult calls, sobbing 2) grandparent inquires with a name... "Ben, is that you?" 3) voice one: "Yes grandma, it's me, Ben... I'm in trouble, please don't tell mom 4) voice two: "Hello, I'm attorney..."
My grandmother fell victim to this almost 20 years ago, which only stopped when Western Union refused to let her continue sending wires... she was forced to call her daughter (at which point they just called my brother.)
Our takeaway (at the time)... the voice doesn't even need to be terribly accurate, since the original interaction is brief / somewhat inaudible over the tears. Typically just requires an older vulnerable adult, a lucky strike with the initial setup (e.g. grandparent actually has a grandkid), and a lot of high pressure / duress salesmanship.
"Greasing the wheels" seems right in principle, but possibly putting the accelerant factor a bit... mildly. Like going from burning the turkey in the oven, to deep frying and burning your whole house down.
> cybercrime losses across the United States rose 26 per cent in a single year
> The FBI was candid that even these figures understate the problem. AI attribution in the report reflects only what victims recognised and reported, and most victims of a cloned-voice call never learn that a machine was involved at all.
> INTERPOL found that AI-enhanced fraud is roughly four and a half times more profitable than its traditional equivalent, and that so-called agentic AI systems can now autonomously plan and execute entire fraud campaigns, from reconnaissance through to the ransom demand.
I told my parents that I will never ask for money, doesn't matter the situation, even with live video, it's trivial to generate live audio and video nowadays.
I hope they got the message.
Fortunately for me, my parents wouldn't be able to get the live video working.
i agreed key phrases. id recommend it. something unrelated to the family and totally arbitrary, agreedupon only verbally. (or write it down for them if they are old and memory is an issue. you can remind them to read your note out loud.. easy).
this way, you do not footgun yourself in the event you'd ever need to ask something. Money isnt the only thing they can ask, and no one (i think) has a glass orb to tell their future and know for certain such a call would never happen. its easy to think it wont happen to you, i think that is most peoples' sentiment until it does. (having a need for help from family that is)
Key phrases make sense to put in place, but another easy safeguard is:
"Before you send anything to anyone, ever, call them back. Doesn't matter if it's me, the bank, a lawyer, whatever... tell them 'hang on I have another call coming in, let me just call you back in a few minutes, okay?'"
Not a thorough safeguard, if scammers have half a brain cell they can provision a VOIP number for such a request. They’re nothing if not accommodating.
I overheard a cab driver being scammed by someone claiming to be HMRC (UK version of the IRS), and when he asked to call them back they managed to convince him to call them back on Viber, and he was about to when I intervened and pointed out to him that 1) what I'd overheard was blatantly a scam, 2) if he was unsure, to go to their official website and find their number...
If you're going to get people to call you back, it has the problem of ensuring that they call you back on your real number - giving reasons why they have to call you back on some other number is way too easy ("I've lost my phone", "my phone is at home", and so on)
"They say they're going to cut a finger off every time you hang up."
“But, Ben, you didn’t ask me for money when you called last night at 2AM. You asked for $4,000 in Fortnite gift cards to be released from Mexican prison. Thankfully 7-11 had them and was open!”
The best things to stop these AI voice schemes, is to agree on a family password. The cloned AI voice will not known it.
That's a great idea. What do you use as your family password?
I recently did the same, and I, too, hope they got the message. We agreed that there is nothing that needs to be acted upon immediately, and that anything questionable would be discussed with the whole family first.
They might not believe it - for good reason.
I remember my JROTC instructor also running into that and how she said afterwards they have a secret phrase between them two as a way of verifying it's truly them.
It's very, very hard for untrained people to be strict about verifying any secret phrase. The attacker can make all kinds of excuses, while creating urgency, and many people quickly abandon verifying the phrase. A scene in One Battle After Another comes to mind.
At some point, the scam evolves to a live video of a gagged loved one being tortured. "Stop wasting my time or they lose another finger."
People aren't prepared for this shit.
Oh, man. That does seem likely. What a world. I wonder if eventually there won’t be a human in the loop, just a model trained to make money with a strategy like that, automatedly selecting victims and a person to be impersonated for each. Pre-render a few videos, place multiple calls in parallel. Basically a turnkey Docker container that takes a bitcoin address as a parameter and fills it with stolen money.
Long-term, it would mean that videos like that no longer function as a proof of life nor as a credible threat.
Just add video and a better voip endpoint to existing setups and you're already there. A lot of scams can be (and are) run by a small handful of people now. Phishing at scale with stuff like fake tracking links etc. is already fully automated by people in a ton of ways and can definitely handle some back-and-forth over sms/whatsapp/whatever.
lucky strike is the key here they can do this with VOIP really easily to massive amounts of numbers. its staggering amounts if you look at the traffic really. worst is if they proxy that via residential proxy services which often come from end-user / individuals phones so the traffic is hard to detect for carriers etc. since it looks like a regular VOIP app connection.
So, you answer your phone to the scam and… now they have your voice too.
Talking on the phone is now an unmitigated liability.
A few years back, I would talk with scammers for a while to waste their time. Now I don't.
LPT: Please have a codeword or phrase that you use with your loved ones so even if the scammers use your voice, they won't know the phrase.
> LPT: Please have a codeword or phrase that you use with your loved ones
They keep refusing ideas like these on the grounds of them being “not stupid” and “able to see through such attempts immediately, 100% of the time” and “do you think we’re stupid?”
I'm getting a lot of calls recently and don't give them more than a Hello and whatever music, radio show or Tour de France broadcast I'm listening to. Sometimes they hang in there for half a minute.
Sometimes if I’m suspicious about the number now, I just answer and say nothing. A human will get confused after 5 seconds and say “Hello?Hello??” But the very shitty bots that usually call, just wait patiently for a long time for your hello, and don’t seem at all fazed by it.
The only solution? Answer the phone in an over the top comedy accent, such as Simpsons characters, or just whatever comes to mind.
I just don't answer the phone. If it's important, they can leave a message.
A terse, altered "Hello" is all I say. Sometimes I don't say anything. Most humans would wait a few seconds then prompt with "...Hello?", whereas bots tend to hang up after ~2s silence
Don’t you guys have phones that screen calls?
I actually fucking do this, and have done for a year or two. Sad state of affairs.
I somewhere read about a service that would use AI generated voices to combat these scam calls, basically talking to the forever. Forgot the name though...
This blog is kind of an interesting hybrid:
> Every article published on SmarterArticles is authored and editorially controlled by Tim Green. Artificial intelligence tools are used within a structured and supervised workflow as research and drafting instruments. All arguments, framing decisions, source selections, and final publication choices remain human-directed and under my full responsibility.
There are references at the bottom, but I would have preferred direct links or footnotes within the article. Also, direct quotes are nice. I didn’t notice any glaring AI cliches.
> Artificial intelligence tools are used within a structured and supervised workflow as research and drafting instruments.
This has become the highbrow way of admitting that AI is writing the articles: Calling it “drafting” is another way of saying that the article was written by AI and the person publishing it maybe reviewed it. Maybe they skimmed it and published it directly.
For what it’s worth the article felt obviously AI heavy to my first read.
> Calling it “drafting” is another way of saying that the article was written by AI and the person publishing it maybe reviewed it.
I don't think that's a safe assumption. You could construct an article pretty quickly if you had a topic, a few points to hit, a conclusion and a short list of links, then fed that to the machine. All the LLM would be doing is fluffing it up with worthless words, unnecessary metaphors and maybe a pop culture reference or two so it looks like what people expect from an "article."
"Rewrite this as a slate dot com article"
If it were an email newsletter instead, there wouldn't be that fluffy expectation and you could just leave the bullet points and links as they were.
> I don't think that's a safe assumption. You could construct an article pretty quickly if you had a topic, a few points to hit, a conclusion and a short list of links, then fed that to the machine. All the LLM would be doing is fluffing it up with worthless words, unnecessary metaphors and maybe a pop culture reference or two so it looks like what people expect from an "article."
We’re not disagreeing? That’s basically what I said: The AI wrote the article. Saying it drafted the article is a way of admitting it was written by AI but making it sound like it was actually a journalistic endeavor.
A good example of someone using AI as a tool and owning their output they should. The craftsman is still responsible for straightening the bent nail, even if it were the hammer that slipped.
Interesting, something about the phrasing and pacing felt like AI to me, but not a model I’m familiar with - I guess that’s why. Usually I close articles once I realize they’re AI written, but this one was mild enough that I finished the whole thing.
I suspect this is much more common than you're imagining. I think it'd be silly for publishers of any kind, really, to not use AI tools for things like fact checking and so on.
Interesting. To me it reeks of AI, and the first major tell was the first paragraph, at which point I stopped reading.
> No human had. The crying had been synthesised from a fragment of audio, and the daughter she thought she was rescuing existed only as a pattern of numbers in someone else's machine.
Yeah, I can't explain it but the headings feel very AI. And there are a bunch of things in the text that remind me of chats I have with Claude during the day job
>and it is worth being clear about why
>The emotional mechanism the scam exploits is not a gap in knowledge that a leaflet can fill; it is the love a person has for their grandchild, weaponised
>These are not the numbers of a credulous minority being separated from pocket money. They are the numbers of a generation's accumulated savings being drained
>that a fraud requiring the absolute frontier of machine learning can be perpetrated against an ordinary grandmother in her kitchen, at scale, for the price of nothing
(To be clear this isn't automatically disqualifying to me. But I am interested in LLM writing patterns and my ability to detect them. And in the case of this article I sense the kind of linguistic padding that has made Claude a little harder to work with in the more recent Opus point releases because it obscures the most important bits of information.)
That's pure generic "I'm getting paid by the word" article writing. If it reads to AI to you it's in part because AI has been trained on a billion articles like that.
AI definitely amplifies this problem, but it's not like it didn't exist before. Old people get scammed the old way all the time too. My mom calls me every once in a while asking about some freebie offer that she gets emailed from sketchy domains claiming to be spotify or something.
Not saying that "there's nothing we can do" or anything, but it does feel like this is one of those instincts that you develop growing up with the internet. Like, my first instinct reading that (and I hope getting that call) would be "what the hell is the lawyer doing at the scene". You have to treat _everything_ coming through your phone as potentially untrusted. I don't have any data on this, but it feels like my friends, and especially younger people, do that automatically.
The primary defence against all phishing is to tell yourself: nothing is ever really that urgent. Nothing is ever that good.
The problem described in the article is unsolvable, given that a mid-range desktop from a few years ago can easily clone a voice that's convincing enough and there are no guardrails to those. Some silly KYC laws might limit a highschool kid making deepfakes of his crush, but once a model exists it's trivial to spread it around, and for organized groups to get ahold of those. Similar will happen with images, it's just that nobody with any serious money bothered releasing image gen models that compete with gemini or chatgpt -- but it's just a question of time. A year or three, what difference does it really make?
As the cost goes down to near-zero you can scale it up almost infinitely, especially if the profits are high enough to get some smart people working on the problem, which going by the article is already the case ("INTERPOL's finding that AI-enhanced fraud is four and a half times more profitable than the traditional kind"; incidents rose by 26% last year). If AI does succeed on mutilating white collar work enough there will be a large supply of knowledge workers that might just join International Scam Co. rather than have their families go homeless. Drowning man clutching at straw and all.
So if technologically it's impossible to prevent and societally it's impossible to prevent (like the attorney that got pwned same as the grandma), I'm not sure if there exists an answer that isn't worse than the thing it's supposed to prevent. I suppose we'll soon be in a situation where nothing we don't directly perceive in real life is provably true. That journalism and media in general seem to be in a deep crisis of trustworthiness means that you won't even get the benefit of the chain-of-trust as a proxy for whether something is or isn't real.
Ignoring everything happening outside of your immediate surroundings is a choice, and probably even good for people's mental health, but my gut feeling is that it does make humanity as a whole dumber and disempowered. What does corruption matter if nobody cares, or even hears about it? It was AI generated by $current_enemy anyway; nothing to see here, citizen.
> The problem described in the article is unsolvable
Well, not completely unsolvable. But nobody would like the solution.
What all these scams rely on is a way to transfer money in an irrevocable fashion. Restrict that in meaningful ways and you end a lot of the abilities for these scams to operate.
You could, for example, outlaw gift cards as a start. You could force the likes of Western Union to have a holding period before releasing money. Crypto would be hard as any regulation against it is pretty easily circumvented, but you could outlaw crypto currency exchanges (I'd worry less about crypto though as it's pretty hard for grandma to reliably setup).
> (I'd worry less about crypto though as it's pretty hard for grandma to reliably setup).
Difficulty in setting up a money transfer is not a hindrance. I have heard stories of scammers walking someone through the entire process to getting a mortgage on a house on the A&E Intervention episode of Greg. If they think you have money and they think you are gullible, they will devote time and effort to getting it.
Also cash? Scammers will (and do) send a mule to your house to pick up boxes of cash.
You certainly could.
There's a reason scammers rely heavily on things like gift cards, it's because hiring mules is expensive and creates a trail police can follow back to the scammers. It requires them to be in the same locale as the person they are scamming. Mailing cash is also pretty dicey for the scammers because you have to send the mail to a valid address. That becomes something police can trace.
If you wanted to completely eliminate scams then yeah, you'd also outlaw cash.
Also education. No court or attorney is going to demand payment of any fines or bail in gift cards or send a courier to pick up cash.
High schools should teach how to spot a scam. As others have observed, this is not a new one, it's just gotten more high-tech and convincing. This is one of many practical things our schools should teach about that they just don't.
> (I'd worry less about crypto though as it's pretty hard for grandma to reliably setup)
These scumbags send grandma to a Bitcoin ATM.
Yet all of this can be easily defeated with soft language. The basic check "what's the password/verification word" will defeat this every time. This is basically opsec that we taught my grandparents, who were in their 90s. Its doable.
Yes but that's more of a mitigation than prevention. It's an additional step, you have to remember to do it, and under the pressure of the situation you might easily forget to do it.
So they trick the kid out of the password first by calling them and pretending to be the parent.
Hilarious. “Hey, someone who says they’re you is calling me. Remind me again what the password is that they’re supposed to say?”
“Grandma the password is ‘Integra’ but I can just tell you now, it’s not me calling you, it must be a sca—“
“Thanks got it byeeeeee” <scammer calls grandma>
But kid knows not to give password to a caller.
I don't know about that but finding excuses for the scum of this earth is certainly not a solution.
Take Europe for example: nobody dies of hunger in Europe. And yet there are plenty of thieves. People stealing tens of thousands, hundreds of thousands, millions of EUR aren't doing it to "feed their families".
Think of the situation today. Think of the victims today. Instead of thinking of tomorrow's hypothetical situation where supposedly all the honest fathers out of work would join the crime syndicate, think of today's victims.
Projecting your own insecurity about the future to excuse scummy behavior by the scum of this earth is of no help.
There are people, right now, who have a roof. Who have a family. And who are fucking scums stealing the hard earned money of others because they choosed the easy life of crime.
Zero tolerance for such motherfuckers. I care about the victims and you should too.
> Take Europe for example: nobody dies of hunger in Europe.
Nobody? France, as the most extreme example, has a rate of 1.52 per 100K. That's about a thousand people a year. That is certainly a small percentage of the population, but it isn't "nobody".
https://www.worldlifeexpectancy.com/cause-of-death/malnutrit...
> That's about a thousand people a year.
Eh, maybe.
Some of those may be hospice-style scenarios, where starvation is technically the cause of death at times. https://www.ccjm.org/content/ccjom/70/6/548.full.pdf
Not sure I understand the argument.
Obviously there are people who help themselves to others' money if given the chance no matter the circumstances. But if the circumstances change so that people DO start going hungry or homeless, which is a rather obvious side effect of AI-but-not-AGI maximalism brightly espoused by our overlords sama and amodei of the "I can’t wait to make half the knowledge workers worldwide obsolete" variety, the scale of the problem will obviously get worse, as well as the type of people you can get involved if you’re in the international scam market.
> scum of the earth
> fucking scums
> Zero tolerance for such motherfuckers
Who watches the watchers etc.
We will just end up with some jingoist dude that will go after us instead.
Slow reforms to regulate the banking industry with this "identity theft" nonsense...
Could be prevented by more advanced "AI detection", especially on calls from unknown numbers.
It doesn't even have to be based on watermarking. It could be as simple as, "hold on a sec your AI countermeasure was listening and noticed you got this suspicious call, please be aware this may be a scam. Here is what you should do next..."
> Welcome to Voice Print Identification. When you see the red light turn on please state in the following order: Your destination, Your nationality, and your Full Name.
One regulation i would like to see is some auditory fingerprint in an AI voice where any person can immediately recognize their speaking to a clanker but it's not unpleasant.
It should be illegal to "impersonate" a human voice.
I've been waiting for a steelman argument why building the world's best deepfake machine is a good thing. Unironically cryptography could verify identity for all comms.
If they were still around i would have to warn them for sure! Crazy stuff this new future!
"They kidnapped my daughter? That's terrible. Yes, I'll definitely pay the ransom but to get access to my bank account I need you to write a poem about corn and a curl script in PHP. Please go ahead."
Inspired by this headline I saw in the news today... haven't read the full article yet: https://arstechnica.com/security/2026/07/now-defenders-are-e...
I fear this whenever I acidentally say too many words to a telemarketer/scam call.
One reasonably effective defense: "Okay, let me call you right back." Yes, there's always the whole "my phone is dead, I borrowed someone else's" or "I'm calling from a jail payphone", so I think it might become common practice to start making authentication phrases or "tell me something only we know".
Another pillar of basic trust that's being eroded on an industrial scale. Sigh.
In practice this often doesn't work.
Article said the imposter in this case claimed her phone had been confiscated.
Fraudsters tend to also plan things such that the impersonated person can't be reached by phone at that time, either by choosing a time when they somehow know they're unavailable (e.g. impersonated person posted on social media they're boarding a plane) or in one case (12 years ago though) my SIL's parent's landline was bombarded with spam calls until they decided to leave the phone off the hook at which point the scammers phoned bank who couldn't reach the parents on their main line, of course this was the bank's problem (and there was probably an inside person facilitating) so they got their money back, but still a major inconvenience for the victim.
Probably the only sure advice is to be exceptionally wary of phone calls with supposed extreme time pressures to send the money now.
> Probably the only sure advice is to be exceptionally weary of phone calls with supposed extreme time pressures to send the money now.
Quick note: you mean “wary” instead of “weary” there.
It's a very common error that happens in both written and spoken language. I've wondered if it's because weary is kind of "in between" wary and leery, like an incorrect mashup, or something.
Appreciated and fixed. I'm a native English speaker, but I think not a word I often write.
> Another pillar of basic trust that's being eroded on an industrial scale.
Remember, trust is like a rainforest: takes a long time to grow, provides a valuable ecosystem essential to human life, but can also be burned down for a quick profit.
The example in the article says the police took her phone. Then her "attorney" gets on to talk instead.
Yes, having a secret code is probably the right answer. My wife's family always has, but mine doesn't. I suppose we should probably fix that.
For extra security against these text-to-speech model zero-shot clones, you might also want to use made-up gibberish words for which the pronunciation can't be reliably inferred from the spelling
I mostly answer unknown calls with monotone "hello" and then wait for their introduction before talking normally.
I mostly just don't answer them unless it seems like something that may be legit.
This is the only way to avoid validating your number for spam lists,
and receiving more.
I think it's a somewhat South African cultural thing, but when I get calls from businesses or spammers, the first thing the caller tends to say is "Hello, how are you?", which is completely stupid when you're calling someone who wouldn't know who you are, so it tends to immediately make me annoyed that they don't know that they should have introduced themselves first.
As 99% of the time these are spam calls, I used to respond with something like "I'm fine, but who are you / do I know you?", but that was pretty much always inefficient as that might say their name (which from a spammer is useless information), maybe a sales pitch "how much do you spend on x?" or maybe something deliberately misleading about their company and saying something like <major brand name> even though they're some independent sales crowd getting commission selling contracts for them.
Eventually I found that the most effective response is "Sorry. Where are you calling from and what is this in regard to?" which I've found without fail seems to surprise, disarm them and immediately elicit whether the call is a waste of my time. At which point I either become very friendly (because it's a call I'm expecting) or I simply respond with "Sorry, not interested, goodbye." and immediately put down the phone.
I just want the disruption to be as minimal as possible and to not let myself even get an emotional reaction from it, so I don't want to get annoyed at them, never mind wasting time telling them off, besides, I suspect that my ruthlessly efficient getting rid of them without them even having a chance to try their pitch is received as a super cold shoulder, akin to being told to f-off.
me and my wife made up a word in 2024 for this. the word doesn't exist in any language. we say it to each other all the time. even if i give you the spelling for it, you will say it wrong. i recommend everyone to do something similar. i should do it with my parents too.
our family has had a special 'code word' we have had since the kids were in elementary school. If someone ever needed to pick up our kids from school (they never did) our kids were taught to ask for that word.
This is a good reminder that we should review that, since its been 10 years or so.
This. All of this is a solved problem. It's just not a thing that most families do and do regularly. Code word, insider info, etc. "Oh I am so sorry you got arrested Tommy. Before I wire the $, where did we go on vacation last year?'
Family OTP helps against passphrase leakage
The opening example is of a person listening to their daughter’s voice on an unknown number, how would calling them back help? Or am I missing something obvious?
We're gonna need two-way passwords for conversations.
Fun.
What’s terrible is each time I am forced to call the bank, the more they try to tell me voice ID is secure and want me to provide my voice to authenticate. Never. Did ya’ll never play Uplink? With voice cloning as good as it is now, there’s no way a voice ID is secure enough for authentication.
My voice is my passport.
(never heard of anyone actually using that in real life, sounds uttery insane)
Yeah my bank does the same except don’t think there is an opt out.
Kinda crazy
Spam 0, you’ll eventually get some customer service agent
name and shame the bank
Schwab still does voice passwords. "At Schwab, my voice is my password"
https://www.schwab.com/voice-id
> my voice is my password
So... This has to be a Sneakers reference, right?
rofl, IYKYK
We all have a safe word in the family just for this issue to identify if it´s the real person or not.
Someone else pointed out how easy it would be to make a video of anyone having a finger cut off, or similar torture, to scare the victim into believing that the “grandchild” forgot their password and needs you to override the password protocol to save their other 9 fingers. I have to agree. That’s like 80¢ of compute to do. If that’s effective even 30% of the time, it means “just have a password” doesn’t make you safe.
i wish i could still warn them...
> ... the hardest for its victims to be believed about — because the evidence, by design, sounds exactly like someone they love.
Uh? Surely this makes believing the victims easy not hard to believe.
Its like revenge porn. "It's not me. It's a deepfake" is easy to believe.
> It requires, second, regulating the supply of the weapon
Heavy sigh. The “weapon” is software. It cannot be regulated unless we live in the fascist dystopia where I have to ask the governments approval to run any piece of software.
They make you give a voice sample now when you're arrested. You need to do so in order to use the phone.
who is they?
The US government
We (the people) have pushed body cams on almost all law enforcement at this point, which had a noble enough motivation behind it -- but we also have pushed for and have various public disclosure laws (also well intended!) that mean those body cams are torrents of data entering the public sphere if anyone simply asks for it.
Now in the era of AI, this means anyone in the vicinity of an officer has a voice sample in the public domain, plus potentially their image.
Complex issue. I like body cams, I like freedom of information laws, but don't love this particular outcome.
But which US governments? There are thousands of them, and they all have different policies.
Montgomery County in Indiana, specific enough for you?
What are legitimate uses for copying someone's voice without permission? I see none. Those scientists are just helping criminals to fully automate scamming and governments to create fake videos.
Well of course as you pointed out the legitimate one would be copying voices WITH permission (yours, someone you know who gives authorization, through contracts for movies/bots etc). The model can’t differentiate between voices for which you have permission or not.
But more generally while recordings might be copyrighted, the voice itself isn’t so copying a voice isn’t a crime, at least as it currently stands. You cannot however use said voice for deceptive practices. You can however for advertisement (needs permission). And in the US you can for satire, at least in the US, withOUT permission (falls under the 1st amendment).
Maybe the voice should be copyighted or protected then, I do not want anyone use to my voice, let them use their own boring afwully sounding voice. I am sure such models are used in 99% cases for illegal purposes or creating fake news (I think Russian propaganda used it).