Imagine your smart home speaker suddenly starts talking in the voice of your deceased relative. Would you find it sweet, or does it sound downright creepy—straight out of a horror film?
No matter what you think about it, Amazon has already begun teaching Alexa to mimic anyone’s voice, dead or alive, from just a one-minute recording.
Rohit Prasad, Amazon’s head scientist for Alexa, said at a live event on Wednesday, July 22 that his team has been instructing Alexa to pick up a voice from a short audio clip and convert it into a longer audio output. The feature was presented at the Amazon re:MARS conference in Las Vegas.
Prasad introduced that while AI cannot eliminate the pain of losing loved ones, “It can definitely make their memories last.” He then played a short video of how people could use Alexa’s voice-changing capability in real life.
The clip showed a child asking Alexa—”Can Grandma finish reading me The Wizard of Oz?” After she says, “okay,” Alexa goes on to speak in a robotic tone in the voice of the child’s grandmother.
Prasad highlighted that the addition of “human attributes” to AI systems was increasingly important “in these times of the ongoing pandemic when so many of us have lost someone we love.” You can watch the full demo of Alexa’s new voice-imitation feature below:
Although Amazon did not say when this feature would be available to public, the specific program is already controversial for many.
AI voice mimicry is not entirely new and has become increasingly common in recent years. These imitations are often known as “audio deep fakes” and are regularly used in industries like podcasting, film and TV, and video games.
That is why rolling out this feature on easy-access devices like Alexa has raised concerns about its potential for abuse and impersonation.
Alexa being able to mimic people’s voices sounds like an obvious recipe for disaster on both a smaller and larger scale. It has vocal deep fake written all over it. Imagine someone impersonating you?? Impersonating public figures?
— Kierra | Data Conversationalist🎙 (@iamKierraD) June 23, 2022
Umm, so how soon will criminals be able to use it to call your family members begging them to Venmo cash? Or ask them for social security numbers? Or bank information?
— 🌈bitty_in_pink 💫 (@bitty_in_pink) June 22, 2022
Meanwhile, others were quick to relate this instance to a Black Mirror episode, a Netflix series about the dark and twisted side of modern technology.
Alexa will be able to mimic people’s voices. But experts say the new feature is a Black Mirror-esque “slippery slope” that raises security and ethical issues.
For ex: who has the rights to the data people leave to the ethers of the World Wide Web? https://t.co/yd220u48xj
— María Luisa Paúl (@marialuisapaulr) June 23, 2022
Amazon’s Prasad said the feature could enable consumers to have “lasting personal relationships” with the deceased. However, not all people would like the idea of their departed loved ones becoming a part of a digital AI.
Source: https://youtu.be/22cb24-sGhg
