Hip-hop artist Jay-Z has long been a source of inspiration for other musicians. However, in a slightly unexpected turn of events, he has recently provided the inspiration for an anonymous YouTube creator who uses AI to generate incredibly realistic Jay-Z audio impersonations.
Videos uploaded to the Vocal Synthesis YouTube channel featuring Jay-Z have included a video of Shakespeare’s “To be, or not to be” soliloquy from Hamlet and a snippet from the Book of Genesis spoken in a synthesised voice mimicking Jay-Z’s vocal tone and flow. These videos are labelled as speech synthesis and say in their descriptions: “The voice in this video was entirely computer-generated using a text-to-speech model trained on the speech patterns of Jay-Z.”
The videos are created using Google’s open-source Tacotron 2 text-to-speech model, which allows for the synthesis of extremely natural-sounding speech from text.
The Vocal Synthesis channel is packed with videos featuring deepfake audio mimicking other musicians, politicians, authors, artists, and actors, often reading content which sounds absurd in their voice, such as the lyrics to Eurodance songs or political statements which are incongruous with the impersonated figure. The audio for some of the videos has been taken and remixed by other creators.
Deepfakes are a type of synthetic media generated by deep-learning algorithms (often generative adversarial networks). Pornographic deepfake videos – in which a person’s likeness is inserted into the place of a performer in a pornographic video – are the most widely shared deepfakes, although similar tools can be used to generate synthetic images, audio and non-pornographic videos.
Deepfakes are overwhelmingly used to insert the likeness of female celebrities into pornographic videos, although a small fraction of deepfake videos created and shared online are for other purposes, such as parody, manipulation, art, or entertainment.
Despite Jay-Z being arguably among the less-maligned victims of deepfakes, his entertainment agency Roc Nation LLC has filed copyright strikes against the YouTube videos in which a synthesised voice mimics the performer’s. According to the channel creator, the notice stated: “This content unlawfully uses an AI to impersonate our client’s voice.”
The copyright claim raises some interesting questions, such as whether it can legitimately be argued that work is infringed upon in videos like these, given that deepfake audios merely impersonate a person’s voice. Many of the videos on this particular channel use text that is out of copyright. If claims like these are upheld – which seems unlikely in this particular case – it could lead to further attempts to use copyright claims to take down deepfake videos, which exist in a grey legal territory. Deepfakes are not explicitly forbidden by law, but they frequently violate legislation such as copyright and revenge pornography laws.
The channel creator posted an update in the form of a notice read by synthesised voices mimicking US Presidents Barack Obama and Donald Trump. In this video, they point out that every video is clearly labelled as “speech synthesis” and intended for entertainment with “no malicious purpose for any of them”.
“Obviously, Donald and I are both disappointed that Jay-Z and Roc Nation have decided to bully a small YouTuber in this way,” the video says, mimicking Obama. “It’s also disappointing that YouTube would choose once again to stifle creativity by reflexively siding with powerful companies over small content creators.
“Specifically, it’s a little ironic that YouTube would accept AI impersonation as a reason for a copyright strike, when Google itself has successfully argued in the case of Authors Guild v Google that machine-learning models trained on copyrighted material should be protected under fair use.”
Although some of the Jay-Z deepfake videos were immediately removed from YouTube, they have since reappeared. A Google spokesperson told technology news website The Verge that YouTube had reviewed the requests and determined that they were incomplete: “Pending additional information from the claimant, we have temporarily reinstated the videos.”