Digital Doppelgängers: The Promise and Peril of Deepfake Tech

Exploring how Deepfakes Fuel a More Immersive and Personalized Virtual Experience

Hello!

Welcome to Trend Hacker’s newest journey! We’re diving into the exciting world of “synthetic media,” spotlighting the rise of deepfakes and voice cloning. In this easy-to-understand guide, we’ll map out the birth and growth of these potent technologies, explore their potential to change various sectors and talk about the challenges they might bring to our society in the following years.

---

Trend Summary

Breaking Down the Trend The Setting: Trend Hacker’s latest edition offers an in-depth look at “synthetic media,” focusing on the role of deepfakes and voice cloning technologies.

The Trend: Deep learning technology, deepfakes, and voice cloning can change voices and create real videos. As these technologies continue to evolve, they raise significant concerns, especially when it comes to misuse that could threaten security, privacy, and the integrity of information. On the bright side, they offer valuable tools in many sectors, from advertising to improving sales support and customer experience.

Impact on Products and Services: Synthetic media has the potential to bring down costs in film production, marketing, and advertising. But it’s not all rosy – deepfakes also threaten voice and facial recognition systems, leading to substantial financial losses due to fraud. Despite these challenges, the rise of deepfake technology is opening up new opportunities for immersive and personalized shopping experiences and enhancing sales and service support.

What Does it Mean for Society? Deepfake technology can significantly shake trust and cause political unrest and financial turbulence.

How Are We Responding? The severe implications of deepfake technology call for regulatory frameworks, corporate responsibility, and public education.

---

So, What Exactly is Synthetic Media?

In a world where technology continuously blurs the lines between reality and artificiality, synthetic media, underpinned by artificial intelligence, emerges as a groundbreaking innovation. This expansive realm includes deepfakes, voice cloning, text generation, virtual reality, augmented reality, Generative Adversarial Networks (GANs), neural style transfer, and image synthesis. Our trend report will focus on two game-changers: deepfakes and voice cloning.

Anders Fogh Rasmussen, Former Secretary General of NATO, who maintains a keen interest in geopolitics, flags an exciting thought. What role will deepfakes play in the future landscape of information warfare? He paints a picture of our digital future where, by 2030, a staggering 90% of all online video content could be synthetic.

In the realm of marketing, synthetic content is making its mark. According to Gartner, by 2025, synthetic media will generate an impressive 30% of all outbound marketing messages from large organizations, a significant leap from less than 2% in 2022. They estimate other AI-powered films taking center stage by 2030, with 90% of the content generated by AI, an astounding evolution from 0% in 2022.

As we stride into this fascinating realm, it’s essential to keep our focus on harnessing its potential responsibly and ethically. This remarkable innovation could redefine how we communicate, market, and entertain, but are we prepared to navigate the potential pitfalls?

Decoding the Trend

Yet the desire to clone is not new. Leonhard Euler, a renowned 18th-century scientist, made significant strides toward understanding sound waves and their role in voice production. In 1773, Euler speculated on creating a mechanical instrument that could mimic human speech by combining similar sounds to form intelligible words. His focus on the unique tonal characteristics of different letters, especially vowels, suggests an early inclination towards what we now know as voice cloning. These advancements marked a potential beginning for the scientific exploration of voice replication. (Wikipedia).

Deepfakes grabbed the public’s attention around 2017. The term “deepfake” – a blend of “deep learning” and “fake” – refers to synthetic videos created using AI techniques. While early examples usually swapped faces in videos, the technology has now evolved to fabricate new content.

Deepfakes use deep learning technology to substitute existing images and videos with source images or videos. Voice cloning follows a similar approach, mimicking an individual’s unique voice patterns, intonations, and speaking style. Combined, these technologies can fabricate personas that sound and look like real people.

Current data suggests a higher demand than supply for deepfakes in darker digital corners, with costs fluctuating between $300 to $20,000 per minute, depending on the subject’s popularity (Kaspersky, 2023). Currently, a paradox exists - the more famous and publicly visible an individual, the cheaper and higher quality their deepfake becomes. However, this dynamic is shifting. SCIP’s research (2023) indicates that we can expect reasonably convincing results with as few as 500 images. As we stand on the brink of this transformative era, How will this shape our digital future?

Implications for Your Product and Business

The rise of synthetic media, particularly deepfakes and voice cloning, presents a mixed bag of opportunities and challenges for businesses across sectors. These implications are broad, and I highlight only selected areas of high impact in this report.

Media, Marketing, and Advertising:

• Deepfake technology can significantly cut costs in film production, boost marketing and advertising efforts, and escalate overall customer engagement. It’s especially effective in crafting hyper-personalized content and making messages feel more personal and less spam-like.

• McKinsey’s 2023 estimates suggest an intriguing shift towards generative AI, with around 75% of the potential value delivered across customer operations, marketing and sales, software engineering, and R&D, with an impressive trillion-dollar impact on marketing and sales alone.

• The creative campaign ‘NotJustACadburyAd’ leveraged AI and machine learning to deepfake Shah Rukh Khan’s face and voice for free personalized advertisements for local store owners. Thanks to Cadbury’s innovative partnership with Rephrase.ai, store owners could generate a unique advertisement where Khan appears to promote their store.

• Deepfake has powered impactful campaigns like Glance’s Har Pal Happening, which created deepfake avatars of historical characters such as Akbar-Birbal and Laila-Majnu. This campaign garnered a staggering engagement with 3 million organic shares and 4 million clicks, reaching over 31 million people. (APN News)

• Other brands, like Lays, have jumped on the bandwagon with their ‘Messi Messages’ initiative, which allowed fans to receive personalized messages from a deepfake Messi, further personalizing the customer experience.

• Pepsi tapped into this innovative technology for a campaign featuring Salman Khan in 2022. They creatively juxtaposed a deepfaked young Salman with his current self to promote their product, showcasing how some things stay consistent over time.

• The key benefits of these deepfake campaigns include significant cost savings, ease of entry into foreign markets due to omnichannel content capabilities, and the delivery of hyper-personalized content to audiences. As Baid notes, these custom messages become an integral part of the customer journey due to their personal touch, distinguishing them from generic promotional content.

New Sales (Support) and Customer Experience

• Deepfake technology transforms the shopping experience, making it more immersive and personalized. Japanese AI Firm Leading the Way: DataGrid has been using deepfake technology since 2019 to create virtual models for fashion advertising, setting the trend for other companies to follow. Brands like Levi’s to Calvin Klein use “Fashion AI” to create custom AI models for a more diverse representation (The Guardian). The first AI Fashion Week in NYC in April 2023 indicates a long-term industry impact.

• Enhancing Sales and Service Support: Deepfake technology creates interactive digital assistants that can answer queries, provide product demonstrations, and guide customers through the purchasing process, improving customer service efficiency.

• UneeQ specializes in creating “digital humans” for revenue teams and business development. These digital human sales assistants can engage in real-time, one-on-one conversations, educate customers, build trust, and guide them toward the right conversion points. Similarly, UneeQ’s digital humans are revolutionizing contact centers by providing automated, engaging service at scale in over 70 languages. They are available 24/7, 365 days a year, on virtually any channel. (UneeQ)

• The first estimated market size for a digital human market in China in 2030 is 13 billion dollars. (Statista, Lai Lin Thomala, 2023)

Security and Authentication

• Deepfake technology poses a significant threat to authentication systems, especially those relying on voice and facial recognition. This threat spans various sectors, including mobile devices, banking, home security, automotive, and social media.

• A study by The Verge in May 2022 revealed a worrying vulnerability in identity verification systems. The research found that 90% of the tested ID verification systems, which used “liveness tests,” could be easily deceived by deepfakes.

• The potential for financial loss due to deepfake impersonation is substantial. For instance, a UK-based energy firm’s CEO was tricked into transferring €220,000 to a fraudulent account, thinking he was obeying his superior’s orders.

• In another disturbing case, a scammer tricked a branch manager of a Japanese company in Hong Kong into authorizing transfers totaling $35 million. The manager believed he was coordinating a legitimate acquisition with the director of his parent business. (Forbes, 2021)

• In China, a fraudster fooled a man into transferring 4.3 million yuan ($622,000) to a supposed friend, using advanced “deepfake” technology. The criminal used AI-powered face-swapping technology to impersonate the victim’s friend during a video call. The victim believed his friend needed the money to deposit in a bidding process. (Reuters, 2023)

• According to a report by Sumsub, the prevalence of deepfake-related frauds in the US doubled from 2022 to 2023, accounting for 3% of all cases in Q1 2023.

• A survey conducted by Regula Forensics across multiple countries, including Australia, France, Germany, Mexico, Turkey, the UAE, the UK, and the USA, found that 37% of organizations had experienced deepfake voice fraud, and 29% had fallen victim to deepfake videos in 2022/2023.

Mirroring Me (Digital Twin)

• Digital-Me Deepfake technology has a high potential to revolutionize cyber-physical systems (games, virtual self-representations, Metaverse). The gaming industry will likely spearhead this shift, using deepfake algorithms to create digital twins that mimic user facial expressions, voices, and full-body movements for a more immersive and personalized gaming experience. Although currently used in the film industry, personal computer processing power limits a complete mirroring of individuals. We might see the first steps in voice cloning and facial expressions in the following years.

• The potential of this technology extends beyond gaming; it’s also crucial in building the Metaverse, a virtual shared space created by the convergence of physical and virtual reality. However, achieving this requires substantial computational power, with experts suggesting a 1,000x increase in collective computing capacity to realize this vision fully. All the parts of the Metaverse will come into place, most likely in the next decade, and until 2033, we might see the virtual self strongly present in our realities.

• Despite the thrilling prospects of incorporating digital twins in the Metaverse and beyond, significant risks emerge, such as impersonations, reputation manipulation, and identity theft, all due to deepfake technology. Developers and gaming companies must focus on privacy and innovate mechanisms to protect players’ identities. These considerations become crucial as we explore the potentials and pitfalls of deepfake technology in creating interactive digital twins within cyber-physical systems and the Metaverse. Even mere voice cloning in games and adding “others” to oneself could lead to complex legal predicaments.

• An extreme scenario of this reality shows in the last season of Black Mirror in the episode “Joan Is Awful,” where Deepfake, Quantum computing, and privacy come together, creating a living nightmare scenario of commercial exploitation.

Deepfake Resurrection

• Deepfake technology, employing artificial intelligence to generate lifelike representations of deceased individuals, is a frontier being explored in 2023 by researchers Lu & Chu. Their work investigates public responses to public service announcements (PSAs) that utilize deepfake resurrections of drunk driving or domestic violence victims.

• The research reveals a common sentiment of discomfort and rejection toward deepfake resurrections. Similarly, Michele K. Lewis, a Professor of Psychology at Winston-Salem State University, discovered that students expressed considerable unease about using deepfake images of the deceased, even in critical social messages against drunk driving and domestic violence.

• Drawing from their research, Lu and Chu propose that our responses to deepfake resurrections depend on our cultural values. The receptivity towards this concept may be more apparent in technologically progressive cultures (Japan, South Korea, China), cultures with ancestor veneration (China, Vietnam, African cultures), secular cultures (Denmark, Sweden), and Western pop cultures (USA, UK).

• This emerging frontier has already attracted the interest of some startups like HearAfter.AI. They’ve partnered with Amazon to “revive” the voices of grandparents narrating stories on Alexa—a compelling testament to the potential of this technology. We find a similar application in the Japanese toy Coemo, synthesizing parents’ voices for bedtime tales.

• This subject isn’t foreign to the world of science fiction, either. For instance, in the second season of the British series Black Mirror, “Be Right Back” explores a scenario where a deceased individual is revived as an AI and subsequently as a robotic avatar.

• In practical terms, deepfake technology has found a place in the music industry, where Tupac, as seen in Snoop Dogg’s work, got resurrected digitally. Cultural institutions are also engaging with this technology, with The Dalí Museum in St. Petersburg, Florida, using it to create an interactive Salvador Dalí exhibit in 2019. The company Uneeq, specializing in “digital humans,” made an Albert Einstein chatbot that fielded 350,000 questions in just two weeks, with 68% of visitors returning for further interactions.

What might be the implications of this trend for society?

Deepfake technology, as our society will soon realize, carries broad implications. It holds the potential to dissolve the trust, provoke discord through counterfeit speeches, generate turbulence in financial markets, and give rise to identity theft. These possible risks emphasize the absolute necessity of government regulations, the duty of organizations to take on corporate responsibilities, and the importance of public education about deepfakes and their possible influences.

• Trust: Deepfakes possess a formidable capability to disintegrate trust by propagating misinformation and producing suspect content. They could be wielded for political subversion, bullying, revenge pornography, and blackmail, inflicting extensive damage to personal reputations. Deepfakes, unlike text, can foster a more profound emotional bond, potentially distorting our memories or embedding spurious ones. An example is that exposure to a deepfake of a political personality could detrimentally color people’s perception of that individual. Research by Hancock et al. (2021) disclosed that exposure to deepfakes augmented uncertainty about media and shrank overall trust in news. Thus, the misuse of deepfakes can prompt far-reaching societal implications, initiating a general decline in trust.

• Scamming, Fraud, and Manipulation: Deepfakes pose an escalated threat of scamming, fraud, and manipulation, inciting considerable emotional turmoil and potential financial damage. This technology can have adverse emotional and economic impacts on individuals, from identity theft to fraudulent use of cloned voices. In commercial and service domains, deepfakes clear a path for new levels of subtle manipulation, such as real-time filters used to modify emotions. For instance, Hancock et al. (2021) discovered that artificially enhancing the amount of smiling in conversational pairs resulted in participants feeling more positive after the interaction and using more affirmative language. These issues highlight the need for corporate responsibility measures for deepfakes, which must extend beyond existing regulatory frameworks (Hancock et al., 2021).

• Regulations: As the evolution of deepfake technology quickens, the demand for inventive legislation and regulation, particularly around voice cloning and deepfakes, becomes increasingly critical. Governments across the globe have already initiated measures to curb the misuse of this technology. For instance, laws in China and South Korea mandate explicit disclosure and consent for deepfake use. Canada emphasizes prevention, detection, and response to deepfakes, while the European Union obliges social media platforms to eliminate misleading information. The United Kingdom reserves funding for research into strategies for detecting and countering deepfakes. In the United States, some states have laws explicitly targeting deepfake pornography. However, enforcing these regulations confronts hurdles due to the anonymous nature of online platforms and potential encroachments on free speech. Therefore, as deepfake technology evolves, regulatory frameworks must balance forestalling misuse and safeguarding free speech and innovation (Responsible AI).

Trend Conclusion

As we steer into the exciting future of synthetic media, we’re witnessing an unprecedented rise in deepfakes and voice cloning technologies. These technologies open up opportunities, from entertainment spectacles to transformative marketing strategies. But as we embrace these advancements, we must also confront the ethical and security dilemmas they present, such as the potential for trust erosion and identity theft. As we venture into this new age of digital communication, the question remains: how can we balance the boundless potential of these technologies with the need for safeguards?

---

Word of Notice

The impact analysis presented here strives to be pragmatic, considering challenges from operational development to social acceptance. However, given the inherent nature of such explorations, it’s speculative.

Lastly, while the analysis explores how this trend can influence the “fabric of our lived reality,” it does so by pre-selecting some contributors from a large pool. Therefore, consider this analysis a thought-provoking, speculative, selected snapshot of what may lie ahead. Applying the trend to your business might require further contextualization and deep diving into the logic and product.