Recently, deepfakes made headlines again, particularly with Grok’s low-guardrail image generator and the potential risks these tools pose to elections globally. However, this threat goes beyond politics- deepfakes are rapidly becoming a major concern for enterprises.
Deepfakes represent an advanced form of social engineering that, while still rare in real-world scenarios, is growing in sophistication and availability. Even organizations with strong security frameworks should reassess their threat models to account for this emerging risk. As deepfake technology becomes increasingly cost-effective, it’s no longer a matter of if, but when these attacks will become more prevalent.
A Growing Concern
For financial institutions, fraud is always a top priority, and deepfakes are rapidly gaining attention. Many banks rely on voice-based authentication, where customers verify their identity by repeating a phrase, such as “At [bank name], my voice is my password.” Selfie images are also commonly used for identity verification in financial transactions.
While deepfake fraud is still relatively rare, it’s a significant concern. The approach to fraud differs from security in that fraud prevention focuses on smaller, everyday attacks, while security aims to prevent large-scale, black swan events. As voice and image-based authentication methods become less reliable, financial institutions are bracing for an increase in deepfake fraud, and many are preparing for the inevitable. Fraud leaders in the industry believe that a more robust authentication system is needed – one that can stand up to the deepfake threat, much like PKI did for the web 25 years ago.
Leveraging Deepfakes for Access
Combining deepfakes with traditional social engineering tactics makes for highly sophisticated attacks. Many access protocols, including wire transfers and sensitive enterprise access, require phone or video calls for verification. As deepfake technology improves, these verification methods are becoming increasingly vulnerable.
Consider the case of cybersecurity firm KnowBe4, which unknowingly hired a North Korean spy using a deepfake and a stolen identity. After being hired, the spy installed malware on the company’s devices, which were then remotely controlled from North Korea. Similarly, Ferrari executives were targeted with a deepfake phone call impersonating the CEO. The executive recognized something was wrong, asked a specific question, and the call ended abruptly. An investigation revealed the deepfake attempt. These cases highlight how challenging deepfake detection can be, especially as the technology becomes more sophisticated.
Brand Devaluation and Loss of Trust
In addition to the financial damage caused by deepfake scams, companies are at risk of brand devaluation. Fake videos impersonating CEOs or executives have already been used to spread scams and manipulate stock prices. While the direct financial damage may fall on consumers, the brand damage can be far-reaching, causing long-term harm to an organization’s reputation.
On the media side, deepfakes present a serious threat to journalistic credibility. If audiences no longer trust the content produced by a media outlet, they may turn elsewhere for news. For companies in the media space, this loss of faith in reporting can lead to a dramatic loss of audience and trust.
Approaches to Detecting and Mitigating Deepfakes
To combat the growing threat of deepfakes, several startups are focusing on detection technologies. Reality Defender, for example, offers a suite of tools that identify AI-generated content, including images, videos, and audio. Their solutions rely on various detection methods, such as analyzing eye movement inconsistencies or examining the frequency domain of images.
Many financial institutions have already integrated these tools into their systems, running real-time detection on voice calls between private wealth management teams and clients. Media companies are also adopting these technologies to investigate potentially manipulated content. This emerging field, known as semantic forensics, is attracting attention from government agencies and organizations like DARPA.
Provenance and Watermarking: Solutions for Authenticity
On the technology front, several cutting-edge solutions are being developed to combat the rising threat of deepfakes, with a particular focus on provenance tracking and watermarking. Provenance tracking is one of the most promising methods to ensure the authenticity of digital content. A key player in this effort is the Coalition for Content Provenance and Authenticity (C2PA), a consortium backed by major companies including Adobe, Google, Microsoft, and OpenAI. The C2PA aims to establish universal standards for verifying the origin of digital content. By embedding embedded metadata directly into images, videos, and other forms of content, the C2PA provides a way to track both the origin and history of digital media. This metadata can include key details such as the creator’s information, location, time, and changes made to the content, which makes it possible to trace the digital media back to its original creation. In turn, this can be incredibly useful in verifying content’s authenticity, preventing manipulation, and supporting users in identifying altered or synthetic material.
Watermarking is another key technique being used in the battle against deepfakes. This method involves embedding a unique identifier directly into the media itself, which cannot be easily removed or altered. Both DeepMind’s SynthID and OpenAI’s DALL·E 3 are implementing watermarking in their image-generation models. These watermarking tools ensure that the content generated by these platforms has an identifiable trace, marking it as AI-generated. This serves as a built-in signifier that the content has been synthesized, helping users and systems distinguish between authentic and artificially generated images. Watermarks can also be imperceptible to the human eye, allowing them to act as a security feature while keeping the media visually unchanged.
However, while provenance tracking and watermarking techniques offer significant improvements in identifying and verifying content, they are not without limitations. One key issue is that bespoke deepfakes—those generated using custom tools or lesser-known AI systems—can easily bypass these detection methods. These custom deepfakes may not carry the metadata associated with C2PA’s standards, nor can they be identified by traditional watermarking. Such content is typically harder to trace, making it more challenging for detection systems to flag these alterations as fake.
That said, provenance and watermarking still offer crucial advantages in the fight against deepfakes. For one, they help reduce the spread of “cheapfakes”—real images and videos that have been taken out of context or manipulated to deceive. With the rapid advancement of generative AI technologies, the ability to trace and authenticate content is becoming more critical, especially in an age where visual manipulation is increasingly difficult to detect by the human eye. Provenance tracking and watermarking provide a valuable line of defense, offering transparent methods of verifying the authenticity of digital media before it is disseminated to the public. They also allow content creators, platforms, and even consumers to verify whether an image or video has been generated by AI, helping to curb misinformation and ensure greater media accountability.
Industry experts are also exploring other strategies, such as semantic forensics, which combines detection algorithms with attribution technologies to trace the origin of AI-generated content and detect malicious intent. As AI-generated content becomes increasingly sophisticated, these detection and validation tools will play an essential role in protecting digital integrity and preventing the misuse of deepfakes in various contexts, including politics, security, and media. As deepfake technology continues to evolve, solutions like provenance tracking and watermarking will need to evolve as well to keep pace with new methods of attack, ensuring a robust defense against this emerging threat.
In addition to these solutions, there is growing interest in Nightshade, a new technique that focuses on identifying and countering deepfake content at the pixel level. Nightshade operates by embedding artificial noise into the visual content that makes it easier for deepfake detection systems to recognize manipulated media. This approach can be used by media creators and tech platforms to verify the authenticity of content before it is shared publicly.
Hygiene, Education, and the Path Forward
While new technical solutions are essential, cybersecurity hygiene and education remain foundational. Many financial institutions are turning to behavioral analysis to catch fraud early, using statistical matching to detect signs of deepfake-related fraud. Establishing strong security protocols, including multi-factor authentication (MFA) and in-person verification for high-risk transactions, will also help reduce the impact of deepfakes.
As the technology behind deepfakes continues to evolve, it’s crucial for organizations to stay ahead of the curve. Founders building solutions in this space must anticipate the rapid advancements in AI and security research to ensure their products remain effective against emerging threats. Furthermore, careful consideration of market timing, product design, and partnerships will be critical for success as deepfake threats continue to grow.
How Can Netizen Help?
Netizen ensures that security gets built-in and not bolted-on. Providing advanced solutions to protect critical IT infrastructure such as the popular “CISO-as-a-Service” wherein companies can leverage the expertise of executive-level cybersecurity professionals without having to bear the cost of employing them full time.
We also offer compliance support, vulnerability assessments, penetration testing, and more security-related services for businesses of any size and type.
Additionally, Netizen offers an automated and affordable assessment tool that continuously scans systems, websites, applications, and networks to uncover issues. Vulnerability data is then securely analyzed and presented through an easy-to-interpret dashboard to yield actionable risk and compliance information for audiences ranging from IT professionals to executive managers.
Netizen is an ISO 27001:2013 (Information Security Management), ISO 9001:2015, and CMMI V 2.0 Level 3 certified company. We are a proud Service-Disabled Veteran-Owned Small Business that is recognized by the U.S. Department of Labor for hiring and retention of military veterans.
Questions or concerns? Feel free to reach out to us any time –
https://www.netizen.net/contact
