Intelligence notes

◉Dashboard↗

Deepfakes and Synthetic Media

16 min read

Deepfakes and Synthetic Media

BLUF

Deepfakes and synthetic media — AI-generated or AI-manipulated video, audio, and image content — have become operational instruments of state and non-state information operations. While the “liar’s dividend” (the epistemic corrosion caused by knowing that authentic content can be fabricated) may ultimately prove more strategically significant than any specific deepfake deployment, documented combat-context use in Ukraine (the “Zelensky surrenders” video, March 2022) and the proliferation of AI-generated imagery across information operations from Taiwan to Gaza demonstrate that synthetic media has graduated from a theoretical threat to an active IO instrument.

The accelerating cost decline of generation tools — from specialized GAN research (2017) to consumer-grade applications (2023–2026) — has democratized production capability to the point where attribution of specific content to state operations is increasingly difficult. The arms race dynamic is structurally asymmetric: generation capabilities improve faster than detection methods can retrain, and the marginal cost of producing a synthetic video now approaches zero while the cost of rebuttal — requiring rapid forensic analysis, platform coordination, and public communication — remains high and time-intensive. This asymmetry is not incidental; it is the operational logic that makes synthetic media strategically attractive.

Technical Generation Methods

GAN Foundations

The conceptual and technical foundation of deepfake video rests on Generative Adversarial Networks (GANs), introduced by Ian Goodfellow et al. in 2014. The GAN architecture pits two neural networks against each other — a generator producing synthetic content and a discriminator evaluating authenticity — in an adversarial training loop that over time produces increasingly photorealistic outputs. The term “deepfake” itself emerged from a Reddit community in late 2017 that applied GAN-based face-swap techniques to non-consensual pornography; the underlying method rapidly migrated to political and IO applications.

Diffusion Models

By 2022–2023, diffusion models — which generate images by learning to reverse a gradual noise-addition process — had largely superseded GANs for image synthesis in terms of photorealism and creative control. Key commercial deployments include:

  • Stable Diffusion (Stability AI, 2022): open-source, downloadable for local inference; enables fine-tuning on specific individuals via LoRA (Low-Rank Adaptation) with as few as 15–20 reference photographs. Assessment [HIGH CONFIDENCE]: The LoRA fine-tuning pipeline represents the most operationally significant development for targeted political deepfakes, as it enables high-fidelity replication of specific individuals with minimal training data and on consumer hardware.
  • DALL-E (OpenAI): closed API access; photorealistic image generation from text prompts.
  • Midjourney: dominant for propaganda poster-style political imagery; documented use in election-period information operations.

Face-Swap and Video Manipulation

  • DeepFaceLab: the dominant open-source video face-swap toolkit; requires GPU compute and training time measured in hours; primary tool for early political deepfake videos (2018–2022).
  • Roop / INSwapper: one-shot face-swap requiring a single reference image; no training phase; dramatically lowered the skill threshold for face-replacement video in 2023.
  • Real-time face replacement (FaceSwap Live, Deep Live Cam): enables live video stream face substitution, raising the risk of deepfake use in real-time video calls, teleconferences, and command communications.

Voice Cloning

Voice synthesis has advanced in parallel with video:

  • ElevenLabs: commercial API; requires approximately 30 seconds of reference audio to clone a voice; the platform was used in documented political disinformation (Slovakia 2023 — see Combat-Documented Cases below) before implementing additional safeguards.
  • XTTS (Coqui): open-source multilingual voice cloning; inference runs locally on consumer hardware; no API or account required.
  • RVC (Retrieval-Based Voice Conversion): converts audio to a cloned voice in near-real-time; widely used in entertainment but also adapted for disinformation.

Assessment [HIGH CONFIDENCE]: The combination of one-shot face-swap and 30-second voice cloning has compressed the technical barrier for producing a convincing audio-visual deepfake of a public figure from weeks of specialized work (2019) to hours of consumer-grade effort (2024–2026).

Video Generation Models

A second-generation threat class involves full video generation — not manipulating existing footage but synthesizing video from scratch:

  • Sora (OpenAI, 2024): text-to-video generation; limited public access as of 2024–2025 but demonstrated capability for photorealistic short-form video.
  • Runway Gen-3 / Gen-4: commercially available; used in film production and increasingly accessible for IO purposes.
  • Kling (Kuaishou, PRC): competitive with Sora; significant for understanding PRC IO toolkit expansion.

Detection Evasion

Sophisticated actors have adopted techniques to degrade automated detection:

  • Re-encoding and compression cycling: re-uploading through multiple platforms adds compression artifacts that mask GAN fingerprints.
  • Film grain and analog filter overlays: obscure spectral anomalies used by forensic classifiers.
  • Adversarial perturbation layers: adding imperceptible pixel-level noise that confuses detection models without affecting human perception.
  • Distribution via low-quality re-shares: deliberately degrading resolution before distribution reduces forensic signal.

Combat-Documented Cases

Ukraine — Zelensky Surrender Video (March 2022)

The most operationally significant documented deepfake deployment in a combat context occurred on 16 March 2022, three weeks after the Russian invasion of Ukraine. A video purporting to show President Volodymyr Zelensky ordering Ukrainian soldiers to surrender was distributed via:

  • A hacked Ukrainian news website (Ukraine 24)
  • Coordinated social media amplification across Russian-aligned networks

The video was rapidly identified as synthetic — Zelensky’s face was noticeably misaligned with his body and neck proportions — and debunked within hours by Ukrainian authorities, major platforms, and media organizations. Zelensky posted an authentic counter-video from Kyiv the same day. Assessment [HIGH CONFIDENCE, multiple corroborating sources]: the deepfake was tactically ineffective as a surrender-inducing instrument but strategically useful as a probe of platform response speed and as evidence for the proposition that state actors are willing to deploy synthetic media in active combat conditions.

Significance: This case is the first documented state-attributed deepfake in a live conflict environment. It establishes a precedent — the wartime use of synthetic media to issue false command communications — that is now embedded in military doctrine planning for adversarial IO in future conflicts.

Moldova — Presidential Election Disinformation (2023)

A deepfake video depicted President Maia Sandu appearing to endorse a pro-Russian political party and announcing her intention to withdraw Moldova from EU accession talks. The video was assessed as synthetic by multiple forensic organizations and attributed to Russian-aligned IO operations targeting the 2023 electoral period. The operation followed Moldova’s pro-EU political trajectory and its severed energy dependency on Russia.

Slovakia — Election Audio Deepfake (September 2023)

Two days before the Slovak parliamentary election, a deepfake audio recording circulated on social media purporting to be a conversation between Michal Šimečka (opposition Progressive Slovakia leader) and journalist Monika Tódová, in which Šimečka appeared to discuss purchasing votes from the Roma community for €50 each.

  • Published by anonymous accounts during the 48-hour pre-election silence period (when candidates legally cannot rebut)
  • Voice cloning assessed as having used ElevenLabs or a comparable commercial tool
  • Tódová and Šimečka publicly denied the conversation; forensic analysis by multiple outlets assessed the audio as synthetic
  • Assessment [MEDIUM-HIGH CONFIDENCE]: the timing — within the pre-election silence window — was deliberate and designed to minimize the rebuttal window. The operation exploited the intersection of electoral law, social media velocity, and voice cloning capability.

Taiwan — 2024 Election Period AI-Generated Imagery

During the lead-up to Taiwan’s January 2024 presidential election, researchers at the Australian Strategic Policy Institute (ASPI) and others documented a sustained campaign of AI-generated imagery distributed across Taiwanese social media platforms depicting:

  • Fabricated endorsements from international figures for pro-Beijing candidates
  • Doctored images of candidate Lai Ching-te (DPP) in compromising or authoritarian-signaling contexts
  • AI-generated protest scenes designed to suggest social unrest

Assessment [HIGH CONFIDENCE]: While no single image achieved the viral impact of the Zelensky video, the Taiwan 2024 case documents the operational use of synthetic imagery as ambient environmental manipulation — a sustained campaign designed to shift the informational baseline rather than achieve a single dramatic effect.

Gaza/Israel (2023–2024)

Both sides of the Gaza conflict saw synthetic media enter the information environment, with AI-generated imagery used to fabricate atrocity imagery, create false humanitarian scenes, and attribute actions to adversaries. Attribution of specific synthetic content to specific actors in this environment is contested and remains assessed [LOW-MEDIUM CONFIDENCE] due to the volume of actors and the difficulty of isolating original source.

Voice Cloning Operations

Voice cloning has distinct IO applications beyond video deepfakes:

CEO and Financial Fraud

In 2019, the CEO of a UK-based energy company transferred €220,000 (approximately $243,000) to a Hungarian bank account after receiving a phone call from a voice he believed to be his parent company’s German CEO. The voice was subsequently assessed to have been a deepfake generated by voice-cloning software. This remains the first publicly documented case of a large-scale financial fraud executed via AI-synthesized voice.

Assessment [HIGH CONFIDENCE, documented by Wall Street Journal reporting]: while this case was criminal rather than state IO, it established the proof-of-concept for voice cloning as a vector for targeted manipulation of high-value individuals, including military and intelligence officials.

Intelligence Operation Risk: Impersonating Commanders

The combat-context risk of voice cloning in military command is assessed as HIGH and currently underweighted in Western military doctrine:

  • A sufficiently convincing audio deepfake of a field commander could be used to issue false withdrawal, surrender, or targeting orders via radio or encrypted messaging systems.
  • Authentication protocols designed for the pre-voice-cloning era — which rely on voice recognition as an implicit authentication factor — are insufficient against modern voice synthesis.
  • The Ukraine conflict demonstrated operational interest in precisely this attack vector (the Zelensky video targeted the surrender-order command frame).

Current assessed state: no confirmed successful military command-impersonation deepfake has been publicly documented; this reflects either non-occurrence, operational security preventing disclosure, or success without attribution. The absence of documented cases should not be read as absence of capability.

The Liar’s Dividend

The concept of the “liar’s dividend” — introduced by legal scholars Robert Chesney and Danielle Citron (2019) in their foundational paper “Deep Fakes: A Looming Crisis for National Security, Democracy, and Privacy” — describes a second-order strategic effect: the mere existence of deepfake capability provides strategic benefit to bad actors even when no specific deepfake is deployed.

The mechanism:

  1. The target audience becomes aware that synthetic video and audio exist and are increasingly indistinguishable from authentic material.
  2. Genuine, authentic footage of a politician, military officer, or public figure can subsequently be credibly dismissed as “a deepfake” — even when it is not.
  3. The attacker’s epistemic cover expands with each new capability announcement and each new publicized deepfake deployment, regardless of whether the attacker produced the specific content being disputed.

Documented Liar’s Dividend Cases

  • Ali Bongo Ondimba (Gabon, 2019): following a long public absence, the Gabonese government released a video of President Bongo addressing the nation. Opposition groups and some international commentators immediately claimed the video was a deepfake — it was not, but the Gabon military used the controversy (and resulting uncertainty about Bongo’s actual health) as partial justification for a coup attempt.
  • Chidi Nwosu (Nigeria, 2019): a politician caught in an apparently compromising video claimed it was a deepfake; it was not, but the claim created sufficient public doubt to blunt the political damage.
  • Multiple US and European cases (2020–2024): politicians across the political spectrum have invoked “deepfake” or “AI manipulation” defenses against authentic footage. In some cases, independent forensic analysis confirmed authenticity; in others, uncertainty persisted.

Assessment [HIGH CONFIDENCE]: The liar’s dividend is already operational. It represents a shift in the threat model — the goal is no longer only to inject false content but to degrade confidence in authentic content, ultimately undermining the evidentiary basis of public political communication.

Detection Methods and Arms Race

Current Forensic Detection Methods

Digital artifact analysis:

  • Compression artifact patterns: GAN and diffusion outputs have characteristic spectral signatures that differ from camera-captured video under frequency-domain analysis.
  • Temporal inconsistency: face-swap models sometimes produce frame-to-frame flickering at the face boundary; motion blur handling differs from authentic footage.
  • Physiological signals (rPPG): remote photoplethysmography detects subtle skin color fluctuations corresponding to heartbeat; synthetic faces do not replicate this signal correctly in current models.
  • GAN fingerprints: each GAN architecture leaves characteristic spectral anomalies; classifiers trained on known GAN outputs can identify these signatures.
  • Eye and teeth rendering: specific failure modes in early-generation deepfakes (2018–2021); largely resolved in current generation models, reducing classifier utility.

Provenance-based approaches:

  • C2PA (Coalition for Content Provenance and Authenticity): an industry consortium standard (Adobe, Microsoft, Intel, BBC, others) that cryptographically signs media at point of capture, creating a tamper-evident chain of custody. Assessment [MEDIUM CONFIDENCE, long-term]: C2PA is the most structurally sound approach to the deepfake problem, but requires camera hardware support, consistent signing infrastructure, and platform-level verification — a multi-year deployment horizon. Current coverage is thin.
  • Content credentials: the C2PA user-facing implementation; displayed as metadata badges by participating platforms.

Detection at Scale

  • Meta, YouTube, TikTok, Google have deployed in-house deepfake classifiers with claimed detection rates that fluctuate with each generation of generation models.
  • The Deepfake Detection Challenge (DFDC) (Facebook/Meta, 2019–2020) produced the largest public dataset for classifier training; top models achieved ~82% accuracy on held-out test data — a figure that has likely degraded as generation models improved.

The Structural Asymmetry

Assessment [HIGH CONFIDENCE]: Generation capability consistently improves faster than detection capability can retrain. This reflects a structural asymmetry:

  • Generation: each new model generation expands realism and degrades existing classifier performance.
  • Detection classifiers: require labeled training data from the generation model they target; there is always a lag between a new generation model’s deployment and the availability of a classifier trained on its outputs.
  • Adversarial fine-tuning: attackers can actively probe detection classifiers (black-box or white-box) and optimize generation outputs to evade them.

The practical implication for IO analysis: forensic detection should not be treated as a scalable solution for the general population. Its current utility is time-bounded (for the window before generation models improve), expert-dependent (requiring access to forensic tools and expertise), and evasion-vulnerable.

State and Commercial Actors

Russian IO Use

Russian information operations have historically invested in video manipulation pre-dating modern deepfakes (the RT/Ruptly practice of selectively editing and recontextualizing authentic video is documented extensively). The Zelensky video (March 2022) — attributed to Russian-aligned IO actors on the basis of distribution patterns, timing, and narrative alignment — represents the known ceiling of documented Russian deepfake deployment in a combat context. Assessment [MEDIUM-HIGH CONFIDENCE]: Russian capability likely exceeds what has been publicly observed, with high-quality synthetic content potentially reserved for escalatory moments not yet reached.

Chinese IO Use (Taiwan-Focused)

PRC-linked information operations targeting Taiwan have increasingly incorporated AI-generated imagery and synthetic media. Documented operations (ASPI, 2024; Mandiant, 2024) have used:

  • AI-generated profile photographs for fake persona networks
  • Synthetic imagery of Taiwanese political figures
  • Fabricated news network aesthetic elements (logos, chyrons) applied to synthetic content

Assessment [HIGH CONFIDENCE]: PRC IO targeting Taiwan represents the most sustained, volume-driven application of AI-generated imagery in a geopolitical information campaign as of 2024–2026. The operational model is ambient saturation rather than high-impact singular events.

NSFW Deepfakes as Political Harassment

Non-consensual intimate imagery (NCII) generation using deepfake tools has been documented as a political weapon specifically targeting female politicians and public figures:

  • Documented cases in South Korea, India, the UK, and multiple European countries
  • The UK’s Online Safety Act 2023 explicitly criminalized NCII generation, in part in response to documented political uses
  • Assessment [HIGH CONFIDENCE]: NCII deepfakes function as a deterrence instrument, creating reputational, psychological, and professional costs for women in public life. This represents a distinct IO application distinct from electoral narrative manipulation.

Commercial Synthetic Media Firms

A growing commercial ecosystem provides synthetic media capabilities with limited enforcement of end-use restrictions:

  • Deepfake-as-a-service platforms operating in jurisdictions with weak enforcement
  • Avatar generation companies providing synthetic video presenters (some with legitimate entertainment/marketing uses) that have been adapted for influence operation propaganda channels (documented by Stanford Internet Observatory)

Regulatory and Counter-IO Response

Legislative Landscape

United States:

  • California AB 739 / SB 926 (2023–2024): prohibits materially deceptive audio/video content depicting candidates for state office within 60 days of election; requires disclosure labels on synthetic media; limited to state elections.
  • Federal proposals (DEEPFAKES Accountability Act, DEFIANCE Act): proposed but not enacted as of 2024–2025; fragmented state-level approach reflects absence of federal consensus.
  • NDAA provisions: Department of Defense directed to assess synthetic media threats to military command and control; classified assessments not public.

European Union:

  • AI Act (2024): classifies deepfake-generating AI systems as high-risk when deployed for biometric manipulation; requires disclosure when synthetic media depicts real persons; enforcement via national authorities.
  • Digital Services Act: requires large platforms to conduct risk assessments for deepfake-driven electoral manipulation; transparency reporting obligations.

UK:

  • Online Safety Act 2023: criminalizes non-consensual intimate deepfakes; Ofcom implementation guidance under development.

Platform Policies

Major platforms have adopted policies requiring synthetic media labeling (Meta, YouTube, TikTok as of 2023–2024) but enforcement consistency is low:

  • Detection-dependent enforcement inherits the structural asymmetry problem described above
  • Cross-platform coordination remains ad hoc despite GIFCT (Global Internet Forum to Counter Terrorism) infrastructure that could theoretically be adapted

Military Doctrine Adaptation

Assessment [MEDIUM CONFIDENCE]: Military and intelligence services are adapting source verification protocols for command communications, including:

  • Multi-factor authentication for high-value command voice communications
  • Out-of-band verification requirements before acting on unexpected command instructions received via voice
  • Development of internal watermarking and authentication standards for authorized communications

The speed of institutional adaptation is assessed as lagging behind the pace of capability development, particularly for smaller and less technically advanced militaries that face the same synthetic media threat environment.

Strategic Assessment

Priority Threat Vectors (ranked by assessed probability × impact, 2026)

VectorAssessed ProbabilityAssessed ImpactNet Priority
Liar’s dividend / epistemic corrosionHIGHHIGHCRITICAL
Election-period audio/visual deepfakeHIGHMEDIUM-HIGHHIGH
Voice cloning for financial fraudVERY HIGHMEDIUMHIGH
Military command impersonationLOW-MEDIUMVERY HIGHHIGH
NSFW deepfake as political harassmentHIGHMEDIUMMEDIUM-HIGH
Full synthetic video (Sora-class) for major IO eventMEDIUM (2026), HIGH (2028+)HIGHMEDIUM-HIGH

Key Analytical Judgment

The transition from deepfakes as discrete events to synthetic media as environmental background is already underway. The strategic problem is no longer “how do we detect and remove specific deepfakes” but “how do societies maintain shared epistemic reference points when the authenticity of any media artifact is structurally contestable.” This is a harder problem and one that current regulatory and technical responses are not adequately scoped to address.

Key Connections

Sources

  • Chesney, Robert and Danielle Citron. “Deep Fakes: A Looming Crisis for National Security, Democracy, and Privacy.” California Law Review 107, no. 6 (2019). — Foundational conceptual framework for liar’s dividend and threat taxonomy.
  • Goodfellow, Ian J. et al. “Generative Adversarial Nets.” Advances in Neural Information Processing Systems 27 (2014). — Original GAN paper.
  • Australian Strategic Policy Institute (ASPI). “Influence Operations Targeting Taiwan’s 2024 Presidential Election.” ASPI Cyber Program, 2024. — Taiwan AI-generated imagery campaign documentation.
  • Meta/Facebook. “Deepfake Detection Challenge Dataset.” Research blog, 2020. — Classifier benchmark data.
  • Coalition for Content Provenance and Authenticity (C2PA). Technical specification, Version 1.3, 2023. https://c2pa.org/specifications/
  • Stanford Internet Observatory. Multiple reports on synthetic media in information operations, 2022–2024.
  • European Parliament and Council. “Artificial Intelligence Act” (Regulation EU 2024/1689). Official Journal of the European Union, 2024.
  • Directive (EU) 2022/2555 (NIS2); European Digital Services Act, Regulation EU 2022/2065.
  • Wall Street Journal. “Fraudsters Used AI to Mimic CEO’s Voice in Unusual Cybercrime Case.” August 30, 2019. — CEO voice cloning fraud case.
  • Atlantic Council Digital Forensic Research Lab (DFRLab). Multiple incident reports on Ukraine war synthetic media, 2022–2024.
  • Mandiant/Google Threat Intelligence. “DRAGONBRIDGE: An Analysis of a PRC-Aligned Influence Operation Network.” Multiple editions, 2022–2024.

Graph View

Backlinks