Can We Still Detect a Fake Voice?
The state of voice deepfake detection in 2026, from acoustic fingerprints to breathing patterns and verifiable provenance
Detection used to mean hunting for artifacts left by generators. Today the field draws on four kinds of evidence: forensic cues, learned representations, signs of human speech production, and provenance. Each one answers a different question. |
A few years ago, spotting a synthetic voice often came down to things text-to-speech systems could not hide: odd pitch transitions, unusual spectral patterns, missing breaths, or speech that was a little too smooth.
That is no longer the case. Modern voice generators reproduce accents, emotion, rhythm and speaker identity with high fidelity. Some generate speech directly from neural audio tokens, others convert a voice in real time. And before the audio reaches any detector, it may have passed through compression, noise suppression, a phone line, or another neural model.
Researchers have responded with better models, but more interestingly, with different ideas about what actually separates human speech from synthetic speech. One line of work looks for artifacts left by the generator. Another learns statistical representations of real and fake audio. A third looks at the physical constraints of how humans produce speech. A fourth stops guessing from the waveform and tries to verify where the audio came from instead.
These are not competing answers to the same question, and that is the main point of this piece.
What exactly are we trying to detect?
The phrase “voice deepfake detection” usually mixes up three separate questions:
Was this speech generated or manipulated by AI? This is the classic classification problem: does the waveform carry evidence of synthesis, voice conversion, or AI editing?
Is this the person they claim to be? This is speaker verification. A cloned voice can sound identical to the enrolled speaker, and a replay attack can use a genuine recording of them. Similarity alone settles neither.
Can we verify who produced or released the audio? This is provenance. It depends on trustworthy claims from the source, not on statistical inference from the waveform.
These overlap, but they are not interchangeable. Synthetic speech can be fully authorized, like an AI-narrated audiobook. A genuine recording can be replayed to deceive. And a valid watermark can tell you which system or publisher issued a file without telling you whether what it says is true.
Acoustic forensics: fingerprints of the generation process
Most deepfake voice detection has been framed as binary classification. Given a waveform x, a detector computes a score s(x) = fθ(φ(x)), where φ extracts an acoustic representation and fθ is a learned classifier. The score can be calibrated for a particular deployment, but by default it is not a probability of fraud.
Classical systems look at frequency and phase structure:
MFCC, LFCC and CQCC coefficients summarize spectral envelopes at different frequency resolutions.
STFTs and spectrograms expose how harmonic and noise energy changes over time.
Phase, group-delay and harmonic-to-noise measures can reveal synthesis irregularities.
Residual and high-frequency features may expose neural vocoder traces not obvious to listeners.
RawNet2 [11] and AASIST [3] moved past hand-engineered features by learning directly from raw waveforms. AASIST uses graph attention to model artifacts that sit across both the spectral and temporal domains.
The idea is that two recordings can sound identical to a listener and still carry different statistical traces of how they were made. The catch is that those traces may not survive a new generator, a telephony codec, a neural audio codec, or heavy post-processing.
ASVspoof 5, with submissions from 53 teams, found exactly this. Many systems performed well, but performance dropped under adversarial attacks and neural encoding or compression [1]. A detector can learn to recognize known synthesis pipelines without becoming a general detector of synthetic speech.
Foundation models: learning a broader representation
A big shift has been the use of pretrained speech encoders like wav2vec 2.0, XLS-R, HuBERT and WavLM. The usual pipeline is waveform → pretrained encoder → pooling or attention → classifier. The embeddings capture a mix of acoustics, phonetic structure, speaker traits and timing.
But a representation built to understand what someone said is not necessarily good at telling how the recording was made. Speech encoders are often trained to be robust to channel variation and noise, which can mean throwing away exactly the evidence a forensic model needs.
Two ICML 2026 papers take different routes around this. SONAR pairs an XLS-R branch for low-frequency content with a learnable high-pass branch that extracts high-frequency residuals, and trains the two to agree on real audio and disagree on fakes [4]. It was evaluated on ASVspoof 2021 and in-the-wild data. Alethia instead pretrains an encoder specifically for deepfake detection and localization. Its authors argue that fine-tuning general speech encoders has reached diminishing returns, and they evaluate on 5 tasks and 56 benchmark datasets [5].
So the question is shifting from which classifier to put on top of a general encoder, to whether the encoder itself should be trained for forensics.
Biological and articulatory evidence: was this produced like human speech?
A newer direction asks a different question: can a detector recognize the physical constraints of human speech production? Speech comes out of breathing, vocal-fold vibration, articulation, muscle control and neurological timing, and these processes leave correlated structure across timescales.
At short timescales, vocal-fold dynamics shape pitch perturbations and harmonic structure. Across phonemes, articulatory movement shapes formants and coarticulation. Over longer stretches, breathing and neuromuscular coordination show up in pauses, intensity and rhythm.
Breathing as evidence
In Every Breath You Don’t Take, researchers at the University of Florida trained a breath detector and used simple breath features (breaths per minute, duration and spacing) to separate human-read from synthesized news audio. On 33.6 hours of in-the-wild audio they reported perfect classification (1.0 AUPRC, 0.0 EER), while a fine-tuned wav2vec 2.0 model failed almost completely on the same samples (0.99 EER) [6]. That is a striking proof of concept, but it is not evidence of universal detection. Synthesis models can learn to insert breaths, and one news-audio dataset does not represent every production pipeline.
BreathNet, a 2026 preprint, goes beyond counting breaths. It adds a breath-conditioned modulation layer trained jointly with an XLS-R encoder, which pushes the encoder to learn breath-related cues, and fuses this with spectral features. Trained on ASVspoof 2019 LA, it reports 1.99% average EER across four benchmarks and 4.70% on In-the-Wild, and 4.94% EER under the ASVspoof 5 protocol [7].
To me, the more interesting question is not whether a breath is present, but whether the breathing fits everything around it: phrase length, articulation, pauses and physiological timing.
Phonemes, formants, and vocal physiology
Other work looks at phoneme-level discrepancies, formant trajectories, harmonic relationships, jitter, shimmer and timing. An October 2026 preprint tested 36 handcrafted respiratory, vocal and temporal markers with logistic regression and SVM classifiers. The best in-domain system reached 21.40% EER, and logistic regression reached 32.77% on the out-of-domain pair [8]. Those numbers are far from the state of the art, but the markers are interpretable, so you can see which speech-production properties push a decision toward real or fake.
It is also worth being careful about calling any single marker uniquely biological. Jitter and shimmer vary with the speaker, vocal health, microphones and post-processing, and breathing patterns vary naturally from person to person.
The more promising hypothesis is consistency across the whole speech-production process, rather than one signal an AI supposedly cannot fake. A generator can insert breaths or pitch fluctuations, but keeping all of these cues consistent with each other over time is a much harder target. Future generators may learn that too.
Speaker identity and provenance
The first three approaches are passive: they infer things from the audio that arrives. Authentication and provenance add evidence that does not depend on whether the waveform looks synthetic.
Speaker verification and interactive checks
Speaker verification compares a recording against an enrolled voice. Interactive challenges ask the caller to respond to unpredictable prompts, which makes replay harder. These answer identity questions, but neither proves the speech came from a human vocal tract, and real-time voice cloning can defeat voice matching and some challenge-response schemes.
Watermarks, signatures, and authorized origin
Provenance tries to establish a verifiable link between a piece of audio and whoever created or published it. It helps to separate two mechanisms here. C2PA-style signed manifests are cryptographic: they bind media and assertions to an identity through keys and signatures, but they live in metadata that platforms often strip. Embedded audio watermarks live in the signal itself and can survive that stripping. Most of them, though, are learned signals rather than cryptographic objects, so their security depends on robustness and on what the embedded payload links back to, not on the mark alone.
All of this needs trustworthy issuance, secure key handling, robust watermarks, and enough coverage to matter. The absence of a watermark is not evidence of human speech. It only means provenance is unavailable. And a valid mark does not, by itself, prove that a speaker consented or that the content is true.
A September 2026 paper, The Watermark Shortcut by Müller and Debus, shows why provenance and detection should stay logically separate [10]. When synthetic speech in a training set is watermarked and human speech is not, detectors learn “watermark means fake” as a shortcut. That causes three problems: worse generalization, fakes that evade detection once the watermark is stripped, and real voices getting flagged as fake once they are watermarked. In their controlled experiment, watermarking real speech raised EER from 16% to 75%, and the same trick fooled a commercial detection API in a black-box test.
This one matters to me directly, since I work on audio watermarking at Verda, and marking authentic human recordings is one of the main uses of provenance. The good news is that the authors also show the shortcut is fixable: retraining with the watermark present on both real and fake audio decorrelates it and restores normal behavior. The broader lesson is that a watermark should be read by a provenance check, not absorbed by a classifier as a proxy for synthesis.
Why benchmark accuracy is not enough
Research results can look contradictory: one system gets near-perfect accuracy in a constrained setting while another falls apart on unseen generators. Both can be true, because they are measuring different conditions.
VoiceWukong (USENIX Security 2025) built a benchmark from 19 commercial and 15 open-source voice tools, with 38 data variants covering six types of manipulation. Of the 12 detectors tested, AASIST2 had the best EER at 13.50%, and every other detector exceeded 20%. A multimodal LLM, Qwen2-Audio, showed no detection ability at all [2].
Deployment brings problems that a single benchmark score can hide:
Unseen generators: the attack comes from an architecture absent from the training set.
Telephony transformations: codec losses, narrowband audio, packet loss, noise suppression, and reverberation alter evidence.
Short speech windows: a few seconds may support acoustic classification but not robust breath or prosody analysis.
Partial manipulation: only a name, phrase, or authorization sentence is synthesized.
Latency: a decision may be needed while the call is still unfolding.
Calibration and base rates: equal error rate is not the same as an operational false-alarm rate at a chosen risk threshold.
Data matters as much as architecture. An ACL 2026 study found that carefully selecting training data can beat simply pooling more of it: their pruning strategy outperformed naive aggregation of datasets while using only 3% of the available data [9]. So when you see an accuracy number, it is worth asking about the training data, evaluation set, deployment channel and operating threshold, not just the architecture.
A practical multi-evidence system
I don’t think the answer is a single best classifier. It is more likely a system that combines independent kinds of evidence, while keeping track of what each signal means and how uncertain it is.
Incoming audio would go through channel-aware acoustic analysis, forensic speech encoders, and longer-context checks on speech-production consistency. Speaker matching and interactive confirmation come in only where they make sense. Watermark and signature checks run as a separate provenance path. A decision policy then combines the calibrated evidence with the cost of an attack, and is allowed to abstain.
The output should avoid a single opaque “real/fake” label wherever possible. It can separately report evidence consistent with synthesis, a mismatch with an enrolled voice, a verified claim from a trusted issuer, no supported provenance, or not enough evidence to decide. Each of these calls for a different response.
The next frontier: detection versus verification
Synthesis models will keep getting better. Many artifacts today’s detectors rely on are quirks of particular generation pipelines, and as those pipelines change, some artifacts disappear and new ones appear. That does not make passive detection useless, but it does mean detectors need continuous evaluation against unknown models and adversarial transformations.
Physiological and articulatory cues may widen the evidence base. Forensic foundation models may learn more general manipulation signatures. Better benchmarks and data selection can improve reliability. But no passive detector can tell you who authorized a piece of audio.
The useful question is no longer just “Does this sound like AI?” It is “What evidence do we have about how this voice was produced, who is speaking, and whether the audio is authentic and authorized?” |
That is why I think voice security is heading toward a combination of forensic detection, speech-production consistency, speaker authentication and provenance. No single approach is enough on its own, and each one answers a different question.
Comments