語音合成、語音轉換與生成式人工智慧技術的快速發展，偽造語音之自然度與可用性持續提升，並逐漸對身分驗證、通訊安全與數位媒體可信度造成潛在威脅，相較於早期品質較低、失真較明顯之合成語音，近年深偽語音已能在音色、韻律與語意層面呈現高度擬真，使傳統偵測方法面臨更高挑戰，因此如何建立一套兼具穩定性、可解釋性與泛化能力之深偽語音偵測系統，已成為當前重要研究議題。
本研究以ASVspoof5 資料集為主要實驗資料，並以輕量化卷積神經網路之語音偵偽架構為基礎，探討韻律資訊、多特徵融合與資料增強策略對深偽語音偵測效能之影響。針對語音韻律資訊，本研究比較語段層級與連續型韻律表示，並透過前置分析建立連續韻律特徵之篩選流程，以選定適合作為輔助分支輸入之代表性特徵；於聲學前端設計方面，本研究比較多種頻譜表徵之辨識能力，並進一步建構特徵融合架構，以驗證雙特徵輸入對偵測效能之提升潛力；最後本研究亦探討資料增強策略於混合架構下之適用性，分析其對辨識表現、分數校準與整體穩定性的影響。
研究結果顯示，韻律資訊適合作為主聲學模型之輔助特徵，而非單獨作為主要分類依據；多特徵融合相較單特徵模型具有較佳之整體表現，顯示不同聲學表徵之間確具互補性；資料增強則能在特定條件下提升模型之辨識能力，本研究最終模型於ASVspoof5 驗證集達到EER 11.58%、minDCF 0.2021，於評估集達到EER 27.77%、minDCF 0.6742。
綜合以上，本研究建立了一套以輕量化卷積神經網路為核心、結合韻律輔助資訊、特徵融合與資料增強策略的深偽語音偵測流程，並透過系統化實驗分析各模組在偵測效能與穩定性上的影響。研究成果除有助於釐清韻律特徵與多特徵融合在語音偵偽任務中的實際角色外，亦可作為後續語音偵偽系統設計與優化之參考。

With the rapid advancement of speech synthesis, voice conversion, and generative artificial intelligence technologies, the naturalness and usability of deepfake speech have significantly improved, posing increasing threats to identity verification, communication security, and the credibility of digital media. Compared with the earlier artificial speech, which was often of poor quality and contained noticeable distortion, recent deepfake speech can now closely mimic authentic voices in terms of timbre, prosody, and semantics, making traditional detection methods face unprecedented challenges. Consequently, developing a speech deepfake detection system that simultaneously ensures robustness, interpretability, and generalization capability has become an important research topic.
This study uses the ASVspoof5 dataset as the primary experimental benchmark and is based on a lightweight convolutional neural network (CNN) framework for speech deepfake detection. It investigates the effects of prosodic information, multi-feature fusion, and data augmentation strategies on detection performance. For prosodic information, both segment-level and continuous prosodic representations are compared, and a feature selection process is designed to identify representative continuous prosodic features as auxiliary branch inputs. In terms of acoustic front-end design, multiple spectrotemporal representations are evaluated, and a feature fusion architecture is constructed to assess the potential performance improvement of dual-feature input. Finally, the applicability of data augmentation strategies within the hybrid architecture is examined, analyzing their impact on detection accuracy, score calibration, and overall stability.
The experimental results indicate that prosodic information is more suitable as auxiliary features to the main acoustic model rather than being used as the sole basis for classification. Moreover, multi-feature fusion achieves better overall performance compared to single-feature models, suggesting that different acoustic representations are indeed complementary. Finally, data augmentation can enhance the model’s discriminative capability under certain conditions. The final model achieved an EER of 11.58% and a minDCF of 0.2021 on the ASVspoof5 validation set, and an EER of 27.77% and a minDCF of 0.6742 on the evaluation set.
In summary, this work establishes a speech deepfake detection framework centered on a lightweight CNN architecture, integrating prosodic auxiliary information, multi-feature fusion, and data augmentation strategies. Through systematic experimental analysis, the study clarifies the respective roles of prosodic and fused acoustic features in speech deepfake detection tasks and provides valuable insights for the future design and optimization of deepfake speech detection systems.

[1] Junichi Yamagishi et al. “ASVspoof 2021: Accelerating Progress in Spoofed and Deepfake Speech Detection”. In: arXiv preprint arXiv:2109.00537 (2021). arXiv:2109.00537.
[2] Xin Wang et al. “ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale”. In: The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024). 2024, pp. 1–8. DOI: 10.21437/ASVspoof.2024-1.
[3] Galina Lavrentyeva et al. “STC Antispoofing Systems for the ASVspoof 2019 Challenge”. In: arXiv preprint arXiv:1904.05576 (2019). arXiv: 1904.05576.
[4] Massimiliano Todisco, Hector Delgado, and Nicholas Evans. “Constant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification”. In: Computer Speech & Language 45 (2017), pp. 516–535.
[5] Xiaofei Cheng, Mingxing Xu, and Thomas Fang Zheng. “Replay Detection Using CQT-Based Modified Group Delay Feature and ResNeWt Network in ASVspoof 2019”. In: 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). 2019, pp. 540–545. DOI: 10.1109/APSIPAASC47483.2019.9023158.
[6] Xiang Wu et al. “A Light CNN for Deep Face Representation with Noisy Labels”. In: IEEE Transactions on Information Forensics and Security 13.11 (2018),pp. 2884–2896.
[7] Judith C. Brown. “Calculation of a Constant Q Spectral Transform”. In: The Journal of the Acoustical Society of America 89.1 (1991), pp. 425–434.
[8] Xin Wang and Junichi Yamagishi. “A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection”. In: arXiv preprint arXiv:2103.11326 (2021). arXiv: 2103.11326.
[9] Kaijie Ma et al. “End-to-End Dual-Branch Network Towards Synthetic Speech Detection”. In: IEEE Signal Processing Letters 30 (2023), pp. 359–363. DOI: 10.1109/LSP.2023.3262419.
[10] Aurosweta Mahapatra, Ismail Rasim Ulgen, and Berrak Sisman. “HuLA:Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech”. In: arXiv preprint arXiv:2509.21676 (2025).arXiv: 2509.21676.
[11] Xin Wang et al. “ASVspoof 5: Evaluation of Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech”. In: arXiv preprint arXiv:2601.03944 (2026). arXiv: 2601.03944.
[12] Kevin Warren et al. “Pitch Imperfect: Detecting Audio Deepfakes Through Acoustic Prosodic Analysis”. In: arXiv preprint arXiv:2502.14726 (2025). arXiv:2502.14726.
[13] Octavian Pascu et al. “Easy, Interpretable, Effective: openSMILE for Voice Deepfake Detection”. In: arXiv preprint arXiv:2408.15775 (2024). arXiv: 2408.15775.
[14] Kai Li et al. “Contributions of Jitter and Shimmer in the Voice for Fake Audio Detection”. In: IEEE Access 11 (2023), pp. 84689–84698.
[15] Paul Boersma. “Accurate Short-Term Analysis of the Fundamental Frequency and the Harmonics-to-Noise Ratio of a Sampled Sound”. In: Proceedings of the Institute of Phonetic Sciences 17 (1993), pp. 97–110.
[16] Yingbo Gao et al. “Revisiting Checkpoint Averaging for Neural Machine Translation”. In: Findings of the Association for Computational Linguistics: AACLIJCNLP 2022. 2022, pp. 188–196. DOI: 10.18653/v1/2022.findings-aacl.18.
[17] Tom Ko et al. “A Study on Data Augmentation of Reverberant Speech for Robust Speech Recognition”. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2017, pp. 5220–5224. DOI: 10.1109/ICASSP.2017.7953152.
[18] Pierre Falez and Tony Marteau. “Whispeak Speech Deepfake Detection Systems for the ASVspoof5 Challenge”. In: The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024). 2024, pp. 32–35. DOI: 10.21437/ASVspoof.2024-5.
[19] David Snyder, Guoguo Chen, and Daniel Povey. “MUSAN: A Music, Speech, and Noise Corpus”. In: arXiv preprint arXiv:1510.08484 (2015). arXiv: 1510.08484.
[20] Daniel S. Park et al. “SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition”. In: Interspeech 2019. 2019, pp. 2613–2617. DOI: 10.21437/Interspeech.2019-2680.
[21] Yassine El Kheir et al. “Two Views, One Truth: Spectral and Self-Supervised Features Fusion for Robust Speech Deepfake Detection”. In: arXiv preprint arXiv:2507.20417 (2025). arXiv: 2507.20417.
[22] Penghui Wen et al. “Robust Audio Anti-Spoofing with Fusion-Reconstruction Learning on Multi-Order Spectrograms”. In: Interspeech 2023. 2023, pp. 271–275. DOI: 10.21437/Interspeech.2023-563.
[23] Santosh Gondi and Vineel Pratap. “Performance Evaluation of Offline Speech Recognition on Edge Devices”. In: Electronics 10.21 (2021), p. 2697. DOI: 10.3390/electronics10212697.
[24] Christiane Attig et al. “System Latency Guidelines Then and Now — Is Zero Latency Really Considered Necessary?” In: Engineering Psychology and Cognitive Ergonomics: Cognition and Design. Vol. 10276. Lecture Notes in Computer Science. Springer, 2017, pp. 3–14. DOI: 10.1007/978-3-319-58475-1_1.
[25] Karla Schäfer, Matthias Neu, and Jeong-Eun Choi. “Robust Audio Deepfake Detection: Exploring Front-/Back-End Combinations and Data Augmentation Strategies for the ASVspoof5 Challenge”. In: The Automatic Speaker Verification Spoofing Countermeasures Workshop (ASVspoof 2024). 2024, pp. 56–63. DOI: 10.21437/ASVspoof.2024-9.
[26] Jee-weon Jung et al. “AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks”. In: arXiv preprint arXiv:2110.01200(2021). arXiv: 2110.01200.
[27] Hemlata Tak et al. “End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection”. In: Proceedings of the ASVspoof 2021 Workshop. 2021, pp. 1–8.
[28] Yuxiong Xu et al. “SZU-AFS antispoofing system for the ASVspoof 5 Challenge”. In: The Automatic Speaker Verification Spoofing Countermeasures Workshop(ASVspoof 2024). 2024, pp. 64–71. DOI: 10.21437/ASVspoof.2024-10.
[29] Nicolas M. Müller et al. “Harder or Different? Understanding Generalization of Audio Deepfake Detection”. In: Interspeech 2024. 2024, pp. 2705–2709. DOI:10.21437/Interspeech.2024-247.
[30] Chin Yuen Kwok et al. “Bona Fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systems”. In: Interspeech 2025. 2025, pp. 2230–2234. DOI:10.21437/Interspeech.2025-172.