Efficient Deception Detection Using Multimodal Facial and Audio Transcript Features

Abstract

This paper presents an efficient deception detection framework leveraging multimodal facial and audio transcript features, achieving a notable 96% accuracy using a lightweight Convolutional Neural Network (Lite-CNN) model. Deception detection, a critical challenge in security, forensic, and psychological analysis, benefits significantly from the integration of multimodal data that captures subtle cues beyond verbal content. The proposed method fuses spatial-temporal facial expressions extracted from video frames with semantic and prosodic features derived from audio transcripts, enabling robust discrimination between truthful and deceptive behavior. Unlike traditional approaches that rely solely on handcrafted features or unimodal inputs, the Lite-CNN architecture is optimized for low-latency environments, providing a balance between computational efficiency and predictive performance. The model was trained and evaluated on benchmark deception datasets, demonstrating superior accuracy and generalization compared to existing state-of-the-art techniques. Key design considerations include attention-based feature fusion and adaptive pooling strategies that enhance discriminative power while minimizing overfitting. The results validate the effectiveness of combining facial micro-expressions with linguistic and paralinguistic signals in deception detection tasks. This study contributes to the growing field of human behavior analysis by offering a practical and scalable solution for real-time deception detection in law enforcement, virtual interviews, and border security applications.