Real-Time Speech Emotion Recognition Using Lightweight Convolutional Neural Network

Speech Emotion Recognition (SER) allows machines to understand human emotional states from their speech, enabling applications in human-computer interaction, call center analytics, and mental health monitoring. Traditional deep CNNs achieve high accuracy; however, they are costly when it comes to computation, which means they cannot be used in real time on edge devices with limited computational power. This paper proposes a lightweight CNN-based architecture for real-time SER using Log-Mel spectrograms as input representations. The proposed model incorporates Conv2D layers, batch normalization, max pooling, and global average pooling to learn discriminative time-frequency features while reducing model complexity and inference overhead. Unlike the existing Artificial Neural Network (ANN) baseline, which was trained using only the RAVDESS dataset, the proposed CNN is trained using a combined RAVDESS and TESS dataset covering eight emotion classes: neutral, calm, happy, sad, angry, fearful, disgust, and surprised. Experimental evaluation shows that theproposedLightweightCNNattained75.70%testaccuracywithaweightedF1-scoreof0.78, compared to the ANN baseline model’s 54.86% accuracyand0.55F1-score. Additionally, compared to the ANN baseline, the proposed Lightweight CNN architecture demonstrated faster real-time performance using microphone input while providing confidences cores for each emotion detected. Overall, the proposed lightweight CNN model provides faster inference with good accuracy for SER applications.