Ai-based Online Interview Behavioral Analysis: An Ai-driven Approach To Proctoring And Integrity Verification

• Author(s):

  Sankar S | Manoj S | Arun Eswar R | Suriya G | Sridharan A

• Keywords:

  Behavioral Analysis, Online Interviews, AI Proctoring, MediaPipe Face Mesh, DeepFace, Speaker Diarization, Eye Gaze Tracking, Head Pose Estimation, Facial Emotion Recognition, Picovoice, Flask Web Application, Suspicion Score, Machine Learning, Computer V

• Abstract:

  The Rapid Proliferation Of Remote Work Culture And Digitally-driven Recruitment Ecosystems Has Fundamentally Reshaped How Organizations Identify And Evaluate Talent. Online Interviews Have Emerged As The Dominant Mode Of Candidate Assessment, Offering Logistical Convenience And Geographic Flexibility. However, This Shift Introduces Unprecedented Vulnerabilities In Maintaining The Integrity And Fairness Of The Hiring Process. Candidates Operating In Unmonitored Remote Environments May Exploit The Absence Of Physical Supervision By Consulting Hidden Reference Materials, Receiving Real-time Verbal Or Textual Coaching From Off-camera Individuals, Utilizing Unauthorized Secondary Displays, Or Leveraging AI-based Answer Generation Tools — All Of Which Fundamentally Compromise The Validity Of The Assessment. This Paper Presents The Online Interview Behavioral Analysis (OIBA) System, A Comprehensive, Privacy-first, AI-driven Behavioral Monitoring Framework Specifically Engineered To Detect And Quantify Integrity Violations During Virtual Job Interviews. Unlike Existing Academic Proctoring Tools That Rely On Cloud-based Video Surveillance Or Simplistic Rule-based Anomaly Detection, OIBA Employs A Multi-modal Micro-module Architecture That Operates Entirely On-device, Ensuring Candidate Data Confidentiality While Delivering Real-time Analytical Precision. The System Integrates Four Parallel Analytical Pipelines. The Visual Analysis Subsystem Leverages Google MediaPipe Face Mesh To Extract 468 Three-dimensional Facial Landmarks Per Frame, Enabling Precise Computation Of Iris Displacement Vectors For Gaze Direction Classification Across Five Categories: Center, Left, Right, Up, And Down. OpenCV's Perspective-n-Point (solvePnP) Algorithm Processes These Landmarks To Compute Pitch, Yaw, And Roll Rotation Vectors, Enabling Head Pose Estimation With Sub-degree Angular Resolution. The Affective Analysis Component Employs DeepFace For Real-time Facial Emotion Recognition, Classifying Expressions Into Seven Categories And Correlating Elevated Stress Or Fear Indicators With Concurrent Behavioral Anomalies To Strengthen Detection Confidence. The Audio Intelligence Subsystem Utilizes Picovoice Falcon For On-device Speaker Diarization, Separating Audio Streams Into Distinct Speaker Tags Without Transmitting Sensitive Data To External Servers, While Picovoice Leopard Performs Speech-to-text Transcription For Subsequent NLP-based Content Matching Between Secondary Speaker Prompts And Candidate Responses. All Four Modules Contribute Weighted Signals To A Unified Suspicion Score (0–100) Rendered On A Flask-based Web Dashboard Via JustGage Visualization. A Key Innovation Of The System Is Its Adaptive Threshold Mechanism, Which Requires Continuous Anomalous Behavior For A Minimum Duration (2 Seconds For Visual Violations, 500ms For Audio Anomalies) Before Registering A Suspicion Event — A Design Decision That Reduced False Positive Rates By 62% In Controlled Testing Without Degrading Sensitivity To Genuine Violations. Experimental Evaluation On 150 Simulated Interview Sessions Demonstrated An Overall Detection Accuracy Of 89%, With Individual Module Accuracies Of 94.2% (eye Gaze), 91.3% (head Pose), 83.1% (emotion Recognition), And 92% (audio Diarization). The System Processes A 30-minute Interview In Approximately 138 Seconds, Representing A 4.4x Speed-up Relative To Real-time. Comparative Analysis Confirms That OIBA Outperforms Existing Commercial Proctoring Solutions Across Accuracy, Latency, And Privacy Preservation Metrics. The OIBA System Demonstrates That Multi-modal Behavioral AI, When Designed With A Privacy-first, On-device Processing Philosophy, Can Serve As A Reliable, Scalable, And Non-intrusive Solution For Maintaining The Integrity Of Remote Hiring Processes — Providing Recruiters With Evidence-backed, Actionable Insights Rather Than Subjective Human Judgment Alone.

---

Other Details

• Paper id:

  IJSARTV12I4105210

• Published in:

  Volume: 12 Issue: 4 April 2026

• Publication Date:

  2026-04-30

Download Article