Case Study 02

Real-Time Face Detection, Gender & Emotion Classification

Two parallel CNNs with Grad-CAM interpretability

PythonKerasOpenCVGrad-CAM

The Problem

Reading emotional state or gender from a face is something humans do intuitively but is genuinely hard for a model — human accuracy on 7-class emotion classification is only about 65% ± 5%, and CNN architectures accurate enough for the task are usually too heavy for real-time, hardware-constrained use. The goal was a general framework for building lighter real-time CNNs, validated by a live system that does face detection, gender classification, and emotion classification simultaneously — and to actually look inside the models rather than treat them as black boxes.

Approach

01Used Haar Cascade classifiers for face detection on each incoming video frame, then extracted two differently-padded crops per face — one RGB crop for gender, one grayscale crop for emotion.
02Compared two lightweight CNN designs: a fully-convolutional network with Global Average Pooling (no fully-connected layers), and a mini-Xception architecture using depth-wise separable convolutions with residual modules — mini-Xception won out for the deployed emotion model.
03Trained the gender classifier on the IMDB faces dataset and the emotion classifier on FER-2013 (7 classes: angry, disgust, fear, happy, sad, surprise, neutral), each independently.
04Applied a 10-frame sliding-window mode filter to both predictions to stop labels from flickering frame-to-frame on live video.
05Implemented guided Grad-CAM visualization to see which pixels each CNN actually used to make its decision, turning both models from black boxes into something inspectable.

Visuals

Architecture diagram showing a live video frame going through Haar Cascade face detection, then splitting into two parallel branches — an RGB crop feeding a Simple CNN for gender classification at 96% accuracy, and a grayscale crop feeding a Mini-Xception model for emotion classification at 66% accuracy — both converging into 10-frame mode smoothing and a live overlay output, with Grad-CAM shown as an interpretability layer on both classifiers — Pipeline: face detection → parallel gender/emotion CNNs → temporal smoothing → live overlay, with Grad-CAM for interpretability.

Results

96% validation accuracy on gender classification (simple CNN, IMDB faces).

66% validation accuracy on 7-class emotion classification — in line with the ~65% human baseline for the same task.

Grad-CAM analysis surfaced a real bias: the gender classifier skewed toward western facial features and accessories, and glasses measurably interfered with emotion classification.

Reflection

The most valuable part of this project wasn't the accuracy numbers — it was using Grad-CAM to find that the gender model was biased toward certain facial features and accessories. It's one thing to know CNNs can be biased in theory; it's another to actually visualize your own model doing it. If I revisited this, I'd want to rebalance the training data to specifically address that bias rather than just document it.

← Back to all projects Next: Real-Time Bitcoin Anomaly Detection →