I have recently been designing and developing a facial recognition and clustering tool to help a client generate statistics on gender representation in linear TV and online streaming services.
The system detects and clusters faces in videos (using a deep learning convolutional neural network model), allowing the user to classify each person and generate statistics on gender representation.
I developed the back-end system using Python (including flask, sklearn, opencv, face_recognition, pafy, multiprocessing, numpy and pandas), and a front-end user interface using ReactJS.
Example
After running the face recognition tool, the user scrolls through groups of automatically clustered faces. Each group of faces can be tagged as ‘Female’, ‘Male’ or ‘Unknown’.
Once the face groupings have all been tagged, statistics and graphs are generated showing when men and women were on screen, and summarising their % screen time.
Finally, the tool outputs the video with the labels applied to each tagged face.
Currently the system is only implemented on a local system for the client, but I plan to port this system over to AWS Lambda to allow it to be used online. Watch this space.