Early warning system: from face recognition by surveillance cameras to social media analysis to detecting suspicious people

Abstract

Surveillance security cameras are increasingly deployed in almost every location for monitoring purposes, including watching people and their actions for security purposes. For criminology, images collected from these cameras are usually used after an incident occurs to analyze who could be the people involved. While this usage of the cameras is important for a post crime action, there exists the need for real time monitoring to act as an early warning to prevent or avoid an incident before it occurs. In this paper, we describe the development and implementation of an early warning system that recognizes people automatically in a surveillance camera environment and then use data from various sources to identify these people and build their profile and network. The current literature is still missing a complete workflow from identifying people/criminals from a video surveillance to building a criminal information extraction framework and identifying those people and their interactions with others We train a feature extraction model for face recognition using convolutional neural networks to get a good recognition rate on the Chokepoint dataset collected using surveillance cameras. The system also provides the function to record people appearance in a location, such that unknown people passing through a scene excessive number of times (above a threshold decided by a security expert) will then be further analyzed to collect information about them. We implemented a queue based system to record people entrance. We try to avoid missing relevant individuals passing through as in some cases it is not possible to add every passing person to the queue which is maintained using some cache handling techniques. We collect and analyze information about unknown people by comparing their images from the cameras to a list of social media profiles collected from Facebook and intelligent services archives. After locating the profile of a person, traditional news and other social media platforms are crawled to collect and analyze more information about the identified person. The analyzed information is then presented to the analyst where a list of keywords and verb phrases are shown. We also construct the person's network from individuals mentioned with him/her in the text. Further analysis will allow security experts to mark this person as a suspect or safe. This work shows that building a complete early warning system is feasible to tackle and identify criminals so that authorities can take the required actions on the spot.