Challenge

Surveillance video cameras are ubiquitous in U.S. cities and are especially prevalent in sensitive areas such as airports, subways, and along international borders. Over 5,000 cameras are deployed throughout Washington, D.C., alone. While such systems can provide critical evidence after a criminal or terrorist act has occurred, their greatest promise is in the capability to detect threatening or suspicious behavior in time to intervene before the event takes place. Realizing this promise has proven difficult, however, due to the sheer amount of data produced by these extensive camera systems. The systems simply produce too much video for a reasonable number of analysts to monitor effectively. Moreover, it is challenging to maintain effective human monitoring for even a modest number of cameras due to analyst fatigue and boredom. To achieve the full interventional potential of these systems, advanced video analytics software must be applied to focus analyst attention. The goal is for the software to simultaneously monitor all available video, alerting a human analyst when suspicious or threatening behavior is observed.

Benefits of SIG’s Approach

SIG’s video analytics software reduces the analyst burden from interpreting hundreds or thousands of video streams to evaluating a small number of software-generated alerts. Thus the software dramatically reduces the number of analysts required to effectively monitor a given number of cameras. SIG’s software automatically detects and tracks individual people and vehicles in surveillance video, maintaining track even when subjects move from the field of view of one camera to another. The robustness of the underlying tracking algorithm has been demonstrated on data from a diverse set of camera systems, collected in dynamic lighting conditions, and with varying levels of occlusion. SIG’s software uses these high-quality tracks to detect specific activities and develop statistical models for typical behavior. In the case of an airport exit lane, for example, the analytics software automatically learns the normal direction of traffic flow and detects individuals attempting to enter the secure zone. Should the individual succeed in accessing the secure area, the algorithm maintains track on the individual as they move from camera to camera, enabling quick and precise interdiction by authorities. The software can also cue on other suspicious behaviors, such as security checkpoint evasion, loitering, and luggage abandonment. SIG’s probabilistic models provide a measure of confidence that a particular behavior is detected (as opposed to a hard declaration), enabling prioritization of alerts and alert sensitivity adjustment.

Products/Solution                                                     

SIG’s fixed-camera video analytics software suite can be deployed for real-time analysis of surveillance video. The software incorporates a multi-threaded parallel architecture to provide a solution that scales well for large multi-camera video surveillance systems. The alert types (e.g., abnormal motion, checkpoint evasion) are configurable to address the specific surveillance goals of individual customers and camera installations. In addition, the sensitivity of the system can be adjusted to control the frequency of alerts. Depending on the number of simultaneous video streams, the software can be installed onto a desktop workstation or a rack server for larger systems..

Technology Summary

SIG’s video analytics algorithms are enabled by SIG’s probabilistic video tracking framework which efficiently and robustly finds and tracks people in dynamic environments. This unique video tracking software is founded on a Bayesian framework that models video pixel statistics for background as well as the appearance, shape, and motion of the people being tracked. The underlying non-parametric statistical models capture the spatial and temporal pixel dynamics of the scene and the person’s articulations to accurately fit to the observed video and adapt to changing environmental conditions. The Bayesian video tracking framework is implemented with modern approximate inference methods for efficient and scalable computation. A higher-level analytics engine utilizes the output of the probabilistic tracker to detect specific activities of interest. Activities are modeled as specific trajectory events (e.g., u-turns near known security checkpoints) and as combinations of trajectory events and shape-derived articulations (e.g., a person walks onto a subway platform with a suitcase, leaves the suitcase, and walks away). In addition, the analytics engine develops time-evolving models of normal aggregate behavior and detects individual behaviors that are anomalous given the learned model. Thus the system augments rule-based alerts by cueing the analyst to behaviors that may be of interest simply because they are unusual. The analytics engine incorporates the full probabilistic output of the underlying tracker to provide a mathematically rigorous confidence that a threatening or suspicious activity was observed.