Vigilant Eye Software detects salient events

© Fraunhofer FIT
(from left) The Vigilant Eye System observes a fan tribune during a soccer match. The software focuses on salient events such as a person on the edge of the pitch (red box). | People are detected in an auto tunnel. | Two people are walking in opposite direction (marked in blue).

Video record of a typical monitoring camera contains mostly common events with only a few frames showing events of particular interest. Same is true for snapshots taken in a crowded place: if anything unusual happens here it takes only a small fraction of the whole scenery and can easily go unnoticed amidst numerous other activities. Vigilant Eye Software learns what is salient in a given scene by building observation models of activities. Vigilant Eye Software then automatically detects unexpected events in a live video stream and guides an active camera to record the detected events in high resolution.

Security relevant events

Saliency is a measure of novelty. Therefore salient events may not be security relevant. Vigilant Eye Software uses the detected salient events to reduce the search for potentially dangerous security relevant events. Vigilant Eye Software learns off-line from a few video samples of security relevant events specified by experts. During operation Vigilant Eye Software analyzes all salient events in the video stream for the specified security events. 

Learning capability

Vigilant Eye Software is self-learning. Only video record of allowed events is required for the software to learn. The Vigilant Eye Software updates its knowledge about the learned events via independent on-line learning. Semi-automatic learning allows incorporation of new security relevant events into the previously learned model.

Application areas

Computer video analysis is often challenged by an infinite variety of video appearances. Yet Vigilant Eye Software continuously learns these natural video variations thus adapting its on-line processing to ever changing illumination, weather or other conditions. This inherent flexibility allows application of the Vigilant Eye Software in very different indoor and outdoor scenarios. The software application ranges from observation of crowded public places to monitoring of secluded zones with restricted entering rights.

Demo 4: Left luggage

An individual leaving a box on a street is detected and tracked.

Software modules and detection scenarios

Vigilant Eye Software modules are freely combined into a processing chain that is optimal for given application scenario and problem to be solved.

Implementation details: Vigilant Eye Software is written in portable c++, has modular structure and exploits multiple levels of parallelization. The software is easily embedded by different SDKs to process video input from various cameras.

  • Operating system: Windows (including .NET), Mac, Linux
  • Hardware platform: modern CPU, cuda-capable nVidia GPU, multiple GPUs and/or CPUs
  • Run-time configuration: text configuration files (.json format), Python scripting

All software modules operate unconstrained in outdoor and indoor environment.

2nd generation of the Vigilant Eye System

Specialized image analysis algorithms allow the system to automatically detect the use of flares at an early stage. The system points out critical situations to surveillance personnel and automatically records high-resolution video footage that can be used in evidence.

Demo 1: Soccer stadium

Demo 2: Soccer stadium

Demo 3: Cyclist