Surveillance of Critical Infrastructures

This scenario refers to Critical Infrastructures surveillance, where Deep Learning is meant to be used to define an intelligent video-based detection system for security in and around the critical infrastructure.

WHY: in this scenario, a currently open issue is the long time that lasts between the installation of a system at a specific site and the point in time at which it becomes reliable. This contradicts a mobile and temporary use of such a system, which is the essence of this use-case. Applications are, for instance, the perimeter of critical infrastructure (power generators) and large events (major political meetings, temporary border controls, peacekeeping operations). Usually, an evaluation period of a few weeks up to multiple months is state of the art. In this period, algorithms are parameterized and tweaked for the optimal state. When using sensor fusion methods over time and multiple sensors, especially in combination with machine learning methods, this problem becomes even more difficult and important. An (offline) learning based method is most effective when data from the actual instance (sensor) has been used in the training phase, which contradicts the goal of mobility.

HOW: in this use-case, DL learning will be used for in three different phases:
a) Algorithm design: transfer learning from one scenario to another, performed off-line;
b) Algorithm adaptation/tuning: involving scene modeling, parsing and understanding (using CNNs);
c) Actual event detection, relying on a scene-aware “tracklet” model (using RNN-LSTM) to identify relevant “tracklets”.

ALOHA intends to exploit a fast workflow to transfer results from one sensor to another. This can be tackled by using the ALOHA tool flow, working on DL-based transfer of training data in the specific context and on tools facilitating the set up of video systems using DL methods. A specific challenge is to use deep learning methods to enable technician after a short training to utilize advanced capabilities of modern surveillance techniques.

Security surveillance application with object tracking (left image) and with classification and prediction of behavior (center and right images)




Project Coordinator
Giuseppe Desoli - STMicroelectronics

Scientific Coordinator
Paolo Meloni - University of Cagliari, EOLAB

Dissemination Manager
Francesca Palumbo - University of Sassari, IDEA Lab