From the Boston attack – to video analytics at the urban space

IBM Smart Surveillance Research

This post is also available in: עברית (Hebrew)

IBM Smart Surveillance Research
IBM Smart Surveillance Research

Urban spaces, wherever they are, present the most complex security challenges. They are densely populated; they rely on a complex array of interrelated local infrastructures with not always clear interactions; they are subject to rapid changes – and as a result are subject to threats the scope and level of which are not known in rural areas. 

However, along with the rise of complexity and threat level, the modern city is also becoming smarter: it is equipped with video cameras, sensors and data collection equipment from the area, from mobile phones to dedicated sensors and traffic light cameras. Cities are becoming more networked, with over two billion people in the cities with internet access, and four billions – cellular phones, and we still have not mentioned the trillion of devices connected at present to the network.

In fact, we are able to collect any data we need from the urban space. The question is how we convert it into information upon which it is possible to base decisions on how to construct smart cities and how to use, for instance, the data in order to secure and protect the urban space and the population – in real time.

The terror attack of Boston Marathon, last month, provided an ultimate proof of the importance of video systems and the analytical tools for video analysis, and of real time video analysis as a vital means of control in the urban space. Hundreds of hours of recorded video that were collected by the cameras spread over the city were fed into the dedicated analytical systems of IBM, and the images were analyzed by means of sophisticated algorithms, providing the interrogators alerts on the terrorists whose behavior fits the profile photographed by the security cameras.

However, the challenge is to move from post-event analysis to real time indications. Can the video analysis be performed in real time? Is it possible to detect unusual phenomena perceived by the camera lenses simultaneously with the streaming and reception of the images? Can the superior data items (meta-data) of the video recordings be used for analysis of the contents of these recordings? To what extent can big data tools be used for video contents and participate in the process of detection, location and automatic keying? Appropriate answers on these questions will make the video analytics tools a cornerstone of policing and security systems at the urban space. And we do not speak only about towns: video analytics is going to become a central tool for securing of facilities, and in fact – anywhere a camera can be installed and operated with communications.

IBM Smart Surveillance Suite
IBM Smart Surveillance Suite

There is also central room for the cellular phone in the video world: it can be used, for instance, for video shooting in the urban space and transferring real time video to command centers or the municipal service.  The system will identify the location of the photographed site according to GPS data of the phone, and combine the incoming video with the database of materials photographed in the same place in the past, or the ones covering the nearby environment.

In order to enable large bodies to cope with the constantly expanding quantities of video materials, IBM developed the capability of real time analysis of video contents, automatic location of defined objects, identification of traffic in areas under surveillance, location and identification of vehicles according to structure patterns and color, and even tracking continuously a suspect person according to his face picture. These capabilities were a central element in the identification of the Boston terrorists – that appeared over and over in apparently “unexpected” places.

Also in Israel, IBM extends now its activity in the field of digital video, and offers the local market advanced video technologies serving aqs infrastructures for smart cities, crime prevention and command and control tasks, involving a large amount of organizations.  The IBM system provides a more convenient infrastructure for interworking of various bodies: it enables several users, in different sites, to hold a video conference against video image presented for the use of each participant of the conference, enabling marking and tagging on top of the video stream and immediate presentation of these symbols in the various workstations. So, for instance, can factors of Home Front, Fire Extinguishers, Police, Red David Shield and municipal departments share information – in order to better cope with the emergency events.

The extension of the activity in Israel came following the success of a number of projects in which a central system was established for management of video contents in real time and upon demand (VOD). These systems were installed for IBM customers in Israel. The system serves many hundreds of users, and enables management of video accepted from various sources, in a large geographic spread. The systems enables real time recording and interrogation, keeping high level of survivability and ensuring a fast and immediate availability of the required information for decision making. The development of the system was done in close cooperation with the IBM R&D Lab in Haifa. The researchers built, for the project, a unique and first of its kind solution for handling of video information, including geographic meta-data and possibility of executing cross-sections and queries based on time, location and additional accompanying information.

IBM believes that handling of video databases will become in the near future an inseparable part of daily activity – in every field. To answer this need, IBM combines video processing tools with analysis technology and analytical computing – areas in which the company is specializing and invests large scale effort and resources.

By: Yoram Ben Yoar, Director of Video, GIS and Analytics Activity, in the IBM Israel Services Group.