New Development to Revolutionize Video Analytics

This post is also available in: עברית (Hebrew)

Image analysis technology will need to become better at understanding human intentions if it is to be employed in a wide range of applications. Computers find it harder to work out people’s next likely actions based on their current behavior. A team of researchers and colleagues has developed a detector that can successfully pick out where human actions will occur in videos, in almost real-time.

Leading researcher Hongyuan Zhu, a computer scientist at A*STAR’s Institute for Infocomm Research in Singapore, said driverless cars must be able to detect police officers and interpret their actions (such as raising a hand in traffic for stop) quickly and accurately, for safe driving. Autonomous systems could also be trained to identify suspicious activities such as fighting, theft, or dropping dangerous items, and alert security officers.

Computers can accurately detect objects in static images thanks to deep learning techniques, which use artificial neural networks to process complex image information. But videos with moving objects are more challenging. “Understanding human actions in videos is a necessary step to build smarter and friendlier machines,” says Zhu.

Previous methods for locating potential human actions in videos did not use deep-learning frameworks and were slow and prone to error.

To overcome this, the team’s “YoTube” detector combines two types of neural networks in parallel: a static neural network, which has already proven to be accurate at processing still images, and a recurring neural network, typically used for processing changing data, for speech recognition. “Our method is the first to bring detection and tracking together in one deep learning pipeline,” says Zhu.

The team tested its new detector on more than 3,000 videos routinely used in computer vision experiments. They report that it outperformed state-of-the-art detectors at correctly picking out potential human actions by approximately 20 percent for videos showing general everyday activities and around 6 percent for sports videos, according to A*STAR institute website.

The detector occasionally makes mistakes if the people in the video are small, or if there are many people in the background. Nonetheless, Zhu says, “We’ve demonstrated that we can detect most potential human action regions in an almost real-time manner.”