In Imaging – Less is More 

This post is also available in: עברית (Hebrew)

Event-based – or neuromorphic – cameras are an emerging class of sensors with demonstrated advantages relative to traditional imagers. These advanced models operate asynchronously and only transmit information about pixels that have changed. This means they produce significantly less data and operate with much lower latency and power. However, they currently lack advanced ‘intelligence’ to perform more difficult perception and control tasks.

Today’s state-of-the-art (SOTA) cameras work well with scenes that have few changes to track and the imagery is relatively simple. Take, for example, a scene of a plane moving through a clear blue sky. SOTA imagers could easily track the movement of the plane. Their capabilities fail, however, in highly cluttered and dynamic scenes, limiting their use among many military applications.

DARPA will develop intelligent event-based imagers. The agency announced that three teams of researchers led by Raytheon, BAE Systems, and Northrop Grumman have been selected to develop event-based infrared (IR) camera technologies under the Fast Event-based Neuromorphic Camera and Electronics (FENCE) program. 

FENCE seeks to develop and demonstrate a low latency, low power, event-based infrared (IR) focal plane array (FPA) and a new class of digital signal processing and learning algorithms to enable intelligent sensors to handle more dynamic scenes. 

The goal is to develop a ‘smart’ sensor that can intelligently reduce the amount of information that is transmitted from the camera, narrowing down the data for consideration to only the most relevant pixels.

FENCE technology could ultimately aid a number of military applications – from autonomous vehicles and robotics to IR search and tracking. To ensure broad applicability, the researchers will also focus on developing a single solution that is flexible and adaptable so it can be used across various mission spaces, according to darpa.mil.