Human-Inspired Computer Vision Can Revolutionize IoT
This post is also available in: 
עברית (Hebrew)
Engineers from the University of Pittsburgh (PITT) Swanson School of Engineering are developing a way to make computer vision more human-like, which would enable computer vision (CV) to work while consuming significantly less energy and data.
Computer vision faces two main challenges today – immense energy and memory consumption. These both restrict its utility for high-speed and resource-deficit edge applications (like hypersonic missile tracking or autonomous navigation). These limitations mean that the potential application’s size, weight, and power constraints make computer vision basically impractical.
To deal with this issue, PITT engineers are working on the next generation of computer vision inspired by how the human visual system works, hoping to use neuromorphic engineering to reinvent visual processing systems.
According to Interesting Engineering, ‘neuromorphic engineering’ is a new field that aims to create the next generation of CV systems inspired by biological processes – by reducing the number of operations required through event-based computation.
Current CV allows computers to perceive visual information using cameras and algorithms, but this method consumes an excessive amount of power and data. Rajkumar Kubendran, a senior member of IEEE, is leading this work and intends to reinvent the CV pipeline’s traditional camera and processing architectures, replacing them with biological-inspired sensors, processors, and algorithms. Kubendran hopes to optimize the CV systems by incorporating biological systems, drastically reducing power consumption and eliminating the need for a large amount of data transfer.
Kubendran explains: “Humans receive visual stimuli through the retina in our eyes, and that data is sent through the optic nerve to our brain’s visual cortex for visual perception. The biological vision pipeline is heavily optimized to transfer the least data. Suppose we have retina-inspired camera sensors and visual cortex-inspired processors. In that case, we can train the processors to learn and understand visual scenes and become capable of versatile spatiotemporal pattern recognition and perception.”
Kubendran’s project uses event-based computer vision and could revolutionize critical areas like healthcare, military defense, IoT, edge computing, and industrial automation.
“Enabling the use of advanced computer vision on personal electronics can revolutionize our lifestyle through technologies such as self-driving vehicles, always-on smart surveillance, and virtual and augmented reality applications,” Kubendran concluded.
