Tiny AI Camera Captures 3D Color Images

image provided by pixabay

This post is also available in: עברית (Hebrew)

Researchers from Karlsruhe Institute of Technology in Germany developed a compact multispectral camera using inkjet printers that is based on something called 3D spectral imaging, which can aid in object and material classification.

So far, managing to capture the spatial and spectral information necessary for this process requires multiple devices or time-intensive scanning procedures. This new “light field camera” deals with this challenge by capturing 3D information and spectral data simultaneously in a single snapshot.

Research team leader Uli Lemmer said that to their knowledge, this is “the most advanced and integrated version of a multispectral light field camera,” and added that they combined it with new AI methods for reconstructing the depth and spectral properties of the scene to create an advanced sensor system for acquiring 3D information.

According to interesting Engineering, light field cameras are specialized imaging devices that can capture both the direction and intensity of light rays, after which computational processing is used to reconstruct 3D image information from the collected data. These cameras typically use microlens arrays aligned with the pixels of a high-resolution camera chip.

The researchers made the new device by using inkjet printing to create a multispectral light field camera. They deposited a single droplet of material to form each individual lens on one side of ultrathin microscope slides, then printed fully aligned color filter arrays on the opposite side of the microscope slides – then they integrated the resulting optical component into a CMOS camera chip.

The inkjet printing method significantly reduced manufacturing complexity and enhanced efficiency. The researchers also discovered that an approach based on deep learning was most effective in extracting the desired information directly from the acquired measurements.

The researchers reportedly tested the camera by recording a scene containing multicolored 3D objects at various distances, then training and testing the image reconstruction algorithm using many synthetic and real multispectral images. The camera worked better than expected, and could, for example, distinguish different objects based on their spectral composition and depth information within a single snapshot.