This post is also available in:
עברית (Hebrew)
Deep learning techniques are advancing the image quality of metalens in a method that leverages AI to upscale images, making metalens cameras useful for a wide range of tasks, from complex microscopy applications to use in mobile devices.
A science team at Southeast University in China used a multi-scale convolutional neural network to enhance resolution, contrast, and distortion in images captured by a small camera they developed (only 3 cm in length) that directly integrates a metal sensor onto a complementary metal-oxide-semiconductor (CMOS) imaging chip. The team reports that this approach could significantly enhance imaging resolution, contrast, and distortion, leading to a substantial improvement in overall image quality.
According to Interesting Engineering, metalenses are incredibly thin (frequently only a few atoms thick) optical devices that control light using nanostructures, whose tiny size allows for incredibly light and compact cameras without conventional optical lenses. However, it is challenging to obtain the required image quality with these optical parts.
The team then developed a deep learning high-quality imaging method using machine learning to address these challenges – deep learning utilizes artificial neural networks with multiple layers to automatically learn features from data and make complex decisions or predictions. The researchers used a convolutional imaging model to produce a large dataset of pairs of high and low-quality images, which were used to train a multi-scale convolutional neural network. This enables the network to identify the characteristics of each image type and transform low-quality images into high-quality ones.
The researchers then applied the new deep learning method to 100 test photos to validate it, examining the peak signal-to-noise ratio and the structural similarity index (two widely used metrics in image processing). They reported there was a noticeable improvement in both measures for the images that the neural network analyzed, and demonstrated how the method could produce high-quality image data quickly and closely to what was directly observed through experiments.
They report they are currently designing metalenses with advanced functionalities (like color and wide-angle imaging) and are developing neural network methods to enhance the imaging quality of these sophisticated metalenses.
The science team emphasizes that in order to make this technology viable for commercial use (like using them in smartphone cameras) there need to be new assembly techniques and image quality enhancement software that is tailored specifically for mobile devices.
