Building AI An Artificial, Multisensory Integrated Neuron “Brain”

This post is also available in: עברית (Hebrew)

Penn State researchers have recently harnessed the biological concept for application in artificial intelligence to develop the first artificial, multisensory integrated neuron, which may forever change the world of AI and robotics.

Associate professor of engineering science and mechanics at Penn State Saptarshi Das explains: “Robots make decisions based on the environment they are in, but their sensors do not generally talk to each other. A collective decision can be made through a sensor processing unit, but is that the most efficient or effective method? In the human brain, one sense can influence another and allow the person to better judge a situation.”

Das further states that the current process robots follow, which involves individual sensors sending data to a central unit to process and send instructions, consumes a lot of energy. Therefore, allowing sensors to communicate directly with each other will be efficient in terms of both energy and speed.

“The senses evolved together in biology, but separately in AI. In this work, we’re looking to combine sensors and mimic how our brains actually work,” said Das.

According to Techxplore, the team focused on integrating a tactile sensor and a visual sensor so that the output of one sensor modifies the other, with the help of visual memory. The researchers created the multisensory neuron by connecting a tactile sensor to a phototransistor. To simulate touch input, the tactile sensor used triboelectric effect, in which two layers slide against one another to produce electricity so the touch stimuli are encoded into electrical impulses. To simulate visual input, the researchers shined a light into a transistor that can remember visual input.

Das explained that an artificial multisensory neuron system could enhance sensor technology’s efficiency, paving the way for more eco-friendly AI uses. This could aid robots, drones, and self-driving vehicles to navigate their environment more effectively while using less energy.