This AI System Turns Thoughts Into Text

This post is also available in: עברית (Hebrew)

Researchers from the University of Texas have developed a system that can translate people’s thoughts into text. The system, called a semantic decoder, could ultimately benefit patients who have lost their ability to physically communicate after suffering from a stroke, paralysis or other degenerative diseases.

By utilizing a model similar to those used by the now famous chatbot ChatGPT, the researchers trained the system to produce text that closely or precisely matches the intended meaning of the participants.

Unlike other language decoding systems in development, this system does not require subjects to have surgical implants, making the process noninvasive.

Brain activity is measured using an fMRI scanner after extensive training of the decoder, in which the individual listens to hours of podcasts in the scanner. Later, provided that the participant is open to having their thoughts decoded, their listening to a new story or imagining telling a story allows the machine to generate corresponding text from brain activity alone, according to cns.utexas.edu.

“For a noninvasive method, this is a real leap forward compared to what’s been done before, which is typically single words or short sentences,” said Alex Huth, an assistant professor of neuroscience and computer science and one of the head researchers of this study.

“We’re getting the model to decode continuous language for extended periods of time with complicated ideas,” explained Huth.

The result is not a word-for-word transcript. Instead, researchers designed it to capture the gist of what is being said or thought, albeit imperfectly. About half the time, when the decoder has been trained to monitor a participant’s brain activity, the machine produces text that closely (and sometimes precisely) matches the intended meanings of the original words.