This post is also available in:
עברית (Hebrew)
Purdue University engineers have discovered that artificial intelligence could allow for intuitive interaction between the passenger and an autonomous vehicle. The study, set to be presented at the IEEE International Conference on Intelligent Transportation Systems, explores how large language models like ChatGPT could enable AVs to better understand and respond to passenger commands, even when they are just implied.
Led by Ziran Wang, an assistant professor at Purdue’s Lyles School of Civil and Construction Engineering, the study examines how integrating large language models with AV systems could enhance their ability to interpret and act on passenger inputs. Current AVs require explicit commands from users, often through complex interfaces or precise voice commands. However, large language models can interpret more nuanced and indirect commands because they are trained on vast amounts of text data and continuously improve their understanding. This could significantly enhance the way AVs interact with passengers.
In their research, Wang and his team used large language models to assist an AV’s existing driving features rather than directly controlling the vehicle. They trained ChatGPT to handle a range of commands, from straightforward requests like “Please drive faster” to more subtle indications such as “I feel a bit motion-sick right now.” The models were programmed to consider various factors, including traffic rules, road conditions, and sensor data such as camera inputs.
According to TechXplore, the team tested their approach with a level four autonomous vehicle, which is one step below full autonomy. During experiments, the vehicle’s speech recognition system detected passenger commands and the large language models provided instructions to the vehicle’s drive-by-wire system, which controls acceleration, braking, steering, and gear shifts. Some experiments even included using a memory system that remembers that passenger’s past preferences.
The experiments included various driving scenarios such as highway speeds and parking maneuvers. Results showed that the vehicle performed well, with participants expressing less discomfort compared to rides in standard level four AVs. The use of large language models improved the vehicle’s performance, with the system adapting to commands even those not previously encountered.
The study revealed that large language models averaged 1.6 seconds to process commands—a duration acceptable for non-critical scenarios but needing improvement for circumstances that require faster responses. Additionally, while the study implemented safety mechanisms to manage any misinterpretations by the AI, “hallucination” or incorrect responses from these models remains a challenge.
Wang’s research also looks ahead, considering the integration of other AI models and the possibility of AVs communicating with each other to improve traffic management. Future projects will explore using large vision models for handling extreme weather conditions. As AI technology advances, integrating these models into AV systems could revolutionize passenger interaction and enhance overall driving experiences.
The paper was published on arXiv.
