Imagine a world where you can control devices freely, simply by thinking about what you want to do. 

A new collaboration between the East China University of Science and Technology (ECUST) and Chinese tech giant Baidu has developed an innovative approach that makes this dream a step closer. 

The developing team integrates  brain-language output systems with large language models (LLMs). This framework utilizes a “single brain-computer interface (BCI) system” to flexibly adapt to various application scenarios, enabling users to truly achieve their desires without being constrained by different languages or control objects, the Global Times learnt.

“The framework integrates Baidu’s LLM, Ernie Bot (also known as Wenxin Yiyan), into the BCI system, allowing users to input their needs through a brain-language interaction module, which is similar to a universal remote control that can interact with devices in multiple scenarios,” Jin Jing, a leading professor of this project, from the Group for Brain-Computer Interface and Control Key Lab at ECUST, told the Global Times in an interview. 

Building a multi-scenario BCI system

In a smart housing scenario, for example, a paralyzed patient is able to control various customized systems such as televisions and music through one BCI system generated by Ernie Bot. Previously, designers had to make a customized system tailored for each device for the entire BCI system to work.

“With the help of Ernie Bot, there is no longer a requirement to design a specific interface, it is automatically generated by AI and ready for immediate use.” 

By integrating an LLM into the BCI system, users can input their requirements using the brain-computer language interaction module. The system then sends the requirement information to Ernie Bot’s software development kit (SDK) via Hypertext Transfer Protocol (HTTP) requests.

In this process, the Ernie Bot’s LLM accurately analyzes the user’s intent and organizes, classifies and infers device information. This reasoning capability enables the system to automatically generate the most suitable BCI setup system for complex device control needs, according to Liu Conglin, a product operations manager at Baidu.

The notion of integrating BCI with AI was seen in other related research and development. London-based startup MindPortal Inc. in December 2023 had unveiled its non-invasive optical BCI that allows human thoughts to navigate ChatGPT, according to Psychology Today. 

According to Jack Baber, CTO and co-founder of MindPortal, “We’re enabling telepathic communication between humans and AI, ensuring humanity thrives alongside artificial intelligence.”

Unlike MindPortal, which merely uses functions of LLMs to interact with an existing BCI system to optimize its performance, Jin’s team aims to use language models to address real technical challenges, providing innovative solutions to the issues that plague BCI technology, Jin told the Global Times.

The technology has struggled with several challenges, such as having only one main function, being limited in how it can be used, and lacking advanced intelligence.

By utilizing a single BCI, the system can flexibly adapt to various application scenarios, helping users achieve their desires without being limited by language barriers or different control objects, Jin said. 

Long way from broad application

In addition to the smart housing demonstration experiments, Jin’s team is also working on training additional scenarios, such as a drone control system, to allow the AI-powered BCI system’s capability to generate more control systems for real-world applications.

In the future, their collaboration aims to dynamically update control options based on the real-time status of the devices and the needs of the user, Jin said.

However, industry observers noted that there are still several steps to take before full implementation of the technological notion can be achieved.

The BCI technology is still a long way from widespread effective application. We must wait for its gradual rollout in the market to see how Jin’s team will unleash more potential, Gao Xiaorong, a professor of Tsinghua University engaging in the BCI, told the Global Times. 

Currently, most BCI technologies are focused on medical applications. The performance of BCIs is still relatively weak, despite their promising future, he said. For instance, one of the main medical applications is for patients with amyotrophic lateral sclerosis (ALS), who are unable to communicate. Even a slight enhancement in their capabilities can significantly improve their quality of life, effectively giving them a new “wing” to communicate, Gao said.

“To some extent, the applications in the medical field determine the pace of BCI development, as it is related to many disciplines and is highly interdisciplinary.”

Jin told the Global Times that he is aware that several steps must be taken to achieve the practical application of this advanced technology, but he is optimistic about its application in the future. 

Apart from medial application, the implanted BCI with an LLM that is able to construct a generative multi-scene brain-computer interaction system can be used by ordinary people as well. “BCI offers a third hand for human beings.” 

Looking ahead, the system can be dynamically changing. We aim to optimize the BCI interaction paradigm based on user data and preferences. As users interact with the system, it will automatically record data and learn their users’ using habits, Jin said.

Gao believes that there is potential for enhanced applications of BCI technology, where someone with a baseline of 100 could be elevated to a level of 300, creating “superhuman” capabilities. These superhuman applications can include enhanced physical abilities, perception, emotions and cognition. 

However, the use of such enhancements is heavily restricted by ethical considerations, as their misuse could lead to social unfairness, Gao added. 

Recognizing the immense potential for future applications, Jin remains grounded in reality. “You could say that the system can help people to have their thoughts translated into actions with the mere flicker of intention. 

“But I don’t want to be metaphysical. Based on reality, we are providing a solution for one of the links that support BCI to be massively applied,” Jin told the Global Times. 

Jin’s team is applying technological patent based on this project. “I hope more teams will join into this research arena to facilitate the depended human and machine intelligence.”