Object-centered control of brain-computer interface systems in three-dimensional spaces using an intuitive motor imagery paradigm

Motor imagery (MI) based Brain-Computer Interfaces (BCIs) have gained increasing attention in recent years. While most of the existing electroencephalogram (EEG) based MI-BCIs have demonstrated satisfactory performance in binary and four-class classification, it is essential to develop intuitive systems with higher dimensionality for object-centered control to support daily living. However, the increase in the dimensionality in MI-BCI has been a great challenge. One of the main reasons is the difficulty of selecting proper MI tasks. To solve this issue, we proposed a novel MI paradigm that selected six daily motions that have the underlying intents aligned with the object-centered control of external devices along three axes: front-back, left-right, and up-down. The user’s perspective of intuitiveness was evaluated through an intuitiveness questionnaire, and the results from twelve participants indicated that closer cognitive matches were achieved by using the proposed MI tasks compared to the traditional ones. An information-preserved data conversion was implemented to keep temporal, spatial and band power features in video-like data. The spatiotemporal convolutional neural network (CNN) was then applied for six-class classification. The highest accuracies reached by the proposed system in six-class within-participant classification were 46.88% for hold-out validation dataset and 39.66% for the test dataset. The average validation and test accuracies were 33.99% and 26.26% respectively for seven healthy participants, which were significantly higher than chance level (16.67%). Furthermore, the key metrics obtained from the confusion matrices suggested that both the similarity of somatotopic representations and the symmetry involved in the MI signal significantly affected the classification performance. In this study, we introduced intuitive movements for use in intuitive MI-BCI paradigms and demonstrated its suitability for controlling external objects in a 3D space. Our results indicate that compound MI signals involving multiple joints are separable using deep neural networks, greatly expanding the available tasks for multiclass MI-BCIs. While large individual differences were observed, the possibility of adopting spatiotemporal CNN classifiers to decode complex MI signals has been shown. Based on our findings, future research with larger sample sizes should consider modifying the classification model architectures specific to the task at hand.

Recommended citation: Huang, Y., Toyokawa, K., Zheng, T., Shimba, K., Kotani, K., & Jimbo, Y. Object-centered control of brain-computer interface systems in three-dimensional spaces using an intuitive motor imagery paradigm. Adv. Biomed. Eng. In press
Download Paper