Neuroergonomics Grand Challenge: Hackathon Passive BCI

Context & challenge:

Passive Brain-Computer Interfaces (BCIs) can estimate users’ states, e.g., their cognitive or affective states, from their brain signals, and use these estimations to adapt a human-computer interaction system accordingly [Zander2011]. As such, passive BCIs have been used for many applications, including to estimate users’ mental workload, in order to adapt education material to students’ cognitive resources [Yuksel2016] or to prevent airplane pilots from being overloaded and thus from missing alarms [Dehais2019]; or to estimate users’ affective states, in order to design adaptive video games maximizing users’ excitement or pleasure [Ewing2016], among many other. 

As such, passive BCIs are a key element for Neuroergonomics [Lotte2019], for the design of numerous real-life studies and applications of neurotechnologies [Ayaz2018]. However, beyond promising proof-of-concepts, really using passive BCIs in everyday life still requires to face numerous challenges. One of them is the well-documented large within-subject variability affecting brain signals such as ElectroEncephaloGraphy (EEG) signals. 

Indeed, EEG signals are highly non-stationary, and can change a lot across days, or even within day, even for the same user [Fairclough2020]. However, so far, the vast majority of passive BCI studies were conducted on a single day (a.k.a. session), making it unclear whether the designed BCI would still work on different days/sessions without re-calibration. 

Similarly, existing public passive BCI data sets provide brain signals recorded on a single day/session [Hinss2021], preventing the design and evaluation of passive BCIs that would work across days/sessions. Ideally, a real-life passive BCI, as envisioned in Neuroergonomics, would need to be effective and efficient at all time, i.e., across sessions, without recalibration.

This competition aims at addressing this scientific challenge, by releasing the first (to the best of our knowledge) public passive BCI data set providing EEG signals across multiple sessions for each user, and by challenging the competitors to come up with the best algorithm to decode mental workload from EEG signals on a new unseen session, from a training data set comprising several sessions.

Dataset

For this passive BCI hackathon, 15 participants (6 female; 9 average 25 y.o.) were invited to the lab for three independent experimental sessions each spaced one week apart (exactly 7 days). Each session involved a short training/warm up period. Following this, a resting state (one minute with eyes open) was recorded. Participants then completed an MATB-II task with three 5-minute blocks, each of a different difficulty level (i.e. different workload level) presented in a pseudorandom manner. By varying the number and complexity of the sub-tasks, 3 levels of workload were elicited (verified through statistical analyzes of both subjective and objective -behavioral and cardiac- data).

The project was validated by the local ethical committee of the University of Toulouse (CER number 2021-342). The dataset is part of a new open EEG database currently under development, and designed to answer a need for more publicly available EEG-based dataset to design and benchmark passive brain-computer interface pipelines (as detailed in [Hinss2021]).

Data formatting

The data are exported as a dataset from EEGlab under the .set and .fdt format. The OBS formatting guidelines were followed. Data are organized as following in the directory:

– One directory per subject
– Two sub-directories for each session
– Inside each session directory one sub-directory for the precise electrode locations (measured via the app) and one for the EEG data
– 5 (.set) files per session. Each of the epoched and preprocessed task conditions, the resting state as well as the raw file for the resting state.
– Each epoch is marked by events to show the condition (difficult, medium, easy, Resting State)
– The electrode locations are provided in a .txt file with the xyz coordinates for each electrode

The task

This competition focuses on a renowned task that elicits various levels of mental/cognitive workload: the Multi-Atribute Task Battery-II (MATB-II) developed by NASA (https://matb.larc.nasa.gov/) assess task-switching and mental workload capacities (https://matb.larc.nasa.gov). Participants have to perform several subtasks simultaneously. Depending on the condition the number of subtasks and their respective difficulty differed.

In the easy condition (label 0) participants engaged in the TRACK and SYSTEM MONITORING task. TRACK is a simulation of manual control, and the participant has to keep a target at the center of a window. The SYSTEM MONITORING TASK demands monitoring of 4 gauges and 2 warning lights.

For the medium condition (label 1) a third task was added. RESOURCE MANAGEMENT presents the participant with a fuel management system where the goal is to maintain a certain fuel level by activating and deactivating a set of pumps that allow for the allocation of fuel to several reservoirs.

Finally to the difficult condition (label 2) COMMUNICATION was added to the three previous tasks: Here the participant has to respond to radio messages by changing the frequencies of different radios. Additionally, the TRACK task was made more demanding in the DIFFICULT condition.

Each of the conditions lasted for 5 minutes. The order was randomized with the other tasks meaning that participants did not necessarily start with the easy task first. Also participants may have performed other tasks in between the conditions.

EEG acquisition

The data acquisition was performed using a 64 active Ag-AgCl Electrode system (ActiCap, Brain Products Gmbh) and an ActiCHamp amplifier (Brain Products, Gmbh). One electrode could not be used and one electrode was dedicated to record cardiac activity, resulting in 62 electrodes, placed according to the international 10-20 system. In addition the precisie electrode location was obtained using a STRUCTURE (https://structure.io) 3D camera and the chanlocs plug-in developed specifically for electrode localisation purposes (https://github.com/sccn/get_chanlocs/wiki). The sampling frequency was set to 500Hz. Impedances were kept below 10 kΩ as much as possible. Markers of all events occurring during the tasks were recorded and synchronized using the LabStreamingLayer.

Preprocessing

The preprocessing of the data was done in Matlab through the use of functions from the EEGlab toolbox. First the data from the resting state as well as the tasks was extracted from the overall recording. The electrode recording cardiac activity was removed and the remaining data was cut into two second non-overlapping epochs. Please see below for the preprocessing that was applied in order to clean the data:

– Epoching into 2-second non-overlapping epochs
– Referencing using right mastoid electrode
– High-pass filter 1 Hz (FIR Filter, pop-filtnew from EEGlab)
– Electrode rejection (average amplitude above 2 sd across channels) + spherical interpolation
– SOBI with subsequent automated IC_Label rejection (muscle, heart and eye components were rejected with a 95% threshold)
– Low-pass filter 40 Hz (FIR Filter)
– Average re-referencing (CAR)
– Down-sampling to 250 Hz

For more information, please contact Dr. Raphaëlle Roy by email.