Translating Brain-Machine Interfacing (BMI) systems outside research laboratories onto real applications requires reliable measurement of their performance. However, there are no accepted, well-defined criteria to assess their effectiveness, usability or safety. Few attempts of benchmarking have mainly focused on off-line comparison of decoding accuracy, neglecting the effects of closed loop interaction, as well as other elements of the BMI loop (e.g. shared-control or hybrid BMI systems). This workshop will work on the identification of specific challenges related to assessing performance of closed-loop BMI systems, and propose specific action points that can lead toward good practices and standards on benchmarking BMI systems. This workshop is supported by CLAIRE (htttp://claire-ai.org), the IEEE EMB standards committee, the IEEE Brain Initiative, the IEEE Standards Association working group on neurotechnologies (http://standards.ieee.org/industry-connections/neurotechnologies-for-brain-machine-interfacing.html).
The workshop is aimed at all individuals interested in the development and translation of neurotechnologies from research prototypes onto clinical and consumer applications. These include novice and experienced researchers, potential innovators and entrepreneurs, as well as representatives of the industry and regulatory bodies among others
1. Participants will improve their knowledge about the current situation and practices regarding performance evaluation and standards relevant to neurotechnologies and brain-machine interfaces
2. Participants will acknowledge the importance agreed metrics and benchmarking have in the successful translation of research onto clinical and consumer applications
3. Participants will be able to identify current scenario and initiatives that can help the definition of common metrics and standards for BMI performance evaluation
Ricardo Chavarriaga: firstname.lastname@example.org