Новое достижение наших партнеров из ВШЭ и их партнеров из МГМСУ и НИИ им. Склифосовского -- "декодирование" речи (26 слов) на основе классификации ЭКоГ и стереоЭЭГ с минимальным числом каналов собственной сверточной сетью:
We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single sEEG shaft or an ECoG stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation. Results: We achieved on average 58% accuracy using only 6 channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 72% accuracy using only 8 channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training.
Artur Petrosyan, Alexey Voskoboinikov, Dmitrii Sukhinin, Anna Makarova, Anastasia Skalnaya, Nastasia Arkhipova, Mikhail Sinkin, Alexei Ossadtchi. Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network. bioRxiv, June 9, 2022. https://www.biorxiv.org/content/10.1101/2022.06.07.495084v1.article-info
We collected one hour of data (from two sessions) in two patients implanted with invasive electrodes. We then used only the contacts that pertained to a single sEEG shaft or an ECoG stripe to decode neural activity into 26 words and one silence class. We employed a compact convolutional network-based architecture whose spatial and temporal filter weights allow for a physiologically plausible interpretation. Results: We achieved on average 58% accuracy using only 6 channels of data recorded with a single minimally invasive sEEG electrode in the first patient and 72% accuracy using only 8 channels of data recorded for a single ECoG strip in the second patient in classifying 26+1 overtly pronounced words. Our compact architecture did not require the use of pre-engineered features, learned fast and resulted in a stable, interpretable and physiologically meaningful decision rule successfully operating over a contiguous dataset collected during a different time interval than that used for training.
Artur Petrosyan, Alexey Voskoboinikov, Dmitrii Sukhinin, Anna Makarova, Anastasia Skalnaya, Nastasia Arkhipova, Mikhail Sinkin, Alexei Ossadtchi. Speech decoding from a small set of spatially segregated minimally invasive intracranial EEG electrodes with a compact and interpretable neural network. bioRxiv, June 9, 2022. https://www.biorxiv.org/content/10.1101/2022.06.07.495084v1.article-info