标题： 基于EEG在语音产生过程中预测超声舌图 显示英文标题

标题： Towards Ultrasound Tongue Image prediction from EEG during speech production

摘要： 先前已经开展了基于语音的脑机接口（BCI）的初步研究，使用脑信号（例如非侵入式EEG和侵入式sEEG/ECoG），但缺乏同时研究非侵入性大脑、发音和语音信号的方法，并分析大脑中的认知过程、发音运动的动力学以及产生的语音信号。本文描述了我们的多模态（脑电图、超声舌成像和语音）分析和合成实验，作为可行性研究。我们扩展了基于超声波发音数据对语音产生期间记录的脑电信号的分析。从EEG测量的大脑信号中，我们使用全连接深度神经网络预测舌头的超声图像。结果显示EEG和超声舌图之间存在一种弱但明显的关系，即网络可以区分发音的语音和中性舌头位置。 显示英文摘要

摘要： Previous initial research has already been carried out to propose speech-based BCI using brain signals (e.g. non-invasive EEG and invasive sEEG / ECoG), but there is a lack of combined methods that investigate non-invasive brain, articulation, and speech signals together and analyze the cognitive processes in the brain, the kinematics of the articulatory movement and the resulting speech signal. In this paper, we describe our multimodal (electroencephalography, ultrasound tongue imaging, and speech) analysis and synthesis experiments, as a feasibility study. We extend the analysis of brain signals recorded during speech production with ultrasound-based articulation data. From the brain signal measured with EEG, we predict ultrasound images of the tongue with a fully connected deep neural network. The results show that there is a weak but noticeable relationship between EEG and ultrasound tongue images, i.e. the network can differentiate articulated speech and neutral tongue position.