标题： 高阶循环神经网络用于声学建模 显示英文标题

标题： High Order Recurrent Neural Networks for Acoustic Modelling

摘要： 消失的长期梯度是训练标准循环神经网络（RNNs）中的主要问题，这可以通过具有记忆单元的长短期记忆（LSTM）模型得到缓解。 然而，与记忆单元相关的额外参数意味着一个LSTM层的参数数量是具有相同隐藏向量大小的RNN的四倍。 本文使用具有来自多个先前时间步骤的附加连接的高阶RNN（HORNN）来解决消失梯度问题。 使用英国英语多流派广播（MGB3）数据进行的语音识别实验表明，针对修正线性单元和sigmoid激活函数提出的HORNN架构在对应RNN上将词错误率（WER）分别降低了4.2%和6.3%，并且在仅使用20%-50%的循环层参数和计算量的情况下，达到了与（投影）LSTM相似的WER。 显示英文摘要

摘要： Vanishing long-term gradients are a major issue in training standard recurrent neural networks (RNNs), which can be alleviated by long short-term memory (LSTM) models with memory cells. However, the extra parameters associated with the memory cells mean an LSTM layer has four times as many parameters as an RNN with the same hidden vector size. This paper addresses the vanishing gradient problem using a high order RNN (HORNN) which has additional connections from multiple previous time steps. Speech recognition experiments using British English multi-genre broadcast (MGB3) data showed that the proposed HORNN architectures for rectified linear unit and sigmoid activation functions reduced word error rates (WER) by 4.2% and 6.3% over the corresponding RNNs, and gave similar WERs to a (projected) LSTM while using only 20%--50% of the recurrent layer parameters and computation.