标题： 递归神经网络在训练于正规语言时的内部状态稳定性 显示英文标题

标题： Stability of Internal States in Recurrent Neural Networks Trained on Regular Languages

摘要： 我们对用于识别正则语言的循环神经网络的稳定性进行了实证研究。 当在激活函数中引入少量噪声时，循环层中的神经元倾向于饱和以补偿变化性。 在这种饱和状态下，网络激活的分析显示出一组类似于有限状态机中的离散状态的聚类。 我们表明，这些状态在响应输入符号时的转换是确定且稳定的。 网络对于任意长的字符串表现出稳定的行为，当对任何状态应用随机扰动时，它们能够恢复，并且其演化会收敛到原始聚类。 这一观察结果强化了将网络视为有限自动机的解释，其中神经元或神经元组编码特定且有意义的输入模式。 显示英文摘要

摘要： We provide an empirical study of the stability of recurrent neural networks trained to recognize regular languages. When a small amount of noise is introduced into the activation function, the neurons in the recurrent layer tend to saturate in order to compensate the variability. In this saturated regime, analysis of the network activation shows a set of clusters that resemble discrete states in a finite state machine. We show that transitions between these states in response to input symbols are deterministic and stable. The networks display a stable behavior for arbitrarily long strings, and when random perturbations are applied to any of the states, they are able to recover and their evolution converges to the original clusters. This observation reinforces the interpretation of the networks as finite automata, with neurons or groups of neurons coding specific and meaningful input patterns.