标题： 基于元学习的深度基于能量的记忆模型 显示英文标题

标题： Meta-Learning Deep Energy-Based Memory Models

摘要： 我们研究了学习关联记忆的问题——一种能够根据失真或不完整版本检索出已存储模式的系统。 吸引子网络提供了一种关联记忆的可靠模型：模式作为网络动态的吸引子被存储下来，并通过从查询模式开始运行动态过程直到收敛到某个吸引子来实现关联检索。 在这些模型中，动态过程通常被实现为最小化能量函数的优化程序，就像经典的Hopfield网络那样。 一般来说，很难推导出既压缩又快速的给定动态和能量的写入规则。 因此，基于能量的记忆研究大多局限于可处理但不够表达能力的能量模型（不足以处理复杂的高维对象如自然图像），或者局限于没有快速写入能力的模型。 我们提出了一种新的基于能量的记忆模型（EBMM）元学习方法，这种方法允许使用任意神经架构作为能量模型，并快速将模式存储在其权重中。 我们通过实验表明，我们的EBMM方法可以为合成数据和自然数据构建压缩记忆，并且在重构误差和压缩率方面优于现有的记忆系统。 显示英文摘要

摘要： We study the problem of learning associative memory -- a system which is able to retrieve a remembered pattern based on its distorted or incomplete version. Attractor networks provide a sound model of associative memory: patterns are stored as attractors of the network dynamics and associative retrieval is performed by running the dynamics starting from a query pattern until it converges to an attractor. In such models the dynamics are often implemented as an optimization procedure that minimizes an energy function, such as in the classical Hopfield network. In general it is difficult to derive a writing rule for a given dynamics and energy that is both compressive and fast. Thus, most research in energy-based memory has been limited either to tractable energy models not expressive enough to handle complex high-dimensional objects such as natural images, or to models that do not offer fast writing. We present a novel meta-learning approach to energy-based memory models (EBMM) that allows one to use an arbitrary neural architecture as an energy model and quickly store patterns in its weights. We demonstrate experimentally that our EBMM approach can build compressed memories for synthetic and natural data, and is capable of associative retrieval that outperforms existing memory systems in terms of the reconstruction error and compression rate.