标题： EgoTrigger：面向全天节能智能眼镜的人类记忆增强的音频驱动图像捕捉 显示英文标题

标题： EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses

摘要： 全天候智能眼镜有望成为能够持续上下文感知的平台，这使它们在我们的日常生活中能够提供前所未有的帮助。 然而，集成用于人类记忆增强的多模态AI代理并执行持续感知，对全天候使用提出了主要的能效挑战。 实现这种平衡需要智能、上下文感知的传感器管理。 我们的方法EgoTrigger利用麦克风的音频提示来选择性地激活耗电的相机，从而在保持人类记忆增强的大量实用性的同时实现高效的感知。 EgoTrigger使用轻量级的音频模型（YAMNet）和自定义分类头，从手-物体交互（HOI）音频提示（如抽屉打开的声音或药瓶打开的声音）触发图像捕获。 除了在QA-Ego4D数据集上进行评估外，我们还引入并在Human Memory Enhancement Question-Answer（HME-QA）数据集上进行了评估。 我们的数据集包含340个由人类标注的第一人称QA对，来自完整的Ego4D视频，这些视频经过筛选以确保它们包含音频，重点关注对于上下文理解和记忆至关重要的HOI时刻。 我们的结果表明，EgoTrigger平均可以使用54%更少的帧，在节省耗电感知组件（例如相机）和下游操作（例如无线传输）的能量方面显著节省，同时在情景记忆任务的数据集上达到相当的性能。 我们认为这种上下文感知的触发策略代表了实现节能、功能强大的全天候智能眼镜的有前途的方向——支持诸如帮助用户回忆他们把钥匙放在哪里或有关他们的日常活动的信息（例如服药）等应用。 显示英文摘要

摘要： All-day smart glasses are likely to emerge as platforms capable of continuous contextual sensing, uniquely positioning them for unprecedented assistance in our daily lives. Integrating the multi-modal AI agents required for human memory enhancement while performing continuous sensing, however, presents a major energy efficiency challenge for all-day usage. Achieving this balance requires intelligent, context-aware sensor management. Our approach, EgoTrigger, leverages audio cues from the microphone to selectively activate power-intensive cameras, enabling efficient sensing while preserving substantial utility for human memory enhancement. EgoTrigger uses a lightweight audio model (YAMNet) and a custom classification head to trigger image capture from hand-object interaction (HOI) audio cues, such as the sound of a drawer opening or a medication bottle being opened. In addition to evaluating on the QA-Ego4D dataset, we introduce and evaluate on the Human Memory Enhancement Question-Answer (HME-QA) dataset. Our dataset contains 340 human-annotated first-person QA pairs from full-length Ego4D videos that were curated to ensure that they contained audio, focusing on HOI moments critical for contextual understanding and memory. Our results show EgoTrigger can use 54% fewer frames on average, significantly saving energy in both power-hungry sensing components (e.g., cameras) and downstream operations (e.g., wireless transmission), while achieving comparable performance on datasets for an episodic memory task. We believe this context-aware triggering strategy represents a promising direction for enabling energy-efficient, functional smart glasses capable of all-day use -- supporting applications like helping users recall where they placed their keys or information about their routine activities (e.g., taking medications).