StoryTeller: Training-Free Narrative Grounding for Long-Form Audio Description· 长音频描述的无训练叙事框架
Long-form audio description (AD) requires more than describing visible actions: it must preserve characters, events, relationships, and story context across scenes so that blind and low-vision (BLV) audiences can follow a film. Modern video-language models (VLMs) are effective on short clips, but they often treat each moment independently, producing descriptions that miss who characters are, why events matter, and how the current scene connects to earlier narrative context. We propose StoryTeller, a training-free framework for story-aware long-form AD. Instead of relying only on local visual cues, StoryTeller maintains a verified narrative memory that carries forward story-relevant information across scenes, enabling later descriptions to remain coherent, grounded, and contextually informative. Given only raw video and a movie title, StoryTeller can optionally retrieve public movie metadata to resolve names and story context, while accepting only facts that are supported by the video through semantic filtering and VLM verification. The method requires no subtitles, scripts, AD transcripts, aligned captions, character banks, precomputed face identities, or task-specific fine-tuning. To evaluate whether generated AD preserves narrative information, we introduce StoryAD-QA, a question-answering benchmark that tests whether a language model can answer story-context questions using only the generated descriptions. Experiments on standard AD benchmarks and diverse long-form videos show that StoryTeller consistently improves narrative coherence, factual grounding, and story comprehension over strong baselines in automatic, QA-based, and human evaluations.
无训练框架 StoryTeller 提升长音频描述的叙事连贯性和信息准确性。
- 核心方法
- StoryTeller 通过维护经过验证的叙事记忆,利用跨场景区别的重要信息,并通过语义过滤和模型验证来接受视频支持的事实,生成故事感知的长音频描述。
- 适合谁读
- 研究者、工程师、视障辅助产品开发者
- 要解决的问题
- 现有的视频-语言模型在长音频描述中缺乏叙事连贯性和上下文信息,无法满足视障观众的需求。
- 关键实验
- 在标准长音频描述基准和多样化的长视频上进行了实验,通过自动、问答基准和人工评估展示其性能提升。
- 主要贡献
- 提出了一种无需训练的叙事感知长音频描述框架,提升描述的连贯性、事实依据和故事理解能力。
- 意义与局限
- 该框架对改善视障观众的观影体验有重要意义,但依赖于公共电影元数据的获取和准确性。