Principled Content Selection to Generate Diverse and Personalized Multi-Document Summaries
Journal:
arXiv
Published Date:
May 28, 2025
Abstract
While large language models (LLMs) are increasingly capable of handling
longer contexts, recent work has demonstrated that they exhibit the "lost in
the middle" phenomenon (Liu et al., 2024) of unevenly attending to different
parts of the provided context. This hinders their ability to cover diverse
source material in multi-document summarization, as noted in the DiverseSumm
benchmark (Huang et al., 2024). In this work, we contend that principled
content selection is a simple way to increase source coverage on this task. As
opposed to prompting an LLM to perform the summarization in a single step, we
explicitly divide the task into three steps -- (1) reducing document
collections to atomic key points, (2) using determinantal point processes (DPP)
to perform select key points that prioritize diverse content, and (3) rewriting
to the final summary. By combining prompting steps, for extraction and
rewriting, with principled techniques, for content selection, we consistently
improve source coverage on the DiverseSumm benchmark across various LLMs.
Finally, we also show that by incorporating relevance to a provided user intent
into the DPP kernel, we can generate personalized summaries that cover relevant
source information while retaining coverage.
