Revisiting Non-Verbatim Memorization in Large Language Models: The Role of Entity Surface Forms
Yuto Nishida, Naoki Shikoda, Yosuke Kishinami, Ryo Fujii, Makoto Morishita + 2 more
TLDR
This paper introduces RedirectQA to show LLMs' factual memorization depends on entity surface forms, not just the underlying fact.
Key contributions
- Introduces RedirectQA, a dataset linking factual triples to diverse entity surface forms using Wikipedia redirects.
- Shows LLM factual predictions often change when only the entity's surface form is altered.
- Reveals LLMs are more robust to minor spelling variations than to aliases or abbreviations.
- Finds both entity and surface frequencies influence accuracy, with entity frequency having an independent effect.
Why it matters
This research highlights a critical blind spot in evaluating LLM factual knowledge, showing that memorization isn't just about the fact itself but also how it's phrased. Understanding this surface-form dependency is crucial for building more reliable and robust LLMs. It pushes for more nuanced evaluation methods.
Original Abstract
Understanding what kinds of factual knowledge large language models (LLMs) memorize is essential for evaluating their reliability and limitations. Entity-based QA is a common framework for analyzing non-verbatim memorization, but typical evaluations query each entity using a single canonical surface form, making it difficult to disentangle fact memorization from access through a particular name. We introduce RedirectQA, an entity-based QA dataset that uses Wikipedia redirect information to associate Wikidata factual triples with categorized surface forms for each entity, including alternative names, abbreviations, spelling variants, and common erroneous forms. Across 13 LLMs, we examine surface-conditioned factual memorization and find that prediction outcomes often change when only the entity surface form changes. This inconsistency is category-dependent: models are more robust to minor orthographic variations than to larger lexical variations such as aliases and abbreviations. Frequency analyses further suggest that both entity- and surface-level frequencies are associated with accuracy, and that entity frequency often contributes beyond surface frequency. Overall, factual memorization appears neither purely surface-specific nor fully surface-invariant, highlighting the importance of surface-form diversity in evaluating non-verbatim memorization.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.