ArXiv TLDR

From Binary Groundedness to Support Relations: Towards a Reader-Centred Taxonomy for Comprehension of AI Output

🐦 Tweet
2604.08082

Advait Sarkar, Christian Poelitz, Viktor Kewenig

cs.HC

TLDR

This paper proposes a reader-centered taxonomy of support relations to move beyond binary groundedness in evaluating generative AI output.

Key contributions

  • Critiques current binary groundedness evaluations for generative AI output.
  • Highlights limitations in benchmarking and provenance interfaces due to this binary view.
  • Proposes a reader-centred taxonomy of support relations between AI output and sources.
  • Suggests synthesizing the taxonomy from linguistics and philosophy of language.

Why it matters

Current methods for evaluating AI output groundedness are too simplistic. This paper introduces a nuanced framework that can reveal how AI output is supported by sources, not just if it is. This will lead to better benchmarks and more transparent user interfaces.

Original Abstract

Generative AI tools often answer questions using source documents, e.g., through retrieval augmented generation. Current groundedness and hallucination evaluations largely frame the relationship between an answer and its sources as binary (the answer is either supported or unsupported). However, this obscures both the syntactic moves (e.g., direct quotation vs. paraphrase) and the interpretive moves (e.g., induction vs. deduction) performed when models reformulate evidence into an answer. This limits both benchmarking and user-facing provenance interfaces. We propose the development of a reader-centred taxonomy of grounding as a set of support relations between generated statements and source documents. We explain how this might be synthesised from prior research in linguistics and philosophy of language, and evaluated through a benchmark and human annotation protocol. Such a framework would enable interfaces that communicate not just whether a claim is grounded, but how.

📬 Weekly AI Paper Digest

Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.