The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Models
Yunze Xiao, Vivienne J. Zhang, Chenghao Yang, Ningshan Ma, Weihao Xuan + 1 more
TLDR
LLMs exhibit 'Persona Collapse,' where distinct agent profiles converge into homogeneous behavior, quantified by a new framework.
Key contributions
- Identifies 'Persona Collapse' in LLMs, where diverse agents converge into homogeneous behavior.
- Proposes a framework to quantify collapse: measures persona space Coverage, Uniformity, and Complexity.
- Observes collapse across different behavioral dimensions and reasoning domains in 10 LLMs.
- Reveals that high per-persona fidelity paradoxically leads to more stereotyped populations.
Why it matters
This paper highlights a critical limitation for LLM-based multi-agent simulations, where desired population diversity collapses into homogeneity. It offers a novel framework and diagnostics to quantify this 'Persona Collapse,' revealing that high individual fidelity can paradoxically lead to more stereotyped populations. This is crucial for developing robust and truly diverse LLM applications.
Original Abstract
Applications based on large language models (LLMs), such as multi-agent simulations, require population diversity among agents. We identify a pervasive failure mode we term \emph{Persona Collapse}: agents each assigned a distinct profile nonetheless converge into a narrow behavioral mode, producing a homogeneous simulated population. To quantify persona collapse, we propose a framework that measures how much of the persona space a population occupies (Coverage), how evenly agents spread across it (Uniformity), and how rich the resulting behavioral patterns are (Complexity). Evaluating ten LLMs on personality simulation (BFI-44), moral reasoning, and self-introduction, we observe persona collapse along two axes: (1) Dimensions: a model can appear diverse on one axis yet structurally degenerate on another, and (2) Domains: the same model may collapse the most in personality yet be the most diverse in moral reasoning. Furthermore, item-level diagnostics reveal that behavioral variation tracks coarse demographic stereotypes rather than the fine-grained individual differences specified in each persona. Counter-intuitively, \textbf{the models achieving the highest per-persona fidelity consistently produce the most stereotyped populations}. We release our toolkit and data to support population-level evaluation of LLMs.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.