Extrapolating Volition with Recursive Information Markets
Abhimanyu Pallavi Sudhir, Long Tran-Thanh
TLDR
This paper formally analyzes recursive information markets using LLM buyers to overcome information asymmetry, with applications in AI alignment.
Key contributions
- Formally analyzes LLM buyers in information markets to mitigate information asymmetry.
- Introduces a novel recursive version of the information market mechanism.
- Addresses the "buyer's inspection paradox" by enabling LLMs to "forget" inspected info.
- Applies the recursive mechanism to AI alignment, relating to Extrapolated Volition.
Why it matters
This paper provides a formal framework for a new type of information market that could significantly improve efficiency by overcoming inherent asymmetries. Its recursive design and application to AI alignment offer a promising path for scalable oversight and safer AI systems.
Original Abstract
One of the impediments to the efficiency of information markets is the inherent information asymmetry present in them, exacerbated by the "buyer's inspection paradox" (the buyer cannot mitigate the asymmetry by "inspecting" the information, because in doing so the buyer obtains the information without paying for it). Previous work has suggested that using Large Language Model (LLM) buyers to inspect and purchase information could overcome this information asymmetry, as an LLM buyer can simply "forget" the information it inspects. In this work, we analyze this mechanism formally through a "value-of-information" paradigm, i.e. whether it incentivizes information to be priced and provided in accordance with its "true value". We focus in particular on our new recursive version of the mechanism, which we believe has a range of applications including in AI alignment research, where it is related to Extrapolated Volition and Scalable Oversight.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.