Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG
Yiqun Sun, Pengfei Wei, Lawrence B. Hsieh
TLDR
Corpus2Skill distills enterprise knowledge into a navigable skill directory, enabling LLM agents to actively navigate and combine evidence for improved RAG.
Key contributions
- Corpus2Skill distills document corpora into hierarchical skill directories offline.
- LLM agents navigate the hierarchy using progressive summaries to find relevant information.
- Enables agents to reason about search paths, backtrack, and combine evidence across topics.
- Achieves state-of-the-art performance on WixQA, outperforming existing RAG baselines.
Why it matters
Current RAG treats LLMs as passive consumers, limiting their ability to reason about corpus organization. This paper introduces Corpus2Skill, enabling LLM agents to actively navigate a hierarchical knowledge base. This significantly enhances RAG performance by allowing more intelligent and flexible information retrieval.
Original Abstract
Retrieval-Augmented Generation (RAG) grounds LLM responses in external evidence but treats the model as a passive consumer of search results: it never sees how the corpus is organized or what it has not yet retrieved, limiting its ability to backtrack or combine scattered evidence. We present Corpus2Skill, which distills a document corpus into a hierarchical skill directory offline and lets an LLM agent navigate it at serve time. The compilation pipeline iteratively clusters documents, generates LLM-written summaries at each level, and materializes the result as a tree of navigable skill files. At serve time, the agent receives a bird's-eye view of the corpus, drills into topic branches via progressively finer summaries, and retrieves full documents by ID. Because the hierarchy is explicitly visible, the agent can reason about where to look, backtrack from unproductive paths, and combine evidence across branches. On WixQA, an enterprise customer-support benchmark for RAG, Corpus2Skill outperforms dense retrieval, RAPTOR, and agentic RAG baselines across all quality metrics.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.