Llama 2: Open Foundation and Fine-Tuned Chat Models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi + 63 more
TLDR
Llama 2 introduces a range of open-source large language models, including fine-tuned chat models that outperform existing open-source alternatives in benchmarks and human evaluations.
Key contributions
- Released Llama 2 models ranging from 7B to 70B parameters, covering pretrained and fine-tuned variants.
- Developed Llama 2-Chat models optimized specifically for dialogue tasks with improved helpfulness and safety.
- Provided detailed methodology on fine-tuning and safety to encourage community-driven responsible LLM development.
Why it matters
This paper matters because it offers the research community and industry accessible, high-performing open-source large language models that rival closed-source counterparts, promoting transparency, collaboration, and safer AI deployment in conversational applications.
Original Abstract
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.