Meta logo
Meta
Open Weights

codellama-70b-instruct

Released Jan 2024

Arena AI
#257
Parameters70B

Code Llama 70B Instruct is a large-scale language model specialized for code generation and instruction following, released by Meta in January 2024. It is the largest variant in the Code Llama family, built upon the Llama 2 architecture and trained on a dataset of 1 trillion tokens of code and code-related information. The model is designed to translate natural language prompts into code and explain technical concepts in a conversational format.\n\nThe model supports various programming languages, including Python, C++, Java, PHP, C#, TypeScript, and Bash. It features a context window capable of handling up to 100,000 tokens, enabling it to analyze large repositories and long sequences of code. The Instruct version is fine-tuned using Reinforcement Learning from Human Feedback (RLHF) to provide safer and more helpful responses when interacting with developers.\n\nCode Llama 70B Instruct achieves high performance on coding benchmarks such as HumanEval, where it has demonstrated results comparable to prominent closed-source models. It is available under the same permissive license as Llama 2, supporting both research and commercial applications.

Rankings & Comparison