Llama 3.3 70B Instruct is a large language model developed by Meta, designed to deliver performance comparable to the Llama 3.1 405B model while operating at a significantly smaller and more efficient scale. It is an instruction-tuned, auto-regressive generative model optimized for multilingual dialogue and complex reasoning tasks.
The model utilizes an optimized transformer architecture featuring Grouped-Query Attention (GQA) to enhance inference scalability. Its development involved a new alignment process using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF), which allows it to excel in instruction-following, coding, and mathematical reasoning. It was pretrained on over 15 trillion tokens of data with a knowledge cutoff of December 2023.
With a context window of 128,000 tokens, Llama 3.3 70B Instruct supports extensive inputs for long-form content generation and analysis. It provides native support for multiple languages, including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai. The model is released under the Llama 3.3 Community License, permitting use for both commercial and research purposes.