Llama 4 Scout is a natively multimodal large language model developed by Meta, released as part of the initial Llama 4 collection. It utilizes a Mixture-of-Experts (MoE) architecture containing 109 billion total parameters, with 17 billion parameters active during any single forward pass across 16 experts. The model is designed for high efficiency, capable of running on a single H100 GPU when using 4-bit quantization. ## Capabilities and Architecture A defining feature of the Scout variant is its 10-million-token context window, which enables the processing of massive datasets, entire codebases, or thousands of pages of documentation in a single prompt. This long-context capability is supported by Interleaved Rotary Positional Embeddings (iRoPE), which alternates between layers with and without positional encodings to maintain performance across extreme sequence lengths. The model is natively multimodal through an early-fusion approach, allowing it to process interleaved text and image inputs within a unified backbone. ## Training and Use Cases The model was trained on a corpus of approximately 40 trillion tokens, including publicly available data and information from Meta's social platforms, with a knowledge cutoff of August 2024. Llama 4 Scout is instruction-tuned for tasks such as visual reasoning, multi-document summarization, and complex multilingual conversations across 12 primary languages. It is released under the Llama 4 Community License, supporting both commercial and research applications.