Phi-4 Mini Instruct is a 3.8 billion parameter lightweight language model developed by Microsoft. Part of the Phi-4 family, it is designed to provide high-quality reasoning, logic, and mathematical capabilities within a compact footprint suitable for memory- and compute-constrained environments. The model is a dense decoder-only Transformer that supports a context window of up to 128K tokens.
The model's training process involved approximately 5 trillion tokens of high-quality synthetic data and filtered web content. This data mix was specifically curated to focus on "reasoning-dense" information, such as textbooks and educational material. Phi-4 Mini utilizes an expanded vocabulary of approximately 200,000 tokens, enhancing its performance across multiple languages and specialized domains.
Technical Capabilities
Compared to its predecessor, Phi-3.5 Mini, the Phi-4 Mini Instruct model introduces several architectural and functional improvements:
- Advanced Reasoning: It incorporates instruction following and reasoning enhancements refined through supervised fine-tuning and direct preference optimization (DPO).
- Function Calling: The model natively supports tool-enabled function calling, allowing it to interact with external APIs and structured data sources.
- Inference Efficiency: It employs Grouped-Query Attention (GQA) with 24 query heads and 8 key/value heads to improve decoding speed and reduce memory overhead.
- Multilingual Support: The model is optimized for 23 languages, including English, Chinese, Arabic, French, German, and Japanese.