Phi-4 is a 14.7 billion parameter language model developed by Microsoft, representing a significant advancement in the Phi family of small language models (SLMs). Released in December 2024, it is a dense decoder-only Transformer designed to deliver high-performance reasoning capabilities while remaining efficient enough for various deployment scenarios. The model was trained on approximately 9.8 trillion tokens over a 21-day period using a cluster of 1,920 NVIDIA H100-80G GPUs.
Training and Data Strategy
The development of Phi-4 continued Microsoft's data-centric philosophy, prioritizing high-quality content over pure volume. The training corpus included a blend of synthetic datasets designed to teach logic and reasoning, filtered public domain websites, and acquired academic books and Q&A datasets. This approach allowed the model to achieve performance levels often associated with larger frontier models, particularly in complex reasoning tasks.
Capabilities
Phi-4 is optimized for advanced reasoning, mathematics, and coding. It demonstrates strong instruction-following capabilities and was refined through a post-training process that included supervised fine-tuning (SFT) and direct preference optimization (DPO). While primarily a language-centric model with a 16K token context window, it serves as a foundation for subsequent specialized versions, including multimodal and reasoning-optimized variants.