Motif-2-12.7B-Reasoning is a 12.7 billion parameter language model developed by Motif Technologies, specifically optimized for complex reasoning and long-context understanding. As an enhanced version of the Motif-2-12.7B-Instruct model, it focuses on logical deduction and multi-step problem-solving. It is an open-weight model designed to provide high-performance reasoning capabilities within a relatively compact parameter footprint.
The model's architecture incorporates Grouped Differential Attention (GDA), which disentangles attention pathways into signal-preserving and noise-control components. This mechanism is intended to improve representational efficiency by focusing computational resources on useful information transmission while maintaining stability. The model was pre-trained on a 5.5 trillion token corpus using a curriculum-driven data scheduler that progressively introduces more complex mathematical, scientific, and programming content.
To refine its reasoning proficiency, the model underwent a multi-stage training process involving two-stage Supervised Fine-Tuning (SFT) and Reinforcement Learning Fine-Tuning (RLFT). These stages are used to stabilize training and align the model's chain-of-thought outputs with verified logical steps, mitigating issues like distribution mismatch and model collapse often seen in smaller reasoning models.
Motif-2-12.7B-Reasoning supports a standard context window of 65,536 tokens, which can be extended to 128,000 tokens through scaling techniques. It has demonstrated proficiency in domains requiring high precision, such as mathematics and coding, where it often rivals the capabilities of larger open-source systems.