North Mini Code is a 30 billion parameter sparse Mixture-of-Experts (MoE) model developed by Cohere, specifically optimized for agentic software engineering and terminal-based tasks. It is the flagship release in the "North" family of models, designed to provide high-performance developer assistance while maintaining a small active footprint of 3 billion parameters per token. This efficiency allows the model to run on single-GPU hardware, such as a single H100, or in local environments while remaining competitive with significantly larger dense models on coding benchmarks.
The model utilizes a decoder-only Transformer architecture featuring 128 total experts, with 8 experts activated for each token. A key technical characteristic is its interleaved attention mechanism, which alternates between sliding-window attention using Rotary Positional Embeddings (RoPE) and global attention (which uses no positional embeddings) in a 3:1 ratio. This hybrid design is intended to facilitate both local code comprehension and the long-range reasoning required for repository-level tasks across its 256,000-token context window.
Cohere trained North Mini Code through a multi-stage process involving supervised fine-tuning (SFT) on over 70,000 verifiable tasks followed by Reinforcement Learning with Verifiable Rewards (RLVR). Unlike models tuned for specific benchmarks, North Mini Code was trained against multiple agent harnesses, such as SWE-Agent and OpenCode, to ensure its capabilities generalize across various agentic scaffolds. It is particularly effective at driving shell tools end-to-end and performing complex code edits within a multi-turn agent loop.
Released under the Apache 2.0 license, the model supports broad commercial application and local fine-tuning. It includes native support for tool-use and is optimized to generate interleaved "thinking" content before providing final code or tool calls. For best performance in agentic workflows, it is recommended to pass these reasoning steps back into the conversation history, allowing the model to maintain context of its internal logic throughout multi-step tasks.