DiffusionGemma 26B A4B

DiffusionGemma 26B A4B is an experimental open-weights generative model developed by Google DeepMind that utilizes discrete text diffusion for sequence generation. Departing from the traditional token-by-token autoregressive approach, the model generates entire blocks of text simultaneously through iterative denoising of a 256-token "canvas." This paradigm shift allows it to achieve inference speeds up to four times faster than standard models, optimized for low-latency interactive workflows by shifting the decoding bottleneck from memory bandwidth to compute.

The model is built on the Gemma 4 Mixture-of-Experts (MoE) backbone, featuring approximately 25.2 billion total parameters with 3.8 billion active during inference. It employs an encoder-decoder architecture where an autoregressive encoder processes the prompt context, while a diffusion-based decoder applies bidirectional attention to refine output in parallel. This structure provides unique advantages for global logical consistency and non-linear tasks such as in-line code editing, rapid iteration, and real-time self-correction.

DiffusionGemma is a multimodal model capable of natively processing interleaved text, image, and video inputs to produce text outputs. It supports a context window of 256,000 tokens and is designed for a variety of high-speed applications, including document parsing, OCR, and agentic workflows. The model is released under the Apache 2.0 license, making it accessible for research and commercial use cases that require high-throughput generation on consumer-grade hardware.