GPT4All-13B-snoozy is a large language model developed by Nomic AI as part of the GPT4All ecosystem. It is a fine-tuned version of the LLaMA-13B foundational model, specifically optimized for instruction-following and assistant-style interactions. The model was trained on a curated corpus of approximately 740,000 prompt-response pairs, which includes diverse data types such as multi-turn dialogues, word problems, code, and creative writing.
The "snoozy" variant was created to improve upon earlier iterations by using a cleaner and more massive training set deduplicated via Nomic's Atlas platform. This dataset incorporates high-quality interaction logs and open-source instruction sets like the Dolly dataset. In benchmarking, the model demonstrated strong performance in common-sense reasoning and linguistic tasks compared to contemporary open-source models of its size.
Architecture and Training
Built on the LLaMA-13B architecture, the model employs a standard transformer-based structure with Rotary Positional Embeddings (RoPE). At 13 billion parameters, it represents a balance between reasoning complexity and the hardware requirements for local execution. It was released alongside the GPT4All technical report to promote open research and the reproducibility of large-scale instruction tuning.