IBM has introduced Granite 4.2, a family of dense, decoder-only reasoning language models (LLMs) that come in three sizes: 3B, 8B, and 30B. This release emphasizes reasoning capabilities, diverging from earlier versions that primarily served as instruction-following assistants.
Architectural Foundations
The Granite 4.2 models are built on a decoder-only dense transformer architecture. Key components include:
- Grouped Query Attention (GQA) with 40 attention heads and 8 KV heads
- Rotary Position Embedding (RoPE) with θ = 10,000,000
- Feed-Forward networks utilizing SwiGLU activation
- RMSNorm for normalization
- Separate input/output embeddings and bfloat16 precision
Each model size shares the same architectural design but varies in parameters, with the 30B model featuring 64 layers and a hidden size of 32,768.
Training Methodology
Granite 4.2 was pre-trained from scratch on approximately 15 trillion tokens using a five-phase strategy. This process extends the context window to 512K tokens, gradually transitioning from broad web-scale data to more curated sources. The supervised fine-tuning (SFT) phase incorporates a diverse dataset, with approximately 7.2 million samples, including both agentic and non-agentic data.
Quality control measures ensure that only high-quality samples enter the final SFT mixture. The data is normalized and assessed for quality using LLM-based judges, with low-scoring samples being excluded.
Reinforcement Learning and Agentic Capabilities
Following SFT, a multi-stage reinforcement learning (RL) pipeline is applied, enhancing the models’ abilities to operate as agents. The 8B and 30B models undergo additional training to learn tool usage, coding, and terminal operations within real sandboxed environments. Each model supports native tool calling and can emit tool calls in the OpenAI function-calling format.
Granite 4.2 models are released under the Apache 2.0 license, making them accessible for further research and development.
Conclusion
Granite 4.2 represents a noteworthy step forward in the evolution of reasoning-focused language models, combining advanced architectural features with a comprehensive training approach. This release not only enhances the models’ reasoning capabilities but also positions them as versatile tools for various applications.
This article was produced by NeonPulse.today using human and AI-assisted editorial processes, based on publicly available information. Content may be edited for clarity and style.
