IBM has announced the release of two new multilingual embedding models, the granite-embedding-311m-multilingual-r2 and granite-embedding-97m-multilingual-r2, both designed to improve multilingual retrieval quality while addressing the size limitations typically faced by such models.
Overview of the New Models
The 311M-parameter full-size model achieves a score of 65.2 on the MTEB Multilingual Retrieval benchmark, ranking second among open models under 500 million parameters. In contrast, the 97M-parameter compact model scores 60.3, making it the highest-performing open multilingual embedding model under 100 million parameters. Both models are capable of handling context lengths up to 32,768 tokens, a significant increase from their predecessors.
Enhanced Language Support
These models support over 200 languages, with enhanced retrieval capabilities for 52 languages and programming code across nine languages. The training process utilized a mix of IBM-curated datasets and publicly available data, ensuring a robust foundation for enterprise applications.
Technical Advancements
The transition from the previous R1 models to the R2 generation involved a complete architectural overhaul, utilizing the ModernBERT encoder. This new architecture incorporates techniques from recent transformer research, leading to improved efficiency and performance. Notably, the models employ rotary position embeddings and support for Flash Attention 2.0, which enhances encoding speed on modern GPUs.
Deployment and Integration
Both models are released under the Apache 2.0 license and are designed for seamless integration with existing frameworks such as LangChain, LlamaIndex, and Haystack. Users can implement these models with minimal changes, allowing for broad accessibility across various applications.
In summary, the Granite Embedding Multilingual R2 models represent a significant advancement in multilingual AI capabilities, balancing performance and size while broadening language support for diverse applications.
This article was produced by NeonPulse.today using human and AI-assisted editorial processes, based on publicly available information. Content may be edited for clarity and style.
