LiquidAI has announced the release of QAD Q4_0 GGUFs, a significant update featuring 4-bit checkpoints for its LFM2.5 models: LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. This development enables developers to utilize these models at Q4_0 memory and speed without the typical decline in quality.
The QAD (Quantization-Aware Distillation) method distills a high-precision teacher model into a quantized student model, preserving both the low memory footprint and high throughput characteristic of Q4_0 GGUFs. Remarkably, these checkpoints recover 97% of the average BF16 accuracy that is typically lost during quantization.
Benchmarking Performance
In a comprehensive evaluation, the QAD Q4_0 checkpoints were benchmarked against GGUFs produced through post-training quantization (PTQ) across various tasks, including reasoning, instruction-following, tool use, and agentic capabilities. The benchmark suite included tests such as GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4, with the BF16 GGUF serving as the performance ceiling.
The results were promising, with the QAD checkpoints retaining 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance across the four models. This substantial improvement underscores the effectiveness of the QAD approach.
Real-World Application
The decode throughput for the models was assessed on various hardware platforms, including the MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. The QAD Q4_0 checkpoints for the 230M and 350M models achieved quality comparable to Q5_K_M, with a 4-33% increase in decode throughput. For the larger models, 1.2B and 2.6B, the QAD Q4_0 checkpoints matched Q4_K_M quality while improving throughput by 3-14%.
Accessing QAD GGUFs
The QAD GGUFs are readily available on Hugging Face, allowing developers to integrate them into their applications. Users can utilize the files with llama.cpp or any runtime that supports GGUF Q4_0 artifacts, facilitating easy deployment and experimentation.
LiquidAI looks forward to seeing the innovative applications that developers will create with these enhanced models.
This article was produced by NeonPulse.today using human and AI-assisted editorial processes, based on publicly available information. Content may be edited for clarity and style.
