QuEST introduces a breakthrough approach for training large language models using only 1-bit weights and activations, potentially slashing computational costs while maintaining performance.

• Achieves stable training through innovative Hadamard normalization technique
• Demonstrates near-lossless performance (within 3% perplexity of full-precision training)
• Enables 20x reduction in memory footprint during both training and inference
• Maintains excellent accuracy across various model scales (125M to 13B parameters)

This research represents a significant engineering advancement for deploying LLMs in resource-constrained environments, making AI more accessible and sustainable while reducing training and deployment costs.

QuEST: Stable Training of LLMs with 1-Bit Weights and Activations