This research evaluates 28 quantized large language models for deployment on resource-constrained edge devices, balancing energy efficiency, accuracy, and speed.

• Quantization reduces model size and computational requirements while maintaining acceptable performance
• Different quantization methods offer varying tradeoffs between energy consumption and output quality
• Edge-optimized LLMs can achieve up to 75% energy reduction with minimal accuracy loss
• Findings provide practical implementation guidance for engineers deploying AI to edge environments

This research matters for engineering teams working on embedded AI, IoT applications, and mobile computing where power and processing constraints are critical considerations.

Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency