ReGLA: Refining Gated Linear Attention
By Peng Lu, Ivan Kobyzev...
Abstract:
Recent advancements in Large Language Models (LLMs) have set themselves apart with their exceptional performance in complex language modelling tasks. However, these models are also known for their significant computational and storage requirements, primarily due to the quadratic computation complexi...
Key points:
- Research on large language models
- Engineering application