AI summary: Microsoft researchers propose a new neural network architecture, Retentive Networks (RetNets), that could outperform Transformers in large language models. RetNets’ innovative retention mechanism allows for efficient sequence data representation, faster training, reduced memory usage, and increased inference speed. The technology could make the development and deployment of massive models more practical, potentially accelerating progress in areas like reasoning and common sense.
Read more at Emsi’s feed…