Innovative research by Meta, Ecole des Ponts ParisTech, and Université Paris-Saclay proposes a significant leap in AI efficiency by training large language models (LLMs) to predict multiple tokens simultaneously, diverging from the traditional single-token prediction approach. This method, which utilizes a modified Transformer architecture with multiple independent output heads, has shown to not only enhance the speed of AI models by up to three times but also improve their performance on generative tasks without requiring additional training time or memory. The study reveals that while multi-token prediction may not universally apply to all model types and language tasks, it holds substantial promise for large models, particularly in tasks like code completion, by offering faster inference and higher accuracy with minimal extra costs. The researchers acknowledge that the technique has room for improvement, including optimizing the number of tokens to predict for different tasks and model sizes. This breakthrough could herald a new era of more efficient and powerful AI applications in various industries.
