Google Research and DeepMind have innovated a method to enhance large language models (LLMs) by integrating them with other specialized language models, potentially revolutionizing the AI field. This technique allows existing models to acquire new capabilities without the need for extensive retraining, which is both time-consuming and expensive. The research utilized Google’s PaLM2-S LLM, comparable to OpenAI’s GPT-4, and demonstrated significant performance boosts in tasks such as translation and coding when augmented with smaller, task-specific models.
For instance, the augmented model exhibited a 13% improvement in translation tasks, particularly when translating less-supported languages into English, a notable challenge in machine learning. In coding tasks, the hybrid model achieved a 40% improvement over the base model, matching the performance of fully fine-tuned models.
This advancement could have profound implications, especially considering the legal challenges facing AI companies over the use of copyrighted data in training their models. If courts rule against the use of such data, the current approach to building LLMs may become unsustainable. However, Google’s method of augmenting LLMs could reduce the scaling requirements and costs associated with developing new models or retraining existing ones, offering a potential solution to this looming issue.
As AI regulation becomes a focus, with countries like Italy prioritizing it during their G7 presidency, Google’s breakthrough could be a timely solution for the industry, ensuring the viability of AI services in a potentially more regulated future.
Read more at Cointelegraph…