Google’s MEENA model, once the best large language model, predicted the integration of language models into everyday life. Despite this, Google’s potential was overlooked until the release of OpenAI’s GPT-3. Now, Google is set to outpace GPT-4’s pre-training FLOPS by 5x by year-end, with a clear path to 20x by next year. The article also discusses the GPU-poor environment, the struggle of startups and open-source researchers, and the potential of Google’s Gemini and Viperfish models to compete with other frontier labs.
