GPT-4: Stanford and MILA’s Hyena Hierarchy technology offers a more efficient alternative to large language models like GPT-4, achieving similar accuracy in AI tasks with up to 100 times less compute power. By replacing the “attention” mechanism with a sub-quadratic approach called Hyena, the technology can handle vast amounts of text without running out of memory. This breakthrough could lead to new possibilities in deep learning, such as processing entire textbooks as context or generating long-form music.
Read more at ZDNET…
