AI summary: OpenAI’s GPT-4 model architecture is not a secret, but a replicable solution with complex tradeoffs. The real challenge lies in scaling AI, particularly in inference, which exceeds training costs. OpenAI’s innovation targets this issue, aiming to decouple training compute from inference compute. The article discusses the cost of training and inference for GPT-4, the architectural decisions, and the issues with scaling AI. It also predicts that tech giants like Google, Meta, and others will soon have models as capable as GPT-4, marking a new era in the AI space race.
