In 2020 OpenAI published a paper on AI Scaling Laws which was supposed to show that the diminishing returns from training models on data can be alleviated by increasing model size at the expense of requiring more power and total compute time or in laymens terms “if the result is a 75% chance of being a thing, 4 attempts are more likely to guess it than one attempt”.
What it unintentionally showed is that the returns from scaling models at optimal compute time each also diminishes.
Then DeepMind corrected their math in a followup study, illustrating that INFINITE compute time and data will never result in above 94% accuracy.
The AI companies know. They all know. They have always known.
I think they don’t know yet.
Guys - recommend z.ai, kimi and deepseek and cancel openai /anthropic /gemini
In 2020 OpenAI published a paper on AI Scaling Laws which was supposed to show that the diminishing returns from training models on data can be alleviated by increasing model size at the expense of requiring more power and total compute time or in laymens terms “if the result is a 75% chance of being a thing, 4 attempts are more likely to guess it than one attempt”.
What it unintentionally showed is that the returns from scaling models at optimal compute time each also diminishes.
Then DeepMind corrected their math in a followup study, illustrating that INFINITE compute time and data will never result in above 94% accuracy.
The AI companies know. They all know. They have always known.