Opinion

The data black hole at the heart of AI

Original title:The data black hole at the center of AI

AI summary

One definition of intelligence is sample efficiency, yet recent AI progress has come mainly from expanding data distributions and adding compute. Reinforcement learning is essentially synthetic data generation — spending large amounts of compute to filter good data through verifiers, then training models to predict correct outputs. This process requires massive human expert examples for every domain and skill, and the data industry already generates billions of dollars in annual revenue. Epoch recently reported that open-source models lag frontier closed models by only four months, because data can be distilled from public APIs while hyperparameters and other elements are hard to copy. Humans encounter about 200 million tokens in a lifetime, while frontier model training spans tens to hundreds of trillions of tokens — a gap of nearly a million times; robotics and autonomous driving show similarly large efficiency gaps.

Key Takeaways

  • One definition of intelligence is sample efficiency,yet recent AI progress has come mainly from…
  • Reinforcement learning is essentially synthetic data generation — spending large amounts of…
  • This process requires massive human expert examples for every domain and skill,and the data…
  • Epoch recently reported that open-source models lag frontier closed models by only four…
  • Humans encounter about 200 million tokens in a lifetime,while frontier model training spans tens…

Quick Facts

Published
Updated
Source
Dwarkesh Patel:Podcast & Blog(RSS)
Category
Opinion
AI score
60
Read the original Read the full article at the original source
Why it matters

Dwarkesh 把 AI 的‘笨’归因于数据效率远低于人类,计算虽简但直指要害,他给出的一个逆向洞察是开源模型四个月追上闭源,正是数据驱动进步的最好证据。

Source & copyright

This article is based on curated titles and AI summaries from the AIHOT public API, with attribution and links back. Original copyright remains with the original publishers and authors; this site does not mirror full texts.

Original article · AIHOT entry

How to Cite

When citing this article, keep the title, date, and permalink, and credit Orange News as the source.

Orange News: The data black hole at the heart of AI. June 20, 2026. https://new.nat6.net/en/news/cmql6vhtu01n0slluxpq3yvzh/

View all →