An audio version of my blog post, Thoughts on AI progress (Dec 2025)

Examines the logical tension between believing AGI is imminent while investing heavily in baking skills into models through RL. If models were truly human-like learners, the current approach of pre-training on every possible task (web browsers, Excel, robotics) would be unnecessary—humans don't need rehearsal for every tool they'll use. This contradiction is most visible in robotics, where true human-like learning would make the hardware problem trivial.

• •Labs spending billions on PhDs/MDs to create training data for specific capabilities reveals models lack general learning ability
• •Robotics as algorithms problem: humans can teleoperate current hardware with minimal training, but AI requires millions of examples per task
• •The 'automated researcher will solve AGI' argument is circular—expecting superhuman research capabilities without basic child-like learning abilities
• •Lab behavior (pre-baking consultant skills like PowerPoint) indicates they expect poor generalization, contradicting AGI-is-near claims

Uses a concrete example of a biologist identifying macrophages in slides to illustrate why current AI can't replace knowledge workers. While image classification is a 'solved' deep learning problem, the economic value isn't in building custom training pipelines for every lab-specific microtask. Human workers are valuable because they learn context-specific skills on the job without requiring bespoke training loops for each new situation.

• •Real work involves 100+ daily tasks requiring judgment and context learned on the job, varying person-to-person and day-to-day
• •Building custom training pipelines for each company/context-specific task (like one lab's slide preparation method) is not economically viable
• •Cannot automate even a single complete job by pre-baking skills—need semantic feedback and self-directed learning like humans
• •True AGI (billions of human-like intelligences that can copy/merge learnings) will be far more transformative than current regime suggests

Directly challenges the argument that slow AI adoption is due to normal technology diffusion lag. Argues that if models had human-level capabilities, they would diffuse much faster than human employees because they can instantly read entire company Slack/Drive, distill skills from other AI instances, and avoid the 'lemons market' problem of hiring. The fact that labs aren't earning trillions (knowledge workers' cumulative wages) proves capability gaps, not diffusion issues.

• •AI employees would be easier to integrate than humans: instant onboarding via Slack/Drive reading, immediate skill transfer between instances
• •No lemons market problem: vetted AI models eliminate costly bad hire risk that plagues human recruitment
• •Knowledge workers earn tens of trillions annually—labs being orders of magnitude below this reveals capability gaps, not adoption friction
• •Companies hire humans constantly; if AI had equivalent capabilities, token spending would already be in trillions

Defends the practice of updating AGI definitions as models improve but fail to deliver expected economic impact. While AI bulls correctly criticize bears for moving goalposts on reasoning/common sense, it's rational to revise definitions when models that would have seemed AGI-level in 2020 (like Gemini) still don't automate half of knowledge work. Predicts this pattern will continue: by 2030, models will have continual learning and earn hundreds of billions, but still won't have automated all knowledge work.

• •Gemini shown in 2020 would have seemed sufficient for automating half of knowledge work, yet clearly isn't
• •Repeatedly solving 'sufficient bottlenecks' (understanding, few-shot learning, reasoning) without achieving AGI reveals intelligence is more complex than assumed
• •Revenue gap (not trillions) is objective evidence that previous AGI definitions were too narrow
• •Expected pattern: models get more impressive at short-timeline rate but more useful at long-timeline rate

Challenges the assumption that reinforcement learning will scale as predictably as pre-training. While pre-training had clean, general trends across multiple orders of magnitude (though on a weak power law), there are no well-established public scaling laws for RL from verifiable reward. Cites Toby Borg's analysis suggesting a million-x scale-up in RL compute is needed for gains equivalent to a single GPT generation leap.

• •Pre-training scaling was predictable 'almost as a physical law' but on a power law (as weak as exponential is strong)
• •No publicly known scaling trends for reinforcement learning from verifiable reward to justify bullish predictions
• •Toby Borg's analysis of O-series benchmarks suggests ~1M x RL compute needed for one GPT-level improvement
• •Laundering pre-training scaling prestige to justify RL predictions is methodologically flawed

Proposes an alternative to recursive self-improvement singularity scenarios: a distributed intelligence explosion driven by continual learning agents deployed across jobs, bringing learnings back to a hive mind for batch distillation. Argues continual learning will progress like in-context learning did—gradual improvement over 5-10 years rather than sudden breakthrough. Predicts fierce competition between labs will prevent runaway advantages, as talent poaching, SF rumor mill, and reverse engineering have neutralized previous supposed flywheels.

• •Future model: specialized agents doing jobs, returning learnings to hive mind for batch distillation (Karpathy's 'cognitive core' + job-specific knowledge)
• •Continual learning will be gradual like in-context learning (GPT-3 showed promise in 2020, still improving today), not one-and-done achievement
• •First continual learning release (likely next year) won't be game-over; human-level on-the-job learning may take 5-10 more years
• •Competition stays fierce: big three rotate monthly on leaderboards, previous flywheels (user engagement, synthetic data) haven't created runaway advantages
• •Talent poaching, SF information sharing, and reverse engineering neutralize single-lab advantages