Confronting the Intelligence Curse, w/ Luke Drago of Workshop Labs, from the FLI Podcast

Luke introduces the intelligence curse concept - when AI becomes the dominant factor of production, incentives shift away from investing in people toward investing in AI systems. He draws parallels to the resource curse in oil-rich states where governments prioritize extractive resources over human capital, leading to reduced quality of life and political power for citizens.

• •Intelligence curse occurs when nonhuman factors of production dominate, reducing incentives to invest in people
• •Similar to resource curse: oil-rich states often invest in extraction over human development, weakening citizens' bargaining power
• •Unlike previous automation, AI targets humanity's core advantage: thinking and cognitive work
• •Economic value is fundamentally tied to political power - losing economic relevance threatens democratic participation

Discussion of why AI represents a categorical shift from industrial revolution-era automation. Previous technologies augmented human capabilities without replacing our fundamental advantage (cognition), while frontier AI labs explicitly aim to automate 'most economically valuable human work,' creating direct competition with human labor.

• •Industrial revolution automated physical labor but preserved human cognitive advantage
• •Current AI goal is to do 'everything any human can do better, faster, and cheaper' - fundamentally different target
• •Companies will naturally choose AI that saves 50% on wages while delivering better results
• •This creates technology designed to replace rather than augment human capabilities

Luke explains the 'pyramid replacement' model for white-collar job automation, where AI first replaces entry-level workers, then progressively moves up the organizational hierarchy. Recent empirical evidence shows declining job postings for 22-25 year olds in automatable fields like software engineering, validating this bottom-up replacement pattern.

• •Entry-level positions are easiest to automate first - firms can hire fewer analysts/junior employees
• •AI capabilities will progressively move up the pyramid as models become more agentic
• •Empirical data (2022-2025) shows shrinking employment in 22-25 age bracket for automatable fields
• •Blue-collar work may see more sudden zero-to-one automation when robotics capabilities arrive

Key metrics to monitor for intelligence curse onset include income inequality, economic mobility, and youth unemployment rates. Luke emphasizes watching for sudden capital accumulation where investment directly converts to output without human intermediaries, and declining pathways for upward mobility.

• •Watch for sudden capital compounding as AI removes talent from the equation
• •Economic mobility is critical - can people still move upward from nothing?
• •Rising unemployment especially among workforce entrants (early career workers)
• •Look for correlation between automatable tasks and employment changes in those sectors

Workshop Labs' core thesis: the bottleneck to AI progress is high-quality data on tacit knowledge and local information. Luke argues individuals currently possess valuable data about their skills and local context that labs desperately want. The strategy is to help users leverage this data privately for their own benefit rather than surrendering it to train replacement systems.

• •Tacit knowledge (hard-to-digitize skills) and local information are the new competitive moats
• •Labs are rushing to capture this data (browser integration, expert RL environments)
• •Workshop Labs aims to let users train private models on their own data without surrendering it
• •Data is 'the new social security number' - protecting it prevents economic automation

Luke paints a concrete picture of intelligence curse failure: 2030 college graduates unable to find entry-level jobs, strained social safety nets as income tax base shrinks while corporations post record profits, increasing social unrest, and eventual institutional instability creating conditions for authoritarian takeover or democratic collapse.

• •Entry-level jobs disappear across sectors, unemployment strains social programs
• •50% of US federal revenue comes from income tax - automation shrinks this base dramatically
• •Corporate profits soar while tax base erodes, creating fiscal crisis
• •Social unrest + institutional weakness creates conditions for coups or authoritarian power grabs

Luke advocates for open-source AI as essential to preventing monopolistic control over intelligence. He challenges the narrative that open weights models will always lag behind, pointing to Chinese models being only 6 months behind frontier and sometimes leading in specific capabilities. Commodifying the intelligence layer prevents excessive rent extraction.

• •Open weights models are critical to preventing intelligence monopolies
• •Chinese open models are ~6 months behind frontier, some leading in specific areas (e.g., Kimi K2 for English writing)
• •The 'open weights will always lag' argument is empirically false
• •Commodifying intelligence is necessary (though not sufficient) to break the curse

Luke argues for 'defensive acceleration' - investing in safety research that makes open weights models tamper-resistant. He cites Kyle O'Brien's work on removing dangerous capabilities during pretraining to create models resistant to later fine-tuning attacks. This enables safe open-source AI rather than forcing centralized control.

• •Defensive acceleration: derisk AI fundamentally so it doesn't require centralized control
• •Kyle O'Brien's research shows removing bio-materials data in pretraining creates tamper-resistant models
• •Catastrophic risk arguments are used to justify centralization - solving safety enables decentralization
• •Technical safety research is more tractable than hoping for warning shots to slow development

Workshop Labs uses encrypted trusted execution environments (NVIDIA Secure Enclaves) to guarantee user data cannot be extracted or used for training. Luke endorses Sam Altman's 'AI privilege' concept - information shared with AI assistants should have legal protections similar to attorney-client privilege, as these systems have unprecedented access to personal information.

• •Encrypted data in transit, decrypted only in trusted execution environments with code attestation
• •Model weights stored encrypted - technically impossible for Workshop to train on user data
• •Endorses 'AI privilege' - legal protections for information shared with AI assistants
• •Privacy guarantees must be technical, not just promises - users shouldn't have to trust the company

Luke urges young people, especially those on traditional prestige tracks, to take moonshot risks now. Entry-level positions at Fortune 500 companies and consulting firms are the first automation targets. N-of-1 specialized roles at smaller companies offer more security than generic positions at large firms, even if they seem riskier by conventional standards.

• •Default prestige paths (consulting, big tech entry-level) are closing - these are prime automation targets
• •N-of-1 roles where you're irreplaceable are safer than generic positions with 1000 peers
• •Now is the time for moonshots - window is still open but closing
• •More specialized, local economy emerging - generic corporate jobs are 'last chopper out of Saigon'