[State of RL/Reasoning] IMO/IOI Gold, OpenAI o3/GPT-5, and Cursor Composer — Ashvin Nair, Cursor

Ashvin discusses his transition from robotics research at Berkeley and OpenAI (2017) to language models, explaining why robotics researchers make good LLM practitioners. He argues robotics is still in the 'GPT-1 to GPT-2 era' despite recent demos from Physical Intelligence and Sunday, and believes LLM agents will be a trillion-dollar market before robotics reaches $10B.

• •Robotics researchers are 'most grounded' at NeurIPS because they work with real-world constraints and look at data constantly
• •Robotics feels furthest from AGI - current state comparable to GPT-1/GPT-2 era despite recent impressive demos
• •Market opportunity: LLM agents likely trillion-dollar market before robotics becomes $10B market
• •Key inflection point for robotics: generalization to new scenarios and objects (OOD performance), similar to what GPT-2 showed

Ashvin joined OpenAI in September 2022 on the Codegen/Codex team, right before ChatGPT launched. The team was working on tool use and making models smarter at programming competitions. He reveals that achieving IOI Gold felt completely unreachable at the time - 'we could all just go on vacation, AI is solved' - yet life hasn't changed much despite reaching that milestone.

• •Joined OpenAI expecting a 'chill research lab' right before ChatGPT explosion changed everything
• •Worked on Codex team focusing on tool use, SAT problems, and programming competitions
• •IOI Gold achievement would have seemed like 'AI is solved' in 2022, yet economic impact remains limited
• •Community keeps moving goalposts and goodharting benchmarks - models spike on specific tasks like IOI while struggling with general programming jobs

Deep dive into why the 2017-2022 era of RL research (DQN, off-policy learning, value functions) didn't pan out despite academic excitement. Ashvin explains how the community implicitly overfit to benchmarks by introducing new tunable knobs, and why academia rewards complex mathematical ideas over simple ones that actually work at scale.

• •RL research from 2017-2022 era largely unused today - community overfit to benchmarks at meta level
• •Academia rewards complex math/theory over simple ideas that generalize, creating implicit tuning knobs
• •Off-policy RL methods hyped during this period haven't delivered practical value
• •Scaling era rewards simple approaches with fewer knobs that generalize broadly - less intellectually interesting but more effective

Ashvin reveals how OpenAI's reasoning models emerged from long-term conviction in RL dating back to Dota (2017), driven by Ilya Sutskever and Jakob Pachocki. The breakthrough came in 2023 when pretrained models became good enough, with early prototypes showing surprisingly accurate reasoning traces on small models that convinced leadership to scale up massively.

• •OpenAI had conviction in RL path to AGI since Dota (2017) - just needed pretrained models to be good enough
• •2023 breakthrough: small models with RL produced 'very interesting reasoning traces' and surprisingly good math scores
• •Internal progress feels smooth and continuous, not the wild swings portrayed externally in media
• •Reasoning team grew from ~dozen people to 300+ as it became productized (o1 to o3)
• •By early 2024, it was 'pretty predictable' they could smash IMO/IOI benchmarks

Critical insight on RL's limitations: it's a 'weird, funny tool' that doesn't generalize beyond training distribution well. Models can completely dominate in-distribution tasks (like IOI) but struggle with general programming jobs. The solution isn't smarter models but bringing economically useful tasks into the training distribution through product design.

• •RL can achieve best-in-world performance on training distribution with 'not that much effort' but generalizes poorly
• •Intelligence isn't the bottleneck - need to bring economically useful tasks in-distribution for RL
• •Products must capture entire context of what someone wants to do (code base, terminal, conversations, graphs)
• •Example: hyperparameter research required year of accumulated knowledge from graphs and sweeps - models need this context to automate the job
• •Co-design product and model together rather than expecting one-model-fits-all approach

Ashvin's perspective on the Sam Altman firing/rehiring crisis. He signed the letter to reinstate Sam but was genuinely conflicted about governance structures. Questions whether nonprofit boards or traditional corporate boards (representing pension funds) are better for AGI governance, noting we haven't solved governance even for unhealthy food or social media.

• •Signed letter to reinstate Sam but willing to 'forget about equity' to have serious governance conversation
• •Nonprofit board's 'erratic actions' made him question if Microsoft-style board (representing pensions) might be more democratic
• •Governance feels really important regardless of 2-year or 10-year AGI timeline
• •Society hasn't solved governance for simpler cases like unhealthy food or social media - AI governance remains unsolved

Why Ashvin left OpenAI's infinite resources for Cursor's 20-25 person ML team. Cursor can co-design products with models in ways impossible at larger orgs, exemplified by online tab with policy updates every 2 hours. The key advantage is bringing the entire software engineering process (not just coding) into the product for RL training.

• •Cursor's small size enables product/ML co-design impossible at OpenAI - policy updates every 2 hours for tab autocomplete
• •Goal: automate software engineering as a process (Datadog, hypotheses, reruns) not just code writing
• •Composer's success comes from being both smart enough to use AND fast enough to stay in the loop
• •All cofounders are high-taste coders who test products themselves, ensuring quality
• •20-25 person ML team with excellent internal tooling (SSH into user environments, close to data)

Discussion of continual learning as potentially paradigm-shifting in the next year. The key insight: models train on trillions of tokens but should be able to learn from millions of deployment tokens without forgetting. Explores the 'hard drive vs CPU' view of neural networks and whether weights are for memorization or computation.

• •Models train on trillions of tokens, so millions of deployment tokens shouldn't overload capacity
• •Humans don't need to filter out bad data - they have value functions that prevent repeating mistakes (touching hot stoves)
• •Models are 'orders of magnitude' less data efficient - will repeat same bugs even within same context
• •Need theoretical understanding of weight capacity: treating weights as hard drive vs CPU (memorization vs computation)
• •Academia could make progress here but lacks compute; industry has compute but prioritizes empirical short-term gains