Navigated to The Startup Powering The Data Behind AGI Pocket Casts Plus Podcasts Playlists Discover New Releases In Progress Starred Bookmarks History Create account Sign in Pocket Casts Plus Get the app Create account Log in Gradient Dissent: Conversations on AI The Startup Powering The Data Behind AGI Play episode Sep 16, 2025 56 mins Bookmarks View Transcript Episode Description In this episode of Gradient Dissent, Lukas Biewald talks with the CEO & founder of Surge AI, the billion-dollar company quietly powering the next generation of frontier LLMs. They discuss Surge's origin story, why traditional data labeling is broken, and how their research-focused approach is reshaping how models are trained. You’ll hear why inter-annotator agreement fails in high-complexity tasks like poetry and math, why synthetic data is often overrated, and how Surge builds rich RL environments to stress-test agentic reasoning. They also go deep on what kinds of data will be critical to future progress in AI—from scientific discovery to multimodal reasoning and personalized alignment. It’s a rare, behind-the-scenes look into the world of high-quality data generation at scale—straight from the team most frontier labs trust to get it right. Timestamps: 00:00 – Intro: Who is Edwin Chen? 03:40 – The problem with early data labeling systems 06:20 – Search ranking, clickbait, and product principles 10:05 – Why Surge focused on high-skill, high-quality labeling 13:50 – From Craigslist workers to a billion-dollar business 16:40 – Scaling without funding and avoiding Silicon Valley status games 21:15 – Why most human data platforms lack real tech 25:05 – Detecting cheaters, liars, and low-quality labelers 28:30 – Why inter-annotator agreement is a flawed metric 32:15 – What makes a great poem? Not checkboxes 36:40 – Measuring subjective quality rigorously 40:00 – What types of data are becoming more important 44:15 – Scientific collaboration and frontier research data 47:00 – Multimodal data, Argentinian coding, and hyper-specificity 50:10 – What's wrong with LMSYS and benchmark hacking 53:20 – Personalization and taste in model behavior 56:00 – Synthetic data vs. high-quality human data Follow Weights & Biases: https://twitter.com/weights_biases https://www.linkedin.com/company/wandb See all episodes Never lose your place, on any device Create a free account to sync, back up, and get personal recommendations. Create account Chapters 0 Chapters Up Next Nothing in Up Next You can queue episodes to play next using the dot menu on an episode row. Press spacebar to reorder.