Sobering Up on AI Progress w/ Dr. Sean McGregor

Sean McGregor and I discuss about why evaluating AI systems has become so difficult; we cover everything from the breakdown of benchmarking, how incentives shape safety work, and what approaches like BenchRisk (his recent paper at NeurIPS) and AI auditing aim to fix as systems move into the real world. We also talk about his history and journey in AI safety, including his PhD on ML for public policy, how he started the AI Incident Database, and what he's working on now: AVERI, a non-profit for frontier model auditing. Chapters (00:00) - Intro (02:36) - What's broken about benchmarking (03:41) - Sean’s wild PhD (14:28) - The phantom internship (19:25) - Sean's journey (22:25) - Market-vs-regulatory modes and AIID (32:13) - Drunk on AI progress (38:34) - BenchRisk (43:20) - Moral hazards and Master Hand (50:34) - Liability, Section 230, and open source (59:20) - AVERI (01:11:30) - Closing thoughts & outro Links Sean McGregor's website AVERI website BenchRisk BenchRisk website NeurIPS paper - Risk Management for Mitigating Benchmark Failure Modes: BenchRisk NeurIPS paper - AI and the Everything in the Whole Wide World Benchmark AIID AI Incident Database website IAAI paper - Preventing Repeated Real World AI Failures by Cataloging Incidents: The AI Incident Database Preprint - Lessons for Editors of AI Incidents from the AI Incident Database AIAAIC website (another incident tracker) Hot AI Summer CACM article - A Few Useful Things to Know About Machine Learning CACM article - How the AI Boom Went Bust Undergraduate Thesis - Analyzing the Prospect of an Approaching AI Winter Tech Genies article - AI History: The First Summer and Winter of AI CACM article - There Was No ‘First AI Winter’ Measuring Generalization Neural Computation article - The Lack of A Priori Distinctions Between Learning Algorithms ICLR paper - Understanding deep learning requires rethinking generalization ICML paper - Model-agnostic Measure of Generalization Difficulty Radiology Artificial Intelligence article - Generalizability of Machine Learning Models: Quantitative Evaluation of Three Methodological Pitfalls Preprint - Quantifying Generalization Complexity for Large Language Models Insurers Exclude AI Financial Times article - Insurers ret