Toggle navigation About Marginal Revolution Categories Date Archives Our Books Our Textbook: Modern Principles of Economics Marginal Revolution University Search Search Thank-you! You've been successfully added to the Marginal Revolution email subscription list. LLMs are Creative Reasoners by Alex Tabarrok September 10, 2024 at 7:18 am in Economics Philosophy Science It’s bizarre to me that there are still people claiming that LLMs are not reasoning or are not creative when by any objective measure they are obviously creative reasoners! By objective measure I mean a test that evaluates creativity and reasoning by evaluating outputs not by idle philosophical speculation that rules AIs out by definition. Here’s a good paper, Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers , which illustrates one such test. The authors asked top researchers in the field of natural language processing to propose research ideas which were then presented in a standardized format to a ratings panel of other NLP experts. The AI created ideas were judged more creative than the human ideas. Now one might argue that the humans weren’t giving their best ideas–some data in the paper suggests they were giving ideas at the median of those for top researchers–and humans might also be looking for ideas that were perhaps easier to get funding precisely because they were less creative but more doable. Either way, however, the AIs are coming up with good ideas that could usefully supplement human generated ideas. 44 Comments Facebook Twitter RSS Feed print Comments Sort by Top Sort by Recent Sort by Controversial Comments for this post are closed Marginal Revolution University See Courses Learn more about Mercatus Center Fellowships Learn More Subscribe via Email Enter your email address to subscribe to updates. Email Address Subscribe RSS Feed Contact Us Alex Tabarrok Email Alex Follow @atabarrok Tyler Cowen Email Tyler Follow @tylercowen Webmaster Report an issue Blogs We Like Interesting People & Sites Our Web Pages Alex Tabarrok's Home Page Alex's TED talk, how ideas trump crises Conversations with Tyler FDAReview.org Tyler Cowen's Personal Web Page Tyler's ethnic dining guide Apply to Emergent Ventures Books Modern Principles of Economics Tyler Cowen & Alexander Tabarrok Marginal Revolution 2026 About Marginal Revolution Categories Date Archives Our Books Our Textbook: Modern Principles of Economics Marginal Revolution University Facebook Twitter RSS Feed Marginal Revolution University See Courses Search Search Privacy Policy Marginal Revolution 2026