> Outsourcing the work deprives you of who you become by writing it. Just because AI can do something that resembles work should not mean outsourcing work to it. Mathematicians should not outsource their work to AI just like programmers should not outsource programming to AI. Humans working with AIs in a tight loop means intellectual work becomes more high-level and creative, but a human should always own the work, validate it and stake their reputation to it. Simply ban any humans who produce low quality work using AI.

> This can be said about pretty much any job on earth. That isn't really true. After push button elevators with floor-logic relays eliminated the need for "elevator operator" to be a job, nobody needed to be an elevator operator anymore. The equipment could do 100% of the job and if the equipment was out of order then you call a repair technician or install a new elevator rather than needing to find an elevator operator to pull out of retirement, since knowing how to repair or install elevators was never part of their job to begin with. The trouble with AI-generated code is that it can't do 100% of the job, so you still need a programmer to do the parts that it can't, but then you need the programmer to understand how to do the parts that it can't, which in turn requires them to also understand how to do the parts that it can.

Yes, but we've already painted ourselves into a corner by almost a century of moving all that work onto computers. Why would we want to sever this last thread of human control? What is there to gain from it? I don't think I have to convince anyone how much there is to lose. The situation being created with an overdependence on AI is looking much more like the burning of Alexandria, and less like a utopian dream or even the oft-warned-about authoritarian hellscape. The AI hype is over and revealed to be delusional and politically motivated.

> much more interesting than the problem’s elementary statement might lead one to expect This is reinforced by the immediate (human) use of the idea to resolve in the negative another significant problem, the sum-product conjecture on reals. Explanation of what was involved: https://www.erdosproblems.com/forum/thread/blog:6

I supervise quite a few Masters students. In my particular setting, believe me, LLM stupid for the top three chatbots is easier to work with than real human stupid now. We passed that threshold earlier this year.

I'm not sure the similarities hold. The best comparison I've found is the construction unions in San Francisco. They frequently block housing that is factory built somewhere else in the Bay area even though that would make it possible to build tonnes of new housing in the city at a fraction of the cost. "But Jobs" they scream and hijack the council to block new housing. I'm sorry folks, the point of housing isn't the jobs it creates, the point of housing is housing! New jobs ARE actually created, they are just higher leverage ones in the house factory. Mathematicians are now facing the same... calculation (pun intended). And I think they are empowered to create a lot more leverage, and they shouldn't be afraid of it. A lot of them are catching on to this [1] [1]: https://x.com/OpenAI/status/2060451757818601808

I think the disruptions are temporary. Using AI still takes someone's time and the skill can be honed, which means there will be people specializing in using AI in different ways that are better than what someone just picking it up can do. It's just a reset on skill floors but the people with talent will rapidly regain ground and find their way to the ceiling again. The ones who learn the most, the fastest, will probably end up being worth more than they were previously. I don't think any of these AI layoffs are actually because AI replaced a human. I think a lot of the layoffs are actually just due to a faltering economy or greedy companies trying desperately to get a piece of the pie, so they're sacrificing their long game for short term gambles. I don't think that's going to pay off for them.

That depends entirely on what's meant by "will be fine" and "AI will liberate them". I was assuming you meant something along the lines of gainful employment. If instead you meant performing a meaningful task then I'd counter that most engineers like to build finished products not wallow in minutia. You can still hand roll assembly but I don't think many developers lament the advent of the modern compiler. Instead people build far more complicated systems than would otherwise have been possible.

Sure, but the compiler didn't promote the assembly programmer, it basically ended assembly programming. So which side of that is the engineer on? You're assuming we're the dev who got the compiler. But if the premise is that AI does the actual problem-solving, then we're the assembly and AI is the compiler. > Engineers like to build finished products, not wallow in minutia This only works if what you loved was having built the thing. If what you loved was the building itself, the solving, then "here's a way bigger system, the AI figured it out" isn't a win. It's just a promotion from maker to manager. And a lot of engineers specifically tried to avoid this promotion in their career.

> Much like for many the point of chess is that it's played by humans, with truly superhuman AI relegated to a training aid It's much more than just training. Humans use the engines to prepare openings and find promising novelties. Over time these novelties unearthed by engines fill out theory. It's easy to fine elite games where neither player is out of book for dozens of moves. Modern players are full hybrids in that sense. Looking back at chess, it seems natural that Mathematics will go the same way.

I think there would still be a place for it if it's beyond human comprehension. For instance, really complex lemmas to solve human-tractable problems. If you can pose a question in a proof assistant language like Lean, have an AI write a Lean program that solves it, you can use that as a Lemma for some other problem. There's quite a bit of math out there that is "correct assuming conjecture X is correct", maybe AI could fill that gap and "still be math".

Surely such AI would also be able to reduce and simplify the math for human understanding. Which is what mathematicians do all the time, from turning base 60 cuneiform into modern number systems to simplifying Maxwell's equations for the students.

In general, most researchers already incorporate LLM into their workflows, as it is quite good at context search. However, the relevant training data is based on the collective works of the field of experts. Collecting current data on that work is what makes the LLM sound relevant, and any improvement of the LLM model requires frequent new data from both researchers and the chat bot users themselves. LLM are not real "AI", and anyone that says otherwise is selling people something. To phrase this differently, LLM companies conduct unauthorized targeted intelligence gathering on peoples work, codify that act of plagiarism or theft as MoE documentation, and sell unaccountable token output to other users. There is a reason output becomes more nonsensical as "AI" companies try to use dynamic weight granularity and conceptual compaction. It is not necessarily "AI" hallucinations, but rather people fooling themselves into believing smart people are no longer needed if they willingly become a hapless exploited data source caste. This simply isn't true, as people will leave the field for awhile. The LLM business model regularly requires copyright theft and plagiarism to persist. It will not magically become sentient/AGI/less-stupid, as these algorithms have been operating for over 40 years. What has changed is the scale of the deployment, data pool size, and the energy consumed. Scientists are still necessary, as they create the world models LLM try to guess at by statistical inference. Hype and FUD ahead of an IPO for a highly dubious revenue company is expected. We look forward to the low cost liquidated GPU hardware in the near future. =3

Your model of what AI is good at is wrong. Generative AI is not good at wandering off into novel esoteric abstract corners while maintaining correctness, it is good at things that are close to its training data. I suspect that humans will long outperform AI in the domain of "novel esoteric abstract useless math" whereas AI will outperform humans in the domains of (1) making connections between already-well-understood concepts, things that seem obvious in retrospect but which no human figured out just because of the accidents of what people happened to focus on, and (2) proving things that require long, tedious, intellectually unsatisfying calculations, which would cause a human mathematician to give up for boredom.

My understanding is that we’re talking about “tool-assisted” proof generation, which provides some guard rails but would still allow significant creativity. Tools like Lean, Coq, etc.

I don't get it. LLMs don't have ego, they don't have the ability to say "no, this should be obvious, I'm not going to explain further", they are just token predictors, and given context, they can generate more tokens. If you don't understand how the answer was derived? You just ask more questions and it isn't going to get bored or annoyed, it will just try to answer the questions. Is that what is offending you so much?

> Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. > Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. This feels like a little bit of a jump to me. AIs arent actually alive so of course someone is going to have to pose the question. They arent going to just do stuff on their own. And of course mathmaticians are going to need to interpret the results if we are to glean anything beyong if the conjecture is true or false. But you seem to be suggesting that mathematicians will have to micromanage every step. That seems like a bit of a jump which i dont see much evidence for.

> Have you ever been exposed to concepts that are so complex that you feel like you could devote your entire lifetime to trying to understand it and still fall short? It’s a very humbling experience, especially if you have classmates who pick it up effortlessly. I do have a PhD so I kind of know how that feels. I watched my entire field (PL) get eaten up by AI though, the problems that I thought were huge 10 years ago are just silly footnotes now. > Without a human holding the reins, consider an LLM a rudderless superboat speeding erratically towards the horizon, finding and proving meaningless theorems that not even your most talented classmate could ever begin to understand. I don't disagree with that. LLMs are a tool, a super fast pattern matcher, research, token predictor. I don't expect it to go out and define its own esoteric (or useful) problems to pursue without human interaction. That's for the humans to do. I don't understand what that has to do with my original comment though. I wasn't addressing what problems the LLMs were answering, just how to review and dissect the answers that they would come up with.

> Sorry but I couldn't agree less. > … > True knowledge is, and will be, a human endeavor, deiven by human curiosity. Promoting curiosity is the sign of a developed society. Unless I misunderstand, it sounds like you do agree? My point is that without human mathematicians LLM output is meaningless, and without human mathematicians holding the reins, LLMs would probably quickly devolve into “proving” things that are not only completely unintelligible by humans, but have no utility. Your examples of esoteric mathematical concepts are anecdata. The vast majority of esoteric mathematics does not have utility. Mathematics is an incredibly large space of concepts. Consider the number of provable theorems in number theory alone, perhaps even related to specific subsets and sequences of numbers. The vast majority of the findings in that domain will not be isomorphic to some real world problem, they will be trivia. We will need mathematicians to separate the signal from the noise.

Once you now something is correct, with a proof. It is MUCH easier to understand why it is correct. Than to start from a slate that you don't even know whether something is correct or not. In that sense AI that can just solve high level math problems is immensely useful. It allows a mathematician to explore ideas at a much more rapid pace.

Not quite. You have machines that can aid and expand your (human) understanding greatly, that wasn't possible without machines. Machines don't think. They aren't human. They have no soul, agency/free will, self-reflection/awareness, moral imperatives or ethics. You've been watching too much Terminator, son.

"You wanna know what the best thing about humans is? You invented us! Giving you a chance to take a rest while we invented everything else!" —Wheatley, Portal 2

> We just let them do their thing, essentially treating them as artists and letting them pursue the craft for its own sake. I think we generally did that because that seemed to be the best known process for maximizing the quantity of useful mathematics that they occasionally stumble upon. It's not like we treat math as a charity project for eccentrics who like blackboards. What we want is new mathematical discoveries that have a huge positive impact on other areas of the world. It's just that math and/or human brains are such that seemingly the best way to find those discoveries was to let mathematicians wander around randomly in mindspace. If a more guided structured process produced more results, we'd probably do that. But it doesn't seem to, so we don't. I don't think anyone knows yet what the best process for producing useful mathematics with humans + AIs looks like.

Even from the most purely instrumental perspective, what we care about is our ability to make use of correct answers, which is quite distinct from the possession of correct answers. There are many theorems that aren't directly interesting, but whose proof requires techniques that are of substantial further interest, that lead to new domains, and/or new practical applications. Simply being handed a proof for those theorems isn't enough--we require the ability to apply those techniques in the real world, or discover further areas of mathematical research that build on that proof or its techniques. It may be that AI can build on its own work for the long-term, but so far, AI does best at exploration in areas that have precisely specified and measurable goals. Actually creating understanding, and making use of mathemtical results outside of pure mathematics is more challenging than simply creating proofs. I think the field will figure out how to make use of AI, and it will be better off for it. But that is not the same as just saying "answers good, grog want more answers."

One of the reasons why over a decade ago, I dived deeply into the OSS world instead of mathematics was that it was so much more accessible: there were docs for everything, and I got direct feedback when something worked vs when something didn't work. Most of my questions had answers on stack overflow, and once I joined Rust (which back then in 2015 didn't have a big stackoverflow presence) I had a community who answered them for me (and in maths I didn't have that). AI makes the math world more accessible than before. If you have a question about a proof in the lecture, you can just ask it. Of course, one can't trust it blindly, but fundamentally it's amazing. I think that's a good thing, but of course this means that a lot has to change in culture and behaviors, also in the research world. The software engineering world is more or less in the same situation, it's also changing. But for now I think it still holds true that someone who knows maths plus an LLM is better than someone who doesn't know maths plus LLM. At least in software it does.

Yeah, among other factors, that "figure it out" mentality put me off in the end. Especially because often you need to show the same mentality unless you want to overkill proofs and spend more time on them than assigned to you. I sometimes miscalibrated and pointed out some details that didn't need pointing out in my proofs while in other proofs, I skipped over too many details for the TA. Of course I agree that if the student just asks LLM to do their homework, they have not learned anything. But it's sad if one can't ask questions about a proof or such. Having the LLM around to review the homework submission is also useful, to make sure that the arguments are solid.

You can ask the LLM for a hint as well.

My vague prediction right now is that in five years LLMs will be heavily used by universities in grant-funded math research but nobody else will be able to afford it, much like supercomputer clusters 25 years ago.

I see any kind automation as a good thing as long as it is reliable enough. We stopped copying books manually a long time ago, and the craft was lost, most of us can't do complex calculations manually etc. but it does not matter as long as we can rely on calculators and computers to do it. At this stage, the current wave of AI is not reliable enough that it would be safe to lose the abilities it can replace. The failures modes are often turned into memes and jokes, but they are the thing we should really pay attention to, IMO.

Right, most professional mathematicians know almost nothing about their neighboring branches of mathematics. An algebraic geometry researcher would be hard pressed to understand a new result from category theory or even something closer like commutative algebra.

As a mathematician by trade I think they’re overblowing it. You can choose to use it or not. I choose not to because I enjoy the process. But I’m not doing formal research or getting paid to do it these days. I will note that the average corporate mathematical modelling is usually a fucking circus so adding AI might make it better.

> You can choose to use it or not This is becoming less and less true unless you're specifically talking about usage of it outside of a work environment. Many work places are requiring people to use it and/or tracking usage. I don't know about in academic settings, but I'd imagine it's becoming heavily used there too?

My academic connections that I keep in touch with never really left the 1990s. And no one is pushing them on AI.

Sure, but their peers (who do use AI) will out-publish them soon enough and solve the open problems before they do.

The choice only remains if using it isn’t a huge multiplier. If it is a huge multiplier/accelerator, then for a while it will be ambiguous and the choice will remain. But as time goes on, the gains of using it will be so apparent and the advantage of the people who use it so great (in publication numbers, hiring, etc) that it will force others to. I don’t say that with any particular relish. But I am skeptical of the choice angle past a certain point.

I don't think all universities or research agencies are particularly pressed on this. I mean my daughter is a notable researcher in a scientific field and they have absolutely no pressure to use AI to pump out papers or deliver value quickly.

Mathematician Ken Ono ( https://en.wikipedia.org/wiki/Ken_Ono ) gives a well nuanced viewpoint on AI in Mathematics (and more) - https://www.youtube.com/watch?v=jGZOi-7haCw He states that he struggled to come up with problems which would be challenging for AI to solve (at the below site) and thus forced to accept that mathematicians have to rethink their profession. FrontierMath: Benchmarking AI against advanced mathematical research by Epoch AI - https://epoch.ai/frontiermath As a follow up to the above, see "First Proof: Mathematicians Putting AI to the Test" featuring eminent mathematicians - https://www.youtube.com/watch?v=AaICCTpkI7Q

> This is all contingent on AI forays into mathematics being slop and low quality It's literally a set of recommendations for researchers on how to use AI to advance the field and prevent slop from overwhelming the people who might do anything with the research produced. For people who are so eager to declare that everyone else is just having an existential crisis because "your culture is commodified", AI people are getting awfully defensive about this document.

AI is the interpolation of the human corpus. Is it suprising AI recombines sucessfully where human attention has not explored all plausible solutions? N,o not especially. The key fallacy is that AI is other than human. This is really no different from computer proofs, e.g. 4 color theorem. The fact the prompt is not linked to the solution by individual human intention alone does not make the solution less human in origin.

Actual "warning": https://leidendeclaration.ai/ Far more interesting as it's outlaying a set of principles for using AI to augment human involvement and science, rather than replacement.

Are maths AI models now using "tools", aka formal solvers? I understand that the "language interface" of a "maths AI" could be some specialized trained LLM (Large Language Model) that to convey, with human language, "high level" mathematical mental contructs and intuition. But then, you would need some models which does the reasoning using formal mathematical solvers (and probably a ton of "scratch" memory, it would be interesting to see how those models end up storing "mathematical" lema data). I guess you can have ML (Machine Learning) for those models on 'general maths', but also we can think about more mathematically focused ML for a specific problem, area, etc. And in the end, ML for maths, would it be mostly permutations of truth statements fed to a neural net? When we were talking about "AI", one decade ago, that was what most had in mind (it may help a bit in physics, but it seems less likely, because reality/experiments are hard to teach to "AI"s). If that becomes a reality (aka easy hardware access, and some "working" models), mathematicians will have to be as good in maths than in maths ML. And this is were there is an issue: training honestely good mathematical human brains may become very hard with some broad availability of good general maths reasoning "AIs".

Two things that I would recommend trying out if you're interested in exploring this further: 1. If you're not paying for a model, the results will be worse. That sucks but the free access models are just not very good for anything where you need to trust the output, even for basic queries. 2. More important than #1 is access to tool use. If the LLM is just producing a nutritional breakdown from its weights, it's almost always going to be wrong. If the LLM is allowed to break the problem down into deterministic steps, it will do a lot better. In the nutritional breakdown case, an LLM with search + tool access can pretty easily break the problem down: - Searching the web for a recipe or ingredient breakdown for the food - Searching the web for nutritional qualities of each ingredient per some volume of the ingredient - Writing and running a script with e.g. Python that takes in the recipe's projected serving output, the desired serving size, the amount of each ingredient etc, and scales the ingredients to match the desired serving size, and sums the nutritional qualities of the scaled ingredients. I've tried this specific case with Claude + Gemini for my own purposes and they both handle it very well. The challenge currently is that the models will not always arrive at this approach when provided with an ambiguous prompt; sometimes they will, but sometimes they'll just vomit up a fully autocompleted response from their weights. Being more specific in the prompt or defining a skill that details the intended approach lets you get more useful + deterministic results while still taking advantage of the fuzzy glue that LLMs can provide here between steps. Same with the classic strawberry r-counting case. IIUC LLMs have trouble with this because of how training data is tokenized, but any LLM will have no trouble farming out to e.g. > echo -n "strawberry" | grep -o "r" | wc -l > 3

There are basically two kinds of applications. One is where you want to correctly solve the problem at least 99 out of 100 times. LLMs generally don't (and not everybody realizes that) so there are a lot of debates and research around how useful and reliable they are or how to make them so. The other kind of application is where you can try 100 times and you only need to be right once. Solving a mathematical research problem is like that.