>> At least for the foreseeable future you still would like people to become interested and develop skills in these fields. These developments, and especially how they are presented, directly discourage that. This assumption may well turn out to be correct, but it is not self-evident. Nearly everyone who has ever got interested in mathematics got discouraged at some point and they left the field. Mathematics is very hard. Those very few that remained certainly have talent, but they also have characteristics that are necessary for success in a competitive field, which are perhaps less valuable per se. Such characteristics as may be over-represented in males for instance. This is not a point about gender differences, but about the intrinsic merit of different success factors. It seems equally possible that the above assumption will turn out to be diametrically incorrect. People that would have been discouraged before LLMs will now retain their curiosity longer. Democratisation is surely a possible outcome. Arguably, chess has never been as popular and accessible. And that discipline fell to AI three decades ago.

Accelerationists may argue that the eroding of proper attribution and proof verification by humans is a meaningless short term struggle of a dying field. Mathematics seems to be entering an era where human + machine maximizes performance, much like chess in the 1990s. However, imagine a future where even talented mathematicians are nothing but noise in the machine (as is the case in chess now). A future where AI generates and verifies proofs without humans in the loop. Where the mathematics may be beyond human comprehension. In that future, does it matter that early career mathematicians are inhibited by these developments? Perhaps not. Programming faces the same issue. As AI crawls up the competence ladder, does it matter that fewer people have opportunities to develop the skillset of a senior engineer? Perhaps not.

Much like for many the point of chess is that it's played by humans, with truly superhuman AI relegated to a training aid, mathematics is in many ways about human comprehension. You can use AI to find and proof new theorems. But if you get to the point where humans can't understand it, is it even still math?

Is it still chess, if humans cannot understand it? Because that's the point we are at in chess. Engines making moves, that humans cannot understand, but somehow they work out to be best or seemingly best. Look at the Leela Zero games, when it came out. These engines play kind of other-worldly chess.

> Much like for many the point of chess is that it's played by humans, with truly superhuman AI relegated to a training aid It's much more than just training. Humans use the engines to prepare openings and find promising novelties. Over time these novelties unearthed by engines fill out theory. It's easy to fine elite games where neither player is out of book for dozens of moves. Modern players are full hybrids in that sense. Looking back at chess, it seems natural that Mathematics will go the same way.

> Think of it as 1000+ ways to think of that word/token I assume you used 1000 because that's in the ballpark of the vector size. But these are not independent scalars, like each might store a certain property. Just like in 2D you can have 4 quadrants (or subdivide further), with a vector of size 1000 you can encode an insane amount of meaning. > Those meanings are baked in at training time. So, LLMs might be able to cross-reference them and solve a class of problems that flew under our radar, but can't come up with revolutionary theories that were never in the training set. There's a lot of jumping to conclusions here, but I'll try to answer more generally. This idea of how LLMs work is mostly to build an intuition, like with a CNN you'd say imagine a layer does edge detection, and so on. And to some degree you can detect those kinds of behavior, but a NN is a VERY general architecture. It needn't work like you say, it can calculate any function and running under a loop and a scratchpad (basically an agent) is turing complete. Even ignoring that, this part is misleading > Those meanings are baked in at training time. Being baked in at training time does not mean it didn't build novel meanings at training time. This is even more significant when you take into account post training RL. A simple proof that transformers can generate novel, superhuman solutions, is that you can build a transformer based chess bot, feed it 0 human games, and train it with RL until it can beat any human, completely novel and unconstrained by human gameplay (because it would've never seen it). You can do that with any task that's verifiable, like coding or math. (Also as a separate fact, as long as a task is easier to verify than solve (basically always), you have somewhat of a million monkeys with a typewriter, and with temperature sampling the model might eventually stumble it's way onto a solution.)