AI models are starting to decipher high-level math problems

Neel Somani, a software engineer, former quantitative researcher, and startup founder, was testing the math skills of OpenAI’s new models last weekend when he made an unexpected discovery. After pasting the problem into ChatGPT and letting it think for 15 minutes, I came back with a complete solution. He evaluated the proof and formalized it using a tool called Harmonic, and everything went well.

“I was interested in establishing a baseline for when LLMs can effectively solve unsolved math problems compared to when they are struggling,” Somani said. What surprised me was that Frontier started to move forward little by little with the latest model.

ChatGPT’s chain of thought is even more impressive, rattling off mathematical axioms such as Legendre’s formula, Bertrand’s postulate, and the Star of David theorem. Eventually, the model found a 2013 Math Overflow post. There, Harvard mathematician Noam Elkies had an elegant solution to a similar problem. However, ChatGPT’s final proof differed from Elkies’ work in important ways and provided a more complete solution to the version of the problem posed by legendary mathematician Paul Erdős. His vast collection of unsolved problems has become a testing ground for AI.

For machine intelligence skeptics, this is a surprising result, but it’s not the only one. From formalization-oriented LLMs like Harmonic’s Aristotle to literature review tools like OpenAI’s Deep Research, AI tools are widespread in mathematics. But since the release of GPT 5.2, which Somani says is “anecdotally more proficient at mathematical reasoning than previous versions,” it has become difficult to ignore the sheer volume of problems solved, raising new questions about the ability of large-scale language models to push the frontiers of human knowledge.

Somani was looking into the Erdos issue. Erdos Problems is a set of over 1,000 conjectures by the Hungarian mathematician maintained online. These problems vary widely in both subject matter and difficulty, making them attractive targets for AI-driven mathematics. The first batch of autonomous solutions was delivered in November with a Gemini-powered model called AlphaEvolve. But recently, Somani and colleagues discovered that GPT 5.2 is very good at high-level mathematics.

Since Christmas, 15 issues have been changed from “open” to “resolved” on the Erdos website, with 11 of the resolutions specifically acknowledging that an AI model is involved in the process.

Respected mathematician Terence Tao offers a more nuanced analysis of the progress on his GitHub page, counting eight different cases where AI models have made meaningful autonomous progress on the Erdos problem, and six other cases where they have discovered and built on prior research. Although we have a long way to go before AI systems can perform mathematics without human intervention, it is clear that large-scale models have an important role to play.

Regarding Mastodon, Tao speculates that the scalable nature of AI systems makes them well-suited to “systematically apply to the ‘long tail’ of Erdos problems, many of which actually have simple solutions.”

“Many of these simple Erdos problems are therefore more likely to be solved by purely AI-based methods than by human or hybrid means,” Tao continued.

Another driver is the recent move toward formalization, a labor-intensive task that facilitates the validation and extension of mathematical reasoning. Formalization does not require the use of AI or computers, but the advent of new automated tools has made the process much easier. Lean, an open source “proof assistant” developed at Microsoft Research in 2013, has become widely used in the field as a way to formalize proofs, and AI tools like Harmonic’s Aristotle are expected to automate much of the formalization work.

For Harmonic founder Tudor Achim, the fact that Erdos’ problem was suddenly solved is less important than the fact that the world’s greatest mathematicians are starting to take these tools seriously. “I’m more concerned about the fact that math and computer science professors are using it. [AI tools]”These people have reputations to protect, so when they say they’re using Aristotle or they’re using ChatGPT, that’s real evidence,” Achim said.