Member-only story
GPT-4 is Amazing but Still Struggles at High School Math Competitions
Its greatest strength is its biggest flaw
GPT-4 is the latest version of the GPT architecture developed by OpenAI, released on 14 March 2023.
We are told on the landing page that GPT-4 “surpasses ChatGPT in its advanced reasoning capabilities”, and presented with some impressive results for the USA Bar Exam and Biology Olympiad.
The GPT-4 Technical Report contains many other simulated exams used to test the reasoning and problem solving ability of GPT-4. When it comes to Mathematics, GPT-4 ranked in the top 11% of scores on the SAT Math Test (a significant improvement from GPT-3.5).
But if you’ve ever sat the SAT, or ever seen the SAT, you’ll know the SAT Math questions are quite predictable and formulaic. Most of them just require the application of a pre-defined set of skills — solving equations, trigonometry laws, index laws, etc.
