Tweet by Ethan Mollick:

Note o1 is showing how unprepared we are for testing high level AIs to figure out what they are good or bad at.

Instead we are turning to experts, like one of the greatest living mathematicians, to give it a vibe check ("mediocre but not completely incompetent, grad student") pic.twitter.com/i1ic1QpOfB