Which dataset is used to compute MATH now that `hendrycks/competition_math` has been served a takedown notice?

#1111
by kyle-cohere - opened

In this thread (issue #494 on GitHub), LenDigLearn pointed to a mirror dataset. Is this being used?

@clefourrier closed the issue, but it's unclear which dataset is the new standard. It's also unclear which data have been used for models evaluated after the takedown notice was served.

Open LLM Leaderboard org

Hi @kyle-cohere :)
We've kept on using the same dataset - it's just no longer openly available to avoid removal requests.
Questions are publicly usable, but it's still under discussion whether answers are also OK to publicly use or not.

clefourrier changed discussion status to closed

Sign up or log in to comment