AI2 Reasoning Challenge (Challenge Set) 排行榜
Grade-school level science questions that require reasoning beyond simple retrieval. The 'Challenge' set contains questions that simple baselines get wrong.
为什么重要： Tests commonsense scientific reasoning. Largely saturated for frontier models but still useful for comparing mid-tier and open-source models.
顶级模型
96.9%
Llama 3.1 405B
平均评分
95.0%
共8个模型
已测试模型
8
指标: accuracy
人类基准
-
评分范围: 0%–100%
ARC-Challenge Scores - Top 8 Models
Ranked by ARC-Challenge score (%)
模型排名
All models with a reported ARC-Challenge score, ranked by highest accuracy.
关于 ARC-Challenge
- 全名
- AI2 Reasoning Challenge (Challenge Set)
- 类别
- Reasoning
- 指标
- accuracy (%)
- 评分范围
- 0%–100%
- 人类基准
- 尚未确定
- 状态
- 饱和
ARC-Challenge is a standardized evaluation that measures AI model performance on specific tasks. It provides comparable scores across different models, helping developers choose the right model for their needs.
Llama 3.1 405B currently holds the top score on the ARC-Challenge benchmark. See our full rankings table above for the complete leaderboard with 8 models.
We update benchmark data from multiple sources including HuggingFace open-source model leaderboards and LMArena. Scores are refreshed regularly as new evaluations are published and new models are released.
No. While ARC-Challenge is an important indicator, real-world performance depends on many factors including pricing, latency, context window, and specific task requirements. We recommend using our composite score which weighs multiple benchmarks and practical factors.