Will an LLM report >50% score on ARC in 2025?
Plus
19
Ṁ3984Dec 31
85%
chance
1D
1W
1M
ALL
ARC is a general-purpose AI eval designed to test intelligence as opposed to memorization. https://arcprize.org/arc
This market resolves to Yes if there are public demonstrations of general-purpose base models from OpenAI, Anthropic or Deepmind that solve at least 50% of the ARC questions before 2026.
(note that this is separate from winning the arc price, which requires only using open-source models)
This question is managed and resolved by Manifold.
Get
1,000and
3.00
1,0003.00Related questions
Related questions
LLM Hallucination: Will an LLM score >90% on SimpleQA before 2026?
60% chance
Will LLMs mostly overcome the Reversal Curse by the end of 2025?
67% chance
Will an LLM be able to solve the Self-Referential Aptitude Test before 2027?
66% chance
Will a publicly-available LLM achieve gold on IMO before 2026?
54% chance
Will the ARC-AGI grand prize be claimed by an LLM?
55% chance
What will be true of Anthropic's best LLM by EOY 2025?
Will an LLM improve its own ability along some important metric well beyond the best trained LLMs before 2026?
58% chance
Will I write an academic paper using an LLM by 2030?
65% chance
6 months from now will I judge that LLMs had already peaked by Nov 2024?
16% chance
Will Apple release its own LLM on par with state of the art LLMs before 2026?
49% chance