What will be true of Gemini 2?
Basic
21
Ṁ1313
resolved Apr 15
Resolved
YES
It will be released before GPT-5
Resolved
YES
It will be released in 2024
Resolved
N/A
>20% (5-shot) on SWEBench (current sota ~ 6%?)
Resolved
N/A
It will get into at least one PR blunder, like the woke issue for Gemini 1 Ultra
Resolved
N/A
It will score better than o1-preview at AIME (44.6% in single pass)
Resolved
N/A
It (the strongest variant) will be better than the best OpenAI model at the moment of release
Resolved
N/A
It will score >70% on GPQA (diamond, pass@1)
Resolved
N/A
It has input modalities other than text, audio, and video (MIDI/code count as text)
Resolved
N/A
It will score >= 85% on MMLU Pro
Resolved
N/A
It will score >=80% on MMMU

#Google Gemini
Get
Ṁ1,000
and
S3.00
Sort by:

@mods creator deleted account. a mod can adopt this and resolve the options, or if no one does so in a week or two they should probably N/A

@Ziddletwix did anyone adopt it yet

@Eliza no.

so how should the benchmark questions be resolved? For a lot of them Gemini 2.0 Flash Thinking is above the threshold, but Gemini 2.0 Flash and 2.0 Pro are not.

bought Ṁ30 NO

shall be resolved

huh, not sure how the benchmark questions should resolve now that 2.0 Flash is out. Maybe wait for info on 2.0 pro? Also relevant that it is called "2.0 Flash Experimental" for now

https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/#gemini-2-0-flash

Current leading benchmarks, excluding o1 (o1-preview gets 73% on GPQA)

Related questions

© Manifold Markets, Inc.•Terms + Mana-only Terms•Privacy•Rules