AI Flash Report
›
Model Release Tracker
› Google Gemini Ultra
Major Release
Google
Multimodal
Proprietary
Google Gemini Ultra
Google's first model to beat GPT-4 on MMLU — 90%+ with CoT
Released
2023-12-06
· ~540B · knowledge cutoff 2023-06
Overview
Google's most capable multimodal AI model
Multimodal reasoning
Text, image, audio, video understanding
Advanced mathematical reasoning
Code generation
Specifications
Developer
Google
Release date
2023-12-06
Model type
Multimodal
Parameters
~540B
Architecture
MoE Transformer
Knowledge cutoff
2023-06
License
Proprietary
Input modalities
text, image, audio, video
Output modalities
text
Benchmarks
Benchmark
Gemini Ultra
PaLM 2
Δ
MMLU
90.0%
78.3%
+11.7
HumanEval
74.4%
77.2%
-2.8
MATH
53.2%
34.3%
+18.9
Availability
Limited Access
Google One AI Premium
Vertex AI
What's new vs PaLM 2
First model to exceed human expert on MMLU
Native multimodal
32K context
Direct comparison available in the
model comparison tool →
Official references
📢 Launch announcement
📝 Blog post