Thanks for the support @AndrewYNg! Completely agree, faster token generation will become increasingly important as a greater proportion of output tokens are consumed by models, such as in multi-step agentic workflows, rather than being read by people.
Shoutout to the team that built artificialanalysis.ai/ . Really neat site that benchmarks the speed of different LLM API providers to help developers pick which models to use. This nicely complements the LMSYS Chatbot Arena, Hugging Face open LLM leaderboards and Stanford's HELM that focus more on the quality of the outputs. I hope benchmarks like this encourage more providers to work on fast token generation, which is critical for agentic workflows!
68
60
939
944,716
In AA-PersonalityBench, Kimi K3.1 Pro shows the biggest personality change we’ve seen compared to a direct predecessor Each model is percentile ranked from 1 to 100 on each of 30 personality traits. Across all 30 traits, Kimi K3.1 Pro's ranks moved by an average of 30 points from Kimi K3's (next largest is 23, for GPT-6 Sol to GPT-6.1 Sol), and 18 of the 30 traits moved by 25 points or more.
38
10
215
31,164
Kimi K3.1 Pro's writing style has changed as well. Compared to Kimi K3, its responses include fewer parenthetical asides, em dashes, contractions, and italics, but more bold text, headings, and tables.
1
17
5,413
We are collaborating with Silicon Data to bring market-based GPU pricing into AA-AgentPerf, our datacenter system performance benchmark Our users and partners want to know how much agent work they can support per dollar of compute. @silicondata's rental price indices allow us to report a reliable value-for-money metric, updated daily as prices change. We currently display Silicon Data indices for H100, H200, B200 and B300, tracked across cloud providers The new metric is Throughput per $/hr, which divides the total throughput of a system and serving configuration by the hourly cost of renting that system. Among systems with available rental pricing, NVIDIA’s B300 generally leads, while B200 matches it at faster output speeds on some models Note: the below chart includes an MI355X rental rate that has been assumed by Artificial Analysis based on recent market quotes
44
13
157
29,128
We’ve released AA-Omniscience v1.1, updating our knowledge and reasoning benchmark with a refreshed public set, removal of saturated questions, and ambiguity fixes AA-Omniscience is our proprietary benchmark measuring model knowledge and hallucinations, including whether models admit when they don’t know an answer. Our primary metric, Net Score, rewards correct answers, penalizes incorrect answers, and treats abstentions neutrally. AA-Omniscience was updated to v1.1 as part of the Artificial Analysis Intelligence Index v5 release. Key changes in AA-Omniscience v1.1: ➤ Updated 3,000-question dataset: Down from 6,000 questions, with previously public and saturated questions removed, errors and ambiguities corrected, and new questions added ➤ Newly calibrated 300-question public set: AA-Omniscience v1.1 has a new public set on @huggingface calibrated to match the difficulty of the private set. At release, public and private scores have a 0.99 Pearson correlation and 0.99 Spearman rank correlation for AA-Omniscience Net Score, with a median difference of 0.0 points. Accuracy and hallucination rate also have 0.99 correlations, with median differences of -0.07 and -0.29 percentage points, respectively ➤ Slightly updated domain distribution: Questions span the same 6 domains and 42 topics as version 1, with a revised distribution: Science, Engineering and Mathematics (600), Software Engineering (600), Finance (450), Health (450), Humanities and Social Sciences (450), and Law (450) Key results for AA-Omniscience v1.1: ➤ Rank order largely preserved: AA-Omniscience v1.1 has a 0.98 Spearman rank correlation with v1. The recently released Gemini 4 Carbon (high) leads with a Net Score of 54, followed by Claude Opus 5.5 (48, max with fallback) and Claude Fable 5.1 (46, max with fallback). Muse Spark Open (xhigh) is the leading open weights model, 28 points behind Gemini 4 Carbon ➤ Scores average ~2 points lower on the refreshed set: Models score 1.9 points lower on Net Score on average compared to v1, with a median decrease of 0.6 points. Some models see much larger declines, particularly MiniMax M3.1 Flash, down 22 points (16 to -6), and MiMo V2.6 Pro, down 13 points (8 to -5) ➤ Slightly lower accuracy: AA-Omniscience Accuracy measures the percentage of questions the model answers correctly. Models answer a median of 1.5 percentage points fewer questions correctly on average compared to v1. Claude Fable 5.1 (max with fallback) has the highest accuracy at 68%, followed by Claude Opus 5.5 (max with fallback) at 66% ➤ Hallucination rates decrease: AA-Hallucination measures how often models choose to answer when they do not know the answer. The median Hallucination Rate falls 1.4 percentage points, with Gemini 4 Argon (high) recording the lowest hallucination rate at 4%
93
50
886
199,012
AA-Omniscience v1.1 has a new public set on @huggingface calibrated to match the difficulty of the private set. At release, public and private scores have a 0.99 correlation for AA-Omniscience Net Score, with a median difference of 0.0 points. Accuracy and hallucination rate also have 0.99 correlations, with median differences of -0.07 and -0.29 percentage points, respectively
2
1
33
10,491
Hyperflow H3 lands at #5 on AA-Video-I2V v2.1 and #3 on AA-Video-Editing v1.1, at under half the price of Dreamina Seedance 2.5 on both Hyperflow H3 is Video Rebirth's 8-step LoRA for MiniMax H3, released in September 2026 with open weights, and now supports Image to Video and Video Editing. Hyperflow H3 places #5 on AA-Video-I2V v2.1. In our testing, it does best on Live-Action Film, where only Dreamina Seedance 2.5 ranks higher, and on Multi-Scene & Narrative, where only Utopai X scores higher. On AA-Video-Editing v1.1, it ranks #3, ahead of MiniMax H3 (768p), the model it builds on. Hyperflow H3 is priced at $9.00 per minute for Image to Video and $18.00 per minute for Video Editing, both at 1088p on Video Rebirth's API. Congratulations to @video_rebirth on the release! See below for our analysis of Hyperflow H3 in Image to Video and Video Editing 🧵
14
11
122
21,445
At $9.00/min for Image to Video and $18.00/min for Video Editing, Hyperflow H3 costs under half of Dreamina Seedance 2.5 on both boards Dreamina Seedance 2.5 is $34.12/min for Image to Video and $40.82/min for Video Editing.
1
5
4,339
We handed GPT-6.1 Sol a robot arm and a pen, and it wrote the three scariest words in AI Happy Halloween from AA-Robotics! 🎃
64
59
1,252
64,525
Utopai X (based on MiniMax H3) takes #2 on AA-Video-I2V v2.1, just behind Dreamina Seedance 2.5 Utopai X is the video model from Utopai Studios, post-trained on MiniMax H3 and available inside PAI, Utopai's production platform, and now supports Image to Video! Utopai X is the only model in the top 2 of both AA-Video-T2V v2.1 and AA-Video-I2V v2.1. On AA-Video-I2V v2.1, it is the highest ranked model built on MiniMax H3, ahead of MiniMax H3 (768p) itself, Hyperflow H3 and MiniMax H3 Max. In our testing, it does best on prompts that test Physics and Multi-Scene & Narrative, and on Frontier, UI/UX & Motion Design and Productivity & Knowledge Work use cases. Utopai X is available in PAI at 93 PAI credits per second of 1080p video, with no public API. PAI plans start at $15 per month for 1,000 credits. Congratulations to @UtopaiStudios on the release! See below for our analysis of Utopai X in Image to Video 🧵
41
19
205
424,458
This translates to Utopai X's best Image to Video use cases: it leads UI/UX & Motion Design and Productivity & Knowledge Work, and narrowly leads Frontier and Retail & Ecommerce Use cases measure the real-world work the video is for. Utopai X is also just behind Dreamina Seedance 2.5 on Marketing & Advertising. Against MiniMax H3 (768p), Utopai X is closer to the frontier on 6 of the 10 use cases. Its largest gains are in Frontier, Animation & Gaming and UI/UX & Motion Design. ➤ UI/UX & Motion Design covers kinetic typography, UI motion demos across devices, and abstract motion graphics. ➤ Productivity & Knowledge Work covers explainers, educational and training videos, animated data visualizations, and scenario simulations. ➤ Live-Action Film, where Utopai X trails the leaders most, covers cinematic shots, microdrama episodes, and previsualization.
1
4
5,844