Every week, I organize 400 to 700 links into roughly 60 categories as part of my ongoing effort to learn about AI. This is my personal notebook, which I enjoy sharing with friends… a hobby and a labor of love, rather than a commercial publication or product.
If you arrived here through a search or shared link, this page collects the links I found for Benchmarks for the week ending August 21, 2026.
As part of my learning process, I like to automate the category covers. It gives me a chance to learn Python and APIs.
This week’s cover prompt was written using Claude Opus 4.7, and the image was generated using Nano Banana 2.1.
Category Cover Image Prompt: Realistic architectural photograph of a 1929 Chicago Art Deco limestone skyscraper on a sunny day with clear deep blue sky, viewed from across the street, showing setbacks and dark steel-framed windows, with a stylized engaged herm emerging from a pier above the entrance holding a tall notched measuring rod against its chest and a carved relief of a balance scale on the pier below, all cut from the same honey-cream Indiana limestone with crisp raking shadows. The word BENCHMARKS is incised in large Art Deco capitals on a frieze band directly beneath the herm, sized and placed like a real building inscription. Perspective-corrected verticals, neighboring buildings receding behind, no other text or signage.
This Week in Benchmarks News
Summary by Claude Sonnet 5.5 based on this week’s links:
- Small open model, big scores: Qwen3.8-27B drew the most attention. Artificial Analysis ranked it #7 on its Agentic Index, and Cline said it became the top local model in its tool after four days. Ethan Mollick cautioned that it is clearly weaker on complex agentic work in practice and urged people to run their own tests.
- Cost joins the leaderboard: Arena added a Pareto frontier to Agent Arena, comparing model performance against median cost per task, and broke its Agent Leaderboard into categories with costs. Google's Gemini 3.7 Flash fit the theme, topping Artificial Analysis' new AA-AnalystAgent and scoring 84.6% on ARC-AGI-2 at $0.25 per task.
- Evals for agents themselves: Several posts focused on measuring agents rather than models. Artificial Analysis launched a Search Index for search APIs, and a dair_ai post flagged a re-evaluation of memory-based self-improving agents where multiple runs and shuffled task orders made results noisier.
This summary was generated by Claude Sonnet 5.5 to help you explore the links below. Rest assured, I select, organize, and check the links by hand in Google Sheets, and write the introduction and personal commentary in The Main Newsletters myself each week as a labor of love.
This week's links related to Benchmarks
Exciting news: Dreamina Seedance-2.5 by @BytePlusGlobal is topping ranks in Video Arena! It currently ranks #1 in Video Edit, the industry’s newest video capability category, which has only 9 models on the leaderboard so far. #1 Video Edit Arena: 1411 pts – a large 23 pts gap”
https://x.com/arena/status/2089448812159045848
We hit No.3 on the Image-editing benchmark! … beating Google’s Nano Banana and Meta’s Muse Image models.”
https://x.com/mustafasuleyman/status/2089766508373176726
Qwen 3.8 27B is #6 among open-weight models on the Vals Index v2 and is #1 on Harvey’s Legal Agent Benchmark among open weights.”
https://x.com/ValsAI/status/2089836844842435040
Qwen 27B is really good local model but, when you use it, it is immediately absolutely and obviously nowhere near as good as the other models listed here for agentic tasks, and especially for the kinds of complex tasks that GDPval-AA proports to measure Do your own benchmarking!”
https://x.com/emollick/status/2089914722464223706
Qwen3.8-27B ranks #7 on Artificial Analysis’ Agentic Index at only 27B parameters, a fraction of the size (and cost) of the other six. Train and deploy it all on one platform with the Baseten Loops SDK:”
https://x.com/baseten/status/2089749674265551077
Pareto frontier is live in Agent Arena! Dive into model performance on real-world agentic tasks, compared to the median cost per task. The current models on the Pareto frontier for Agent Arena are: – Claude Opus 5 (High) by @AnthropicAI (+12.34%/ $1.78) – Kimi K3 (Max) by”
https://x.com/arena/status/2090137780932538549
excited to be working with @NVIDIAAI on bringing SkillEvaluator to ClawHub! our goal is to make it easy for your Claw to discover the best skills, not just based on vibes but on quantitative proof having evals that show a clear “skill lift”, eg how much the skill actually”
https://x.com/Pat_Erichsen/status/2090142910499205145
Gemini 3.7 Flash just took #1 on @ArtificialAnlys new AA-AnalystAgent. AA-AnalystAgent evaluates against 80 real-world quantitative analysis tasks across 14 business and scientific domains (finance, healthcare, hydrology, government appropriations). The Agent run inside an”
https://x.com/_philschmid/status/2090063976872751408
The real benchmark for an AI agent isn’t what it says. It’s what it gets done. AutoClaw, @Zai_org ‘s AI Agent for Work, is built to turn intent into finished output. Give it a complex task. AutoClaw breaks it down, operates tools, works across files, coordinates multiple”
https://x.com/AutoClawAIer/status/2090446256342708724
Muse Spark 1.2 (xHigh) by @AIatMeta is now in Agent Arena, with a net improvement of +2.1%! Agent Arena measures models on millions of real-world, long-horizon agentic tasks. We use causal tracing methodology to measure a model’s net improvement, indicating how much it improves”
https://x.com/arena/status/2090484142408618033
Today we’re also previewing WildArtifactBench, an internal evaluation framework designed to assess agents on complex, real-world tasks across diverse deliverable formats. By using win rates and Elo scores from human and agentic preference judges rather than strict ground-truth”
https://x.com/AIatMeta/status/2090505413817246050
🚀Today we launched LangSmith Tuned Evaluators, starting with Perceived Error. Tuned Evaluators run on production traces to catch undesirable agent behavior and attach feedback that you can use in your agent improvement processes. In our benchmark, our tuned model beat frontier”
https://x.com/hwchase17/status/2089755542931865901
A few months ago,@sh_reya and I released eval skills plugin. We iterated on it a bunch and recently made it better The biggest change is a new error-discovery skill. Give your coding agent a file of AI outputs or traces, and it builds a custom review app w/intelligent”
https://x.com/HamelHusain/status/2089438973714440196
Agent Categories and Costs are now in the Agent Leaderboard! Two questions guide model selection: task-specific performance and cost to completion. Based on analysis of 1.7M+ real-world Agent Mode sessions, the leaderboard now lets users: – Filter the leaderboard by Code, Work,”
https://x.com/arena/status/2089464753567797321
Announcing the Artificial Analysis Search Index, benchmarking how search API providers perform on quality, cost, and speed when used by an agent. We are initiating coverage with Parallel, Exa, Firecrawl, You (dot) com, Tavily, Keenable, and Brave Search is one of the most”
https://x.com/ArtificialAnlys/status/2089755262915936661
It’s felt like harness month on Twitter. We’re seeing much faster and cheaper gains on a bunch of benchmarks by focusing on harness improvement rather than model improvement. Allowing the harness to be updated during use is just another form of continual learning. The common”
https://x.com/bradenjhancock/status/2090114460828766567
Why an LLM leaderboard matters for agent builders
https://www.algolia.com/blog/ai/llm-leaderboard?%3Futm_campaign=tldr_global_b2x_ecomm_ecomm_tof_reach
Ornith-1.5 is out and it serves in vLLM today. Congrats @ornith_ on a self-improving, MIT-licensed family from 9B to 397B, SOTA among open models on coding and agentic tasks. Already serving in vLLM: vllm serve ornith-ai/Ornith-1.5-9B”
https://x.com/vllm_project/status/2090243605147586955
Finally a good paper testing whether memory-based self-improving agents actually improve. The re-evaluation adds two things prior work skipped, multiple runs to measure variance and randomly shuffled task orders. Both hurt. Agent evaluation is already noisy on multi-step”
https://x.com/dair_ai/status/2090559561128407336
(60) WF2026: Autoresearch & Keynotes ft. Anthropic, Google DeepMind, Amazon AGI, Sonar, Arena, Recursive – YouTube
https://www.youtube.com/watch?v=4sX_He5c4sI
Aloha! 🌺Introducing Ornith-1.5, a family of open-source LLMs spanning 9B Dense, 35B MoE, and 397B MoE, trained with self-improving strategies. It achieves state-of-the-art performance among open-source models of comparable size and delivers performance comparable to Claude Opus”
https://x.com/ornith_/status/2090074077084127302
Gemini 3.7 Flash from @Google on ARC-AGI (Verified): – ARC-AGI-2: 84.6%, $0.25/task – ARC-AGI-1: 95.5%, $0.12/task Gemini 3.7 Flash stands out for its low cost and high scores on ARC-AGI-1 and ARC-AGI-2 relative to other frontier models.”
https://x.com/arcprize/status/2090500144550539327
great initiative (RSI benchmark) from @ScaleAILabs and @mhrezaeics”
https://x.com/alexandr_wang/status/2089759883914822033
With retained reasoning and compaction, GPT-5.6 Sol improved from 13.3% to 38.3% on ARC-AGI-3 while using roughly 6x fewer output tokens.”
https://x.com/OpenAIDevs/status/2089374232040132764
The full results for Grok 4.6 are in, and its no surprise that it excels on legal benchmarks. It ranks #3 of 49 on Legal Research Bench at 48.1%, at a fraction of the price of the two models above it.”
https://x.com/ValsAI/status/2090119651204423763
Cartesia’s Sonic 3.6 takes the #1 spot on both the Provider Voice and Controlled Voice Artificial Analysis Speech Arena leaderboards, surpassing Speechify AI’s Simba 3.2 and Alibaba’s Qwen-Audio-3.0-TTS-Plus, with Sonic 3.5 holding #2 on Controlled Voice Sonic 3.6 is the latest”
https://x.com/ArtificialAnlys/status/2089400880688976062
Qwen3.8 vs Qwen3.6 vs Gemma 4: 24GB GPU Test
https://kingy.ai/blog/qwen3-8-27b-vs-qwen3-6-27b-vs-gemma-4-31b/
Artificial Analysis Intelligence Index puts Qwen3.8-27B at DeepSeek V4-Pro and GPT 5.6 Luna performance. This is the first time a local model has scored frontier model capability. We weren’t expecting this pace of local progress anywhere near this soon.”
https://x.com/cline/status/2089425906569977896
Qwen3.8-27B is now the #1 local model in Cline after just 4 days. This ends a *4 month* streak by the previous winner Qwen2.5-Coder-7B which has been the top local model since April.”
https://x.com/cline/status/2089825294677143973
We’re releasing new Qwen3.8-27B GGUFs with 10% higher accuracy. Unsloth Dynamic V3 outperforms others by >10% on Div-300, KLD & more benchmarks. We also release 1-bit quants that retain 77% accuracy. Run on 8GB RAM. Blog:
https://t.co/tHsBexyh2K GGUF:
https://x.com/UnslothAI/status/2090103470015828184
4 days to the top.🏆Thank you to every builder who pushed Qwen3.8-27B to #1 on Cline! @cline”
https://x.com/Alibaba_Qwen/status/2089919106522976337
GLM-5.3 (Max) by @Zai_org has shifted the Pareto frontier for Code Arena: WebDev! GLM-5.3 (Max) would sit at #2 among open models (upon open weights release, and #8 overall) with 1597 pts in the Code Arena: WebDev. At $3.65/M, GLM-5.3 (Max) is on par with Qwen3.8 (Max)”
https://x.com/arena/status/2090581559262798055
very interesting part on CoT monitoring > estimation is ~20% compute overhead with a multi level monitoring system > confirmed they didn’t have monitor for eval and training run before > monitor expected to catch concerning activity within 30min > safety, security AND (not or)”
https://x.com/eliebakouch/status/2089780991988502633
MiniMax H3 image+audio to video feed an audio into h3 instead of generating it. i was testing this flow, got shocked with how good the lipsync and movement is implemented it on the i2v version in diffusers, made a demo for it. what do you think? 🔊 ▶️
https://x.com/multimodalart/status/2089418659370357191
Gemini 3.7 Flash isn’t just fast ⚡ It’s now ranked #1 on @ArtificialAnlys’ AA-AnalystAgent leaderboard for complex, real-world data analysis. In a test of 80 real-world tasks across 14 business and scientific domains, Gemini 3.7 Flash combined reasoning and speed to deliver the”
https://x.com/NewsFromGoogle/status/2090120394141266141
What % of global AI token spend do you think is people running evals?”
https://x.com/OfficialLoganK/status/2090235603745165684
This is super cool! Sota video generation models like our MiniMax H3 understand the movement of a walking character so well, while some people report that latest image generation model still struggles to keep walking motion consistent. MiniMax H3 is here for your game sprites”
https://x.com/MiniMax_AI/status/2089420340728610890
BREAKING: Muse Spark 1.2 by @AIatMeta takes 1st for Video-to-Website with an Elo rating of 1279 and impressive scores across all of our multimodal code categories. Muse Spark 1.2 also takes 2nd for Image-to-HTML with an Elo rating of 1252 and 3rd for Image-to-Frontend with Elo”
https://x.com/DesignArena/status/2090498670685020639
MAI-Image-2.5 is now #1 on the Artificial Analysis leaderboard for image editing! Amazing hillclimbing… compounding the gains!”
https://x.com/mustafasuleyman/status/2090132085562347786
GLM 5.3 is #2 on Terminal Bench, #3 on Legal Bench, and #6 on Skills Bench among-open weight models.”
https://x.com/ValsAI/status/2090192848780136668
GLM-5.3 achieves 60 on the Artificial Analysis Intelligence Index, on par with Kimi K3 and up 7 points from GLM-5.2. Once the weights are released it will be tied as the leading open weights model @Zai_org has just launched GLM-5.3, which ties Kimi K3 (60) for the most”
https://x.com/ArtificialAnlys/status/2089830890709135426
Cool MiniMax-H3 use case unlocked for game sprites creation, here’s the workflow: 1. one character design still, on a flat magenta background 2. one 5s H3 clip per move (walk, jump, punch, kick, block…) 3. cut each clip down to its pose extremes 4. key out the magenta, pack”
https://x.com/victormustar/status/2089310616854892818
OpenAI has scored a major win with 5.6 Luna, not just in terms of PR, but also due to the efficiency that makes it possible to offer the service to everyone for free, as is the case here. Luna is cheap *and* good (enough) for several tasks. I still find it impressive that a”
https://x.com/kimmonismus/status/2090111297039765703
AI Model & API Providers Analysis | Artificial Analysis
https://artificialanalysis.ai/#intelligence
dig.bench
https://digbench.ai/
What are the big questions about AI capabilities that govern the impact AI will have on the world? In this Gradient Update, @GregHBurnham lays out his top questions, and how he thinks Epoch’s work on benchmarking can help answer them.”
https://x.com/EpochAIResearch/status/2088315292141441396
The summer Math fell to the machines… – YouTube
https://www.youtube.com/watch?v=iuZPTE5qsJY
Leave a Reply