Image created with gemini-2.5-flash-image with claude-sonnet-4-5. Image prompt: Photorealistic Times Square at dusk, all billboards displaying massive glowing leaderboards with racing bar graphs, percentage scores, and AI model rankings in neon colors, crowds looking up at the competitive performance charts, vibrant urban atmosphere with digital scoreboards dominating every surface
Sonnet 4.5 crushing GPT-5 high on ARC-AGI 2 https://x.com/scaling01/status/1973081750189334587
Anthropic supremacy in the coding category of lmarena Claude 4.5 Sonnet on tied first with Claude 4.1 Opus https://x.com/scaling01/status/1973836516205134135
The most important thing here: the models completed these tasks 100x faster and cheaper than the industry experts”” / X https://x.com/scaling01/status/1971431825433374866
🚀 Ring-1T-preview: Deep Thinking, No Waiting The first 1 trillion open-source thinking model -> Early results in natural language: AIME25/92.6, HMMT25/84.5, ARC-AGI-1/50.8, LCB/78.3, CF/94.7 -> Solved IMO25 Q3 in one shot, with partial solutions for Q1/Q2/Q4/Q5 Still evolving! https://x.com/AntLingAGI/status/1972711364876697612
The “”90% of code written by AI”” thing is increasingly true for very good independent coders, not just a thing the labs say about the future. (The remaining 10% is still very important and very human, though)”” / X https://x.com/emollick/status/1972710280137257214
Today, we are launching “Help Me Choose” in @yupp_ai – a new product feature where multiple AIs critique each other and debate among themselves to help users synthesize diverse perspectives and get the best answer out of their own “AI council”. https://x.com/lintool/status/1973874173157257485
🚨 Leaderboard Update: we have a four-way tie for #1 in the Arena! 🏆 The very top tier is now tied across the strongest models in the world: 🏆 Claude Sonnet 4.5 32k Thinking 🏆 Claude Sonnet 4.5 standard 🏆 Claude Opus 4.1 🏆 Gemini 2.5 Pro All separated by just a few Arena https://x.com/arena/status/1974215622474293262
My review of Sonnet 4.5 based on ~30 hours of Claude Code use is that it’s basically the same as Opus 4.1. Which is quite good! But not as good as GPT-5 (codex thinking=high). Claude Code is still much more polished than Codex. But I find GPT-5 much stronger as a model.”” / X https://x.com/finbarrtimbers/status/1973922679418974298
Sonnet 4.5 was just released – it takes the #1 spot on our finance and programming benchmarks! We had the chance to test the new @claudeai model ahead of launch and found it to be exceptionally capable, especially for agentic applications. (1/6) https://x.com/_valsai/status/1972707249748582454
We evaluated Anthropic’s Sonnet 4.5 with our minimal agent. New record on SWE-bench verified: 70.6%! Same price/token as Sonnet 4, but takes more steps, ending up being more expensive. Cost analysis details & link to full trajectories in 🧵 https://x.com/klieret/status/1972766908878667877
Sonnet 4.5 – the best non-reasoning model on ARC-AGI-2 (GPT-5 is at 0%) https://x.com/scaling01/status/1973083158812782802
Sonnet 4.5 ranking 4th in the overall on LiveBench #1 in Coding #1 in Mathematics https://x.com/scaling01/status/1973088409359982623
Sonnet 4.5 ranking #1 on Deep Research Bench https://x.com/scaling01/status/1973088829138460987
Veo3 high quality on Freepik 😀. Again I use Grok4 +Kimi K2 to write my prompts if you are not using them you are missing big time 👀 Veo3 on Flow refuses to generate the prompt for violating the safety, while the API and Gemini App did not have the problem 🤔🤔🤔 { https://x.com/IamEmily2050/status/1951248894035402973
Final update! MMLU is saturated and has become (rightfully) less popular. What should replace it? – Other knowledge evals like GPQA, MMLU-Pro – Code evals like LiveCodeBench – Agentic evals like BFCL – Other? https://x.com/maximelabonne/status/1972615048511250647
and the task selection is obviously biased, so the eval is not too hard for LLMs it’s a small subset of real world tasks”” / X https://x.com/scaling01/status/1971433067266089395
If people lose trust in your evals – they lose trust in you.”” This is exactly why you need human alignment in your Evals! If there’s ONE thing you listen to this weekend, make it @lennysan latest podcast episode with @HamelHusain and @sh_reya They absolutely demolish common https://x.com/bnicholehopkins/status/1971683830269350161
@syhw Just to clarify some confusion: The SweBench Verified number I provided refers to *TTS* pass@1. This figure is correct. For additional details, please refer to the paper, specifically, the caption of Figure 2 on page 3.”” / X https://x.com/alexandr_wang/status/1971603685559140663
ARC Prize @ MIT – Sept 27, 2025 Measuring Intelligence with Interactive Benchmarks Join us live in Boston to hear a conversation from @fchollet, @mikeknoop, @GregKamradt, Josh Tenenbaum, Laura Schulz, @gershbrain, @jacobandreas and @JunyiChu https://x.com/arcprize/status/1971609644004200693
the best eval for LLMs usefulness is industry adoption, which directly translates to tokens used / money spent on tokens”” / X https://x.com/scaling01/status/1971433691848262084
We’re launching a new feature at Yupp: Help Me Choose. Don’t know which AI’s response is better? Help Me Choose lets a third AI give its own analysis. Then, the two models that answered you analyze each others’ responses – and their own. Final step? You pick your favorite! https://x.com/yupp_ai/status/1973882910907470237
As predicted, the “”downturn in AI adoption”” that everyone was speculating about here on X appears to have been a data artifact and not actually a reversal of the trend.”” / X https://x.com/emollick/status/1973243767982531009
IKEA furniture assembly with only the tools provided in the flatpack would be a great AGI test. https://x.com/TheHumanoidHub/status/1972386261106393100
like surely the human graders were influenced by style similar to lmarena”” / X https://x.com/scaling01/status/1971432758817050970
Ollama’s cloud models just got an update! ollama run kimi-k2:1t-cloud Welcome Kimi K2 by @Kimi_Moonshot for the 1 trillion parameter model! ollama run deepseek-v3.1:671b-cloud DeepSeek V3.1 Terminus by @deepseek_ai for the updated updated that is better in language”” / X https://x.com/ollama/status/1971750071483167010
Results on Test-Time Scaling: Training Llama3.1-8B-Instruct on AIME24, AMC23, MATH500, RESTRAIN delivers +11.0% average gain over TTRL and ETMR. 🧵(4/5) https://x.com/jaseweston/status/1974000970192544248
Right behind the leaders, a bundle of models from @openAI: 🥈ChatGPT-4o-latest 🥈GPT-4.5 preview 🥈GPT-5 high 🥈o3 All clustered within a single rating point of the top. With tens of thousands of votes, they’re effectively in striking distance of the peak.”” / X https://x.com/arena/status/1974215628757066077
Last week we found an issue with SWE-Bench, allowing agents to cheat by looking at future commits. Instead of celebrating the SWE-Bench Devs for quickly fixing the issue and being transparent, the HN crowd is dunking on them and drawing wildly inaccurate conclusions about”” / X https://x.com/TacoCohen/status/1966421688846778561
LLaDA-MoE: A Sparse MoE Diffusion Language Model “”LLaDA-MoE achieves state-of-the-art performance among diffusion language models with larger parameters, surpassing previous diffusion language models LLaDA, LLaDA 1.5, and Dream across multiple benchmarks. The instruct-tuned https://x.com/iScienceLuvr/status/1972971896036851767
Dude. MLX team really went all out. How is it that you can get almost 2.5x the performance of the same quant of a model just by using the MLX version of it? Granite 4 H Tiny 4bit GGUF = 47 tokens/sec Granite 4 H Tiny 4bit MLX w/ DWQ = 115 tokens/sec This is on my M3 Max https://x.com/JorgeConsulting/status/1974168414391619672
We tested whether coding agents with CLI tools are all you need for complex document search and analysis tasks 🧪 Our new SemTools benchmark used 1000 @arxiv papers to compare agents with and without semantic search capabilities: 🔍 Agents with semantic search provided more https://x.com/llama_index/status/1973783798044307741
Another paper pointing out the inadequacy of older public benchmarks for determining whether AI is actually good for tasks like medicine. Models are clearly memorizing or using heuristics for some answers. A new wave of benchmarks based on real-world data will help, more needed. https://x.com/emollick/status/1971658338594443287
Lots of new models; time for another AI video vibe check”” / X https://x.com/bilawalsidhu/status/1971126232415862840
