Technical and Dev: AI News Week Ending 02/28/2025

“Autellix An Efficient Serving Engine for LLM Agents as General Programs https://x.com/_akhaliq/status/1892434670597345474

“Craw4LLM Efficient Web Crawling for LLM Pretraining https://x.com/_akhaliq/status/1892435546703638628

Paper page – Distill Any Depth: Distillation Creates a Stronger Monocular Depth Estimator
https://huggingface.co/papers/2502.19204

“Qwen2.5-VL Technical Report just dropped https://x.com/_akhaliq/status/1892433462910501170

“SIFT Grounding LLM Reasoning in Contexts via Stickers https://x.com/_akhaliq/status/1893862923195310285

“Think Inside the JSON Reinforcement Strategy for Strict LLM Schema Adherence https://x.com/_akhaliq/status/1893861347911217257

“Thinking Preference Optimization https://x.com/_akhaliq/status/1892431954085024189

“Qwen2.5-VL Technical Report was just released https://x.com/arankomatsuzaki/status/1892422884049768473

“Introducing Helix, our newest AI that thinks like a human To bring robots into homes, we need a step change in capabilities Helix can generalize to any household item 🧵 https://x.com/adcock_brett/status/1892577936869327233

“What’s truly exciting is that these robots can now generally pick up any household item For instance, we asked it to “Pick up the desert item” Helix identifies the toy cactus, chooses the nearest hand, and executes precise motor commands to grasp it securely! https://x.com/adcock_brett/status/1892579136956186947

“📣 Come join us at MLSys 2025 Young Professionals Symposium on May 12 in Santa Clara! Submit a 1-page abstract by April 7 and present at the poster session.” / X https://x.com/realDanFu/status/1894576091777700128

“🚀 Day 1 of #OpenSourceWeek: FlashMLA Honored to share FlashMLA – our efficient MLA decoding kernel for Hopper GPUs, optimized for variable-length sequences and now in production. ✅ BF16 support ✅ Paged KV cache (block size 64) ⚡ 3000 GB/s memory-bound & 580 TFLOPS” / X https://x.com/deepseek_ai/status/1893836827574030466

“System prompts are a pretty limited way to control LLM behavior. You can always convince the LLM to leak them (a cause of embarrassment several times in the past) and they tend to react in weird ways with user requests that are unpredictable at best, but often counterproductive.” / X https://x.com/emollick/status/1893778354601443616

“The lack of benchmarks for writing, telling stories, persuasion, creativity, emotional intelligence, perceived empathy, and doing office work are… (1) holding back AI advances, (2) hiding big differences between models & (3) obscuring how good these models are for real work” / X https://x.com/emollick/status/1894410123059089758

“Why did AI benchmarking decide on using consensus@64 or pass@64? Is there an intellectual or statistical basis for choosing 64 trials or even using the pass/consensus approaches?” / X https://x.com/emollick/status/1893047065724264678

“Exa Websets is now live! 🚀 Supercharged search awaits. 👇 https://x.com/ExaAILabs/status/1894446779233702204

Implementing Character.AI’s Memory Optimizations in nanoGPT | njkumarr https://njkumar.com/implementing-characterais-memory-optimizations-in-nanogpt/

“Extracting structured data from unstructured documents is a huge use case for our customers. We’ve just made it a lot simpler with LlamaExtract, now in public beta! LlamaExtract enables you to: ➡️ Define and customize schemas for data extraction, either programmatically or in https://x.com/llama_index/status/1895164615010722233

“There are three real lessons to be learned here: 1) Super-straightforward CUDA code like that has NO CHANCE of ever being faster than optimized cublas kernels. If it is, something is wrong. 2) If your benchmarking results are mysterious and inconsistent, something is wrong. 3)” / X https://x.com/giffmana/status/1892510744224182661

“Scaling pretraining and scaling thinking are two different dimensions of improvement. They are complementary, not in competition.” / X https://x.com/polynoamial/status/1895207166799401178

“A canonical example in DSPy. Decoupling the system’s specification from the mechanics of the ML paradigm of your choice. Even if AI/ML looks very different in 10 years, the system is defined in the same paradigm-agnostic way. Resolves the bitter part of the the bitter lesson. https://x.com/lateinteraction/status/1893783797717504248

“A lesson reconfirmed dozens of times in IR: granular scoring generalizes far better than dense dot products when tasks are hard. Late interaction ftw!” / X https://x.com/lateinteraction/status/1894662639839842346

“You need *late alignment* after all the facts are there — late alignment of tokens for IR, late alignment of downstream systems for DSPy/RL. Precrastination (doing things too soon!) is bad.” / X https://x.com/lateinteraction/status/1894666144055005693

“AI-Toolkit UI is coming along. Most of the hard stuff is done. Just need to tie it all together and clean up the ui a bit. Then I can start adding fun features. https://x.com/ostrisai/status/1892424544356294978

“Deespeek released their MLA implementaion, here is how it works 💡 Multi-head Latent Attention (MLA) speeds up LLM inference and reduces memory needs. It uses “low-rank joint compression” to shrink the Key-Value (KV) to reduce memory usage by up to 93.3% and improve throughput https://x.com/_philschmid/status/1894017216640901302

“🚀 We released the tech report of Qwen2.5VL ( https://x.com/Alibaba_Qwen/status/1892576737848160538

“MoBA, Mixture of Block Attention, from @Kimi_Moonshot improves handling long-context tasks with no fixed attention patterns. Applying ideas from Mixture of Experts (MoE) to attention, MoBA lets the model dynamically decide where to focus. This allows MoBA to be 6.5x faster than https://x.com/TheTuringPost/status/1894711958353629617

Utility Engineering https://www.emergent-values.ai/

“As always, vLLM going at lightning fast speed here and integrating EP ⚡️ https://x.com/reach_vb/status/1894266500271223021

“Fluid Dropdown component built with v0 https://x.com/spoonyvu/status/1889766220569665969

“BBH (big bench hard led by @_jasonwei in the past) was super influential partly because there were a zillion big bench tasks and no one knew what to do with it back then. It’s been 2 years and it might be too easy now. Check out BBEH (extra hard!! 👿) by Mehran (@kazemi_sm)” / X https://x.com/YiTayML/status/1894939679943991661

“The AI CUDA Engineer Archive The team has made available an archive of more than 17000 verified CUDA kernels. These can be used for downstream fine-tuning of LLMs. There is also a website to explore verified CUDA kernels. https://x.com/omarsar0/status/1892621450340921345

“why don’t people build transformers with recursive blocks? instead of the typical transformer block – one softmax (t tokens of dim d) – one MLP (d -> 4d, 4d -> d) we could have – log(d) softmaxes, each with half the dimensionality of the previous – log(D) MLPs, each half as https://x.com/jxmnop/status/1894567121793028158

“Use case 1: Optimal Routing If we know which models are best per-prompt, that makes optimal routing easy! – Performance: P2L-router (experimental-router-0112) is #1 on Chatbot Arena in Jan 2025 with a score of 1395. (+20 than the best model candidate) – We also develop https://x.com/lmarena_ai/status/1894767012490174779

“Use case 2: Domain-Specific Leaderboards P2L can aggregate rankings of prompts within a category to produce an adaptive category ranking → e.g., Find the best models for SQL queries instantly! https://x.com/lmarena_ai/status/1894767014767673744

“vLLM v0.7.3 now supports @deepseek_ai’s Multi-Token Prediction module! It delivers up to 69% speedup boost. You can turn it on with `–num-speculative-tokens=1` and an optional `–draft-tensor-parallel-size=1`. We saw 81-82.3% acceptance rate on the ShareGPT. https://x.com/vllm_project/status/1892646680719216960

“👀 We just merged initial EP support in vLLM https://x.com/vllm_project/status/1894215122966507801

“@LinusEkenstam i dont think they lied at all! They were extremely clear in the blog post that they were providing consensus@64. Could they have been clearer in the original release what the 2 colors were? Yes but it was obvious whatever the 2 modes (that correspond to the 2 colors) were, the https://x.com/nrehiew_/status/1892469273446035924

“this is huge, Hugging Face Inference providers now supports over 8 different providers including sambanova, together, fireworks, hyperbolic, replicate, fal and more and close to 100 models https://x.com/_akhaliq/status/1892628229871030602

“We further slashed memory usage of GRPO (the algorithm behind R1) in @UnslothAI to 90% savings! 20K context length GRPO with Unsloth uses 54GB vs 510GB in other trainers! I coded up a memory efficient GRPO algo & found some cool things about GRPO as well! 1. Reverse KL, https://x.com/danielhanchen/status/1892643424538595611

“🚀 Day 3 of #OpenSourceWeek: DeepGEMM Introducing DeepGEMM – an FP8 GEMM library that supports both dense and MoE GEMMs, powering V3/R1 training and inference. ⚡ Up to 1350+ FP8 TFLOPS on Hopper GPUs ✅ No heavy dependency, as clean as a tutorial ✅ Fully Just-In-Time compiled” / X https://x.com/deepseek_ai/status/1894553164235640933

““Heteroscadasticity” is not a word that has an easily intuitable meaning. “Scadasticity” sounds like something from Kung Fu Panda. https://x.com/ID_AA_Carmack/status/1894852822706872658

“I’m very excited about this work! If SAEs work we hypothesised they should help probe in difficult regimes, but in 5 regimes and 100+ datasets linear probes won This was a negative update on SAEs for me and highlights the value of grounding interpretability with downstream tasks” / X https://x.com/NeelNanda5/status/1894749262757634405

“”Training, concatenation, and retrieval are three different ways to condition the LM on stuff — and they pose very different tradeoffs. We need all three.” All three are still necessary a year later, but I wonder whether we actually need a fourth mechanism beyond those.” / X https://x.com/lateinteraction/status/1894669414454485033

“Composition and abstraction are the most foundational ideas in computer science, yet they’re conspicuously absent in modern ML’s perception of itself. This is not because they’re not crucial for intelligent software. It’s only because it takes a lot of experience building” / X https://x.com/lateinteraction/status/1894719046760968550

“Some quick thoughts: On why we gave the ColBERT paradigm the name “late interaction” instead of “multi-vector”, a term that emerged later and that has proven to be more intuitive. **The mechains is actually not about having multiple vectors at all.** You can see this in four” / X https://x.com/lateinteraction/status/1894696983077785980

“It’s always interesting seeing a more distant outside impression of my work. Work on AI safety will likely increasingly touch on econ and psychology. I talk more about making deals with in a recent podcast. (Linked in thread.)” / X https://x.com/RyanPGreenblatt/status/1894515270108283368

“Fair enough. Here’s my compilation of all results from relevant sources on AIME 2025 performance of Grok and OpenAI models, plus extrapolations of cons@64 for DeepSeek models and o1. I think this is significantly easier to understand than chart crimes of these frontier labs. https://x.com/teortaxesTex/status/1892471638534303946

“does anyone have a good explanation for why the MLP in transformers has to project the representation up to a much larger dimensionality, then back down again? i’m trying to figure out why all the weight matrices in a Transformer can’t just be square” / X https://x.com/jxmnop/status/1894527828630147562

“Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case! P2L trains an LLM to generate “prompt-specific” leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt. The model is trained on the 2M https://x.com/lmarena_ai/status/1894767009977811256

Exa Websets
https://websets.exa.ai/

“Learning to Reason from Feedback at Test-Time This new paper proposes a new paradigm called FTTT (Feedback-based Test-Time Training) that enables LLMs to learn iteratively from environment feedback during inference. Key highlights include: • Test-time optimization for https://x.com/dair_ai/status/1894419591780340065

“@deepseek_ai Love the DualPipe diagram in the paper as well! I added the other 2 in the table for comparison – 1F1B and ZB1P https://x.com/danielhanchen/status/1894937006352031832

“Stay up to date with the latest SholtoBench results via the official benchmark website at https://x.com/nearcyan/status/1892470292758614148

“In the past 2-3 years, pretraining researchers learned a more bitter lesson. Model performance hinges on rigorous data curation. Now, post-training researchers are learning the same lesson – but they don’t have access to high quality annotations. A short thread and essay…” / X https://x.com/sarahcat21/status/1894413202706022570

“⚖️ Start Evaluating LLMs with OpenEvals ⚖️ Evals are a vital part of bringing LLM apps to production. To make it easier to get started, today we’re announcing a new OSS repo: OpenEvals! OpenEvals contains prebuilt evaluators that help you quickly add evals to your app, https://x.com/LangChainAI/status/1894821108018262297

Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs 49 This paper contains model-generated content that might be offensive. 49 https://martins1612.github.io/emergent_misalignment_betley.pdf

“GneissWeb: Preparing High Quality Data for LLMs at Scale “In this paper, we introduce GneissWeb, a large dataset yielding around 10 trillion tokens that caters to the data quality and quantity requirements of training LLMs.” “The GneissWeb dataset consists of 10T high quality https://x.com/iScienceLuvr/status/1893869596379365452

“🧠 NaturalReasoning: A new cool instruction dataset dropped LLMs still struggle with complex reasoning across diverse domains. This paper presents a new way to create better instruction data to improve their reasoning capabilities without human annotation. → Quality beats https://x.com/maximelabonne/status/1892539204875227642

“LightThinker: Thinking Step-by-Step Compression Efficient reasoning is all we need! Jokes aside, there’s been a lot of interesting and novel papers on improving reasoning efficiency. This new paper proposes a novel approach to dynamically compress reasoning steps in LLMs, https://x.com/omarsar0/status/1894068783700218205

“While Amplify strongly believes that FMs will transform drug development, this can’t happen w/o high quality data generated from sophisticated approaches. Vevo Therapeutics is not only unlocking data, but also future cures, with the OSS release of the Tahoe-100M dataset.” / X https://x.com/sarahcat21/status/1894784421611680209

“Desire #4: More of you should be taking advantage of the techniques I outline in the RetroInstruct Guide To Synthetic Data to add useful mental motions and transformations to the English corpus. I know you’re all creative enough for it. If everyone did one we’d have a great set.” / X https://x.com/jd_pressman/status/1893792595228107064

Tencent/llm.hunyuan.turbo-s https://github.com/Tencent/llm.hunyuan.turbo-s

“Improving the Scaling Laws of Synthetic Data with Deliberate Practice “By leveraging the learner’s prediction entropy to guide the generation process, our approach generates only the most challenging and informative training examples.” “DP can improve the validation accuracy https://x.com/iScienceLuvr/status/1893866542565392511

“AIs are both incredibly accessible because anyone can use them and incredibly complicated in terms of understanding their abilities & failure states. Even very smart people misunderstand what LLMs can do in fundamental ways. Fixing this through better training & tools would help” / X https://x.com/emollick/status/1894514021979611563

[2502.18394] The FFT Strikes Back: An Efficient Alternative to Self-Attention https://arxiv.org/abs/2502.18394

“Github 👨‍🔧: MoBA: Mixture of Block Attention for Long-Context LLMs ————- → Addresses the quadratic computational complexity of traditional attention in long-context LLMs. → Introduces Trainable Block Sparse Attention, dividing context into blocks for efficient https://x.com/rohanpaul_ai/status/1893067589867446549

“Github 👨‍🔧: The media player for language learning, with dual subtitles, AI-generated subtitles, realtime-OCR, translation, word lookup, and more! ————- → Displays two subtitles at once for comparative language learning. → Generates automatic subtitles in real time https://x.com/rohanpaul_ai/status/1893077345961156672

[2502.17419v1] From System 1 to System 2: A Survey of Reasoning Large Language Models https://arxiv.org/abs/2502.17419v1

“I started Thinking Machines Lab alongside a remarkable team of scientists, engineers, and builders. We’re building three things: – Helping people adapt AI systems to work for their specific needs – Developing strong foundations to build more capable AI systems – Fostering a” / X https://x.com/miramurati/status/1891918876029616494

[2502.15589] LightThinker: Thinking Step-by-Step Compression https://arxiv.org/abs/2502.15589

adenta/fire_red_agent https://github.com/adenta/fire_red_agent

“An Efficient Alternative to Self-Attention This paper presents FFTNet, a framework that replaces costly self-attention with an adaptive spectral filtering technique based on the Fast Fourier Transform (FFT). Key components: • Global token mixing via FFT – Instead of pairwise https://x.com/omarsar0/status/1894757821587296614

“The metrics argument going on right now over AI benchmarks is ridiculous because any one benchmark is pretty sketchy in the first place. It is the overall trend across models & benchmarks that is more important and interesting than the difference between any two recent models.” / X https://x.com/emollick/status/1892441931545821442

Various approaches to parallelizing Muon · main https://main-horse.github.io/posts/parallelizing-muon/

“This example from their paper ( https://x.com/main_horse/status/1892446384910987718

“RAD Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning https://x.com/_akhaliq/status/1892435007412621429

[2502.14698v1] General Uncertainty Estimation with Delta Variances https://arxiv.org/abs/2502.14698v1

“Introducing Elicit Systematic Reviews! Elicit now supports automated search, title & abstract screening, and data extraction, all in one step-by-step flow. We built this to accelerate power researchers without asking them to sacrifice control. You can do the whole thing https://x.com/elicitorg/status/1892592908563534221

[2502.13909v1] Lost in Sequence: Do Large Language Models Understand Sequential Recommendation? https://arxiv.org/abs/2502.13909v1

“Our AI team working on Helix is scaling up considerably > Helix, Training Infra > Helix, Large Scale Training > Helix, Manipulation Engineer > Helix, Large Scale Model Evals > Helix, Reinforcement Learning https://x.com/adcock_brett/status/1892579357182345588

“Wild numbers. If you plot trajectory of the non-stochastic diffusion sampling, 99.8% of the latent of the entire trajectory can be explained with first two principle components. Roughly speaking, your entire diffusion trajectory is 99.8% two dimensional. https://x.com/cloneofsimo/status/1894086577632284975

“Large Language Diffusion with Masking (LLaDA) are here – and their generation looks so fucking dope! 🤯 True to @ylecun’s vision, Ditch the auto-regressive bits and approximate the language distribution via Maximum Likelihood Estimation! So cool to watch the model denoise text https://x.com/reach_vb/status/1894847408619794493

“o3-mini-high figured out the issue with @SakanaAILabs CUDA kernels in 11s. It being 150x faster is a bug, the reality is 3x slower. I literally copy-pasted their CUDA code into o3-mini-high and asked “what’s wrong with this cuda code”. That’s it! Proof: https://x.com/giffmana/status/1892510741242036468

“This is true in spirit but RL over foundation models is normally explored at a heinous level of abstraction. This is the main blocker for progress IMO. We have all the AI/ML primitives (search & learning) we need. We just need reliable composition and meta-programming. Every” / X https://x.com/lateinteraction/status/1893778310553030763

“🐬 Just released DolphinFlow v0.1.0 – my new PyTorch optimizer! Drop-in replacement for your existing optimizers, improves stability, reduces overfit, and encourages grokking. Try it today: `pip install dolphinflow` https://x.com/cognitivecompai/status/1895030753022431686