“InfiniteHiP Extending Language Model Context Up to 3 Million Tokens on a Single GPU https://x.com/_akhaliq/status/1890443550426300769
“What is Mixture-of-Mamba (MoM)? MoM expands Mixture-of-Experts (MoE) concept on State Space Models SSMs). This development brings a new architecture that can handle all modalities by applying modality-aware sparsity inside the core of the Mamba block. ▪️ What is this https://x.com/TheTuringPost/status/1892695756290834941
“Scaling Test-Time Compute Without Verification or RL is Suboptimal “In this paper, we prove that finetuning LLMs with verifier-based (VB) methods based on RL or search is far superior to verifier-free (VF) approaches based on distilling or cloning search traces, given a fixed https://x.com/iScienceLuvr/status/1891839822257586310
“Update: Combining evolutionary optimization with LLMs is powerful but can also find ways to trick the verification sandbox. We are fortunate to have readers, like @main_horse test our CUDA kernels, to identify that the system had found a way to “cheat”. For example, the system” / X https://x.com/SakanaAILabs/status/1892992938013270019
“self-training for reasoning & retrieval before it was cool + some thoughts on what novelty means in modern ML ColBERT-QA: https://x.com/lateinteraction/status/1892687808508387483
Academic researchers find a way to train an AI reasoning model for less than $50 https://techxplore.com/news/2025-02-academic-ai.html
“A simple reimpl. of “Inference-time scaling diffusion models beyond denoising steps” 🤗 I did the simplest random search strategy but results can be improved w/ better guided search. Supports Gemini 2 Flash & Qwen2.5 as verifiers for “LLMGrading”. Code & comments ⬇️ https://x.com/RisingSayak/status/1890223516773167375
“Is Noise Conditioning Necessary for Denoising Generative Models? “Motivated by research on blind image denoising, we investigate a variety of denoising-based generative models in the absence of noise conditioning. To our surprise, most models exhibit graceful degradation, and in https://x.com/iScienceLuvr/status/1892053059221717486
“Large Language Diffusion Models Presents LLaDA, a 8B diffusion LM, trained entirely from scratch, rivaling LLaMA3 8B in performance despite being trained on 7x fewer tokens (2T tokens). https://x.com/arankomatsuzaki/status/1891343406334693879
“in what sense is this diffusion? I see no SDE, no probability flow, no noise. not every iterative sampling method is diffusion! this paper is genuinely impressive but it’s a new thing I don’t see how I would port diffusion intuitions over to it.” / X https://x.com/gallabytes/status/1891356261582557438
[2502.08391v1] ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification https://arxiv.org/abs/2502.08391v1
“Diffusion Models without Classifier-free Guidance Directly learning the modified score from classifier-free guidance during training, leading to faster convergence and eliminating the need for two model forward passes during inference. Achieves new SOTA FID on ImageNet 256×256 https://x.com/iScienceLuvr/status/1891847953087619147
“Current diffusion-based video generation is computationally and memory intensive, making it inaccessible on smartphones. This paper introduces On-device Sora. It is a pioneering solution to enable diffusion-based text-to-video generation directly on mobile devices. On-device https://x.com/rohanpaul_ai/status/1891408704496652446
[2502.12154v1] Diffusion Models without Classifier-free Guidance https://arxiv.org/abs/2502.12154v1
Large Language Diffusion Models https://ml-gsai.github.io/LLaDA-demo/
“Large Language Diffusion Models (LLaDA) Proposes a diffusion-based approach that can match or beat leading autoregressive LLMs in many tasks. If true, this could open a new path for large-scale language modeling beyond autoregression. More on the paper: Questioning https://x.com/omarsar0/status/1891568386494300252
“Forget “tapestry” or “delve” these are the actual unique giveaway words for each model, relative to each other. https://x.com/emollick/status/1892047167713427687
“The AI-native (edge and LLM) proxy for agents. Move faster by letting Arch handle all the pesky heavy lifting in securing, processing, routing, and tracing prompts. Built by the contributors of Envoy. Key features include: 🛡️ Guardrails at the edge: reject jailbreak attempts https://x.com/_akhaliq/status/1891289618978316749
“@Entelligence ❤️’s OSS! You can now understand any codebase with realtime documentation, tutorials, codebase chat, code reviews and more by simply changing https://x.com/Aiswarya_Sankar/status/1890502637360992451
“We’ve been cooking a bunch of improvements to @lovable_dev’s @supabase integration for a while Today we’re introducing ALL these improvements: //1 https://x.com/antonosika/status/1889005519928463442
“MLX uses nanobind (by @wenzeljakob) to bind C++ to Python. It’s a critical piece of MLX infra and is why running Python code is nearly the same speed as running C++ directly. Also makes it super easy to move arrays between frameworks. Examples: https://x.com/awnihannun/status/1890495434021326974
Chatbot Arena (formerly LMSYS): Free AI Chat to Compare & Test Best AI Chatbots https://lmarena.ai/
“AI labs have products that people increasingly rely on for serious things & build workflows around. Every update breaks some of those and enables new ones Provide a changelog, testing info, anything indicating what happened! Mysterious drops are fun for X, bad for everyone else. https://x.com/emollick/status/1890822253274706046
HadaCore: Tensor Core Accelerated Hadamard Transform Kernel | PyTorch https://pytorch.org/blog/hadacore/
“I’m in NY for AI Eng Summit! Who else is around?” / X https://x.com/HamelHusain/status/1892025630079787214
“FastHTML.” / X https://x.com/jeremyphoward/status/1893013559904477604
“Reinforcement Learning Quickstart Guide” / X https://x.com/jsuarez5341/status/1854855861295849793
“@nearcyan well there’s a difference between prompting with chain-of-thought (which people were doing mostly with base models i think) and RL on chain-of-thought which is what is becoming more popular recently” / X https://x.com/iScienceLuvr/status/1892863034865177038
“Improving the Diffusability of Autoencoders “In this work, we perform a spectral analysis of modern autoencoders and identify inordinate high-frequency components in their latent spaces, which are especially pronounced in the autoencoders with a large bottleneck channel size. We https://x.com/iScienceLuvr/status/1892877363958206766
“The best context length is < 32k. You will get very good quality output when it’s below this. If your task can be decomposed into sub tasks that are each under that context then you can have decent working agents” / X https://x.com/abacaj/status/1893025241078665238
[2502.13923] Qwen2.5-VL Technical Report https://arxiv.org/abs/2502.13923
Paper page – An Open Recipe: Adapting Language-Specific LLMs to a Reasoning Model in One Day via Model Merging https://huggingface.co/papers/2502.09056
X-Dyna https://x-dyna.github.io/xdyna.github.io/
“@barret_zoph and I recently gave a talk at Stanford on post-training and our experience working together on ChatGPT. Unfortunately the talk wasn’t recorded, but here are the slides: https://x.com/johnschulman2/status/1891539960743743756
“The performance of LLMs in code generation significantly drops for low-resource programming languages due to limited training data. This paper explores methods to bridge this performance gap. Proposes and compares in-context learning and fine-tuning strategies to enhance LLM https://x.com/rohanpaul_ai/status/1892177949354295652
[2502.07316] CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction https://arxiv.org/abs/2502.07316
Paper page – SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models https://huggingface.co/papers/2502.09604
[2502.08524] LLM Pretraining with Continuous Concepts https://arxiv.org/abs/2502.08524
[2502.08923v1] CopySpec: Accelerating LLMs with Speculative Copy-and-Paste Without Compromising Quality https://arxiv.org/abs/2502.08923v1
[2502.11986] Selective Task Group Updates for Multi-Task Optimization https://arxiv.org/abs/2502.11986
“Github 👨🔧: A Comprehensive Toolkit for High-Quality PDF Content Extraction → Integrates leading document parsing models for layout detection, formula detection, formula recognition, OCR, and table recognition. → Achieves high-quality parsing across diverse document types due https://x.com/rohanpaul_ai/status/1890552874155090388
“The problem is that merging multiple task-specific models into one multi-task model still results in a performance drop compared to individual task-specific models. This paper proposes “Isotropic Merging”. It flattens the singular value spectrum of merged task matrices to https://x.com/rohanpaul_ai/status/1891407113034850656
“Learning to Reason at the Frontier of Learnability “we adapt a method from the reinforcement learning literature—sampling for learnability—and apply it to the reinforcement learning stage of LLM training. Our curriculum prioritises questions with high variance of success, i.e. https://x.com/iScienceLuvr/status/1892156610304438297
“This new paper shows people could not tell the difference between the written responses of ChatGPT-4o & expert therapists, and that they preferred ChatGPT’s responses. Effectiveness is not measured. Given that people use LLMs for therapy now, this is an important topic for study https://x.com/emollick/status/1890649701185130654
“Current sub-branch quantization techniques often lead to overfitting and increased latency. This paper introduces Feedback Quantization, called FBQuant, to address these issues. FBQuant uses a feedback mechanism and kernel fusion to optimize sub-branch quantization. This https://x.com/rohanpaul_ai/status/1891043872517808196
[2502.09316v1] A Judge-free LLM Open-ended Generation Benchmark Based on the Distributional Hypothesis https://arxiv.org/abs/2502.09316v1
“Score-of-Mixture Training: Training One-Step Generative Models Made Simple “We propose Score-of-Mixture Training (SMT), a novel framework for training one-step generative models by minimizing a class of divergences called the α-skew Jensen–Shannon divergence. At its core, SMT https://x.com/iScienceLuvr/status/1890319794790101098
“This paper addresses the issue of inefficient inference when using long contexts in transformer models. Introduces a method to reduce inference costs. It attends only to the most important tokens at each step using a top-k selection, enhancing efficiency for long contexts. https://x.com/rohanpaul_ai/status/1892178684473414061
“This Paper introduces a Data-Efficient Approach to Long Chain-of-Thought Reasoning for LLM Paper addresses how large language models can reliably generate detailed chain-of-thought with minimal data overhead They employ supervised fine-tuning or LoRA with only 17k structured https://x.com/rohanpaul_ai/status/1891044576485310875
“What if we could change time with the click of a button? Built this component using @v0, @mattgperry Motion library, and @paper for the background gradients. Probably the longest I’ve spent on a single component😅 Now, time to package it into the library! https://x.com/tonyblu331/status/1889332320328417426
“it’s been over two years since arxiv papers were published on it: https://x.com/nearcyan/status/1892862381803655240
[2502.13144] RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning https://arxiv.org/abs/2502.13144
[2502.12618] Uncertainty-Aware Graph Structure Learning https://arxiv.org/abs/2502.12618
“My sister’s exams were coming up, but she kept getting distracted by ads while listening to study music. So, I built her an ad-free productivity app overnight using @Replit Lately, I’ve been creating apps left and right on Replit, huge thanks to @amasad for making it so easy! https://x.com/idleshubh/status/1889398006912585879
“The paper addresses the perplexing phenomenon of “grokking” in neural networks. Networks initially memorize training data but suddenly generalize to unseen data after extended training. The paper explores how training data size, model architecture, and optimization methods https://x.com/rohanpaul_ai/status/1891043560138383824
Animate Anyone 2 https://humanaigc.github.io/animate-anyone-2/
[2502.09254v1] AnomalyGFM: Graph Foundation Model for Zero/Few-shot Anomaly Detection https://arxiv.org/abs/2502.09254v1
“A very interesting study from Korean @kaist_ai and DeepAuto AI introduced InfiniteHiP. It’s an inference framework that handles long context up to 3M tokens on a single GPU with ~19x speed boost. But how? The key improvements are: • Offloading memory — it moves less-used data https://x.com/TheTuringPost/status/1890346417560121430
“First LLM benchmark of its kind? > ZSEval uses multiplayer games to pit models against each other on knowledge, reasoning, and planning. > It uses DSPy optimization to test the self-improvement capability of models and ensure the competition between models is fair.” / X https://x.com/lateinteraction/status/1891288503201259843
[2502.12678] Multi-Step Alignment as Markov Games: An Optimistic Online Gradient Descent Approach with Convergence Guarantees https://arxiv.org/abs/2502.12678
“LLMs leak up to 27.5% of sensitive training data PII (Personally Identifiable Information like emails, SSNs, VINs, Bitcoin wallets). @Secludy makes it easy to generate privacy-guaranteed synthetic data that is a near replica of the original unstructured dataset but better. How? https://x.com/rohanpaul_ai/status/1892314450478104825
[2502.11896] CAMEL: Continuous Action Masking Enabled by Large Language Models for Reinforcement Learning https://arxiv.org/abs/2502.11896
“🚨 New paper & dataset! 🚨 NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions – Synthesizes 2.8M challenging and diverse questions which require multi-step reasoning, along with reference answers – Shows steeper data scaling curve for knowledge distillation https://x.com/jaseweston/status/1892041992127021300
jina-ai/node-DeepResearch: Keep searching, reading webpages, reasoning until it finds the answer (or exceeding the token budget) https://github.com/jina-ai/node-DeepResearch
“If you’re curious why I didn’t tweet about Sakana’s CUDA kernel paper, it’s because it was just too fishy for my taste.” / X https://x.com/arankomatsuzaki/status/1892999981209956623
“No matter whether you like the benchmark or not. #1 across all categories is impressive” / X https://x.com/scaling01/status/1891790801631314103
Paper page – The Stochastic Parrot on LLM’s Shoulder: A Summative Assessment of Physical Concept Understanding https://huggingface.co/papers/2502.08946
“The paper addresses the problem of inefficient training of large models due to the high variance inherent in stochastic gradient methods, especially for LLMs. Existing variance reduction techniques have not been widely adopted in deep learning. This paper introduces MARS (Make https://x.com/rohanpaul_ai/status/1891404523757076688
“Grokking is when a model initially memorizes training data and overfits, but later, after prolonged training, it suddenly generalizes well to unseen data. This paper proposes to explain grokking using concepts from thermodynamics and stochastic gradient Langevin dynamics. It https://x.com/rohanpaul_ai/status/1891047293446422823
“MVP that started as a tweet from @NWischoff to a fully functional prototype using AgentStack from @AgentOpsAI If you know anyone in the loan approval space, tag them below” / X https://x.com/AtomSilverman/status/1890132937150197909
“Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity https://x.com/arankomatsuzaki/status/1892059638943637960
[2502.13063] Cramming 1568 Tokens into a Single Vector and Back Again: Exploring the Limits of Embedding Space Capacity https://arxiv.org/abs/2502.13063
“This paper explores the energy consumption and performance trade-offs when fine-tuning different sized language models for research highlight generation. The study compares two smaller pre-trained language models and a larger LLM to analyze their environmental impact alongside https://x.com/rohanpaul_ai/status/1891046780583784515
“@IgorIlyinsky Equivalent to the best possible model right now (o3-full with deep research) for 30 bucks instead of 200$” / X https://x.com/Teknium1/status/1891726810494210449
“Axolotl v0.7.0 is out! – GRPO support – Process Reward Model support – KD Training from offline top-k logprobs – Multi-GPU LoRA kernels – Deploy your training and evaluation workloads straight to Modal from the axolotl CLI – Sweeps – Chat template parsing improvements – Improved” / X https://x.com/winglian/status/1891933173858304413
Exploring the structural changes driving protein function with BioEmu-1 – Microsoft Research https://www.microsoft.com/en-us/research/blog/exploring-the-structural-changes-driving-protein-function-with-bioemu-1/
