‘It gets more and more confused’: can AI replace translators? | Books | The Guardian
“Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models Author’s Explanation:
“DeepSeek is back! JanusFlow 1.3B – Unified multimodal LLM 🔥 > Key Finding: Rectified flow can be trained within the large language model framework without complex modifications. > Base Model: Built on DeepSeek-LLM-1.3b-base. > Vision Encoder: SigLIP-L, supports 384 x 384
[2411.04996] Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models
[2411.04732] Convolutional Differentiable Logic Gate Networks
“@Saraht0n1n But I disagree with the premise if that is it. Rather I take issue with the logical order; it matters whether AI systems we are building and likely to continue building (and which are protested by people who plead to stop scaling) belong to the class of objects for which this” / X
“Introducing Predicted Outputs—dramatically decrease latency for gpt-4o and gpt-4o-mini by providing a reference string.
Are people speedrunning training GPTs now? : r/LocalLLaMA
[2411.06959v1] ENAT: Rethinking Spatial-temporal Interactions in Token-based Image Synthesis
[2411.08378v1] Physics Informed Distillation for Diffusion Models
[2411.07462v1] MureObjectStitch: Multi-reference Image Composition
“timm is known for models, lots of image encoders / backbones. There are also many useful optimizers, all tested but not necessarily passing the high bar of inclusion to torch.optim. After recently working on including the Adopt and Big Vision variant of Adafactor, I decided to” / X
“Top-rated papers from ICLR 2025 Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imposing Consistent Light Transport – Rating: 9.0 –
“🚀 Introducint JanusFlow: harmonizing autoregressive LLMs with rectified flow! By adopting the best practices in both fields, JanusFlow excels at both image understanding & generation in a single model. ⚡️ Powerful, simple, flexible, & your next-gen of Janus is here! 🔥 📄
“Automatically generate cloud configurations with RAGformation! 🏗️ Describe your use case in natural language, get a tailored cloud architecture 🖼️ Visualize your setup with dynamically generated flow diagrams 💰 Receive pricing estimates for the generated architecture 🔄 Refine
“Whether you’re at #EMNLP2024 in person or following from your feed, here are 5️⃣ research papers being presented by AI research teams at Meta to add to your reading list. 1️⃣ Distilling System 2 into System 1:
“[2/7] We first study the common technique of post-train quantizing model weights, finding that the longer you train/the more data seen during pretraining, the more sensitive the model becomes to quantization at inference-time, explaining why Llama-3 may be harder to quantize. In
Nexusflow.ai | Blog :: Introducing Athene-V2: Advancing Beyond the Limits of Scaling with Targeted Post-training
“This is the most important paper in a long time . It shows with strong evidence we are reaching the limits of quantization. The paper says this: the more tokens you train on, the more precision you need. This has broad implications for the entire field and the future of GPUs🧵
“There is a lot of energy going into fine-tuning models, but specialized medical AI models lost to their general versions 38% of the time, only won 12%. Before spending millions on specialized training, might be worth exploring what base models can do with well-designed prompts.
“It’s just a browser extension – so anyone can report bugs w/ technical details. Even after the bug just happened! You can click instant replay, and Jam will create a detailed report with real-time data and video up to the last 2 minutes.” / X
“This paper, “Scaling Laws for Precision” [1], is pretty cool and has some important insights. For example, they say “the more data seen during pretraining, the more sensitive the model becomes to quantization at inference-time” [2], which has real implications for data curation,” / X
“But yes saturation is normal — and in general this has what has been most suprising about the huge emphasis on scaling blindly and treating scaling laws as if they are reliable rules to govern future progress. I have had this as a grumpy part of all my talks for the last few” / X
“[1/7] New paper alert! Heard about the BitNet hype or that Llama-3 is harder to quantize? Our new work studies both! We formulate scaling laws for precision, across both pre and post-training
[2401.00448] Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws
“That is: Feel free to compete with o1, or to just use models as infrastructure on top of which you build ML software. Either way, please don’t turn the academic or OSS communities into noisy replication machines for black-box work from closed labs.” / X
“The way Discord is gaining use in so many communities makes me daydream about a parallel universe where IRC instead of HTTP became the dominant protocol for information exchange in society. Chat rooms over web pages. Chat apps over web apps, etc.” / X
“PureML uses LLMs to automatically clean up and refactor ML datasets 🧼🤖 🕳️ Context-aware null handling 🔍 Intelligent feature creation from existing data 🔄 Data consolidation for consistency PureML leverages LlamaIndex, @OpenAI’s GPT-4, and @getreflex to create an efficient,
Speculations on Test-Time Scaling (o1) – YouTube
“Does Incorrect Reasoning + Explanation help improve Chain-of-Thought (CoT) prompting? A new paper measures how “extended” few-shot CoT prompting with both correct and incorrect reasoning paths boosts LLM reasoning. Implementation 1️⃣ Select your base LLM (e.g. GPT-3.5, GPT-4,
“I’m a bit confused by the “scaling is over” thing. Nobody really believed we’ll get the full-fledged reasoning and creative AI god machine by training a ginormous 2019 GPT on 100x more data, right?” / X
Isotonic regression – Joseph Salmon
“Clarification Regarding AutoGen Forks” / X
“Our own @bhancock_ai just dropped a new tutorial! 🚀 🔁 CrewAI Flows: AI Feedback Loop 🔁 Brandon covers how to set up a flow that evaluates its outputs, iterates with feedback, and polishes each result until it meets your standards! Watch the full tutorial here:
“Gathering stakeholder requirements is a big part of data engineering. In the Data Engineering certificate you get to watch @colleenfotsch and Joe Reis have a simulated conversation so you can see how they actually go! Learn more about the program ➡️
“Program synthesis and test-time fine-tuning of a DL model are conceptually similar: in both cases, you start from a preexisting bank of reusable functions, and you recombine those functions at test time into a program adapted to a new task. They are however at different points” / X
“Something more important than looking at the data is using your own product” / X
“Software engineering 101, lesson 1: if it’s not broken, don’t fix it!” / X
“🚨 Adaptive Decoding via Latent Preference Optimization 🚨 – New layer added to Transformer, selects decoding params automatically *per token* – Learnt via new method, Latent Preference Optimization – Outperforms any fixed temperature decoding method, choosing creativity or
“If you’re used to `htop`, drop everything and install `btop` real quick. Maybe alias it too. You’ll thank me later. btop is beautiful! For the typical person: $ sudo apt install btop $ echo “alias htop=btop” >> ~/.bashrc
“An LLM’s IQ is a type error. LLMs don’t preserve the positive manifold of correlated scores human IQ summarizes. An LLM’s IQ is 160 or 60 depending on the task and no human of any IQ is similar. The jagged frontier is not a chalk outline of the midwit mind.” / X
“Any code that has no tests is broken code. There are a few skills more important than learning how to evaluate LLM-powered applications. Building an LLM demo is straightforward, but every professional developer can tell you how hard it is to build production-grade systems that
“Very critical work given widespread quantization of llms — efficiency introduces disparate trade-offs ⚖️ across languages. Led by amazing @cheeesio with @TheyCallMeMr_ @hongyucharlie @d_aumiller @ahmetustun89 @seb_ruder Talk to @d_aumiller and @ahmetustun89 presenting at” / X
Why the deep learning boom caught almost everyone by surprise
“I say this sincerely, you build a strong intuition for models that are just “overfit” eg they can only do well on the benchmark format and it is literally vibes based (hasn’t let me down yet)” / X
Leave a Reply