Multimodal: AI News Week Ending 09/04/2026

Every week, I organize 400 to 700 links into roughly 60 categories as part of my ongoing effort to learn about AI. This is my personal notebook, which I enjoy sharing with friends… a hobby and a labor of love, rather than a commercial publication or product.

If you arrived here through a search or shared link, this page collects the links I found for Multimodal for the week ending September 4, 2026.

As part of my learning process, I like to automate the category covers. It gives me a chance to learn Python and APIs.

This week’s cover prompt was written using Claude Opus 4.7, and the image was generated using Nano Banana 2.1.

Category Cover Image Prompt: A vertical psychedelic screen-printed gig poster of a giant blooming flower whose petals are different sensory icons (eye, ear, speech bubble, hand, filmstrip, musical note) melting and swirling into one another in electric magenta, acid orange, deep teal and lime green on cream paper, with autumn leaves and a harvest moon in an ornate decorative border, and the word Multimodality hand-lettered in bold 1960s Fillmore-style psychedelic title text arcing above the bloom. Flat ink layers with slight registration offset, intricate linework, visible paper texture, no photorealism.

This Week in Multimodal News

Summary by Claude Sonnet 5.5 based on this week’s links:

Atlas from World Labs: World Labs launched Atlas, described as a multimodal world model that generates frames with precise camera control and reconstructs them in 3D. Ben Mildenhall said he built a London museum flythrough from three images found on Google Images.
Real2sim for robotics: Researchers like Jim Fan and Yunzhu Li pointed to Atlas as a step toward turning a few phone photos into a simulator where robots can practice navigation. World Labs co-founder Justin Johnson predicted training a robot for a new environment in minutes from about 5 photos.
Interactive world models: Runway announced GWM Worlds 2, which it says generates real-time interactive video simulations at 720p and 24 fps and merges navigation, manipulation and human interaction. Runway also showed Solaris, a model that generates interfaces frame by frame with no code.

This summary was generated by Claude Sonnet 5.5 to help you explore the links below. Rest assured, I select, organize, and check the links by hand in Google Sheets, and write the introduction and personal commentary in The Main Newsletters myself each week as a labor of love.

This week's links related to Multimodal

10 months between these two visions of how ChatGPT agents work.
https://x.com/emollick/status/2095257172896841964

@DrJimFan @theworldlabs Indeed Jim, a camera conditioned world model with spatial context has so much horizontal usage including real2sim for robotics!
https://x.com/drfeifei/status/2094910083444707551

@theworldlabs Amazing work! Great step towards real2sim for robotics!
https://x.com/DrJimFan/status/2094905169460736291

NEW WORLD MODEL: @ylecun’s team is back with an efficient model! This project involves @ylecun, @lukaskuhn77, @lucasmaes_, @quentinlldc, and @randall_balestr. A couple definitions first: – DINO: self-DIstillation with NO labels. A self-supervised image model (Meta, 2021) where
https://x.com/LeoKharon/status/2094395060636803122

Another glimpse of Real2Sim with Atlas: capture a real environment with just a number of casual photos from a phone, turn them into a sim, then simulate diverse robots navigating different trajectories. Atlas generates the RGB and depth observations they would see along the way.
https://x.com/YunzhuLiYZ/status/2094926835649790103

Cool update by World Labs; I’m most stoked about doing large scale 3d reconstruction using a sparse set of images. That way you capture what’s most important to get right, and AI fills in the rest of the gaps in the scene.
https://x.com/bilawalsidhu/status/2094853004826452337

Introducing Atlas: The world’s first multimodal world model that generates image and video frames with pixel-perfect camera control and reconstructs them in 3D. Model the world, move the camera, and simulate space & time.
https://x.com/theworldlabs/status/2094839756329041984

our new model Atlas also happens to be a capable text-to-image generator, providing it with a strong foundation of “world knowledge”… this generalizes across both single and multi view domains, meaning you can step into even highly stylized scenes with precise 3D control
https://x.com/BenMildenhall/status/2094859820100952575

World Labs co-founder @jcjohnss predicts a real-to-sim-to-real future where Atlas could train a robot for any new environment in minutes using just 5 photos from your phone: “One is this notion of fully in-context learning. Maybe I’ve got a robotics foundation model, then I can
https://x.com/MTSlive/status/2094950206240600308

Today, we’re sharing new research on Solaris, our first Interface World Model. Solaris is a new kind of operating system that generates interactive interfaces frame by frame, in real time, with no code. We find that Solaris outperforms frontier LLMs when generating new
https://x.com/runwayml/status/2094463070466646019

At the limit, world models will become the RL environments for everything. Excited to announce GWM Worlds 2, the first world model with a general action space, merging navigation, manipulation and human interaction in one model. We’re already using it to evaluate
https://x.com/agermanidis/status/2095597719574466676

Atlas is live! A spatial foundation model, trained in-house from scratch. After shipping RTFM last year, I was convinced a single camera-conditioned model could unify most spatial generation tasks. Atlas is that bet at scale. The core idea👇
https://x.com/KeunhongP/status/2094840790061301795

Atlas: A World Model for Spatial Intelligence | World Labs
https://www.worldlabs.ai/blog/atlas

I’m so excited that our @theworldlabs team has achieved a major milestone today! Introducing Atlas – a first of its kind multimodal world model trained from scratch! 🚀 Atlas is capable of generating frames with pixel-perfect camera control, reconstructing large scenes from as
https://x.com/drfeifei/status/2094840371675283673

Runway Research | Introducing GWM Worlds 2
https://runway.com/research/introducing-gwm-worlds-2

Two of us filmed this on a couple phones, and we used Atlas to turn it into a short film. This is just a clip from the @theworldlabs tech blog Atlas is able to generate the parts of the scene that none of the cameras actually filmed. It can freeze time, allowing you to reframe
https://x.com/davidpantera_/status/2094841083805266401

Atlas is out! What excites me most is the 3D capability, months of work with @bardienus : an AR omni model that beats every specialized model at 3D reconstruction. First clip: fly through the Porsche museum, 500+ frames streamed into a single consistent point cloud; feed-forward
https://x.com/HaoZhang623/status/2094841315783754213

🎬Video generation faster than playback! 🚀MiniMax H3 on vLLM-Omni + FastVideo’s FastH3: a complete 10.1s MP4 – video AND synchronized audio – rendered in 8.7s!⚡️ Thanks to @MiniMax_AI for the great Minimax H3 release, the FastVideo team @haoailab for open-sourcing FastH3 and
https://x.com/vllm_project/status/2094849929487552663

You can now use up to 14 references in your videos. Images, voices, character references, and more. Use “@” to tag each reference in your prompt.
https://x.com/imagine/status/2095249317875622255

MAI-Transcribe-2 is the fastest, most accurate and cheapest speech recognition model in the world | Microsoft AI
https://microsoft.ai/news/mai-transcribe-2-is-the-fastest-most-accurate-and-cheapest-speech-recognition-model-in-the-world/

Build with Gemini Omni 1.1 Flash
https://blog.google/innovation-and-ai/technology/developers-tools/build-with-gemini-omni-1-1-flash/

We’re bringing agentic video understanding to our latest Gemini models. They can now analyze videos with better accuracy while using up to 88% fewer tokens. 🧵
https://x.com/GoogleDeepMind/status/2094840179676660097

Astra had made Codex more essential than ever due to its unique strengths. The computer use, vision and self-verification capabilities work incredibly with it. Using it in a terminal almost feels silly
https://x.com/theo/status/2095609789711831286

Wan 3.0 debuts at #1 on the Artificial Analysis Video Editing Leaderboard, and is a close #2 in Text to Video with Audio Wan 3.0 is Alibaba’s new all-in-one video generation and editing model, positioned as a single system for turning multimodal creative direction into video. It
https://x.com/ArtificialAnlys/status/2095349174799888760

125B/6B, 1M context, multimodal, now in OpenCode Go. ⚡ Happy coding with Qwen3.8-Flash!
https://x.com/Alibaba_Qwen/status/2093227357951897687

Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment:
https://x.com/OpenAI/status/1635687373060317185?s=20

Must-read papers of the week ▪️ From Generation to Simulation: How Far Are World Models from Being True Simulators? ▪️ Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning ▪️ Can Video World Models Track Unobserved World States? ▪️
https://x.com/TheTuringPost/status/2094942136865333544

We are sharing a major update to our General World Models efforts. GWM Worlds 2 can generate full, interactive, real-time video simulations in one continuous 720p stream at 24 fps and audio at 48000 Hz, responding to your inputs as you explore. It is the first world model that
https://x.com/c_valenzuelab/status/2095548906281042144

I’m particularly excited by Atlas’s ability to reconstruct scenes from a very small number of input image — was able to create this flythrough of London’s Natural History Museum by combining 3 input images I found from completely separate sources on Google Images
https://x.com/BenMildenhall/status/2094891871730581609

When @ylecun, @demishassabis, and @drfeifei talk about building world models, what do they mean? → They want AI systems that can build an internal map of their surroundings to anticipate what happens next. But that doesn’t mean they’re building the same machine. @kseniase is
https://x.com/TheTuringPost/status/2094573808006938746

One image. A world to train in, shoot in & explore. 🌍 👀Meet HYPER3D #WorldGen: independent foreground meshes + #3DGS background, built for interaction. For robotics🦾, film🎬, games🎮, DCC💎 & XR. 🔥Powered by our Best Paper Award Project 「CAST」. Let’s Connect the Dots⬇️
https://x.com/DeemosTech/status/2094440163246256523

Here’s what launched this week: … Gemini 3.5 Transcribe, our most precise speech-to-text model yet, designed to deliver intelligent transcriptions … Gemini Omni 1.1 Flash, bringing expanded creative capabilities and controls for video generation and editing … @GeminiApp’s Live
https://x.com/GoogleAI/status/2093356767992279503

Photon 2.1 is out, adding text-to-speech models and B200 support to our realtime multimodal inference engine.
https://x.com/vikhyatk/status/2095230035707977947

MiniMax H3: An Open Model Breaking the Boundaries Between Tasks and Modalities – MiniMax Research | MiniMax
https://www.minimax.io/blog/minimax-h3

jeez, kinda bearish for oai? astra seems really strong on vision (which is under-indexed on AA). but i’m very surprised that it’s not a significant leap in coding (despite being an obv larger model)
https://x.com/rishdotblog/status/2095601577918943697

This is what open-source intelligence is for. Built on vLLM-Omni and @haoailab’s FastH3, with @NVIDIAAI hardware support – all public. Real-time generation makes interactive video possible; an open baseline makes it improvable for everyone. Take it and go faster! 🚀
https://x.com/MiniMax_AI/status/2094926136333787512

What used to take years now takes a week. @TAMU’s VISION, an NVIDIA DGX SuperPOD and the most powerful academic supercomputer on the June 2026 TOP500, is giving researchers across seven institutions the compute to tackle challenges at unprecedented scale and speed.
https://x.com/nvidia/status/2093484389682270429

DeepSeek-V4-Flash-Vision-Exp weights are released. Now there’s some feature parity with Moonshot and GLM.
https://x.com/teortaxesTex/status/2094375909868368213

Gemini video understanding is truly bonkers 🤯 Given a 2 hour football match, it scans the video for yellow cards, finds them, 3.8 Flash adds them to a 2D field so you can see where they occurred, then you can click on it and go to the exact part of the video! Check it out 👇
https://x.com/JackWoth98/status/2095520018561630691

We’re rolling out Gemini Omni 1.1 Flash to make generative video highly controllable, faster to iterate on, and more polished for production-grade use. Here’s how you can try it in @FlowbyGoogle and more →
https://x.com/GoogleDeepMind/status/2093338200580256172

Today, we’re excited to launch Extract Turbo ⚡️ – the fastest VLM-powered document extraction solution. It is 3-5x faster than all other comparable OCR solutions, including our other LlamaParse tiers, at equivalent or higher accuracy.
https://x.com/jerryjliu0/status/2095622647375651100

GPT-6 Astra brings stronger visual judgment to front-end design. Give Astra a sketch, reference, or existing UI, and ask it to: • Turn the reference into a working UI • Refine layout, typography, and spacing • Adjust color and interactions • Use screenshots to guide
https://x.com/OpenAIDevs/status/2095596149654868092