Every week, I organize 400 to 700 links into roughly 60 categories as part of my ongoing effort to learn about AI. This is my personal notebook, which I enjoy sharing with friends… a hobby and a labor of love, rather than a commercial publication or product.
If you arrived here through a search or shared link, this page collects the links I found for Multimodal for the week ending August 28, 2026.
As part of my learning process, I like to automate the category covers. It gives me a chance to learn Python and APIs.
This week’s cover prompt was written using Claude Opus 4.7, and the image was generated using Nano Banana 2.1.
Category Cover Image Prompt: A 16:9 mid-1980s murder-mystery movie poster of a wide black-and-white engraved Victorian mansion on pure black, with warm amber windows showing costumed suspects (a chanteuse with microphone, a painter with dripping brush, a film director with clapperboard) and evidence windows crowded with cameras, waveforms, speech bubbles, paint tubes and film reels, plus one window with a comic silhouette of a parrot mid-squawk. The central open front door glows warm, with the same slim silver-haired butler in black tailcoat, pencil mustache and white gloves holding a silver tray with an envelope sealed in wax shaped like three overlapping eye-ear-mouth circles. Above the roofline, fat rounded extra-heavy white sans-serif letters spell MULTIMODALITY like a movie title, dense crosshatching, high-contrast engraving, campy spooky theatrical.
This Week in Multimodal News
Summary by Claude Sonnet 5.5 based on this week’s links:
- Gemini 3.5 Transcribe: Google launched a speech-to-text model with sub-second streaming, 85+ languages and cleanup of filler words like "um" and "ah." Google's Phil Schmid cited 2.6% word error rate on non-streaming and 4.0% on streaming.
- Gemini Omni 1.1 Flash for video: Google's video generation and editing model adds first and last frame control, scene extension, and 360p drafts that upscale to 4K. The Arena leaderboard put it at #1 in Text-to-Video and #2 in Image-to-Video.
- Vision arrives in Chinese models: DeepSeek released an experimental vision version of V4-Flash on its API, and Z.ai launched GLM-5.3-Flash as a natively multimodal model with a 1M-token context window. One tester, though, called GLM 5.3 Flash pretty bad at vision tasks on aerial and satellite images.
This summary was generated by Claude Sonnet 5.5 to help you explore the links below. Rest assured, I select, organize, and check the links by hand in Google Sheets, and write the introduction and personal commentary in The Main Newsletters myself each week as a labor of love.
This week's links related to Multimodal
Deepseek releases experimental Flash vision model that rivals Opus 4.8 on agent benchmarks
https://the-decoder.com/deepseek-releases-experimental-flash-vision-model-that-rivals-opus-4-8-on-agent-benchmarks/
You can just use your voice | ChatGPT Work – YouTube
https://www.youtube.com/watch?v=ceBruD6v5Bk
Make information visual with ChatGPT – YouTube
https://www.youtube.com/watch?v=jzmNh8lbSp8
How to Manage Your Workspace With ChatGPT Work – YouTube
https://www.youtube.com/watch?v=29SyCndnMZs
How to Build a Personalized Meal Planner with ChatGPT Work – YouTube
https://www.youtube.com/watch?v=t64oZKCdG8Q
Build a Shareable Site – YouTube
https://www.youtube.com/watch?v=0qK3KqvtRrg
Plugins & Skills – YouTube
https://www.youtube.com/watch?v=5QPBVrAAdBk
Use Your Computer and Browser – YouTube
https://www.youtube.com/watch?v=981SivztzOc
Talk to ChatGPT Work – YouTube
https://www.youtube.com/watch?v=DLcQSp_CIIc
Getting Started with ChatGPT Work – YouTube
https://www.youtube.com/watch?v=Rk4VyQxDq5s
Create Slides, Docs, and Templates – YouTube
https://www.youtube.com/watch?v=syML5KT-HzI
Scheduled Tasks – YouTube
https://www.youtube.com/watch?v=urXc4xxixVU
Use ChatGPT Work on Your Phone – YouTube
https://www.youtube.com/watch?v=yfnh8SuMbNw
Build agent-ready sites with WebMCP – YouTube
https://www.youtube.com/watch?v=Is2NHa7awWY
Asked God’s Eye View to explain why there’s so much military activity in this area — all using voice, while it sees what I’m seeing on the 3D map. Turns out Fort Rucker is a hot spot for helicopter training. So I jumped into cockpit view to track one of the choppers live in the”
https://x.com/bilawalsidhu/status/2092334246044074237
Introducing Gemini 3.5 Transcribe
https://blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-3-5-transcribe/
Today Hark is announcing a multi-year partnership with NVIDIA with gigawatt-scale compute capacity on next-generation Vera Rubin platforms Hark is building multimodal systems that remember who you are and what you plan to do. Our work with NVIDIA will support frontier model”
https://x.com/adcock_brett/status/2092989234189869211
Introducing GLM-5.3-Flash – Leading capabilities at a highly competitive price – Natively multimodal with a 1M-token context window – A 320B-A18B model released under the MIT License – Previously previewed as Ox Alpha, running entirely on Chinese AI chips Blog:”
https://x.com/Zai_org/status/2092616204787626030
The sense of touch is the most criminally under-explored modality in robotics. Imagine doing sleight of hand wearing thick oven mitts. That’s exactly how a robot feels today if it were alive. A magnetic piece snapping into place, a paper cup peeling out of a stack, a USB”
https://x.com/DrJimFan/status/2090832821036470626
I interviewed Sundar Pichai, and he made an interesting prediction for where we’ll be in 3 years: “Any content can go from any modality to any modality. It doesn’t matter how the content was originally created — people can consume it in many different forms.” “I’m very”
https://x.com/rowancheung/status/2092276652944593257
We’re excited to release the best open-weight model for robotics. Isaac 0.5 is a 36B, 2.5B active per token model, one sparse backbone serving video perception, embodied reasoning and robot control.”
https://x.com/ArmenAgha/status/2092682391794155885
Today, we’re launching Gemini 3.5 Transcribe, our new speech-to-text model with sub-second streaming and intelligent post-processing for agent interfaces. 2.6% WER on non-streaming and 4.0% on streaming. Supports 85+ languages, cleans up conversational disfluencies (“um”, “ah”,”
https://x.com/_philschmid/status/2092659659866030112
Mixedbread is retrieval infrastructure for AI agents to find the right evidence across text, PDFs, tables, images, audio, and videos. We process millions of documents and searches every week. Our control plane runs on @planetscale Metal. → 0.05 ms p99 for our busiest”
https://x.com/mixedbreadai/status/2092654670988628223
DeepSeek-V4-Flash-Vision-Exp is now live on the DeepSeek API Platform! 🚀 🔹 This experimental multimodal model matches DeepSeek-V4-Flash on text capabilities… including agents, reasoning, and world knowledge. 🔹 On multimodal agent benchmarks, V4-Flash-Vision-Exp makes a major”
https://x.com/deepseek_ai/status/2090730032574631962
nac v0.1.3 is out. What’s new: • Project-backed session organization • Skill prompt expansion & composer autocomplete • Sandboxed Git worktrees per session • Vision-aware image reading • Shiki syntax highlighting for code & diffs ➡️ Upgrade now with: nac-web upgrade”
https://x.com/arcee_ai/status/2090821442409562524
GLM-5.3-Flash is live, day 0, on Baseten Model APIs. – Useful for general intelligence + agentic coding – More intelligent and 90% cheaper than GLM-5.2 – Supports vision natively – 1M token context Try it here:
https://x.com/baseten/status/2092720341432799426
The mystery model Ox Alpha is now available in @AutoClawAIer by @Zai_org It’s official name is GLM-5.3-Flash, a next-generation multimodal model for vision-language understanding, code generation, and long-horizon agentic tasks. Limited-time rewards, Aug 27-30: • Log in to”
https://x.com/AutoClawAIer/status/2092650193158389929
This is very nice work from NVIDIA. Like all high-performing approaches on ARC-AGI-3, it uses deep learning-guided on-the-fly synthesis of symbolic world models, i.e. navigating the world by generating programs to represent what you know. To be clear, like with several other”
https://x.com/fchollet/status/2090838046937645398
as expected, GLM 5.3 Flash is pretty bad at vision interesting how different models end up specializing in completely different things some are great at coding, others at computer vision no AGI yet, I guess 1/n aerial and satellite images, crops link:
https://x.com/skalskip92/status/2092748209802154201
Exciting news: Qwen3.8-27B by @Alibaba_Qwen just landed in Code Arena: WebDev at #9 overall with 1595 pts. It is the only model in its size class in the top 10, and also reshapes the Pareto Frontier! It is only 6 ranks behind the much larger Qwen3.8-Max. For scale: Gemma 4-31B”
https://x.com/arena/status/2091920512796725272
2026 Chinese model landscape in one list ▪️ Qwen3.8-27B, Qwen3.8-Max ▪️ GLM-5.3 ▪️ Kimi K3 ▪️ DeepSeek-V4 ▪️ ERNIE 5.1 ▪️ Baichuan-M3-235B, Baichuan-Omni-1.5 ▪️ Yi 1.5 ▪️ MiniCPM-o 4.5, MiniCPM-V 4.5, MiniCPM-V 4.6 They’re getting interesting in very different ways. Here we”
https://x.com/TheTuringPost/status/2091665921849098363
Throwback to the “dinosaur input device” that was used to animate the stellar T-Rex dinosaur in the first Jurassic Park movie. Everything that has happened before will happen again, so now it’s a spatial 3D harness you can control with a Vision Pro and hand tracking.”
https://x.com/bilawalsidhu/status/2091641023487226143
Excited to bring Gemini Omni Flash 1.1 to the world, this is an update to our anything in, anything out world model. It now supports: – 360p drafts, 4K up samplers – up to 10 seconds of video context when extending – video extensions in 10 second increments – video references”
https://x.com/OfficialLoganK/status/2093014204324417881
Joining the MTS stream today at 10am PST to yap about world models and spatial intelligence!”
https://x.com/bilawalsidhu/status/2090836155906289667
Learning How the World Evolves Extrapolative Video World Models via Latent Dynamics Reasoning paper:
https://x.com/_akhaliq/status/2091958146596041142
Thanks so much for sharing! @_akhaliq 🥰 The core idea is to learn how the world evolves, rather than directly predict what the future looks like. LDR maps past frames into structured latent states and rolls them forward with kinematic integration, learning only the higher-order”
https://x.com/haodongli00/status/2091961954562887884
Introducing Gemini 3.5 Transcribe, our new speech to text model with smart transcription, function calling, more precise transcription (lower WER), custom vocabulary support, multi-speaker identification, and support for over 85 languages! Also with realtime streaming support!”
https://x.com/OfficialLoganK/status/2092660925509890397
Say hello to Gemini 3.5 Transcribe! – Build apps that understand user speech / intent, even w/ multiple speakers! – Auto-detection of 85+ languages out of the box – Custom vocab adaptation for specialized jargon… SGTM:) API available now in @GoogleAIStudio and Gemini”
https://x.com/sundarpichai/status/2092659467284517088
We’re introducing Gemini 3.5 Transcribe, our most precise speech-to-text model yet 🔊 It turns audio into precise transcription in 85+ languages, removing filler words like “ums” and “ahs” while handling self-corrections and capturing your intent so you can get things done using”
https://x.com/Google/status/2092659278632894576
Big news: Gemini Omni 1.1 Flash has landed #1 in the Text-to-Video Arena and #2 in the Image-to-Video Arena! For Text-to-Video the latest 1.1 model is +20pts above FLUX 3 Video at #3 (1495 pts). For Image-to-Video the release is a strong +25pt improvement from Gemini Omni Flash”
https://x.com/arena/status/2093015572212846673
Inference on steroid: NVIDIA’s Groq 3 LPX is now in full production, adding a dedicated token-generation accelerator to the Vera Rubin platform. NVIDIA says it reached 3,400 output tokens per second running Gemma 4 31B with a 100,000-token context in Artificial Analysis”
https://x.com/kimmonismus/status/2091926070085759448
Multimodal API support 🔌 🔹 Set model=’deepseek-v4-flash-vision-exp’ 🔹 Images are tokenized for billing: up to 384 tokens each, at V4-Flash pricing 🔹 Supports Chat Completions, Messages & Responses 🔹 Supports mixed text + image input; images can be provided via base64,”
https://x.com/deepseek_ai/status/2090730039973392531
Build with Gemini Omni 1.1 Flash
https://blog.google/innovation-and-ai/technology/developers-tools/build-with-gemini-omni-1-1-flash/
If you’re building with @GoogleDeepMind Gemini Omni 1.1 Flash today, we just published a new prompting guide: – Role binding with tags: Explicitly tell the model how to use your inputs instead of guessing: • <FIRST_FRAME>: Starting frame • <LAST_FRAME>: Target ending frame”
https://x.com/_philschmid/status/2093012878211072183
Gemini Omni 1.1 Flash is our newest multimodal model for video generation and editing. It delivers a new suite of creative capabilities and controls for developers 🎥 With this update you can: 🎬 Extend your scenes 🎯 Specify starting and ending frames of a shot ➕ Add video”
https://x.com/Google/status/2093008576487072064
introducing Gemini Omni 1.1 Flash this model brings a new suite of creative controls and generative video capabilities to developers – extend scenes for longer storytelling – specify first and last frames – draft videos more efficiently in 360p – upscale up to 4K resolution -“
https://x.com/GoogleAIStudio/status/2093008678118998298
Meet Gemini Omni 1.1 Flash ⚡️ Our newest multimodal model for video generation and editing. It now features your favorite creative controls from Veo, plus brand new capabilities. Enjoy features like 4K upscaling, first / last frame control, and fast 360p drafting. But, the”
https://x.com/GoogleAI/status/2093008998987403303
New models to know this week ▪️ MoE-ViE … Meta’s sparse mixture-of-experts vision encoder for efficient image and video understanding. ▪️ τ0-VLA – robot foundation model that “thinks” through possible outcomes before acting (AgiBot Finch) ▪️ 4DAnyone – turns a single casual”
https://x.com/TheTuringPost/status/2092420798539833839
GLM-5.3 Flash has arrived. @Zai_org’s first natively multimodal GLM-5 model packs 320B parameters, 18B active, 1M context, and hybrid attention. On DeepSWE, it nearly MATCHES Luna’s performance while getting more than TWICE as much work done for the same budget.”
https://x.com/togethercompute/status/2093015257560281099
I’m highly confident now it’s GLM and specifically either 5.3 vision or even smaller/faster one Just preregistration that I’m not retarded and accept uncertainty”
https://x.com/teortaxesTex/status/2090734081751310344
Congrats to HAI Founding Director Fei-Fei Li (@drfeifei) on being named to TimeAI 100! A pioneer in AI and computer vision, Fei-Fei has revolutionized the field through ImageNet, co-founded @StanfordHAI, and champions human-centered AI. Well deserved!”
https://x.com/StanfordHAI/status/2093010247569051685
[1/7] 🚨 We’re releasing LAION-BVD: a 10-million-hour open video dataset for multimodal pre-training. – 1.3B video URLs from CommonCrawl – 80M downloaded videos – 10M video hours – 55M captioned clips – 300M frame-caption pairs 🌐:
https://t.co/dNGwHsygF7 🧵👇”
https://x.com/ahochlehnert/status/2092648676829413778
Leave a Reply