Canon R5 photo. a person in casual attire standing in the middle of a bustling, futuristic city square. The person is wearing an advanced AR headset that projects holographic data and interfaces around them. The city is a blend of real and digitally augmented structures, with skyscrapers that have virtual extensions reaching into the sky, digital flora that reacts to physical movements, and interactive holographic advertisements floating beside real shops. Virtual avatars of people walk alongside real pedestrians, each interacting in a space that seamlessly merges physical and digital realities. The sky above is a vivid sunset, casting warm hues over the scene, enhancing the blend of technology and natural beauty. This visualization encapsulates the future of urban life enhanced by AR, VR, and AI technologies, highlighting the potential for digital and physical world convergence. --ar 16:9

Augmented and Virtual Reality (AR/VR) News: Week Ending 03/22/2024

March 22, 2024

Apple Vision Pro

NVIDIA Omniverse Expands Worlds Using Apple Vision Pro | NVIDIA Blog – https://blogs.nvidia.com/blog/omniverse-apple-vision-pro/?ncid=so-twit-288101

Super cool windows into panoramas (virtual windows)

“Vision Pro app developers have found a MUCH more interesting use case for Panoramas than Apple You can also spawn multiples of these windows, even though I only show one in the video https://twitter.com/SadlyItsBradley/status/1768723856288612409

“NVIDIA announced today at GTC 2024 is bringing OpenUSD Omniverse enterprise digital twins to the Apple Vision Pro! See this demo now 👇🏼 This show the capabilities of spatial computing by blending 3D photo real environments with the real world. https://twitter.com/hughhoufilm/status/1769900268051988924

“Apple now has the Vision Pro patent regarding Immersive Video Streaming using View-Adaptive Prefetching and Buffer Control! 🤯 Does this mean Apple now has a monopoly on pushing immersive video to the next level of clarity and resolution? View adaptive streaming was used five… https://twitter.com/hughhoufilm/status/1769130003051290861

“Create 3D in your Apple Vision Pro 𖡼 Turn text + images into 3D in seconds, then spawn it into your world to share with friends. 10x higher quality than last version. Works on iPhone too. Try it now at https://twitter.com/MattHoerl/status/1770859015368520057

Splats and Nerfs

“Google presents RadSplat Radiance Field-Informed Gaussian Splatting for Robust Real-Time Rendering with 900+ FPS Recent advances in view synthesis and real-time rendering have achieved photorealistic quality at impressive rendering speeds. While Radiance Field-based https://twitter.com/_akhaliq/status/1770669543565734194

“We just shipped the biggest update to @Scaniverse yet: Gaussian Splatting, which can reconstruct photorealistic 3D scenes like these, entirely on your phone. It took a lot of hard work from the entire team, and we’d love for you to try it! Get the app at: https://twitter.com/keeeto/status/1770596013428166984

3D Modeling

“Today, we are releasing Stable Video 3D, a generative model based on Stable Video Diffusion. This new model advances the field of 3D technology, delivering greatly improved quality and multi-view. The model is available now for commercial and non-commercial use with a Stability… https://twitter.com/StabilityAI/status/1769817136799855098

“NVIDIA dropped another banger in the text-to-3D space – LATTE3D. The need for high quality 3d assets spans media entertainment, robotics and simulation to name just a few. This leap is huge. DreamFusion circa 2022 was slow and low quality, but kicked off this generative 3D… https://twitter.com/bilawalsidhu/status/1770829596809687242

“Stability AI released Stable Video 3D. The new AI model is based on Stable Video Diffusion and creates high-quality and consistent 3D generations from a single image. Stability AI just keeps shipping. It’s impressive. https://twitter.com/rowancheung/status/1769943138674642960

“😱Massive 3D release!! 🥳Introducing 𝐌𝐕𝐄𝐝𝐢𝐭: A powerful 3D toolbox for creating stunning 3D objects from text and images! Watch the supercool and feature-rich app (akin to Auto1111🔥) in action! MVEdit generates high-quality textured meshes from multi-view images. 🖼️➡️🗿 https://twitter.com/Gradio/status/1770028925856387464

Robot Training

“Today is the beginning of our moonshot to solve embodied AGI in the physical world. I’m so excited to announce Project GR00T, our new initiative to create a general-purpose foundation model for humanoid robot learning. The GR00T model will enable a robot to understand multimodal… https://twitter.com/DrJimFan/status/1769860044324319658

“Nvidia showcases AI-driven digital twin technology for warehouse operations. Features a 100,000 sq. ft. warehouse digital twin built with Nvidia Omniverse. Integrates digital workers, AMRs, and infrastructure. Utilizes Nvidia Isaac, Metropolis, and Kuopo software for… https://twitter.com/hadi__co/status/1770121360892670195

World Models

“Midjourney continues to reiterate in office hours that their goal is real-time world simulation. They said: “We’re really trying to get to the world simulation. We’re building 3D midjourney, video MJ, real-time MJ where things move really really fast. Put them all…” / X – https://twitter.com/nickfloats/status/1770549117116191196

“I got to try this demo in person today at NVIDIA HQ — mind blown 🤯 Immaculate streaming quality, full-res CAD models with real-time ray tracing. Stay tuned for a deep dive video after GTC. I’ll also be interviewing the VP of XR behind these efforts. https://twitter.com/bilawalsidhu/status/1769919085847208136

“I asked Jensen about his iconic AI prediction: ‘How far are we from a world where every pixel is generated at real-time frame rates?’ Jensen thinks we’re ‘5-8 years away’ and is already seeing signs of traversing the S-curve of innovation. The entire gaming and simulation… https://twitter.com/bilawalsidhu/status/1770157890546123177

“At NVIDIA GTC, the company announced Earth-2. It’s a cloud platform that uses AI and digital twin tech to forecast climate change and weather with high resolution and speed. AI use cases like this will be massive for predicting extreme weather. https://twitter.com/rowancheung/status/1770294655579525551

“Introducing Buildbox 4 Alpha, the AI-first game engine where you simply type to create. Make games with the power of AI. Sign up today at → https://twitter.com/buildbox/status/1770086034002300959?s=20

Other AR/VR News

Fully AI-generated games by 2030? That’s what Nvidia’s CEO believes – but what exactly will that mean for PC gamers? | TechRadar – https://www.techradar.com/computing/gpu/fully-ai-generated-games-by-2030-thats-what-nvidias-ceo-believes-but-what-exactly-will-that-mean-for-pc-gamers

ScoreHMR – ScoreHMR:

Score-Guided Diffusion for 3D Human Recovery

https://statho.github.io/ScoreHMR

VR Headsets Are Approaching the Eye’s Resolution Limits – IEEE Spectrum – https://spectrum.ieee.org/virtual-reality-head-set-8k

Roblox Introduces AI-Powered Avatar and Texturing Technologies to Accelerate 3D Creation – Roblox – https://corp.roblox.com/2024/03/18/roblox-introduces-ai-powered-avatar-and-texturing-technologies-to-accelerate-3d-creation/

MineDreamer – MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control https://sites.google.com/view/minedreamer/main

Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.

Be Sure To Read This Week’s Main Post:

This week’s executive overview and top links are here:

AI News #25: Week Ending 03/22/2024 with Executive Summary and Top 55 Links

The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.

Credits/Sources

Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.