a dark movie theater. a man sits in the front row playing a tuba. The movie screen says “Video”. –chaos 30 –ar 4:3 –style raw –personalize qko3v53 –v 6.1

“Knowing the training data for any AI video maker has a lot of typical YouTube stuff, may I present: A man reacts to rocks, a chef puts far too much salt on some food (nice salt physics), hard math is explained, a woman reacts to a potato, a Twitch streamer, & flaming car review

Knowing the training data for any AI video maker has a lot of typical YouTube stuff, may I present:

A man reacts to rocks, a chef puts far too much salt on some food (nice salt physics), hard math is explained, a woman reacts to a potato, a Twitch streamer, & flaming car review pic.twitter.com/oVobyqwSmr
— Ethan Mollick (@emollick) July 5, 2024

“🔊 Sound on! 🤯 This new AI Space generates audio for silent videos. Imagine the possibilities: – Bringing old silent films to life – Adding ambiance sounds – Enhancing game development What would you use this for? Try it yourself:

🔊 Sound on! 🤯
This new AI Space generates audio for silent videos.

Imagine the possibilities:
– Bringing old silent films to life
– Adding ambiance sounds
– Enhancing game development

What would you use this for?

Try it yourself: https://t.co/726pCAd5nz #AI #AudioGeneration pic.twitter.com/RYozFFWOeO
— Florent Daudens (@fdaudens) July 3, 2024

“In October 2022, “an otter on a plane using wifi” typed into an jmage generator created nonsense first try. In July, 2024 the same thing typed into a video generator (plus “on a computer”) gets this first try. Not quite an otter, but look at the lighting & details. Quick pace…

In October 2022, “an otter on a plane using wifi” typed into an jmage generator created nonsense first try.

In July, 2024 the same thing typed into a video generator (plus “on a computer”) gets this first try. Not quite an otter, but look at the lighting & details.

Quick pace… https://t.co/AWfhJXJ95A pic.twitter.com/jT2aCltDZK
— Ethan Mollick (@emollick) July 2, 2024

FlexiFilm: Long Video Generation with Flexible Conditions

https://y-ichen.github.io/FlexiFilm-Page

Krea

“The decreasing cost and increasing ability of open source AI image tools is leading to an explosion of interesting services and experiments. This is me playing with https://www.krea.ai/home  (free, for now?) which lets you “paint” in realtime with shapes.

The decreasing cost and increasing ability of open source AI image tools is leading to an explosion of interesting services and experiments.

This is me playing with https://t.co/dW3rswxg5e (free, for now?) which lets you "paint" in realtime with shapes. https://t.co/g9VgelRYjK pic.twitter.com/JggsBLAYvL
— Ethan Mollick (@emollick) July 5, 2024

Runway

“Gen-3 Alpha Text to Video is now available to everyone. A new frontier for high-fidelity, fast and controllable video generation. Try it now at

Gen-3 Alpha Text to Video is now available to everyone.

A new frontier for high-fidelity, fast and controllable video generation.

Try it now at https://t.co/ekldoIshdw pic.twitter.com/miNbHdK5hX
— Runway (@runwayml) July 1, 2024

“Runway Gen-3 videos are next level. 7 impressive ones:” / X

Runway Gen-3 videos are next level.

7 impressive ones:
— Borriss (@_Borriss_) June 30, 2024

“Runway Gen-3 highlight reel. Everything made with text-to-video. – Strong prompt adherence; can do trippy transitions – Impressive humans; though control is limited – Can do realistic volumetric and particle effects – Nicely integrates HUDs and AR elements – Often blends live

Runway Gen-3 highlight reel. Everything made with text-to-video.

– Strong prompt adherence; can do trippy transitions
– Impressive humans; though control is limited
– Can do realistic volumetric and particle effects
– Nicely integrates HUDs and AR elements
– Often blends live… pic.twitter.com/gFJvvLhLgQ
— Bilawal Sidhu (@bilawalsidhu) July 1, 2024

“Lmao. Gen-3 can generate retro 16-bit video games on your theme of choice. Here’s 1776 🇺🇸

Lmao. Gen-3 can generate retro 16-bit video games on your theme of choice.

Here's 1776 🇺🇸 pic.twitter.com/UqZs43OQLL
— Bilawal Sidhu (@bilawalsidhu) June 30, 2024

Runway, an AI Video Startup, in Talks With General Atlantic for $4 Billion–Valuation Fundraise — The Information

https://www.theinformation.com/articles/runway-an-ai-video-startup-in-talks-with-general-atlantic-for-4-billion-valuation-fundraise

“@AndroidAuth Runway announced that its AI video generator, Gen-3 Alpha, is now available to all users. Gen-3 Alpha feels like the biggest leap AI video has taken since Sora, but the generation limits and wait queues so far have been rough

Runway announced that its AI video generator, Gen-3 Alpha, is now available to all users.

Gen-3 Alpha feels like the biggest leap AI video has taken since Sora, but the generation limits and wait queues so far have been rough pic.twitter.com/g1OuqJpC8Q
— Rowan Cheung (@rowancheung) July 3, 2024

“The “don’t believe anything you see online” moment from AI is continuing to expand as video becomes trivial. Gen-3 is good enough that you can make cryptid videos that, while not perfect, beat a lot of the stuff that people spread around on social media. I did this in one go.

The "don't believe anything you see online" moment from AI is continuing to expand as video becomes trivial.

Gen-3 is good enough that you can make cryptid videos that, while not perfect, beat a lot of the stuff that people spread around on social media. I did this in one go. pic.twitter.com/9fWEtBhC83
— Ethan Mollick (@emollick) July 4, 2024

“I created a fully AI-generated commercial. Everything, including the videos, voiceover, logo, ending images, sound effects, and even the script, was done with AI. Runway has opened a world of endless possibilities.

I created a fully AI-generated commercial.

Everything, including the videos, voiceover, logo, ending images, sound effects, and even the script, was done with AI.

Runway has opened a world of endless possibilities. pic.twitter.com/q1gDawJSf1
— Alvaro Cintas (@dr_cintas) July 4, 2024

““Hyperspeed POV shot: going through a crystal cave, then through a portal, then through a burning city.” First try. Runway Gen-3 is clearly the best text-to-video system. Yet, its closer to a world model, but isn’t quite there. Also not controllable enough yet for full use.

“Hyperspeed POV shot: going through a crystal cave, then through a portal, then through a burning city.” First try.

Runway Gen-3 is clearly the best text-to-video system.

Yet, its closer to a world model, but isn’t quite there. Also not controllable enough yet for full use. pic.twitter.com/3fuQS6sFXP
— Ethan Mollick (@emollick) July 2, 2024

“Runway, make “the most American video, ever.” There were versions that were coherent, but this one has a vibe: ✅Eagle w/American flag tail flying over fields ✅Folks with too many hands pointing at fireworks ✅An American flag driving a muscle car past a distorted Mt. Rushmore

Runway, make "the most American video, ever."

There were versions that were coherent, but this one has a vibe:
✅Eagle w/American flag tail flying over fields
✅Folks with too many hands pointing at fireworks
✅An American flag driving a muscle car past a distorted Mt. Rushmore pic.twitter.com/LCo6VLZPXy
— Ethan Mollick (@emollick) July 5, 2024

“I compiled most of the videos where I prompted Runway to generate “FPS shooters in the style of famous works of art/artists” into one video. Surprisingly good job from the AI: stained glass, the Voynich manuscript, Klimt, Monet, Hans Holbien the Younger and more…

I compiled most of the videos where I prompted Runway to generate “FPS shooters in the style of famous works of art/artists” into one video.

Surprisingly good job from the AI: stained glass, the Voynich manuscript, Klimt, Monet, Hans Holbien the Younger and more… pic.twitter.com/AgPd2TI6SH
— Ethan Mollick (@emollick) July 3, 2024

“I generated this video in one go using Runway Gen 3 with the prompt “tracking shot: wheeled robot rolling through an office, overhead lighting” There are many obvious tells that this is AI, but I am shocked by how fast the tech is advancing. It is going to be hard to tell soon.

I generated this video in one go using Runway Gen 3 with the prompt "tracking shot: wheeled robot rolling through an office, overhead lighting"

There are many obvious tells that this is AI, but I am shocked by how fast the tech is advancing. It is going to be hard to tell soon. pic.twitter.com/C7YyuMTxRV
— Ethan Mollick (@emollick) July 3, 2024

Synthesia

Synthesia’s hyperrealistic deepfakes will soon have full bodies | MIT Technology Review

https://www.technologyreview.com/2024/06/24/1094170/synthesias-hyperrealistic-deepfakes-will-soon-have-full-bodies

The Rest of Video News

Incredible demo

“🤣 on the AI internet, you can be anyone you want to be

🤣 on the AI internet, you can be anyone you want to be pic.twitter.com/yVlTXZdBwS
— Ate-a-Pi (@8teAPi) July 6, 2024

“I’m playing around with generative AI tools and stitching them together into visual stories. Here I took the first few sentences of Pride and Prejudice and made it into a video. The gen stack used for this one: – @AnthropicAI Claude took the first chapter, generated the scenes

I'm playing around with generative AI tools and stitching them together into visual stories. Here I took the first few sentences of Pride and Prejudice and made it into a video.

The gen stack used for this one:
– @AnthropicAI Claude took the first chapter, generated the scenes… pic.twitter.com/vX64avfRUY
— Andrej Karpathy (@karpathy) July 4, 2024

Text2Bricks: Fine-tuning Open-Sora in 1,000 GPU Hours | lego – Weights & Biases

https://wandb.ai/lambdalabs/lego/reports/Text2Bricks-Fine-tuning-Open-Sora-in-1-000-GPU-Hours–Vmlldzo4MDE3MTky

Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.

Be Sure To Read This Week’s Main Post:

This week’s executive overview and top links are here:

AI News #40: Week Ending 07/05/2024 with Executive Summary and Top 65 Links

The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.

Credits/Sources

Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.