AI Audio News

SonicVisionLM can generate sound effects for videos. It uses vision language models (VLMs) to identify events within videos and generate sounds that match the video content.

AI generated videos don't have sound yet, but that is going to change!

SonicVisionLM can generate sound effects for videos. It uses vision language models (VLMs) to identify events within videos and generate sounds that match the video content.https://t.co/qV8SyDD0XG pic.twitter.com/PaPdvoJcZ9
— Dreaming Tulpa 🥓👑 (@dreamingtulpa) January 17, 2024

It turns out that an artificial deep neural network that models auditory information in the brain spontaneously creates units tuned to music, even without learning music directly

Even #AI wants to hear music: It turns out that an artificial deep neural network that models auditory information in the brain spontaneously creates units tuned to music, even without learning music directly @natcomms https://t.co/izo3F1zlZW
— Mary Elizabeth (@meharpist) January 16, 2024

Introducing ‘Text to Music’ on http://beatoven.ai.  Now get more creative control on background music for your content.

!! Launch alert !! Introducing ‘Text to Music’ on https://t.co/GsXQfSKV5R. Now get more creative control on background music for your content. Experience the revolution at your fingertips with https://t.co/GsXQfSKV5R. #BeatovenAi #texttomusic #AI #creators #aimusic pic.twitter.com/0GWRJpldAn
— Beatoven.ai (@beatovenai) January 18, 2024

https://www.beatoven.ai

Meta

MAGNeT : AI from META to generate music on your Mac Silicon

MAGNeT is a text-to-music and text-to-sound model capable of generating high-quality audio samples conditioned on text descriptions. It is a masked generative non-autoregressive Transformer trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz.

View at Medium.com

Click to access 2401.04577.pdf

https://github.com/facebookresearch/audiocraft/blob/main/docs/MAGNET.md

Examples: https://twitter.com/rowancheung/status/1747845930491990042

This is the first part of many posts I am writing to consolidate learnings on how to finetune Large Language Models (LLMs) to process audio, with the eventual goal of being able to build and host a LLM able to describe human voices.https://paul.mou.dev/posts/2023-12-31-listening-with-llm/

Be Sure To Read “This Week in AI“

This week’s executive overview and top links are here:
AI News #16: Week Ending 01/19/2024 with Executive Summary and Top 16 Stories

This is an extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.