Ethan B. Holland

Over 62,700 manually organized AI links and counting

AI News #12: Week Ending 12/22/2023 with Executive Summary and Top 7 Stories

This week’s cover theme is “segmentation”. Segmentation is the ability for a computer to tell things apart within an image, identify them, track them as they move. For example, tracking cars or soccer players, separating an image into all of its objects for editing, masking, or removal. The cover references a scene from the movie Inception, since it would be helpful to have a computer track the characters as they jump from one dream to another. Segmentation is a powerful part of augmented and virtual reality – a second theme this week and related to Inception. I digitally inserted a pair of VR goggles resting on the steps (bottom right). The image was created with Photoshop’s “select subject” automated masking and Adobe’s generative fill tool, which is incredible.

Executive Summary

  • MidJourney v.6: The leading text-to-image generation tool, MidJourney, released a new version, and the results are breathtaking.  Be sure to click the examples below.
  • iPhone AI: Apple is working on an offline AI model that can run directly on an iPhone, without an internet connection.  This is big news for Siri, dictation, app developers, health tracking, images, video… anything on our phone..
  • Augmented Reality: Last year, Meta was mocked for investing too heavily in the Metaverse. 2024 might be the year Mark Zuckerberg shows he was right.  This week saw several announcements underscoring the pace of AR/VR.  Apple released technology that separates objects from still images and turns them into 3D renders. Apple is also building AR/VR into its phones, in preparation for their VR headsets shipping later this year.
  • Object Tracking (aka Segmentation): Apple is making advancements in object tracking and editing.  Microsoft and another small group of researchers also released tracking tools.  Knowing what’s in a photo and following it across video is also a big part of augmented and virtual reality, and might sound boring… but it’s shocking when you see it.
  • Video: Leading AI companies Pika and Runway continue to improve text-to-video generation.  Pika released ‘modify region’ which lets users change objects within a video, while maintaining the rest.  New user examples of Runway videos show just how powerful it’s getting, with creative applications.
  • Retrospectives: Sam Altman, Bill Gates, and others share their thoughts on last year as well as the year ahead.
  • OpenAI Preparedness: OpenAI created a preparedness team to research and report back how society and companies will manage once AI outperforms humans.  There are rumors that OpenAI is heading toward a breakthrough in the coming months.

Top Stories

These are the must-click links if you only have time for a few.  Even if they look boring, click them!  I did the work, so you don’t have to worry.  All are 10/10 would recommend.

The Rest: AI News of The Week

Don’t let the volume overwhelm you.  Have fun and skim it. The links are organized by topic, sorted from ‘coolest’ to ‘least cool’, and each topic is clearly defined with a headline.  I’ve added a description and glossary of what the topics mean, beneath each label, in plain language.  I do the work so you don’t have to!   The links descriptions are often pulled directly from tweets or articles, so it’s not always my voice.  Pause when you see something that interests you.  Reach out to me any time.  I enjoy sharing and discussing these items!

Audio

This week’s audio section is all about generative AI song creation and a new tool called Suno.  Enter some text, and out comes a song.   Suno song-writing must be experienced to be believed.  It’s a bit silly and awkward, but it’s the worst it’s ever going to be… and it creates a custom tune + lyrics, and outputs it in a matter of seconds

Suno

Try it for yourself

https://app.suno.ai/create/

https://app.suno.ai/song/862a3b88-aec9-4039-9311-8a69ddc7b4c6

“You can make great music, whether you’re a shower singer or a charting artist. No instrument needed, just imagination.”

Turn your ideas into songs with Suno on Microsoft Copilot

“We are excited to share that we have partnered with Suno, a leader in artificial intelligence-based music creation to bring their capabilities to Microsoft Copilot. Through this partnership, people will have at their fingertips the ability, regardless of musical background, to create fun, clever, and personalized songs with a simple prompt. Suno has been a leader in AI music technology, pioneering the ability to generate complete songs—lyrics, instrumentals, and singing voices—from a single sentence.”

https://blogs.bing.com/search/december-2023/Turn-your-ideas-into-songs-with-Suno-on-Microsoft-Copilot

Segmentation (this week’s cover image theme)

Segmentation is the ability for a computer to tell things apart within an image, identify them, track them as they move.  For example, tracking cars or soccer players, separating an image into all of its objects for editing, masking, or removal.  This week it really comes into play with augmented reality.  Apple is making huge strides in taking still photos, segmenting the people and objects, and using a technique called “gaussian splatting” to turn these still objects into 3D representations.  

Also, this week’s cover photo references segmentation, since it would be helpful to have segmentation to track the people in the movie Inception as they move from one dream to another.  

Apple Breakthroughs

“Apple AI avatar research that takes your short 2D video clip and pulls apart an editable 3D avatar and environment.  The modeling of hair, clothing etc is immaculate and looks far more realistic than rigging a 3D photo scan.”

“Introducing HUGS: Human Gaussian Splats – capable of creating animatable (3DGS) avatars from a casual video (50-100 frames) in ~30 mins. Our avatars can easily be embedded into other (NeRF) scenes.”

TAO

Tracking Any Object Amodally

“Infers complete object boundaries, even when certain portions are occluded.”

Microsoft Kosmos-2

“Think of an LLM that can find entities in a given image, describe the image and answers questions about it, without hallucinating.  Kosmos-2 released by Microsoft is a very underrated model that can do that. “

“Introducing SemiVL – a semi-supervised semantic segmentation method exploiting priors from vision-language models”

Translation

Meta is the star with two big translation tools.  The first is a tool that retains the original tone, volume, and inflection while mapping audio to a new language.  The second is a streaming translation tool, which translates 100 languages with only a two-second delay.

Meta Translation AI Models

“Seamless Communication: A significant step towards removing language barriers through expressive, fast and high-quality AI translation – A family of AI research models that enable more natural and authentic communication across languages”

https://ai.meta.com/research/seamless-communication/

Meta: SeamlessExpressive

“Translations should capture the nuances of human expression. While existing translation tools are skilled at capturing the content within a conversation, they typically rely on monotone, robotic text-to-speech systems for their output. SeamlessExpressive aims to preserve intricacies of speech; such as pauses and speech rate, in addition to vocal style and emotional tone.”

https://seamless.metademolab.com/expressive

SeamlessStreaming

SeamlessStreaming is the first massively multilingual model that delivers translations with around two-seconds of latency and nearly the same accuracy as an offline model. Built upon SeamlessM4T v2, SeamlessStreaming supports automatic speech recognition and speech-to-text translation for nearly 100 input and output languages, in addition to speech-to-speech translation for nearly 100 input languages and 36 output languages.

https://ai.meta.com/resources/models-and-libraries/seamless-communication-models/#seamlessstreaming

Images

The biggest news is MidJourney – an update of the text-to-image generation tool creates imagery which is nearly impossible to tell apart from an actual photo.  Take time to click all the examples.  It’s amazing how much the technology has progressed in a few months.

The second biggest news is a continuation of “upscaling”.  This takes a decent image (AI or otherwise) and doubles or quadruples its resolution… and can be given freedom to be creative while doing so…  to the point that it can turn an emoji into a realistic image.

Upscaling

Magnific.ai upscales simple emojis into highly realistic art pieces. 

https://www.linkedin.com/feed/update/urn:li:activity:7140706133148688384/

MidJourney 6

MidJourney v6 is finally here!!!!

“Here are some side-by-sides, –v 5.2 versus –v 6, as well as some new highly detailed prompts and camera angle tests. These are all unaltered and unedited, straight out of MidJourney .  v6 is a HUGE leap forward”

One day passed since MidJourney v6 got released into the wild..  X is already flooded with tests.  Here are 9 that impressed me a lot:

The most powerful example of MidJourney 5 v. 6.   Same prompt, two comparison images:

“portrait art, stunningly beautiful woman, silver textured skin, purple hair, magical, enchanting, sparkly –ar 16:9”

Mastering MidJourney v6 : A Deep Dive into the New Version!

https://youtu.be/LJ36dMlw5C8?si=rGAaWIPO7q7gpA0I

Hello Kitty Invades the United States

The new version of MidJourney now does a really good job integrating text.

“I did a series of images of some famous meals in literature, with the names of the authors included in the image. Joyce’s plate of peas at a pub, Melville’s clam chowder with pork, Proust’s madeleine…”

Other image news:

Sketch anything and render your unique character with guaranteed consistency, every time

Google released an upgrade to it’s Magic Eraser (see also the “segmentation” section)

The photo editor now uses Gen AI to fill in details when users remove unwanted objects from photos.

DiffusionLight: Light Probes for Free by Painting a Chrome Ball

Given an input image, we seek to estimate the scene’s lighting as an HDR environment map. The idea is to inpaint a chrome ball into the image using a diffusion model and unwarp it to an environment map. We tackle two key challenges of (1) how to consistently generate chrome balls and (2) how to use an LDR diffusion model to generate HDR chrome balls.

https://diffusionlight.github.io/

Snapchat+ subscribers can now create and send AI-generated images

Subscribers can now create and send AI-generated images based on a text prompt. In addition to getting access to a new AI extend tool, subscribers can now also use the app’s Dream selfie feature with friends.

https://techcrunch.com/2023/12/12/snapchat-subscribers-can-now-create-and-send-ai-generated-images/

Improving Text in Generative Images

A Unified Framework for High-quality Text Synthesis in Arbitrary Images via Character-aware Diffusion Models

https://udifftext.github.io/

Agents

An agent is something that does a task for you (i.e. ‘travel agent’).  In AI, an agent often refers to using a language interface like Chat GPT to do something in real time (i.e. “look up the weather”).   It’s a broad term that’s both technical (a web-browser like Chrome is an agent) and common-sense (Google search is an agent).   This week the big story is the trend of using an AI that can “see” to use your device for you, like a person.  This is very different from traditional computer agents, which connect using a data connection and disclose themselves.  If an AI can actually “use your phone”… then the agent will appear to be your phone.  If I give my friend my phone to text for me… no one knows it’s my friend…. Which means no one will know it’s an AI bot.  This means “blocking AI” will be a lot harder, since it’s not discernible from a person.

Tencent announces AppAgent

Multimodal Agents as Smartphone Users

“Recent advancements in large language models (LLMs) have led to the creation of intelligent agents capable of performing complex tasks. This paper introduces a novel LLM-based multimodal agent framework designed to operate smartphone applications. Our framework enables the agent to operate smartphone applications through a simplified action space, mimicking human-like interactions such as tapping and swiping.”

https://huggingface.co/papers/2312.13771

Will AI Kill Traditional Web Scraping? (GPT4V + Mistral Medium Project)

Google Chrome just got enhanced with new AI-powered page summaries.

Google AI researchers have developed ‘Hold for Me’.

The AI tech processes audio directly on its Pixel smartphone devices and can determine whether you’ve been put on hold or if someone has picked up a call.

No more waiting on hold for hours to up your credit card limit!

Augmented/Virtual Reality

The big news this week is from Meta.  They have so many photos and videos on Facebook and Instagram (set to public) that they can leverage them to build incredible training tools for virtual reality.  Apple is also doing big things by integrating AR into the camera of their new phones, even before people have the headsets.  They are building the foundation for an AR/VR future.

Meta

“Meta has been building an absolutely wild dataset comprised entirely of first person or ‘egocentric’ views. Not just imagery but also time-synced audio, IMU, eye gaze, head poses & 3D point clouds of the environment. Perfect for training an AI Jarvis?”

Meta announces ControlRoom3D

Room Generation using Semantic Proxy Rooms

“ControlRoom3D creates diverse and plausible 3D room meshes aligning well with user-defined room layouts and textual descriptions of the room style.”

“Oh you know, just hitting a few golf balls in my dining room through a magic portal in mixed reality on the Quest 3. No big deal.“

Meta’s Ray-Ban Glasses Added AI That Can See What You’re Seeing

https://www.cnet.com/tech/computing/metas-ray-ban-glasses-added-ai-that-can-see-what-youre-seeing/

The rest of AR/VR

“Announcing Holodeck, a promptable system that can generate diverse, customized, and interactive 3D simulated environments ready for Embodied AI applications.”

“HAAR is a generative model of 3D hair. Given a text description, HAAR generates an encoding of the 3D hair strands. These are decoded into an animation-ready hair model. This connects text-driven generative models with standard graphics that can be used today, eg in Unreal. ⅕”

“Neural perception with vision and touch yields robust tracking and reconstruction of novel objects for in-hand manipulation.”

Why Vision Pro Will Change Photography

Why Vision Pro Will Change Photography

Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

https://research.nvidia.com/labs/toronto-ai/AlignYourGaussians/

Splatter Image: Ultra-Fast Single-View 3D Reconstruction

https://szymanowiczs.github.io/splatter-image

Paint Anything 3D with Lighting-Less Texture Diffusion Models

“AGAP supports various ways of 3D editing by applying different 2D image processing tools on the canonical images without re-optimization.”

Nvidia just announced Align Your Gaussians

Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models

HUGS: Human Gaussian Splats

Click to access 2311.17910.pdf

News/Journalism

OpenAI just gave out 13 grants to local news organizations to drive specific AI projects in their newsrooms.

OpenAI Partnership with the American Journalism Project

Through our partnership with American Journalism Project, its new Product & AI Studio has selected the first cohort of newsrooms to receive grants— enabling 13 organizations to pilot and experiment with a variety of AI applications to assist local journalism.

This summer, the American Journalism Project launched our new Product & AI Studio, a new program to explore the smart application of emerging technologies in local journalism.  We’re starting our work in the Studio with a set of pilot investments; these direct grants will go to a cohort of 13 of our portfolio organizations, which will receive one- to two-year grants, ranging from $25,000 to $200,000. They are: Centro de Periodismo Investigativo (CPI), Cityside, Civic News Company, CT Mirror, inewsource, The Marshall Project, Montana Free Press, Outlier Media, the Nebraska Journalism Trust, Open Campus, Sahan Journal, Spotlight PA, and THE CITY.

View at Medium.com

Legal/Ethics

It’s notable that robots are better than people at solving CAPTCHAs.  Also, OpenAI is causing a stir by creating a “preparedness” team.   Many interpret this as OpenAI getting close to major breakthroughs.  Lastly, in existential news, Imran Khan deployed a clone to campaign from jail in Pakistan.

Robots are now better than humans at solving CAPTCHAs

An Empirical Study & Evaluation of Modern CAPTCHAs 

Click to access 2307.12108.pdf

OpenAI Preparedness Team

“The study of frontier AI risks has fallen far short of what is possible and where we need to be. To address this gap and systematize our safety thinking, we are adopting the initial version of our Preparedness Framework. It describes OpenAI’s processes to track, evaluate, forecast, and protect against catastrophic risks posed by increasingly powerful models.”

https://openai.com/safety/preparedness

“Imran Khan deploys AI clone to campaign from behind bars in Pakistan”

https://www.theguardian.com/world/2023/dec/18/imran-khan-deploys-ai-clone-to-campaign-from-behind-bars-in-pakistan

“ChatGPT’s decisions are largely rational in risk, time, social, and food choices and it demonstrates higher rationality scores than those of humans”

“Rite Aid banned from using facial recognition software after falsely identifying shoplifters

FTC says the company’s ‘reckless use’ of AI humiliated customers”

Rite Aid banned from using facial recognition software after falsely identifying shoplifters

People buy brand-new Chevrolets for $1 from a ChatGPT chatbot

https://the-decoder.com/people-buy-brand-new-chevrolets-for-1-from-a-chatgpt-chatbot/

“‘We cannot let China get these chips’: Commerce Secretary Raimondo says more funding needed for AI export controls”

https://fortune.com/2023/12/02/ai-chip-export-controls-china-nvidia-raimondo/

If AI is making the Turing test obsolete, what might be better?

“The Turing test focuses on the ability to chat—can we test the ability to think?”

https://arstechnica.com/ai/2023/12/do-ai-improvements-call-for-something-better-than-the-turing-test/

TikTok 

TikTok made the newsletter this week, because they got caught using OpenAI to train a competitor.  They also have been making some interesting AI tools that take a still image of a person and animate it.  This makes sense since TikTok has so many videos of people dancing.  They also made a tool that animates a face from a photo.  This ties into the theme of segmentation and AR/VR.

TikTok Gets Busted

ByteDance is secretly using OpenAI’s tech to build a competitor

https://www.theverge.com/2023/12/15/24003151/bytedance-china-openai-microsoft-competitor-llm

OpenAI suspends ByteDance’s account after it used GPT to train its own AI model.

https://www.theverge.com/2023/12/15/24003542/openai-suspends-bytedances-account-after-it-used-gpt-to-train-its-own-ai-model

Other TikTok News

Bytedance announces DREAM-Talk

Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generation

https://huggingface.co/papers/2312.13578

Bytedance announces DiffPortrait3D

Controllable Diffusion for Zero-Shot Portrait View Synthesis

Given an unposed portrait image, diffportrait3d can synthesize plausible but consistent facial details with retained both identity and facial expression.

Apple

The big news with Apple is that they are working feverishly to get AI running on a phone, locally.  This means no internet connection needed.  Obviously, Siri would get much better.  As would autofill, etc.  But beyond that, so would anything that gets your data.  Heart rate, pedometer, location.  Phones are the holy grail of data, and having a locally run AI on the phone would be a ‘force multiplier’ for Apple, app developers, and users.  Apple is also working to integrate their new VR headset into everything on the phone (camera, interface, etc).

Apple’s Local LLM Effort 

Researchers working on enabling ‘Apple GPT’ and more to run on iPhone

“A new research paper appears to be geared to enabling a ChatGPT-style system to run on iPhones.  A second Apple AI paper looks at ways to generate animated 3D avatars from standard video, with obvious application to Vision Pro.”

Researchers working on enabling ‘Apple GPT’ and more to run on iPhone

Apple Develops Breakthrough Method for Running LLMs on iPhones

https://www.macrumors.com/2023/12/21/apple-ai-researchers-run-llms-iphones/

Apple’s latest AI research could completely transform your iPhone

Apple’s latest AI research could completely transform your iPhone

“A new research paper from Apple demonstrates a breakthrough technique to run advanced AI on iPhones.  It involves streamlining bulky LLMs using flash storage optimization. If (or when) Apple integrates advanced AI into the iPhone — it will be another 0 to 1 moment for adoption.”

Video

AI generated video is moving extremely quickly, but it’s still limited to short clips.  The quality of the clips is improving exponentially, but for critics, videos are way too short and clunky to matter.  Following video closely is more for anticipation – if you see the trend and extrapolate it, it’s thrilling.  If you look at the current state, it’s still clunky.  The two biggest names in AI video are Pika and Runway.  However, Meta and Google both made the news this week.  Meta released “Fairy” which allows a user to take an image, animate it, and change the style using text prompts, in basically real time.   Google’s “Video Poet” is doing something called ‘zero shot’ text to video… which means it creates new videos from nothing but text prompts, without having been trained on the content in advance. 

Pika

Pika: ‘Modify Region’ lets you change specific areas in the frame. 

Runway

Made a movie using Midjourney V6 + Runway and Live Action. The time is now for AI to push our creative at any level. Here is the teaser for a short and feature screenplay that I wrote.

Runway: Generate voice overs and dialogue with our simple to use and highly expressive Text to Speech. Available for all plans.

Runway: Aspect ratios. Quickly and easily change the ratio of your generations to better suit the channels you’re creating for. Choose from 16:9, 9:16, 1:1, 4:3, 3:4.

Meta: Fairy

Meta just announced Fairy

Fast Parallelized Instruction-Guided Video-to-Video Synthesis

Introducing Fairy, a minimalist yet robust adaptation of image-editing diffusion models, enhancing them for video editing applications. Our approach centers on the concept of anchor-based cross-frame attention, a mechanism that implicitly propagates diffusion features across frames, ensuring superior temporal coherence and high-fidelity synthesis.

https://huggingface.co/papers/2312.13834

Google: Video Poet

VideoPoet: Google’s large language model for zero-shot video generation

https://blog.research.google/2023/12/videopoet-large-language-model-for-zero.html

emu2

Generate video from any prompt sequence with emu2

Multimodality

Multimodality is when an AI can process more than one type of media (text, photos, videos).  Meta is doing a lot with video learning, as we saw (above) with their AR/VR video research.  The demo is worth a look.

Meta Video Learing

Introducing Ego-Exo4D: A foundational dataset for research on video learning and multimodal perception

“The result of a two-year effort by Meta’s FAIR (Fundamental Artificial Intelligence Research), Meta’s Project Aria, and 15 university partners, the centerpiece of Ego-Exo4D is its simultaneous capture of both first-person “egocentric” views, from a participant’s wearable camera, as well as multiple “exocentric” views, from cameras surrounding the participant.”

https://ai.meta.com/blog/ego-exo4d-video-learning-perception/

Google Gemini

Google got in a lot of PR trouble for over-promising and under-delivering their competitor to OpenAI’s GPT4.   This stemmed from a product demo when Google launched Gemini.  Google edited the demo to make it look better than it was in real life (specifically to beat GPT).  To make things worse, the more people test Gemini, the more they discover Gemini is not only worse than GPT4, it’s not even as good as GPT 3.5.

Google Gemini is not even as good as GPT-3.5 Turbo, researchers find

“Less than a month after Google unveiled its long-rumored ChatGPT competitor Gemini to the world in a glossy demo video — only for the company to face criticism for what appeared and was ultimately confirmed to be staged interactions between the presenter and the AI — new research finds that the most powerful version of Gemini available now to consumers, Gemini Pro, falls behind OpenAI’s GPT-3.5 Turbo large language model (LLM) in terms of most tasks.”

Google Gemini is not even as good as GPT-3.5 Turbo, researchers find

Google brings Gemini Pro to Vertex AI

“Gemini Pro, a lightweight version of a more capable Gemini model, Gemini Ultra, currently in private preview for a “select set” of customers, is now accessible in public preview in Vertex AI, Google’s fully managed AI dev platform, via the new Gemini Pro API.”

Google brings Gemini Pro to Vertex AI

Artificial General Intelligence (AGI)

Artificial General Intelligence is a term for when computers beat humans at most tasks, not just a few (like playing chess).   There’s no official definition, as opposed to a ‘vibe’… aka “we’ll know when we see it”.   More than just intelligence, many people include robots on this list.  A robot would have to outperform a person at physical tasks like agility, mobility, strength, nuance.  The day that a robot surgeon can get in your car and drive it home, make a cup of coffee, do the laundry, do your taxes… and write a screenplay… is a layperson’s example.   AGI is a major line in the sand when it comes to AI benchmarks… almost impossible by design.

Open AI employee 12/18/2023: “brace yourselves, agi is coming”

Metacalculus: When will the first weakly general AI system be devised, tested, and publicly announced?  Community Prediction: Apr 25, 2026

https://www.metaculus.com/questions/3479/date-weakly-general-ai-is-publicly-known/

IMF Report: AI may be on a trajectory to surpass human intelligence; we should be prepared

https://www.imf.org/en/Publications/fandd/issues/2023/12/Scenario-Planning-for-an-AGI-future-Anton-korinek

Business/Enterprise

This is a catch-all category for announcements that affect how we work and do business.  From job loss or job creation, to funding, or entrepreneurialism. 

Job Loss/Creation

Google might already be replacing some Ad sales jobs with AI

“When AI can make assets and text for ads, you don’t need humans to do it anymore.”

https://arstechnica.com/gadgets/2023/12/report-google-ads-restructure-could-replace-some-sales-jobs-with-ai/

Plot twist: Consulting firm Deloitte is reportedly using AI to prevent layoffs

The firm is deploying AI technology to assess its current employees’ skills and figure out how to move them into areas of the business that are more promising, according to Bloomberg. The firm is essentially making a bet that AI will not only help it avoid mass layoffs, but also moderate its hiring growth in the coming years, Bloomberg reported. 

https://www.businessinsider.com/deloitte-trying-to-avoid-layoffs-with-ai-2023-12

“New study where entrepreneurs in Kenya were given access to AI mentors. This is different than other studies as the AI was limited to advice, not doing.  High performers got a big 20% boost to performance, but low performers took a hit as they could not implement the advice.”

Performance

Airbnb using AI to help clampdown on New Year’s Eve parties globally

https://news.airbnb.com/airbnb-using-ai-to-help-clampdown-on-new-years-eve-parties-globally/

“JPMorgan and Morgan Stanley are both developing custom, sector-focused LLMs for investment advice. JP Morgan has applied to trademark ‘IndexGPT’ in May. Morgan Stanley appears to have been working with OpenAI on their internal LLM since the launch of GPT-4.”

Atlassian welcomes AI to the team

AI capabilities are now generally available across Jira Software, Confluence, Jira Service Management and more.

Atlassian welcomes AI to the team

Funding

Anthropic is reportedly in talks to raise an additional $750M in a funding round.

This could propel Anthropic’s pre-funding valuation to an impressive $18.4B!

Legal AI Harvey raises $80M Series B from Elad Gil, Kleiner Perkins, OpenAI and Sequoia

https://www.harvey.ai/blog/series-b

Generative AI is churning out unicorns like it’s 2021

These startups are taking far less time to get to $1B+ valuations than most of their unicorn peers: Across the 13 genAI unicorns, the average time to reach unicorn status is 3.6 years but for the unicorn club as a whole the average is 7 years — almost twice as long.

https://www.cbinsights.com/research/generative-ai-unicorns-valuations-revenues-headcount/

Robotics/Embodiment

Embodiment refers to when AI is integrated into a robot.  The most common simple example of that would be Siri or Alexa.  You talk to your device in plain language, and it responds.   A more cutting edge example would be talking to your car’s autopilot in plain language “Let’s stop at the grocery store on the way home” or asking a robot to get something that is not explicitly programmed to do (using the language model to translate your request into code), i.e. “Go to the dining room and get my shoes”.   It can also simply refer to using AI to train a robot to get better at something (like chess or soccer).

“My jaw hit the floor when I watched an AI master, Labrinth, one of the world’s toughest physical games in just six hours”

https://www.techradar.com/computing/artificial-intelligence/my-jaw-hit-the-floor-when-i-watched-an-ai-master-one-of-the-worlds-toughest-physical-games-in-just-six-hours

“Waymo finds that its driverless cars ‘significantly outperformed’ humans”

Waymo finds that its driverless cars ‘significantly outperformed’ humans

“TomTom creates AI-based conversational assistant for vehicles with Microsoft”

https://www.reuters.com/technology/tomtom-creates-ai-based-conversational-assistant-vehicles-with-microsoft-2023-12-19/

“Watch ‘world’s most powerful’ humanoid robot withstand brutal kicks – China”

https://interestingengineering.com/innovation/worlds-most-powerful-robot-withstand-kicks

“Agility is using large language models to communicate with its humanoid robots”

Agility is using large language models to communicate with its humanoid robots

Wearables

AI wearables refer to always-on devices that you passively wear rather than pick up to use.  Smart glasses, pins, watches, rings…. Anything that you wear that can collect data, act on commands, or help you on-demand through voice activation, usually without a keypad or traditional interface.

OpenAI, Meta, Microsoft Chase Wearable AI

“It’s a vision many of the companies have discussed or worked on for years, but they have a new reason to think they can pull it off: the sudden rise of multimodal AI that understands drawings, charts, objects and hand gestures in addition to text and audio.”

https://www.theinformation.com/articles/tech-giants-chase-wearable-ai

Explainers/Discussion

There are some good end of year retrospectives this week.  Sam Altman, CEO of OpenAI, wrote a blog post about what he wished someone had told him before this last year.  Words of wisdom from a guy in the hot seat.  The head of technology from Meta/Facebook also wrote a similar post about what lies ahead. Bill Gates did the same.   Brian Solis created a comprehensive infographic of the AI ecosystem. 

Retrospectives

Sam Altman: What I Wish Someone Had Told Me

https://blog.samaltman.com/what-i-wish-someone-had-told-me

Boz’s End of Year Blog: 2023 was a wild year for technological progress. At Meta, our two biggest long-term bets on the technologies of the future both make great leaps forward but more excitingly, they began to converge. I wrote about it here: 

Living in the Future

Bill Gates: The road ahead reaches a turning point in 2024

This year signaled the start of a new era. Here’s why I believe next year is an opportunity to shape the world’s next chapter for the better.

https://www.gatesnotes.com/The-Year-Ahead-2024

AI Infographic

Introducing the “The GenAI Prism,” the single most comprehensive representation of the generative AI universe in one stunning infographic.

Introducing the GenAI Prism Infographic: A Framework for Collaborating with Generative AI

Other Explainers

A short tour of wonders.

(or how to get a sense of the current state of Gen AI progress, all free, no skills required):

Midjourney v6, Altman ‘Age Reversal’ and Gemini 2 

A 100T Transformer Model Coming? Plus ByteDance Saga and the Mixtral Price Drop

Phi-2, Imagen-2, Optimus-Gen-2: Small New Models to Change the World?

15min History of Reinforcement Learning and Human Feedback

Can Google truly moonshot AI to win it all?

https://stratechery.com/2023/googles-true-moonshot/

Mixture of Experts Explained

https://huggingface.co/blog/moe

Science/Medicine

Spatial Computing glasses for surgeons

We are proud to announce that NuLoupes has received FDA approval!

The authors got GPT-4 to autonomously research, plan, and conduct chemical experiments, including learning how to use lab equipment by reading documentation. Most of the work was done completely by the AI writing code for the lab equipment, but one task had to be done by humans.

https://www.linkedin.com/posts/emollick_this-is-quite-the-paper-in-nature-on-ai-and-activity-7143427105630912512-cRAj

Discovery of a structural class of antibiotics with explainable deep learning

https://www.nature.com/articles/s41586-023-06887-8

Big AI discovery: a new structural class of antibiotics (the last one took 38 years) with multiple compounds effective vs methicillin-resistant Staph aureus, without toxicity

https://www.nature.com/articles/s41586-023-06887-8

The Brain-Implant Company Going For Neuralink’s Jugular

Synchron’s electrodes are delivered via blood vessel

https://spectrum.ieee.org/synchron-bci

AI trained on millions of life stories can predict risk of early death

A model trained on 6 million people’s health, employment and financial records can predict death more accurately than tools used by the insurance industry

https://www.newscientist.com/article/2408480-ai-trained-on-millions-of-life-stories-can-predict-risk-of-early-death/

OpenAI

GPT 4.5 Leaks?

The GPT 4.5 speculation started on Thursday with a ‘leaked‘ image showing a new GPT-4.5 model with new advanced multimodal capabilities and new pricing.

more leaks surfaced over the weekend, with multiple users reporting ChatGPT responding with ‘GPT 4.5-turbo’ when asked what model is currently running.

You can now archive your chats in ChatGPT

Archive removes chats from your sidebar without deleting them. You can see your archived chats in Settings.

OpenAI buffs safety team and gives board veto power on risky AI

OpenAI is expanding its internal safety processes to fend off the threat of harmful AI. A new “safety advisory group” will sit above the technical teams and make recommendations to leadership, and the board has been granted veto power — of course, whether it will actually use it is another question entirely.

OpenAI buffs safety team and gives board veto power on risky AI

OpenAI Prompt Engineering Guide

This guide shares strategies and tactics for getting better results from large language models (sometimes referred to as GPT models) like GPT-4. The methods described here can sometimes be deployed in combination for greater effect. We encourage experimentation to find the methods that work best for you.

https://platform.openai.com/docs/guides/prompt-engineering

OpenAI Prompt Examples

https://platform.openai.com/examples

Open Source

Now there is yet another small open source GPT-3.5 class model, OpenChat. It claims to beat all of the other models of similar size, as well as both the original ChatGPT (3.5) & Grok, in common benchmarks.

https://openchat.team/

Not AI but… 

Significantly Enhancing Adult Intelligence With Gene Editing May Be Possible

https://www.lesswrong.com/posts/JEhW3HDMKzekDShva/significantly-enhancing-adult-intelligence-with-gene-editing

US nuclear-fusion lab enters new era: achieving ‘ignition’ over and over

Researchers at the National Ignition Facility are consistently creating reactions that make more energy than they consume.

https://www.nature.com/articles/d41586-023-04045-8

Chips/Hardware

Transformers etched into silicon

New Company – Etched 

By burning the transformer architecture into our chips, we’re creating the world’s most powerful servers for transformer inference.

https://www.etched.com/

Intel unveils new data center chip with focus on AI growth

https://finance.yahoo.com/news/intel-unveils-new-data-center-chip-with-focus-on-ai-growth-150001734.html

Stable Diffusion Benchmarks: 45 Nvidia, AMD, and Intel GPUs Compared

Which graphics card offers the fastest AI performance?

https://www.tomshardware.com/pc-components/gpus/stable-diffusion-benchmarks

Technical/Dev/IT

VERSES AI claims to have a far better AI technology than the one that builds our current LLMs.

On December 19th 2023, VERSES took out a full page ad in the New York Times with an open letter to the board of OpenAI, citing the assist clause in their charter, which states: “if a value-aligned, safety-conscious project comes close to building AGI before we do, we commit to stop competing with and start assisting this project.”

https://www.verses.ai/blogs/science-and-standards-behind-the-breakthrough

The AI community is predicting that 2024 is going to be the year of the rise of synthetic data. And this is really true. Interested to know more about how those are generated (especially for tabular and text)?

https://blog.premai.io/the-synthetic-data-revolution-paving-the-way-for-agi/

2023 has been incredible for open releases, so I made a year review in Open LLMs

https://huggingface.co/blog/2023-in-llms

Introducing gpt-engineer App👶

since gpt-engineer became the world’s most popular  codegen project I have been tinkering with the next step: how to make it practical, ie allow anyone to build and deploy web–apps with plain english

Chatbot Arena

It allows you to compare and rank the output of 25+ LLMs right from your browser.

It then crowdsources the scores to build a ranking of the top closed/open models.

Here are some interesting takeaways:

– OpenAI remains the king of LLMs

– Closed models still outperform open models (but the gap is closing)

– Mistral 8x7b is the best open-source model right now

– Yi-34B is flying under the radar

LM Studio

Discover, download, and run local LLMs

https://lmstudio.ai/

LangChain State of AI 2023

https://blog.langchain.dev/langchain-state-of-ai-2023/

PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU

Click to access powerinfer-20231219.pdf

Unmasking Deepfake Faces from Videos Using An Explainable Cost-Sensitive Deep Learning Approach

https://arxiv.org/abs/2312.10740v1

Fine Tuning Mistral 7B on Magic the Gathering Drafts

https://generallyintelligent.substack.com/p/fine-tuning-mistral-7b-on-magic-the

testing self-hosted GPT-4 alternative – CogVLM deployed on AWS

you can deploy state-of-the-art LMM on a single NVIDIA T4 GPU

open-source LMMs are getting better and cheaper

Credits/Sources

Most of these links come from just a few incredible sources.  Please follow them:

2 responses to “AI News #12: Week Ending 12/22/2023 with Executive Summary and Top 7 Stories”

  1. The Viggle AI Meme's Impact on Image-to-Video Awareness – Ethan B. Holland

    […] https://ethanbholland.com/2023/12/24/ai-news-11-week-ending-12-22-2023-with-executive-summary-and-to…Here are a few more articles, if you’re interested in skimming about segmentation. I usually put any new links into the AR/VR category of this newsletter. […]

    Reply
  2. A short video of a panda frying an egg in a modern kitchen with a Dodgers hat on – Ethan B. Holland

    […] if you’re not technical, you can follow trends and see patterns and clusters of tools: object segmentation, generative video stitching (like this example), video-to-image mapping (Viggle, LivePortrait), […]

    Reply

Leave a Reply

Trending

Discover more from Ethan B. Holland

Subscribe now to keep reading and get access to the full archive.

Continue reading