This week’s cover depicts an oil painting of Dr. Christian Szell from the movie “Marathon Man”. In the movie, Szell repeatedly asks “Is it safe?”. This week Anthropic released a paper warning that language models’ safety guardrails can be thwarted by flooding their memory with repeated questions. In the movie, Szell is a dentist and holds dental equipment. On the cover, Szell is depicted questioning a phone. The oil painting nods to Anthropic’s model, Claude, through the style of Claude Monet. Image created in MidJourney. Title font is layered bold Arial Black to mimic the movie poster’s style. Gradient and noise added by hand using the paint tool in Photoshop.
Executive Summary
- Apple announces “on-device model that ‘substantially outperforms’ GPT-4”: To me, this sounds a lot like ‘AI that can fit on your phone and beats GPT-4’. Better yet, the model can understand what is showing on your phone’s screen, what apps are active (on or off screen), and interact between them. That sounds a lot like Rabbit or Open Interpreter, both of which could kill the concept of an app store. Expect Apple to drop some big announcements at the Apple event on June 10th.
- Anthropic’s Claude is still ahead of OpenAI’s GPT-4: Do you know about Claude? You should. “Claude 3 is the best model in the market right now, overtaking GPT-4. Claude 3 Opus is #1 in the Arena Leaderboard (beating GPT-4.) Opus is a huge model, but Claude 3 Haiku is cheap and fast. And it’s also beating GPT-4 0613! – @svpino
- Google’s Waymo driverless cars are delivering Uber Eats: “The new offering is limited to Uber Eats users in Phoenix, Chandler, Mesa and Tempe, with ordering available from select merchants in those cities.”
- Google might put AI search results behind a paywall: According to the Financial Times, Google is looking to add AI-powered search features to its premium subscription services.
- Anthropic discovers a major security issue: As long as a language model has a memory limit, it can be tricked simply by repeating questions until the memory is full. In a rare press release, Anthropic announced their findings, warned the community, and asked for a unified effort to fix it.
- Google Gemini is great at understanding video: “AIs are really impressive at figuring out context from video. I uploaded a minute of me playing Balatro, a new roguelike deckbuilding poker game, and Gemini 1.5 was able to figure out the core gameplay loop from watching me play once.” –Ethan Mollick
- AI search company Perplexity plans to sell ads: Generative AI search engine Perplexity, which claims to be a Google competitor and recently snagged a $73.6 million Series B funding from investors like Jeff Bezos, is going to start selling ads, the company told ADWEEK.
- X releasing Grok 1.5 and training Grok 2: “Announcing Grok-1.5: improved reasoning capabilities and a context length of 128,000 tokens”
- Cohere releases Command R: an open source model with a focus on multiple language support (English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese) as well as RAG (basically you can upload documents and it can refer to them) capabilities.
Top 48 Links of The Week
These are the must-click links, in order by topic. Even if they look boring, click them! I did the work, so you don’t have to worry. All are 10/10 would recommend.
- AI gets attention but CGI is king:
- Forget AI video, look at this video and pick your jaw off the floor. Why is no one up in arms about CGI? How do we define what’s AI? “Cyberpunk 2077: POV Photorealistic Motorbike Ride in 8K | Real Life Graphics Mod – YouTube”
- Apple is making their move:
- “In a research paper, Apple researchers introduced ReALM. It’s a new AI system that can understand on-screen tasks, conversational context, and background processes. Researchers said it ‘substantially outperformed’ GPT-4. Read more:
- Apple AI researchers boast useful on-device model that ‘substantially outperforms’ GPT-4
- Apple’s VR headset launches “personas”
- “First big Vision Pro update: Shared experiences. “Spatial personas” This was at the top of my list of what was missing from Vision Pro – the floating head is a little odd looking, but the principle is there: Interact with the same content/game/object as someone else in realtime https://twitter.com/MKBHD/status/1775178427328614717
- Now Apple Vision Pro Personas can float freely across different apps – The Verge – https://www.theverge.com/2024/4/2/24118863/apple-vision-pro-spatial-persona-shareplay
- Media Analysis
- “I used Anthropic Claude 3 Opus model to analyse all 63 Podcasts from Dwarkesh Patel’s channel and extracted useful book recommendations, career advices, interesting ideas, learning tips etc 1.49Million tokens processed, costs $23
- “AIs are really impressive at figuring out context from video. I uploaded a minute of me playing Balatro, a new roguelike deckbuilding poker game, and Gemini 1.5 was able to figure out the core gameplay loop from watching me play once. Surprisingly good insights.
- AI agents
- Open AI’s case study on legal AI agent, Harvey. OpenAI customer story: Harvey
- “I asked Gemini 1.5 Pro to analyze tech startup Rubrik’s S-1, the IPO document, and tell me if I should buy or sell. It read ~200k words, wrote a ~500 word memo in ~61s with NO hallucinations. Wall Street pays analysts $3000+ for a week of “specialized” work, and this cost $1.5!
- “We created an AI platform that can analyse hundrets of pitch decks with a single upload or as embedding field on an incubator/VC page. Seems obvious to utilize when having thousands of pitch decks to go through each month.
- “Cool. An LLM chat tool for SEC filings and earnings transcripts.
- “pretty crazy that GPT3.5 can outperform GPT4 through an agentic workflow reminds me of folks I’ve worked with in my career who outperformed people who were much smarter merely because they had a better process
- “I gave Devin (an AI agent) my Netify key & asked it to build a website on how to rebuild civilization from scratch. It built a site with react quizzes, downloadable preparedness documents & free images. Kind of impressive, actually. Try it, all just Devin:
- “The more I play with early versions of AI agents the more I think that this is going to be a big move forward in the near future. Giving AIs planning and delegation seems to help a lot. Both Devin & MAGIS agents perform 8x better than base GPT-4 in resolving real Github issues.
- Scientists create AI models that can talk to each other and pass on skills with limited human input
- Security Risks
- “One of the most impressive (and concerning) demos of AI security I’ve seen. If your company uses pip or npm you’re not just bugged, you’re bug bug bug bugged bugged bugged bugged hundreds of times after every update. Snyk/npmregistry DON’T catch most of these Yes you. Your data.”
- “New Anthropic research paper: Many-shot jailbreaking. We study a long-context jailbreaking technique that is effective on most large language models, including those developed by Anthropic and many of our peers. Read our blog post and the paper here:
- Many-shot jailbreaking \ Anthropic
- “This is the most effective, reliable, and hard to train away jailbreak I know of. It’s also principled (based on in-context learning) and predictably gets worse with model scale and context length.”
- “New jailbreaking technique: pure repetition. AIs are getting big context windows, it turns out if you fill a lot of it with examples of bad behavior, the AI becomes much more willing to breach its own guardrails. Security people are used to rules-based systems. This is weirder.
- Top recent ten AI developments
- “Research on LLMs is moving quickly, and even models / techniques that have been state-of-the-art for a long time (e.g., GPT-4 and Mixtral) are being quickly dethroned. Here’s a list of my top ten AI developments (each with a brief summary) over the last few months:
- AI song editing
- “welp, figured out how to change singers and edit lyrics of ALREADY generated Suno songs and… ya, I’m crying RN😂 https://twitter.com/amli_art/status/1774875267820872175
- Self-driving Waymo cars are now delivering Uber Eats:
- Waymo self-driving cars are delivering Uber Eats orders for first time
- Phoenix residents can now experience Uber Eats delivery with the Waymo Driver
- Talent wars and funding
- Meta Pursues AI Talent With Quick Offers, Emails From Zuckerberg — The Information
- Elon Musk Boosts AI Engineer Pay in ‘Craziest Talent War’ – WSJ
- Tesla Raising Pay for AI Engineers To Counter Poaching, Musk Says
- “The AI tsunami is coming. @ycombinator’s W24 mega-batch of 157 AI startups touching 20+ different sectors gives us a glimpse of where AI is headed this year. Here’s a breakdown of the top 6 themes 🧵👇
- “$6.4B+ raised by AI Startups in March across 170+ rounds (up from $5.1B in February) Here’s a recap of the biggest deals:
- Removing 40% of an LLM with no decrease in performance. LLabotomy? Seems sketchy.
- “A new method was able to delete 40% of LLM layers with no drop in accuracy. This makes them mich cheaper and faster. The method combines pruning, quantization and PEFT. They tested this across various open source models. Each family of models had a maximum amount of layers… https://twitter.com/AlphaSignalAI/status/1774858806817906971
- AI investment metric
- “WSJ: The AI industry spent 17x more on Nvidia chips than it brought in in revenue
- Earthquake shines a light on risk
- Taiwan Earthquake Puts TSMC Advanced Chips Production At Risk – Bloomberg
- AI Surveillance
- Lumana launches new AI video surveillance system | Security Systems News
- Open AI Dalle Image Editing
- Editing your images with DALL·E | OpenAI Help Center
- DALL-E now lets you edit images in ChatGPT – The Verge
- Google AI search paywall?
- Google looks to AI paywall option, claims report – https://www.bbc.com/news/business-68727857
- Google might make SGE paid, not working on ad-free Search – https://9to5google.com/2024/04/03/google-ad-free-search/
- Science and Medicine
- “It is remarkable how routine it has become for careful studies to show that GPT-4 (not trained specifically for medicine) outperforms most doctors in key aspects of diagnosis. That doesn’t mean that GPT-4 is reliable in all circumstances, but it still seems like a big deal.
- Generative AI develops potential new drugs for antibiotic-resistant bacteria | News Center | Stanford Medicine
- FDA approves AI-driven test for sepsis made by Prenosis – The Washington Post –
- “How good is AI at clinical reasoning? An LLM was better than physicians in processing medical data and clinical reasoning using recognizable frameworks as measured by R-IDEA”
- Human-Level Performance in Data Science Competitions
- The downfall of Stability AI
- How Stability AI Founder Emad Mostaque Tanked His Billion-Dollar Startup
- Powerful new open-source model, Cohere, focuses on language and document referral
- “Command R+ (⌘ R+) is our most capable model (with open weights!) yet! I’m particularly excited about its multilingual capabilities. It should do pretty well in 10 languages (English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic, and Chinese).
- “In a lot of other models, Arabic/اللغة العربية (and other non-English languages) are more expensive to use because the tokenizer basically punishes other languages by requiring a lot more tokens to represent their text. Command R+ improves the tokenization, requiring less tokens
- “As RAG gets more complicated, it starts to look more agentic 🤖 @cohere released a new open weights (!!) model that is a perfect fit for this @RLanceMartin made a killer video on it – check it out below thanks to @aidangomez and everyone at cohere for this!!”
- Introducing Command R+: A Scalable LLM Built for Business
AI Visuals and Charts: Week Ending 04/05/2024
Just how much can you paste into a prompt in Gemini? Ten Harry Potter books!
Image via: https://www.linkedin.com/in/eduardoviteri/
The cost of AI reasoning over time. – by Karina Nguyen
The Rest: AI News of The Week
Don’t let the volume overwhelm you. Have fun and skim these. The links are organized by topic, sorted from ‘coolest’ to ‘least cool’, and each topic is clearly defined with a headline. I’ve added a description and glossary of what the topics mean, beneath each label, in plain language. I do the work so you don’t have to! When you visit the pages, note that the links and descriptions are often pulled directly from tweets or articles, so it’s not always my voice. Pause when you see something that interests you. Reach out to me any time. I enjoy sharing and discussing these items.
Agency/Agents/Copilots News of the Week: Agency is when AI can do things for you (like Googling an actress name or fetching the latest weather forecast). An agent is one step further, when AI given autonomy to take action on your behalf (“Alexa, book a reservation for three at Peak in Hudson Yards for Friday night”). A co-pilot is an assistant (like spell check or autofill).
This weeks’s latest agent news: https://ethanbholland.com/2024/04/06/agents-and-copilots-ai-news-week-ending-04-05-2024/
Anthropic News of the Week:
Anthropic is a company that builds LLMs like OpenAI, Mistral, Meta, etc. Their main AI brand is Claude. As with Amazon and Apple, individual Anthropic company posts will often be placed in the categories they match (image, audio, agents, robots, etc). Occasionally, I’ll dedicate space to a company’s news if it’s broad or a major product release.
This week’s Anthropic news: https://ethanbholland.com/2024/04/06/anthropic-news-week-ending-04-05-2024/
Apple News of the Week: As with Amazon, individual Apple company products will often be placed in the categories they match (image, audio, agents, robots, etc). Occasionally, I’ll dedicate space to a company’s news if it’s broad or a major product release.
This weeks’ latest Apple AI news: https://ethanbholland.com/2024/04/06/apple-ai-news-week-ending-04-05-2024/
Augmented and Virtual Reality (AR/VR) News of the Week: Augmented reality is when you see images or information on top of the real world. A car windshield with a heads-up display of the speed. Or glasses that have facial recognition and overlay the names of everyone in view. Virtual reality is when you are transported into another place, usually wearing goggles, but a flight simulator could also be considered virtual reality.
This weeks’s latest AR/VR news: https://ethanbholland.com/2024/04/06/augmented-and-virtual-reality-ar-vr-news-week-ending-04-05-2024/
Autonomous Vehicles/Driverless Cars News of the Week: Driverless car news doesn’t always get its own category, because it’s so close to robot embodiment. I go with my gut each week around what to place in each category. My recommendation would be to follow Robotics/Embodiment also, as the two fields are converging.
This week’s autonomous vehicle news: https://ethanbholland.com/2024/04/06/autonomous-vehicles-news-week-ending-04-05-2024/
AI Audio News of the Week: In this case, AI audio can mean a few things. The first is “generative audio” which refers to creating sounds with AI, much like ChatGPT writes words or MidJourney creates images. For example, asking for the “sound of waves crashing on the beach” would be text to sound. Another example would be an AI ‘watching’ a video and adding sound to it, like a foley artist would add footsteps or a creaking door to a movie scene. Lastly, AI audio can refer to microphones that only pick up certain speaker’s voices or headsets that cancel out all voices but your friends. This week’s latest AI audio news: https://ethanbholland.com/2024/04/06/ai-audio-news-week-ending-04-05-2024/
Business/EnterpriseAI News of the Week: This broad category is for stories that impact corporations and large scale AI implementation. Enterprise refers to a type of AI that is often custom built for a business or leverage an API to connect secure data to an AI model.
This weeks’s latest enterprise AI news: https://ethanbholland.com/2024/04/06/business-and-enterprise-ai-news-week-ending-04-05-2024/
Chips and Hardware AI News of the Week: Most of the chip news is NVIDA usually, yet more and more Meta, Google, and OpenAI are starting toward their own manufacturing. I have to make the call whether to put Meta, Google, and OpenAI’s chip news under this section or their company sections. Lately, I’m putting each company’s chips news into the company category, rather than the chips category. This is the rest of the chips headlines.
This weeks’s latest chips and hardware news: https://ethanbholland.com/2024/04/06/chips-and-hardware-week-ending-04-05-2024/
Ethics/Legal/Security AI News of the Week: This section focuses on the impact AI is having on ethics (deep fakes, war, trust, false information, plagiarism, job loss, income), legal (rights, laws, regulations), and security (hacking, phishing, national interests, safety). For huge news stories like the NY Times suing OpenAI, I usually put them under the main section or give them their own page.
This weeks’s latest AI ethics/legal/security news: https://ethanbholland.com/2024/04/06/ethics-legal-security-ai-news-week-ending-04-05-2024/
Google AI News of the Week: Individual company products will often be placed in the categories they match (image, audio, agents, robots, etc). Occasionally, I’ll dedicate space to a company’s news if it’s broad
This weeks’s latest Google AI news: https://ethanbholland.com/2024/04/06/google-ai-news-week-ending-04-05-2024/
Imagery News of the Week: AI imagery covers “generative AI” image tools. This usually text-to-image, where a user enters a prompt (“a polar bear walking through NYC”) and a tool like Dalle or MidJourney generates an image in the likeness of the description. This is different than AI vision, where an AI “looks at” an image and can derive context, details, and contents. AI vision is a subset of AI called multimodality. Imagery, in this case, is for image creation and modification/editing. Adobe Photoshop’s AI tools would fall into this category. I’ll also include things like automatic masking and object removal, even though that’s in between imagery and vision… but practically speaking it fits into editing.
This weeks’s latest AI image news: https://ethanbholland.com/2024/04/06/imagery-news-week-ending-04-05-2024/
International AI News of the Week: A lot of international news will get cross listed in the chips, security, or open-source categories, however it’s nice to have a separate category for worldwide AI news.
This week’s latest international AI news: https://ethanbholland.com/2024/04/06/international-ai-news-week-ending-04-05-2024/
Locally Run AI Models News of the Week: This is a niche mostly for serious AI followers. It refers to AI that can be privately downloaded and run on a device without an internet connection. These have an array of powerful implications, from ethics of rogue users with untethered agents, to practical uses like Apple running a full AI on your phone, to corporate installations for security, to embodied robots with AI running in their virtual brain.
This weeks’s latest locally run AI news: https://ethanbholland.com/2024/04/20/locally-run-ai-models-news-week-ending-04-05-2024/
Meta AI News of the Week: This is a space dedicated for Meta specific AI advancements and news stories.
This weeks Meta AI news: https://ethanbholland.com/2024/04/06/meta-ai-news-week-ending-04-05-2024/
Microsoft AI News of the Week: This is a space dedicated for Microsoft specific AI advancements and news stories.
This weeks Microsoft AI news: https://ethanbholland.com/2024/04/06/microsoft-ai-news-week-ending-04-05-2024/
Mobile AI News of the Week: In April, 2024 I added a dedicated category for mobile. Prior, I put all most the mobile news into either the company (Apple v. Google v. Microsoft) or locally run AI. It also ended up in the chips and hardware section, or the consumer products category. There is enough mobile news to at least start cross linking it all in one place. This weeks’s latest mobile AI news: https://ethanbholland.com/2024/04/06/mobile/
Multimodal AI News of the Week: This is a broad topic for an single AI model that demonstrates an ability to interact with more than one modality (imagery, video, audio, text). Often multimodal news will end up in one of these categories. I’m playing it by ear on a case by case basis. Please be patient with my organizational challenges.
This week’s multimodal AI news: https://ethanbholland.com/2024/04/06/multimodality-news-week-ending-04-05-2024/
OpenAI: OpenAI is the leading force in the AI boom of 2023 and now 2024. This section focuses on news that is specific to OpenAI. This section will compete with all of the other sections (imagery, vision, ethics, etc) because OpenAI is so broad. I won’t be able to consistently pick when to put things under OpenAI or other sections, so bear with me.
This weeks’s latest OpenAI news: https://ethanbholland.com/2024/04/06/openai-news-week-ending-04-05-2024/
Open Source Models: An open source AI model refers to a class of artificial intelligence models with public source code. They can be inspected, copied, installed, and customized on private computers. In contrast, a closed source model is proprietary and owned by a company that you pay to use (like PowerPoint or Photoshop). One of the most famous open source language models is a French model called Mistral. Its code is completely publicly available, and anyone can download it and customize it. On one hand, open source is a transparent and powerful way to democratize AI, but on the other hand, open source models circumvent the guard rails and copyright protections that private companies implement. Open source models are the wild west of artificial intelligence, but also the potential saving grace (depending on who you ask). It’s a bit like gun control debates but for computing power.
This weeks’s latest open source news: https://ethanbholland.com/2024/04/06/open-source-ai-news-week-ending-04-05-2024/
Podcast/YouTube Clips of the Week: This is for more general interviews and explainer videos and podcasts that provide access to leadership, demos of new products, and walkthroughs and tutorials. Videos focused on specific topics will live in the topic category (i.e. images), but broader videos will live here.
This weeks’s latest podcasts and YouTube clips: https://ethanbholland.com/2024/04/06/podcasts-youtube-op-eds-week-ending-04-05-2024/
Publishing AI News of the Week: These are stories about AI’s impact on the publishing industry. From copyright and crawling to the death of page views or even the end of browsers.
This weeks’s latest publishing AI news: https://ethanbholland.com/2024/04/06/publishing-news-week-ending-04-05-2024/
Robotics/Embodiment News of the Week: This is the most intense area of AI. Embodiment refers to putting an AI inside of a machine. It’s “embodying” the object and therefore giving a robot agency in the real world. An example would be using a large language model as an interface to a complex coding task. Just as you ask “Alexa, play Bad Blood by Taylor Swift on Spotify” using plain language, with embodiment you could ask a robot to “Go to the laundry basket and bring me all of the red shirts”. The language model in the robot would translate your request into the proper code to go get the red shirts. The robot was never trained on the task. Another type of embodiment would be training a robot using virtual reality simulations. Using an simulation, a robot could be trained on thousands of scenarios until the real world can be swapped out and the robot doesn’t “notice”. This section also includes factory automation and human prosthetics. There will be some overlap with other categories like autonomous vehicles. I first learned about embodiment from Alan Thompson. I highly recommend his video explainer: https://youtu.be/peLqYP9BAUg?si=2FzrvDlw-qaQFaCx.
This week’s latest robot and embodiment AI news: https://ethanbholland.com/2024/04/06/robotics-and-embodiment-news-week-ending-04-05-2024/
Science/Medicine AI News of the Week: AI’s strength is learning patterns. This applies nicely to medical diagnosis and identifying trends. When combined with data and AI vision, this means AI is good at looking at x-rays. Language models are helping with patient interface, and robotics and augmented reality are advancing surgery. Powerful enterprise models like Google’s Alphafold can master protein folding. Other models can read ancient scrolls without opening them.
This weeks’s latest AI science and medicine news: https://ethanbholland.com/2024/04/06/science-and-medicine-news-week-ending-04-05-2024/
AI Video News of the Week: AI video in this case refers to generative video. Much like imagery meant generative imagery. This usually text-to-video, where a user enters a prompt (“a wizard walking out of a flaming building”) and a tool like Pika or Runway generates an video in the likeness of the description. It also covers animation of still images, where an image is given motion (like a photo of a waterfall appearing to have flowing water). As with images, this is different than AI vision, where an AI “looks at” an image or video and can derive context, details, and contents. Video, in this case, is video creation and modification/editing.
This weeks’s latest AI video news: https://ethanbholland.com/2024/04/20/video-news-week-ending-04-05-2024/
X/Twitter/Grok: Grok is one of several AI’s developed by X, and it’s a bit blended in with Telsa and other Elon Musk technology. Not every week will have a Grok section, but like Meta, Google, Apple, and OpenAI, X will be in the news enough to have its own section.
This week’s latest X news: https://ethanbholland.com/2024/04/06/tech-and-development-news-week-ending-04-05-2024/
Technical and AI Developer News of the Week: Everything that is too technical for general consumption goes here. These are stories I think are important, but might be inaccessible and confusing. It’s also a space for developer news and deep dives into how AI works, under the hood.
This weeks technical and dev AI news: https://ethanbholland.com/2024/04/06/twitter-x-grok-week-ending-04-05-2024/
Credits/Sources
Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.
- Robert Scoble: https://x.com/Scobleizer
- Ethan Mollick: https://www.linkedin.com/in/emollick/
- Alan Thompson: https://lifearchitect.ai/
- Theoretically Media: https://www.youtube.com/@TheoreticallyMedia
- The Rundown: https://www.therundown.ai/
- Bilawal Sidhu: https://twitter.com/bilawalsidhu/
- TLDR: https://tldr.tech/ai
- Jeremiah Owyang: https://twitter.com/jowyang
- Nick St. Pierre: https://twitter.com/nickfloats
- Dr. Jim Fan: https://twitter.com/DrJimFan
- All About AI: https://www.youtube.com/@AllAboutAI
- Marshall Kirkpatrick: https://aitimetoimpact.com/
- AI News (Smol Talk): https://buttondown.email/ainews/archive/
For previous issues, please visit the archives!
Thanks for reading!
Leave a Reply