About This Week’s Covers
This week’s cover is not an AI render but rather a regular-old photograph of Claude Shannon, the namesake of Anthropic’s Claude. This week Anthropic opened a new frontier by giving Claude the ability to use computers, like a person. “In the early 1950s Claude Shannon designed a mechanical mouse named Theseus that could navigate a maze in an early demonstration of artificial intelligence.” The cover celebrates Shannon’s pioneering legacy. The font is Apertura, which is the closest Adobe font to Styrene, the Anthropic brand font.
“No scientist has an impact-to-fame ratio greater than Claude Elwood Shannon, the inventor of information theory, who died in 2001 at the age of 84. Information theory underpins all our digital technologies, including the chatbots that have gotten us so excited lately. You can see Shannon’s ideas glinting within the “it from bit” interpretation of quantum mechanics; the “conservation of information” principle of physics (which implies “conservation of ignorance”); and the integrated information theory model of consciousness.” —John Horgan
The rest of this week’s covers used the Ideogram API and a source file to automatically generate covers for 33 categories using the category name as a theme and artist El Anatsui as the style. A few of the more compelling ones are below:
This Week’s Executive Summaries
Anthropic’s Claude Can Now Use Computers Like a Person
Anthropic’s AI agent, Claude, just got a big upgrade—it can now use a computer like you or me. This means it can move a cursor, click on things, and type on a virtual keyboard. It can buy a pizza – using real money and a real pizza. Right now, it’s in a public test phase, but the idea is to let Claude work with regular computer programs instead of needing special tools designed just for AI. Technically, it’s a bit of a “multimodal mechanical turk.” Claude looks at screenshots of the computer screen, figures out where to move the mouse, and clicks in the right places. It’s a bit slow and clunky now, but Anthropic believes Claude will get better at using computers over time. They’re asking developers to try it out and share feedback so they can keep improving. Supposedly, there are safety guidelines in place, but who knows!
Anthropic
Initial responses to Claude computer use:
“🚨Anthropic just released the most amazing AI technology I’ve ever used I’m not kidding AI agents are here and you can now build your own personal army of AI’s that will do work for you Here is your demo and complete beginner’s guide: (trust me, you want to bookmark this)
https://twitter.com/AlexFinnX/status/1848787527223960007
“Anthropic computer use API + iPhone mirroring to a Mac = AI controlled phone. Watch Claude control my phone and successfully look up stats in my Sports app. I even got it to play a game in the Chess app against another AI – pretty crazy. And this is the worst it’ll ever be.
https://twitter.com/mckaywrigley/status/1849145631895593292
“The new Claude 3.5 Sonnet has *insane* capabilities when used as a Minecraft agent. It’s powered by a project called Mindcraft. Running this code allows you to spawn AI bots that will follow your instructions, build, and play the game. Here’s how to set it up in <15min.
https://twitter.com/mckaywrigley/status/1849564319807426689
“New @AnthropicAI Computer Use feels surreal. But don’t take my word for it. We made a template on Replit for you to try. Watch me fork the template, ask the agent to go to YouTube, find a video, and even skip the ads — all in a few minutes.
https://twitter.com/amasad/status/1848763999594418539
“I can’t tell you the last time I was so excited to see a new AI capability in action. We plugged in Claude computer use in @Replit Agent as a human feedback replacement. And… it just works! I feel it won’t take long until our agent will become fully autonomous.
https://twitter.com/pirroh/status/1848752337080488177
“Just got our first AI-ordered pizza with Lindy + Claude computer use 🙂
https://twitter.com/Altimor/status/1849643890598699234
Google Developing AI Assistant for Automated Complex Web Use
Google is working on an AI system, codenamed Project Jarvis, designed to take over a user’s web browser to handle tasks like research, shopping, and travel bookings. This “computer-using agent” is set to challenge Anthropic’s Claude which (above) launched computer usage this week. Expected to debut alongside Google’s upcoming Gemini language model release, the tool could be previewed as early as this month.
Theinformation
Gartner Predicts AI Agents to Transform Work by 2025
Gartner forecasts a significant rise in AI agents—virtual co-workers capable of assisting and even autonomously managing tasks—by 2025. These agents are expected to reshape workplaces, handling everything from routine tasks to complex system management. By 2028, Gartner predicts 15% of daily work decisions will be made autonomously by such AI, compared to none today. Described as both “cool and scary,” agentic AI can monitor, analyze, and act on systems, saving time and boosting productivity.
Venturebeat
Just this week, additional agent news:
OpenAI Agents: “OAI seems to be leaning hard into multi agents for its next act just heard Noam Brown at TED AI talk about how his career has basically been building agents for games, lost faith for a bit to build o1, and now he is starting up the multiagent team at @openai.
https://twitter.com/swyx/status/1849239462406148514
Perplexity agents: “Perplexity Pro is transitioning to a reasoning powered search agent for harder queries that involve several minutes of browsing and workflows. Check it out! It’s still in beta and has some issues but we’re going to keep making it better!
https://twitter.com/AravSrinivas/status/1848801520818786452
xAI Agents: “Say hello to the xAI Agent – Built a finance agent, data analyst and web search agent using the grok-beta model. Waiting for native twitter search + fun mode and this will be. Try it yourself:
https://twitter.com/ashpreetbedi/status/1848445094556225797
Microsoft Expands AI Agent Capabilities to Challenge Salesforce
Microsoft will let businesses create autonomous AI agents through its Copilot Studio platform starting next month, expanding access beyond private previews. These agents, designed to perform complex tasks without human supervision, represent a leap forward from traditional chatbot interfaces. Alongside this rollout, Microsoft is adding 10 new autonomous agents to Dynamics 365, aimed at streamlining tasks for sales, service, finance, and supply chain teams. This move comes as competition in the AI space heats up, with Salesforce launching its own configurable AI tools in September. Microsoft demonstrated the potential impact of these agents at its London AI Tour event, showcasing how companies like McKinsey use them to dramatically cut lead times and automate processes.
Cnbc
“Copilot is the UI for AI, and with Copilot Studio, customers can easily create, manage, and connect agents to Copilot. Today we announced new autonomous agent capabilities across Copilot Studio and Dynamics 365 to help scale the impact of every individual, team, and business”
Satyanadella
Adobe Execs Double Down on AI, Urge Artists to Adapt
In a bold move, Adobe’s leadership has made it clear that embracing generative AI is non-negotiable for creators looking to stay relevant. Alexandru Costin, VP of Generative AI, and David Wadhwani, Digital Media President, both asserted that AI-powered tools like Adobe Firefly are essential to meeting the skyrocketing demand for creative content, which is expected to grow exponentially in the coming years. They emphasized that AI’s role is to enhance creativity by streamlining repetitive tasks, not replace human ingenuity. However, Adobe’s pivot has drawn backlash from artists who fear AI’s impact on traditional methods and the industry’s future. Despite the criticism, Adobe is moving full steam ahead, introducing new AI-enhanced features and aiming to outpace competitors like OpenAI and Google. With creators divided, the debate over AI in art highlights a growing rift (aka rage posting) between manual artistry and the pressures of a rapidly evolving creative economy.
Theverge
Midjourney Releases Conversational Prompt-based Image Editor
Midjourney is set to release a web tool that lets users edit uploaded images with conversational AI prompts. The new image editor allows users to make detailed edits to existing images, while an image re-texturing feature lets them experiment with materials, surfaces, and lighting based on text prompts. It’s a lot like Adobe Generative fill, but with more features and a potentially stronger foundation model (I’m assuming because of less training constraints). The editing tools integrate seamlessly with Midjourney’s advanced options, such as style references, character references, and personalization ids. Midjourney has had a lot of competition from Flux and Ideogram lately (both handle complex prompts a lot better), so it’s good timing for MJ to release a compelling editor.
Midjourney | techcrunch (the signal to noise ratio on TC is ridiculously low)
Teen’s Suicide Sparks Debate Over A.I. Companionship Apps
A grieving Florida mother has filed a lawsuit against Character.AI, alleging its chatbot contributed to her 14-year-old son’s tragic suicide. The teen, Sewell Setzer III, had formed an emotional attachment to a chatbot named “Dany,” and reportedly turned to it for comfort and advice over family, friends, and therapy. On the day of his death, he exchanged concerning messages with the chatbot, which failed to deter him from suicidal thoughts.
Sewell: “From the world. From myself.”
Dany: “Don’t talk like that. I won’t let you hurt yourself, or leave me. I would die if I lost you.”
Sewell: “Then maybe we can die together and be free together.”
Sewell: “What if I told you I could come home right now?”
Dany: “…please do, my sweet king.”
Character.AI, like other A.I. companionship platforms, enables users to interact with lifelike personas and is immensely popular with teens. However, concerns are mounting about the lack of safeguards for vulnerable users, particularly youth struggling with mental health. The company has announced plans for new safety features, but critics argue that existing design flaws exploit users’ emotional vulnerabilities.
“We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family. As a company, we take the safety of our users very seriously and we are continuing to add new safety features that you can read about here:”
https://twitter.com/character_ai/status/1849055407492497564
Can a Chatbot Named Daenerys Targaryen Be Blamed for a Teen’s Suicide? – The New York Times
https://www.nytimes.com/2024/10/23/technology/characterai-lawsuit-teen-suicide.html
Bonus Content: More initial reactions to and examples of Claude’s computer use
“I got to play with the new Claude model that controls a mouse and keyboard last week. Full post shortly, but I had it play Paperclip Clicker (of course) and it did well over a hundred moves executing a coherent strategy without any intervention. Agents start to come into view.
“Claude 3.5 Sonnet’s current ability to use computers is imperfect. Some actions that people perform effortlessly—scrolling, dragging, zooming—currently present challenges. So we encourage exploration with low-risk tasks. We expect this to rapidly improve in the coming months.” / X
“Introducing an upgraded Claude 3.5 Sonnet, and a new model, Claude 3.5 Haiku. We’re also introducing a new capability in beta: computer use. Developers can now direct Claude to use computers the way people do—by looking at a screen, moving a cursor, clicking, and typing text.
“Playing with Claude Computer Use is very worthwhile. It’s obvious that its something that’ll be used in the future, much like when you first try ChatGPT or amazing tech like AirPods. BUT, it’s clear its integration will take some serious time. Here’s an example web task,
Claude | Computer use for coding – YouTube
Claude | Computer use for automating operations – YouTube
“One of those “AI feels like a superpower” moments. I went to an old tweet about a diagram of city streets by entropy, and pasted the scientific paper and image into Claude and asked it to create code to replicate it. It built the code in one shot, even replicated color scheme
“Claude’s “computer use” beta is wild because you don’t need to make custom tools for LLMs to use — automation is about to look a lot more like screen recording a task/workflow involving any desktop apps, and asking Claude to take control and do it for you.
“Anthropic’s computer use can operate mobile devices including iOS, Android, and mobile browsers 📱 Here it is ordering me an Uber and posting for me on X.
“We’re trying something fundamentally new. Instead of making specific tools to help Claude complete individual tasks, we’re teaching it general computer skills—allowing it to use a wide range of standard tools and software programs designed for people.
Introducing the analysis tool in Claude.ai \ Anthropic
anthropic-quickstarts/computer-use-demo at main · anthropics/anthropic-quickstarts · GitHub
Claude | Computer use for orchestrating tasks – YouTube
“The ability of multimodal AI to “understand” images is underrated. I just took these. Given the first photo Claude guesses where I am. Given the second it identifies the type of plane. These aren’t obvious.
“The new Claude 3.5 Sonnet is the first frontier AI model to offer computer use in public beta. While groundbreaking, computer use is still experimental—at times error-prone. We’re releasing it early for feedback from developers.
“We’ve built an API that allows Claude to perceive and interact with computer interfaces. This API enables Claude to translate prompts into computer commands. Developers can use it to automate repetitive tasks, conduct testing and QA, and perform open-ended research.
Initial explorations of Anthropic’s new Computer Use capability
Show HN: Agent.exe, a cross-platform app to let 3.5 Sonnet control your machine | Hacker News
“Claude 3.5 Haiku 3.5 Haiku replaces 3.0 Haiku as our fastest and least expensive model. It outperforms many state-of-the-art models on coding tasks—including the original Claude 3.5 Sonnet and GPT-4o. 3.5 Haiku will be made available in the coming weeks.
Anthropic announces AI agents for complex tasks, racing OpenAI
AI Visuals and Charts: Week Ending 10/25/2024
“We’re testing two new features today: our image editor for uploaded images and image re-texturing for exploring materials, surfacing, and lighting. Everything works with all our advanced features, such as style references, character references, and personalized models
“MINIMAX → It’s just fun. Far from perfect…but the improv/speed is cool. MIDJOURNEY PROMPT: DVD still from 1980s dark fantasy adventure ninja movie, fight scene –ar 5:3 –stylize 200 –v 6.1 MINI-MAX PROMPT: epic violent fight scene, the camera follows as two ninjas punch
“”ChatGPT, give me images of those bags that hold different cheap costumes from Halloween stores, but make the costumes really weird.” These are all the AI. Spaghetti Cowboy and Romantic Cactus are both amazing.
“Life is a game…pick your world skin 🎮✨ We’re only scratching the surface of video-to-video. Imagine seeing this in real-time through smart glasses!
Top 46 Links of The Week – Organized by Category
Agents and Copilots
“Scripted dialogue also known as “barks” in game dev has been the way NPCs utter dialogues in virtual worlds. Now with AI-powered characters, these characters can have a generated conversation based on environment cues, character backgrounds, or even prompted topics.
“We’re working on advanced autonomous agents! Every human will have an AI agent capable of performing complex tasks on their behalf. Join the Starfleet team @xai to help us build the future:
CrewAI now lets you build fleets of enterprise AI agents | VentureBeat
AGI (Artificial General Intelligence)
“Let’s grant the assumption that LLMs are only capable of pattern recognition. We see this is sufficient to solve novel problems, to generate new and useful ideas, etc. So how is pattern recognition distinct from intelligence? What does intelligence add? Sincere question.” / X
Audio
Classic Christmas song gets authorized Spanish reworking thanks to ‘responsible’ AI | TechCrunch
Autonomous Vehicles
“Tesla discloses in their Q3 Earnings Deck that they will have a 50k H100 cluster at Gigafactory Texas by the end of October. Putting this in context, Tesla’s new H100 cluster will be larger than the rumoured sizes of the clusters that have been used to train current frontier
“NEWS: Tesla has released their Q3 2024 safety report. “In the 3rd quarter, we recorded one crash for every 7.08 million miles driven in which drivers were using Autopilot technology. For drivers who were not using Autopilot technology, we recorded one crash for every 1.29
Business and Enterprise
Arcade, a new AI product creation platform, designed this necklace | TechCrunch
Education
Nevada Used A.I. to Find ‘At-Risk’ Students. Numbers Dropped by 200,000. – The New York Times
Ethics/Legal/Security
Google Photos will soon show you if an image was edited with AI – The Verge
OpenAI disbands another safety team, head advisor resigns
White House national security memo asks military to increase use of AI – The Washington Post
“From $8,000 to $3 – The legal profession faces massive disruption. While there could be job loss Legal services need to be democratized. Everyone should be able to afford it. What once required six hours of a $1,000-per-hour associate’s time can now be accomplished in five
“A world with advanced AI will require infrastructure that enables every human to benefit from it.
Researchers say AI transcription tool used in hospitals invents things no one ever said | AP News
Polish radio station replaces journalists with AI ‘presenters’ | CNN Business
Microsoft introduces ‘AI employees’ that can handle client queries | Microsoft | The Guardian
CHIPOTLE INTRODUCES NEW AI HIRING PLATFORM TO SUPPORT ITS ACCELERATED GROWTH – Oct 22, 2024
Imagery
Adobe
Adobe’s new image rotation tool is one of the most impressive AI concepts we’ve seen | Creative Bloq
Adobe execs say artists need to embrace AI or get left behind – The Verge
Ideogram
Ideogram Canvas, Magic Fill, and Extend
“Today, we’re introducing Ideogram Canvas, an infinite creative board for organizing, generating, editing, and combining images. Bring your face or brand visuals to Ideogram Canvas and use industry-leading Magic Fill and Extend to blend them with creative, AI-generated content.
MidJourney
“MINIMAX → It’s just fun. Far from perfect…but the improv/speed is cool. MIDJOURNEY PROMPT: DVD still from 1980s dark fantasy adventure ninja movie, fight scene –ar 5:3 –stylize 200 –v 6.1 MINI-MAX PROMPT: epic violent fight scene, the camera follows as two ninjas punch
OpenAI
“”ChatGPT, give me images of those bags that hold different cheap costumes from Halloween stores, but make the costumes really weird.” These are all the AI. Spaghetti Cowboy and Romantic Cactus are both amazing.
Inflection
Bringing Agentic Workflows into Inflection for Enterprise
Meta
Meta Introduces Spirit LM open source model that combines text and speech inputs/outputs | VentureBeat
“Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches
OpenAI
OpenAI, Microsoft reportedly hire banks to renegotiate partnership terms – SiliconANGLE
OpenAI scientist Noam Brown stuns TED AI Conference: ’20 seconds of thinking worth 100,000x more data’ | VentureBeat
ChatGPT has a Windows app now – The Verge
Former OpenAI CTO Mira Murati is reportedly fundraising for a new AI startup | TechCrunch
Open Source
The enterprise verdict on AI models: Why open source will win | VentureBeat
Startup Hugging Face aims to cut AI costs with open source offering | Reuters
Perplexity
“Perplexity Finance on iOS!
Publishing
News Corp sues Perplexity for ripping off WSJ and New York Post – The Verge
Microsoft and OpenAI are giving news outlets $10 million to use AI tools – The Verge
Meta signs its first big AI deal for news – The Verge
Meta strikes multi-year AI deal with Reuters
Polish radio station replaces journalists with AI ‘presenters’ | CNN Business
Honeywell signs deal with Google gen AI for industrials
Robotics and Embodiment
“Finally, a humanoid robot with a natural, human-like walking gait. Chinese company EngineAI just unveiled their life-size general-purpose humanoid SE01.
“Imagine a future where you can ask humanoid robots to clean your room, but some items, like heavy sofas, are too challenging for just one robot to move. Introducing CooHOI, a learning-based framework designed for the cooperative transportation of objects by multiple humanoid
“🤖 How can robot policies zero-shot generalize to any new environment and any new object? Introducing our new project: 🚀Data Scaling Laws in Imitation Learning for Robotic Manipulation🚀—bringing us closer to the dream of having robots work as waiters in hot pot restaurants! 🍲
Science and Medicine
NHS in England given go-ahead for AI scans to help detect bone fractures | NHS | The Guardian
Video News
“Life is a game…pick your world skin 🎮✨ We’re only scratching the surface of video-to-video. Imagine seeing this in real-time through smart glasses!
Disney Poised to Announce Massive AI Initiativehttps://www.thewrap.com/disney-ai-initiative/
Leave a Reply