a map lays on the ground in a forest with a trail sign that reads “Open Source” –ar 5:3 –style raw
This week’s category cover theme is a sign in a forest. Each category image prompt is a derivative of the formula “an [category themed object] in a forest with a trail sign that reads “[category name]”. Using a theme each week takes the cover creation time down to about 20 minutes, rather than several hours.
“📰New open models this week: multilinguality, long contexts, and VLMs 🔥 – CogVLM2: multimodal conversational – Yi 1.5 long context – M2-BERT-V2, long-context encoder models – Phi 3 small and medium + vision – Falcon VLM – Mistral 7B 0.3 – Aya 23: multilingual
“BREAKING: California’s newly passed AI bill 📌 “Covered models” trained with over 10^26 FLOPS must be incapable of enabling certain critical harms like creation of WMDs, even if fine-tuned. 📌 Developers must implement the capability to promptly enact a full shutdown of covered
“This California Bill Makes Zero Sense And Is Targeted At Banning Open-Source AI The bill stipulates that if you train a “model” with some arbitrary amount of compute, then a whole bunch of rules and restrictions apply. Imagine if I stopped training just short of that compute
IBM makes more AI models open source and lands Saudi Arabia deal
Open LLM Leaderboard: DROP deep dive
“6/ on the national security front, the US has won by leading. restricting open-source AI won’t stop determined adversaries, only slow U.S. innovation and cede that leadership. openness keeps us on the offense. we must shape AI with western values and norms.” / X
“strong disagree. 1/ open source isn’t a charity, it’s a strategy for both building and selling. I grew up on linux. the kernel has over 20,000 individual contributors and more than 1,300 contributing companies. linux’s large community helps it remain robust, secure & versatile” / X
“Reminder: open-source is the foundation of all AI (including closed-source AI)!” / X
“What a week in the ML world! Here is a recap thread on all the exciting open ML updates!🔥 VLMs: Salesforce, Kosmos 2.5, PaliGemma, Cumo LLM: Yi 1.5, Falcon 2, DeepSeek v2 lite Diffusion: HunyuanDiT, Lumina next Keep reading 👇” / X
“I am a little confused about the reason for the plethora of open weights models being offered by providers right now. Given that a few models dominate the leaderboards across many skills, come in multiple sizes & are getting cheaper, fast, what is the point of using the others?
“LLMs are plateauing and the gap between closed vs. open is almost closed!! If you are look at MMLU open-source is caught up to closed source and we are seeing the LLMs plateau It’s time to move on to different benchmarks that measure LLM capabilities on hard problems The key
“You can now fine-tune models using an AI assistant! And it works with any open-source model. No-code fine-tuning and deployment is as good as it gets! Seriously, it’s mind-blowing how far we’ve come. I recorded a video to show you how to use a GPT to fine-tune a model. You
Cohere
Aya | Cohere For AI
Cohere For AI Launches Aya 23, 8 and 35 Billion Parameter Open Weights Release
@romainhuet Cohere just launched Aya 23 — a family of multilingual LLMs with open weights and support for 23 different languages. Access to top models will soon become crucial for many parts of the world, so democratizing access is a massive step forward.
“Switching from French to German to Chinese in the same discussion 😅 Impressive to see @CohereForAI’s new Aya model multilingual capabilities. – C4AI Aya 23 is a research open weights release – 8 and 35 billion parameter models – 23 languages supported You can try it out here:
Gemma
PaliGemma: Open Source Multimodal Model by Google
Grok
“Wow this is powerful. Grok is able to accurately give me the last closing price of a stock option, and correctly explain the reason for this price. Congrats @grok team, your RAG capabilities are very impressive and useful, Grok will be my exclusive personal assistant from now on
Elon Musk’s xAI is working on making Grok multimodal – The Verge
Open Release of Grok-1
Hugging Face
“No cloud, no cost, no data sent to anyone, no problem. Welcome to local AI on Hugging Face!
Hugging Face commits $10 million in free shared GPUs – The Verge
Experimental Moondream WebGPU – a Hugging Face Space by Xenova
Meta/Llama
“The first open-source implementation of the paper that will change automatic test generation is now available! In February, Meta published a paper introducing a tool to automatically increase test coverage, guaranteeing improvements over an existing code base. This is a big
“Build a Full-Stack Job Search Assistant with @gokoyeb, @MongoDB, and @llama_index 🧑💼🔎 This is a comprehensive end to end tutorial by @rishi_raj_jain_ on building a RAG-powered assistant that streams its response in real-time but can also continuously update its internal
“Let’s build a crew of AI agents to scrape the web and write blog posts for you, powered by Llama-3 (100% local):” / X
“Welcome CogVLM 2 ⚡ > Beats GPT4 V/ Gemini Pro on TextVQA, DocVQA and ChartQA – by a decent margin 🔥 > 19B params > Llama 3 8B (Instruct) text backbone > Supports 8K context length > Upto 1344 X 1344 resolution supported > Works with both Chinese and English > Open access with
What’s up with Llama 3? Arena data analysis | LMSYS Org
LLM Comparison/Test: Llama 3 Instruct 70B + 8B HF/GGUF/EXL2 (20 versions tested and compared!)
Should Meta open-source Llama 3 400B or not?
“Meta open-sourcing Llama-3 400b will make them the biggest hero of our age! Nothing is more important or more urgent!” / X
“Meta plans to not open the weights for its 400B model. The hope is that we would quietly not notice / let it slide. Don’t let it slide.” / X
“Head of Ted is calling Meta reckless for releasing Open source models. While its a fact that Meta is really the leader for OSS contributions. Beyond Llama-3 – we have all the below from them (and this not an exhaustive list) – React – PyTorch – React Native – GraphQL – Jest –
Mistral
“New @MistralAI 7B base and instruct 🔥 No magnet link, but a @huggingface repository! 🤗👀 🔡 Extended Vocabulary from 32000 to 32768 🔨 Function calling support 🔓 Apache 2.0 license ❌ No evaluation details Base:
“Made a free Colab for Mistral v3! You can QLoRA finetune 2x faster, use 70% less VRAM with no accuracy degradations with @UnslothAI! You can export to vLLM, GGUF, HF inference is 2x faster & we support 4x longer context windows than FA2 (24GB= 56K vs 14K)
“Checkout the new Mistral v0.3 models with MLX LM. Pre-quantized models in the 🤗 MLX community
“Let’s fucking go! Mistral just released 7B v0.3 🔥 > Base + Instruct model checkpoints released > Extended vocabulary to 32768 > Supports new v3 Tokenizer > Supports function calling > Uncensored (no moderation tactics used during fine-tuning) Thanks for the sweet surprise,” / X
“BREAKING : Mistral-7B v0.3 has been released 🎇 – Extended vocabulary to 32768 – Supports v3 Tokenizer – Supports function calling Their github repo is claiming that Mixtral 8x7B Instruct and Mixtral 8x7B will be updated soon, probably also in the same fashion as Mistral 7B
Mistral AI and Harvey Partnership
Phi
Microsoft Phi-Silica: 3.3B small AI model made for Copilot+ PC NPUs | VentureBeat
New models added to the Phi-3 family, available on Microsoft Azure | Microsoft Azure Blog
“Phi-3 small & medium are now available under the MIT license! 🚀@Microsoft has just launched Phi-3 small (7B) and medium (14B) 🤯. The Phi-3 small model claims to outperform @AIatMeta’s Llama 3 and @MistralAI, and the Phi-3 medium model GPT-3.5 and @cohere Command R+. 🤔 TL;DR:
“Small Models Are Improving Exponentially – Phi-3 14B Is Phenomenal The new Phi-3 14B model scores phenomenally on all benchmarks. On key numbers, it seems to be pretty close to Llama-3-Instruct 🤯🤯 As small models become more and more powerful, we will see 7b-sized GPT-4 class
“False alarm on the phi-3 models (did very poorly on a few offline benchmarks I have), still using llama-3 fine tuned models for a few specialized services. The phi-3 models seem very sensitive to prompts (not a good thing imo)” / X
“Phi-3-vision with 4.2B parameters
“The more I look at these numbers the more magical it looks. 😯🔥 Phi-3-small with only 7B parameters beats GPT-3.5T across a variety of language, reasoning, coding, and math benchmarks. Next, GPT4 level model in my pocket GPY by this year end 😯
Other Open Source News
“Great Yi-1.5-34B with much longer context window.” / X
“You asked for longer contexts🎤 and we heard you!👂 The following models are now available on @huggingface by popular demand: ✅Yi-1.5-34B-32K ✅Yi-1.5-34B-Chat-16K ✅Yi-1.5-9B-32K ✅Yi-1.5-9B-Chat-16K Happy building!
Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.
Be Sure To Read This Week’s Main Post:
This week’s executive overview and top links are here:
AI News #34: Week Ending 05/24/2024 with Executive Summary and Top 47 Links
The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.
- Agents/Copilots
- Amazon
- Apple
- Artificial General Intelligence (AGI)
- Augmented and Virtual Reality (AR/VR)
- Autonomous Vehicles
- AI Audio
- Business and Enterprise AI
- Chips and Hardware
- Consumer Products
- Education
- Ethics/Legal Security
- Images/Photos
- International AI News
- Locally Run AI Models
- Mobile
- Meta
- Microsoft
- OpenAI
- Open Source
- Podcasts/YouTube
- Publishing and News
- Retrieval-Augmented Generation (RAG) News
- Robots and Embodiment
- Science and Medicine
- Video
- Vision/Multimodality
- X/Twitter/Grok
- Tech and Development
Credits/Sources
Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.
- Robert Scoble: https://x.com/Scobleizer
- Ethan Mollick: https://www.linkedin.com/in/emollick/
- Alan Thompson: https://lifearchitect.ai/
- Theoretically Media: https://www.youtube.com/@TheoreticallyMedia
- The Rundown: https://www.therundown.ai/
- Bilawal Sidhu: https://twitter.com/bilawalsidhu/
- TLDR: https://tldr.tech/ai
- Jeremiah Owyang: https://twitter.com/jowyang
- Nick St. Pierre: https://twitter.com/nickfloats
- Dr. Jim Fan: https://twitter.com/DrJimFan
- All About AI: https://www.youtube.com/@AllAboutAI
- Marshall Kirkpatrick: https://aitimetoimpact.com/
- AI News (Smol Talk): https://buttondown.email/ainews/archive/
For previous issues, please visit the archives!
Thanks for reading!
Leave a Reply