record album cover for a band named "Open" --chaos 40 --ar 4:3 --style raw --personalize t1hfffm

Open Source AI News: Week Ending 06/28/2024

June 28, 2024

record album cover for a band named “Open” –chaos 40 –ar 4:3 –style raw –personalize t1hfffm

DeepSeek

“Deepseek Coder v2 Is The New Open-Source King And Beats The Best Gemini Model. Deepseek-coder has officially put OSS in third position, just behind OAI and Anthropic. It excels at coding and reasoning and scores very high on Livebench AI, the ONLY benchmark you can’t game! –

Deepseek Coder v2 Is The New Open-Source King And Beats The Best Gemini Model.

Deepseek-coder has officially put OSS in third position, just behind OAI and Anthropic.

It excels at coding and reasoning and scores very high on Livebench AI, the ONLY benchmark you can't game!

-… pic.twitter.com/6GZbTlXhHK
— Bindu Reddy (@bindureddy) June 25, 2024

“DeepSeek-Coder-V2 – competes with closed-sourced models on code and math generation tasks; achieves 90.2% on HumanEval and 75.7% on MATH; these results are higher than GPT-4-Turbo-0409 performance according to their report; includes a 16B and 236B parameter model with 128K”

2/ DeepSeek-Coder-V2 – competes with closed-sourced models on code and math generation tasks; achieves 90.2% on HumanEval and 75.7% on MATH; these results are higher than GPT-4-Turbo-0409 performance according to their report; includes a 16B and 236B parameter model with 128K…
— DAIR.AI (@dair_ai) June 23, 2024

“Chatbot Arena update! @deepseek_ai DeepSeek-Coder-v2 has climbed to #4 in Coding Arena, nearing GPT-4-Turbo levels! It’s now the top open model for coding. It’s also strong in Hard Prompts, ranking #11, but sits at #20 in Overall generic questions. Meanwhile, @ChatGLM GLM-0520 https://t.co/UAS5tuKHuY”

Gemma

“Google just dropped Gemma 2 27B & 9B Both are showing interesting performances: – 27B beats Llama3 70B, Qwen2 72B and Command R+ in LMSYS Chatbot Arena – 9B beats Mistral 7B and Llama3 8B on various benchmarks Can’t wait to see the evals on the Open LLM Leaderboard with

Google just dropped Gemma 2 27B & 9B

Both are showing interesting performances:

– 27B beats Llama3 70B, Qwen2 72B and Command R+ in LMSYS Chatbot Arena
– 9B beats Mistral 7B and Llama3 8B on various benchmarks

Can't wait to see the evals on the Open LLM Leaderboard with… pic.twitter.com/4C3YsvMNYX
— Florent Daudens (@fdaudens) June 27, 2024

Google launches Gemma 2, its next generation of open models

https://blog.google/technology/developers/google-gemma-2

“Our benchmarks show: 🔘 9B delivers class-leading performance against other open models in its size category 🔘 27B outperforms some models more than twice its size… … and is optimized to run efficiently on a single TPU host.

Our benchmarks show:
🔘 9B delivers class-leading performance against other open models in its size category
🔘 27B outperforms some models more than twice its size…

… and is optimized to run efficiently on a single TPU host. https://t.co/RmRLEdFPMV pic.twitter.com/BtE3VdSxZC
— Google DeepMind (@GoogleDeepMind) June 27, 2024

“We’re excited to unveil Gemma 2. 🛠️ Available in both 9B and 27B parameters, it delivers the best performance for its size – unlocking more possibilities for developers to build and deploy with AI. →

We're excited to unveil Gemma 2. 🛠️

Available in both 9B and 27B parameters, it delivers the best performance for its size – unlocking more possibilities for developers to build and deploy with AI. → https://t.co/RmRLEdFPMV pic.twitter.com/hE1tdSM3DB
— Google DeepMind (@GoogleDeepMind) June 27, 2024

“When training Gemma 2, we followed robust internal safety processes. 🌐 This included filtering pre-training data and performing rigorous testing and evaluation against a comprehensive set of metrics to identify and mitigate potential biases and risks. →

When training Gemma 2, we followed robust internal safety processes. 🌐

This included filtering pre-training data and performing rigorous testing and evaluation against a comprehensive set of metrics to identify and mitigate potential biases and risks. → https://t.co/RmRLEdFPMV pic.twitter.com/dTYESWydwf
— Google DeepMind (@GoogleDeepMind) June 27, 2024

Meta/Llama

Meta’s LLM Compiler is the latest AI breakthrough to change the way we code | VentureBeat

Meta’s LLM Compiler is the latest AI breakthrough to change the way we code

“Today we’re announcing Meta LLM Compiler, a family of models built on Meta Code Llama with additional code optimization and compiler capabilities. These models can emulate the compiler, predict optimal passes for code size, and disassemble code. They can be fine-tuned for new

Today we’re announcing Meta LLM Compiler, a family of models built on Meta Code Llama with additional code optimization and compiler capabilities. These models can emulate the compiler, predict optimal passes for code size, and disassemble code. They can be fine-tuned for new… pic.twitter.com/GFDZDbZ1VF
— AI at Meta (@AIatMeta) June 27, 2024

Nvidia

“🤖 From this week’s issue: @nvidia announced Nemotron-4 340B, a family of open models that developers can use to generate synthetic data for training large language models.

🤖 From this week's issue: @nvidia announced Nemotron-4 340B, a family of open models that developers can use to generate synthetic data for training large language models.https://t.co/WYle0xJUka
— Deep Learning Weekly (@dl_weekly) June 23, 2024

Other Open Source News

“Introducing Cambrian-1, a fully open project from our group at NYU. The world doesn’t need another MLLM to rival GPT-4V. Cambrian is unique as a vision-centric exploration & here’s why I think it’s time to shift focus from scaling LLMs to enhancing visual representations.🧵[1/n]

Introducing Cambrian-1, a fully open project from our group at NYU. The world doesn't need another MLLM to rival GPT-4V. Cambrian is unique as a vision-centric exploration & here's why I think it's time to shift focus from scaling LLMs to enhancing visual representations.🧵[1/n] pic.twitter.com/RyIHb2jvSl
— Saining Xie (@sainingxie) June 26, 2024

Open-LLM performances are plateauing, let’s make the leaderboard steep again – a Hugging Face Space by open-llm-leaderboard

https://huggingface.co/spaces/open-llm-leaderboard/blog

“Look at that 👀 Actual benchmarks have become too easy for recent models, much like grading high school students on middle school problems makes little sense. So the team worked on a new version of the Open LLM Leaderboard with new benchmarks. Stellar work from @clefourrier

Look at that 👀

Actual benchmarks have become too easy for recent models, much like grading high school students on middle school problems makes little sense. So the team worked on a new version of the Open LLM Leaderboard with new benchmarks.

Stellar work from @clefourrier… https://t.co/qzzCesnp71 pic.twitter.com/jdmfp8oSM3
— Florent Daudens (@fdaudens) June 26, 2024

Training a 70B model from scratch: open-source tools, evaluation datasets, and learnings – imbue

https://imbue.com/research/70b-intro

“It is difficult for open-source software to be truly open when it is used as a strategic vehicle for corporate interests.” / X

It is difficult for open-source software to be truly open when it is used as a strategic vehicle for corporate interests.
— François Chollet (@fchollet) June 24, 2024

Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.

Be Sure To Read This Week’s Main Post:

This week’s executive overview and top links are here:

AI News #39: Week Ending 06/28/2024 with Executive Summary and Top 67 Links

The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.

Credits/Sources

Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.