a realistic baroque oil painting in an ornate gilded frame, depicting puzzle pieces, a small, elegant nameplate at the bottom of the fame is engraved with the title: "Open Source" --chaos 20 --ar 4:3 --style raw --personalize t9u6ckr --v 6.1

Open Source AI News: Week Ending 07/12/2024

July 12, 2024

a realistic baroque oil painting in an ornate gilded frame, depicting puzzle pieces, a small, elegant nameplate at the bottom of the fame is engraved with the title: “Open Source” –chaos 20 –ar 4:3 –style raw –personalize t9u6ckr –v 6.1

Gemma

Gemma 2

Improving Open Language Models at a Practical Size

In this post, we take a deep dive into the architectural components of Gemma 2 such as Grouped Query Attention, Sliding Window Attention, RoPE Embeddings, Logit soft-capping & Model-merging!

https://amaarora.github.io/posts/2024-07-07%20Gemma.html

Meta/Llama

Meta to reportedly launch largest Llama 3 model on July 23 – Breaking The News

https://breakingthenews.net/Article/Meta-to-reportedly-launch-largest-Llama-3-model-on-July-23/62364570

Meta Platforms To Release Largest Llama 3 Model on July 23 — The Information

https://www.theinformation.com/briefings/meta-platforms-to-release-largest-llama-3-model-on-july-23

Meta drops AI bombshell: Multi-token prediction models now open for research | VentureBeat

Meta drops AI bombshell: Multi-token prediction models now open for research

“As a special weekend release, we’ve implemented an initial working implementation of GraphRAG concepts in @llama_index [in beta]. GraphRAG from Microsoft has a lot of cool ideas, and we’ve implemented the following main concepts: 1. Graph Generation: Build graph and community

As a special weekend release, we’ve implemented an initial working implementation of GraphRAG concepts in @llama_index [in beta].

GraphRAG from Microsoft has a lot of cool ideas, and we’ve implemented the following main concepts:
1. Graph Generation: Build graph and community… pic.twitter.com/dMJ4RQiPQg
— LlamaIndex 🦙 (@llama_index) July 14, 2024

Other Open Source News

“”What the industry leaders OpenAI, Anthropic, Mistral and Microsoft have never revealed is their formula for building the collection of texts used to train their models.” With FineWeb, @huggingface has done it. And @lemondefr went behind the scenes to see how it was done.” / X

"What the industry leaders OpenAI, Anthropic, Mistral and Microsoft have never revealed is their formula for building the collection of texts used to train their models."

With FineWeb, @huggingface has done it.

And @lemondefr went behind the scenes to see how it was done.…
— Florent Daudens (@fdaudens) July 9, 2024

“Nvidia launched a new competition with RTC 4080 GPUs + cash as a prize. The ICCAD contest is to create an open-source code dataset for training LLMs on hardware design, with the aim to eventually automate the development of future GPUs.

Nvidia launched a new competition with RTC 4080 GPUs + cash as a prize.

The ICCAD contest is to create an open-source code dataset for training LLMs on hardware design, with the aim to eventually automate the development of future GPUs. pic.twitter.com/2fg2G3DGX5
— Rowan Cheung (@rowancheung) July 9, 2024

“Researchers at UCSD and MIT introduced Open-TeleVision It’s an open-source tele-op system that allows users to control robots from thousands of miles away It’s also accessible from any device with a web browser, including VR

Researchers at UCSD and MIT introduced Open-TeleVision

It's an open-source tele-op system that allows users to control robots from thousands of miles away

It's also accessible from any device with a web browser, including VRhttps://t.co/Fu6I2QLIoz
— Brett Adcock (@adcock_brett) July 7, 2024

“Introduce Open-𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧🤖: ⁣ We need an intuitive and remote teleoperation interface to collect more robot data. 𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳. Open-sourced!

Introduce Open-𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧🤖: ⁰⁣
We need an intuitive and remote teleoperation interface to collect more robot data.
𝐓𝐞𝐥𝐞𝐕𝐢𝐬𝐢𝐨𝐧 lets you immersively operate a robot even if you are 3000 miles away, like in the movie 𝘈𝘷𝘢𝘵𝘢𝘳.
Open-sourced! pic.twitter.com/629fQKtCWU
— Xuxin Cheng (@xuxin_cheng) July 2, 2024

“Open science at work: This model solved 29 out of 50 math problems in the AI Math Olympiad.” / X

Open science at work: This model solved 29 out of 50 math problems in the AI Math Olympiad. https://t.co/g1jJxDVG0h
— Florent Daudens (@fdaudens) July 10, 2024

“Launching GPT4All 3.0: The Open-Source Local LLM Desktop App – Completely Private Experience – Supports 1000’s of models and all major operating systems – Major UI/UX Improvements – Local File Chat – MIT Licensed Get started below:

Launching GPT4All 3.0: The Open-Source Local LLM Desktop App

– Completely Private Experience
– Supports 1000’s of models and all major operating systems
– Major UI/UX Improvements
– Local File Chat
– MIT Licensed

Get started below: pic.twitter.com/I2cUOfACHW
— Nomic AI (@nomic_ai) July 2, 2024

“Lynx is a new hallucination detection model for #LLMs that is especially suited for real-world applications in industries like healthcare and fintech. @PatronusAI trained Lynx on @databricks Mosaic AI using Composer, our open source @PyTorch-based training library.” / X

Lynx is a new hallucination detection model for #LLMs that is especially suited for real-world applications in industries like healthcare and fintech. @PatronusAI trained Lynx on @databricks Mosaic AI using Composer, our open source @PyTorch-based training library. https://t.co/zxwDzYpzbe
— Databricks Mosaic Research (@DbrxMosaicAI) July 11, 2024

Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.

Be Sure To Read This Week’s Main Post:

This week’s executive overview and top links are here:

AI News #41: Week Ending 07/12/2024 with Executive Summary and Top 58 Links

The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.

Credits/Sources

Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.