Open Source AI News: Week Ending 02/02/2024

February 7, 2024

Open Source

The Mistral CEO, confirmed that a mysterious model called ‘Miqu’ that neared GPT-4 performance posted on HuggingFace was leaked. The model is said to be a fine-tune of Meta’s open-source Llama-2. Open-source AI is rapidly catching up to industry leaders!

The Mistral CEO, confirmed that a mysterious model called ‘Miqu’ that neared GPT-4 performance posted on HuggingFace was leaked.

The model is said to be a fine-tune of Meta's open-source Llama-2.

Open-source AI is rapidly catching up to industry leaders! https://t.co/D18yqDTvdw
— Rowan Cheung (@rowancheung) February 1, 2024

Mistral CEO confirms ‘leak’ of new open source AI model nearing GPT-4 performance

Mistral CEO confirms ‘leak’ of new open source AI model nearing GPT-4 performance

OLMo: Open Language Model

A State-Of-The-Art, Truly Open LLM and Framework

https://blog.allenai.org/olmo-open-language-model-87ccfc95f580

InternLM2-Math: New SOTA math LLMs

It achieves 90% GPT-4 performance on various math benchmarks with only 20B parameters.

Theorem proving and problem solving with *Lean*.

It can act as a verifier and augmentor.

🔥InternLM2-Math: New SOTA math LLMs
🚀 It achieves 90% GPT-4 performance on various math benchmarks with only 20B parameters.
👍Theorem proving and problem solving with *Lean*.
🧐It can act as a verifier and augmentor.
GitHub: https://t.co/QlWKFhKBem pic.twitter.com/iz9DPKKqPc
— OpenMMLab (@OpenMMLab) January 23, 2024

We are excited to announce that we have open-sourced FireLLaVA under the Llama 2 Community License. It is the first LLaVA multi-modality model with a commercially permissive license.

https://fireworks.ai/blog/firellava-the-first-commercially-permissive-oss-llava-model

We are thrilled to release LLaVA-1.6, with improved reasoning, OCR, and world knowledge. It supports higher-res inputs, more tasks, and exceeds Gemini Pro on several benchmarks! It maintains the data efficiency of LLaVA-1.5, and LLaVA-1.6-34B is trained ~1 day with 32 A100s.

🚀We are thrilled to release LLaVA-1.6, with improved reasoning, OCR, and world knowledge. It supports higher-res inputs, more tasks, and exceeds Gemini Pro on several benchmarks! 🤯 It maintains the data efficiency of LLaVA-1.5, and LLaVA-1.6-34B is trained ~1 day with 32 A100s.… pic.twitter.com/nGRpLX8FQv
— Haotian Liu (@imhaotian) January 31, 2024

https://llava.hliu.cc

https://llava-vl.github.io/blog/2024-01-30-llava-next

Introducing Nomic Embed – the first fully open long context text embedder to beat OpenAI

– Open source, open weights, open data

– Beats OpenAI text-embeding-3-small and Ada on short and long context benchmarks

Introducing Nomic Embed – the first fully open long context text embedder to beat OpenAI

– Open source, open weights, open data
– Beats OpenAI text-embeding-3-small and Ada on short and long context benchmarks
– Day 1 integrations with @langchain, @llama-index, @MongoDB pic.twitter.com/miDGh2OVhv
— Nomic AI (@nomic_ai) February 1, 2024

SMAUG – The BEST 30B class open-source LLM In the world. Not gonna lie; it feels GREAT to be on top of the LLM leaderboard. Yes, Abacus AI is thrilled to drop the top-performing 30B open-source model in the world – SMAUG. Abacus SMAUG has an MMLU of 76.66 and an overall average score of 77.24. This is a good 2 points over all other models in its class.

SMAUG – The BEST 30B class open-source LLM In the world!

Not gonna lie; it feels GREAT to be on top of the LLM leaderboard!

Yes, Abacus AI is thrilled to drop the top-performing 30B open-source model in the world – SMAUG.

Abacus SMAUG has an MMLU of 76.66 and an overall… pic.twitter.com/iaDh7D6fgo
— Bindu Reddy (@bindureddy) January 29, 2024

Airavata – an instruction-tuned model for Hindi built by finetuning OpenHathi LLM. OpenHathi is an open-source foundational model for Hindi, developed by extending Llama 2.

🚀 Airavata – Instruction Tuned Hindi LLM

✅ Airavata – an instruction-tuned model for Hindi built by finetuning OpenHathi LLM.

☑️ OpenHathi is an open-source foundational model for Hindi, developed by extending Llama 2.

✅ OpenHathi was introduced by Sarvam AI, a promising AI… pic.twitter.com/mf87cdVkom
— Kalyan KS (@kalyan_kpl) January 29, 2024

Introducing two brand new, general purpose AI models:

10.7B Parameters, Tess-10.7B-v1.5b: https://huggingface.co/migtissera/Tess-10.7B-v1.5b

Apache-2.0, Trained with the Tess-v1.5b dataset, SOLAR-10.7B base, 4K Context Window

34B Parameters, Tess-34B-v1.5b: https://huggingface.co/migtissera/Tess-34B-v1.5b

Yi-34B Licence, Trained with the Tess-v1.5b dataset, Yi-34B-200K base, 200K Context Window

Introducing two brand new, general purpose AI models:

10.7B Parameters, Tess-10.7B-v1.5b: https://t.co/IeDtjQWNYF
Apache-2.0, Trained with the Tess-v1.5b dataset, SOLAR-10.7B base, 4K Context Window

34B Parameters, Tess-34B-v1.5b: https://t.co/dsh1YzOMwO
Yi-34B Licence, Trained… pic.twitter.com/SodPgQm0MR
— Migel Tissera (@migtissera) January 28, 2024

Vision LLM for edge computing ? Check out this new release on Hugging Face. Open BMB, who OS’ed the UltraFeedback dataset before, released a series of strong eco-friendly yet powerful LLMs

– MiniCPM: 2B model that competes with Mistral-7B

– MiniCPM-V: 3B vision LLM on edge

Vision LLM for #edgecomputing ? Check out this new release on @huggingface @OpenBMB, who OS'ed the UltraFeedback dataset before, released a series of strong eco-friendly yet powerful LLMs

– MiniCPM: 2B model that competes with Mistral-7B

– MiniCPM-V: 3B vision LLM on edge! pic.twitter.com/Cz8HP4EUUL
— Tiezhen WANG (@Xianbao_QIAN) February 1, 2024

BAAI releases BGE-M3, a new member to BGE model series. M3 stands for Multi-linguality (100+ languages), Multi-granularities (input length up to 8192), Multi-Functionality (unification of dense, lexical, multi-vec retrieval).

BAAI releases BGE-M3, a new member to BGE model series. M3 stands for Multi-linguality (100+ languages), Multi-granularities (input length up to 8192), Multi-Functionality (unification of dense, lexical, multi-vec retrieval). 🔥 https://t.co/5Un8e6NFqu
— BAAI (@BAAIBeijing) January 30, 2024

Introducing Eagle-7B

Soaring past Transformers with 1 Trillion Tokens Across 100+ Languages (RWKV-v5)

Based on the RWKV-v5 architecture, bringing into opensource space, the strongest

– multi-lingual model

(beating even mistral)

– attention-free transformer today

(10-100x+ lower inference)

https://twitter.com/RWKV_AI/status/1751797147492888651 https://blog.rwkv.com/p/eagle-7b-soaring-past-transformers

META’s new OPEN SOURCE Coding AI beats out GPT-4 | Code Llama 70B

Be Sure To Read “This Week In AI”

This week’s executive overview and top links are here:

AI News #18: Week Ending 02/02/2024 with Executive Summary and Top 12 Stories

The post you just read is an extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.