Ethan B. Holland

Over 57,200 manually organized AI links and counting

AI Tech and Development News: Week Ending 01/19/2024

January 20, 2024

Technical/Dev

Moore’s law is coming to an end. Extropic AI is pioneering the next-gen system for end-to-end neural information processing at the thermodynamic limits of computing.

Moore's law is coming to an end. @Extropic_AI is pioneering the next-gen system for end-to-end neural information processing at the thermodynamic limits of computing.

Many more details to come. 🔥🫡

Thanks to @willknight for the feature in @WIRED today. pic.twitter.com/VcpJxakHAK
— Guillaume Verdon (@GillVerd) January 18, 2024

Check out our International Conference on Learning Representations [oral] paper on learning state-of-the-art ViTs from a single video from scratch! One of the coolest things is that multi-object tracking emerges from the different heads in the plain ViTs (three heads visualised below in R,G,B).

Check out our @iclr_conf [oral] paper on learning state-of-the-art ViTs from a single video from scratch!

One of the coolest things is that multi-object tracking emerges from the different heads in the plain ViTs (three heads visualised below in R,G,B). https://t.co/RbdpRH1fzv pic.twitter.com/puuUOFVHVz
— Yuki (@y_m_asano) January 19, 2024

Introducing our new flagship LLM, Nous-Hermes 2 on Mixtral 8x7B.

Our first model that was trained with RLHF, and the first model to beat Mixtral Instruct in the bulk of popular benchmarks!

Introducing our new flagship LLM, Nous-Hermes 2 on Mixtral 8x7B.

Our first model that was trained with RLHF, and the first model to beat Mixtral Instruct in the bulk of popular benchmarks!

We are releasing the SFT only and SFT+DPO model, as well as a qlora adapter for the DPO…
— Nous Research (@NousResearch) January 15, 2024

What is Low-Rank Adaptation (LoRA) | explained by the inventor

More and more evidence that “running out of human-made training data” is not going to be the thing that stops LLM development.

More and more evidence that “running out of human-made training data” is not going to be the thing that stops LLM development. https://t.co/KZvaoeqtG1
— Ethan Mollick (@emollick) January 19, 2024

Portkey’s AI Gateway is the interface between your app and hosted LLMs. It streamlines API requests to OpenAI, Anthropic, Mistral, LLama2, Anyscale, Google Gemini and more with a unified API

Portkey's AI Gateway is the interface between your app and hosted LLMs. It streamlines API requests to OpenAI, Anthropic, Mistral, LLama2, Anyscale, Google Gemini and more with a unified API https://t.co/AUrZnZqpsS
— Samuel Wong (@samuel_wong_) January 11, 2024

Hidden Changes in GPT-4, Uncovered

https://dmicz.github.io/machine-learning/openai-changes

State-of-the-art Code Generation with AlphaCodium – From Prompt Engineering to Flow Engineering

https://www.codium.ai/blog/alphacodium-state-of-the-art-code-generation-for-code-contests

8bit HippoAttention: Up to 3X Faster Compared to FlashAttentionV2

https://blog.hippoml.com/8bit-hippoattention-up-to-3x-faster-compared-to-flashattentionv2-8f9def90b482

Be Sure To Read “This Week in AI“

This week’s executive overview and top links are here:
AI News #16: Week Ending 01/19/2024 with Executive Summary and Top 16 Stories

This is an extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.