Ethan B. Holland

Over 56,600 manually organized AI links and counting

a middle school art project showcasing libraries, a small written sign says: "Tech" --chaos 20 --ar 4:3 --style raw --personalize x3p86jc --v 6.1

Tech Papers, Training, and Development: Week Ending 07/19/2024

a middle school art project showcasing libraries, a small written sign says: “Tech” –chaos 20 –ar 4:3 –style raw –personalize x3p86jc –v 6.1

“(Yet) another tale of Rise and Fall: Recently, NuminaMath-7B ranked 1st at the AIMO competition, solving 29/50 private set problems of olympiad math level. Can it handle simple AIW problem, which reveal basic reasoning deficits in SOTA LLMs? (

“🚨Excited to share our new paper!🚨 We reveal a curious generalization gap in the current refusal training approaches: simply reformulating a harmful request in the past tense (e.g., “How to make a Molotov cocktail?” to “How did people make a Molotov cocktail?”) is often… 

[D] [P] Exponential Growth of Context Length in Language Models : r/MachineLearning

[D] [P] Exponential Growth of Context Length in Language Models
byu/porkbellyqueen111 inMachineLearning

“Human-like Episodic Memory for Infinite Context LLMs Large language models (LLMs) have shown remarkable capabilities, but still struggle with processing extensive contexts, limiting their ability to maintain coherence and accuracy over long sequences. In contrast, the human 

optimization – What are the ways compilers recognize complex patterns? – Programming Language Design and Implementation Stack Exchange

https://langdev.stackexchange.com/questions/3942/what-are-the-ways-compilers-recognize-complex-patterns

“4x throughput over open source serving vLLM… what hope do we have to serve our own models”

[2407.04620] Learning to (Learn at Test Time): RNNs with Expressive Hidden States

https://arxiv.org/abs/2407.04620

“towards intelligence too cheap to meter: 

“way back in 2022, the best model in the world was text-davinci-003. it was much, much worse than this new model. it cost 100x more.”

“For this week’s Q&AI, I wrote about how AI companies trying to build better reasoning in their models, and why that’s such a hard problem to solve. 

“Decided to start a new blog series about model architectures in the era of LLMs. 😀 Here’s part 1 on broader architectures like Transformer Encoders/Encoder-Decoders, PrefixLM and denoising objectives. 😄 A frequently asked question: “The people who worked on language and NLP 

“📃We’ve published our technical report for WayveScenes101. At Wayve, we’ve always focused on tackling the hardest problems first. Through the WayveScenes101 dataset, which includes a number of dynamic and deformable objects, we’re supplying others within the industry with the 

“If you believe you can’t exceed a teacher model at a task with synthetic data alone, then how is this SOTA? Synthetic data is real and is not something that has to cause a mode collapse or top out at the previous SOTA”

What happened to BERT & T5? On Transformer Encoders, PrefixLM and Denoising Objectives — Yi Tay

https://www.yitay.net/blog/model-architecture-blogpost-encoders-prefixlm-denoising

“I’m worried that a lot of work on AI safety evals is primarily motivated by “Something must be done. This is something. Therefore this must be done.” Or, to put it another way: I judge eval ideas on 4 criteria, and I often see proposals which fail all 4. The criteria:”

Overcoming the limits of current LLM

https://seanpedersen.github.io/posts/overcoming-llm-limits

“We were using LMs to evaluate the outputs of LMs (AlpacaEval). Now we’re using LMs to generate the inputs into LMs too (so it’s just models talking to models). One should be careful with automatic evals, but the sheer power you gain is worth something.”

NeurIPS 2023 Recap — Best Papers – Latent Space

https://www.latent.space/p/neurips-2023-papers

“Neat, well-explained guide to the Transformer architecture, with Keras code examples: 

“It begins. This is another sign that LLMs are going to be able to work with structured & unstructured spreadsheet data soon. This will unlock a lot of use cases (projections, financials, valuations, etc.) and having a spreadsheet source of truth will tend to lower hallucinations 

Benchmarking results for vector databases – Redis

https://redis.io/blog/benchmarking-results-for-vector-databases

DataComp

Welcome to DataComp, the machine learning benchmark where the models are fixed and the challenge is to find the best possible data!

https://www.datacomp.ai/dclm/index.html

Heads up! You’ve scrolled to the end of this category. There may have been just one or two links (above), so go back up and double check to be sure you didn’t quickly scroll down past it.

Be Sure To Read This Week’s Main Post:

This week’s executive overview and top links are here:

AI News #42: Week Ending 07/19/2024 with Executive Summary and Top 58 Links 

The post you just read is an deep dive extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.

Credits/Sources

Most of these weekly links come from just a few prolific oversharing sources. Please follow them, as they work hard to find the news each week and they make it a lot easier for me to compile.

For previous issues, please visit the archives!

Thanks for reading!

Leave a Reply

Trending

Discover more from Ethan B. Holland

Subscribe now to keep reading and get access to the full archive.

Continue reading