Meta

Meta is training LLaMa 3 on 600,000 H100s…. That’s $20 BILLION worth of GPUs…. L.F.G.

Holy shit. Meta is training LLaMa 3 on 600,000 H100s…. That's $20 BILLION worth of GPUs…. L.F.G. https://t.co/1Ily4hyc3H
— JJ (@JosephJacks_) January 18, 2024

By the end of the year, Meta will have 600k H100-equivalent GPUs.

Can finally talk some GPU numbers publicly 🙃
By the end of the year, Meta will have 600k H100-equivalent GPUs.
Feel free to guess what's already deployed and being used 😉! pic.twitter.com/z4j2MIBOCr
— Soumith Chintala (@soumithchintala) January 18, 2024

Can finally talk some GPU numbers publicly

By the end of the year, Meta will have 600k H100-equivalent GPUs.

Can finally talk some GPU numbers publicly 🙃
By the end of the year, Meta will have 600k H100-equivalent GPUs.
Feel free to guess what's already deployed and being used 😉! pic.twitter.com/z4j2MIBOCr
— Soumith Chintala (@soumithchintala) January 18, 2024

https://www.threads.net/@zuck/post/C2QBoRaRmR1

This is wild. 600k H100 equivalents at Meta is roughly 8x Tesla’s roadmap by end of 2024 (assuming 1 H100 = 4 A100). The 350k H100 target by end of 2024 suggests a more than doubling of total GPU compute since H100 started volume shipping in Q1 2023. To be fair, Tesla’s relative compute increase is also insane, targeted to grow their base by ~20x in a year and a half

This is wild. 600k H100 equivalents at Meta is roughly 8x Tesla's roadmap by end of 2024 (assuming 1 H100 = 4 A100).

The 350k H100 target by end of 2024 suggests a more than doubling of total GPU compute since H100 started volume shipping in Q1 2023

To be fair, Tesla's relative… https://t.co/51imuMXRsW pic.twitter.com/SUVTWYXeiO
— Frank Downing (@downingARK) January 18, 2024

Mark Zuckerberg’s new goal is creating artificial general intelligence

And he doesn’t want to control it. Maybe.

https://www.theverge.com/2024/1/18/24042354/mark-zuckerberg-meta-agi-reorg-interview

Be Sure To Read “This Week in AI“

This week’s executive overview and top links are here:
AI News #16: Week Ending 01/19/2024 with Executive Summary and Top 16 Stories

This is an extension of my weekly newsletter, This Week In AI, an executive summary of the top things to know in AI. Each week, I create an accessible overview for laypeople to feel confident they are conversant with the week’s AI developments. I include a curated list of must-click links of the week, to offer everyone a hands-on opportunity to explore the most intriguing updates in artificial intelligence across various categories, including robotics, imagery, video, AR/VR, science, ethics, and more. Beyond the overview, I post these topic-based deeper dives (below). If you haven’t read this week’s overview, I recommend starting there.