DeepSummary
CoreWeave co-founder Brannin McBee discusses the company's transition from cryptocurrency mining to providing cloud infrastructure for AI workloads. He explains how GPUs are more power-efficient than CPUs for parallelizable AI tasks like training large language models. However, the rapid adoption of AI is creating huge demand for GPU infrastructure that current cloud providers struggle to meet.
McBee highlights the engineering challenges of building specialized cloud fabrics with tens of thousands of GPUs interconnected via high-speed networks like Infiniband. He predicts continuing Nvidia dominance in training due to their software ecosystem, while inference may see more diversity with specialized chips like LPUs.
Looking ahead, McBee expects AI to rapidly transform industries like advertising through personalized generative ads. However, meeting the insatiable demand for AI compute will require massive investments in new data centers and sustainable energy sources like nuclear power over the next decade.
Key Episodes Takeaways
- CoreWeave pivoted from cryptocurrency mining to building specialized cloud infrastructure for AI and GPU-accelerated workloads.
- Scaling AI infrastructure requires building high-performance GPU fabrics interconnected with high-speed networks like Infiniband.
- Nvidia currently dominates AI training due to its CUDA software ecosystem, while inference may see more chip diversity.
- AI is rapidly being adopted across industries like advertising, driving huge demand for compute that is outstripping supply.
- Meeting future AI compute needs will require massive investments in new data centers and sustainable energy sources.
- Emerging use cases like personalized generative advertising will further increase AI's energy footprint.
- Bottlenecks in data center capacity and energy availability may constrain AI growth over the next decade.
- Cloud providers face an "innovator's dilemma" in adapting existing infrastructure for modern AI workloads.
Top Episodes Quotes
- “I mean, we're building at, I think it's 28 data centers this year across North America were one of the largest operators of this infrastructure in the world, and we are unable to keep up with demand, and we really dont see that subsiding for years to come.“ by Brannin McBee
- “An H 100 hosted at AWS is very different than an H 100 hosted at core weave because of the way that we run that infrastructure differently from a software perspective and the way we build it from a physical perspective.“ by Brannin McBee
- “Instead, if you could use generative AI to create on demands, always on ads for people that are 100% specified to the metadata associated with that user, those are going to be much more highly effective.“ by Brannin McBee
- “It honestly could be the end of this decade until you see this rebalancing of supply and demand. And not to say that that's overbuilding like thats just still on a heavy growth trajectory.“ by Brannin McBee
Entities
Company
Product
Person
Episode Information
This Week in Startups
Jason Calacanis
4/4/24
This Week in Startups is brought to you by…
OpenPhone. Create business phone numbers for you and your team that work through an app on your smartphone or desktop. TWiST listeners can get an extra 20% off any plan for your first 6 months at http://www.openphone.com/twist
Gusto is easy online payroll, benefits, and HR built for modern small businesses. Get three months free when you run your first payroll at Gusto.com/twist.
Northwest Registered Agent. When starting your business, it's important to use a service that will actually help you. Northwest Registered Agent is that service. They'll form your company fast, give you the documents you need to open a business bank account, and even provide you with mail scanning and a business address to keep your personal privacy intact. Visit http://www,northwestregisteredagent.com/twist to get a 60% discount on your next LLC.
*
Todays show:
Brannin McBee joins Jason to discuss CoreWeave’s transition from cryptocurrency to AI (3:36), the energy dynamics of GPUs (14:04), innovations in data center cooling (19:28), and the future challenges and impacts of AI infrastructure on industries like search, advertising, and e-commerce (45:36).
*
Timestamps:
(00:00) CoreWeave’s Brannin McBee joins Jason
(3:36) The founding story of CoreWeave and the transition from the cryptocurrency market to AI
(12:49) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist
(14:04) Energy reliance of GPUs, the future of data centers, and the cost and efficiency of GPU cloud infrastructure
(19:28) Direct-to-chip liquid cooling and other cooling solutions for GPUs
(24:52) Demand trends for GPU capacity and the growth of inference linked to user growth
(29:06) Gusto - Get three months free when you run your first payroll at http://gusto.com/twist
(30:28) LPUs Vs. GPUs
(34:13) Dominance of Nvidia in training models, implications of open source chips and chip architecture
(39:01) Northwest Registered Agent - Get a 60% discount on your next LLC at http://www.northwestregisteredagent.com/twist
(40:00) Challenges in training large language models and the use of infrastructure as a weapon by big tech
(45:36) The potential impact of AI on search engines, advertising sector, and ecommerce
(52:46) Timeline for supply-demand balance in AI infrastructure and the operational feat of running multiple data centers
*
Subscribe to This Week in Startups on Apple: https://rb.gy/v19fcp
*
Follow Brannin:
X: https://twitter.com/branninmcbee
LinkedIn: https://www.linkedin.com/in/branninmcbee
*
Follow Jason:
LinkedIn: https://www.linkedin.com/in/jasoncalacanis
*
Thank you to our partners:
(12:49) OpenPhone - Get 20% off your first six months at http://www.openphone.com/twist
(29:06) Gusto - Get three months free when you run your first payroll at http://gusto.com/twist
(39:01) Northwest Registered Agent - Get a 60% discount on your next LLC at http://www.northwestregisteredagent.com/twist
*
Great 2023 interviews: Steve Huffman, Brian Chesky, Aaron Levie, Sophia Amoruso, Reid Hoffman, Frank Slootman, Billy McFarland
*
Check out Jason’s suite of newsletters: https://substack.com/@calacanis
*
Follow TWiST:
Substack: https://twistartups.substack.com
Twitter: https://twitter.com/TWiStartups
YouTube: https://www.youtube.com/thisweekin
Instagram: https://www.instagram.com/thisweekinstartups
TikTok: https://www.tiktok.com/@thisweekinstartups
*
Subscribe to the Founder University Podcast: https://www.founder.university/podcast