December 6, 2024 • 1hr 35min

Chetan Puttagunta and Modest Proposal - Capital, Compute & AI Scaling - [Invest Like the Best, EP.399]

Invest Like the Best with Patrick O'Shaughnessy

This episode features Chetan Puttagunta, General Partner at Benchmark, and Modest Proposal, an anonymous public markets investor, discussing a potential pivotal shift in AI development. The conversation explores how leading AI labs are hitting scaling limits and transitioning from pre-training to test-time compute approaches, with implications for both public and private markets.

0:00

-0:00

Key Takeaways

Major shift in AI development from pre-training scaling to test-time compute as labs hit scaling limits with synthetic data
Test-time compute paradigm focuses on having LLMs analyze problems, generate multiple potential solutions in parallel, and use verifiers to iterate
Democratization of AI development as small teams can now reach frontier model performance with orders of magnitude less capital
Infrastructure implications as test-time compute may require different network architectures than massive pre-training clusters
Application layer opportunities expanding rapidly across enterprise software categories with AI-first solutions
Cost curves dropping dramatically for AI inference, enabling high-margin AI applications
Silicon Valley renaissance as hub of AI innovation across labs, infrastructure, and applications

Introduction

This episode features Chetan Puttagunta, General Partner at Benchmark, and Modest Proposal, an anonymous public markets investor, discussing a potential pivotal shift in AI development. The conversation explores how leading AI labs are hitting scaling limits and transitioning from pre-training to test-time compute approaches, with implications for both public and private markets.

Topics Discussed

The Shift from Pre-Training to Test-Time Compute (5:20)

Chetan explains how AI labs have hit plateaus in pre-training scaling, where throwing more compute at training larger models was the primary path to improvement. The industry is now shifting to "test-time compute" where models analyze problems and pursue multiple solution paths in parallel.

Pre-training scaling limits emerged as labs exhausted available human-generated text data
Synthetic data generation has not enabled continued scaling as hoped
Test-time compute focuses on reasoning capabilities during inference rather than just larger models
New scaling paradigm measures intelligence against time (on logarithmic scale) rather than compute

Implications for Public Tech Companies (11:27)

Modest Proposal discusses how this shift impacts public market valuations and capital expenditure plans for major tech companies.

AI theme permeates 40-45% of market cap across tech, industrials, and utilities
Better alignment of revenue and expenditures with test-time compute vs massive upfront training costs
Infrastructure architecture may shift from massive training clusters to distributed inference optimization
Capital efficiency improvements as companies can better match spending to actual usage

Democratization of Model Development (17:29)

Chetan describes how small teams are now able to reach frontier model performance with minimal capital, enabled by open source models like Meta's LLaMA.

Teams of 2-5 people matching performance of leading labs
Orders of magnitude less capital required compared to previous approaches
Open source models like LLaMA enabling rapid innovation
Vertical specialization allowing focused optimization for specific use cases

Strategic Positioning of Major AI Players (29:52)

The discussion analyzes the positioning of OpenAI, Anthropic, xAI and other leading AI companies in light of these changes.

OpenAI's consumer mindshare advantage but challenge from free alternatives
Anthropic's technical excellence but unclear strategic path
Meta's embedded optionality and distribution advantages
Google's innovative dilemma as incumbent trying to adapt

Application Layer Opportunities (49:06)

Chetan outlines the expanding opportunities for AI-first applications across enterprise software categories.

Sales automation with companies like 11X
Legal tech increasing lawyer productivity
Accounting/financial modeling automation
Game development transformation
Circuit board design optimization
Ad network efficiency improvements

Infrastructure and Cost Implications (1:05:00)

The conversation explores how infrastructure needs and economics are changing with the shift to test-time compute.

Dramatic cost reductions in inference (100-200x)
Different infrastructure requirements for bursty inference vs sustained training
Hyperscaler advantages in providing reliable, cost-effective compute
Capital efficiency improvements for AI companies

Future of AI Development (1:28:58)

The discussion concludes with thoughts on artificial general intelligence (AGI) and artificial superintelligence (ASI).

AGI potentially achievable by 2025 according to some definitions
Breakthrough possibilities in mathematics, physics, and other sciences
Recursive self-improvement as potential path to ASI
Silicon Valley's continued role as innovation hub

Conclusion

The shift from pre-training to test-time compute represents a potentially significant change in AI development, with major implications for both public and private markets. This transition may enable more capital-efficient development of AI capabilities while creating new opportunities in infrastructure and applications. The discussion suggests we are entering a new phase of AI development that could be more sustainable and broadly accessible while still driving significant technological advancement.

The conversation highlights how this shift could lead to a more rational and predictable development path for AI, while still maintaining rapid progress toward increasingly capable systems. Silicon Valley appears positioned to remain at the center of this innovation, with both established players and startups contributing to advancement across multiple layers of the AI stack.