The AI Bubble Isn't Going To Pop - Building the Foundations of Intelligence Takes Time

We still need to stay pragmatic and realistic though about expectations

Andrej Karpathy’s recent podcast and follow-up reflections present one of the most nuanced, data-driven, and technically grounded views of artificial intelligence today. Contrary to speculation that the “AI bubble” is deflating, Karpathy’s perspective reinforces that we are in a period of foundational build-out, not decline.

The industry has moved beyond the experimental hype phase and is entering an engineering and infrastructure phase — where the key breakthroughs will come from integration, optimization, and new learning paradigms rather than single algorithmic miracles.

We are still in the early stages of understanding how to scale, generalize, and operationalize intelligence safely and reliably.

1. The Decade of Agents

Karpathy calls the coming period the “Decade of Agents.” His estimate of five to ten years for AGI is both conservative compared to Silicon Valley optimism and aggressive in historical context. The distinction reflects the duality of AI progress — extraordinary momentum paired with unfinished engineering.

The next decade will focus less on raw model breakthroughs and more on integration: connecting intelligence with memory, tools, sensors, and reasoning layers.

The current stage is about deployment, not discovery — ensuring that models work safely and effectively in real-world environments.

A ten-year path to AGI is historically fast. The Industrial Revolution, the transistor age, and the Internet all required decades of infrastructure buildup before widespread application.

Karpathy’s outlook is not bearish; it is precise. The road to intelligence is measured not by hype cycles, but by the steady expansion of systems that make intelligence useful.

2. The Scaffolding Challenge

Today’s AI capabilities exceed the systems built to support them — a phenomenon Karpathy calls model overhang. The gap between model potential and real-world implementation is now the largest bottleneck in the field.

Scaffolding refers to the ecosystem of memory, context, reasoning, and safety mechanisms surrounding the model. These systems determine whether AI is reliable, consistent, and safe.

Even if no new model were released for five years, there would be enough work to keep thousands of teams busy refining interfaces, orchestration frameworks, and long-term memory architectures.

The self-driving car analogy is apt: moving from 90% reliability to 99.9% reliability requires exponentially more work, data, and edge-case handling.

Unlike self-driving, AI agents add incremental value at every stage. A model that automates a 1-minute task today can automate 10 minutes next year and an hour within a few years.

Scaffolding is not a temporary layer; it is the primary barrier between impressive demos and enduring utility.

3. Animals, Ghosts, and the Nature of Learning

Karpathy’s “animals vs. ghosts” analogy captures the philosophical divide in AI.

Animals (including humans) are born with evolutionary priors — instincts, perception modules, and reasoning patterns encoded by millions of years of adaptation.

Ghosts (AI systems) learn by prediction, not embodiment. They lack instincts or physical grounding but can generalize across vast textual and multimodal domains.

Biological intelligence generalizes from minimal data; machine intelligence memorizes and scales through exposure.

The goal is not to recreate evolution but to design systems that combine both approaches — predictive learning enhanced by experiential feedback.

Future progress depends on bridging abstraction with embodiment: teaching AI to reason through interaction, not imitation.

Karpathy’s view reframes AI as a new form of intelligence — one that will evolve alongside humans, not replicate them.

4. Reinforcement Learning and the Case for Agentic Interaction

Karpathy is skeptical of traditional reinforcement learning (RL), describing it as “sucking supervision through a straw.” The issue lies in the inefficiency of reward signals and the noise inherent in the training loop.

Correct reasoning steps can be penalized if the outcome is wrong, while bad reasoning can be rewarded if the model stumbles onto a correct result.

The signal-to-compute ratio is poor — immense computational cost for limited learning benefit.

Instead, Karpathy advocates for agentic interaction: systems that learn by doing.

Learning emerges through environmental feedback, not static data.

Inspired by AlphaGo and AlphaZero, agentic interaction allows AI to self-improve through iterative simulation and exploration.

Emerging labs such as World Labs and DeepMind’s Genie are experimenting with world simulations — letting AI systems discover cause, effect, and physical reasoning.

For robotics, defense, and enterprise automation, this shift from memorization to experiential learning will define the next generation of intelligent systems.

Agentic learning represents the next step beyond RL — a more scalable, self-sustaining approach to intelligence.

5. The Cognitive Core

The Cognitive Core is Karpathy’s term for the next phase of model design — compact systems that focus on learning and adaptability rather than memorization and size.

Human cognition thrives on constraint; limited memory forces generalization. AI, by contrast, often overfits and loses flexibility.

Distillation and quantization now reduce model size by up to 90% while maintaining over 95% of accuracy.

Models under 10 billion parameters can now perform at levels once reserved for systems over 100 billion parameters.

Edge computing is rapidly expanding: by 2028, an estimated 40% of AI workloads will run locally instead of in centralized data centers.

Karpathy argues that intelligence will become distributed — large frontier models for discovery, paired with billions of small, composable models embedded in devices, networks, and workflows.

6. The Expanding Landscape of Training Paradigms

AI is not a single discipline but an evolving framework of complementary learning paradigms. Each plays a distinct role in creating adaptable intelligence.

Supervised Learning: Establishes foundational pattern recognition through labeled data.

Self-Supervised Pretraining: Enables models to predict missing tokens and build structure from unlabeled data.

Reinforcement Learning: Useful for decision-making but limited in scalability and interpretability.

RLHF (Human Feedback): Aligns output with human judgment and ethical preferences.

Curriculum Learning: Gradually increases task complexity to stabilize learning.

Contrastive Learning: Improves multimodal reasoning by distinguishing similarities and differences.

Chain-of-Thought Fine-Tuning: Encourages step-by-step reasoning and transparency.

System-Prompt Learning: Optimizes context and instruction rather than weights — a potential successor to RL for alignment.

Tool Use and API Integration: Expands real-world utility by allowing models to act, not just respond.

Distillation & Quantization: Makes intelligence portable and affordable.

Continual Learning: Keeps systems adaptive post-deployment.

Together, these paradigms form the architecture of machine understanding — an ongoing experiment in teaching intelligence how to learn.

7. Infrastructure Before Intelligence

The current stage of AI resembles the Internet buildout of the 1990s — a period of massive infrastructure investment preceding mainstream adoption. Before the web could transform communication, the world had to build servers, routers, and fiber networks. The same principle applies today.

$1.5 trillion: projected global AI spending by 2025.

19% CAGR: expected enterprise AI market growth through 2030.

$190 billion+: 2024 capital expenditure from cloud providers like Microsoft, Google, and Amazon, largely directed toward AI infrastructure.

$50 billion+: annual revenue from NVIDIA’s data center segment — up nearly 400% year-over-year.

500 terawatt-hours (TWh): expected annual energy consumption from AI data centers by 2030, roughly equal to France’s total electricity use.

These figures reflect a sector that is not overheating but industrializing. The foundations of the AI economy are being laid in silicon, power, and data.

8. The Vertical AI Thesis

At Six Point Ventures, we focus on Vertical AI — systems built for specific industries, combining proprietary data with expert context. While general-purpose models are powerful, they lack the domain fluency needed to operate in complex, regulated environments.

Proprietary datasets create defensible moats that general models cannot replicate.

Domain expertise ensures relevance in high-stakes fields like healthcare, defense, and compliance.

Regulatory alignment builds trust and enables adoption in sensitive industries.

Operational integration allows AI to embed directly within existing enterprise workflows.

Vertical AI is not about replacing professionals; it is about augmenting them. These companies will define the next generation of enduring, economically resilient AI businesses — deep, not wide; applied, not theoretical.

9. Market Metrics and Capital Formation

AI remains one of the fastest-growing sectors in modern history. The data points are conclusive.

$34 billion: private investment in generative AI in 2024, up nearly 20% year-over-year.

60%: proportion of Fortune 500 companies with active AI pilots or internal task forces.

110+: number of AI-native startups globally valued over $1 billion, up from fewer than 20 just three years ago.

10 million+: projected AI workforce worldwide by 2030, roughly double today’s figure.

$15–20 trillion: estimated contribution of AI to global GDP by 2030, representing approximately 15% of total global economic growth.

These metrics describe not a speculative bubble, but a generational reallocation of capital, labor, and intellectual energy.

The AI Bubble Isn't Going To Pop - Building the Foundations of Intelligence Takes Time