We still need to stay pragmatic and realistic though about expectations
Andrej Karpathy’s recent podcast and follow-up reflections present one of the most nuanced, data-driven, and technically grounded views of artificial intelligence today.
Contrary to speculation that the “AI bubble” is deflating, Karpathy’s perspective reinforces that we are in aperiod of foundational build-out, not decline.
The industry has moved beyond the experimental hype phase and is entering anengineering and infrastructure phase— where the key breakthroughs will come from integration, optimization, and new learning paradigms rather than single algorithmic miracles.
https://www.dwarkesh.com/p/andrej-karpathy
We are still in the early stages of understanding how to scale, generalize, and operationalize intelligence safely and reliably.
Karpathy calls the coming period the“Decade of Agents.”His estimate of five to ten years for AGI is both conservative compared to Silicon Valley optimism and aggressive in historical context. The distinction reflects the duality of AI progress — extraordinary momentum paired with unfinished engineering.
The next decade will focus less on raw model breakthroughs and more onintegration: connecting intelligence with memory, tools, sensors, and reasoning layers.
The current stage is aboutdeployment, not discovery — ensuring that models work safely and effectively in real-world environments.
A ten-year path to AGI is historically fast. The Industrial Revolution, the transistor age, and the Internet all required decades of infrastructure buildup before widespread application.
Karpathy’s outlook is not bearish; it is precise. The road to intelligence is measured not by hype cycles, but by the steady expansion of systems that make intelligence useful.
Today’s AI capabilities exceed the systems built to support them — a phenomenon Karpathy callsmodel overhang.The gap between model potential and real-world implementation is now the largest bottleneck in the field.
Scaffoldingrefers to the ecosystem of memory, context, reasoning, and safety mechanisms surrounding the model. These systems determine whether AI is reliable, consistent, and safe.
Even if no new model were released for five years, there would be enough work to keep thousands of teams busy refining interfaces, orchestration frameworks, and long-term memory architectures.
Theself-driving car analogyis apt: moving from 90% reliability to 99.9% reliability requires exponentially more work, data, and edge-case handling.
Unlike self-driving, AI agents addincremental valueat every stage. A model that automates a 1-minute task today can automate 10 minutes next year and an hour within a few years.
Scaffolding is not a temporary layer; it is the primary barrier between impressive demos and enduring utility.
Karpathy’s “animals vs. ghosts” analogy captures the philosophical divide in AI.
Animals(including humans) are born with evolutionary priors — instincts, perception modules, and reasoning patterns encoded by millions of years of adaptation.
Ghosts(AI systems) learn by prediction, not embodiment. They lack instincts or physical grounding but can generalize across vast textual and multimodal domains.
Biological intelligence generalizes from minimal data; machine intelligence memorizes and scales through exposure.
The goal isnotto recreate evolution but to design systems that combine both approaches — predictive learning enhanced by experiential feedback.
Future progress depends on bridging abstraction with embodiment: teaching AI to reason through interaction, not imitation.
Karpathy’s view reframes AI as a new form of intelligence — one that will evolve alongside humans, not replicate them.
Karpathy is skeptical oftraditional reinforcement learning (RL), describing it as “sucking supervision through a straw.” The issue lies in the inefficiency of reward signals and the noise inherent in the training loop.
Correct reasoning steps can be penalized if the outcome is wrong, while bad reasoning can be rewarded if the model stumbles onto a correct result.
Thesignal-to-compute ratiois poor — immense computational cost for limited learning benefit.
Instead, Karpathy advocates foragentic interaction: systems that learn by doing.
Learning emerges throughenvironmental feedback, not static data.
Inspired byAlphaGoandAlphaZero, agentic interaction allows AI to self-improve through iterative simulation and exploration.
Emerging labs such asWorld LabsandDeepMind’s Genieare experimenting with world simulations — letting AI systems discover cause, effect, and physical reasoning.
For robotics, defense, and enterprise automation, this shift from memorization to experiential learning will define the next generation of intelligent systems.
Agentic learning represents the next step beyond RL — a more scalable, self-sustaining approach to intelligence.
TheCognitive Coreis Karpathy’s term for the next phase of model design — compact systems that focus on learning and adaptability rather than memorization and size.
Human cognition thrives on constraint; limited memory forces generalization. AI, by contrast, often overfits and loses flexibility.
Distillation and quantizationnow reduce model size by up to90%while maintaining over95%of accuracy.
Models under10 billion parameterscan now perform at levels once reserved for systems over100 billion parameters.
Edge computing is rapidly expanding: by2028, an estimated40% of AI workloadswill run locally instead of in centralized data centers.
Karpathy argues that intelligence will becomedistributed— large frontier models for discovery, paired with billions of small, composable models embedded in devices, networks, and workflows.
AI is not a single discipline but an evolving framework of complementary learning paradigms. Each plays a distinct role in creating adaptable intelligence.
Supervised Learning:Establishes foundational pattern recognition through labeled data.
Self-Supervised Pretraining:Enables models to predict missing tokens and build structure from unlabeled data.
Reinforcement Learning:Useful for decision-making but limited in scalability and interpretability.
RLHF (Human Feedback):Aligns output with human judgment and ethical preferences.
Curriculum Learning:Gradually increases task complexity to stabilize learning.
Contrastive Learning:Improves multimodal reasoning by distinguishing similarities and differences.
Chain-of-Thought Fine-Tuning:Encourages step-by-step reasoning and transparency.
System-Prompt Learning:Optimizes context and instruction rather than weights — a potential successor to RL for alignment.
Tool Use and API Integration:Expands real-world utility by allowing models to act, not just respond.
Distillation & Quantization:Makes intelligence portable and affordable.
Continual Learning:Keeps systems adaptive post-deployment.
Together, these paradigms form the architecture of machine understanding — an ongoing experiment in teaching intelligence how to learn.
The current stage of AI resembles theInternet buildout of the 1990s— a period of massive infrastructure investment preceding mainstream adoption. Before the web could transform communication, the world had to build servers, routers, and fiber networks. The same principle applies today.
$1.5 trillion: projected global AI spending by 2025.
19% CAGR: expected enterprise AI market growth through 2030.
$190 billion+: 2024 capital expenditure from cloud providers like Microsoft, Google, and Amazon, largely directed toward AI infrastructure.
$50 billion+: annual revenue from NVIDIA’s data center segment — up nearly400% year-over-year.
500 terawatt-hours (TWh): expected annual energy consumption from AI data centers by 2030, roughly equal to France’s total electricity use.
These figures reflect a sector that is not overheating butindustrializing. The foundations of the AI economy are being laid in silicon, power, and data.
AtSix Point Ventures, we focus onVertical AI— systems built for specific industries, combining proprietary data with expert context. While general-purpose models are powerful, they lack the domain fluency needed to operate in complex, regulated environments.
Proprietary datasetscreate defensible moats that general models cannot replicate.
Domain expertiseensures relevance in high-stakes fields like healthcare, defense, and compliance.
Regulatory alignmentbuilds trust and enables adoption in sensitive industries.
Operational integrationallows AI to embed directly within existing enterprise workflows.
Vertical AI is not about replacing professionals; it is aboutaugmenting them. These companies will define the next generation of enduring, economically resilient AI businesses — deep, not wide; applied, not theoretical.
AI remains one of the fastest-growing sectors in modern history. The data points are conclusive.
$34 billion: private investment in generative AI in 2024, up nearly20% year-over-year.
60%: proportion of Fortune 500 companies with active AI pilots or internal task forces.
110+: number of AI-native startups globally valued over $1 billion, up from fewer than 20 just three years ago.
10 million+: projected AI workforce worldwide by 2030, roughly double today’s figure.
$15–20 trillion: estimated contribution of AI to global GDP by 2030, representing approximately15% of total global economic growth.
These metrics describe not a speculative bubble, but a generational reallocation of capital, labor, and intellectual energy.
Big Desk Energy | Subscribe | Big Desk Energy
startup insights, stories, and vibes sent to your inbox every Tuesday
mail.bigdeskenergy.com/subscribe?_bhba=a912eba6-7a35-4c1b-a9cb-9721b5c72389
Outdated tax tools drain time, increase audit risk, and limit strategy. Inthis on-demand webinar, see how Longview Tax helps you cut manual work, boost accuracy, and get back to what matters most.
Or copy and paste this link to others:{{rp_refer_url_no_params}}
© 2026 Trace Cohen's Vertical Ai Investor Newsletter
1 Unicorn Ranch
Port Washington, New York 11050, United States