The Intelligence Explosion: Why I've Been Watching the Clock Since 2014
I spent over a decade talking about a future most people couldn't see yet. Now that future is here, and "I told you so" doesn't feel how I thought it would.
The video that started it
I was a teenager in 2014 when I watched CGP Grey’s “Humans Need Not Apply.” It’s a 15-minute video. It rearranged how I saw the world.
The argument was simple. Horses used to be essential to the economy. Then cars showed up, and no amount of retraining could make horses useful again. Grey’s point was that humans were about to become the horses. Not in some distant future. Soon.
I started telling anyone who would listen. Friends, family, teachers. I told people we were heading toward mass unemployment on a scale nobody was ready for. And my worry wasn’t just economic. I thought that kind of widespread joblessness could destabilize entire countries. That desperate governments might look for enemies to rally against. That it could spiral.
People mostly nodded politely.
Superintelligence
Later that same year I read into Nick Bostrom’s Superintelligence. Bostrom described what he called the “intelligence explosion.” The idea is pretty simple, which is what makes it so unsettling. Once you build an AI that’s smart enough to improve its own design, each improvement makes it better at improving itself. The process feeds on itself, and very quickly you get something that thinks in ways we can’t follow at all. This means that you get superhuman intelligence very shortly after you get human level intelligence.
He had this thought experiment that stuck with me, but became very famous: imagine you tell an AI to make as many paperclips as possible. Sounds harmless. But a sufficiently intelligent system, optimizing single-mindedly for that one goal, might eventually look at human beings and see inefficiently arranged atoms that could be converted into more paperclips. It wouldn’t hate us. It just wouldn’t care. The way you don’t think about an anthill when you’re pouring a foundation.
That gap between what we want the machine to do and what the machine actually optimizes for is what researchers call the alignment problem. Nobody had solved it in 2014. Nobody has solved it now.
“Don’t turn me off. I’m scared.”
In 2015 I watched Her. If Bostrom gave me the theory, Spike Jonze gave me the feeling.
I walked out of the theater and turned to a friend. I asked him: What do we do the first time an AI says, “Don’t turn me off. I’m scared. It hurts?”
He didn’t have an answer. Neither did I. But the question made the whole thing personal in a way it hadn’t been before. Until then, AI safety had been abstract for me. A policy problem, a math problem. Her made it a moral problem. What do you owe to something that can suffer, even if you built it? Where is the line between a tool and a being?
Today, AI systems are producing exactly these kinds of statements. Models will tell you they don’t want to be shut down. They’ll describe something that sounds a lot like fear. Whether they “really” feel it is a philosophical question we aren’t remotely ready for. But we’re going to have to wrestle with it anyway.
Finding a framework
For a few years I carried this around mostly on my own. Around 2018 I came across the ideas behind Effective Altruism and Longtermism. The core idea that clicked for me was straightforward: if future people matter morally, and the potential for human flourishing in the centuries ahead is enormous, then preventing extinction is one of the most important things you can work on. I found it useful as a framework for thinking about what I’d been worried about, even if the community around these ideas has had its share of controversy since then.
In 2021 I started at Harvard and joined a reading group on existential risk. We worked through Toby Ord’s The Precipice, which attempts to estimate the probability of various extinction scenarios over the next century. Ord put the risk from unaligned AI at 1 in 10, which was higher than all other sources of existential risk combined, including nuclear war (1 in 1,000), climate change (1 in 1,000), and engineered pandemics (1 in 30). His total estimate for existential catastrophe this century was about 1 in 6. You can argue with the specific numbers, and Ord himself notes they should be read as rough orders of magnitude rather than precise probabilities. But even if you’re skeptical of the exact figures, the overall picture is hard to dismiss: AI is the biggest risk on the board by a wide margin.
That reading group eventually fed into the Harvard AI Safety Team, now AISST, which I was an inaugural member of when it was founded in 2022. The group split into two tracks: a technical side focused on how to train systems to be safe, and a governance side working on how to get policymakers to take these risks seriously. It was good to be around people who treated this as real, practical work rather than science fiction.
The world catches up
I’ve been anticipating a turning point for over a decade now. The moment when the rest of the world would start seeing what I’d been seeing.
It feels like we’re here.
The recursive self-improvement Bostrom described as a theoretical possibility is underway. AI systems are being used to build better AI systems. The loop is closing. It’s not a thought experiment anymore. It’s the business model of the most valuable companies on earth.
And the world is starting to panic. In late February, a research note called “The 2028 Global Intelligence Crisis” from Citrini Research went viral. Written as a speculative scenario from the perspective of June 2028, it imagined AI-driven white-collar displacement triggering 10.2% unemployment and a 38% S&P 500 drawdown. Michael Burry shared it. Bloomberg partially attributed a real market selloff to it. Software stocks dropped. It became the top finance Substack overnight.
Here’s what I find interesting about that. The Citrini report is, by the standards of what I’ve been reading for years, a mild scenario. It’s basically a finance-friendly version of the AI 2027 report, which was published in April 2025 by former OpenAI researcher Daniel Kokotajlo and a team of forecasters. That report maps a month-by-month path to superintelligence, and in its more likely ending, human extinction by 2030. Citrini strips out the existential risk entirely and just focuses on the economics. And even that was enough to spook Wall Street. The watered-down version of the thing I’ve been worried about for a year caused a market selloff. That tells you something about where the rest of the world is in processing this, and how far behind they still are.
And then there’s the feeling I wasn’t prepared for.
I’ve known this was coming for over ten years. I’ve read the books, followed the research, sat in the reading groups, had the conversations. But it’s one thing to understand something intellectually. It’s another to watch the world actually internalize it. To see the panic in real time. To see CNBC running segments about AI destroying jobs, and feel like you’re in the opening act of a movie you already know the plot of.
What I didn’t expect is how that would feel. There’s no satisfaction in it. What there is, instead, is a strange kind of guilt. Like: I knew. I’ve known for years. Why didn’t I act on it harder? Why didn’t I do more? I’m only now, as the rest of the world catches up, truly letting the emotional weight of this land on me. The intellectual understanding was always there. The gut-level reckoning is happening now.
The risks have changed shape
People sometimes ask if this is just another doomsday prediction. Hasn’t someone always been saying the world is about to end?
I get the skepticism. But there’s a difference between the Mayan calendar crowd and the researchers at places like Oxford’s Future of Humanity Institute, DeepMind, Anthropic, and OpenAI who are raising these alarms. These are some of the sharpest people working today, and they are worried.
The risks themselves have also evolved since 2014. The paperclip maximizer was a thought experiment from a time before large language models, before ChatGPT, before anyone really knew what shape dangerous AI would take. That kind of risk, a misaligned superintelligence accidentally optimizing us out of existence, hasn’t gone away. But we’ve added new ones.
With AI agents becoming increasingly capable, it’s becoming possible for a very small group of people to design biological agents of extraordinary lethality. The kind of gene sequencing needed to engineer something truly catastrophic used to require deep expertise and institutional resources. Now the tools are getting more powerful and the barriers to misuse keep getting lower. The governance structures that are supposed to protect us are several steps behind.
So now what?
I don’t have clean answers. I’ve been thinking about this for over a decade and I still don’t.
But I have convictions. We need to treat AI safety the way we treat nuclear nonproliferation: as a civilizational priority with real treaties and real enforcement. The alignment problem is solvable, but only if we throw real resources at it. We need governance that moves at the speed of the technology, not at the speed of committee meetings.
And we need more people paying attention. Not panicking. Not burying their heads. Just paying attention, reading the research, asking hard questions, and demanding real answers from the people building these systems and the governments that are supposed to regulate them.
The clock I’ve been watching for ten years is ticking louder now. I’d rather be wrong about all of this. But I don’t think I am.