☰ The Intelligence Curse
Art by Nomads & Vagabonds
The Intelligence Curse
by Luke Drago and Rudolf Laine
We will soon live in the intelligence age. What you do with that information will determine your place in history.
The imminent arrival of AGI has pushed many to try to seize the levers of power as quickly as possible, leaping towards projects that, if successful, would comprehensively automate all work. There is a trillion-dollar arms race to see who can achieve such a capability first, with trillions more in gains to be won.
Yes, that means you’ll lose your job. But it goes beyond that: this will remove the need for regular people in our economy. Powerful actors—like states and companies—no longer have an incentive to care about regular people. We call this the intelligence curse.
If we do nothing, the intelligence curse will work like this:
- Powerful AI will push automation through existing organizations, starting from the bottom and moving to the top.
- AI will obsolete even outlier human talent. Social mobility will stop, ending the social dynamism and progress that it drives.
- Non-human factors of production, like capital, resources, and control over AI, will become overwhelmingly more important than humans.
- This will usher in incentives for powerful actors around the world that break the modern social contract.
- This could result in the gradual—or sudden—disempowerment of the vast majority of humanity.
But this prophecy is not yet fulfilled; we reject the view that this path is inevitable. We see a different future on the horizon, but it will require a deliberate and concerted effort to achieve it.
We aim to change the incentives driving the intelligence curse, maintaining human economic relevance and strengthening our democratic institutions to withstand what will likely be the greatest societal disruption in history.
To break the intelligence curse, we should chart a different path on the tech tree_,_ building technology that lets us:
- Avert AI catastrophes by hardening the world against them, both because it is good in itself and because it removes the security threats that drive calls for centralization.
- Diffuse AI, to get it in the hands of regular people. In the short-term, build AI that augments human capabilities. In the long-term, align AI directly to individual users and give everyone control in the AI economy.
- Democratize institutions, making them more anchored to the needs of humans even as they are buffeted by the changing incentive landscape and fast-moving events of the AGI transition.
In this series of essays, we examine the incoming crisis of human irrelevance and provide a map towards a future where people remain the masters of their destiny.
Chapters
1. Introduction (this page)
We will soon live in the intelligence age. What you do with that information will determine your place in history.
2. Pyramid Replacement
Increasingly powerful AI will trigger pyramid replacement: a systematic hollowing out of corporate structures that starts with entry-level hiring freezes and moves upward through waves of layoffs.
3. Capital, AGI, and Human Ambition
AI will make non-human factors of production more important than human ones. The result may be a future where today's power structures become permanent and frozen, with no remaining pathways for social mobility or progress.
4. Defining the Intelligence Curse
With AGI, powerful actors will lose their incentive to invest in regular people–just as resource-rich states today neglect their citizens because their wealth comes from natural resources rather than taxing human labor. This is the intelligence curse.
5. Shaping the Social Contract
The intelligence curse will break the core social contract. While this suggests a grim future, understanding how economic incentives reshape societies points to a solution: we can deliberately develop technologies that keep humans relevant.
6. Breaking the Intelligence Curse
Avert AI catastrophes with technology for safety and hardening without requiring centralizing control. Diffuse AI that differentially augments rather than automates humans and decentralizes power. Democratize institutions, bringing them closer to regular people as AI grows more powerful.
7. History is Yours to Write
You have a roadmap to break the intelligence curse. What will you do with it?
Acknowledgements
Thanks to David Duvenaud, Tom Everitt, Nitarshan Rajkumar, Anton Leicht, Herbie Bradley, Bilal Chughtai, Liam Patell, Henry Sleight, Max Kaufmann, Nathan Darmon, Lara Thurnherr, Stephen Clare, Edward Kembery, Connor A. Stewart Hunter, Dhruv Sumathi, Oscar Moxon, Tom Reed, Alejandro Ortega, Cristian Trout, Clay Wren, Aviel Parrack, Lysander Mawby, Jacob Pfau, James Campbell, Joseph Miller, Shannon Yang, Jasmin Kaur, Sam Cherry, Luca Gandrud, Riya Kataria, Li-Lian Ang, and Nicholas Osaka for reviewing early drafts of the essays in this series.
Thank you to Duncan McClements, Deric Cheng, Sam Manning, Josh Priest, and Caleb Peppiatt for reviewing portions of the economic and political economy work.
Thank you to Brendan McCord, Richard Ngo, Iason Gabriel, and Joe Pollard for conversations that informed the political theory, social contract, and institutions sections.
Thank you to Séb Krier and Jamie Bernardi for conversations that informed the tech solutions sections.
Thank you to Mahmoud Ghanem and Jack Miller for conversations that informed our thoughts on cybersecurity.
Thank you to Andrew Synder-Beattie for conversations that informed our biosecurity agenda.
Special thanks to Nomads & Vagabonds for the custom artwork used on this site.
Subscribe to follow our work:
Subscribe
Contact us
>> Next: Pyramid Replacement
Follow our work: