P(doom) n=5
Taking a pulse on the odds of AI killing us all.
Probability of Doom.
p(doom). It’s a dark in-joke that some consider one of the most important questions facing humanity.
“p” stands for probability.
“doom” is a more general term but is generally understood to be a scenario in which a sophisticated AI causes the end of human existence.
And so, your p(doom) is your best guess at the odds of AI killing us all. The specific circumstances on which these happen are wide and varied- from accidental paperclip maximizing to bioterrorists using AI to build highly infectious deadly diseases.
If you’re not exposed to these circles, the concept of p(doom) likely seems something reserved for extremists and philistines, at the edges of intellectual society- especially if they declare a p(doom) greater than, say, 1%. And yet, many public and influential figures (more commonly within the AI industry itself) have declared their p(doom) at alarmingly high levels.
The rest of this post is a compilation of notable figures’ declarations of their p(doom) along with their reasoning (if included). I hope this serves to provide a balanced perspective on what the industry, regulators, and commentators think at this snapshot in time about the dangers of AI.
Lina Khan (Chairwoman of FTC)
p(doom): 15%
Source: Hard Fork Podcast
Lina Khan stated in a recent podcast of Hard Fork that she “has to be optimistic” and that 15% was her p(doom).
I don’t know about you, dear reader, but hearing that a highly-acclaimed high-ranking government official thinks that 15% is an optimistic p(doom) is more than slightly terrifying. I couldn’t find very many instances of other high-ranking political figures giving their p(doom)s, so we may need to use Lina Khan as a stand-in for regulators.
I think Kevin Roose and Casey Newton’s response aptly capture my thoughts-
Kevin: “Oh”
Casey Newton “All Right”
Kevin: “Yeah.”
Casey: “And they say there are no techno-optimists in the government.”
Dario Amodei (CEO of Anthropic AI)
p(doom): 15%
Source: Logan Bartlett Podcast
Dario Amodei is the CEO of Anthropic AI, which has 3.3 billion dollars in funding. Also seems extremely high, but perhaps a publically declared high p(doom) is to be expected for an executive in the AI space.
Conjecture.dev Employees (AI startup)
Average p(doom): 71%
Source: Archived Corporate Blog Post
Conjecture.dev is a company that is attempting to build software to help “ensure the controllable, safe development of advanced AI technology.” Their specific offerings
The average p(doom) listed above is the result of an internal survey, which they’ve since taken down, asking their (23) employees for their perspectives on AI doom and progress. Their p(doom) may seem exceptionally high, but it is worth noting that they recruit/receieved investment heavily from people concerned with AGI.
Paul Christiano (AI Researcher)
p(doom): 10-20%
Source: Bankless Podcast
Paul Christiano used to run OpenAI’s alignment team and now runs the Alignment Research Center.
Manifold Markets (Tech People)
p(doom): 11%
Source: Manifold Market Market
Manifold Markets is a prediction market which lets users bet the website’s currency on the odds that a specific event in the future will ocur. This market is set around the prediction that AI will wipe out humanity by 2023- hence, the odds of the market can serve as a rough approximation for the site’s p(doom)- if a user thought that p(doom) was actually lower or higher, they would in theory be incentivized to bet towards that outcome.
Around 235 of the site’s users have bet on this market; however, it is important to note that this market may not be the most accurate as, if you vote for AGI doom and win, you won’t really be alive to enjoy your victory- however, you could still ride the waves of public opinion before then.
Further Consumption related to P(doom)
The Hard Fork podcast: Lovely podcast with Casey Newton and Kevin Roose, two hilarious but very well-informed reporters. Remember Sydney (the chatbot) asking that NYT reporter to leave his wife for it? That was Kevin Roose. They ask many of their guests for their genreal thoughts on the progression of AI, and are always ahead of the curve on covering the trends in AI.
Manifold Markets: Definitely worth browsing for seeing what a more futuristic tech crowd is interested in and betting on.
Logan Bartlett: Not very AI-specific, but one of my favorite podcacsts. I think of it as providing great insights into the tech industry.
This Discussion between George Hotz and Eliezer Yuddowsky: Two differing perspectives on AGI/p(doom)- although to people uninitiated to their perspectives, equally novel. Hosted on the fantastic Dwarkesh Patel Podcast. George Hotz’s post-debate debrief
Call for more p(doom)s
I will continue to update this post with notable p(doom)s, so if you see any influential or well-informed figures speculate about the p(doom), please do send an email my way [my name sf sanchez at gmail . com]!