On July 16th, 1945, when the world’s first nuclear explosion shook the plains of New Mexico, J. Robert Oppenheimer, who led the project, quoted the Bhagavad Gita, “Now I am become Death, the destroyer of worlds.” And indeed, he had. The world was never truly the same after nuclear power became a reality.
Today, however, we have lost that reverence for the power of technology. Instead of proceeding deliberately and with caution, we rush ahead. In his Techno-Optimist Manifesto, tech investor Marc Andreessen implied that AI regulation was a form of murder. Defense Secretary Pete Hegseth punished Anthropic when it tried to impose limits on its own technology.
Clearly, we’ve been here before and shown that we can meet the challenge. We contained the nuclear threat and put useful limits on the use of genomics, while still allowing the technology to develop. Yet when we’ve failed to heed warnings, as we did with financial engineering, we’ve paid a heavy price. That choice between recklessness and prudence, is what we have before us now.
How We Put The Nuclear Genie Back In The Bottle
The story of nuclear weapons didn’t start with Oppenheimer, not by a long shot. In fact, if we were going to attribute the Manhattan Project to a single person, it would probably be a Hungarian immigrant physicist named Leo Szilard, who was one of the first to conceive of the possibility of a nuclear chain reaction.
In 1939, upon hearing of the discovery of nuclear fission in Germany he, along with fellow Hungarian émigré Eugene Wigner, decided that the authorities needed to be warned. Szilard then composed a letter warning of the possibility of a nuclear bomb. The letter was eventually signed by Albert Einstein and sent to President Roosevelt. That’s what led to the Manhattan Project that developed the nuclear bomb.
Yet after the explosions at Hiroshima and Nagasaki, many of the scientists who worked to develop the bomb wanted to educate the public of its dangers. In 1955, the philosopher Bertrand Russell issued a manifesto signed by a number of scientific luminaries. This led to a series of conferences at Pugwash, Nova Scotia were convened to discuss different approaches to protect the world from weapons of mass destruction.
These efforts involved far more than talk. They helped to shape the non-proliferation agenda and led to concrete achievements such as the Partial Test Ban Treaty. In fact, these contributions were so crucially important that the organizers of the Pugwash conferences were awarded the Nobel Peace Prize in 1995, and they continue even today.
Putting Limits On What We Do With The Code Of Life
While the nuclear age started with a bang, the genetic age began with a simple article in the scientific journal Nature, written by two relatively unknown scientists named James Watson and Francis Crick. To the untrained eye, it seemed like a run-of-the-mill paper about the structure of an obscure molecule. Yet the final sentence belied an earthshaking insight.
“It has not escaped our notice that the specific pairing we have postulated immediately suggests a possible copying mechanism for the genetic material.”
It was one of those rare watershed moments when an entirely new branch of science arose from a single event. The field progressed quickly and, roughly 20 years later, a brilliant researcher named Paul Berg discovered that you could merge human DNA with that from other living things, creating new genetic material that didn’t exist in nature. Much like Oppenheimer, Berg understood that, due to his work, humanity stood on a precipice and it wasn’t quite clear where the edge was.
He organized a conference at Asilomar State Beach in California to establish guidelines. Importantly, participation wasn’t limited to scientists. A wide swath of stakeholders were invited, including public officials, members of the media, and ethical specialists. The result, now known as the Berg Letter, called for a moratorium on the riskiest experiments until the dangers were better understood. These norms were respected for decades.
Today, we’re undergoing another revolution in genomics and synthetic biology. New technologies, such as CRISPR and mRNA techniques, have opened up incredible possibilities, but also serious dangers. Yet here again, pioneers in the field like Jennifer Doudna are taking the lead in devising sensible guardrails and using the technology responsibly.
Carol’s Journey
In 2019, a Facebook researcher set up a fictitious account for “Carol Smith,” a politically conservative mother from Wilmington, North Carolina. Carol then liked a few mainstream, but conservative-leaning pages. Within days, Facebook’s algorithm sent her down a rabbit hole of QAnon conspiracies and white supremacist content.
According to whistleblower complaints, top Facebook executives, including Mark Zuckerberg, were notified that their platform was radicalizing its users, but chose profits and growth over safety. This was not an isolated incident, but part of an established pattern of how Facebook does business.
In 2016, Nobel Laureate Maria Ressa warned company leaders that Facebook was being exploited by bot networks to influence elections. The Wall Street Journal published a series of reports showing that the company knew that its product was harming its users, especially teenage girls, but took no action to mitigate the damage. More recently, a court of law confirmed the accusations and found the firm liable for damages.
The contrast between Silicon Valley and other technological breakthroughs is startling. It was, after all, the nuclear scientists who alerted us to the dangers of nuclear energy, just as it was the biologists who raised the alarm about recombinant DNA. We’ve proved time and time again that technology can be contained and its dangers mitigated.
Yet with massive profits at stake, Silicon Valley executives have shown that they are unwilling to do the same.
It’s The Institutions, Stupid
In 1945, Vannevar Bush published a long essay in The Atlantic entitled As We May Think, which envisioned a “memex,” a machine that sounded strikingly like the internet of today. He wrote:
“Consider a future device … in which an individual stores all his books, records, and communications, and which is mechanized so that it may be consulted with exceeding speed and flexibility. It is an enlarged intimate supplement to his memory.”
Yet in envisioning the future he saw both possibility and peril. He predicted much of what we use the Internet for today, including doctors being able to track down symptoms of obscure cases and lawyers being able to quickly retrieve relevant case law. Yet he also foresaw much of what we struggle with, such as information overload and the use of technology for war.
Bush was, at the time, a figure something akin to Elon Musk, but if anything more prominent. An engineer of the highest order, he invented a proto-computer at MIT. He also co-founded the company Raytheon, oversaw the U.S. government scientific programs during World War II, including the development of the atomic bomb, radar, and penicillin.
Yet probably more than anything else, he was a master at designing institutions. When the war was winding down, President Roosevelt asked him to deliver a report about how to continue America’s scientific prowess. That report, Science, The Endless Frontier, delivered to President Truman in 1945, laid out the basic architecture of programs, such as the National Science Foundation, that would transform the US into a technological superpower.
To paraphrase James Carville, it’s the institutions, stupid. If we are going to seize the promise of AI and other cutting-edge areas such as quantum computing and synthetic biology, while minimizing the peril, we need structures to organize our collective will for the common good, or we will end up subjugating our will to the technologies we fail to govern.
The choices made by those who came before shaped the world we live in today. The choices we make now will shape the world we leave behind.
