Why we don't really need to build AGI
Is AGI the most important thing we could be working on?
In this essay, I intend to thoroughly explore the unfolding debates surrounding the pursuit of artificial general intelligence (AGI) – AI systems envisioned to possess generalized cognitive abilities on par with humans.
I have been deeply intrigued by the topic of AGI since my adolescent years, with this longstanding interest recently reinvigorated upon reading a thoughtful essay by my friend
who writes the Absent Minded Professor Substack.He raised probing questions about the motivations behind AGI research that inspired me to crystallize my perspective. I aim to share views on both the considerable promise and potential perils of AGI, proposing what I believe could be a prudent path forward.
Defining the Concept of AGI
First, what exactly is AGI? Also referred to as strong AI, AGI refers to AI systems exhibiting expansive capacities for learning, reasoning, creativity, and autonomous function comparable to the full range of human cognition. Rather than specializing in narrow domains like chess or image recognition, AGI aspires toward flexible, human-level intelligence that could grasp virtually any intellectual challenge within the scope of the human mind.
AGI is envisioned as artificial yet general cognition - able to transfer learning and skills between contexts fluidly like humans. Proposed AGI capabilities range from scientific discovery to medical diagnosis to social interaction to artistic creation and beyond. AGI represents a grand vision of AI’s fullest possibilities.
The Hopes Surrounding AGI
Proponents of AGI argue it represents a necessary milestone, perhaps even a pivotal inflexion point, for humanity to overcome systemic challenges holding civilization back. By profoundly enhancing human capabilities and automating routine cognitive work, AGI could catalyze transformative leaps in science, medicine, education, sustainability, space exploration and more.
Some view human-level AGI as the essential missing piece needed to create an abundant, flourishing future for humanity, envisioning Star Trek-like futures where society enjoys universal prosperity. Powerful technologies inevitably carry immense consequences, both beneficial and hazardous. The arc of history curves with their emergence. There is no denying AGI’s momentous potential if guided prudently.
Causes For Caution Around Unfettered AGI Pursuit
However, in our understandable eagerness to realize AGI’s remarkable possibilities, we must also frankly confront the risks of rushing into irresponsible development and deployment of advanced AI. The very capabilities making AGI so prized also lend it incredible capacity for unintended consequences or deliberate misuse if handled without ethics, care, and wisdom.
History has shown time and again that powerful technologies often radically outpace humanity’s capacity to foresee and safeguard against their hazards. Just consider disasters like leaded gasoline, CFCs, and thalidomide. AGI would represent the most impactful technology in human history to date(on the same level as electricity, fire, and petroleum) - we cannot afford to repeat past mistakes of reckless adoption.
While resonating with the grand vision, I believe the most prudent near-term approach is focusing efforts on specialized narrow AI solutions tailored to defined real-world problems. Systems like DeepMind’s AlphaFold, wholly dedicated to the singular challenge of protein-folding, can already drive meaningful progress in their domains even if less revolutionary than AGI dreams. Targeted AI may deliver tangible benefits years before AGI’s uncertainties resolve.
Additionally, we must seriously contemplate our societal preparedness for AGI's disruptions. My brother’s computer science teacher believing today's AI equals AGI exemplifies the lack of accurate public understanding about existing capabilities. How can we responsibly manage technologies we scarcely comprehend? We would be reckless to blithely unleash something as impactful as AGI without profound philosophical preparation.
Moreover, today’s AI systems remain remarkably fragile and limited compared to human cognition despite impressive advances. Further paradigm-shifting discoveries (on the level of Reinforcement Learning with Human Feedback) may be essential before AGI is technically feasible to ensure alignment with ethics and human values. Overpromising near-term AGI timelines risks disillusionment and loss of momentum. A patient, incremental approach is most prudent.
A Balanced Path Forward
To conclude, while AGI’s possibilities are profound, we must pursue them with nuance, wisdom and care. I am not stating we should abandon efforts to build AGI altogether. However, we need a practical, measured approach given the immense risks and uncertainties involved. A prudent way forward would be to shift more foundational AGI research into academic environments, while the industry focuses on specialized AI applications.
Academia offers a fertile incubator for nurturing risky, speculative AGI research. Without profit-driven pressures, academics have the creative freedom to patiently work on high-risk, high-reward ideas. They can devote time to deeply exploring the philosophical foundations and ethical alignment for advanced AI. Even if progress is slower than commercially driven AGI research, critical groundwork can be laid.
Universities also attract brilliant, unconventional thinkers who relish working on open-ended theoretical problems regardless of commercial viability. The interdisciplinary nature of academia fosters the exchange of diverse viewpoints on challenging issues like AI safety and control. Allowing AGI research to percolate here organically, while the industry solves near-term issues, balances society's short and long-term needs.
Additionally, the peer review system in academia provides checks and balances for vetting AGI ideas rigorously. Commercial AGI research may privilege speed and secrecy over academic discourse and transparency. But given AGI's profound implications, informed debate should guide its development. A vibrant academic ecosystem keeps AGI rooted in ethics, not just profit incentives.
In essence, universities provide the ideal incubator for nurturing AGI research as an intellectual endeavour and pillar of human progress, not just a commercial race. This environment cultivates the wisdom and care needed to craft beneficial AGI while addressing society's immediate challenges.
Please check out Josh’s essay on this topic, it serves as the foundation of this piece.
Well done, Edem - and thanks, Josh, for the thoughtful contribution to this conversation.
I'm glad to hear the role of academia highlighted. I think we've done our very best work as humans when we've been able to tap into all three arenas: universities, government, and industry. If we can figure out how to use all three effectively and in tandem, we're going to have something unimaginably powerful, and probably closer to unambiguously good than if we just let one stay in the driver's seat for too long.
Academia's role is definitely to explore those outer limits, and that's sensible, but I think back to the last revolution like this- that of the transistor, and how industry became the academic center (I'm talking about Bell Labs specifically here), and how the most powerful minds were actually working on this, there, and not in a university at all.
Thanks Edem, glad you found the piece helpful!