Eliezer Shlomo Yudkowsky (born September 11, 1979) is an American AI researcher and writer best known for popularising the idea of friendly artificial intelligence. He is a co-founder and research fellow at the Machine Intelligence Research Institute (MIRI), a private research nonprofit based in Berkeley, California. He has no formal education, never having attended high school or college. His work on the prospect of a runaway intelligence explosion was an influence on Nick Bostrom's Superintelligence: Paths, Dangers, Strategies.
Video Eliezer Yudkowsky
Work in artificial intelligence safety
Goal learning and incentives in software systems
Yudkowsky's views on the safety challenges posed by future generations of AI systems are discussed in the undergraduate textbook in AI, Stuart Russell and Peter Norvig's Artificial Intelligence: A Modern Approach. Noting the difficulty of formally specifying general-purpose goals by hand, Russell and Norvig cite Yudkowsky's proposal that autonomous and adaptive systems be designed to learn correct behavior over time:
Yudkowsky (2008) goes into more detail about how to design a Friendly AI. He asserts that friendliness (a desire not to harm humans) should be designed in from the start, but that the designers should recognize both that their own designs may be flawed, and that the robot will learn and evolve over time. Thus the challenge is one of mechanism design - to design a mechanism for evolving AI under a system of checks and balances, and to give the systems utility functions that will remain friendly in the face of such changes.
Citing Steve Omohundro's idea of instrumental convergence, Russell and Norvig caution that autonomous decision-making systems with poorly designed goals would have default incentives to treat humans adversarially, or as dispensable resources, unless specifically designed to counter such incentives: "even if you only want your program to play chess or prove theorems, if you give it the capability to learn and alter itself, you need safeguards".
In response to the instrumental convergence concern, Yudkowsky and other MIRI researchers have recommended that work be done to specify software agents that converge on safe default behaviors even when their goals are misspecified. The Future of Life Institute (FLI) summarizes this research program in the Open Letter on Artificial Intelligence research priorities document:
If an AI system is selecting the actions that best allow it to complete a given task, then avoiding conditions that prevent the system from continuing to pursue the task is a natural subgoal (and conversely, seeking unconstrained situations is sometimes a useful heuristic). This could become problematic, however, if we wish to repurpose the system, to deactivate it, or to significantly alter its decision-making process; such a system would rationally avoid these changes. Systems that do not exhibit these behaviours have been termed corrigible systems, and both theoretical and practical work in this area appears tractable and useful. For example, it may be possible to design utility functions or decision processes so that a system will not try to avoid being shut down or repurposed, and theoretical frameworks could be developed to better understand the space of potential systems that avoid undesirable behaviors.
Yudkowsky argues that as AI systems become increasingly intelligent, new formal tools will be needed in order to avert default incentives for harmful behavior, as well as to inductively teach correct behavior. These lines of research are discussed in MIRI's 2015 technical agenda.
System reliability and transparency
Yudkowsky studies decision theories that achieve better outcomes than causal decision theory in Newcomblike problems. This includes decision procedures that allow agents to cooperate with equivalent reasoners in the one-shot prisoner's dilemma. Yudkowsky has also written on theoretical prerequisites for self-verifying software.
Yudkowsky argues that it is important for advanced AI systems to be cleanly designed and transparent to human inspection, both to ensure stable behavior and to allow greater human oversight and analysis. Citing papers on this topic by Yudkowsky and other MIRI researchers, the FLI research priorities document states that work on defining correct reasoning in embodied and logically non-omniscient agents would be valuable for the design, use, and oversight of AI agents.
Capabilities forecasting
In their discussion of Omohundro and Yudkowsky's work, Russell and Norvig cite I. J. Good's 1965 prediction that when computer systems begin to outperform humans in software engineering tasks, this may result in a feedback loop of increasingly capable AI systems. This raises the possibility that AI's impact could increase very quickly after it reaches a certain level of capability.
In the intelligence explosion scenario inspired by Good's hypothetical, recursively self-improving AI systems quickly transition from subhuman general intelligence to superintelligent. Nick Bostrom's 2014 book Superintelligence: Paths, Dangers, Strategies sketches out Good's argument in greater detail, while making a broader case for expecting AI systems to eventually outperform humans across the board. Bostrom cites writing by Yudkowsky on inductive value learning and on the risk of anthropomorphizing advanced AI systems, e.g.: "AI might make an apparently sharp jump in intelligence purely as the result of anthropomorphism, the human tendency to think of 'village idiot' and 'Einstein' as the extreme ends of the intelligence scale, instead of nearly indistinguishable points on the scale of minds-in-general."
The Open Philanthropy Project, an offshoot of the charity evaluator GiveWell, credits Yudkowsky and Bostrom with several (paraphrased) arguments for expecting future AI advances to have a large societal impact:
Over a relatively short geological timescale, humans have come to have enormous impacts on the biosphere, often leaving the welfare of other species dependent on the objectives and decisions of humans. It seems plausible that the intellectual advantages humans have over other animals have been crucial in allowing humans to build up the scientific and technological capabilities that have made this possible. If advanced artificial intelligence agents become significantly more powerful than humans, it seems possible that they could become the dominant force in the biosphere, leaving humans' welfare dependent on their objectives and decisions. As with the interaction between humans and other species in the natural environment, these problems could be the result of competition for resources rather than malice.
In comparison with other evolutionary changes, there was relatively little time between our hominid ancestors and the evolution of humans. There was therefore relatively little time for evolutionary pressure to lead to improvements in human intelligence relative to the intelligence of our hominid ancestors, suggesting that the increases in intelligence may be small on some absolute scale. [...T]his makes it seem plausible that creating intelligent agents that are more intelligent than humans could have dramatic real-world consequences even if the difference in intelligence is small in an absolute sense.
Russell and Norvig raise the objection that there are known limits to intelligent problem-solving from computational complexity theory; if there are strong limits on how efficiently algorithms can solve various computer science tasks, then intelligence explosion may not be possible. Yudkowsky has debated the likelihood of intelligence explosion with economist Robin Hanson, who argues that AI progress is likely to accelerate over time, but is not likely to be localized or discontinuous.
Maps Eliezer Yudkowsky
Rationality writing
Between 2006 and 2009, Yudkowsky and Robin Hanson were the principal contributors to Overcoming Bias, a cognitive and social science blog sponsored by the Future of Humanity Institute of Oxford University. In February 2009, Yudkowsky founded LessWrong, a "community blog devoted to refining the art of human rationality". Overcoming Bias has since functioned as Hanson's personal blog. LessWrong has been covered in depth in Business Insider.
Yudkowsky has also written several works of fiction. His fanfiction story, Harry Potter and the Methods of Rationality, uses plot elements from J.K. Rowling's Harry Potter series to illustrate topics in science. The New Yorker describes Harry Potter and the Methods of Rationality as a retelling of Rowling's original "in an attempt to explain Harry's wizardry through the scientific method".
Over 300 blogposts by Yudkowsky have been released as six books, collected in a single ebook titled Rationality: From AI to Zombies by the Machine Intelligence Research Institute in 2015. His latest ebook is titled Inadequate Equilibria: Where and How Civilizations Get Stuck.
Personal views
Yudkowsky identifies as an atheist and a "small-l libertarian." He supports cryonics and is signed up with the Cryonics Institute.
Family
His younger brother, Yehuda Nattan Yudkowsky, died in 2004 at the age of nineteen.
He had an open marriage with Brienne Yudkowsky, whom he married in 2013. They have no children, and reside in Berkeley, California, in the San Francisco Bay Area.. He announced his divorce on 31st March 2018 in a Facebook post
Academic publications
- Yudkowsky, Eliezer (2007). "Levels of Organization in General Intelligence" (PDF). Artificial General Intelligence. Berlin: Springer.
- Yudkowsky, Eliezer (2008). "Cognitive Biases Potentially Affecting Judgement of Global Risks" (PDF). In Bostrom, Nick; ?irkovi?, Milan. Global Catastrophic Risks. Oxford University Press. ISBN 978-0199606504.
- Yudkowsky, Eliezer (2008). "Artificial Intelligence as a Positive and Negative Factor in Global Risk" (PDF). In Bostrom, Nick; ?irkovi?, Milan. Global Catastrophic Risks. Oxford University Press. ISBN 978-0199606504.
- Yudkowsky, Eliezer (2011). "Complex Value Systems in Friendly AI" (PDF). Artificial General Intelligence: 4th International Conference, AGI 2011, Mountain View, CA, USA, August 3-6, 2011. Berlin: Springer.
- Yudkowsky, Eliezer (2012). "Friendly Artificial Intelligence". In Eden, Ammon; Moor, James; Søraker, John; et al. Singularity Hypotheses: A Scientific and Philosophical Assessment. Berlin: Springer. ISBN 978-3-642-32559-5.
- Bostrom, Nick; Yudkowsky, Eliezer (2014). "The Ethics of Artificial Intelligence" (PDF). In Frankish, Keith; Ramsey, William. The Cambridge Handbook of Artificial Intelligence. New York: Cambridge University Press. ISBN 978-0-521-87142-6.
- LaVictoire, Patrick; Fallenstein, Benja; Yudkowsky, Eliezer; Bárász, Mihály; Christiano, Paul; Herreshoff, Marcello (2014). "Program Equilibrium in the Prisoner's Dilemma via Löb's Theorem". Multiagent Interaction without Prior Coordination: Papers from the AAAI-14 Workshop. AAAI Publications.
- Soares, Nate; Fallenstein, Benja; Yudkowsky, Eliezer (2015). "Corrigibility". AAAI Workshops: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence, Austin, TX, January 25-26, 2015. AAAI Publications.
See also
- AI box
- Friendly artificial intelligence
- Less Wrong
- Open Letter on Artificial Intelligence
References
External links
- Official website
Source of the article : Wikipedia
0 Comments