Content deleted Content added
m adds link |
Rescuing 11 sources and tagging 0 as dead.) #IABot (v2.0.9.3 |
||
Line 13:
== Risks of unfriendly AI ==
{{main|Existential risk from artificial general intelligence}}
The roots of concern about artificial intelligence are very old. Kevin LaGrandeur showed that the dangers specific to AI can be seen in ancient literature concerning artificial humanoid servants such as the [[golem]], or the proto-robots of [[Gerbert of Aurillac]] and [[Roger Bacon]]. In those stories, the extreme intelligence and power of these humanoid creations clash with their status as slaves (which by nature are seen as sub-human), and cause disastrous conflict.<ref>{{cite journal|url=https://www.academia.edu/704751|author=Kevin LaGrandeur|title=The Persistent Peril of the Artificial Slave|journal=Science Fiction Studies|year=2011|volume=38|issue=2|page=232|doi=10.5621/sciefictstud.38.2.0232|access-date
In modern times as the prospect of [[Superintelligence|superintelligent AI]] looms nearer, philosopher [[Nick Bostrom]] has said that superintelligent AI systems with goals that are not aligned with human ethics are intrinsically dangerous unless extreme measures are taken to ensure the safety of humanity. He put it this way:
Line 19:
<blockquote>Basically we should assume that a 'superintelligence' would be able to achieve whatever goals it has. Therefore, it is extremely important that the goals we endow it with, and its entire motivation system, is 'human friendly.'</blockquote>
In 2008 Eliezer Yudkowsky called for the creation of "friendly AI" to mitigate [[existential risk from advanced artificial intelligence]]. He explains: "The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else."<ref>{{cite book |author=[[Eliezer Yudkowsky]] |year=2008 |chapter-url=http://intelligence.org/files/AIPosNegFactor.pdf |chapter=Artificial Intelligence as a Positive and Negative Factor in Global Risk |title=Global Catastrophic Risks |pages=308–345 |editor1=Nick Bostrom |editor2=Milan M. Ćirković |access-date=2013-10-19 |archive-date=2013-10-19 |archive-url=https://web.archive.org/web/20131019182403/http://intelligence.org/files/AIPosNegFactor.pdf |url-status=live }}</ref>
[[Steve Omohundro]] says that a sufficiently advanced AI system will, unless explicitly counteracted, exhibit a number of [[Instrumental convergence#Basic AI drives|basic "drives"]], such as resource acquisition, self-preservation, and continuous self-improvement, because of the intrinsic nature of any goal-driven systems and that these drives will, "without special precautions", cause the AI to exhibit undesired behavior.<ref>{{cite journal |last=Omohundro |first=S. M. |date=February 2008 |title=The basic AI drives |journal=Artificial General Intelligence |volume=171 |pages=483–492 |citeseerx=10.1.1.393.8356}}</ref><ref>{{cite book|last1=Bostrom|first1=Nick|title=Superintelligence: Paths, Dangers, Strategies|date=2014|publisher=Oxford University Press|___location=Oxford|isbn=9780199678112|title-link=Superintelligence: Paths, Dangers, Strategies |chapter=Chapter 7: The Superintelligent Will}}</ref>
[[Alexander Wissner-Gross]] says that AIs driven to maximize their future freedom of action (or causal path entropy) might be considered friendly if their planning horizon is longer than a certain threshold, and unfriendly if their planning horizon is shorter than that threshold.<ref>{{cite web | last=Dvorsky | first=George | title=How Skynet Might Emerge From Simple Physics | website=Gizmodo | date=2013-04-26 | url=https://gizmodo.com/how-skynet-might-emerge-from-simple-physics-482402911 | access-date=2021-12-23 | archive-date=2021-10-08 | archive-url=https://web.archive.org/web/20211008105300/https://gizmodo.com/how-skynet-might-emerge-from-simple-physics-482402911 | url-status=live }}</ref><ref>{{cite journal | last1 = Wissner-Gross | first1 = A. D. | author-link1 = Alexander Wissner-Gross | last2 = Freer | first2 = C. E. | author-link2 = Cameron Freer | year = 2013 | title = Causal entropic forces | journal = Physical Review Letters | volume = 110 | issue = 16 | page = 168702 | doi = 10.1103/PhysRevLett.110.168702 | pmid = 23679649 | bibcode = 2013PhRvL.110p8702W | doi-access = free }}</ref>
Luke Muehlhauser, writing for the [[Machine Intelligence Research Institute]], recommends that [[machine ethics]] researchers adopt what [[Bruce Schneier]] has called the "security mindset": Rather than thinking about how a system will work, imagine how it could fail. For instance, he suggests even an AI that only makes accurate predictions and communicates via a text interface might cause unintended harm.<ref name=MuehlhauserSecurity2013>{{cite web|last1=Muehlhauser|first1=Luke|title=AI Risk and the Security Mindset|url=http://intelligence.org/2013/07/31/ai-risk-and-the-security-mindset/|website=Machine Intelligence Research Institute|access-date=15 July 2014|date=31 Jul 2013|archive-date=19 July 2014|archive-url=https://web.archive.org/web/20140719205835/http://intelligence.org/2013/07/31/ai-risk-and-the-security-mindset/|url-status=live}}</ref>
In 2014, Luke Muehlhauser and Nick Bostrom underlined the need for 'friendly AI';<ref name=think13>{{Cite journal|last1=Muehlhauser|first1=Luke|last2=Bostrom|first2=Nick|title=Why We Need Friendly AI|date=2013-12-17|journal=Think|volume=13|issue=36|pages=41–47|doi=10.1017/s1477175613000316|s2cid=143657841|issn=1477-1756}}</ref> nonetheless, the difficulties in designing a 'friendly' superintelligence, for instance via programming counterfactual moral thinking, are considerable.<ref name=boyles2019>{{Cite journal|last1=Boyles|first1=Robert James M.|last2=Joaquin|first2=Jeremiah Joven|date=2019-07-23|title=Why friendly AIs won't be that friendly: a friendly reply to Muehlhauser and Bostrom|journal=AI & Society|volume=35|issue=2|pages=505–507|doi=10.1007/s00146-019-00903-0|s2cid=198190745|issn=0951-5666}}</ref><ref>{{Cite journal|last=Chan|first=Berman|date=2020-03-04|title=The rise of artificial intelligence and the crisis of moral passivity|journal=AI & Society|volume=35|issue=4|pages=991–993|language=en|doi=10.1007/s00146-020-00953-9|s2cid=212407078|issn=1435-5655|url=https://philpapers.org/rec/CHATRO-56
==Coherent extrapolated volition==
Yudkowsky advances the Coherent Extrapolated Volition (CEV) model. According to him, coherent extrapolated volition is people's choices and the actions people would collectively take if "we knew more, thought faster, were more the people we wished we were, and had grown up closer together."<ref name=cevpaper />
Rather than a Friendly AI being designed directly by human programmers, it is to be designed by a "seed AI" programmed to first study [[human nature]] and then produce the AI which humanity would want, given sufficient time and insight, to arrive at a satisfactory answer.<ref name=cevpaper>{{cite web |url=https://intelligence.org/files/CEV.pdf |title=Coherent Extrapolated Volition |publisher=Singularity Institute for Artificial Intelligence |year=2004 |access-date=2015-09-12 |author=Eliezer Yudkowsky |archive-date=2015-09-30 |archive-url=https://web.archive.org/web/20150930035316/http://intelligence.org/files/CEV.pdf |url-status=live }}</ref> The appeal to an [[evolutionary psychology|objective through contingent human nature]] (perhaps expressed, for mathematical purposes, in the form of a [[utility function]] or other [[decision theory|decision-theoretic]] formalism), as providing the ultimate criterion of "Friendliness", is an answer to the [[metaethics|meta-ethical]] problem of defining an [[moral universalism|objective morality]]; extrapolated volition is intended to be what humanity objectively would want, all things considered, but it can only be defined relative to the psychological and cognitive qualities of present-day, unextrapolated humanity.
==Other approaches==
{{See also|AI control problem#Alignment|AI safety}}
[[Steve Omohundro]] has proposed a "scaffolding" approach to [[AI safety]], in which one provably safe AI generation helps build the next provably safe generation.<ref name=Hendry2014>{{cite news|last1=Hendry|first1=Erica R.|title=What Happens When Artificial Intelligence Turns On Us?|url=http://www.smithsonianmag.com/innovation/what-happens-when-artificial-intelligence-turns-us-180949415/|access-date=15 July 2014|work=Smithsonian Magazine|date=21 Jan 2014|archive-date=19 July 2014|archive-url=https://web.archive.org/web/20140719142131/http://www.smithsonianmag.com/innovation/what-happens-when-artificial-intelligence-turns-us-180949415/|url-status=live}}</ref>
[[Seth Baum]] argues that the development of safe, socially beneficial artificial intelligence or artificial general intelligence is a function of the social psychology of AI research communities, and so can be constrained by extrinsic measures and motivated by intrinsic measures. Intrinsic motivations can be strengthened when messages resonate with AI developers; Baum argues that, in contrast, "existing messages about beneficial AI are not always framed well". Baum advocates for "cooperative relationships, and positive framing of AI researchers" and cautions against characterizing AI researchers as "not want(ing) to pursue beneficial designs".<ref>{{Cite journal|last=Baum|first=Seth D.|date=2016-09-28|title=On the promotion of safe and socially beneficial artificial intelligence|journal=AI & Society|volume=32|issue=4|pages=543–551|doi=10.1007/s00146-016-0677-0|s2cid=29012168|issn=0951-5666}}</ref>
Line 53:
[[James Barrat]], author of ''[[Our Final Invention]]'', suggested that "a public-private partnership has to be created to bring A.I.-makers together to share ideas about security—something like the International Atomic Energy Agency, but in partnership with corporations." He urges AI researchers to convene a meeting similar to the [[Asilomar Conference on Recombinant DNA]], which discussed risks of biotechnology.<ref name=Hendry2014 />
[[John McGinnis]] encourages governments to accelerate friendly AI research. Because the goalposts of friendly AI are not necessarily eminent, he suggests a model similar to the [[National Institutes of Health]], where "Peer review panels of computer and cognitive scientists would sift through projects and choose those that are designed both to advance AI and assure that such advances would be accompanied by appropriate safeguards." McGinnis feels that peer review is better "than regulation to address technical issues that are not possible to capture through bureaucratic mandates". McGinnis notes that his proposal stands in contrast to that of the [[Machine Intelligence Research Institute]], which generally aims to avoid government involvement in friendly AI.<ref name=McGinnis2010>{{cite journal|last1=McGinnis|first1=John O.|title=Accelerating AI|journal=Northwestern University Law Review|date=Summer 2010|volume=104|issue=3|pages=1253–1270|url=http://www.law.northwestern.edu/LAWREVIEW/Colloquy/2010/12/|access-date=16 July 2014|archive-date=1 December 2014|archive-url=https://web.archive.org/web/20141201201600/http://www.law.northwestern.edu/LAWREVIEW/Colloquy/2010/12/|url-status=live}}</ref>
According to [[Gary Marcus]], the annual amount of money being spent on developing machine morality is tiny.<ref>{{cite magazine|last1=Marcus|first1=Gary|title=Moral Machines|url=http://www.newyorker.com/news/news-desk/moral-machines|access-date=30 July 2014|magazine=[[The New Yorker]]|date=24 November 2012|archive-date=8 August 2014|archive-url=https://web.archive.org/web/20140808031914/http://www.newyorker.com/news/news-desk/moral-machines|url-status=live}}</ref>
==Criticism==
{{see also|Technological singularity#Criticisms}}
Some critics believe that both human-level AI and superintelligence are unlikely, and that therefore friendly AI is unlikely. Writing in ''[[The Guardian]]'', Alan Winfield compares human-level artificial intelligence with faster-than-light travel in terms of difficulty, and states that while we need to be "cautious and prepared" given the stakes involved, we "don't need to be obsessing" about the risks of superintelligence.<ref>{{cite news|last1=Winfield|first1=Alan|title=Artificial intelligence will not turn into a Frankenstein's monster|url=https://www.theguardian.com/technology/2014/aug/10/artificial-intelligence-will-not-become-a-frankensteins-monster-ian-winfield|access-date=17 September 2014|work=[[The Guardian]]
Some philosophers claim that any truly "rational" agent, whether artificial or human, will naturally be benevolent; in this view, deliberate safeguards designed to produce a friendly AI could be unnecessary or even harmful.<ref>{{cite journal | last=Kornai | first=András | title=Bounding the impact of AGI | journal=Journal of Experimental & Theoretical Artificial Intelligence | publisher=Informa UK Limited | volume=26 | issue=3 | date=2014-05-15 | issn=0952-813X | doi=10.1080/0952813x.2014.895109 | pages=417–438 | s2cid=7067517 |quote=...the essence of AGIs is their reasoning facilities, and it is the very logic of their being that will compel them to behave in a moral fashion... The real nightmare scenario (is one where) humans find it advantageous to strongly couple themselves to AGIs, with no guarantees against self-deception.}}</ref> Other critics question whether it is possible for an artificial intelligence to be friendly. Adam Keiper and Ari N. Schulman, editors of the technology journal ''[[The New Atlantis (journal)|The New Atlantis]]'', say that it will be impossible to ever guarantee "friendly" behavior in AIs because problems of ethical complexity will not yield to software advances or increases in computing power. They write that the criteria upon which friendly AI theories are based work "only when one has not only great powers of prediction about the likelihood of myriad possible outcomes, but certainty and consensus on how one values the different outcomes.<ref>{{cite magazine |url=http://www.thenewatlantis.com/publications/the-problem-with-friendly-artificial-intelligence |first1=Adam |last1=Keiper |first2=Ari N. |last2=Schulman |title=The Problem with 'Friendly' Artificial Intelligence |journal=The New Atlantis |number=32 |date=Summer 2011 |page=80-89 |access-date
==See also==
| |||