Human-based computation: Difference between revisions

Content deleted Content added
Now, I'll delete the superfluous note
Cleanup cite template: extracted original URL from 5 refs archived at web.archive.org
Line 10:
A concept of the automatic [[Turing test]] pioneered by [[Moni Naor]] (1996)<ref>{{cite web |url=http://www.wisdom.weizmann.ac.il/~naor/PAPERS/human_abs.html |author=Naor, Moni |title=Verification of a human in the loop or Identification via the Turing Test |access-date=12 May 2021}}</ref> is another precursor of human-based computation. In Naor's test, the machine can control the access of humans and computers to a service by challenging them with a [[natural language processing]] (NLP) or [[computer vision]] (CV) problem to identify humans among them. The set of problems is chosen in a way that they have no algorithmic solution that is both effective and efficient at the moment. If it existed, such an algorithm could be easily performed by a computer, thus defeating the test. In fact, Moni Naor was modest by calling this an automated Turing test. The [[imitation game]] described by [[Alan Turing]] (1950) didn't propose using CV problems. It was only proposing a specific NLP task, while the Naor test identifies and explores a large [[AI-complete|class]] of problems, not necessarily from the ___domain of NLP, that could be used for the same purpose in both automated and non-automated versions of the test.
 
Finally, [[Human-based genetic algorithm]] (HBGA)<ref>{{cite web |url=https://ieeexplore.ieee.org/document/972056 |title=Human-based genetic algorithm |access-date=12 May 2022}}</ref> encourages human participation in multiple different roles. Humans are not limited to the role of evaluator or some other predefined role, but can choose to perform a more diverse set of tasks. In particular, they can contribute their innovative solutions into the evolutionary process, make incremental changes to existing solutions, and perform intelligent recombination.<ref>{{cite web |url=http://gpbib.cs.ucl.ac.uk/gecco2005lbp/papers/56-hammond.pdf |author=Hammond, Michelle O.; and Terence C. Fogarty|title=Co-operative OuLiPian (Ouvroir de littérature potentielle) Generative Literature Using Human-Based Evolutionary Computing |access-date=12 May 2022}}</ref> In short, HBGA allows humans to participate in all operations of a typical [[genetic algorithm]]. As a result of this, HBGA can process solutions for which there are no computational innovation operators available, for example, natural languages. Thus, HBGA obviated the need for a fixed representational scheme that was a limiting factor of both standard and interactive EC.<ref>{{cite web |url=https://ieeexplore.ieee.org/document/949485 |title=Interactive evolutionary computation: fusion of the capabilities of EC optimization and human evaluation, pp.&nbsp;1275-1296 |author=Takagi, Hideyuki |access-date=12 May 2022}}</ref> These algorithms can also be viewed as novel forms of social organization coordinated by a computer, according to Alex Kosorukoff and David Goldberg.<ref>{{cite web |url=https://web.archive.org/web/20110707063732/http://research.3form.com/alex/pub/gecco-2002-18.pdf |title=Evolutionary Computation as a Form of Organization, pp.&nbsp;965-972 |access-date=12 May 2022 |archive-url=https://web.archive.org/web/20110707063732/http://research.3form.com/alex/pub/gecco-2002-18.pdf |archive-date=7 July 2011 |url-status=dead}}</ref>
 
==Classes of human-based computation==
Line 43:
* (HH<sub>1</sub>) [[Social search]] applications accept contributions from users and attempt to use human evaluation to select the fittest contributions that get to the top of the list. These use one type of human-based innovation. Early work was done in the context of HBGA. [[Digg]] and [[Reddit]] are recently popular examples. See also [[Collaborative filtering]].
* (HC) Computerized tests. A computer generates a problem and presents it to evaluate a user. For example, [[CAPTCHA]] tells human users from computer programs by presenting a problem that is supposedly easy for a human and difficult for a computer. While CAPTCHAs are effective security measures for preventing automated abuse of online services, the human effort spent solving them is otherwise wasted. The [[reCAPTCHA]] system makes use of these human cycles to help digitize books by presenting words from scanned old books that optical character recognition cannot decipher.<ref>{{cite web |url=https://www.cs.cmu.edu/~biglou/reCAPTCHA_Science.pdf |title=reCAPTCHA: Human-Based Character Recognition via Web Security Measures |author=von Ahn, Luis; Benjamin Maurer, Colin McMillen, David Abraham, and Manuel Blum |access-date=12 May 2022}}</ref>
* (HC) Interactive online games: These are programs that extract knowledge from people in an entertaining way.<ref>{{cite web |url=https://web.archive.org/web/20000229091147/http://www.20q.net/index.html |author=Burgener, Robin |title=20Q . net. Twenty Questions. The neural-net on the Internet. Play Twenty Questions |access-date=12 May 2022 |archive-url=https://web.archive.org/web/20000229091147/http://www.20q.net/index.html |archive-date=29 February 2000 |url-status=dead}}</ref><ref>{{cite web |url=https://www.cs.cmu.edu/~biglou/ESP.pdf |title=Labeling Images with a Computer Game |author=von Ahn, Luis, and Laura Dabbish |access-date=12 May 2022}}</ref><ref>{{cite web |url=https://www.cs.cmu.edu/~biglou/Verbosity.pdf |title=Verbosity: A Game for Collecting Common-Sense Facts |author=von Ahn, Luis; Mihir Kedia, and Manuel Blum |access-date=12 May 2022}}</ref><ref>{{cite web |url=https://www.cs.cmu.edu/~biglou/Phetch.pdf |title=Improving Accessibility of the Web with a Computer Game |author=von Ahn, Luis; Shiri Ginosar, Mihir Kedia, Ruoran Liu, and Manuel Blum |access-date=12 May 2022}}</ref><ref>{{cite web |url=https://patft.uspto.gov/netacgi/nph-Parser?Sect1=PTO1&Sect2=HITOFF&d=PALL&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.htm&r=1&f=G&l=50&s1=7980953.PN.&OS=PN/7980953&RS=PN/7980953 |title=Method for labeling images through a computer game |author=von Ahn, Luis |date=19 July 2011 |access-date=12 May 2022}}{{US patent|7980953}}</ref>
* (HC) "Human Swarming" or "Social Swarming". The UNU platform for human swarming establishes real-time closed-loop systems around groups of networked users molded after biological swarms, enabling human participants to behave as a unified [[collective intelligence]].<ref>{{cite web |url=http://sites.lsa.umich.edu/collectiveintelligence/wp-content/uploads/sites/176/2015/05/Rosenberg-CI-2015-Abstract.pdf |title=Human Swarms: a real-time paradigm for Collective intelligence |author=Rosenberg, Louis B.|access-date=12 May 2021}}</ref><ref>http://sites.lsa.umich.edu/collectiveintelligence/wp-content/uploads/sites/176/2015/05/Rosenberg-CI-2015-Abstract.pdf {{Bare URL PDF |date=March 2022}}</ref><ref>{{cite web |url=https://web.archive.org/web/20151027132802/https://mitpress.mit.edu/sites/default/files/titles/content/ecal2015/ch117.html |title=Swarms: a real-time paradigm for Collective intelligence |access-date=12 May 2022 |archive-url=https://web.archive.org/web/20151027132802/https://mitpress.mit.edu/sites/default/files/titles/content/ecal2015/ch117.html |archive-date=27 October 2015 |url-status=dead}}</ref><ref>{{cite web |url=https://papers.ssrn.com/sol3/papers.cfm?abstract_id=924249 |title=Infotopia: How Many Minds Produce Knowledge |author=Sunstein, Cass R. |access-date=12 May 2022}}</ref><ref>{{cite web |url=https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1381502 |title=Harnessing Crowds: Mapping the Genome of Collective Intelligence |author=Malone, Thomas W.; Robert Laubacher, and Chrysanthos Dellarocas |access-date=12 May 2022}}</ref><ref>{{cite web |url=https://mitpress.mit.edu/sites/default/files/titles/content/ecal2015/ch117.html |title=Archived copy |access-date=October 12, 2015 |archive-url=https://web.archive.org/web/20151027132802/https://mitpress.mit.edu/sites/default/files/titles/content/ecal2015/ch117.html |archive-date=October 27, 2015 |url-status=dead |df=mdy-all }}</ref><ref>http://news.discovery.com/human/life/swarms-of-humans-power-a-i-platform-150603.htm {{Dead link|date=March 2022}}</ref>
* (NHC) Natural Human Computation involves leveraging existing human behavior to extract computationally significant work without disturbing that behavior.<ref>[https://arxiv.org/abs/1306.6376 Estrada, Daniel, and Jonathan Lawhead, "Gaming the Attention Economy" in ''The Springer Handbook of Human Computation'', Pietro Michelucci (ed.), (Springer, 2014)]</ref> NHC is distinguished from other forms of human-based computation in that rather than involving outsourcing computational work to human activity by asking humans to perform novel computational tasks, it involves taking advantage of previously unnoticed computational significance in existing behavior.
 
Line 66:
* Increasing online reputation/recognition
 
Many projects had explored various combinations of these incentives. See more information about motivation of participants in these projects in Kosorukoff,<ref>{{cite web |url=https://web.archive.org/web/20110707063544/http://research.3form.com/alex/pub/classtre.pdf |title=Social classification structures. Optimal decision making in an organization |author=Kosorukoff, Alexander |access-date=12 May 2022 |archive-url=https://web.archive.org/web/20110707063544/http://research.3form.com/alex/pub/classtre.pdf |archive-date=7 July 2011 |url-status=dead}}</ref> and Von Hippel.<ref>{{cite web |url=http://web.mit.edu/evhippel/www/democ1.htm |title=Democratizing Innovation |author=Von Hippel, Eric |access-date=12 May 2022}}</ref>
 
==Human-based computation as a form of social organization==
 
Viewed as a form of social organization, human-based computation often surprisingly turns out to be more robust and productive than traditional organizations.<ref>{{cite web |url=https://web.archive.org/web/20110707063732/http://research.3form.com/alex/pub/gecco-2002-18.pdf |title=Evolutionary Computation as a Form of Organization |author=Kosorukoff, Alexander, and David Goldberg |date=2002 |access-date=12 May 2022 |archive-url=https://web.archive.org/web/20110707063732/http://research.3form.com/alex/pub/gecco-2002-18.pdf |archive-date=7 July 2011 |url-status=dead}}</ref> The latter depend on obligations to maintain their more or less fixed structure, be functional and stable. Each of them is similar to a carefully designed mechanism with humans as its parts. However, this limits the freedom of their human employees and subjects them to various kinds of stresses. Most people, unlike mechanical parts, find it difficult to adapt to some fixed roles that best fit the organization. Evolutionary human-computation projects offer a natural solution to this problem. They adapt organizational structure to human spontaneity, accommodate human mistakes and creativity, and utilize both in a constructive way. This leaves their participants free from obligations without endangering the functionality of the whole, making people happier. There are still some challenging research problems that need to be solved before we can realize the full potential of this idea.
 
The algorithmic outsourcing techniques used in human-based computation are much more scalable than the manual or automated techniques used to manage outsourcing traditionally. It is this scalability that allows to easily distribute the effort among thousands of participants. It was suggested recently that this mass outsourcing is sufficiently different from traditional small-scale outsourcing to merit a new name [[crowdsourcing]].<ref>{{cite web |url=https://www.wired.com/2006/06/crowds/ |title=The Rise of Crowdsourcing |author=Howe, Jeff |date=June 2006|access-date=12 May 2022}}</ref> However, others have argued that crowdsourcing ought to be distinguished from true human-based computation.<ref>{{cite web |url=https://www.springer.com/gp/book/9781461488057 |title=Handbook of Human Computation |author=Michelucci, Pietro |access-date=12 May 2022}}</ref> Crowdsourcing does indeed involve the distribution of computation tasks across a number of human agents, but Michelucci argues that this is not sufficient for it to be considered human computation. Human computation requires not just that a task be distributed across different agents, but also that the set of agents across which the task is distributed be ''mixed:'' some of them must be humans, but others must be traditional computers. It is this mixture of different types of agents in a computational system that gives human-based computation its distinctive character. Some instances of crowdsourcing do indeed meet this criterion, but not all of them do.