Computer game bot Turing test: Difference between revisions

Revision as of 00:53, 10 December 2021

Latest revision as of 03:02, 27 December 2024

(17 intermediate revisions by 10 users not shown)

Line 1:

{{short description|Variant of the Turing test}}

{{primary sources|date=August 2017}}

The '''[[Videocomputer game bot|Computer Game Bot]] Turing Testtest''' is a variant of the [[Turing Testtest]], where a human judge viewing and interacting with a [[virtual world]] must distinguish between other humans and [[video game botsbot]]s, both interacting with the same virtual world. This variant was first proposed in 2008 by Associate Professor Philip Hingston<ref>{{Cite web|url=http://philiphingston.com/Homepage/Homepage.html|title=Philip Hingston &#124; Home}}</ref><ref name="turing">{{Cite journal|last=Hingston | publisherjournal=IEEE Transactions on Computational Intelligence and AI in Games|first=Philip | last=Hingston|title=A Turing Test for Game Bots | date=September 2009|doi=10.1109/TCIAIG.2009.2032534 |s2cid=13988179 |url=http://www.cs.ucf.edu/~gitars/cap6671-2010/Presentations/turing.pdf}}</ref> of [[Edith Cowan University]], and implemented through a tournament called the 2K BotPrize.<ref name="BotPrize">{{Cite web | url=http://botprize.org | title=The 2K Bot Prize : Home | archive-url=https://web.archive.org/web/20080913202526/http://botprize.org/ | archive-date=2008-09-13}}</ref>

[[File:UT^2 bot combats an opponent in the BotPrize.jpg|thumb|alt=The UT^2 bot combats an opponent in the BotPrize.|A bot combats a human opponent in the game ''[[Unreal Tournament 2004]]''.]]

==History==

The Computercomputer Gamegame Botbot Turing Testtest was proposed to advance the fields of [[Artificialartificial Intelligenceintelligence]] (AI) and [[Computationalcomputational Intelligenceintelligence]] with respect to video games. It was considered that a poorly implemented bot implied a subpar game, so a bot that would be capable of passing this test, and therefore might be indistinguishable from a human player, would directly improve the quality of a game. It also served to debunk a flawed notion that "game AI is a solved problem."<ref name="turing"/>

Emphasis is placed on a game bot that interacts with other players in a [[Multiplayer video game|multiplayer]] environment. Unlike a bot that simply needs to make optimal human-like decisions to play or beat a game, this bot must make the same decisions while also convincing another in-game player of its human-likeness.{{cn|date=February 2023}}

==Implementation==

The Computercomputer Gamegame Botbot Turing Testtest was designed to test a bot's ability to interact with a game environment in comparison with a human player,; simply 'winning' was insufficient. This evolved into a contest with a few important goals in mind:<ref name="turing"/>

* There are three participants: a human player, a computer-game bot, and a judge.

Line 18 ⟶ 20:

* Human participants were of a moderate skill range, with no participant either ignorant to the game or capable of playing at a professional level.

In 2008, the first 2K BotPrize tournament took place.<ref name="BotPrize 2008">{{Cite web |url=http://botprize.org/2008.html |title=ArchivedBotprize copy: 2008 |access-date=2013-02-03 |archive-url=https://web.archive.org/web/20130225095014/http://botprize.org/2008.html |archive-date=2013-02-25 |url-status=dead }}</ref> The contest was held with the game ''[[Unreal Tournament 2004]]'' as the platform. Contestants created their bots in advance using the GameBots<ref>http://gamebots.sourceforge.net</ref> interface. GameBots had some modifications made so as to adhere to the above conditions, such as removing data about vantage points or weapon damage that unfairly informed the bots of relevant strengths/weaknessweaknesses that a human would otherwise need to learn.{{cn|date=November 2022}}

==Tournament==

The first BotPrize Tournament was held in [[Perth]], [[Australia]], on 17 December 2008, as part of the 2008 IEEE Symposium on Computational Intelligence and Games in Australia.<ref name="BotPrize 2008"/><ref>{{Cite web|url=http://www.csse.uwa.edu.au/cig08/|title=2008 IEEE Symposium on Computational Intelligence and Games (CIG'08)}}</ref> Each competing team was given time to set up and adjust their bots to the modified game client, although no coding changes were allowed at that point. The tournament was run in rounds, each a 10-minute death match. Judges were the last to join the server and every judge observed every player and every bot exactly once, although the pairing of players and bots did change. When the tournament ended, no bot was rated as more human than any player.<ref>{{Cite web |last=Mer |first=Kold |title=New Round-Up 6 SB |url=https://www.academia.edu/30383516}}</ref>{{cn|date=November 2022}}

In subsequent tournaments, run during 2009–2011,<ref>{{Cite web |url=http://botprize.org/2009.html |title=ArchivedBotprize copy: 2008 |access-date=2013-02-03 |archive-url=https://web.archive.org/web/20130226091236/http://botprize.org/2009.html |archive-date=2013-02-26 |url-status=dead }}</ref><ref>{{Cite web |url=http://botprize.org/2010.html |title=ArchivedBotprize copy: 2010 |access-date=2013-02-03 |archive-url=https://web.archive.org/web/20121230045906/http://botprize.org/2010.html |archive-date=2012-12-30 |url-status=dead }}</ref><ref>{{Cite web |url=http://botprize.org/2011.html |title=ArchivedBotprize copy: 2011 |access-date=2013-02-03 |archive-url=https://web.archive.org/web/20121229083121/http://www.botprize.org/2011.html |archive-date=2012-12-29 |url-status=dead }}</ref> bots achieved scores that were increasingly human-like, but no contestant had won the BotPrize in any of these contests.

In 2012, the annual 2K BotPrize was held once again, and two teams programmed bots that achieved scores greater than those of human players.<ref name="BotPrize"/>

==Successful bots==

To date, there have been two successfully programmed bots that passed the Computercomputer Gamegame Botbot Turing Testtest:

* UT^2, a team from the [[University of Texas at Austin]], emphasized a bot that adjusted its behaviour based on previously observed human behaviour and [[neuroevolution]]. The team has made their bot available,<ref>{{Cite web|url=http://nn.cs.utexas.edu/?ut2|title = NNRG Software - UT^2: Winning Botprize 2012 Entry}}</ref> although a copy of ''[[Unreal Tournament 2004]]'' is required. A short video of their bot is available on YouTube.<ref>Archived at [https://ghostarchive.org/varchive/youtube/20211209/VwIrZ3X4b6c Ghostarchive]{{cbignore}} and the [https://web.archive.org/web/20140421193458/http://www.youtube.com/watch?v=VwIrZ3X4b6c Wayback Machine]{{cbignore}}: {{cite AV media| url = https://www.youtube.com/watch?v=VwIrZ3X4b6c| title = UT bot kills human judge | website=[[YouTube]]}}{{cbignore}}</ref>

* Mihai Polceanu, a doctoral student from [[Romania]], focused on creating a bot that would mimic opponent reactions, in a sense 'borrowing' the human-like nature of the opponent.<ref name="BotPrize"/> These victors succeeded in the year 2012, [[Alan Turing]]'s centenary year.

Comments from the winners can be found in detail at the BotPrize website.<ref name="BotPrize"/> These victors succeeded in the year 2012, [[Alan Turing]]'s centenary year.

==Aftermath==

The outcome of a bot that appears more human-like than a human player is possibly overstated, since in the tournament in which the bots succeeded, the average 'humanness' rating of the human players was only 41.4%.<ref>{{Cite web |url=http://botprize.org/result.html |title=ArchivedBotprize copy2012 : Result |access-date=2013-02-04 |archive-url=https://web.archive.org/web/20130225095906/http://botprize.org/result.html |archive-date=2013-02-25 |url-status=dead }}</ref> This showcases some limits of this Turing Testtest, since the results demonstrate that human behaviour is more complicated and quantitative than was accounted for.<ref>{{Cite news|title=How did this game bot score higher than humans on a Turing Test?|date=October 1, 2012|last=Dvorsky|first=George|url=http://io9.com/5947796/how-can-a-game-bot-score-higher-than-humans-on-a-turing-test}}</ref> In light of this, the BotPrize competition organizers will increase the difficulty in upcoming years with new challenges, forcing competitors to improve their bots.<ref>{{Cite news|title=More human than human: AI game bots pass Turing Test|date=September 26, 2012|first=Darren|last=Quick|url=http://www.gizmag.com/turing-test-ut2004-botprize/24308/}}</ref>

It is also believed that methods and techniques developed for the Computercomputer Gamegame Botbot Turing Testtest will be useful in fields other than video games, such as [[virtual training]] environments and in improving [[Human–robot interaction|robot-human interaction]].<ref>{{Cite news|title=Artificially Intelligent Game Bots Pass the Turing Test on Turing's Centenary|date=September 26, 2012|url=https://www.sciencedaily.com/releases/2012/09/120926133235.htm}}</ref>

==Contrasts to the Turing Testtest==

The Computercomputer Gamegame Botbot Turing test differs from the traditional or generic [[Turing test]] in a number of ways:<ref name="turing"/>

* Unlike the traditional Turing Testtest, for example the [[Chatterbot]]-style contest held annually by the [[Loebner Prize]] competition, the humans who played against the Computer Game Bots are not trying to convince judges they are the human; rather, they want to win the game (i.e., by achieving the highest kill score).

* Judges are not restricted to awarding only one participant in a match as the 'human' and the other as the 'non-human.' This emphasizes more qualitative rather than polarized findings.

* With regards to a successful computervideo game bot, this is not to be confused with a claim that the bot is 'intelligent,' whereas a machine that 'passed' the Turing Testtest would arguably have some evidence for its Chatterbot's 'intelligence.'

* The game ''[[Unreal Tournament 2004]]'' was chosen for its commercial availability and its interface for creating bots, GameBots. This limitation on medium is a sharp contrast to the Turing Testtest, which emphasizes a conversation, where possible questions are vastly more numerous than the set of possible actions available in any specific video game.

* The available information to the participants, humans and bots, is not equal. Humans interact through vision and sound, whereas bots interact with data and events.

* The judges cannot introduce new events (e.g., a lava pit) to aid in differentiating between human and bot, whereas in a Chatterbot designed system, judges may theoretically ask any question in any manner.

Line 53 ⟶ 54:

* [[Virtual reality]]

* [[Turing test]]

* [[Graphics Turing Testtest]]

* The [[Loebner Prize]], a contest that implements the 'traditional' Turing Test

* [[Rog-O-Matic]], a 1984 bot that plays the 1980s dungeon crawler ''[[Rogue (video game)|Rogue]]''

==References==