Modality (human–computer interaction): Difference between revisions

Revision as of 08:26, 15 May 2015

Revision as of 00:43, 17 March 2016

Line 10:

** [[Hearing (sense)|Audition]] - various audio outputs

** [[Haptics|Tactition]] - vibrations or other movement

* Uncommon modalities

** [[Taste|Gustation]] (taste)

Line 18 ⟶ 17:

** [[Equilibrioception]] (balance)

Any human sense can be used as a computer to human modality. However, the modalities of [[visual perception|seeing]] and [[hearing (sense)|hearing]] are the most commonly employed since they are capable of transmitting information at a higher speed than other modalities, 250 to 300<ref name=Ziefle98>{{cite journal|last1=Ziefle|first1=M|title=Effects of display resolution on visual performance.|journal=Human factors|date=December 1998|volume=40|issue=4|pages=554–68|pmid=9974229}}</ref> and 150 to 160<ref>Williams, J. R. (1998). Guidelines for the use of multimedia in instruction, Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting, 1447–1451</ref> [[words per minute]], respectively. Though not commonly implemented as computer-human modality, tactition can achieve an average of 125 wpm <ref>{{cite web|title=Braille|url=http://www.acb.org/node/67|website=ACB|publisher=American Council of the Blind|accessdate=21 April 2015}}</ref> through the use of a [[refreshable Braille display]]. Other more common forms of tactition are smartphone and game controller vibrations.

==Human–Computer Modalities==

Computers can be equipped with various types of [[input devices]] and sensors to allow them to receive information from humans. Common input devices are often interchangeable if they have a standardized method of communication with the computer and [[Affordance|afford]] practical adjustments to the user. Certain modalities can provide a richer interaction depending on the context, and having options for implementation allows for more robust systems.<ref>{{Cite book|title = Berkshire Encyclopedia of Human-computer Interaction|last = Bainbridge|first = William|publisher = Berkshire Publishing Group LLC|year = 2004|isbn = 9780974309125|___location = |pages = 483|url = https://books.google.com/books?id=568u_k1R4lUC&dq}}</ref>

Line 26:

** [[Pointing device]]

** [[Touchscreen]]

* Complex modalities

** [[Computer Vision]]

Line 32 ⟶ 31:

** [[Accelerometer|Motion]]

** [[Orientation (geometry)|Orientation]]

With the increasing popularity of [[smartphones]], the general public are becoming more comfortable with the more complex modalities. Speech recognition was a major selling point of the [[iPhone 4S]] and following [[Apple Inc.|Apple]] products, with the introduction of [[Sirius|Siri]].<ref>{{Cite news|url = http://bgr.com/2011/11/02/siri-said-to-be-driving-force-behind-huge-iphone-4s-sales/|title = Siri said to be driving force behind huge iPhone 4S sales|last = Epstein|first = Zach|date = Nov 2, 2011|work = |access-date = April 21, 2015|via = }}</ref> This technology gives users an alternative way to communicate with computers when typing is less desirable. However, in a loud environment, the audition modality is not quite effective. This exemplifies how certain modalities have varying strengths depending on the situation.<ref>{{Cite book|title = Multimodality in Mobile Computing and Mobile Devices: Methods for Adaptable Usability|last = Kurkovsky|first = Stan|publisher = IGI Global|year = 2009|isbn = 9781605669793|___location = |pages = 210-211210–211|url = https://books.google.com/books?id=kqxpqs32muQC&dq}}</ref> Other complex modalities such as computer vision in the form of [[Microsoft]]'s [[Kinect]] or other similar technologies can make sophisticated tasks easier to communicate to a computer especially in the form of three dimensional movement.<ref>{{Cite book|title = Human-Computer Interaction: Interaction Modalities and Techniques|last = Kurosu|first = Masaaki|publisher = Springer|year = 2013|isbn = 9783642393303|___location = |pages = 366|url = https://books.google.com/books?id=p5W6BQAAQBAJ&dq}}</ref>

==Using Multiple Modalities==

Having multiple modalities in a system gives more [[affordance]] to users and can contribute to a more robust system. Having more also allows for greater [[accessibility]] for users who work more effectively with certain modalities. Multiple modalities can be used as backup when certain forms of communication are not possible. This is especially true in the case of redundant modalities in which two or more modalities are used to communicate the same information. Certain combinations of modalities can add to the expression of a computer-human or human-computer interaction because the modalities each may be more effective at expressing one form or aspect of information than others.

There are six types of cooperation between modalities, and they help define how a combination or fusion of modalities work together to convey information more effectively.<ref name=":0">{{Cite book|title = Multimodal Human Computer Interaction and Pervasive Services|last = Grifoni|first = Patrizia|publisher = IGI Global|year = 2009|isbn = 9781605663876|___location = |pages = 37|url = https://books.google.com/books?id=O8CqMtIKSWwC&source=gbs_navlinks_s}}</ref>

Line 44 ⟶ 43:

* '''Transfer: '''a modality produces information that another modality consumes

* '''Concurrency:''' multiple modalities take in separate information that is not merged

Complimentary-redundant systems are those which have multiple sensors to form one understanding or dataset, and the more effectively the information can be combined without duplicating data, the more effectively the modalities cooperate. Having multiple modalities for communication is common, particularly in smartphones, and often their implementations work together towards the same goal, for example gyroscopes and accelerometers working together to track movement.<ref name=":0">{{Cite book|title = Multimodal Human Computer Interaction and Pervasive Services|last = Grifoni|first = Patrizia|publisher = IGI Global|year = 2009|isbn = 9781605663876|___location = |pages = 37|url = https://books.google.com/books?id=O8CqMtIKSWwC&source=gbs_navlinks_s}}</ref>

==See also==

Line 50 ⟶ 49:

* [[User-interface]]

* [[Multisensory integration]]

== References ==

{{Reflist}}