Component-based usability testing: Difference between revisions

Revision as of 20:02, 20 November 2010

Revision as of 15:03, 25 January 2011

Line 4:

==Theory==

CBUT is based on both software architectural views such as [[Model–View–Controller]] (MVC), [[Presentation-Abstraction-Control]] (PAC), ICON and CNUCE agent models that split up the software in parts, and [[cognitive psychology]] views where a person’s mental process is split up in smaller mental processes. Both software architecture and cognitive architecture use the principle of hierarchical layering, in which low level processes are more elementary and for humans often more physical in nature, such as the coordination movement of muscle groups. Processes that operate on higher level layers are more abstract and focus on a person’s main goal, such as writing an application letter to get a job. The Layered Protocol Theory<ref name="Farrel1999">Farrell, P.S.E., Hollands, J.G., Taylor, M.M., Gamble, H.D., (1999). Perceptual control and layered protocols in interface design: I. Fundamental concepts. ''International Journal of Human-Computer Studies'' 50 (6), 489-520. [http://dx.doi.org/doi:10.1006/ijhc.1998.0259 online] </ref> (LPT), which is a special version of [[Perceptual Control Theory]] (PCT), brings these views together by suggesting that users interact with a system across several layers by sending messages. Users interact with components on high layers by sending messages, such as pressing keys, to components operating on lower layers, which on their turn relay a series of these messages into a single high level message, such as ‘DELETE *.*’, to a component on a higher layer. Components operating on higher layers, communicate back to the user by sending messages to components operating on lower level layers. Whereas this layered-interaction model explains how the interaction is established, control loops explain the purpose of the interaction. LPT sees the purpose of the users’ behaviour as the users’ attempt to control their perception, in this case the state of the component they perceive. This means that users will only act if they perceive the component to be in an undesirable state. For example, if a person have an empty glass but want a full glass of water, he or she will act (e.g. walk to the tap, turning the tap on to fill the glass). The action of filling the glass will continue until the person perceives the glass as full. As interaction with components takes places on several layers, interacting with a single device can include several control loops. The amount of effort put into operating a control loop is seen as an indicator for the usability of an interaction component.

==Testing==

CBUT can be categorized according to two testing paradigms, the Single-Version Testing Paradigm (SVTP) and the Multiple-Versions Testing Paradigm (MVTP). In SVTP only one version of each interaction component in a system is tested. The focus is to identify interaction components that might reduce the overall usability of the system. SVTP is therefore suitable as part of a software-integration test. In MVTP on the other hand, multiple versions of a single component are tested while the remaining components in the system remain unchanged. The focus is on identifying the version with the highest usability of specific interaction component. MVTP therefore is suitable for component development and selection. Different CBUT methods have been proposed for SVTP and MVTP, which include measures based on recorded user interaction and questionnaires. Whereas in MVTP the recorded data can directly be interpreted by making a comparison between two versions of the interaction component, in SVTP log file analysis is more extensive as interaction with both higher and lower components must be considered <ref name="Brinkman2007">Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2007), Towards an empirical method of efficiency testing of system parts: a methodological study, ''Interacting with Computers'', vol. 19, no. 3, pp. 342-356. [http://mmi.tudelft.nl/~willem-paul/WP_Papers_online_versie/Towards_an_empirical_method_of_efficiency_testing_of_system_parts_a_methodological_study_preliminary_version.pdf preliminary version] [http://dx.doi.org/doi:10.1016/j.intcom.2007.01.002 online] </ref>.

Meta-analysis on the data from several lab experiments that used CBUT measures suggests that these measures can be statistically more powerful than overall (holistic) usability measures <ref name="Brinkman2008">Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2008). Component-Specific Usability Testing,'' IEEE Transactions on Systems, Man, and Cybernetics - Part A'', vol. 38, no. 5, pp. 1143-1155, September 2008. [http://mmi.tudelft.nl/~willem-paul/WP_Papers_online_versie/Component_specific_usability_testing_preliminary_version.pdf preliminary version] [http://dx.doi.org/doi:10.1109/TSMCA.2008.2001056 online]</ref>.

==Usability questionnaire==

Whereas holistic oriented usability questionnaires such as [[System Usability Scale]] (SUS) examine the usability of a system on several dimensions such as defined in [[ISO 9241]] Part 11 standard effectiveness, efficiency and satisfaction, a Component-Based Usability Questionnaire (CBUQ) <ref name="Brinkman2009"> Brinkman, W.-P., Haakma, R., & Bouwhuis, D.G. (2009), Theoretical foundation and validity of a component-based usability questionnaire, ''Behaviour and Information Technology'', 2, no. 28, pp. 121 - 137. [http://mmi.tudelft.nl/~willem-paul/WP_Papers_online_versie/The_theoretical_foundation_and_Validity_of_a_component_based_usability_questionnaire_preliminary_version.pdf preliminary version] [http://mmi.tudelft.nl/~willem-paul/mp3player/Intro.htm MP3 example study] [http://dx.doi.org/DOI:10.1080/01449290701306510 online]</ref>

is a questionnaire which can be used to evaluate the usability of individual interaction components, such as the volume control or the play control of a MP3 player. To evaluate an interaction component, the six Perceived Ease-Of-Use (PEOU) statements from the [[Technology acceptance model]] are taken with a reference to the interaction component, instead of to the entire system, for example:

Line 30:

== External links ==

* [http://mmi.tudelft.nl/~willem-paul/mp3player/Intro.htm Example study] that uses Component-based Usability Questionnaire including [http://mmi.tudelft.nl/~willem-paul/mp3player/study.htm instructions, questionnaires] and [http://mmi.tudelft.nl/~willem-paul/mp3player/results.htm data analysis].

<!--- Categories --->

[[Category:Usability]]

[[Category:Software testing]]