Content deleted Content added
m WP:CHECKWIKI error fixes, added orphan, uncategorised tags using AWB (11979) |
m Corrected spelling |
||
| (48 intermediate revisions by 33 users not shown) | |||
Line 1:
{{Short description|Human research factorization and quantification system}}
'''Human performance modeling''' ('''HPM''') is a method of quantifying human behavior, cognition, and processes
▲'''Human performance modeling''' ('''HPM''') is a method of quantifying human behavior, cognition, and processes; a tool used by human factors researchers and practitioners for both the analysis of human function and for the development of systems designed for optimal user experience and interaction .<ref name=":0">Sebok, A., Wickens, C., & Sargent, R. (2013, September). Using Meta-Analyses Results and Data Gathering to Support Human Performance Model Development. In ''Proceedings of the Human Factors and Ergonomics Society Annual Meeting'' (Vol. 57, No. 1, pp. 783-787). SAGE Publications.</ref> It is a complementary approach to other usability testing methods for evaluating the impact of interface features on operator performance.<ref name="Carolan, T. 2000, pp. 650-653">Carolan, T., Scott-Nash, S., Corker, K., & Kellmeyer, D. (2000, July). An application of human performance modeling to the evaluation of advanced user interface features. In ''Proceedings of the Human Factors and Ergonomics Society Annual Meeting'' (Vol. 44, No. 37, pp. 650-653). SAGE Publications.</ref>
== History ==
The [[Human Factors and Ergonomics Society]] (HFES) formed the [https://sites.google.com/view/hfes-hpmtg/ Human Performance Modeling Technical Group] in 2004. Although a recent discipline, [[Human factors and ergonomics|human factors]] practitioners have been constructing and applying models of human performance since [[World War II]]. Notable early examples of human performance models include Paul
|
|
| last2 = Pew
| first2 = Richard W.
| date = 2009-06-01
| title = A History and Primer of Human Performance Modeling
| journal = Reviews of Human Factors and Ergonomics
| language = en
Line 28 ⟶ 26:
Individual models vary in their origins, but share in their application and use for issues in the human factors perspective. These can be models of the products of human performance (e.g., a model that produces the same decision outcomes as human operators), the processes involved in human performance (e.g., a model that simulates the processes used to reach decisions), or both. Generally, they are regarded as belonging to one of three areas: perception & attention allocation, command & control, or cognition & memory; although models of other areas such as emotion, motivation, and social/group processes continue to grow burgeoning within the discipline. Integrated models are also of increasing importance<s>.</s> Anthropometric and biomechanical models are also crucial human factors tools in research and practice, and are used alongside other human performance models, but have an almost entirely separate intellectual history, being individually more concerned with static physical qualities than processes or interactions.<ref name=":1" />
The models are applicable in many number of industries and domains including military,<ref>Lawton, C. R., Campbell, J. E., & Miller, D. P. (2005). ''Human performance modeling for system of systems analytics: soldier fatigue'' (No. SAND2005-6569). Sandia National Laboratories.</ref><ref>Mitchell, D. K., & Samms, C. (2012). An Analytical Approach for Predicting Soldier Workload and Performance Using Human Performance Modeling. ''Human-Robot Interactions in Future Military Operations''.</ref> aviation,<ref>Foyle, D. C., & Hooey, B. L. (Eds.). (2007). ''Human performance modeling in aviation''. CRC Press.</ref> nuclear power,<ref>O’Hara, J. (2009). ''Applying Human Performance Models to Designing and Evaluating Nuclear Power Plants: Review Guidance and Technical Basis''. BNL-90676-2009). Upton, NY: Brookhaven National Laboratory.</ref> automotive,<ref>{{cite journal | last1 = Lim
== Model Categories ==
Line 39 ⟶ 37:
==== Pointing ====
Pointing at stationary targets such as buttons, windows, images, menu items, and controls on computer displays is commonplace and has a well-established modeling tool for analysis - [[Fitts
==== [[Control theory|Manual Control Theory]] ====
Complex motor tasks, such as those carried out by musicians and athletes, are not well modeled due to their complexity. Human target-tracking behavior, however, is one complex motor task that is an example of successful HPM.
The history of manual control theory is extensive, dating back to the
Analysis methods were developed that predicted the required control systems needed to enable stable, efficient control of these systems (James, Nichols, & Phillips, 1947). Originally interested in temporal response - the relationship between sensed output and motor output as a function of time - James et al. (1947) discovered that the properties of such systems are best characterized by understanding temporal response after it had been transformed into a frequency response; a ratio of output to input amplitude and lag in response over the range of frequencies to which they are sensitive. For systems that respond linearly to these inputs, the frequency response function could be expressed in a mathematical expression called a ''transfer function''.<ref name=":1" /> This was applied first to machine systems, then human-machine systems for maximizing human performance. Tustin (1947), concerned with the design of gun turrets for human control, was first to demonstrate that nonlinear human response could be approximated by a type of transfer function. McRuer and Krenzel (1957) synthesized all the work since Tustin (1947), measuring and documenting the characteristics of the human transfer function, and ushered in the era of manual control models of human performance. As electromechanical and hydraulic flight control systems were implemented into aircraft, automation and electronic artificial stability systems began to allow human pilots to control highly sensitive systems These same [[transfer function]]s are still used today in [[control engineering]].
Line 60 ⟶ 58:
===== [[Visual search|Visual Search]] =====
A developed area in attention is the control of visual attention - models that attempt to answer, "where will an individual look next?" A subset of this concerns the question of visual search: How rapidly can a specified object in the visual field be located? This is a common subject of concern for human factors in a variety of domains, with a substantial history in cognitive psychology. This research continues with modern conceptions of [[Salience (neuroscience)|salience]] and [http://www.scholarpedia.org/article/Saliency_map salience maps]. Human performance modeling techniques in this area include the work of Melloy, Das, Gramopadhye, and Duchowski (2006) regarding [[Markov models]] designed to provide upper and lower bound estimates on the time taken by a human operator to scan a homogeneous display.<ref>{{cite journal | last1 = Melloy
==== Visual Sampling ====
Many domains contain multiple displays, and require more than a simple discrete yes/no response time measurement. A critical question for these situations may be "How much time will operators spend looking at X relative to Y?" or "What is the likelihood that the operator will completely miss seeing a critical event?" Visual sampling is the primary means of obtaining information from the world.<ref name=":3">Cassavaugh, N. D., Bos, A., McDonald, C., Gunaratne, P., & Backs, R. W. (2013). Assessment of the SEEV Model to Predict Attention Allocation at Intersections During Simulated Driving. In ''7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design'' (No. 52).</ref> An early model in this ___domain is Sender's (1964, 1983) based upon operators monitoring of multiple dials, each with different rates of change.<ref>Senders, J. W. (1964). The human operator as a monitor and controller of multidegree of freedom systems. ''Human Factors in Electronics, IEEE Transactions on'', (1), 2-5.</ref><ref>Senders, J. W. (1983). ''Visual sampling processes'' (Doctoral dissertation, Universiteit van Tilburg).</ref> Operators try, as best as they can, to reconstruct the original set of dials based on discrete sampling. This relies on the mathematical [[Nyquist–Shannon sampling theorem|Nyquist theorem]] stating that a signal at W Hz can be reconstructed by sampling every 1/W seconds. This was combined with a measure of the information generation rate for each signal, to predict the optimal sampling rate and dwell time for each dial. Human limitations prevent human performance from matching optimal performance, but the predictive power of the model influenced future work in this area, such as Sheridan's (1970) extension of the model with considerations of access cost and information sample value.<ref name=":1" /><ref>{{cite journal | last1 = Sheridan
A modern conceptualization by Wickens et al. (2008) is the salience, effort, expectancy, and value (SEEV) model. It was developed by the researchers (Wickens et al., 2001) as a model of scanning behavior describing the probability that a given area of interest will attract attention (AOI). The SEEV model is described by '''''p(A) = sS - efEF + (exEX)(vV)''''', in which ''p(A)'' is the probability a particular area will be samples, ''S'' is the ''salience'' for that area; ''EF'' represents the ''effort'' required in reallocating attention to a new AOI, related to the distance from the currently attended ___location to the AOI; ''EX'' (''expectancy'') is the expected event rate (bandwidth), and ''V'' is the value of the information in that AOI, represented as the product of Relevance and Priority (R*P).<ref name=":3" /> The lowercase values are scaling constants. This equation allows for the derivation of optimal and normative models for how an operator should behave, and to characterize how they behave. Wickens et al., (2008) also generated a version of the model that does not require absolute estimation of the free parameters for the environment - just the comparative salience of other regions compared to region of interest.<ref name=":1" />
Line 74 ⟶ 72:
==== [[Workload]] ====
Although an exact definition or method for measurement of the construct of workload is debated by the human factors community, a critical part of the notion is that human operators have some capacity limitations and that such limitations can be exceeded only at the risk of degrading performance. For physical workload, it may be understood that there is a maximum amount that a person should be asked to lift repeatedly, for example. However, the notion of workload becomes more contentious when the capacity to be exceeded is in regard to attention - what are the limits of human attention, and what exactly is
Byrne and Pew (2009) consider an example of a basic workload question: "To what extent do task A and B interfere?" These researchers indicate this as the basis for the ''[[psychological refractory period]]'' (PRP) paradigm. Participants perform two choice reaction-time tasks, and the two tasks will interfere to a degree - especially when the participant must react to the stimuli for the two tasks when they are close together in time - but the degree of interference is typically smaller than the total time taken for either task. The ''response selection bottleneck model'' (Pashler, 1994) models this situation well - in that each task has three components: perception, response selection (cognition), and motor output. The attentional limitation - and thus locus of workload - is that response selection can only be done for one task at a time. The model makes numerous accurate predictions, and those for which it cannot account are addressed by cognitive architectures (Byrne & Anderson, 2001; Meyer & Kieras, 1997). In simple dual-task situations, attention and workload are quantified, and meaningful predictions made possible.<ref name=":1" />
Line 80 ⟶ 78:
Horrey and Wickens (2003) consider the questions: To what extent will a secondary task interfere with driving performance, and does it depend on the nature of the driving and on the interface presented in the second task? Using a model based on ''[[Workload|multiple resource theory]]'' (Wickens, 2002, 2008; Navon & Gopher, 1979), which proposes that there are several loci for multiple-task interference (the stages of processing, the codes of processing, and [[Modality (human–computer interaction)|modalities]]), the researchers suggest that cross-task interference increases proportional to the extent that the two tasks use the same resources within a given dimension: Visual presentation of a read-back task should interfere more with driving than should auditory presentation, because driving itself makes stronger demands on the visual modality than on the auditory.<ref name=":1" />
Although multiple resource theory the best known workload model in human factors, it is often represented qualitatively. The detailed computational implementations are better alternatives for
Numerical typing is an important perceptual-motor task whose performance may vary with different pacing, finger strategies and urgency of situations. Queuing network-model human processor (QN-MHP), a computational architecture, allows performance of perceptual-motor tasks to be modelled mathematically. The current study enhanced QN-MHP with a top-down control mechanism, a close-loop movement control and a finger-related motor control mechanism to account for task interference, endpoint reduction, and force deficit, respectively. The model also incorporated neuromotor noise theory to quantify endpoint variability in typing. The model predictions of typing speed and accuracy were validated with Lin and
The psychological refractory period (PRP) is a basic but important form of dual-task information processing. Existing serial or parallel processing models of PRP have successfully accounted for a variety of PRP phenomena; however, each also encounters at least 1 experimental counterexample to its predictions or modeling mechanisms. This article describes a queuing network-based mathematical model of PRP that is able to model various experimental findings in PRP with closed-form equations including all of the major counterexamples encountered by the existing models with fewer or equal numbers of free parameters. This modeling work also offers an alternative theoretical account for PRP and demonstrates the importance of the theoretical concepts of “queuing” and “hybrid cognitive networks” in understanding cognitive architecture and multitask performance.<ref>{{Cite journal|title = Queuing network modeling of the psychological refractory period (PRP).
=== Cognition & Memory ===
The paradigm shift in psychology from behaviorism to the study of cognition had a huge impact on the field of Human Performance Modeling. Regarding memory and cognition, the research of Newell and Simon regarding artificial intelligence and the [[General Problem Solver]] (GPS; Newell & Simon, 1963), demonstrated that computational models could effectively capture fundamental human cognitive behavior. Newell and Simon were not simply concerned with the amount of information - say, counting the number of bits the human cognitive system had to receive from the perceptual system - but rather the actual computations being performed. They were critically involved with the early success of comparing cognition to computation, and the ability of computation to simulate critical aspects of cognition - thus leading to the creation of the sub-discipline of [[artificial intelligence]] within [[computer science]], and changing how cognition was viewed in the psychological community. Although cognitive processes do not literally flip bits in the same way that discrete electronic circuits do, pioneers were able to show that any universal computational machine could simulate the processes used in another, without a physical equivalence (Phylyshyn, 1989; Turing, 1936). The [[cognitive revolution]] allowed all of cognition to be approached by modeling, and these models now span a vast array of cognitive domains - from simple list memory, to comprehension of communication, to problem solving and decision making, to imagery, and beyond.<ref name=":1" />
One popular example is the Atkinson-Shiffrin (1968) [[Atkinson–Shiffrin memory model|"modal" model of memory]]. Also, please see [[Cognitive models|Cognitive Models]] for information not included here..
==== Routine Cognitive Skill ====
One area of memory and cognition regards modeling routine cognitive skills; when an operator has the correct knowledge of how to perform a task and simply needs to execute that knowledge. This is widely applicable, as many operators are practiced enough that their procedures become routine. The GOMS (goals, operators, methods, and selection rules) family of Human Performance Models popularized and well-defined by researchers in the field (Card et al., 1983; John & Kieras, 1996a, 1996b) were originally applied to model users of computer interfaces, but have since been extended to other areas. They are useful HPM tools, suitable for a variety of different concerns and sizes of analysis, but are limited in regard to analyzing user error (see Wood & Kieras, 2002, for an effort to extend GOMS to handling errors).<ref name=":1" />
The simplest form of a GOMS model is a ''keystroke-level model'' (KLM) - in which all physical actions are listed (e
Detailed versions of GOMS exist, including:
Line 106 ⟶ 103:
Another critical cognitive activity of interest to human factors is that of judgement and decision making. These activities starkly contrast to routine cognitive skills, for which the procedures are known in advance, as many situations require operators to make judgments under uncertaintly - to produce a rating of quality, or perhaps choose among many possible alternatives. Although many disciplines including mathematics and economics make significant contributions to this area of study, the majority of these models do not model human behavior but rather model optimal behavior, such as ''[[Expected utility hypothesis|subjective expected utility theory]]'' (Savage, 1954; von Neumann & Morgenstern, 1944). While models of optimal behavior are important and useful, they do not consider a baseline of comparison for human performance - though much research on human decision making in this ___domain compares human performance to mathematically optimal formulations. Examples of this include Kahneman and Tversky's (1979) ''[[prospect theory]]'' and Tversky's (1972) ''elimination by aspects model''. Less formal approaches include Tversky and Kahneman's seminal work on heuristics and biases, Gigerenzer's work on 'fast and frugal' shortcuts (Gigerenzer, Todd, & ABC Research Group, 2000), and the descriptive models of Paune, Bettman, and Johnson (1993) on adaptive strategies.<ref name=":1" />
Sometimes optimal performance is uncertain, one powerful and popular example is the '''''lens model''''' (Brunswick, 1952; Cooksey, 1996; Hammond, 1955), which deals with ''policy capturing'', ''[[cognitive control]]'', and ''cue utilization'', and has been used in aviation (Bisantz & Pritchett, 2003), command and control (Bisantz et al., 2000); to investigate human judgement in employment interviews (Doherty, Ebert, & Callender, 1986), financial analysis (Ebert & Kruse, 1978), physicians' diagnoses (LaDuca, Engel, & Chovan, 1988), teacher ratings (Carkenord & Stephens, 1944), and numerous others.<ref name=":1" /> Although the model does have limitations [described in Byrne & Pew (2009)], it is very powerful and remains underutilized in the human factors profession.<ref name=":1" />
===== [[Situation awareness|Situation Awareness]] (SA) =====
Models of SA range from descriptive (Endsley, 1995) to computational (Shively et al., 1997).<ref name=":2" /><ref>{{cite journal | last1 = Endsley
model of situational awareness instantiated in MIDAS.
Proceedings of the Ninth International Symposium on Aviation
Psychology (pp. 1454-1459). Columbus, OH: University of Ohio.
</ref> The most useful model in HPM is that of McCarley et al. (2002) known as the [http://hsi.arc.nasa.gov/groups/HCSL/publications/mccargohhorhf02.pdf '''A-SA model'''] (Attention/Situation Awareness). It incorporates two semi-independent components: a perception/attention module and a cognitive SA-updated module.<ref name=":2" /> The P/A model of this A-SA model is based on the Theory of Visual Attention.<ref>Bundesen, C. (1990). A theory of visual attention. Psychological
Line 134 ⟶ 125:
When a modeler builds a network model of a task, the first step is to construct a flow chart decomposing the task into discrete sub-tasks - each sub-task as a node, the serial and parallel paths connecting them, and the gating logic that governs the sequential flow through the resulting network. When modeling human-system performance, some nodes represent human decision processes and.or human task execution, some represent system execution sub-tasks, and some aggregate human/machine performance into a single node. Each node is represented by a statistically specified completion time distribution and a probability of completion. When all these specifications are programmed into a computer, the network is exercised repeatedly in Monte Carlo fashion to build up distributions of the aggregate performance measures that are of concern to the analyst. The art in this is in the modeler's selection of the right level of abstraction at which to represent nodes and paths and in estimating the statistically defined parameters for each node. Sometimes, human-in-the-loop simulations are conducted to provide support and validation for the estimates.. Detail regarding this, related, and alternative approaches may be found in Laughery, Lebiere, and Archer (2006) and in the work of Schwieckert and colleagues, such as Schweickert, Fisher, and Proctor (2003).<ref name=":1" />
Historically, Task Network Modeling stems from queuing theory and modeling of engineering reliability and quality control. Art Siegel, a psychologist, first though of extending reliability methods into a Monte Carlo simulation model of human-machine performance (Siegel & Wolf, 1969). In the early 1970s, the U.S. Air Force sponsored the development of '''SAINT''' (Systems Analysis of Integrated Networks of Tasks), a high-level programming language specifically designed to support the programming of Monte Carlo simulations of human-machine task networks (Wortman, Pritsker, Seum, Seifert, & Chubb, 1974). A modern version of this software is [[Micro Saint Sharp]] (Archer, Headley, & Allender, 2003). This family of software spawned a tree of special-purpose programs with varying degrees of commonality and specificity with Micro Saint. The most prominent of these is the
The network approach to modeling using these programs is popular due to its technical accessibility to individual with general knowledge of computer simulation techniques and human performance analysis. The flowcharts that result from task analysis lead naturally to formal network models. The models can be developed to serve specific purposes - from simulation of an individual using a human-computer interface to analyzing potential traffic flow in a hospital emergency center. Their weakness is the great difficulty required to derive performance times and success probabilities from previous data or from theory or first principles. These data provide the model's principle content.
Line 141 ⟶ 132:
Cognitive Architectures are broad theories of human cognition based on a wide selection of human empirical data and are generally implemented as computer simulations. They are the embodiment of a scientific hypothesis about those aspects of human cognition relatively constant over time and independent of task (Gray, Young, & Kirschenbaum, 1997; Ritter & young, 2001). Cognitive architectures are an attempt to theoretically unify disconnected empirical phenomena in the form of computer simulation models. While theory is inadequate for the application of human factors, since the 1990s cognitive architectures also include mechanisms for sensation, perception, and action. Two early examples of this include the Executive Process Interactive Control model (EPIC; Kieras, Wood, & Meyer, 1995; Meyer & Kieras, 1997) and the ACT-R (Byrne & Anderson, 1998).
A model of a task in a cognitive architecture, generally referred to as a cognitive model, consists of both the architecture and the knowledge to perform the task. This knowledge is acquired through human factors methods including task analyses of the activity being modeled.
Examples of cognitive architectures include the EPIC system (Hornof & Kieras, 1997, 1999)
The Queuing Network-Model Human Processor model was used to predict how drivers perceive the operating speed and posted speed limit, make choice of speed, and execute the decided operating speed. The model was sensitive (average d’ was 2.1) and accurate (average testing accuracy was over 86%) to predict the majority of unintentional speeding<ref name=":4" />
ACT-R has been used to model a wide variety of phenomena. It consists of several modules, each one modeling a different aspect of the human system. Modules are associated with specific brain regions, and the ACT-R has thus successfully predicted neural activity in parts of those regions. Each model essentially represents a theory of how that piece of the overall system works - derived from research literature in the area. For example, the declarative memory system in ACT-R is based on series of equations considering frequency and recency and that incorporate
=== Group Behavior ===
Line 155 ⟶ 146:
=== Modeling Approaches ===
'''Computer Simulation Models/Approaches'''
Example: [[IMPRINT (Improved Performance Research Integration Tool)]]
'''Mathematical Models/Approaches'''
Line 169 ⟶ 159:
== Benefits ==
Numerous benefits may be gained from using modeling techniques in the human [[performance ___domain]].
=== Specificity ===
Line 192 ⟶ 182:
=== Abstraction ===
The abstraction necessary for understandable models competes with accuracy. While generality, simplicity, and understandability are important to the application of models in human factors practice, many valuable human performance models are inaccessible to those without graduate, or postdoctoral training. For example, while [[Fitts's law]] is straightforward for even undergraduates, the lens model requires an intimate understanding of multiple regression, and construction of an ACT-R type model requires extensive programming skills and years of experience. While the successes of complex models are considerable, a practitioner of HPM must be aware of the trade-offs between accuracy and usability.<ref name=":1" />
=== Free Parameters ===
Line 209 ⟶ 199:
-'''[[Coefficient of determination]]''' '''([[Coefficient of determination|R Square]])''': A line or curve indicate how well the data fit a statistic model.
-[[Root mean square|'''Root Mean Square''']] '''([[Root mean square|RMS]])''': A statistical measure defined as the square root of the arithmetic mean of the squares of a set of numbers.<ref>{{Cite
| title = A Dictionary of Physics (6 ed.). Oxford University Press. 2009. ISBN 9780199233991.
| chapter-url = http://www.oxfordreference.com/view/10.1093/acref/9780199233991.001.0001/acref-9780199233991-e-2676
| isbn = 9780199233991
| chapter = Root-mean-square value
| publisher = Oxford University Press
| year = 2009
}}</ref>
Line 252 ⟶ 246:
{{Reflist}}
[[Category:Scientific modelling]]
[[Category:Software optimization]]
| |||