|
#REDIRECT [[Markov model]] {{R with history}}
{{Userspace draft|source=ArticleWizard|date=December 2011}} <!-- Please leave this line alone! -->
Hierarchical Markov Models have been recently applied to recognize human behavior at different levels of abstraction. The term ''behavior recognition'' is used to refer to the task of determining a high-level activity that a person is performing (e.g., cooking) based on a sequence of low-level observations (e.g., the ___location of the person in a room) often captured by devices such as video cameras and motion sensors. This article briefly introduces two kinds of Hierarchical Markov Models--[[Hierarchical hidden Markov model|Hierarchical Hidden Markov Models]] and Abstract Hidden Markov Models--and then discusses how they have been used for behavior recognition.
== Examples of Hierarchical Markov Models ==
=== Hierarchical Hidden Markov Model ===
A [[Hierarchical hidden Markov model|Hierarchical Hidden Markov Model]] (HHMM) <ref name="HHMM">S. Fine and Y. Singer [http://dl.acm.org/citation.cfm?id=325879 The hierarchical hidden markov model: Analysis and applications]. Journal of Machine Learning, vol. 32, 1998.</ref> is an extension of a [[Hidden Markov model|Hidden Markov Model]] that contains states at different levels of abstraction. Thus, the structure of an HHMM is more suitable for representing the hierarchy in human activities. For instance, the activity of eating dinner could be decomposed in the activities of preparing ingredients, cooking, cleaning, and washing dishes. A corresponding HHMM would have a high-level state corresponding to the activity of eating dinner and low-level states corresponding to the sub-activities. The reader is referred to the article on [[Hierarchical hidden Markov model|Hierarchical Hidden Markov Models]] for more detailed information.
=== Abstract Hidden Markov Model ===
An Abstract Hidden Markov Model (AHMM) <ref name="AHMM">H. H. Bui, S. Venkatesh, and G. West [http://dl.acm.org/citation.cfm?id=1622824 Policy recognition in the abstract hidden markov model]. Journal of Artificial Intelligence Research, vol. 17, p. 451–499, 2002.</ref> is an extension of a [[Hierarchical hidden Markov model|Hierarchical Hidden Markov Model]] that allows the choice of how a high-level activity (policy) will be decomposed into a sequence of lower-level activities (policies) to be dependent on the current state of the environment. Thus, simple types of context-sensitive behaviors can be captured by an AHMM <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning"> S. Luhr, H. H. Bui, S. Venkatesh, and G. A. W. West [http://dl.acm.org/citation.cfm?id=826390 Recognition of Human Activity through Hierarchical Stochastic Learning]. PERCOM '03 Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. </ref>.
== Applications to Human Behavior Recognition ==
[[Image:Kitchen.png|right|thumb|600px|
Figure 1. Layout for kitchen. (a) the “food preparation first” and (b) the “washing dishes first” meal preparation sequences. <br>
Image from <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" />]]
'''Hierarchical Hidden Markov Models (HHMMs)''' have been used in the context of eldercare aiming to identify behaviors such as eating dinner, watching television, and cooking <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" />. The goal is to learn the pattern of executing such activities normally, so that when an abnormal execution occurs (e.g., an elder person falls down while cooking), it can be automatically and instantaneously detected. Then, the appropriate action (e.g., provide audio/video cues to help the elderly or notify the appropriate caretaker that the elderly needs help) can then be taken before negative consequences for the elder person's health occur.
[[Supervised learning]] (a class of [[machine learning]] techniques) was used in <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" /> to learn an HHMM for four simplistic ways for dinner preparation ("spending some time preparing the food and rummaging through the fridge before the meal is cooked", "washing dishes prior to cooking on the stove", "going to the sink to wash the dishes then spending time at the food preparation area and the fridge before finally cooking the meal", "transition between each area of interest in a round robin fashion, starting and ending at the stove, before leaving the room") and five typical living room activities ("watch television", "read a book on the couch", "eat dinner", "eat dinner while watching television" and "there is nothing good on TV, read a book instead"). The person's position in the room was captured at short intervals of time and a sequence of observations was thus created. Observation sequences were labelled with the corresponding behavior and were used to learn HHMMs for the high-level dinner preparation and living room behaviors. Two example kitchen behaviors are shown in Figure 1. Figure 2 shows an HHMM learned for the "food preparation first" kitchen behavior.
[[Image:LearnedHHMM.png|center|thumb|800px|
Figure 2. Learned HHMM for "food preparation first" kitchen behavior. <br>
Image from <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" />]]
Once the HHMMs for the kitchen and living room behaviors have been learned, they can be used to classify new sequences of observations. The experiments with recognizing simplistic kitchen and living room behaviors in <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" /> report high classification accuracy--83% for the cooking sequences and 80% for the living room sequences.
'''Abstract Hidden Markov Models (AHMM)''' have been used successfully to recognize human behavior based on the human's position in a building as recorded by video cameras<ref name="AHMM"/>. For instance, if the person is close to a computer, the behavior recognition method can infer that the person is using the computer, or if the person is moving in a hallway towards the north exit of the building, the method infers that the person intends to exit the building through the north exit.
== Advantages of Hierarchical Markov Models for Behavior Recognition ==
The ability of Hierarchical Markov Models to capture hierarchical decomposition makes them an appealing vehicle for capturing the hierarchical nature of human activities. It also allows for model re-usability <ref name="HierarchicalLearningAndPlanningInPOMDPs"> G. Theocharous [http://dl.acm.org/citation.cfm?id=936140 Hierarchical Learning and Planning in Partially Observable Markov Decision Processes]. PhD thesis, 2002. </ref> and certain conditional independence properties between different levels of abstraction in the model allow for faster learning and inference <ref name="AHMM" /> <ref name="RecognitionOfHumanActivityThroughHierarchicalStochasticLearning" />.
== References ==
{{Reflist}}
<!--- Categories --->
[[Category:Machine_learning]]
[[Category:Artificial_intelligence]]
| 