Graphical model: Difference between revisions

Revision as of 17:31, 10 February 2021

Revision as of 15:15, 17 May 2021

Line 2:

{{More footnotes|date=May 2017}}

A '''graphical model''' or '''probabilistic graphical model''' ('''PGM''') or '''structured probabilistic model''' is a [[probabilistic model]] for which a [[Graph (discrete mathematics)|graph]] expresses the [[conditional dependence]] structure between [[random variable]]s. They are commonly used in [[probability theory]], [[statistics]]—particularly [[Bayesian statistics]]—and [[machine learning]].

[[File:Graph model.svg|thumb|right|alt=An example of a graphical model.| An example of a graphical model. Each arrow indicates a dependency. In this example: D depends on A, B, and C; and C depends on B and D; whereas A and B are each independent.]]

==Types of graphical models==

Line 22 ⟶ 20:

|url-status=dead

}}</ref>

===Undirected Graphical Model===

[[File:Examples of an Undirected Graph.svg|thumb|alt=An undirected graph with four vertices.|An undirected graph with four vertices.]]

The undirected graph shown may have one of several interpretations; the common feature is that the presence of an edge implies some sort of dependence between the corresponding random variables. From this graph we might deduce that <math>B,C,D</math> are all mutually independent, once <math>A</math> is known, or (equivalently in this case) that

:<math>P[A,B,C,D] = Pf_{AB}[A,B] \cdot Pf_{AC}[BA,C] \cdot Pf_{AD}[C,D|A,BD]</math>

for some non-negative functions <math>f_{AB}, f_{AC}, f_{AD}</math>.

===Bayesian network===

{{main|Bayesian network}}

[[File:Example of a Directed Graph.svg|thumb|alt=Example of a directed acyclic graph on four vertices.|Example of a directed acyclic graph on four vertices.]]

If the network structure of the model is a [[directed acyclic graph]], the model represents a factorization of the joint [[probability]] of all random variables. More precisely, if the events are <math>X_1,\ldots,X_n</math> then the joint probability satisfies

Line 30 ⟶ 39:

:<math>P[X_1,\ldots,X_n]=\prod_{i=1}^nP[X_i|\text{pa}(X_i)]</math>

where <math>\text{pa}(X_i)</math> is the set of parents of node <math>X_i</math> (nodes with edges directed towards <math>X_i</math>). In other words, the [[joint distribution]] factors into a product of conditional distributions. For example, the graphical model in the Figure shown above (which is actually not a directed acyclic graph, butshown an [[ancestral graph]]) consists ofin the random variablesFigure <math>A,this B,factorization C,would D</math>be

:<math>P[A,B,C,D] = P[A]\cdot P[B | A]\cdot P[C | A] \cdot P[D|A,C]</math>.

with a joint probability density that factors as

:<math>P[A,B,C,D] = P[A]\cdot P[B]\cdot P[C,D|A,B]</math>

Any two nodes are [[Conditional independence|conditionally independent]] given the values of their parents. In general, any two sets of nodes are conditionally independent given a third set if a criterion called [[d-separation|''d''-separation]] holds in the graph. Local independences and global independences are equivalent in Bayesian networks.

This type of graphical model is known as a directed graphical model, [[Bayesian network]], or belief network. Classic machine learning models like [[hidden Markov models]], [[neural networks]] and newer models such as [[variable-order Markov model]]s can be considered special cases of Bayesian networks.

===Cyclic Directed Graphical Models===

[[File:Graph model.svg|thumb|right|alt=An example of a directed graphical model.| An example of a directed, cyclic graphical model. Each arrow indicates a dependency. In this example: D depends on A, B, and C; and C depends on B and D; whereas A and B are each independent.]]

The next figure depicts a graphical model with a cycle. This may be interpreted in terms of each variable 'depending' on the values of its parents in some manner.

withThe particular graph shown suggests a joint probability density that factors as

:<math>P[A,B,C,D] = P[A]\cdot P[B]\cdot P[C,D|A,B]</math>,

but other interpretations are possible.

<ref name=richardson95causal>{{cite journal

|first=Thomas |last=Richardson

|title=A discovery algorithm for directed cyclic graphs

|book-title=Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence

|year=1996}}

</ref>

===Other types===