Markov decision process: Difference between revisions

Revision as of 11:15, 25 May 2025

Revision as of 02:40, 27 June 2025

Line 75:

====Value iteration====

In value iteration {{harv|Bellman|1957}}, which is also called [[backward induction]],

the <math>\pi</math> function is not used; instead, the value of <math>\pi(s)</math> is calculated within <math>V(s)</math> whenever it is needed. Substituting the calculation of <math>\pi(s)</math> into the calculation of <math>V(s)</math> gives the combined step;{{explain|reason=The derivation of the substituion is needed|date=July 2018}}:

:<math> V_{i+1}(s) := \max_a \left\{ \sum_{s'} P_a(s,s') \left( R_a(s,s') + \gamma V_i(s') \right) \right\}, </math>