Content deleted Content added
Erel Segal (talk | contribs) |
Erel Segal (talk | contribs) No edit summary |
||
| (52 intermediate revisions by 24 users not shown) | |||
Line 1:
{{Short description|Computing the fixed point of a function}}
'''Fixed-point computation''' refers to the process of computing an exact or approximate [[Fixed point (mathematics)|fixed point]] of a given function.<ref name=":1">{{Cite book |url=https://link.springer.com/book/10.1007/978-3-642-50327-6 |title=The Computation of Fixed Points and Applications |language=en |doi=10.1007/978-3-642-50327-6}}</ref> In its most common form, we are given a function ''f'' that satisfies the condition to the [[Brouwer fixed-point theorem]], that is: ''f'' is continuous and maps the unit [[N-cube|''n''-cube]] to itself. The [[Brouwer fixed-point theorem]] guarantees that ''f'' has a fixed point, but the proof is not constructive. Various algorithms have been devised for computing an approximate fixed point. Such algorithms are used in economics for computing a [[market equilibrium]], in game theory for computing a [[Nash equilibrium]], and more.▼
{{CS1 config|mode=cs1}}
▲'''Fixed-point computation''' refers to the process of computing an exact or approximate [[Fixed point (mathematics)|fixed point]] of a given function.<ref name=":1">{{
* [[Nash equilibrium computation]],
* [[Market equilibrium computation]],
* [[Dynamic system]] analysis.
== Definitions ==
[[File:Fixed point example.svg|alt=an example function with three fixed points|thumb|The graph of an example function with three fixed points]]
The unit interval is denoted by
A '''fixed point''' of <math>f</math> is a point <math>x</math> in <math>E^d</math> such that <math>f(x) = x</math>. By the [[Brouwer fixed-point theorem]], any continuous function from <math>E^d</math> to itself has a fixed point. But for general functions, it is impossible to compute a fixed point precisely, since it can be an arbitrary real number. Fixed-point computation algorithms look for ''approximate'' fixed points. There are several criteria for an approximate fixed point. Several common criteria are:<ref name=":3">{{cite journal |last1=Shellman |first1=Spencer |last2=Sikorski |first2=K. |title=A recursive algorithm for the infinity-norm fixed point problem |journal=Journal of Complexity |date=December 2003 |volume=19 |issue=6 |pages=799–834 |doi=10.1016/j.jco.2003.06.001 |doi-access=free }}</ref>
* The '''absolute criterion''': given an approximation parameter <math>\delta>0</math>, A '''δ-absolute fixed-point of''' '''<math>f</math>''' is a point <math>x</math> in <math>E^d</math> such that <math>|x-x_0| \leq \delta</math>, where <math>x_0</math> is any fixed-point of <math>f</math>.
* The '''relative criterion''': given an approximation parameter <math>\delta>0</math>, A '''δ-relative fixed-point of''' '''<math>f</math>''' is a point ''x'' in <math>E^d</math> such that <math>|x-x_0|/|x_0|\leq \delta</math>, where <math>x_0</math> is any fixed-point of <math>f</math>.
For Lipschitz-continuous functions, the absolute criterion is stronger than the residual criterion: If <math>f</math> is Lipschitz-continuous with constant <math>L</math>, then <math>|x-x_0|\leq \delta</math> implies <math>|f(x)-f(x_0)|\leq L\cdot \delta</math>. Since <math>x_0</math> is a fixed-point of <math>f</math>, this implies <math>|f(x)-x_0|\leq L\cdot \delta</math>, so <math>|f(x)-x|\leq (1+L)\cdot \delta</math>. Therefore, a δ-absolute fixed-point is also an {{mvar|ε}}-residual fixed-point with <math>\varepsilon = (1+L)\cdot \delta</math>.
The most basic step of a fixed-point computation algorithm is a '''value query''': given any <math>x</math> in <math>E^d</math>, the algorithm is provided with an oracle <math>\tilde{f}</math> to <math>f</math> that returns the value <math>f(x)</math>. The accuracy of the approximate fixed-point depends upon the error in the oracle <math>\tilde{f}(x)</math>.
=== One dimension ===▼
For a 1-dimensional function (''d''=1), a δ-near fixed-point can be found using <math>O(\log(1/\delta))</math> queries using the [[bisection method]]: start with the interval E := [0,1]; at each iteration, let ''x'' be the center of the current interval, and compute ''f''(''x''); if ''f''(''x'')>''x'' then recurse on the sub-interval to the right of ''x''; otherwise, recurse on the interval to the left of ''x''. Note that the current interval always contains a fixed point, so after <math>O(\log(1/\delta))</math> queries, any point in the remaining interval is a δ-near fixed-point of ''f.'' Setting <math>\delta := \varepsilon/(L+1)</math>, where ''L'' is the Lipschitz constant, gives an {{mvar|ε}}-fixed-point, using <math>O(\log(L/\varepsilon) = \log(L) + \log(1/\varepsilon))</math> queries.<ref name=":0" />▼
The function <math>f</math> is accessible via '''evaluation''' queries: for any <math>x</math>, the algorithm can evaluate <math>f(x)</math>. The run-time complexity of an algorithm is usually given by the number of required evaluations.
=== Two or more dimensions: algorithms ===▼
== Contractive functions ==
* The first algorithm to approximate a fixed point was developed by [[Herbert Scarf]] in 1967.<ref>{{Cite journal |last=Scarf |first=Herbert |date=1967-09-01 |title=The Approximation of Fixed Points of a Continuous Mapping |url=http://epubs.siam.org/doi/10.1137/0115116 |journal=SIAM Journal on Applied Mathematics |language=en |volume=15 |issue=5 |pages=1328–1343 |doi=10.1137/0115116 |issn=0036-1399}}</ref><ref>H. Scarf found the first algorithmic proof: {{SpringerEOM|title=Brouwer theorem|first=M.I.|last=Voitsekhovskii|isbn=1-4020-0609-8}}.</ref> Scarf's algorithm finds an approximate fixed point by finding a fully-labelled "primitive set", in a construction similar to [[Sperner's lemma]]. A subtle aspect of Scarf's algorithm is that it finds an {{mvar|ε}}-fixed-point of ''f'', that is, a point that is {{em|almost fixed}} by a function ''f'', but in general cannot find a δ-near fixed-point of ''f,'' that is, ''a'' point that is close to an actual fixed point. ▼
A Lipschitz-continuous function with constant <math>L</math> is called '''[[contractive]]''' if <math>L<1</math>; it is called '''[[weakly-contractive]]''' if <math>L\le 1</math>. Every contractive function satisfying Brouwer's conditions has a ''unique'' fixed point. Moreover, fixed-point computation for contractive functions is easier than for general functions.
* A later algorithm by [[Harold W. Kuhn|Harold Kuhn]]<ref>{{Cite journal |last=Kuhn |first=Harold W. |date=1968 |title=Simplicial Approximation of Fixed Points |url=https://www.jstor.org/stable/58762 |journal=Proceedings of the National Academy of Sciences of the United States of America |volume=61 |issue=4 |pages=1238–1242 |issn=0027-8424}}</ref> used simplices and simplicial partitions instead of primitive sets.▼
[[File:Fixed point anime.gif|alt=computing a fixed point using function iteration|thumb|Computing a fixed point using function iteration]]
* Developing the simplicial approach further, Orin Harrison Merrill<ref>{{Cite web |title=APPLICATIONS AND EXTENSIONS OF AN ALGORITHM THAT COMPUTES FIXED POINTS OFCERTAIN UPPER SEMI-CONTINUOUS POINT TO SET MAPPINGS - ProQuest |url=https://www.proquest.com/openview/9bd010ff744833cb3a23ef521046adcb/1?pq-origsite=gscholar&cbl=18750&diss=y |access-date=2023-04-13 |website=www.proquest.com |language=en}}</ref> presented the ''restart algorithm''.▼
The first algorithm for fixed-point computation was the '''[[fixed-point iteration]]''' algorithm of Banach. [[Banach fixed point theorem|Banach's fixed-point theorem]] implies that, when fixed-point iteration is applied to a contraction mapping, the error after <math>t</math> iterations is in <math>O(L^t)</math>. Therefore, the number of evaluations required for a <math>\delta</math>-relative fixed-point is approximately <math>\log_L(\delta) = \log(\delta)/\log(L) = \log(1/\delta)/\log(1/L) </math>. Sikorski and Wozniakowski<ref name=":5">{{cite journal |last1=Sikorski |first1=K |last2=Woźniakowski |first2=H |title=Complexity of fixed points, I |journal=Journal of Complexity |date=December 1987 |volume=3 |issue=4 |pages=388–405 |doi=10.1016/0885-064X(87)90008-2 |doi-access=free }}</ref> showed that Banach's algorithm is optimal when the dimension is large. Specifically, when <math>d\geq \log(1/\delta)/\log(1/L) </math>, the number of required evaluations of ''any'' algorithm for <math>\delta</math>-relative fixed-point is larger than 50% the number of evaluations required by the iteration algorithm. Note that when <math>L</math> approaches 1, the number of evaluations approaches infinity. No finite algorithm can compute a <math>\delta</math>-absolute fixed point for all functions with <math>L=1</math>.<ref name=":4">{{cite book |last1=Sikorski |first1=Krzysztof A. |title=Optimal Solution of Nonlinear Equations |date=2001 |publisher=Oxford University Press |isbn=978-0-19-510690-9 }}{{page needed|date=April 2023}}</ref>
* B. Curtis Eaves<ref>{{Cite journal |last=Eaves |first=B. Curtis |date=1972-12-01 |title=Homotopies for computation of fixed points |url=https://doi.org/10.1007/BF01584975 |journal=Mathematical Programming |language=en |volume=3 |issue=1 |pages=1–22 |doi=10.1007/BF01584975 |issn=1436-4646}}</ref> presented the ''[[homotopy]] algorithm''. The algorithm works by starting with an affine function that approximates ''f'', and deforming it towards ''f'', while following the fixed point''.'' A book by Michael Todd<ref name=":1" /> surveys various algorithms developed until 1976.▼
* [[David Gale]]<ref>{{cite journal |author=David Gale |year=1979 |title=The Game of Hex and Brouwer Fixed-Point Theorem |journal=The American Mathematical Monthly |volume=86 |issue=10 |pages=818–827 |doi=10.2307/2320146 |jstor=2320146}}</ref> showed that computing a fixed point of an ''n''-dimensional function (on the unit ''d''-dimensional cube) is equivalent to deciding who is the winner in an ''d''-dimensional game of [[Hex (board game)|Hex]] (a game with ''d'' players, each of whom needs to connect two opposite faces of an ''d''-cube). Given the desired accuracy ''{{mvar|ε}}''▼
** Construct a Hex board of size ''kd'', where ''k''>1/''{{mvar|ε}}''. Each vertex ''z'' corresponds to a point ''z''/''k'' in the unit ''n''-cube.▼
** Compute the difference ''f''(''z''/''k'')-''z''/''k''; note that the difference is an ''n''-vector.▼
** Label the vertex ''z'' by a label in 1,...,''d'', denoting the largest coordinate in the difference vector.▼
** The resulting labeling corresponds to a possible play of the ''d''-dimensional Hex game among ''d'' players. This game must have a winner, and Gale presents an algorithm for constructing the winning path.▼
** In the winning path, there must be a point in which ''f<sub>i</sub>''(''z''/''k'')-''z''/''k'' is positive, and an adjacent point in which ''f<sub>i</sub>''(''z''/''k'')-''z''/''k'' is negative. This means that there is a fixed point of f between these two points.▼
In the worst case, the number of function evaluations required by all these algorithms is exponential, that is, <math>\Omega(1/\varepsilon)</math>.▼
When <math>L</math> < 1 and ''d'' = 1, the optimal algorithm is the '''Fixed Point Envelope''' (FPE) algorithm of Sikorski and Wozniakowski.<ref name=":5" /> It finds a ''δ''-relative fixed point using <math>O(\log(1/\delta) + \log \log(1/(1-L))) </math> queries, and a ''δ''-absolute fixed point using <math>O(\log(1/\delta)) </math> queries. This is faster than the fixed-point iteration algorithm.<ref>{{cite book |doi=10.1007/978-1-4615-9552-6_4 |chapter=Fast Algorithms for the Computation of Fixed Points |title=Robustness in Identification and Control |year=1989 |last1=Sikorski |first1=K. |pages=49–58 |isbn=978-1-4615-9554-0 }}</ref>
Hirsch, [[Christos Papadimitriou|Papadimitriou]] and Vavasis proved that<ref name=":0">{{Cite journal |last=Hirsch |first=Michael D |last2=Papadimitriou |first2=Christos H |last3=Vavasis |first3=Stephen A |date=1989-12-01 |title=Exponential lower bounds for finding Brouwer fix points |url=https://www.sciencedirect.com/science/article/pii/0885064X89900174 |journal=Journal of Complexity |language=en |volume=5 |issue=4 |pages=379–416 |doi=10.1016/0885-064X(89)90017-4 |issn=0885-064X}}</ref> ''any'' algorithm based on function evaluations, that finds an {{mvar|ε}}-fixed-point of ''f,'' requires <math>\Omega(L'/\varepsilon)</math> function evaluations, where <math>L'</math> is the Lipschitz constant of the function <math>f(x)-x</math> (note that <math>L-1 \leq L' \leq L+1</math>). More precisely:▼
When <math>d>1</math> but not too large, and <math>L\le 1</math>, the optimal algorithm is the interior-ellipsoid algorithm (based on the [[ellipsoid method]]).<ref>{{cite journal |last1=Huang |first1=Z |last2=Khachiyan |first2=L |last3=Sikorski |first3=K |title=Approximating Fixed Points of Weakly Contracting Mappings |journal=Journal of Complexity |date=June 1999 |volume=15 |issue=2 |pages=200–213 |doi=10.1006/jcom.1999.0504 |doi-access=free }}</ref> It finds an {{mvar|ε}}-residual fixed-point using <math>O(d\cdot \log(1/\varepsilon)) </math> evaluations. When <math>L<1</math>, it finds a <math>\delta</math>-absolute fixed point using <math>O(d\cdot [\log(1/\delta) + \log(1/(1-L))]) </math> evaluations.
* For a 2-dimensional function (''d''=2), they prove a tight bound <math>\Theta(L'/\varepsilon)</math>.▼
* For any d ≥ 3, finding an {{mvar|ε}}-fixed-point of a ''d''-dimensional function requires <math>\Omega((L'/\varepsilon)^{d-2})</math> queries and <math>O((L'/\varepsilon)^{d})</math> queries. ▼
The latter result leaves a gap in the exponent. Chen and Deng<ref name=":2" /> closed the gap. They proved that, for any ''d'' ≥ 2 and <math>1/\varepsilon > 4 d</math> and <math>L'/\varepsilon > 192 d^3</math>, the number of queries required for computing an {{mvar|ε}}-fixed-point is in <math>\Theta((L'/\varepsilon)^{d-1})</math>.▼
Shellman and Sikorski<ref name=":3" /> presented an algorithm called '''PFix''' for computing an {{mvar|ε}}-residual fixed-point of a ''d''-dimensional function with ''L ≤'' 1, using <math>O(\log^d(1/\varepsilon))</math> queries. When <math>L</math> < 1, '''PFix''' can be executed with <math>\varepsilon = (1-L)\cdot \delta</math>, and in that case, it computes a δ-absolute fixed-point, using <math>O(\log^d(1/[(1-L)\delta]))</math> queries. It is more efficient than the iteration algorithm when <math>L</math> is close to 1. The algorithm is recursive: it handles a ''d''-dimensional function by recursive calls on (''d''-1)-dimensional functions.
== Algorithms for contraction mappings ==▼
== General functions ==
▲* Shellman and Sikorski<ref>{{Cite journal |last=Shellman |first=Spencer |last2=Sikorski |first2=K. |date=2002-06-01 |title=A Two-Dimensional Bisection Envelope Algorithm for Fixed Points |url=https://www.sciencedirect.com/science/article/pii/S0885064X01906259 |journal=Journal of Complexity |language=en |volume=18 |issue=2 |pages=641–659 |doi=10.1006/jcom.2001.0625 |issn=0885-064X}}</ref> presented an algorithm called BEFix (Bisection Envelope Fixed-point) for computing an {{mvar|ε}}-fixed-point of a two-dimensional function with Lipschitz constant 1, using only <math>2 \lceil\log_2(1/\varepsilon)\rceil+1</math> queries. They later<ref>{{Cite journal |last=Shellman |first=Spencer |last2=Sikorski |first2=K. |date=2003-09-01 |title=Algorithm 825: A deep-cut bisection envelope algorithm for fixed points |url=https://doi.org/10.1145/838250.838255 |journal=ACM Transactions on Mathematical Software |volume=29 |issue=3 |pages=309–325 |doi=10.1145/838250.838255 |issn=0098-3500}}</ref> presented an improvement called BEDFix (Bisection Envelope Deep-cut Fixed-point), with the same worst-case guarantee but better empirical performance.
For functions with Lipschitz constant <math>L</math> > 1, computing a fixed-point is much harder.
▲=== One dimension ===
▲For a 1-dimensional function (''d'' = 1), a
For functions in two or more dimensions, the problem is much more challenging. Shellman and Sikorski<ref name=":3" /> proved that for any integers ''d'' ≥ 2 and <math>L</math> > 1, finding a δ-absolute fixed-point of ''d''-dimensional <math>L</math>-Lipschitz functions might require infinitely many evaluations. The proof idea is as follows. For any integer ''T'' > 1 and any sequence of ''T'' of evaluation queries (possibly adaptive), one can construct two functions that are Lipschitz-continuous with constant <math>L</math>, and yield the same answer to all these queries, but one of them has a unique fixed-point at (''x'', 0) and the other has a unique fixed-point at (''x'', 1). Any algorithm using ''T'' evaluations cannot differentiate between these functions, so cannot find a δ-absolute fixed-point. This is true for any finite integer ''T''.
Several algorithms based on function evaluations have been developed for finding an {{mvar|ε}}-residual fixed-point
▲* The first algorithm to approximate a fixed point of a general function was developed by [[Herbert Scarf]] in 1967.<ref>{{
▲* A later algorithm by [[Harold W. Kuhn|Harold Kuhn]]<ref>{{Cite journal |last=Kuhn |first=Harold W. |date=1968 |title=Simplicial Approximation of Fixed Points |
▲* Developing the simplicial approach further, Orin Harrison Merrill<ref>{{
▲* B. Curtis Eaves<ref>{{
* A book by Michael Todd<ref name=":1" /> surveys various algorithms developed until 1976.
▲* [[David Gale]]<ref>{{cite journal |
▲** Construct a Hex board of size ''kd'', where
▲** Compute the difference
▲** Label the vertex ''z'' by a label in 1, ..., ''d'', denoting the largest coordinate in the difference vector.
▲** The resulting labeling corresponds to a possible play of the ''d''-dimensional Hex game among ''d'' players. This game must have a winner, and Gale presents an algorithm for constructing the winning path.
▲** In the winning path, there must be a point in which ''f<sub>i</sub>''(''z''/''k'') - ''z''/''k'' is positive, and an adjacent point in which ''f<sub>i</sub>''(''z''/''k'') - ''z''/''k'' is negative. This means that there is a fixed point of <math>f</math> between these two points.
▲In the worst case, the number of function evaluations required by all these algorithms is exponential in the binary representation of the accuracy, that is, in <math>\Omega(1/\varepsilon)</math>.
==== Query complexity ====
▲Hirsch, [[Christos Papadimitriou|Papadimitriou]] and Vavasis proved that<ref name=":0">{{
▲* For a 2-dimensional function (''d''=2), they prove a tight bound <math>\Theta(L'/\varepsilon)</math>.
▲* For any d ≥ 3, finding an {{mvar|ε}}-residual fixed-point of a ''d''-dimensional function requires <math>\Omega((L'/\varepsilon)^{d-2})</math> queries and <math>O((L'/\varepsilon)^{d})</math> queries.
▲The latter result leaves a gap in the exponent. Chen and Deng<ref name=":2" /> closed the gap. They proved that, for any ''d'' ≥ 2 and <math>1/\varepsilon > 4 d</math> and <math>L'/\varepsilon > 192 d^3</math>, the number of queries required for computing an {{mvar|ε}}-residual fixed-point is in <math>\Theta((L'/\varepsilon)^{d-1})</math>.
== Discrete fixed-point computation ==
A '''discrete function''' is a function defined on a subset of ''<math>\mathbb{Z}^d</math>'' (the ''d''-dimensional integer grid). There are several [[
Let
Chen and Deng<ref>{{cite journal |last1=Chen |first1=Xi |last2=Deng |first2=Xiaotie |title=On the complexity of 2D discrete fixed point problem |journal=Theoretical Computer Science |date=October 2009 |volume=410 |issue=44 |pages=4448–4456 |doi=10.1016/j.tcs.2009.07.052 |s2cid=2831759 }}</ref> define a different discrete-fixed-point problem, which they call '''2D-BROUWER'''. It considers a discrete function <math>f</math> on <math>\{0,\dots, n\}^2</math> such that, for every ''x'' on the grid, <math>f</math>(''x'') - ''x'' is either (0, 1) or (1, 0) or (-1, -1). The goal is to find a square in the grid, in which all three labels occur. The function <math>f</math> must map the square <math>\{0,\dots, n\}^2</math>to itself, so it must map the lines ''x'' = 0 and ''y'' = 0 to either (0, 1) or (1, 0); the line ''x'' = ''n'' to either (-1, -1) or (0, 1); and the line ''y'' = ''n'' to either (-1, -1) or (1,0). The problem can be reduced to '''2D-SPERNER''' (computing a fully-labeled triangle in a triangulation satisfying the conditions to [[Sperner's lemma]]), and therefore it is [[PPAD-complete]]. This implies that computing an approximate fixed-point is PPAD-complete even for very simple functions.
== Relation between fixed-point computation and root-finding algorithms ==
Given a function
Fixed-point computation is a special case of root-finding: given a function
The opposite is not true: finding an approximate root of a general function may be harder than finding an approximate fixed point. In particular, Sikorski<ref>{{
However, there are classes of functions for which finding an approximate root is equivalent to finding an approximate fixed point. One example<ref name=":2">{{
== Communication complexity ==
Roughgarden and Weinstein<ref>{{
== References ==
{{reflist}}
== Further reading ==
* {{cite journal |last1=Yannakakis |first1=Mihalis |title=Equilibria, fixed points, and complexity classes |journal=Computer Science Review |date=May 2009 |volume=3 |issue=2 |pages=71–85 |doi=10.1016/j.cosrev.2009.03.004 |url=https://drops.dagstuhl.de/opus/volltexte/2008/1311/|arxiv=0802.2831 }}
[[Category:Fixed-point theorems]]
| |||