Time hierarchy theorem: Difference between revisions

Revision as of 03:31, 13 April 2004

Latest revision as of 19:58, 12 August 2025

(177 intermediate revisions by 88 users not shown)

Line 1:

{{short description|Given more time, a Turing machine can solve more problems}}

In [[computational complexity theory]], the '''time hierarchy theorems''' are important statements that ensure the existence of certain "hard" problems which cannot be solved in a given amount of time. As a consequence, the run-time hierarchy of problems does not completely collapse. One theorem deals with deterministic computations and the other with non-deterministic ones.

In [[computational complexity theory]], the '''time hierarchy theorems''' are important statements about time-bounded computation on [[Turing machine]]s. Informally, these theorems say that given more time, a Turing machine can solve more problems. For example, there are problems that can be solved with ''n''² time but not ''n'' time, where ''n'' is the input length.

The time hierarchy theorem for [[Turing machine|deterministic multi-tape Turing machines]] was first proven by [[Richard E. Stearns]] and [[Juris Hartmanis]] in 1965.<ref>{{Cite journal

Both theorems use the notion of a '''time-constructible function'''. A [[function (mathematics)|function]] ''f'' : '''N''' &rarr; '''N''' is time-constructible if there exists a deterministic [[Turing machine]] such that for every ''n'' in '''N''', if the machine is started with an input of ''n'' ones, it will halt after precisely ''f''(''n'') steps. All [[polynomial]]s with non-negative integral coefficients are time-constructible, as are exponential functions such as 2^''n''.

| last1 = Hartmanis | first1 = J. | author1-link = Juris Hartmanis

| last2 = Stearns | first2 = R. E. | author2-link = Richard E. Stearns

| doi = 10.2307/1994208

| journal = [[Transactions of the American Mathematical Society]]

| pages = 285–306

| title = On the computational complexity of algorithms

| volume = 117

| date = 1 May 1965

| issn = 0002-9947

| publisher = American Mathematical Society

| mr = 0170805

| jstor = 1994208| doi-access = free

}}

</ref> It was improved a year later when F. C. Hennie and Richard E. Stearns improved the efficiency of the [[Universal Turing machine#Efficiency|universal Turing machine]].<ref>{{cite journal

| last1 = Hennie

| first1 = F. C.

| last2 = Stearns

| first2 = R. E.

| author-link2 = Richard E. Stearns

|date=October 1966

| title = Two-Tape Simulation of Multitape Turing Machines

| journal = J. ACM

| volume = 13

| issue = 4

| pages = 533–546

| ___location = New York, NY, USA

| publisher = ACM

| issn = 0004-5411

| doi = 10.1145/321356.321362| s2cid = 2347143

| doi-access= free

}}</ref> Consequent to the theorem, for every deterministic time-bounded [[complexity class]], there is a strictly larger time-bounded complexity class, and so the time-bounded hierarchy of complexity classes does not completely collapse. More precisely, the time hierarchy theorem for deterministic Turing machines states that for all [[constructible function|time-constructible function]]s ''f''(''n''),

:<math>\mathsf{DTIME}\left(o\left(f(n)\right)\right) \subsetneq \mathsf{DTIME}(f(n){\log f(n)})</math>,

where [[DTIME]](''f''(''n'')) denotes the complexity class of [[decision problem]]s solvable in time [[big O notation|O]](''f''(''n'')). The left-hand class involves [[little o]] notation, referring to the set of decision problems solvable in asymptotically '''less''' than ''f''(''n'') time.

In particular, this shows that <math>\mathsf{DTIME}(n^a) \subsetneq \mathsf{DTIME}(n^b)</math> if and only if <math>a < b</math>, so we have an infinite time hierarchy.

== Deterministic time hierarchy theorem ==

The time hierarchy theorem for [[nondeterministic Turing machine]]s was originally proven by [[Stephen Cook]] in 1972.<ref>{{cite conference

| title = A hierarchy for nondeterministic time complexity

| first = Stephen A.

| last = Cook

| author-link = Stephen Cook

| year = 1972

| conference = STOC '72

| book-title = Proceedings of the fourth annual ACM symposium on Theory of computing

| publisher = ACM

| ___location = Denver, Colorado, United States

| pages = 187–192

| doi = 10.1145/800152.804913| doi-access= free

}}</ref> It was improved to its current form via a complex proof by Joel Seiferas, [[Michael J. Fischer|Michael Fischer]], and [[Albert R. Meyer|Albert Meyer]] in 1978.<ref>{{cite journal

| last1 = Seiferas

| first1 = Joel I.

| last2 = Fischer

| first2 = Michael J.

| author-link2 = Michael J. Fischer

| last3 = Meyer

| first3 = Albert R.

| author-link3 = Albert R. Meyer

|date=January 1978

| title = Separating Nondeterministic Time Complexity Classes

| journal = J. ACM

| volume = 25

| issue = 1

| pages = 146–167

| ___location = New York, NY, USA

| publisher = ACM

| issn = 0004-5411

| doi = 10.1145/322047.322061| s2cid = 13561149

| doi-access= free

}}</ref> Finally in 1983, Stanislav Žák achieved the same result with the simple proof taught today.<ref>{{cite journal

| first1 = Stanislav

| last1 = Žák

|date=October 1983

| title = A Turing machine time hierarchy

| journal = Theoretical Computer Science

| volume = 26

| issue = 3

| pages = 327–333

| publisher = Elsevier Science B.V.

| doi = 10.1016/0304-3975(83)90015-4| doi-access= free

}}</ref> The time hierarchy theorem for nondeterministic Turing machines states that if ''g''(''n'') is a time-constructible function, and ''f''(''n''+1) = [[Little O notation|o]](''g''(''n'')), then

:<math>\mathsf{NTIME}(f(n)) \subsetneq \mathsf{NTIME}(g(n))</math>.

The analogous theorems for space are the [[space hierarchy theorem]]s. A similar theorem is not known for time-bounded probabilistic complexity classes, unless the class also has one bit of [[advice (complexity)|advice]].<ref>{{Cite book|doi=10.1109/FOCS.2004.33|title=45th Annual IEEE Symposium on Foundations of Computer Science|year=2004|author=Fortnow, L.|pages=316|last2=Santhanam|first2=R.|chapter=Hierarchy Theorems for Probabilistic Polynomial Time|isbn=0-7695-2228-9|s2cid=5555450}}</ref>

==Background==

Both theorems use the notion of a [[constructible function|time-constructible function]]. A [[function (mathematics)|function]] <math>f:\mathbb{N}\rightarrow\mathbb{N}</math> is time-constructible if there exists a deterministic [[Turing machine]] such that for every <math>n\in\mathbb{N}</math>, if the machine is started with an input of ''n'' ones, it will halt after precisely ''f''(''n'') steps. All [[polynomial]]s with non-negative integer coefficients are time-constructible, as are exponential functions such as 2^''n''.

==Proof overview==

We need to prove that some time class '''TIME'''(''g''(''n'')) is strictly larger than some time class '''TIME'''(''f''(''n'')). We do this by constructing a machine which cannot be in '''TIME'''(''f''(''n'')), by [[Cantor's diagonal argument|diagonalization]]. We then show that the machine is in '''TIME'''(''g''(''n'')), using a [[Simulation#Computer science|simulator machine]].

==Deterministic time hierarchy theorem==

===Statement===

<blockquote>'''Time Hierarchy Theorem.''' If ''f''(''n'') is a time-constructible function, then there exists a [[decision problem]] which cannot be solved in worst-case deterministic time ''o''(''f''(''n'')) but can be solved in worst-case deterministic time ''O''(''f''(''n'')log ''f''(''n'')). Thus

:<math>\mathsf{DTIME}(o(f(n))) \subsetneq \mathsf{DTIME}\left (f(n)\log f(n) \right).</math>

Equivalently, if <math>f, g</math> are time-constructable, and <math>f(n) \ln f(n) = o(g(n))</math>, then

<math display="block">\mathsf{DTIME}(f(n)) \subsetneq \mathsf{DTIME} (g(n))</math></blockquote>

If ''f''(''n'') is a time-constructible function, then there exists a [[decision problem]] which cannot be solved in worst-case deterministic time ''f''(''n'') but can be solved in worst-case deterministic time ''f''(''n'')². In other words, the complexity class TIME(''f''(''n'')) is a strict subset of TIME(''f''(''n'')²).

'''Note 1.''' ''f''(''n'') is at least ''n'', since smaller functions are never time-constructible.<br>

'''Example.''' <math>\mathsf{DTIME}(n) \subsetneq \mathsf{DTIME} (n (\ln n)^2) </math>.

===Proof===

We include here a proof of a weaker result, namely that '''DTIME'''(''f''(''n'')) is a strict subset of '''DTIME'''(''f''(2''n'' + 1)³), as it is simpler but illustrates the proof idea. See the bottom of this section for information on how to extend the proof to ''f''(''n'')log''f''(''n'').

To prove this, we first define the language of the encodings of machines and their inputs which cause them to halt within ''f''(|''x''|) steps:

We include here a proof that TIME(''f''(''n'')) is a strict subset of TIME(''f''(2''n'' + 1)³) as it is simpler. See the bottom of this section for information on how to extend the proof to ''f''(''n'')².

: <math> H_f = \left\{ ([M], x)\ |\ M \ \text{accepts}\ x \ \text{in}\ f(|x|) \ \text{steps} \right\}. </math>

To prove this, we first define a language as follows:

Notice here that this is a time-class. It is the set of pairs of machines and inputs to those machines (''M'',''x'') so that the machine ''M'' accepts within ''f''(|''x''|) steps.

: <math> H_f = \left\{ ([M], x)\ |\ M \ \mbox{accepts}\ x \ \mbox{in}\ f(|x|) \ \mbox{steps} \right\} </math>

Here, ''M'' is a deterministic Turing machine, and ''x'' is its input (the initial contents of its tape). [''M''] denotes an input that encodes the Turing machine ''M''. Let ''m'' be the size of the tuple ([''M''], ''x'').

We know that we can decide membership of ''H_f'' by way of a deterministic Turing machine ''R'', that simulates ''M'' for ''f''(''x'') steps by first calculatescalculating ''f''(|''x''|), and then writeswriting out a row of 0s of that length, and then usesusing this row of 0s as a "clock" or "counter" to simulate ''M'' for at most that many steps. At each step, the simulating machine needs to look through the definition of ''M'' to decide what the next action would be. It is safe to say that this takes at most ''f''(''m'')³ operations (since it is known that a simulation of a machine of time complexity ''T''(''n'') for can be achieved in time <math>O(T(n)\cdot|M|)</math> on a multitape machine, sowhere |''M''| is the length of the encoding of ''M''), we have that:

: <math> H_f \in \mathsf{TIME}\left(f(m)^3\right). </math>

The rest of the proof will show that

: <math> H_f \notin \mathsf{TIME}\left(f\left( \left\lfloor \frac{m/}{2} \right\rfloor \right)\right) </math>

so that if we substitute 2''n'' + 1 for ''m'', we get the desired result. Let us assume that ''H_f'' is in this time complexity class, and we will attempt to reach a contradiction.

If ''H_f'' is in this time complexity class, itthen meansthere weexists can construct somea machine ''K'' which, given some machine description [''M''] and input ''x'', decides whether the tuple ([''M''], ''x'') is in ''H_f'' within <math> \mathsf{TIME}(f( \left\lfloor m/2 \right\rfloor )) </math>.

:<math>\mathsf{TIME}\left(f\left( \left\lfloor \frac{m}{2} \right\rfloor \right)\right). </math>

Therefore we can use this ''K'' to construct another machine, ''N'', which takes a machine description [''M''] and runs ''K'' on the tuple ([''M''], [''M'']), and then accepts only if ''K'' rejects, and rejects if ''K'' accepts. If now ''n'' is the length of the input to ''N'', then ''m'' (the length of the input to ''K'') is twice ''n'' plus some delimiter symbol, so ''m'' = 2''n'' + 1. ''N''&#39;s running time is thus <math> \mathsf{TIME}(f( \left\lfloor m/2 \right\rfloor )) = \mathsf{TIME}(f( \left\lfloor (2n+1)/2 \right\rfloor )) = \mathsf{TIME}(f(n)) </math>.

NowWe ifuse wethis feed [''NK''] asto inputconstruct intoanother machine, ''N'', itself (which makestakes a machine description [''nM''] and runs ''K'' on the lengthtuple of([''M''], [''NM'']), andie. askM theis simulated on its own questioncode whetherby ''K'', and then ''N'' accepts itsif own''K'' descriptionrejects, asand input,rejects weif get:''K'' accepts.

* If ''N'' '''accepts''' [''N''] (which we know it does in at most ''f''(''n'') operations), this means that ''K'' '''rejects''' ([''N''], [''N'']), so ([''N'', ''N'']) is not in ''H_f'', and thus ''N'' does not accept [''N''] in ''f''(''n'') steps. Contradiction!

* If ''Nn'' '''rejects'''is [''N'']the (whichlength weof knowthe itinput does in at mostto ''fN''(, then ''nm'') operations),(the thislength meansof thatthe input to ''K'') '''accepts'''is ([''N''],twice [''Nn'']), soplus some ([''N''delimiter symbol, ''N''])so '''is'm'' in= 2''H_fn'', and+ thus1. ''N'' '''does'''s acceptrunning [''N'']time inis ''f''(''n'') steps. Contradiction!thus

:<math> \mathsf{TIME}\left(f\left( \left\lfloor \frac{m}{2} \right\rfloor \right)\right) = \mathsf{TIME}\left(f\left( \left\lfloor \frac{2n+1}{2} \right\rfloor \right)\right) = \mathsf{TIME}\left(f(n)\right). </math>

Now if we feed [''N''] as input into ''N'<nowiki/>'' and ask the question whether ''N'' accepts its ''N''' description as input, we get:

* If ''N'' accepts' [''N'<nowiki/>'''''<nowiki/>''' ] (which we know it does in at most f(n) operations since K halts on ([''N'' ], [''N'<nowiki/>'''''<nowiki/>''']) in f(n) steps), this means that K rejects ([''N'<nowiki/>''], [''N'<nowiki/>'']), so ([''N''], [''N'<nowiki/>'']) is not in ''H_f'', and so by the definition of ''H_f'', this implies that ''N'' does not accept [''N'<nowiki/>''] in ''f''(''n'') steps. Contradiction.

* If N rejects' [''N'<nowiki/>''] (which we know it does in at most f(n) operations), this means that K accepts ([''N'<nowiki/>''], [''N'<nowiki/>'''''<nowiki/>''']), so ([''N'' ], [''N'<nowiki/>'']) is in ''H_f'', and thus ''N'' does accept [''N'<nowiki/>''] in ''f''(''n'') steps. Contradiction.

We thus conclude that the machine ''K'' does not exist, and so

: <math> H_f \notin \mathsf{TIME}\left(f\left( \left\lfloor \frac{m/}{2} \right\rfloor \right)\right). </math>

===Extension===

The reader may have realised that the proof gives the weaker result because we have chosen a simple Turing machine simulation for which we know that

: <math> H_f \in \mathsf{TIME}(f(m)^3). </math>

It is known<ref>{{cite book |last1=Sipser |first1=Michael |title=Introduction to the Theory of Computation |date=27 June 2012 |publisher=CENGAGE learning |isbn=978-1-133-18779-0 |edition=3rd}}</ref> that a more efficient simulation exists which establishes that

: <math> H_f \in \mathsf{TIME}(f(m) \log f(m)) </math>.

==Non-deterministic time hierarchy theorem==

If ''g''(''n'') is a time-constructible function, and ''f''(''n''+1) = [[Big O notation|o]](''g''(''n'')), then there exists a decision problem which cannot be solved in non-deterministic time ''f''(''n'') but can be solved in non-deterministic time ''g''(''n''). In other words, the complexity class '''[[NTIME]]'''(''f''(''n'')) is a strict subset of '''NTIME'''(''g''(''n'')).

==Consequences==

The time hierarchy theorems guarantee that the deterministic and non-deterministic versions of the [[exponential hierarchy]] are genuine hierarchies: in other words '''[[P (complexity)|P]]''' ⊊ '''[[EXPTIME]]''' ⊊ '''[[2-EXP]]''' ⊊ ... and '''[[NP (complexity)|NP]]''' ⊊ '''[[NEXPTIME]]''' ⊊ '''2-NEXP''' ⊊ ....

For example, <math>\mathsf{P} \subsetneq \mathsf{EXPTIME}</math> since <math>\mathsf{P} \subseteq \mathsf{DTIME} (2^n)\subsetneq \mathsf{DTIME} (2^{2n}) \subseteq \mathsf{EXPTIME}</math>. Indeed, <math>\mathsf{DTIME}\left(2^n\right) \subseteq \mathsf{DTIME}\left(o\left(\frac{2^{2n}}{2n}\right)\right) \subsetneq \mathsf{DTIME}(2^{2n})</math> from the time hierarchy theorem.

The theorem also guarantees that there are problems in '''P''' requiring arbitrarily large exponents to solve; in other words, '''P''' does not collapse to '''DTIME'''(''n''^''k'') for any fixed ''k''. For example, there are problems solvable in ''n''⁵⁰⁰⁰ time but not ''n''⁴⁹⁹⁹ time. This is one argument against [[Cobham's thesis]], the convention that '''P''' is a practical class of algorithms. If such a collapse did occur, we could deduce that '''P''' ≠ '''PSPACE''', since it is a well-known theorem that '''DTIME'''(''f''(''n'')) is strictly contained in '''DSPACE'''(''f''(''n'')).

The reader may have realised that the proof is simpler because we have chosen a simple Turing machine simulation for which we can be certain that

: <math> H_f \in \mathsf{TIME}(f(m)^3) </math>

However, the time hierarchy theorems provide no means to relate deterministic and non-deterministic complexity, or time and space complexity, so they cast no light on the great unsolved questions of [[computational complexity theory]]: whether [[P = NP problem|'''P''' and '''NP''']], '''NP''' and '''[[PSPACE]]''', '''PSPACE''' and '''EXPTIME''', or '''EXPTIME''' and '''NEXPTIME''' are equal or not.

It is possible to find a provably more efficient model of simulation that establishes that

: <math> H_f \in \mathsf{TIME}(f( \left\lfloor m/2 \right\rfloor )^2) </math>

==Sharper hierarchy theorems==

but since this model of simulation is rather involved, it is not included here.

The gap of approximately <math>\log f(n)</math> between the lower and upper time bound in the hierarchy theorem can be traced to the efficiency of the device used in the proof, namely a universal program that maintains a step-count. This can be done more efficiently on certain computational models. The sharpest results, presented below, have been proved for:

* The unit-cost [[random-access machine]]<ref>{{cite journal |last1=Sudborough |first1=Ivan H. |last2=Zalcberg |first2=A. |title=On Families of Languages Defined by Time-Bounded Random Access Machines |journal=SIAM Journal on Computing |date=1976 |volume=5 |issue=2 |pages=217–230 |doi=10.1137/0205018}}</ref>

* A [[programming language]] model whose programs operate on a binary tree that is always accessed via its root. This model, introduced by [[Neil D. Jones]]<ref>{{cite book |last1=Jones |first1=Neil D. |title=Proceedings of the twenty-fifth annual ACM symposium on Theory of computing - STOC '93 |chapter=Constant time factors ''do'' matter |date=1993 |pages=602–611 |doi=10.1145/167088.167244|isbn=0-89791-591-7 |s2cid=7527905 }}</ref> is stronger than a deterministic Turing machine but weaker than a random-access machine.

For these models, the theorem has the following form:

<blockquote>If ''f''(''n'') is a time-constructible function, then there exists a decision problem which cannot be solved in worst-case deterministic time ''f''(''n'') but can be solved in worst-case time ''af''(''n'') for some constant ''a'' (dependent on ''f'').</blockquote>

Thus, a constant-factor increase in the time bound allows for solving more problems, in contrast with the situation for Turing machines (see [[Linear speedup theorem]]). Moreover, Ben-Amram proved<ref>{{cite journal |last1=Ben-Amram |first1=Amir M. |title=Tighter constant-factor time hierarchies |journal=Information Processing Letters |date=2003 |volume=87 |issue=1 |pages=39–44|doi=10.1016/S0020-0190(03)00253-9 }}</ref> that, in the above models, for ''f'' of polynomial growth rate (but more than linear), it is the case that for all <math>\varepsilon > 0</math>, there exists a decision problem which cannot be solved in worst-case deterministic time ''f''(''n'') but can be solved in worst-case time <math>(1+\varepsilon)f(n)</math>.

===Consequences=See also==

* [[Space hierarchy theorem]]

==References==

As a consequence, one can show that the class [[Complexity classes P and NP|'''P''']] is a strict subset of [[EXPTIME]], which is defined as:

{{Reflist}}

==Further reading==

: <math> \mathsf{EXPTIME} = \bigcup_{k=1}^\infty \mathsf{TIME} (2^{n^k}) </math>

* {{Cite book | author = Michael Sipser | year = 1997 | title = Introduction to the Theory of Computation | publisher = PWS Publishing | isbn = 0-534-94728-X | url-access = registration | url = https://archive.org/details/introductiontoth00sips | author-link = Michael Sipser }} Pages 310&ndash;313 of section 9.1: Hierarchy theorems.

* {{Cite book|author = Christos Papadimitriou | year = 1993 | title = Computational Complexity | publisher = Addison Wesley | edition = 1st | isbn = 0-201-53082-1| author-link = Christos Papadimitriou }} Section 7.2: The Hierarchy Theorem, pp.&nbsp;143&ndash;146.

{{Use dmy dates|date=September 2019}}

== Non-deterministic time hierarchy theorem ==

{{DEFAULTSORT:Time Hierarchy Theorem}}

If ''g''(''n'') is a time-constructible function, and ''f''(''n''+1) = [[Big O notation|o]](''g''(''n'')), then there exists a decision problem which cannot be solved in non-deterministic time ''f''(''n'') but can be solved in non-deterministic time ''g''(''n''). In other words, the complexity class NTIME(''f''(''n'')) is a strict subset of NTIME(''g''(''n'')).

[[Category:Structural complexity theory]]

[[Category:Theorems in computational complexity theory]]

[[Category:Articles containing proofs]]