|
==Historical background==
The sampling theorem was implied by the work of [[Harry Nyquist]] in 1928,<ref>{{cite journal | authorlast=Nyquist, |first=Harry | author-link =Harry Nyquist | title =Certain topics in telegraph transmission theory | journal =Trans.Transactions of the AIEE | volume =47 | issue =2 | pages =617–644 | date =April 1928 | doi=10.1109/t-aiee.1928.5055024| bibcode =1928TAIEE..47..617N }} [http://www.ieee.org/publications_standards/publications/proceedings/nyquist.pdf Reprint as classic paper] in: ''Proc.Proceedings of the IEEE'', Vol. 90, No. 2, FebFebruary 2002]. {{webarchive|url=https://web.archive.org/web/20130926031230/http://www.ieee.org/publications_standards/publications/proceedings/nyquist.pdf |date=2013-09-26 }}</ref> in which he showed that up to 2''B''<math>2B</math> independent pulse samples could be sent through a system of bandwidth ''<math>B''</math>; but he did not explicitly consider the problem of sampling and reconstruction of continuous signals. About the same time, [[Karl Küpfmüller]] showed a similar result<ref>{{cite journal |first=Karl |last=Küpfmüller |title=Über die Dynamik der selbsttätigen Verstärkungsregler |journal=Elektrische Nachrichtentechnik |volume=5 |issue=11 |pages=459–467 |year=1928 |language=de}} [http://ict.open.ac.uk/classics/2.pdf (English translation 2005)].</ref> and discussed the sinc-function impulse response of a band-limiting filter, via its integral, the step-response [[sine integral]]; this bandlimiting and reconstruction filter that is so central to the sampling theorem is sometimes referred to as a ''Küpfmüller filter'' (but seldom so in English).
The sampling theorem, essentially a [[duality (mathematics)|dual]] of Nyquist's result, was proved by [[Claude E. Shannon]].<ref name="Shannon49"/> The mathematician [[E. T. Whittaker]] published similar results in 1915,<ref>{{cite journal |authorlast=Whittaker, |first=E. T. |author-link=E. T. Whittaker |title=On the Functions Which are Represented by the Expansions of the Interpolation Theory |journal=Proc.Proceedings R.of Soc.the Royal Society of Edinburgh |volume=35 |pages=181–194 |date=1915 |doi=10.1017/s0370164600017806|url=https://zenodo.org/record/1428702 }} ({{lang|de|"Theorie der Kardinalfunktionen"}}).</ref> J. M. Whittaker in 1935,<ref>{{cite book | authorlast=Whittaker, |first=J. M. | author-link =J. M. Whittaker | title =Interpolatory Function Theory | url=https://archive.org/details/in.ernet.dli.2015.219870 | publisher =Cambridge Univ.University Press | date =1935 | ___location =Cambridge, England}}.</ref> and [[Dennis Gabor|Gabor]] in 1946 ("Theory of communication").
In 1948 and 1949, Claude E. Shannon published the two revolutionary articles in which he founded the information theory.<ref>{{cite journal |ref=refShannon48 |authorlast=Shannon, |first=Claude E. |author-link=Claude Shannon |title=A Mathematical Theory of Communication |journal=Bell System Technical Journal |volume=27 |issue=3 |pages=379–423 |date=July 1948 |doi=10.1002/j.1538-7305.1948.tb01338.x|hdl=11858/00-001M-0000-002C-4317-B |hdl-access=free }}.</ref><ref>{{cite journal |ref=refShannon48oct |authorlast=Shannon, |first=Claude E. |author-link=Claude Shannon |title=A Mathematical Theory of Communication |journal=Bell System Technical Journal |volume=27 |issue=4 |pages=623–666 |date=October 1948 |doi=10.1002/j.1538-7305.1948.tb00917.x|hdl=11858/00-001M-0000-002C-4314-2 |hdl-access=free }}</ref><ref name="Shannon49"/> In [[#refShannon48oct|Shannon 1948]] the sampling theorem is formulated as “Theorem"Theorem 13”13": Let ''<math>f''(''t'')</math> contain no frequencies over W. Then
:<math display="block">f(t) = \sum_{n=-\infty}^\infty X_n \frac{\sin \pi(2Wt - n)}{\pi(2Wt - n)},</math> where <math>X_n = f\left(\frac n {2W} \right)</math>.
It was not until these articles were published that the theorem known as “Shannon’s sampling theorem” became common property among communication engineers, although Shannon himself writes that this is a fact which is common knowledge in the communication art.{{efn-ua|group=bottom|[[#refShannon49|Shannon 1949]], p. 448.}} A few lines further on, however, he adds: "but in spite of its evident importance, [it] seems not to have appeared explicitly in the literature of communication theory". ▼where <math>X_n = f\left(\frac n {2W} \right)</math>.
▲It was not until these articles were published that the theorem known as “Shannon’s"Shannon's sampling theorem”theorem" became common property among communication engineers, although Shannon himself writes that this is a fact which is common knowledge in the communication art.{{efn-ua|group=bottom|[[#refShannon49|Shannon 1949]], p. 448.}} A few lines further on, however, he adds: "but in spite of its evident importance, [it] seems not to have appeared explicitly in the literature of communication theory".
===Other discoverers===
Others who have independently discovered or played roles in the development of the sampling theorem have been discussed in several historical articles, for example, by Jerri<ref>{{cite journal | last=Jerri | first=Abdul | author-link=Abdul Jerri | title=The Shannon Sampling Theorem—Its Various Extensions and Applications: A Tutorial Review | journal=Proceedings of the IEEE | volume=65 | issue=11 | pages=1565–1596 | date=November 1977 | doi=10.1109/proc.1977.10771 | bibcode=1977IEEEP..65.1565J | s2cid=37036141 }} See also {{cite journal | last=Jerri | first=Abdul | title=Correction to "'The Shannon sampling theorem—Its various extensions and applications: A tutorial review"' | journal=Proceedings of the IEEE | volume=67 | issue=4 | page=695 | date=April 1979 | doi=10.1109/proc.1979.11307 }}</ref> and by Lüke.<ref>{{cite journal | last=Lüke | first=Hans Dieter | title =The Origins of the Sampling Theorem | journal =IEEE Communications Magazine | pages =106–108 | date =April 1999 | issue=4 | doi =10.1109/35.755459 | volume=37| url=http://www.hit.bme.hu/people/papay/edu/Conv/pdf/origins.pdf | citeseerx=10.1.1.163.2887 }}</ref> For example, Lüke points out that H. Raabe, an assistant to Küpfmüller, proved the theorem in his 1939 Ph.D. dissertation; the term ''Raabe condition'' came to be associated with the criterion for unambiguous representation (sampling rate greater than twice the bandwidth). Meijering<ref name="EM">{{cite journal | last =Meijering | first =Erik | title =A Chronology of Interpolation From Ancient Astronomy to Modern Signal and Image Processing | journal =Proc.Proceedings of the IEEE | volume =90 | issue =3 | pages =319–342 | date =March 2002 | doi =10.1109/5.993400 | url =http://bigwww.epfl.ch/publications/meijering0201.pdf }}</ref> mentions several other discoverers and names in a paragraph and pair of footnotes:
<blockquote>
As pointed out by Higgins [135], the sampling theorem should really be considered in two parts, as done above: the first stating the fact that a bandlimited function is completely determined by its samples, the second describing how to reconstruct the function using its samples. Both parts of the sampling theorem were given in a somewhat different form by J. M. Whittaker [350, 351, 353] and before him also by Ogura [241, 242]. They were probably not aware of the fact that the first part of the theorem had been stated as early as 1897 by Borel [25].<sup>27</sup> As we have seen, Borel also used around that time what became known as the cardinal series. However, he appears not to have made the link [135]. In later years it became known that the sampling theorem had been presented before Shannon to the Russian communication community by [[Vladimir Kotelnikov|Kotel'nikov]] [173]. In more implicit, verbal form, it had also been described in the German literature by Raabe [257]. Several authors [33, 205] have mentioned that Someya [296] introduced the theorem in the Japanese literature parallel to Shannon. In the English literature, Weston [347] introduced it independently of Shannon around the same time.<sup>28</sup> ▼
{{quote|
<sup>27</sup> Several authors, following Black [16], have claimed that this first part of the sampling theorem was stated even earlier by Cauchy, in a paper [41] published in 1841. However, the paper of Cauchy does not contain such a statement, as has been pointed out by Higgins [135].
▲As pointed out by Higgins [135], the sampling theorem should really be considered in two parts, as done above: the first stating the fact that a bandlimited function is completely determined by its samples, the second describing how to reconstruct the function using its samples. Both parts of the sampling theorem were given in a somewhat different form by J. M. Whittaker [350, 351, 353] and before him also by Ogura [241, 242]. They were probably not aware of the fact that the first part of the theorem had been stated as early as 1897 by Borel .{{refn|group= [25]Meijering|Several authors, following Black, have claimed that this first part of the sampling theorem was stated even earlier by Cauchy, in a paper published in 1841. However, the paper of Cauchy does not contain such a statement, as has been pointed out by Higgins. <sup>27</sup>}} As we have seen, Borel also used around that time what became known as the cardinal series. However, he appears not to have made the link [135]. In later years it became known that the sampling theorem had been presented before Shannon to the Russian communication community by [[Vladimir Kotelnikov|Kotel'nikov] ] [173]. In more implicit, verbal form, it had also been described in the German literature by Raabe [257]. Several authors [33, 205] have mentioned that Someya [296] introduced the theorem in the Japanese literature parallel to Shannon. In the English literature, Weston [347] introduced it independently of Shannon around the same time. <sup>28</sup>{{refn|group= Meijering|As a consequence of the discovery of the several independent introductions of the sampling theorem, people started to refer to the theorem by including the names of the aforementioned authors, resulting in such catchphrases as "the Whittaker–Kotel'nikov–Shannon (WKS) sampling theorem" or even "the Whittaker–Kotel'nikov–Raabe–Shannon–Someya sampling theorem". To avoid confusion, perhaps the best thing to do is to refer to it as the sampling theorem, "rather than trying to find a title that does justice to all claimants".}}
{{reflist|group= Meijering}}|Eric Meijering, "A Chronology of Interpolation From Ancient Astronomy to Modern Signal and Image Processing" (citations omitted)
<sup>28</sup> As a consequence of the discovery of the several independent introductions of the sampling theorem, people started to refer to the theorem by including the names of the aforementioned authors, resulting in such catchphrases as “the Whittaker–Kotel’nikov–Shannon (WKS) sampling theorem" [155] or even "the Whittaker–Kotel'nikov–Raabe–Shannon–Someya sampling theorem" [33]. To avoid confusion, perhaps the best thing to do is to refer to it as the sampling theorem, "rather than trying to find a title that does justice to all claimants" [136].
}}
</blockquote>
===Why Nyquist?===
Exactly how, when, or why [[Harry Nyquist]] had his name attached to the sampling theorem remains obscure. The term ''Nyquist Sampling Theorem'' (capitalized thus) appeared as early as 1959 in a book from his former employer, [[Bell Labs]],<ref>{{cite book | title = Transmission Systems for Communications | author = Members of the Technical Staff of Bell Telephone Lababoratories | year = 1959 | publisher = AT&T | pages = 26–4 (Vol.|volume=2)}}</ref>{{verification needed|reason=The page range "26–4" seems to mean "26–24" which makes no sense. If it is a single page numbered 26–4, use page, not pages|date=October 2023}} and appeared again in 1963,<ref>{{cite book | title = Theory of Linear Physical Systems | publisher = Wiley | year = 1963 | url = https://books.google.com/books?id=jtI-AAAAIAAJ |first=Ernst Adolph |last=Guillemin| isbn = 9780471330707 }}</ref> and not capitalized in 1965.<ref>{{cite book |first1=Richard A. |last1=Roberts |first2=Ben F. |last2=Barton |title=Theory of Signal Detectability: Composite Deferred Decision Theory |year=1965 }}</ref> It had been called the ''Shannon Sampling Theorem'' as early as 1954,<ref>{{cite journal |first=Truman S. |last=Gray |title=Applied Electronics: A First Course in Electronics, Electron Tubes, and Associated Circuits |journal=Physics Today |year=1954 |volume=7 |issue=11 |page=17 |doi=10.1063/1.3061438 |bibcode=1954PhT.....7k..17G |hdl=2027/mdp.39015002049487 |hdl-access=free }}</ref> but also just ''the sampling theorem'' by several other books in the early 1950s.
In 1958, Blackman and Tukey cited Nyquist's 1928 article as a reference for ''the sampling theorem of information theory'',<ref>{{cite book |first1=R. B. |last1=Blackman |first2=J. W. |last2=Tukey |title=The Measurement of Power Spectra : From the Point of View of Communications Engineering |___location=New York |publisher=Dover |year=1958 |url=http://alcatel-lucent.com/bstj/vol37-1958/articles/bstj37-1-185.pdf |archive-url=https://ghostarchive.org/archive/20221009/http://alcatel-lucent.com/bstj/vol37-1958/articles/bstj37-1-185.pdf |archive-date=2022-10-09 |url-status=live }}{{Dead link|date=April 2020 |bot=InternetArchiveBot |fix-attempted=yes }}</ref> even though that article does not treat sampling and reconstruction of continuous signals as others did. Their glossary of terms includes these entries: ▼
{{quote|
▲In 1958, Blackman and Tukey cited Nyquist's 1928 article as a reference for ''the sampling theorem of information theory'',<ref>{{cite book |first1=R. B. |last1=Blackman |first2=J. W. |last2=Tukey |title=The Measurement of Power Spectra : From the Point of View of Communications Engineering |___location=New York |publisher=Dover |year=1958 |url=http://alcatel-lucent.com/bstj/vol37-1958/articles/bstj37-1-185.pdf |archive-url=https://ghostarchive.org/archive/20221009/http://alcatel-lucent.com/bstj/vol37-1958/articles/bstj37-1-185.pdf |archive-date=2022-10-09 |url-status=live }}{{Dead link|date=April 2020 |bot=InternetArchiveBot |fix-attempted=yes }}</ref> even though that article does not treat sampling and reconstruction of continuous signals as others did. Their glossary of terms includes these entries:
{{glossary}}
; Sampling theorem (of information theory): Nyquist's result that equi-spaced data, with two or more points per cycle of highest frequency, allows reconstruction of band-limited functions. (See ''Cardinal theorem''.) ▼{{term|Sampling theorem (of information theory)}}
; Cardinal theorem (of interpolation theory): A precise statement of the conditions under which values given at a doubly infinite set of equally spaced points can be interpolated to yield a continuous band-limited function with the aid of the function <math display="block">\frac{\sin (x - x_i)}{x - x_i}.</math> ▼▲; Sampling theorem (of information theory): {{defn|Nyquist's result that equi-spaced data, with two or more points per cycle of highest frequency, allows reconstruction of band-limited functions. (See ''Cardinal theorem''.) }}
{{term|Cardinal theorem (of interpolation theory)}}
▲; Cardinal theorem (of interpolation theory): {{defn|A precise statement of the conditions under which values given at a doubly infinite set of equally spaced points can be interpolated to yield a continuous band-limited function with the aid of the function <math display="block">\frac{\sin (x - x_i)}{x - x_i}.</math> }}
{{glossary end}}}}
Exactly what "Nyquist's result" they are referring to remains mysterious.
When Shannon stated and proved the sampling theorem in his 1949 article, according to Meijering,<ref name="EM" /> "he referred to the critical sampling interval <math>T = \frac 1 {2W}</math> as the ''Nyquist interval'' corresponding to the band ''<math>W''</math>, in recognition of Nyquist’sNyquist's discovery of the fundamental importance of this interval in connection with telegraphy". This explains Nyquist's name on the critical interval, but not on the theorem.
Similarly, Nyquist's name was attached to ''[[Nyquist rate]]'' in 1953 by [[Harold Stephen Black|Harold S. Black]]:
{{blockquote|"If the essential frequency range is limited to ''<math>B''</math> cycles per second, 2''B''<math>2B</math> was given by Nyquist as the maximum number of code elements per second that could be unambiguously resolved, assuming the peak interference is less half a quantum step. This rate is generally referred to as '''signaling at the Nyquist rate''' and <math>\frac 1 {2B}</math> has been termed a ''Nyquist interval''."|Harold Black, ''Modulation Theory''<ref>{{cite book |first=Harold S. |last=Black |title=Modulation Theory |year=1953 }}</ref> (bold added for emphasis; italics as in the original)}}
According to the ''[[OEDOxford English Dictionary]]'', this may be the origin of the term ''Nyquist rate''. In Black's usage, it is not a sampling rate, but a signaling rate.
== See also ==
| 