Discrete cosine transform: Difference between revisions

Revision as of 04:51, 11 September 2024

Latest revision as of 19:07, 9 August 2025

(23 intermediate revisions by 12 users not shown)

Line 1:

{{Short description|Technique used in signal processing and data compression}}

{{Jagged 85 cleanup|date=July 2022}}

A '''discrete cosine transform''' ('''DCT''') expresses a finite sequence of [[data points]] in terms of a sum of [[cosine]] functions oscillating at different [[frequency|frequencies]]. The DCT, first proposed by [[Nasir Ahmed (engineer)|Nasir Ahmed]] in 1972, is a widely used transformation technique in [[signal processing]] and [[data compression]]. It is used in most [[digital media]], including [[digital images]] (such as [[JPEG]] and [[HEIF]]), [[digital video]] (such as [[MPEG]] and {{nowrap|[[H.26x]]}}), [[digital audio]] (such as [[Dolby Digital]], [[MP3]] and [[Advanced Audio Coding|AAC]]), [[digital television]] (such as [[SDTV]], [[HDTV]] and [[Video on demand|VOD]]), [[digital radio]] (such as [[AAC+]] and [[DAB+]]), and [[speech coding]] (such as [[AAC-LD]], [[Siren (codec)|Siren]] and [[Opus (audio format)|Opus]]). DCTs are also important to numerous other applications in [[science and engineering]], such as [[digital signal processing]], [[telecommunication]] devices, reducing [[network bandwidth]] usage, and [[spectral method]]s for the numerical solution of [[partial differential equations]].

Line 12 ⟶ 11:

== History ==

The DCT was first conceived by [[Nasir Ahmed (engineer)|Nasir Ahmed]], T. Natarajan and [[K. R. Rao]] while working at [[Kansas State University]]. The concept was proposed to the [[National Science Foundation]] in 1972. The DCT was originally intended for [[image compression]].<ref name="Ahmed" /><ref name="Stankovic"/> Ahmed developed a practical DCT algorithm with his PhD students T. Raj Natarajan, Wills Dietrich, and Jeremy Fries, and his friend Dr. [[K. R. Rao]] at the [[University of Texas at Arlington]] in 1973.<ref name="Ahmed"/> They presented their results in a January 1974 paper, titled ''Discrete Cosine Transform''.<ref name="pubDCT" /><ref name="pubRaoYip" /><ref name="t81">{{cite web|date=September 1992|title=T.81 – Digital compression and coding of continuous-tone still images – Requirements and guidelines|url=https://www.w3.org/Graphics/JPEG/itu-t81.pdf|access-date=12 July 2019|publisher=[[CCITT]]}}</ref> It described what is now called the type-II DCT (DCT-II),<ref name="Britanak2010" />{{rp|page = [https://books.google.com/books?id=iRlQHcK-r_kC&pg=PA51 51]}} as well as the type-III inverse DCT (IDCT).<ref name="pubDCT"/>

Since its introduction in 1974, there has been significant research on the DCT.<ref name="t81"/> In 1977, Wen-Hsiung Chen published a paper with C. Harrison Smith and Stanley C. Fralick presenting a fast DCT algorithm.<ref name="A Fast Computational Algorithm for">{{cite journal |last1=Chen |first1=Wen-Hsiung |last2=Smith |first2=C. H. |last3=Fralick |first3=S. C. |title=A Fast Computational Algorithm for the Discrete Cosine Transform |journal=[[IEEE Transactions on Communications]] |date=September 1977 |volume=25 |issue=9 |pages=1004–1009 |doi=10.1109/TCOM.1977.1093941}}</ref><ref name="t81"/> Further developments include a 1978 paper by M. J. Narasimha and A. M. Peterson, and a 1984 paper by B. G. Lee.<ref name="t81"/> These research papers, along with the original 1974 Ahmed paper and the 1977 Chen paper, were cited by the [[Joint Photographic Experts Group]] as the basis for [[JPEG]]'s lossy image compression algorithm in 1992.<ref name="t81"/><ref name="chen">{{cite journal |last1=Smith |first1=C. |last2=Fralick |first2=S. |title=A Fast Computational Algorithm for the Discrete Cosine Transform |journal=IEEE Transactions on Communications |date=1977 |volume=25 |issue=9 |pages=1004–1009 |doi=10.1109/TCOM.1977.1093941 |issn=0090-6778}}</ref>

Line 60 ⟶ 59:

**[[Image processing]] — [[digital image processing]],<ref name="Stankovic"/> [[image analysis]], [[content-based image retrieval]], [[corner detection]], directional block-wise [[Sparse approximation|image representation]], [[edge detection]], [[image enhancement]], [[image fusion]], [[image segmentation]], [[interpolation]], [[image noise]] level estimation, mirroring, rotation, [[Just-noticeable difference|just-noticeable distortion]] (JND) profile, [[spatiotemporal]] masking effects, [[foveated imaging]]<ref name="Ochoa"/>

**[[Image quality]] assessment — DCT-based quality degradation metric (DCT QM)<ref name="Ochoa"/>

**[[Image reconstruction]] — directional [[image texture|textures]] auto inspection, [[image restoration]], [[inpainting]], [[photo recovery|visual recovery]]<ref name="Ochoa"/>

*[[Medical technology]]

**[[Electrocardiography]] (ECG) — [[vectorcardiography]] (VCG)<ref name="Ochoa"/>

Line 258 ⟶ 257:

===Compression artifacts===

A common issue with DCT compression in [[digital media]] are blocky [[compression artifacts]],<ref name="Katsaggelos">{{cite book |last1=Katsaggelos |first1=Aggelos K. |last2=Babacan |first2=S. Derin |last3=Chun-Jen |first3=Tsai |title=The Essential Guide to Image Processing |date=2009 |publisher=[[Academic Press]] |isbn=9780123744579 |pages=349–383|chapter=Chapter 15 - Iterative Image Restoration}}</ref> caused by DCT blocks.<ref name="Alikhani">{{cite web |last1=Alikhani |first1=Darya |title=Beyond resolution: Rosa Menkman's glitch art |url=http://postmatter.merimedia.com/articles/archive-2012-2016/2015/51-rosa-menkman/ |website=POSTmatter |date=April 1, 2015 |access-date=19 October 2019 |archive-date=19 October 2019 |archive-url=https://web.archive.org/web/20191019082218/http://postmatter.merimedia.com/articles/archive-2012-2016/2015/51-rosa-menkman/ |url-status=dead }}</ref> In a DCT algorithm, an image (or frame in an image sequence) is divided into square blocks which are processed independently from each other, then the DCT blocks is taken within each block and the resulting DCT coefficients are [[Quantization (signal processing)|quantized]]. This process can cause blocking artifacts, primarily at high [[data compression ratio]]s.<ref name="Katsaggelos"/> This can also cause the [[mosquito noise]] effect, commonly found in [[digital video]].<ref>{{cite web |title=Mosquito noise |url=https://www.pcmag.com/encyclopedia/term/55914/mosquito-noise |website=[[PC Magazine]] |access-date=19 October 2019}}</ref><!--[[User:Kvng/RTH]]-->

DCT blocks are often used in [[glitch art]].<ref name="Alikhani"/> The artist [[Rosa Menkman]] makes use of DCT-based compression artifacts in her glitch art,<ref name="Menkman">{{cite book |last1=Menkman |first1=Rosa |title=The Glitch Moment(um) |url=https://networkcultures.org/_uploads/NN%234_RosaMenkman.pdf |publisher=Institute of Network Cultures |isbn=978-90-816021-6-7 |date=October 2011 |access-date=19 October 2019}}</ref> particularly the DCT blocks found in most [[digital media]] formats such as [[JPEG]] digital images and [[MP3]] [[digital audio]].<ref name="Alikhani"/> Another example is ''Jpegs'' by German photographer [[Thomas Ruff]], which uses intentional [[JPEG]] artifacts as the basis of the picture's style.<ref>{{cite book|chapter=jpegs|first=Thomas|last=Ruff|title=Aperture|date=May 31, 2009|page=132|publisher=Aperture |isbn=9781597110938}}</ref><ref>{{cite web|url=http://jmcolberg.com/weblog/2009/04/review_jpegs_by_thomas_ruff/|title=Review: jpegs by Thomas Ruff|first=Jörg|last=Colberg|date=April 17, 2009}}</ref>

==Informal overview==

Like any Fourier-related transform, discrete [[Cosine transform|cosine transforms]] (DCTs) express a function or a signal in terms of a sum of [[sinusoid]]s with different [[frequencies]] and [[amplitude]]s. Like the [[discrete Fourier transform]] (DFT), a DCT operates on a function at a finite number of [[Discrete signal|discrete]] data points]]. The obvious distinction between a DCT and a DFT is that the former uses only cosine functions, while the latter uses both cosines and sines (in the form of [[complex exponential]]s). However, this visible difference is merely a consequence of a deeper distinction: a DCT implies different [[boundary condition]]s from the DFT or other related transforms.

The Fourier-related transforms that operate on a function over a finite [[___domain of a function|___domain]], such as the DFT or DCT or a [[Fourier series]], can be thought of as implicitly defining an ''extension'' of that function outside the ___domain. That is, once you write a function <math>f(x)</math> as a sum of sinusoids, you can evaluate that sum at any <math>x</math>, even for <math>x</math> where the original <math>f(x)</math> was not specified. The DFT, like the Fourier series, implies a [[periodic function|periodic]] extension of the original function. A DCT, like a [[Sine and cosine transforms|cosine transform]], implies an [[even and odd functions|even]] extension of the original function.

[[Image:DCT-symmetries.svg|thumb|right|350px|Illustration of the implicit even/odd extensions of DCT input data, for ''N''=11 data points (red dots), for the four most common types of DCT (types I-IV). Note the subtle differences at the interfaces between the data and the extensions: in DCT-II and DCT-IV both the end points are replicated in the extensions but not in DCT-I or DCT-III (and a zero point is inserted at the sign reversal extension in DCT-III).]]

However, because DCTs operate on ''finite'', ''discrete'' sequences, two issues arise that do not apply for the continuous cosine transform. First, one has to specify whether the function is even or odd at ''both'' the left and right boundaries of the ___domain (i.e. the min-''n'' and max-''n'' boundaries in the definitions below, respectively). Second, one has to specify around ''what point'' the function is even or odd. In particular, consider a sequence ''abcd'' of four equally spaced data points, and say that we specify an even ''left'' boundary. There are two sensible possibilities: either the data are even about the sample ''a'', in which case the even extension is ''dcbabcd'', or the data are even about the point ''halfway'' between ''a'' and the previous point, in which case the even extension is ''dcbaabcd'' (''a'' is repeated).

Each boundary can be either even or odd (2 choices per boundary) and can be symmetric about a data point or the point halfway between two data points (2 choices per boundary), for a total of 2 × 2 × 2 × 2 = 16 possibilities. These choices lead to all the standard variations of DCTs and also [[discrete sine transform]]s (DSTs). Half of these possibilities, those where the ''left'' boundary is even, correspond to the 8 types of DCT; the other half are the 8 types of DST.

Each boundary can be either even or odd (2 choices per boundary) and can be symmetric about a data point or the point halfway between two data points (2 choices per boundary), for a total of 2 × 2 × 2 × 2 = 16 possibilities. Half of these possibilities, those where the ''left'' boundary is even, correspond to the 8 types of DCT; the other half are the 8 types of DST.

These different boundary conditions strongly affect the applications of the transform and lead to uniquely useful properties for the various DCT types. Most directly, when using Fourier-related transforms to solve [[partial differential equation]]s by [[spectral method]]s, the boundary conditions are directly specified as a part of the problem being solved. Or, for the [[Modified discrete cosine transform|MDCT]] (based on the type-IV DCT), the boundary conditions are intimately involved in the MDCT's critical property of time-___domain aliasing cancellation. In a more subtle fashion, the boundary conditions are responsible for the "''energy compactification"'' properties that make DCTs useful for image and audio compression, because the boundaries affect the rate of convergence of any Fourier-like series.

In particular, it is well known that any [[Classification of discontinuities|discontinuities]] in a function reduce the [[rate of convergence]] of the Fourier series, so that more sinusoids are needed to represent the function with a given accuracy. The same principle governs the usefulness of the DFT and other transforms for signal compression; the smoother a function is, the fewer terms in its DFT or DCT are required to represent it accurately, and the more it can be compressed. ({{efn|Here, we think of the DFT or DCT as approximations for the [[Fourier series]] or [[cosine series]] of a function, respectively, in order to talk about its "smoothness".)}} However, the implicit periodicity of the DFT means that discontinuities usually occur at the boundaries: any random segment of a signal is unlikely to have the same value at both the left and right boundaries. ({{efn|A similar problem arises for the DST, in which the odd left boundary condition implies a discontinuity for any function that does not happen to be zero at that boundary.)}} In contrast, a DCT where ''both'' boundaries are even ''always'' yields a continuous extension at the boundaries (although the [[slope]] is generally discontinuous). This is why DCTs, and in particular DCTs of types I, II, V, and VI (the types that have two even boundaries) generally perform better for signal compression than DFTs and DSTs. In practice, a type-II DCT is usually preferred for such applications, in part for reasons of computational convenience.

== Formal definition ==

Line 287 ⟶ 286:

\qquad \text{ for } ~ k = 0,\ \ldots\ N-1 ~.</math>

Some authors further multiply the <math>x_0 </math> and <math> x_{N-1} </math> terms by <math> \sqrt{2\,}\, ,</math> and correspondingly multiply the <math> X_0 </math> and <math> X_{N-1}</math> terms by <math> 1/\sqrt{2\,} \,,</math> which makes the DCT-I matrix [[orthogonal matrix|orthogonal]], if one further multiplies by an overall scale factor of <math> display="inline">\sqrt{\tfrac{2}{N-1\,}\,} ,</math>, makes the DCT-I matrix [[orthogonal matrix|orthogonal]] but breaks the direct correspondence with a real-even [[Discrete Fourier transform|DFT]].

The DCT-I is exactly equivalent (up to an overall scale factor of 2), to a [[Discrete Fourier transform|DFT]] of <math> 2(N-1) </math> real numbers with even symmetry. For example, a DCT-I of <math>N = 5 </math> real numbers <math> a\ b\ c\ d\ e </math> is exactly equivalent to a DFT of eight real numbers {{not a typo|<math> a\ b\ c\ d\ e\ d\ c\ b </math>}} (even symmetry), divided by two. (In contrast, DCT types II-IV involve a half-sample shift in the equivalent DFT.)

Note, however, that the DCT-I is not defined for <math> N </math> less than 2, while all other DCT types are defined for any positive <math> N .</math>.

Thus, the DCT-I corresponds to the boundary conditions: <math> x_n </math> is even around <math> n = 0 </math> and even around <math> n = N - 1 </math>; similarly for <math> X_k .</math>.

=== DCT-II ===

Line 300 ⟶ 299:

\qquad \text{ for } ~ k = 0,\ \dots\ N-1 ~.</math>

The DCT-II is probably the most commonly used form, and is often simply referred to as "the ''DCT"''.<ref name="pubDCT"/><ref name="pubRaoYip"/>

This transform is exactly equivalent (up to an overall scale factor of 2) to a [[Discrete Fourier transform|DFT]] of <math>4N</math> real inputs of even symmetry, where the even-indexed elements are zero. That is, it is half of the [[Discrete Fourier transform|DFT]] of the <math>4N</math> inputs <math> y_n ,</math> where <math> y_{2n} = 0 ,</math>, <math> y_{2n+1} = x_n </math> for <math> 0 \leq n < N ,</math>, <math> y_{2N} = 0 ,</math>, and <math> y_{4N-n} = y_n </math> for <math> 0 < n < 2N .</math>. DCT-II transformation is also possible using 2{{mvar|N}}<math>2N</math> signal followed by a multiplication by half shift. This is demonstrated by [[John Makhoul|Makhoul]].{{cn|date=April 2025}}

Some authors further multiply the <math> X_0 </math> term by <math> 1/\sqrt{N\,} \, </math> and multiply the rest of the matrix by an overall scale factor of <math display="inline">\sqrt{{2}/{N}}</math> (see below for the corresponding change in DCT-III). This makes the DCT-II matrix [[orthogonal matrix|orthogonal]], but breaks the direct correspondence with a real-even [[Discrete Fourier transform|DFT]] of half-shifted input. This is the normalization used by [[Matlab]], for example, see.<ref>{{cite web |url=https://www.mathworks.com/help/signal/ref/dct.html |title=Discrete cosine transform - MATLAB dct |website=www.mathworks.com |access-date=2019-07-11}}</ref> In many applications, such as [[JPEG]], the scaling is arbitrary because scale factors can be combined with a subsequent computational step (e.g. the [[Quantization (signal processing)|quantization]] step in JPEG<ref>{{cite book |isbn=9780442012724 |title=JPEG: Still Image Data Compression Standard |last1=Pennebaker |first1=William B. |last2=Mitchell |first2=Joan L. |date=31 December 1992|publisher=Springer }}</ref>), and a scaling can be chosen that allows the DCT to be computed with fewer multiplications.<ref>{{cite journal |url=https://search.ieice.org/bin/summary.php?id=e71-e_11_1095 |first1=Y. |last1=Arai |first2=T. |last2=Agui |first3=M. |last3=Nakajima |title=A fast DCT-SQ scheme for images |journal=IEICE Transactions |volume=71 |issue=11 |pages= 1095–1097 |year=1988}}</ref><ref>{{cite journal |doi=10.1016/j.sigpro.2008.01.004 |title=Type-II/III DCT/DST algorithms with reduced number of arithmetic operations |year=2008 |last1=Shao |first1=Xuancheng |last2=Johnson |first2=Steven G. |journal=Signal Processing |volume=88 |issue=6 |pages=1553–1564 |arxiv=cs/0703150 |bibcode=2008SigPr..88.1553S |s2cid=986733}}</ref>

The DCT-II implies the boundary conditions: <math> x_n </math> is even around <math> n = -1/2 </math> and even around <math> n = N - 1/2 \,;</math>; <math> X_k </math> is even around <math> k = 0 </math> and odd around <math> k = N .</math>.

=== DCT-III ===

Line 314 ⟶ 313:

\qquad \text{ for } ~ k = 0,\ \ldots\ N-1 ~.</math>

Because it is the inverse of DCT-II up to a scale factor (see below), this form is sometimes simply referred to as "the inverse DCT" ("IDCT").<ref name="pubRaoYip"/>

Some authors divide the <math>x_0</math> term by <math>\sqrt{2}</math> instead of by 2 (resulting in an overall <math>x_0/\sqrt{2}</math> term) and multiply the resulting matrix by an overall scale factor of <math display="inline"> \sqrt{2/N}</math> (see above for the corresponding change in DCT-II), so that the DCT-II and DCT-III are transposes of one another. This makes the DCT-III matrix [[orthogonal matrix|orthogonal]], but breaks the direct correspondence with a real-even [[Discrete Fourier transform|DFT]] of half-shifted output.

The DCT-III implies the boundary conditions: <math>x_n</math> is even around <math>n = 0</math> and odd around <math>n = N ;</math> <math>X_k</math> is even around <math>k = -1/2</math> and even around <math>k = N - 1/2.</math>

Line 329 ⟶ 328:

A variant of the DCT-IV, where data from different transforms are ''overlapped'', is called the [[modified discrete cosine transform]] (MDCT).<ref>{{harvnb|Malvar|1992}}</ref>

The DCT-IV implies the boundary conditions: <math> x_n </math> is even around <math>n = -1/2</math> and odd around <math>n = N - 1/2;</math>; similarly for <math>X_k.</math>.<!--[[User:Kvng/RTH]]-->

=== DCT V-VIII ===

DCTs of types I–IV treat both boundaries consistently regarding the point of symmetry: they are even/odd around either a data point for both boundaries or halfway between two data points for both boundaries. By contrast, DCTs of types V-VIII imply boundaries that are even/odd around a data point for one boundary and halfway between two data points for the other boundary.

In other words, DCT types I–IV are equivalent to real-even [[Discrete Fourier transform|DFT]]sDFTs of even order (regardless of whether <math> N </math> is even or odd), since the corresponding DFT is of length <math> 2(N-1) </math> (for DCT-I) or <math> 4 N </math> (for DCT-II & III) or <math> 8 N </math> (for DCT-IV). The four additional types of discrete cosine transform<ref>{{harvnb|Martucci|1994}}</ref> correspond essentially to real-even DFTs of logically odd order, which have factors of <math> N \pm {1}/{2} </math> in the denominators of the cosine arguments.

However, these variants seem to be rarely used in practice. One reason, perhaps, is that [[Fast Fourier transform|FFT]] algorithms for odd-length DFTs are generally more complicated than [[Fast Fourier transform|FFT]] algorithms for even-length DFTs (e.g. the simplest radix-2 algorithms are only for even lengths), and this increased intricacy carries over to the DCTs as described below.

Line 343 ⟶ 342:

Using the normalization conventions above, the inverse of DCT-I is DCT-I multiplied by 2/(''N''&nbsp;−&nbsp;1). The inverse of DCT-IV is DCT-IV multiplied by 2/''N''. The inverse of DCT-II is DCT-III multiplied by 2/''N'' and vice versa.<ref name="pubRaoYip"/>

Like for the [[discrete Fourier transform|DFT]], the normalization factor in front of these transform definitions is merely a convention and differs between treatments. For example, some authors multiply the transforms by <math display="inline">\sqrt{2/N}</math> so that the inverse does not require any additional multiplicative factor. Combined with appropriate factors of {{sqrt|2}} (see above), this can be used to make the transform matrix [[orthogonal matrix|orthogonal]].

== Multidimensional DCTs ==

Line 507 ⟶ 506:

Specialized DCT algorithms, on the other hand, see widespread use for transforms of small, fixed sizes such as the {{nobr| 8 × 8 }} DCT-II used in [[JPEG]] compression, or the small DCTs (or MDCTs) typically used in audio compression. (Reduced code size may also be a reason to use a specialized DCT for embedded-device applications.)

In fact, even the DCT algorithms using an ordinary FFT are sometimes equivalent to pruning the redundant operations from a larger FFT of real-symmetric data, and they can even be optimal from the perspective of arithmetic counts. For example, a type-II DCT is equivalent to a DFT of size <math>~ 4N ~</math> with real-even symmetry whose even-indexed elements are zero. One of the most common methods for computing this via an FFT (e.g. the method used in [[FFTPACK]] and [[Fastest Fourier Transform in the West|FFTW]]) was described by {{harvtxt|Narasimha|Peterson|1978}} and {{harvtxt|Makhoul|1980}}, and this method in hindsight can be seen as one step of a radix-4 decimation-in-time Cooley–Tukey algorithm applied to the "logical" real-even DFT corresponding to the DCT-II.{{efn|

The radix-4 step reduces the size <math>~ 4N ~</math> DFT to four size <math>~ N ~</math> DFTs of real data, two of which are zero, and two of which are equal to one another by the even symmetry. Hence giving a single size <math>~ N ~</math> FFT of real data plus <math>~ \mathcal{O}(N) ~</math> [[butterfly (FFT algorithm)|butterflies]], once the trivial and / or duplicate parts are eliminated and / or merged.

}}

Line 518 ⟶ 517:

==Example of IDCT==

[[File:DCT filter comparison.png|thumb|right|An example showing eight different filters applied to a test image (top left) by multiplying its DCT spectrum (top right) with each filter.]]

Consider this {{resx|8x8}} [[grayscale image]] of capital letter A.

[[File:letter-a-8x8.png|frame|center|Original size, scaled 10x (nearest neighbor), scaled 10x (bilinear).]]

Line 634 ⟶ 633:

* Takuya Ooura: General Purpose FFT Package, [http://www.kurims.kyoto-u.ac.jp/~ooura/fft.html FFT Package 1-dim / 2-dim]. Free C & FORTRAN libraries for computing fast DCTs (types II–III) in one, two or three dimensions, power of 2 sizes.

* Tim Kientzle: Fast algorithms for computing the 8-point DCT and IDCT, [http://drdobbs.com/parallel/184410889 Algorithm Alley].

* [httphttps://ltfat.sourceforge.net/ LTFAT] is a free Matlab/Octave toolbox with interfaces to the FFTW implementation of the DCTs and DSTs of type I-IV.

{{Compression Methods|state=expanded}}

Line 647 ⟶ 646:

[[Category:Data compression]]

[[Category:Image compression]]

[[Category:Indian inventions]]

[[Category:H.26x]]

[[Category:JPEG]]