Video coding format: Difference between revisions

Revision as of 15:43, 15 January 2025

Revision as of 15:45, 15 January 2025

Line 28:

Practical [[video compression]] emerged with the development of [[motion compensation|motion-compensated]] [[Discrete cosine transform|DCT]] (MC DCT) coding,<ref name="Lea"/><ref name="Ghanbari"/> also called block motion compensation (BMC)<ref name="ITU"/> or DCT motion compensation. This is a hybrid coding algorithm,<ref name="ITU"/> which combines two key [[data compression]] techniques: [[discrete cosine transform]] (DCT) coding<ref name="Lea"/><ref name="Ghanbari"/> in the [[spatial dimension]], and predictive [[motion compensation]] in the [[temporal dimension]].<ref name="ITU"/>

DCT coding is a [[lossy]] block compression [[transform coding]] technique that was first proposed by [[Nasir Ahmed (engineer)|Nasir Ahmed]], who initially intended it for [[image compression]], while he was working at [[Kansas State University]] in 1972. It was then developed into a practical image compression algorithm by Ahmed with T. Natarajan and [[K. R. Rao]] at the [[University of Texas]] in 1973, and was published in 1974.<ref name="Ahmed">{{cite journal |last=Ahmed |first=Nasir |author-link=N. Ahmed |title=How I Came Up With the Discrete Cosine Transform |journal=[[Digital Signal Processing (journal)|Digital Signal Processing]] |date=January 1991 |volume=1 |issue=1 |pages=4–5 |doi=10.1016/1051-2004(91)90086-Z |bibcode=1991DSP.....1....4A |url=https://www.scribd.com/doc/52879771/DCT-History-How-I-Came-Up-with-the-Discrete-Cosine-Transform}}</ref><ref name="pubDCT">{{Citation |first1=Nasir |last1=Ahmed |author1-link=N. Ahmed |first2=T. |last2=Natarajan |first3=K. R. |last3=Rao |title=Discrete Cosine Transform |journal=IEEE Transactions on Computers |date=January 1974 |volume=C-23 |issue=1 |pages=90–93 |doi=10.1109/T-C.1974.223784|s2cid=149806273 }}</ref><ref name="pubRaoYip">{{Citation |last1=Rao |first1=K. R. |author-link1=K. R. Rao |last2=Yip |first2=P. |title=Discrete Cosine Transform: Algorithms, Advantages, Applications |publisher=Academic Press |___location=Boston |year=1990 |isbn=978-0-12-580203-1}}</ref>

The other key development was motion-compensated hybrid coding.<ref name="ITU"/> In 1974, Ali Habibi at the [[University of Southern California]] introduced hybrid coding,<ref name="Habibi">{{cite journal |last1=Habibi |first1=Ali |title=Hybrid Coding of Pictorial Data |journal=IEEE Transactions on Communications |date=1974 |volume=22 |issue=5 |pages=614–624 |doi=10.1109/TCOM.1974.1092258}}</ref><ref>{{cite journal |last1=Chen |first1=Z. |last2=He |first2=T. |last3=Jin |first3=X. |last4=Wu |first4=F. |title=Learning for Video Compression |journal=IEEE Transactions on Circuits and Systems for Video Technology |date=2019 |volume=30 |issue=2 |pages=566–576 |doi=10.1109/TCSVT.2019.2892608 |arxiv=1804.09869 |s2cid=13743007 }}</ref><ref>{{cite book |last1=Pratt |first1=William K. |title=Advances in Electronics and Electron Physics: Supplement |date=1984 |publisher=[[Academic Press]] |isbn=9780120145720 |page=158 |url=https://books.google.com/books?id=OX00AAAAIAAJ |quote=A significant advance in image coding methodology occurred with the introduction of the concept of hybrid transform/DPCM coding (Habibi, 1974).}}</ref> which combines predictive coding with transform coding.<ref name="ITU"/><ref>{{cite book |last1=Ohm |first1=Jens-Rainer |title=Multimedia Signal Coding and Transmission |date=2015 |publisher=Springer |isbn=9783662466919 |pages=364 |url=https://books.google.com/books?id=e7xnBwAAQBAJ&pg=PA364}}</ref> He examined several transform coding techniques, including the DCT, [[Hadamard transform]], [[Fourier transform]], slant transform, and [[Karhunen-Loeve transform]].<ref name="Habibi"/> However, his algorithm was initially limited to [[intra-frame]] coding in the spatial dimension. In 1975, John A. Roese and Guner S. Robinson extended Habibi's hybrid coding algorithm to the temporal dimension, using transform coding in the spatial dimension and predictive coding in the temporal dimension, developing [[inter-frame]] motion-compensated hybrid coding.<ref name="ITU"/><ref name="Roese">{{cite journal |last1=Roese |first1=John A. |last2=Robinson |first2=Guner S. |editor-first1=Andrew G. |editor-last1=Tescher |title=Combined Spatial And Temporal Coding Of Digital Image Sequences |journal=Efficient Transmission of Pictorial Information |date=October 30, 1975 |volume=0066 |pages=172–181 |doi=10.1117/12.965361 |publisher=International Society for Optics and Photonics |bibcode=1975SPIE...66..172R |s2cid=62725808 }}</ref> For the spatial transform coding, they experimented with different transforms, including the DCT and the [[fast Fourier transform]] (FFT), developing inter-frame hybrid coders for them, and found that the DCT is the most efficient due to its reduced complexity, capable of compressing image data down to 0.25-[[bit]] per [[pixel]] for a [[videotelephone]] scene with image quality comparable to a typical intra-frame coder requiring 2-bit per pixel.<ref>{{cite book |last1=Huang |first1=T. S. |title=Image Sequence Analysis |date=1981 |publisher=[[Springer Science & Business Media]] |isbn=9783642870378 |page=29 |url=https://books.google.com/books?id=bAirCAAAQBAJ&pg=PA29}}</ref><ref name="Roese"/>

The DCT was applied to video encoding by Wen-Hsiung Chen,<ref name="Stankovic">{{cite journal |last1=Stanković |first1=Radomir S. |last2=Astola |first2=Jaakko T. |title=Reminiscences of the Early Work in DCT: Interview with K.R. Rao |journal=Reprints from the Early Days of Information Sciences |date=2012 |volume=60 |url=http://ticsp.cs.tut.fi/reports/ticsp-report-60-reprint-rao-corrected.pdf |access-date=October 13, 2019}}</ref> who developed a fast DCT algorithm with C.H. Smith and S.C. Fralick in 1977,<ref>{{cite journal |last1=Chen |first1=Wen-Hsiung |last2=Smith |first2=C. H. |last3=Fralick |first3=S. C. |title=A Fast Computational Algorithm for the Discrete Cosine Transform |journal=[[IEEE Transactions on Communications]] |date=September 1977 |volume=25 |issue=9 |pages=1004–1009 |doi=10.1109/TCOM.1977.1093941}}</ref><ref name="t81">{{cite web |title=T.81 – Digital compression and coding of continuous-tone still images – Requirements and guidelines |url=https://www.w3.org/Graphics/JPEG/itu-t81.pdf |publisher=[[CCITT]] |date=September 1992 |access-date=July 12, 2019}}</ref> and founded [[Compression Labs, Inc.|Compression Labs]] to commercialize DCT technology.<ref name="Stankovic"/> In 1979, [[Anil K. Jain (electrical engineer, born 1946)|Anil K. Jain]] and Jaswant R. Jain further developed motion-compensated DCT video compression.<ref>{{cite book |last1=Cianci |first1=Philip J. |title=High Definition Television: The Creation, Development and Implementation of HDTV Technology |date=2014 |publisher=McFarland |isbn=9780786487974 |page=63 |url=https://books.google.com/books?id=0mbsfr38GTgC&pg=PA63}}</ref><ref name="ITU"/> This led to Chen developing a practical video compression algorithm, called motion-compensated DCT or adaptive scene coding, in 1981.<ref name="ITU"/> Motion-compensated DCT later became the standard coding technique for video compression from the late 1980s onwards.<ref name="Ghanbari"/><ref name="Li">{{cite book |last1=Li |first1=Jian Ping |title=Proceedings of the International Computer Conference 2006 on Wavelet Active Media Technology and Information Processing: Chongqing, China, 29-31 August 2006 |date=2006 |publisher=[[World Scientific]] |isbn=9789812709998 |page=847 |url=https://books.google.com/books?id=FZiK3zXdK7sC&pg=PA847}}</ref>

Line 39:

[[MPEG-1]], developed by the [[Moving Picture Experts Group]] (MPEG), followed in 1991, and it was designed to compress [[VHS]]-quality video.<ref name="history"/> It was succeeded in 1994 by [[MPEG-2]]/[[H.262]],<ref name="history"/> which was developed with patents licensed from a number of companies, primarily [[Sony]], [[Technicolor SA|Thomson]] and [[Mitsubishi Electric]].<ref name="mp2-patents"/> MPEG-2 became the standard video format for [[DVD]] and [[SD digital television]].<ref name="history"/> Its motion-compensated DCT algorithm was able to achieve a [[compression ratio]] of up to 100:1, enabling the development of [[digital media]] technologies such as [[video on demand]] (VOD)<ref name="Lea"/> and [[high-definition television]] (HDTV).<ref name="Shishikui">{{cite journal |last1=Shishikui |first1=Yoshiaki |last2=Nakanishi |first2=Hiroshi |last3=Imaizumi |first3=Hiroyuki |title=An HDTV Coding Scheme using Adaptive-Dimension DCT |journal=Signal Processing of HDTV: Proceedings of the International Workshop on HDTV '93, Ottawa, Canada |date=October 26–28, 1993 |pages=611–618 |doi=10.1016/B978-0-444-81844-7.50072-3 |url=https://books.google.com/books?id=j9XSBQAAQBAJ&pg=PA611 |publisher=[[Elsevier]] |isbn=9781483298511}}</ref> In 1999, it was followed by [[MPEG-4 Visual|MPEG-4]]/[[H.263]], which was a major leap forward for video compression technology.<ref name="history"/> It uses patents licensed from a number of companies, primarily Mitsubishi, [[Hitachi]] and [[Panasonic]].<ref name="mp4-patents"/>

The most widely used video coding format {{as of|2019|lc=y}} is [[H.264/MPEG-4 AVC]].<ref name="Bitmovin">{{cite web |url=https://cdn2.hubspot.net/hubfs/3411032/Bitmovin%20Magazine/Video%20Developer%20Report%202019/bitmovin-video-developer-report-2019.pdf |title=Video Developer Report 2019 |website=[[Bitmovin]] |year=2019 |access-date=November 5, 2019}}</ref> It was developed in 2003, and uses patents licensed from a number of organizations, primarily Panasonic, [[Godo kaisha|Godo Kaisha IP Bridge]] and [[LG Electronics]].<ref name="avc-patents"/> In contrast to the standard DCT used by its predecessors, AVC uses the [[Discrete cosine transform|integer DCT]].<ref name="Stankovic"/><ref name="Wang">{{cite journal |last1=Wang |first1=Hanli |last2=Kwong |first2=S. |last3=Kok |first3=C. |title=Efficient prediction algorithm of integer DCT coefficients for H.264/AVC optimization |journal=IEEE Transactions on Circuits and Systems for Video Technology |date=2006 |volume=16 |issue=4 |pages=547–552 |doi=10.1109/TCSVT.2006.871390|s2cid=2060937 }}</ref> H.264 is one of the video encoding standards for [[Blu-ray Disc]]s; all Blu-ray Disc players must be able to decode H.264. It is also widely used by streaming internet sources, such as videos from [[YouTube]], [[Netflix]], [[Vimeo]], and the [[iTunes Store]], web software such as the [[Adobe Flash Player]] and [[Microsoft Silverlight]], and also various [[HDTV]] broadcasts over terrestrial ([[ATSC standards]], [[ISDB-T]], [[DVB-T]] or [[DVB-T2]]), cable ([[DVB-C]]), and satellite ([[DVB-S2]]).<ref>{{Cite web|title=Digital Video Broadcasting (DVB); Specification for the use of video and audio coding in DVB services delivered directly over IP|url=https://www.etsi.org/deliver/etsi_ts/102000_102099/102005/01.01.01_60/ts_102005v010101p.pdf}}</ref>

A main problem for many video coding formats has been [[patent]]s, making it expensive to use or potentially risking a patent lawsuit due to [[submarine patent]]s. The motivation behind many recently designed video coding formats such as [[Theora]], [[VP8]], and [[VP9]] have been to create a ([[Free software|libre]]) video coding standard covered only by royalty-free patents.<ref>{{Cite web|url=https://blogs.cisco.com/collaboration/world-meet-thor-a-project-to-hammer-out-a-royalty-free-video-codec|title = World, Meet Thor – a Project to Hammer Out a Royalty Free Video Codec|date = August 11, 2015}}</ref> Patent status has also been a major point of contention for the choice of which video formats the mainstream [[web browser]]s will support inside the [[HTML video]] tag.