Advanced Video Coding: Difference between revisions

Revision as of 21:07, 26 March 2024

Revision as of 16:38, 25 April 2024

Line 33:

== Naming ==

The H.264 name follows the [[ITU-T]] [[ITU-T#Recommendation categorization|naming convention]], where Recommendations are given a letter corresponding to their series and a recommendation number within the series. H.264 is part of "H-Series Recommendations: Audiovisual and multimedia systems". H.264 is further categorized into "H.200-H.499: Infrastructure of audiovisual services" and "H.260-H.279: [[Video encoding|Coding]] of moving video".<ref>{{Cite web |title=ITU-T Recommendations |url=https://www.itu.int:443/en/ITU-T/publications/Pages/recs.aspx |access-date=2022-11-01 |website=ITU |language=en-US}}</ref> The MPEG-4 AVC name relates to the naming convention in [[International Organization for Standardization|ISO]]/[[International Electrotechnical Commission|IEC]] [[MPEG]], where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG-4. The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG-4 AVC, or MPEG-4/H.264 AVC, to emphasize the common heritage. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon. For example, the video compression standard known as MPEG-2 also arose from the partnership between [[MPEG]] and the ITU-T, where MPEG-2 video is known to the ITU-T community as H.262.<ref>{{cite web|url=http://itu.int/rec/T-REC-H.262|title=H.262 : Information technology&nbsp;— Generic coding of moving pictures and associated audio information: Video|access-date=2007-04-15}}</ref>) Some software programs (such as [[VLC media player]]) internally identify this standard as AVC1.

== History ==

Line 52:

=== Multiview video coding ===

The next major feature added to the standard was [[Multiview Video Coding]] (MVC). Specified in Annex H of H.264/AVC, MVC enables the construction of bitstreams that represent more than one view of a video scene. An important example of this functionality is [[stereoscopy|stereoscopic 3D]] video coding. Two profiles were developed in the MVC work: Multiview High profile supports an arbitrary number of views, and Stereo High profile is designed specifically for two-view stereoscopic video. The Multiview Video Coding extensions were completed in November 2009.

=== 3D-AVC and MFC stereoscopic coding ===

Line 98:

== Applications ==

{{Further|List of video services using H.264/MPEG-4 AVC}}

The H.264 video format has a very broad application range that covers all forms of digital compressed video from low bit-rate Internet streaming applications to HDTV broadcast and Digital Cinema applications with nearly lossless coding. With the use of H.264, bit rate savings of 50% or more compared to [[MPEG-2 Part 2]] are reported. For example, H.264 has been reported to give the same Digital Satellite TV quality as current MPEG-2 implementations with less than half the bitrate, with current MPEG-2 implementations working at around 3.5 &nbsp;Mbit/s and H.264 at only 1.5 &nbsp;Mbit/s.<ref>{{cite journal |author=Wenger|title=RFC 3984 : RTP Payload Format for H.264 Video |newspaper=Ietf Datatracker |date=February 2005 |url=http://tools.ietf.org/html/rfc3984#page-2 |page=2|doi=10.17487/RFC3984 |display-authors=etal}}</ref> Sony claims that 9 &nbsp;Mbit/s AVC recording mode is equivalent to the image quality of the [[HDV]] format, which uses approximately 18–25 &nbsp;Mbit/s.<ref>{{cite web|title=Which recording mode is equivalent to the image quality of the High Definition Video (HDV) format?|website=Sony eSupport|url=https://ca.en.kb.sony.com/app/answers/detail/a_id/41994|access-date=December 8, 2018|archive-url=https://web.archive.org/web/20171109054553/https://ca.en.kb.sony.com/app/answers/detail/a_id/41994|archive-date=November 9, 2017|url-status=dead|df=mdy-all}}</ref>

To ensure compatibility and problem-free adoption of H.264/AVC, many standards bodies have amended or added to their video-related standards so that users of these standards can employ H.264/AVC. Both the [[Blu-ray Disc]] format and the now-discontinued [[HD DVD]] format include the H.264/AVC High Profile as one of three mandatory video compression formats. The Digital Video Broadcast project ([[Digital Video Broadcasting|DVB]]) approved the use of H.264/AVC for broadcast television in late 2004.

Line 113:

[[AVC-Intra]] is an [[Video compression#Encoding theory|intraframe]]-only compression format, developed by [[Panasonic]].

[[XAVC]] is a recording format designed by Sony that uses level 5.2 of H.264/MPEG-4 AVC, which is the highest level supported by that video standard.<ref name=SonyXAVCrecordingformat>{{cite news |title=Sony introduces new XAVC recording format to accelerate 4K development in the professional and consumer markets |publisher=Sony |url=http://www.sony.co.uk/pro/article/broadcast-xavc-codec-1012 |date=2012-10-30 |access-date=2012-11-01}}</ref><ref name=SonyXAVCrecordingformatpdf>{{cite news |title=Sony introduces new XAVC recording format to accelerate 4K development in the professional and consumer markets |publisher=Sony |url=http://www.xavc-info.org/resource/1351573070000/xavcsite/share/data/XAVC_30-Oct-2012_NewsRelease.pdf |date=2012-10-30 |access-date=2012-11-01 }}{{dead link|date=October 2017 |bot=InternetArchiveBot |fix-attempted=yes }}</ref> XAVC can support [[4K resolution]] (4096 × 2160 and 3840 × 2160) at up to 60&nbsp;[[frames per second]] (fps).<ref name=SonyXAVCrecordingformat/><ref name=SonyXAVCrecordingformatpdf/> Sony has announced that cameras that support XAVC include two [[CineAlta]] cameras—the Sony PMW-F55 and Sony PMW-F5.<ref name=Engadget4KCineAltaF55F5>{{cite news |title=Sony goes Red-hunting with PMW-F55 and PMW-F5 pro CineAlta 4K Super 35mm sensor camcorders |publisher=Engadget |url=https://www.engadget.com/2012/10/30/sony-goes-red-hunting-with-pmw-f55-and-pmw-f5-pro-cinealta-4k/ |author=Steve Dent |date=2012-10-30 |access-date=2012-11-05}}</ref> The Sony PMW-F55 can record XAVC with 4K resolution at 30&nbsp;fps at 300 [[Mbit/s]] and 2K resolution at 30&nbsp;fps at 100 &nbsp;Mbit/s.<ref name=SonyPMW-F55datasheet>{{cite news |title=F55 CineAlta 4K the future, ahead of schedule |publisher=Sony |url=http://pro.sony.com/bbsccms/assets/files/show/highend/pdf/F55_Camera.pdf |date=2012-10-30 |access-date=2012-11-01 |url-status=dead |archive-url=https://web.archive.org/web/20121119001306/http://pro.sony.com/bbsccms/assets/files/show/highend/pdf/F55_Camera.pdf |archive-date=November 19, 2012 |df=mdy-all }}</ref> XAVC can record 4K resolution at 60&nbsp;fps with 4:2:2 chroma sampling at 600 &nbsp;Mbit/s.<ref name=SonySxSPro4Kvideo>{{cite news |title=Ultra-fast "SxS PRO+" memory cards transform 4K video capture |publisher=Sony |url=http://www.sony.co.uk/pro/article/km-rme-sxspro+-launch |access-date=2012-11-05 |url-status=dead |archive-url=https://web.archive.org/web/20130308090520/http://www.sony.co.uk/pro/article/km-rme-sxspro+-launch |archive-date=March 8, 2013 |df=mdy-all }}</ref><ref name=SonySxSPro4Kvideopdf>{{cite news |title=Ultra-fast "SxS PRO+" memory cards transform 4K video capture |publisher=Sony |url=http://www.sony.co.uk/res/attachment/file/60/1237488996260.pdf |access-date=2012-11-05 |url-status=dead |archive-url=https://web.archive.org/web/20150402123222/http://www.sony.co.uk/res/attachment/file/60/1237488996260.pdf |archive-date=April 2, 2015 |df=mdy-all }}</ref>

== Design ==

Line 233:

|-

! [[Color depth|Bit depth]] (per sample)

| {{Yes|8}} || {{Yes|8}} || {{Yes|8}} || {{Yes|8}} || {{Yes|8}} || {{Yes|8}} || {{yes|8 to 10}} || {{yes|8 to 10}} || {{yes|8 to 14}}

|-

! [[Chroma subsampling|Chroma]] formats

| {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{Yes|4:2:0<br /><br />&nbsp;}} || {{yes|4:2:0/<br />4:2:2<br />&nbsp;}} || {{yes|4:2:0/<br />4:2:2/<br />4:4:4}}

|-

! [[Flexible macroblock ordering|Flexible macroblock ordering (FMO)]]

| {{no}} || {{yes}} || {{yes}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}}

|-

! [[Arbitrary slice ordering|Arbitrary slice ordering (ASO)]]

| {{no}} || {{yes}} || {{yes}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}}

|-

! Redundant slices (RS)

| {{no}} || {{yes}} || {{yes}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}}

|-

! Data Partitioning

| {{no}} || {{no}} || {{yes}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}}

|-

! SI and SP slices

| {{no}} || {{no}} || {{yes}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}}

|-

! Interlaced coding (PicAFF, MBAFF)

| {{no}} || {{no}} || {{yes}} || {{yes}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! B slices

| {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|- style="display:none"

! Multiple reference frames

Line 269:

|-

! [[CABAC|CABAC entropy coding]]

| {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! 4:0:0 ([[Monochrome]])

| {{no}} || {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! 8×8 vs. 4×4 transform adaptivity

| {{no}} || {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! Quantization scaling matrices

| {{no}} || {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! Separate C_B and C_R QP control

| {{no}} || {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}

|-

! Separate color plane coding

| {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{yes}}

|-

! Predictive lossless coding

| {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{no}} || {{yes}}

|}

=== Levels ===

As the term is used in the standard, a "''level''" is a specified set of constraints that indicate a degree of required decoder performance for a profile. For example, a level of support within a profile specifies the maximum picture resolution, frame rate, and bit rate that a decoder may use. A decoder that conforms to a given level must be able to decode all bitstreams encoded for that level and all lower levels.

<!-- Please don't change the default state of the table to collapsed using the mw-collapsed option since that causes problems with some web browsers, such as making the customtoggle in the table not work when the page is refreshed or revisited. -->

{| class="wikitable" style="text-align:right;"

Line 428:

=== Decoded picture buffering ===

Previously encoded pictures are used by H.264/AVC encoders to provide predictions of the values of samples in other pictures. This allows the encoder to make efficient decisions on the best way to encode a given picture. At the decoder, such pictures are stored in a virtual ''decoded picture buffer'' (DPB). The maximum capacity of the DPB, in units of frames (or pairs of fields), as shown in parentheses in the right column of the table above, can be computed as follows:

: {{mono|''DpbCapacity'' {{=}} min(floor(''MaxDpbMbs'' / (''PicWidthInMbs'' * ''FrameHeightInMbs'')), 16)}}

Line 557:

ASIC encoders with H.264 encoder functionality are available from many different semiconductor companies, but the core design used in the ASIC is typically licensed from one of a few companies such as [[Chips&Media]], Allegro DVT, [[On2]] (formerly Hantro, acquired by Google), [[Imagination Technologies]], NGCodec. Some companies have both FPGA and ASIC product offerings.<ref>{{cite web |url=http://www.design-reuse.com/sip/?q=H.264+encoder |title=Design-reuse.com |publisher=Design-reuse.com |date=1990-01-01 |access-date=2010-05-17}}</ref>

Texas Instruments manufactures a line of ARM + DSP cores that perform DSP H.264 BP encoding 1080p at 30fps.<ref>{{cite web |url=http://processors.wiki.ti.com/index.php/Category:DM6467 |title=Category:DM6467 - Texas Instruments Embedded Processors Wiki |publisher=Processors.wiki.ti.com |date=2011-07-12 |access-date=2011-07-30 |archive-date=July 17, 2011 |archive-url=https://web.archive.org/web/20110717053351/http://processors.wiki.ti.com/index.php/Category:DM6467 |url-status=dead }}</ref> This permits flexibility with respect to codecs (which are implemented as highly optimized DSP code) while being more efficient than software on a generic CPU.

== Licensing ==

Line 565:

A private organization known as [[MPEG LA]], which is not affiliated in any way with the MPEG standardization organization, administers the licenses for patents applying to this standard, as well as other [[patent pool]]s, such as for MPEG-4 Part 2 Video, HEVC and MPEG-DASH. The patent holders include [[Fujitsu]], [[Panasonic]], [[Sony]], [[Mitsubishi]], [[Apple Inc.|Apple]], [[Columbia University]], [[KAIST]], [[Dolby Laboratories|Dolby]], [[Google]], [[JVC Kenwood]], [[LG Electronics]], [[Microsoft]], [[NTT Docomo]], [[Philips]], [[Samsung]], [[Sharp Corporation|Sharp]], [[Toshiba]] and [[ZTE]],<ref>{{cite web |title=Licensors Included in the AVC/H.264 Patent Portfolio License |url=https://www.mpegla.com/programs/avc-h-264/licensors/ |website=[[MPEG LA]] |access-date=18 June 2019}}</ref> although the majority of patents in the pool are held by [[Panasonic]] ({{formatnum:{{#expr:1137+60}}|}} patents), [[Gōdō gaisha|Godo Kaisha]] IP Bridge ({{formatnum:{{#expr:1111+19}}|}} patents) and [[LG Electronics]] ({{#expr:949+(40+1)}} patents).<ref name="patents">{{cite web |title=AVC/H.264 {{ndash}} Patent List |url=https://www.mpegla.com/wp-content/uploads/avc-att1.pdf |website=MPEG LA |access-date=6 July 2019}}</ref>

On August 26, 2010, MPEG LA announced that royalties won't be charged for H.264 encoded Internet video that is free to end users.<ref>{{cite web|url=http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/74/n-10-08-26.pdf|title=MPEG LA's AVC License Will Not Charge Royalties for Internet Video that is Free to End Users through Life of License|publisher=MPEG LA|date=2010-08-26|access-date=2010-08-26|archive-date=November 7, 2013|archive-url=https://web.archive.org/web/20131107135621/http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/74/n-10-08-26.pdf|url-status=dead}}</ref> All other royalties remain in place, such as royalties for products that decode and encode H.264 video as well as to operators of free television and subscription channels.<ref>{{cite news|url=https://www.pcmag.com/article2/0,2817,2368359,00.asp |title=MPEG LA Cuts Royalties from Free Web Video, Forever |publisher=pcmag.com |date=2010-08-26 |access-date=2010-08-26 |first=Mark |last=Hachman}}</ref> The license terms are updated in 5-year blocks.<ref>{{cite web |url=http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx |title=AVC FAQ |publisher=MPEG LA |date=2002-08-01 |access-date=2010-05-17 |archive-url=https://web.archive.org/web/20100507102710/http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx |archive-date=May 7, 2010 |url-status=dead }}</ref>

Since the first version of the standard was completed in May 2003 ({{age|month=May|year=2003}} years ago) and the most commonly used profile (the High profile) was completed in June 2004{{Citation needed|date=December 2023}} ({{age|month=June|year=2004}} years ago), a number of the relevant patents that apply to the standard expires every year,<ref>{{Cite web |url=https://www.mpegla.com/wp-content/uploads/avc-att1.pdf |title=AVC Attachment 1 |website=mpegla.com |access-date=August 1, 2022}}</ref> although one of the US patents in the MPEG LA H.264 pool lasts at least until November 2030.<ref>{{cite web |url=https://patents.google.com/patent/US9356620B2/ |title=United States Patent 9,356,620 Baese, et al. |access-date=August 1, 2022}} with an earliest priority date of September 14, 2001 has a 2,998 day term extension.</ref>

In 2005, Qualcomm sued Broadcom in US District Court, alleging that Broadcom infringed on two of its patents by making products that were compliant with the H.264 video compression standard.<ref name="case">See [http://caselaw.lp.findlaw.com/data2/circs/fed/071545p.pdf Qualcomm Inc. v. Broadcom Corp.], No. 2007-1545, 2008-1162 (Fed. Cir. December 1, 2008). For articles in the popular press, see signonsandiego.com, [http://www.signonsandiego.com/news/business/20070127-9999-1b27verdict.html "Qualcomm loses its patent-rights case"] and [http://www.signonsandiego.com/news/business/20070126-9999-1b26qualcomm.html "Qualcomm's patent case goes to jury"]; and bloomberg.com [https://www.bloomberg.com/apps/news?pid=20601204&sid=aLX_DFMCEYWU&refer=technology "Broadcom Wins First Trial in Qualcomm Patent Dispute"]</ref> In 2007, the District Court found that the patents were unenforceable because Qualcomm had failed to disclose them to the JVT prior to the release of the H.264 standard in May 2003.<ref name="case" /> In December 2008, the US Court of Appeals for the Federal Circuit affirmed the District Court's order that the patents be unenforceable but remanded to the District Court with instructions to limit the scope of unenforceability to H.264 compliant products.<ref name="case" />

In October 2023 [[Nokia]] sued [[HP Inc.|HP]] and [[Amazon (company)|Amazon]] for H.264/H.265 patent infringement in USA, UK and other locations.<ref>{{Cite web |title=nokia h264 |url=https://www.nokia.com/blog/nokia-seeks-compensation-for-amazons-use-of-our-patented-multimedia-inventions/}}</ref>