Advanced Video Coding: Difference between revisions

Content deleted Content added
Licensing: Removed wrong information regarding original patents
Tags: Mobile edit Mobile web edit
Rescuing 4 sources and tagging 0 as dead.) #IABot (v2.0.9.5
 
(23 intermediate revisions by 18 users not shown)
Line 1:
{{Short description|Most widelyWidely used standard for video compression}}
{{Redirect-confused|AVC1|AV1|VC-1}}
{{Lead too long|date=May 2023}}
{{Use mdy dates|date=February 2015}}
 
{{Infobox technology standard
| title = Advanced Video Coding / H.264 / MPEG-4 Part 10
| long_name = Advanced video coding for generic audiovisual services
| status = In force
| year_started = {{Start date and age|2003|p= 2003y}}
| first_published = {{Start date|2004|08|17|df=y}}
| version = 1415.0
| version_date = {{Start date|20212024|08|2213|df=y}}
| organization = [[ITU-T]], [[ISO]], [[International Electrotechnical Commission|IEC]]
| committee = [[ITU-T Study Group 16|SG16]] ([[VCEG]]), [[MPEG]]
| base_standards = [[H.261]], [[H.262]] (aka [[MPEG-2 Video]]), [[H.263]], [[ISO/IEC 14496-2]] (aka MPEG-4 Part 2)
| related_standards = [[H.265]] (aka HEVC), [[H.266]] (aka VVC)
| abbreviationpredecessor = [[H.263]]
| ___domain successor = [[Video compressionH.265]]
| abbreviation =
| license = [[MPEG LA]]<ref>{{cite tech report |publisher=Library of Congress |___location=Washington, D.C. |series=Sustainability of Digital Formats |type=Full draft |title=MPEG-4, Advanced Video Coding (Part 10) (H.264) |date=5 December 2011 |url=https://www.loc.gov/preservation/digital/formats/fdd/fdd000081.shtml |access-date=1 December 2021}}</ref>
| ___domain = [[Video compression]]
| website = {{URL|https://www.itu.int/rec/T-REC-H.264}}
| license = [[MPEG LA]]<ref>{{cite tech report |publisher=Library of Congress |___location=Washington, D.C. |series=Sustainability of Digital Formats |type=Full draft |title=MPEG-4, Advanced Video Coding (Part 10) (H.264) |date=5 December 2011 |url=https://www.loc.gov/preservation/digital/formats/fdd/fdd000081.shtml |access-date=1 December 2021}}</ref>
| website = {{URL|https://www.itu.int/rec/T-REC-H.264}}
}}
 
[[File:H.264 block diagram with quality score.jpg|thumb|Block diagram of video coding layer of H.264 encoder with perceptual quality score]]
 
'''Advanced Video Coding''' ('''AVC'''), also referred to as '''H.264''' or '''MPEG-4 Part 10''', is a [[video compression standard]] based on block-oriented, [[motion compensation|motion-compensated]] coding.<ref>{{Cite web|url=https://www.itu.int/rec/T-REC-H.264|title=H.264 : Advanced video coding for generic audiovisual services|website=www.itu.int|url-status=live|archive-url=https://web.archive.org/web/20191031100750/https://www.itu.int/rec/T-REC-H.264|archive-date=2019-10-31|access-date=2019-11-22}}</ref> It is by far the most commonly used format for the recording, compression, and distribution of video content, used by 9184–86% of video industry developers {{as of|SeptemberNovember 20192023|lc=on}}.<ref>{{cite web|url=https://go.bitmovin.com/hubfsdownloads/Bitmovinbitmovin-Video7th-Developer-Report-2018.pdf|title=Video Developer Report 2018 |website=[[Bitmovin]] |date=September 2019}}</ref><ref>{{cite web |url=https://go.bitmovin.com/video-developer-report-2019 2023-2024.pdf|title=Video Developer Report 20192023/2024 |website=[[Bitmovin]] |date=SeptemberNovember 20192023}}</ref> It supports a maximum resolution of [[8K resolution|8K UHD]].<ref>{{Cite news|url=http://www.mysterybox.us/blog/2017/2/21/delivering-8k-using-avch264|archive-url=https://web.archive.org/web/20210325084239/https://www.mysterybox.us/blog/2017/2/21/delivering-8k-using-avch264 | archive-date=March 25, 2021|title=Delivering 8K using AVC/H.264|work=Mystery Box|access-date=2017-08-23|language=en-US}}</ref><ref name="Wang" />
 
The intent of the H.264/AVC project was to create a standard capable of providing good video quality at substantially lower [[bit rate]]s than previous standards (i.e., half or less the bit rate of [[H.262/MPEG-2 Part 2|MPEG-2]], [[H.263]], or [[MPEG-4 Part 2]]), without increasing the complexity of design so much that it would be impractical or excessively expensive to implement. This was achieved with features such as a reduced-complexity integer [[discrete cosine transform]] (integer DCT),<ref name="apple"/> variable block-size segmentation, and multi-picture [[inter frame|inter-picture prediction]]. An additional goal was to provide enough flexibility to allow the standard to be applied to a wide variety of applications on a wide variety of networks and systems, including low and high bit rates, low and high resolution video, [[Broadcasting|broadcast]], [[DVD]] storage, [[Real-time Transport Protocol|RTP]]/[[Internet Protocol|IP]] packet networks, and [[ITU-T]] multimedia [[telephony]] systems. The H.264 standard can be viewed as a "family of standards" composed of a number of different profiles, although its "High profile" is by far the most commonly used format. A specific decoder decodes at least one, but not necessarily all profiles. The standard describes the format of the encoded data and how the data is decoded, but it does not specify algorithms for encoding video{{snd}} thatencoding—that is left open as a matter for encoder designers to select for themselves, and a wide variety of encoding schemes have been developed. H.264 is typically used for [[lossy compression]], although it is also possible to create truly [[lossless compression|lossless-coded]] regions within lossy-coded pictures or to support rare use cases for which the entire encoding is lossless.
 
H.264 was standardized by the [[ITU-T]] [[Video Coding Experts Group]] (VCEG) of [[ITU-T Study Group 16|Study Group 16]] together with the [[ISO/IEC JTC 1]] [[Moving Picture Experts Group]] (MPEG). The project partnership effort is known as the Joint Video Team (JVT). The ITU-T H.264 standard and the ISO/IEC [[MPEG-4]]&nbsp;AVC standard (formally, ISO/IEC&nbsp;14496-10&nbsp;– MPEG-4 Part 10, Advanced Video Coding) are jointly maintained so that they have identical technical content. The final drafting work on the first version of the standard was completed in May 2003, and various extensions of its capabilities have been added in subsequent editions. [[High Efficiency Video Coding]] (HEVC), a.k.a. H.265 and MPEG-H Part 2 is a successor to H.264/MPEG-4&nbsp;AVC developed by the same organizations, while earlier standards are still in common use.
 
H.264 is perhaps best known as being the most commonly used video encoding format on [[Blu-ray Disc]]s. It is also widely used by streaming Internet sources, such as videos from [[Netflix]], [[Hulu]], [[Amazon Prime Video]], [[Vimeo]], [[YouTube]], and the [[iTunes Store]], Web software such as the [[Adobe Flash Player]] and [[Microsoft Silverlight]], and also various [[HDTV]] broadcasts over terrestrial ([[Advanced Television Systems Committee standards|ATSC]], [[ISDB-T]], [[DVB-T]] or [[DVB-T2]]), cable ([[DVB-C]]), and satellite ([[DVB-S]] and [[DVB-S2]]) systems.
 
H.264 is restricted by [[patent]]s owned by various parties. A license covering most (but not all{{cn|date=February 2024}}) patents essential to H.264 is administered by a [[patent pool]] formerly administered by [[MPEG LA]]. Via Licensing Corp acquired MPEG LA in April 2023 and formed a new patent pool administration company called [[Via-LA|Via Licensing Alliance]].<ref>{{citation |url=https://www.streamingmedia.com/Articles/ReadArticle.aspx?ArticleID=158547 |title=Via LA's Heath Hoglund Talks MPEG LA/Via Licensing Patent Pool Merger |first=Jan |last=Ozer |date=2023-05-08 |publisher=StreamingMedia.com}}</ref> The commercial use of patented H.264 technologies requires the payment of royalties to Via and other patent owners. MPEG LA has allowed the free use of H.264 technologies for streaming Internet video that is free to end users, and [[Cisco]] paid royalties to MPEG LA on behalf of the users of binaries for its [[Open-source software|open source]] H.264 encoder [[openH264]].
 
== Naming ==
The H.264 name follows the [[ITU-T]] [[ITU-T#Recommendation categorization|naming convention]], where Recommendations are given a letter corresponding to their series and a recommendation number within the series. H.264 is part of "H-Series Recommendations: Audiovisual and multimedia systems". H.264 is further categorized into "H.200-H.499: Infrastructure of audiovisual services" and "H.260-H.279: [[Video encoding|Coding]] of moving video".<ref>{{Cite web |title=ITU-T Recommendations |url=https://www.itu.int:443/en/ITU-T/publications/Pages/recs.aspx |access-date=2022-11-01 |website=ITU |language=en-US}}</ref> The MPEG-4 AVC name relates to the naming convention in [[International Organization for Standardization|ISO]]/[[International Electrotechnical Commission|IEC]] [[MPEG]], where the standard is part 10 of ISO/IEC 14496, which is the suite of standards known as MPEG-4. The standard was developed jointly in a partnership of VCEG and MPEG, after earlier development work in the ITU-T as a VCEG project called H.26L. It is thus common to refer to the standard with names such as H.264/AVC, AVC/H.264, H.264/MPEG-4 AVC, or MPEG-4/H.264 AVC, to emphasize the common heritage. Occasionally, it is also referred to as "the JVT codec", in reference to the Joint Video Team (JVT) organization that developed it. (Such partnership and multiple naming is not uncommon. For example, the video compression standard known as MPEG-2 also arose from the partnership between [[MPEG]] and the ITU-T, where MPEG-2 video is known to the ITU-T community as H.262.<ref>{{cite web|url=http://itu.int/rec/T-REC-H.262|title=H.262 : Information technology&nbsp;— Generic coding of moving pictures and associated audio information: Video|access-date=2007-04-15}}</ref>) Some software programs (such as [[VLC media player]]) internally identify this standard as AVC1.
 
== History ==
=== Overall history ===
In early 1998, the [[Video Coding Experts Group]] (VCEG&nbsp;– ITU-T SG16 Q.6) issued a call for proposals on a project called H.26L, with the target to double the coding efficiency (which means halving the bit rate necessary for a given level of fidelity) in comparison to any other existing video coding standards for a broad variety of applications. [[VCEG]] was chaired by [[Gary Sullivan (engineer)|Gary Sullivan]] ([[Microsoft]], formerly [[PictureTel]], U.S.). The first draft design for that new standard was adopted in August 1999. In 2000, [[Thomas Wiegand]] ([[Fraunhofer Institute for Telecommunications|Heinrich Hertz Institute]], Germany) became VCEG co-chair.
 
In December 2001, VCEG and the Moving Picture Experts Group ([[MPEG]]&nbsp;– [[ISO/IEC JTC 1/SC 29]]/WG 11) formed a Joint Video Team (JVT), with the charter to finalize the video coding standard.<ref name=JVTsite>[http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/jvt.aspx Joint Video Team], [[ITU-T]] Web site.</ref> Formal approval of the specification came in March 2003. The JVT was (is) chaired by [[Gary Sullivan (engineer)|Gary Sullivan]], [[Thomas Wiegand]], and Ajay Luthra ([[Motorola]], U.S.: later [[Arris Group|Arris]], U.S.). In July 2004, the Fidelity Range Extensions (FRExt) project was finalized. From January 2005 to November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex (G) called [[Scalable Video Coding]] (SVC). The JVT management team was extended by [[Jens-Rainer Ohm]] ([[RWTH Aachen University]], Germany). From July 2006 to November 2009, the JVT worked on [[Multiview Video Coding]] (MVC), an extension of H.264/AVC towards [[3D television]] and limited-range [[free-viewpoint television]]. That work included the development of two new profiles of the standard: the Multiview High Profile and the Stereo High Profile.
Line 52 ⟶ 56:
 
=== Multiview video coding ===
The next major feature added to the standard was [[Multiview Video Coding]] (MVC). Specified in Annex H of H.264/AVC, MVC enables the construction of bitstreams that represent more than one view of a video scene. An important example of this functionality is [[stereoscopy|stereoscopic 3D]] video coding. Two profiles were developed in the MVC work: Multiview High profile supports an arbitrary number of views, and Stereo High profile is designed specifically for two-view stereoscopic video. The Multiview Video Coding extensions were completed in November 2009.
 
=== 3D-AVC and MFC stereoscopic coding ===
Line 79 ⟶ 83:
* Version 18 (Edition 8): (April 13, 2013) Amendment to specify the coding of depth map data for 3D stereoscopic video, including a Multiview Depth High profile.<ref name=AVC8June2013ITURecommendations/>
* Version 19 (Edition 8): (April 13, 2013) Corrigendum to correct an error in the sub-bitstream extraction process for multiview video.<ref name=AVC8June2013ITURecommendations/>
* Version 20 (Edition 8): (April 13, 2013) Amendment to specify additional [[color space]] identifiers (including support of [[Rec. 2020|ITU-R Recommendation BT.2020]] for [[Ultra-high-definition television|UHDTV]]) and an additional model type in the tone mapping information SEI message.<ref name=AVC8June2013ITURecommendations/>
* Version 21 (Edition 9): (February 13, 2014) Amendment to specify the Enhanced Multiview Depth High profile.<ref name=AVC9February2014ITURecommendations>{{cite news |title=ITU-T Recommendation H.264 (02/2014) |publisher=ITU |url=http://www.itu.int/ITU-T/recommendations/rec.aspx?rec=12063 |date=2014-11-28 |access-date=2016-02-28}}</ref>
* Version 22 (Edition 9): (February 13, 2014) Amendment to specify the multi-resolution frame compatible (MFC) enhancement for 3D stereoscopic video, the MFC High profile, and minor corrections.<ref name=AVC9February2014ITURecommendations/>
Line 87 ⟶ 91:
* Version 26 (Edition 13): (June 13, 2019) Amendment to specify additional SEI messages for ambient viewing environment, content light level information, content color volume, equirectangular projection, cubemap projection, sphere rotation, region-wise packing, omnidirectional viewport, SEI manifest, and SEI prefix.<ref>{{Cite web|date=June 13, 2019|title=H.264: Advanced video coding for generic audiovisual services - Version 26 (Edition 13)|url=https://handle.itu.int/11.1002/1000/13903|url-status=live|archive-url=https://web.archive.org/web/20211104102930/https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=13903&lang=en|archive-date=2021-11-04|access-date=2021-11-03|website=www.itu.int}}</ref>
*Version 27 (Edition 14): (August 22, 2021) Amendment to specify additional SEI messages for annotated regions and shutter interval information, and miscellaneous minor corrections and clarifications.<ref>{{Cite web|date=August 22, 2021|title=H.264: Advanced video coding for generic audiovisual services - Version 27 (Edition 14)|url=https://handle.itu.int/11.1002/1000/14659|url-status=live|archive-url=https://web.archive.org/web/20211104103832/https://www.itu.int/ITU-T/recommendations/rec.aspx?rec=14659&lang=en|archive-date=2021-11-04|access-date=2021-11-03|website=www.itu.int}}</ref>
*Version 28 (Edition 15): (August 13, 2024) Amendment to specify additional SEI messages for neural-network postfilter characteristics, neural-network post-filter activation, and phase indication, additional colour type identifiers, and miscellaneous minor corrections and clarifications.<ref>{{Cite web|date=August 13, 2024|title=H.264: Advanced video coding for generic audiovisual services - Version 28 (Edition 15)|url=https://handle.itu.int/11.1002/1000/15935|access-date=2025-02-12|website=www.itu.int}}</ref>
 
=== Patent holders ===
{{See|MPEG LA}}
 
{{Transcluded section|MPEG LA|H.264/MPEG-4 AVC licensors|part=yes}}
 
{{trim|{{#section::MPEG LA|AVC patents}} }}
|}
 
== Applications ==
{{Further|List of video services using H.264/MPEG-4 AVC}}
The H.264 video format has a very broad application range that covers all forms of digital compressed video from low bit-rate Internet streaming applications to HDTV broadcast and Digital Cinema applications with nearly lossless coding. With the use of H.264, bit rate savings of 50% or more compared to [[MPEG-2 Part 2]] are reported. For example, H.264 has been reported to give the same Digital Satellite TV quality as current MPEG-2 implementations with less than half the bitrate, with current MPEG-2 implementations working at around 3.5&nbsp;Mbit/s and H.264 at only 1.5&nbsp;Mbit/s.<ref>{{cite journal |author=Wenger|title=RFC 3984 : RTP Payload Format for H.264 Video |newspaper=Ietf Datatracker |date=February 2005 |url=http://tools.ietf.org/html/rfc3984#page-2 |page=2|doi=10.17487/RFC3984 |display-authors=etal|url-access=subscription }}</ref> Sony claims that 9&nbsp;Mbit/s AVC recording mode is equivalent to the image quality of the [[HDV]] format, which uses approximately 18–25&nbsp;Mbit/s.<ref>{{cite web|title=Which recording mode is equivalent to the image quality of the High Definition Video (HDV) format?|website=Sony eSupport|url=https://ca.en.kb.sony.com/app/answers/detail/a_id/41994|access-date=December 8, 2018|archive-url=https://web.archive.org/web/20171109054553/https://ca.en.kb.sony.com/app/answers/detail/a_id/41994|archive-date=November 9, 2017|url-status=dead|df=mdy-all}}</ref>
 
To ensure compatibility and problem-free adoption of H.264/AVC, many standards bodies have amended or added to their video-related standards so that users of these standards can employ H.264/AVC. Both the [[Blu-ray Disc]] format and the now-discontinued [[HD DVD]] format include the H.264/AVC High Profile as one of three mandatory video compression formats. The Digital Video Broadcast project ([[Digital Video Broadcasting|DVB]]) approved the use of H.264/AVC for broadcast television in late 2004.
 
The [[Advanced Television Systems Committee]] (ATSC) standards body in the United States approved the use of H.264/AVC for broadcast television in July 2008, although the standard is not yet used for fixed ATSC broadcasts within the United States.<ref>{{cite web |url=http://www.atsc.org/cms/standards/a_72_part_1.pdf |title=ATSC Standard A/72 Part 1: Video System Characteristics of AVC in the ATSC Digital Television System |access-date=2011-07-30 |url-status=dead |archive-url=https://web.archive.org/web/20110807103348/http://www.atsc.org/cms/standards/a_72_part_1.pdf |archive-date=August 7, 2011 |df=mdy-all }}</ref><ref>{{cite web |url=http://www.atsc.org/cms/standards/a_72_part_2.pdf |title=ATSC Standard A/72 Part 2: AVC Video Transport Subsystem Characteristics |access-date=2011-07-30 |archive-url=https://web.archive.org/web/20110807103411/http://www.atsc.org/cms/standards/a_72_part_2.pdf |archive-date=August 7, 2011 |url-status=dead |df=mdy-all }}</ref> It has also been approved for use with the more recent [[ATSC-M/H]] (Mobile/Handheld) standard, using the AVC and SVC portions of H.264.<ref>{{cite web |url=http://atsc.org/cms/standards/a153/a_153-Part-7-2009.pdf |title=ATSC Standard A/153 Part 7: AVC and SVC Video System Characteristics |access-date=2011-07-30 |url-status=dead |archive-url=https://web.archive.org/web/20110726151422/http://atsc.org/cms/standards/a153/a_153-Part-7-2009.pdf |archive-date=July 26, 2011 |df=mdy-all }}</ref>
 
The [[CCTV]]closed-circuit (Closed Circuit TV)television|closed-circuit-television and [[Video Surveillancevideo-surveillance]] markets have included the technology in many products.
 
Many common [[DSLR]]s use H.264 video wrapped in QuickTime MOV containers as the native recording format.
Line 113 ⟶ 110:
[[AVC-Intra]] is an [[Video compression#Encoding theory|intraframe]]-only compression format, developed by [[Panasonic]].
 
[[XAVC]] is a recording format designed by Sony that uses level 5.2 of H.264/MPEG-4 AVC, which is the highest level supported by that video standard.<ref name=SonyXAVCrecordingformat>{{cite news |title=Sony introduces new XAVC recording format to accelerate 4K development in the professional and consumer markets |publisher=Sony |url=http://www.sony.co.uk/pro/article/broadcast-xavc-codec-1012 |date=2012-10-30 |access-date=2012-11-01}}</ref><ref name=SonyXAVCrecordingformatpdf>{{cite news |title=Sony introduces new XAVC recording format to accelerate 4K development in the professional and consumer markets |publisher=Sony |url=http://www.xavc-info.org/resource/1351573070000/xavcsite/share/data/XAVC_30-Oct-2012_NewsRelease.pdf |datearchive-url=2012https://web.archive.org/web/20230323100304/http://www.xavc-10info.org/resource/1351573070000/xavcsite/share/data/XAVC_30-30Oct-2012_NewsRelease.pdf |accessurl-datestatus=2012-11-01 }}{{dead link|archive-date=OctoberMarch 201723, 2023 |botdate=InternetArchiveBot2012-10-30 |fixaccess-attempteddate=yes2012-11-01 }}</ref> XAVC can support [[4K resolution]] (4096 × 2160 and 3840 × 2160) at up to 60&nbsp;[[frames per second]] (fps).<ref name=SonyXAVCrecordingformat/><ref name=SonyXAVCrecordingformatpdf/> Sony has announced that cameras that support XAVC include two [[CineAlta]] cameras—the Sony PMW-F55 and Sony PMW-F5.<ref name=Engadget4KCineAltaF55F5>{{cite news |title=Sony goes Red-hunting with PMW-F55 and PMW-F5 pro CineAlta 4K Super 35mm sensor camcorders |publisher=Engadget |url=https://www.engadget.com/2012/10/30/sony-goes-red-hunting-with-pmw-f55-and-pmw-f5-pro-cinealta-4k/ |author=Steve Dent |date=2012-10-30 |access-date=2012-11-05}}</ref> The Sony PMW-F55 can record XAVC with 4K resolution at 30&nbsp;fps at 300 [[Mbit/s]] and 2K resolution at 30&nbsp;fps at 100&nbsp;Mbit/s.<ref name=SonyPMW-F55datasheet>{{cite news |title=F55 CineAlta 4K the future, ahead of schedule |publisher=Sony |url=http://pro.sony.com/bbsccms/assets/files/show/highend/pdf/F55_Camera.pdf |date=2012-10-30 |access-date=2012-11-01 |url-status=dead |archive-url=https://web.archive.org/web/20121119001306/http://pro.sony.com/bbsccms/assets/files/show/highend/pdf/F55_Camera.pdf |archive-date=November 19, 2012 |df=mdy-all }}</ref> XAVC can record 4K resolution at 60&nbsp;fps with 4:2:2 chroma sampling at 600&nbsp;Mbit/s.<ref name=SonySxSPro4Kvideo>{{cite news |title=Ultra-fast "SxS PRO+" memory cards transform 4K video capture |publisher=Sony |url=http://www.sony.co.uk/pro/article/km-rme-sxspro+-launch |access-date=2012-11-05 |url-status=dead |archive-url=https://web.archive.org/web/20130308090520/http://www.sony.co.uk/pro/article/km-rme-sxspro+-launch |archive-date=March 8, 2013 |df=mdy-all }}</ref><ref name=SonySxSPro4Kvideopdf>{{cite news |title=Ultra-fast "SxS PRO+" memory cards transform 4K video capture |publisher=Sony |url=http://www.sony.co.uk/res/attachment/file/60/1237488996260.pdf |access-date=2012-11-05 |url-status=dead |archive-url=https://web.archive.org/web/20150402123222/http://www.sony.co.uk/res/attachment/file/60/1237488996260.pdf |archive-date=April 2, 2015 |df=mdy-all }}</ref>
 
== Design ==
Line 127 ⟶ 124:
** The ability to use any macroblock type in [[Video compression picture types#Bi-directional predicted frames/slices (B-frames/slices)|B-frames]], including I-macroblocks, resulting in much more efficient encoding when using B-frames. This feature was notably left out from [[MPEG-4 ASP]].
** Six-tap filtering for derivation of half-pel luma sample predictions, for sharper subpixel motion-compensation. Quarter-pixel motion is derived by linear interpolation of the halfpixel values, to save processing power.
** [[Qpel|Quarter-pixel]] precision for motion compensation, enabling precise description of the displacements of moving areas. For [[Chrominance|chroma]] the resolution is typically halved both vertically and horizontally (see [[4:2:0]]) therefore the motion compensation of chroma uses one-eighth chroma pixel grid units.
** Weighted prediction, allowing an encoder to specify the use of a scaling and offset when performing motion compensation, and providing a significant benefit in performance in special cases—such as fade-to-black, fade-in, and cross-fade transitions. This includes implicit weighted prediction for B-frames, and explicit weighted prediction for P-frames.
* Spatial prediction from the edges of neighboring blocks for [[Intra-frame|"intra"]] coding, rather than the "DC"-only prediction found in MPEG-2 Part 2 and the transform coefficient prediction found in H.263v2 and MPEG-4 Part 2. This includes luma prediction block sizes of 16×16, 8×8, and 4×4 (of which only one type can be used within each [[macroblock]]).
Line 144 ⟶ 141:
** Logarithmic step size control for easier bit rate management by encoders and simplified inverse-quantization scaling
** Frequency-customized quantization scaling matrices selected by the encoder for perceptual-based quantization optimization
* An in-loop [[Deblocking filter (video)|deblocking filter]] that helps prevent the blocking artifacts common to other DCT-based image compression techniques, resulting in better visual appearance and compression efficiency
* An [[entropy encoding|entropy coding]] design including:
** [[Context-adaptive binary arithmetic coding]] (CABAC), an algorithm to losslessly compress syntax elements in the video stream knowing the probabilities of syntax elements in a given context. CABAC compresses data more efficiently than CAVLC but requires considerably more processing to decode.
** [[Context-adaptive variable-length coding]] (CAVLC), which is a lower-complexity alternative to CABAC for the coding of quantized transform coefficient values. Although lower complexity than CABAC, CAVLC is more elaborate and more efficient than the methods typically used to code coefficients in other prior designs.
** A common simple and highly structured [[Variable-length code|variable length coding]] (VLC) technique for many of the syntax elements not coded by CABAC or CAVLC, referred to as [[Exponential-Golomb coding]] (or Exp-Golomb).
* Loss resilience features including:
** A [[Network Abstraction Layer]] (NAL) definition allowing the same video syntax to be used in many network environments. One very fundamental design concept of H.264 is to generate self-contained packets, to remove the header duplication as in MPEG-4's Header Extension Code (HEC).<ref name="rfc3984_3"/> This was achieved by decoupling information relevant to more than one slice from the media stream. The combination of the higher-level parameters is called a parameter set.<ref name="rfc3984_3"/> The H.264 specification includes two types of parameter sets: Sequence Parameter Set (SPS) and Picture Parameter Set (PPS). An active sequence parameter set remains unchanged throughout a coded video sequence, and an active picture parameter set remains unchanged within a coded picture. The sequence and picture parameter set structures contain information such as picture size, optional coding modes employed, and macroblock to slice group map.<ref name="rfc3984_3">RFC 3984, p.3</ref>
** [[Flexible macroblock ordering]] (FMO), also known as slice groups, and arbitrary slice ordering (ASO), which are techniques for restructuring the ordering of the representation of the fundamental regions (''macroblocks'') in pictures. Typically considered an error/loss robustness feature, FMO and ASO can also be used for other purposes.
** Data partitioning (DP), a feature providing the ability to separate more important and less important syntax elements into different packets of data, enabling the application of unequal error protection (UEP) and other types of improvement of error/loss robustness.
Line 178 ⟶ 175:
;Extended Profile (XP, 88): Intended as the streaming video profile, this profile has relatively high compression capability and some extra tricks for robustness to data losses and server stream switching.
;Main Profile (MP, 77): This profile is used for standard-definition digital TV broadcasts that use the MPEG-4 format as defined in the DVB standard.<ref>{{cite web|url=http://www.etsi.org/deliver/etsi_ts/101100_101199/101154/01.09.01_60/ts_101154v010901p.pdf |title=TS 101 154&nbsp;– V1.9.1&nbsp;– Digital Video Broadcasting (DVB); Specification for the use of Video and Audio Coding in Broadcasting Applications based on the MPEG-2 Transport Stream |access-date=2010-05-17}}</ref> It is not, however, used for high-definition television broadcasts, as the importance of this profile faded when the High Profile was developed in 2004 for that application.
;High Profile (HiP, 100): The primary profile for broadcast and disc storage applications, particularly for high-definition television applications (for example, this is the profile adopted by the [[Blu-ray Disc]] storage format and the [[Digital Video Broadcasting|DVB]] HDTV broadcast service).
;Progressive High Profile (PHiP, 100 with constraint set 4): Similar to the High profile, but without support of field coding features.
;Constrained High Profile (100 with constraint set 4 and 5): Similar to the Progressive High profile, but without support of B (bi-predictive) slices.
Line 271 ⟶ 268:
| {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}
|-
! 4:0:0 ([[MonochromeGreyscale]])
| {{no}} || {{no}} || {{no}} || {{no}} || {{yes}} || {{yes}} || {{yes}} || {{yes}} || {{yes}}
|-
Line 488 ⟶ 485:
== Implementations ==
[[File:YouTube H264 video with Opus audio stat screenshot.png|upright=1.2|thumb|A YouTube video statistics with AVC (H.264) video codec and [[Opus (audio format)|Opus]] audio format]]
In 2009, the [[WHATWG|HTML5 working group]] was split between supporters of Ogg [[Theora]], a free video format which is thought to be unencumbered by patents, and H.264, which contains patented technology. As late as July 2009, Google and Apple were said to support H.264, while Mozilla and Opera support Ogg Theora (now Google, Mozilla and Opera all support Theora and [[WebM]] with [[VP8]]).<ref>{{cite web|url=https://arstechnica.com/open-source/news/2009/07/decoding-the-html-5-video-codec-debate.ars |title=Decoding the HTML 5 video codec debate |website=Ars Technica |date=2009-07-06 |access-date=2011-01-12}}</ref> Microsoft, with the release of Internet Explorer 9, has added support for HTML 5 video encoded using H.264. At the Gartner Symposium/ITXpo in November 2010, Microsoft CEO Steve Ballmer answered the question "HTML 5 or [[Silverlight]]?" by saying "If you want to do something that is universal, there is no question the world is going HTML5."<ref>{{cite web|url=https://www.youtube.com/watch?v=iI47b3a9cEI | archive-url=https://ghostarchive.org/varchive/youtube/20211030/iI47b3a9cEI| archive-date=2021-10-30|title= Steve Ballmer, CEO Microsoft, interviewed at Gartner Symposium/ITxpo Orlando 2010 |publisher=Gartnervideo |date=November 2010|access-date=2011-01-12}}{{cbignore}}</ref> In January 2011, Google announced that they were pulling support for H.264 from their Chrome browser and supporting both Theora and [[WebM]]/[[VP8]] to use only open formats.<ref>{{cite web|url=https://blog.chromium.org/2011/01/html-video-codec-support-in-chrome.html |title=HTML Video Codec Support in Chrome |date=2011-01-11 |access-date=2011-01-12}}</ref>
 
On March 18, 2012, [[Mozilla]] announced support for H.264 in Firefox on mobile devices, due to prevalence of H.264-encoded video and the increased power-efficiency of using dedicated H.264 decoder hardware common on such devices.<ref>{{cite web|url=https://hacks.mozilla.org/2012/03/video-mobile-and-the-open-web/ | title=Video, Mobile, and the Open Web|date=2012-03-18|access-date=2012-03-20}}</ref> On February 20, 2013, Mozilla implemented support in Firefox for decoding H.264 on Windows 7 and above. This feature relies on Windows' built in decoding libraries.<ref>{{cite web|title=WebRTC enabled, H.264/MP3 support in Win 7 on by default, Metro UI for Windows 8 + more&nbsp;– Firefox Development Highlights|url=https://hacks.mozilla.org/2013/02/webrtc-enabled-h-264mp3-support-in-win-7-on-by-default-metro-ui-for-windows-8-more-firefox-development-highlights/|work=hacks.mozilla.org|publisher=mozilla|access-date=2013-03-15|date=2013-02-20}}</ref> Firefox 35.0, released on January 13, 2015, supports H.264 on OS X 10.6 and higher.<ref>{{cite web|url=https://www.mozilla.org/en-US/firefox/35.0/releasenotes/|title=Firefox — Notes (35.0)|website=Mozilla}}</ref>
 
On October 30, 2013, [[Rowan Trollope]] from [[Cisco Systems]] announced that Cisco would release both binaries and source code of an H.264 video codec called [[OpenH264]] under the [[Simplified BSD license]], and pay all royalties for its use to MPEG LA for any software projects that use Cisco's precompiled binaries, thus making Cisco's OpenH264 ''binaries'' free to use. However, any software projects that use Cisco's source code instead of its binaries would be legally responsible for paying all royalties to MPEG LA. Target CPU architectures include x86 and ARM, and target operating systems include Linux, Windows XP and later, Mac OS X, and Android; iOS was notably absent from this list, because it doesn'tdoes not allow applications to fetch and install binary modules from the Internet.<ref>{{cite web |url=http://blogs.cisco.com/collaboration/open-source-h-264-removes-barriers-webrtc |title=Open-Sourced H.264 Removes Barriers to WebRTC |date=2013-10-30 |access-date=2013-11-01 |archive-url=https://web.archive.org/web/20150706222941/http://blogs.cisco.com/collaboration/open-source-h-264-removes-barriers-webrtc |archive-date=July 6, 2015 |url-status=dead }}</ref><ref name=OpenH264faq>{{cite web|url=http://www.openh264.org/faq.html |title=Cisco OpenH264 project FAQ |access-date=2021-09-26}}</ref><ref>{{cite web|url=https://github.com/cisco/openh264/blob/master/LICENSE |title=OpenH264 Simplified BSD License |website=[[GitHub]] |date=2013-10-27 |access-date=2013-11-21}}</ref> Also on October 30, 2013, [[Brendan Eich]] from [[Mozilla]] wrote that it would use Cisco's binaries in future versions of Firefox to add support for H.264 to Firefox where platform codecs are not available.<ref>{{cite web|url=https://blog.mozilla.org/blog/2013/10/30/video-interoperability-on-the-web-gets-a-boost-from-ciscos-h-264-codec/ |title=Video Interoperability on the Web Gets a Boost From Cisco's H.264 Codec |date=2013-10-30 |access-date=2013-11-01}}</ref> Cisco published the source code to OpenH264 on December 9, 2013.<ref>{{cite web|url=https://github.com/cisco/openh264/commit/59dae50b1069dbd532226ea024a3ba3982ab4386|title=Updated README · cisco/openh264@59dae50|website=GitHub}}</ref>
 
Although iOS was not supported by the 2013 Cisco software release, Apple updated its Video Toolbox Framework with [[iOS 8]] (released in September 2014) to provide direct access to hardware-based H.264/AVC video encoding and decoding.<ref name=OpenH264faq/>
Line 553 ⟶ 550:
The 2nd generation [[Intel]] "[[Sandy Bridge]]" [[Intel Core|Core i3/i5/i7]] processors introduced at the January 2011 CES ([[Consumer Electronics Show]]) offer an on-chip hardware full HD H.264 encoder, known as [[Intel Quick Sync Video]].<ref>{{cite web |url=http://software.intel.com/en-us/articles/quick-reference-guide-to-intel-integrated-graphics/ |title=Quick Reference Guide to generation Intel Core Processor Built-in Visuals |publisher=Intel Software Network |date=2010-10-01 |access-date=2011-01-19}}</ref><ref>{{cite web |url=http://www.intel.com/content/www/us/en/architecture-and-technology/quick-sync-video/quick-sync-video-general.html |title=Intel Quick Sync Video|publisher=www.intel.com |date=2010-10-01 |access-date=2011-01-19}}</ref>
 
A hardware H.264 encoder can be an [[Application-specific integrated circuit|ASIC]] or an [[Field-programmable gate array|FPGA]].
 
ASIC encoders with H.264 encoder functionality are available from many different semiconductor companies, but the core design used in the ASIC is typically licensed from one of a few companies such as [[Chips&Media]], Allegro DVT, [[On2]] (formerly Hantro, acquired by Google), [[Imagination Technologies]], NGCodec. Some companies have both FPGA and ASIC product offerings.<ref>{{cite web |url=http://www.design-reuse.com/sip/?q=H.264+encoder |title=Design-reuse.com |publisher=Design-reuse.com |date=1990-01-01 |access-date=2010-05-17}}</ref>
 
Texas Instruments manufactures a line of ARM + DSP cores that perform DSP H.264 BP encoding 1080p at 30fps30&nbsp;fps.<ref>{{cite web |url=http://processors.wiki.ti.com/index.php/Category:DM6467 |title=Category:DM6467 - Texas Instruments Embedded Processors Wiki |publisher=Processors.wiki.ti.com |date=2011-07-12 |access-date=2011-07-30 |archive-date=July 17, 2011 |archive-url=https://web.archive.org/web/20110717053351/http://processors.wiki.ti.com/index.php/Category:DM6467 |url-status=dead }}</ref> This permits flexibility with respect to codecs (which are implemented as highly optimized DSP code) while being more efficient than software on a generic CPU.
 
== Licensing ==
{{see also|Microsoft Corp. v. Motorola Inc.|Qualcomm Inc. v. Broadcom Corp.}}
In countries where [[software patent|patents on software algorithms]] are upheld, vendors and commercial users of products that use H.264/AVC are expected to pay patent licensing royalties for the patented technology that their products use.<ref>{{cite web |url=http://www.mpegla.com/main/programs/AVC/Documents/avcweb.pdf |title=Briefing portfolio |website=www.mpegla.com |access-date=December 1, 2016 |archive-date=November 28, 2016 |archive-url=https://web.archive.org/web/20161128050907/http://www.mpegla.com/main/programs/AVC/Documents/avcweb.pdf |url-status=dead }}</ref> This applies to the Baseline Profile as well.<ref>{{cite web|url=http://blogs.sun.com/openmediacommons/entry/oms_video_a_project_of|title=OMS Video, A Project of Sun's Open Media Commons Initiative|access-date=2008-08-26|archive-url=https://web.archive.org/web/20100511060302/http://blogs.sun.com/openmediacommons/entry/oms_video_a_project_of|archive-date=May 11, 2010|url-status=dead|df=mdy-all}}</ref>
 
A private organization known as [[MPEG LA]], which is not affiliated in any way with the MPEG standardization organization, administers the licenses for patents applying to this standard, as well as other [[patent pool]]s, such as for MPEG-4 Part 2 Video, HEVC and MPEG-DASH. The patent holders include [[Fujitsu]], [[Panasonic]], [[Sony]], [[Mitsubishi]], [[Apple Inc.|Apple]], [[Columbia University]], [[KAIST]], [[Dolby Laboratories|Dolby]], [[Google]], [[JVC Kenwood]], [[LG Electronics]], [[Microsoft]], [[NTT Docomo]], [[Philips]], [[Samsung]], [[Sharp Corporation|Sharp]], [[Toshiba]] and [[ZTE]],<ref>{{cite web |title=Licensors Included in the AVC/H.264 Patent Portfolio License |url=https://www.mpegla.com/programs/avc-h-264/licensors/ |website=[[MPEG LA]] |access-date=18 June 2019 |archive-date=October 11, 2021 |archive-url=https://web.archive.org/web/20211011082410/https://www.mpegla.com/programs/avc-h-264/licensors/ |url-status=dead }}</ref> although the majority of patents in the pool are held by [[Panasonic]] ({{formatnum:{{#expr:1137+60}}|}} patents), [[Gōdō gaisha|Godo Kaisha]] IP Bridge ({{formatnum:{{#expr:1111+19}}|}} patents) and [[LG Electronics]] ({{#expr:949+(40+1)}} patents).<ref name="patents">{{cite web |title=AVC/H.264 {{ndash}} Patent List |url=https://www.via-la.com/licensing-2/avc-h-264/avc-h-264-patent-list/ |website=Via Licensing Alliance |access-date=28 April 2024}}</ref>
 
On August 26, 2010, MPEG LA announced that royalties won'twould not be charged for H.264 encoded Internet video that is free to end users.<ref>{{cite web|url=http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/74/n-10-08-26.pdf|title=MPEG LA's AVC License Will Not Charge Royalties for Internet Video that is Free to End Users through Life of License|publisher=MPEG LA|date=2010-08-26|access-date=2010-08-26|archive-date=November 7, 2013|archive-url=https://web.archive.org/web/20131107135621/http://www.mpegla.com/Lists/MPEG%20LA%20News%20List/Attachments/74/n-10-08-26.pdf|url-status=dead}}</ref> All other royalties remain in place, such as royalties for products that decode and encode H.264 video as well as to operators of free television and subscription channels.<ref>{{cite news|url=https://www.pcmag.com/article2/0,2817,2368359,00.asp |title=MPEG LA Cuts Royalties from Free Web Video, Forever |publisher=pcmag.com |date=2010-08-26 |access-date=2010-08-26 |first=Mark |last=Hachman}}</ref> The license terms are updated in 5-year blocks.<ref>{{cite web |url=http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx |title=AVC FAQ |publisher=MPEG LA |date=2002-08-01 |access-date=2010-05-17 |archive-url=https://web.archive.org/web/20100507102710/http://www.mpegla.com/main/programs/AVC/Pages/FAQ.aspx |archive-date=May 7, 2010 |url-status=dead }}</ref>
 
Since the first version of the standard was completed in May 2003 ({{age|month=May|year=2003}} years ago) and the most commonly used profile (the High profile) was completed in June 2004{{Citation needed|date=December 2023}} ({{age|month=June|year=2004}} years ago), some of the relevant patents are expired by now, <ref name="patents" /> while others are still in force in jurisdictions around the world and one of the US patents in the MPEG LA H.264 pool (granted in 2016, priority from 2001) lasts at least until November 2030.<ref>{{cite web |url=https://patents.google.com/patent/US9356620B2/ |title=United States Patent 9,356,620 Baese, et al. |access-date=August 1, 2022}} with an earliest priority date of September 14, 2001 has a 2,998 day term extension.</ref>
Line 592 ⟶ 589:
== Further reading ==
{{Refbegin}}
* {{Cite journal |url = http://ip.hhi.de/imagecom_G1/assets/pdfs/csvt_overview_0305.pdf |first1 = Thomas |last1 = Wiegand |first2 = Gary J. |last2 = Sullivan |first3 = Gisle |last3 = Bjøntegaard |first4 = Ajay |last4 = Luthra |title = Overview of the H.264/AVC Video Coding Standard |journal = IEEE Transactions on Circuits and Systems for Video Technology |date = July 2003 |volume = 13 |issue = 7 |pages = 560–576 |doi = 10.1109/TCSVT.2003.815165 |access-date = January 31, 2011 |archive-date = April 29, 2011 |archive-url = https://web.archive.org/web/20110429013301/http://ip.hhi.de/imagecom_G1/assets/pdfs/csvt_overview_0305.pdf |url-status = dead }}
* {{Cite journal |url = http://www.fastvdo.com/spie04/spie04-h264OverviewPaper.pdf |first1 = Pankaj |last1 = Topiwala |first2 = Gary J. |last2 = Sullivan |first3 = Ajay |last3 = Luthra |editor1-first = Andrew G |editor1-last = Tescher |s2cid = 2308860 |title = The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions |journal = SPIE Applications of Digital Image Processing XXVII |series = Applications of Digital Image Processing XXVII |date = August 2004 |volume = 5558 |page = 454 |doi = 10.1117/12.564457 |bibcode = 2004SPIE.5558..454S |access-date = January 31, 2011 }}
* {{Cite journalFTP |url = ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/CASM_2004_MN.pdf |archive-url = https://web.archive.org/web/20170706033551/ftp://ftp.tnt.uni-hannover.de/pub/papers/2004/CASM_2004_MN.pdf |url-status = dead |archive-date = 2017-07-06 |first1 = J. |last1 = Ostermann |first2 = J. |last2 = Bormans |first3 = P. |last3 = List |first4 = D. |last4 = Marpe |first5 = M. |last5 = Narroschke |first6 = F. |last6 = Pereira |first7 = T. |last7 = Stockhammer |first8 = T. |last8 = Wedi |s2cid = 11105089 |title = Video coding with H.264/AVC: Tools, Performance, and Complexity |journal = IEEE Circuits and Systems Magazine |date = 2004 |volume = 4 |issue = 1 |pages = 7–28 |doi = 10.1109/MCAS.2004.1286980 |server = IEEE Circuits and Systems Magazine |url-status = dead |access-date = January 31, 2011 }}
* {{Cite journal |url = https://archive.org/download/video-coding-using-the-h.-264-mpeg-4-avc-compression-standard/Video%20coding%20using%20the%20H.264%3AMPEG-4%20AVC%20compression%20standard.pdf |first1 = Atul |last1 = Puri |first2 = Xuemin |last2 = Chen |first3 = Ajay |last3 = Luthra |title = Video coding using the H.264/MPEG-4 AVC compression standard |journal = Signal Processing: Image Communication |date = October 2004 |volume = 19 |issue = 9 |pages = 793–849 |access-date = March 30, 2011 |doi = 10.1016/j.image.2004.06.003 }}
* {{Cite journal |url = http://ip.hhi.de/imagecom_G1/assets/pdfs/pieee_sullivan_wiegand_2005.pdf |first1 = Gary J. |last1 = Sullivan |first2 = Thomas |last2 = Wiegand |s2cid = 1362034 |title = Video Compression—From Concepts to the H.264/AVC Standard |journal = Proceedings of the IEEE |date = January 2005 |volume = 93 |issue = 1 |pages = 18–31 |access-date = January 31, 2011 |doi = 10.1109/jproc.2004.839617 }}
* {{cite web |url=http://www.vcodex.com/h264.html |title=Learn about video compression and H.264 |work=VCODEX |publisher=Vcodex Limited |first1=Iain E. G. |last1=Richardson |access-date=January 31, 2011 |date=January 2011 |archive-date=January 28, 2011 |archive-url=https://web.archive.org/web/20110128164552/http://www.vcodex.com/h264.html |url-status=dead }}
{{Refend}}
 
== External links ==
* {{Official website|http://www.itu.int/rec/T-REC-H.264|ITU-T publication page: H.264: Advanced video coding for generic audiovisual services}}
* [httphttps://forum.doom9.org/showthread.php?t=96059 MPEG-4 AVC/H.264 Information] Doom9's Forum
* [https://web.archive.org/web/20040603025720/http://www.vcodex.com/h264.html H.264/MPEG-4 Part 10 Tutorials (Richardson)]
* {{cite web|url=http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=56538|work=ISO publication page: ISO/IEC 14496-10:2010&nbsp;– Information technology&nbsp;— Coding of audio-visual objects | title = Part 10: Advanced Video Coding}}
* {{cite web|url=http://iphome.hhi.de/suehring/tml/|work=IP Homepage | title = H.264/AVC JM Reference Software|access-date=2007-04-15}}
Line 612 ⟶ 609:
* {{cite web|url=http://www.networkwebcams.com/ip-camera-learning-center/2009/04/03/using-h264-video-compression-in-ip-video-surveillance-systems/| title=Discussion on H.264 with respect to IP cameras in use within the security and surveillance industries| date=April 3, 2009}} (dated April 2009)
* {{cite web|url=http://compression.ru/video/codec_comparison/h264_2010/| title=Sixth Annual H.264 video codecs comparison | publisher = [[Moscow State University]]}} (dated May 2010)
* {{cite web|url=https://support.etcconnect.com/ETC/FAQ/SMPTE_QuickGuide/ | title= SMPTE QuickGuide | date= May 14, 2018 | publisher = ETC}}
 
{{Compression formats}}