High Efficiency Image File Format: Difference between revisions

Content deleted Content added
Explanation of official MP4RA registar and ISO jargon. BRANDS.
 
(37 intermediate revisions by 26 users not shown)
Line 18:
| standard = ISO/IEC 23008-12 (MPEG-H Part 12)
}}
'''High Efficiency Image File Format''' ('''HEIF''') is a digital [[container format]] for storing individual [[digital image]]s and image sequences. The standard covers [[multimedia]] files that can also include other media streams, such as [[timed text]], audio and video.<ref name="JCTVC" />
'''High Efficiency Image File Format''' ('''HEIF''') is an international standard defined by MPEG-H Part 12 (ISO/IEC 3008-12), first published by ISO in 2017. It is designed as a container for photographic images in any image encoding. '''HEIF is a special case of the general ISO BMFF format''', in which all data is encapsulated in typed boxes, with a mandatory ftyp box that is used to indicate particular file types. The initial specification for HEIF provided usage details for three compression schemes, the widely supported JPEG encoding for still raster images and two video encodings that are also applicable to still image items, namely Advanced Video Coding (AVC, aka MPEG-4, Part 10 and H.264) and High Efficiency Video Coding (HEVC aka PEG-H Part 2 and H.265).
 
HEIF can store images encoded with multiple coding formats, for example both [[Standard-dynamic-range video|SDR]] and [[High-dynamic-range video|HDR]] images. [[High Efficiency Video Coding|HEVC]] is an image and video encoding format and the default image codec used with HEIF. HEIF files containing HEVC-encoded images are also known as '''HEIC''' files. Such files require less storage space than the equivalent quality [[JPEG]].<ref name="CNET" /><ref name="LesShu" />
ISO Base Media File Format official specification (of which HEIF is a special case) defines multiple subtypes in relations to possible encodings:
 
HEIF files are a special case of the [[International Organization for Standardization|ISO]] Base Media File Format ([[ISO base media file format|ISOBMFF]], ISO/IEC 14496-12), first defined in 2001 as a shared part of [[MP4]] and [[JPEG 2000]]. Introduced in 2015, it was developed by the [[Moving Picture Experts Group]] (MPEG) and is defined as Part 12 within the [[MPEG-H]] media suite (ISO/IEC 23008-12).
* MJP2_FF, Motion JPEG 2000 File Format
* MP4_FF_2, MPEG-4 File Format, version 2
* MP4_FF_AVCE, MPEG-4 File Format for AVC (Extensions; Part 15)
* JP2_FF, JPEG 2000 Part 1 (Core) jp2 File Format
* JPX_FF, JPEG 2000 Part 2 (Extensions) jpf File Format
* JPM_FF, JPEG 2000 Part 1 (Compound) jpm File Format
* JPEG_XL, JPEG XL Image Encoding
* JXL, JPEG XL File Format
* HEIF, High Efficiency Image File Format. Some internal structures required for HEIF were added into the ISO_BMFF specification in a 2018 amendment. (ISO/IEC 14496-12:2015/Amd 2:2018)
 
 
 
'''BMFF HEIC/HEIX Registered Brands'''
 
Specific implementations of HEIF encoding are registered by MP4RA and are officially called "brands".
 
According to official specifications Apple HEIC registered brand is using Main or Main Still Picture profile of HEVC with an 8-bit per channel color depth and chroma subsampling of 4:2:0, while Sony and Canon HIEX registered brand may additionally increase the bit-depth to 10 bits per channel and chroma subsampling to 4:2:2 or 4:4:4.
 
== History ==
Line 50 ⟶ 34:
On some systems, pictures stored in the HEIC format are converted automatically to the older JPEG format when they are sent outside of the system, although incompatibility has led to problems such as US [[Advanced Placement]] test takers failing due to their phones uploading unsupported HEIC images by default,<ref name=":1">{{cite web |url=https://www.theverge.com/2020/5/20/21262302/ap-test-fail-iphone-photos-glitch-email-college-board-jpeg-heic |date=2020-05-02 |access-date=2022-08-25 |title=Students are failing AP tests because the College Board can't handle iPhone photos |publisher=[[The Verge]] |author= Monica Chin}}</ref> leading the [[College Board]] to request students change the settings to send only JPEG files.<ref>{{cite tweet |user=CollegeBoard |number=1260208235132436480 |title=If you want to submit a photo of a handwritten AP Exam answer from an iPhone or iPad, make sure to change your camera settings so your photos are saved as JPEGs, not HEICs.}}</ref>
 
Although HEIC is gaining in popularity, thereit areis issuesnot withuniversally compatibility,supported; [[Adobe Photoshop]] is an example of a popular image editing software that only supports 8-bit HEIC and not 10-bit or 12-bit HEIC yet.<ref>{{Cite web |title=File formats in Adobe Photoshop |url=https://helpx.adobe.com/au/photoshop/using/file-formats.html}}</ref>
 
Camera hardware (including mobile devices) are increasingly supporting outputting HEIC files and with color depth often higher than 8-bit color.<ref>{{Cite web |title=A bit confused: what's a HEIF and why do we need 10-bit stills and video? |url=https://www.dpreview.com/learn/6184595294/10-bit-stills-a-look-at-raw-log-and-the-future-of-photography |access-date=2023-11-11 |website=DPReview}}</ref>
Line 61 ⟶ 45:
; Image sequences:Storage of multiple time-related and/or temporally predicted images (like a [[burst mode (photography)|burst-photo shot]] or [[cinemagraph]] animation), their properties and thumbnails. Different prediction options can be used in order to exploit the temporal and spatial similarities between the images. Hence, file sizes can be drastically reduced when many images are stored in the same HEIF file.
; Auxiliary image items: Storage of image data, such as an [[alpha channel|alpha plane]] or a [[depth map]], which complements another image item. These data are not displayed as such, but used in various forms to complement another image item.
; Image metadata: Storage of [[Exif|EXIF]], [[Extensible Metadata Platform|XMP]] and similar metadata which accompany the images stored in the HEIF file.
 
== Encodings inside the container ==
The HEIF container can store the following encodings:
<!-- Improve the below paragraph as it doesn't adhere to Wikipedia's standards. -->HEIF container specification describes various encodings that can be stored inside:
 
* JFIF (JPEG)
Line 70 ⟶ 54:
* HEVC
 
These leads to confusion, asAs users cannot easily tell what kindencoding ofand encoding hasparameters beenan used.image Simplywas sayingstored thatin, “HEIFthe isHEIF bettercontainer thanformat JPEG”can asbe promotionalconfusing materialand doesn'tmakes meancomparison anythingstatements tolike the“HEIF is better than JPEG” vague and userinaccurate.
 
Simply knowing a file is in the HEIF container does not reveal much information, as it could be:
Inside the HEIF container, JFIF (JPEG) can be stored. It can also contain AV1 encoding. The user would also need to know the encoding parameters in which the encoding happened (e.g., color quality "q", if delta in chroma planes were used, if some advanced tuning for ssim/psnr was used, or what quantization mode was applied).
 
If a user creates a ".HEIF" photo with their phone, the user or the receiver of the photo cannot determine which image format inside the file was used. It could be:
 
* a JFIF (JPEG);
Line 81 ⟶ 63:
* an HEVC with poor quality parameters; or
* an HEVC with high quality parameters.
 
Hence, simply stating in promotional materials that “HEIF is better” doesn't mean anything at all, as HEIF is just a container specification. What is contained inside the container must be described.
 
It's impossible to easily determine exactly what you receive in HEIF container. At best, users can use graphics magic to determine if it's impossible by visual inspection (gradients transitions are broken in 8-bit photos) to tell if the photo is 12-bit or 10-bit per channel, or 8-bit.
 
{{anchor|MIAF}}{{anchor|MPEG-A}}
Line 93 ⟶ 71:
 
{{anchor|HEIC}}{{anchor|HEVC}}
 
=== HEIC: HEVC in HEIF ===
[[High Efficiency Video Coding]] (HEVC, ITU-T H.265)<ref>{{cite web|url=https://tidbits.com/2020/06/26/what-the-heic-apples-highly-compressed-image-format-still-confuses/|title=What the HEIC? Apple's Highly Compressed Image Format Still Confuses|publisher=TidBITS|first=Glenn|last=Fleishman|date=26 June 2020|access-date=28 June 2020|archive-date=28 June 2020|archive-url=https://web.archive.org/web/20200628103743/https://tidbits.com/2020/06/26/what-the-heic-apples-highly-compressed-image-format-still-confuses/|url-status=live}}</ref> is an encoding format for graphic data, first standardized in 2013.
It is the primarily used and implied default codec for HEIF as specified in the normative Annex B to ISO/IEC 23008-12 ''HEVC Image File Format''.
 
While not introduced formally in the standard, the acronym <dfn><abbr title="High-Efficiency Image Codec">HEIC</abbr></dfn> (High-Efficiency Image Codec) is used as a ''brand'' and in the MIME subtypes <code>image/heic</code> and <code>image/heic-sequence</code>. If the content conforms to certain [[HEVC#Profiles|HEVC profiles]], more specific brands can be used: <dfn>HEIX</dfn> for Main 10 of HEVC, <dfn>HEIM</dfn> for (Multiview) Main profile, and <dfn>HEIS</dfn> for (Scalable) Main (10) profile of L-HEVC.
 
A HEIC photo takes up about half the space of an equivalent quality JPEG file.<ref name=HEVCApril2012I0461>{{cite news |title=On HEVC still picture coding performance |first1=Jani |last1=Lainema |first2=Kemal |last2=Ugur |publisher=JCT-VC |url=http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=5721 |date=2012-04-20 |access-date=2013-01-22}}</ref> The initial HEIF specification already defined the means of storing HEVC-encoded [[Video compression picture types|intra images]] (''i-frames'') and HEVC-encoded image sequences in which [[inter frame|inter prediction]] is applied in a constrained manner.
 
HEVC<!-- or HEIF? --> image players are required to support rectangular cropping and rotation by one, two, and three quarter-turns. The primary use case for the mandatory support for rotation by 90 degrees is for images where the camera orientation is incorrectly detected or inferred. The rotation requirement makes it possible to manually adjust the orientation of a still image or an image sequence without needing to re-encode it. Cropping enables the image to be re-framed without re-encoding. The HEVC file format also includes the option to store pre-derived images.<ref name="Nokia">{{cite web |url=https://nokiatech.github.io/heif/technical.html |title=HEIF Technical Information |subject=High Efficiency Image File Format |publisher=Nokia |website=nokiatech.github.io |at=Images in HEIF Files |access-date=2018-02-01 |archive-date=2018-02-01 |archive-url=https://web.archive.org/web/20180201152802/http://nokiatech.github.io/heif/technical.html |url-status=live }}</ref>
 
Samples in image sequence tracks must be either intra-coded images or inter-picture predicted images with reference to only intra-coded images. These constraints of inter-picture prediction reduce the decoding latency for accessing any particular image within a HEVC image sequence track.
 
The ''.heic'' and ''.heics'' file name extensions are conventionally used for HEVC-coded HEIF files.<ref name="WWDC2017 511">{{cite web |url=https://developer.apple.com/videos/play/wwdc2017/511 |title=Working with HEIF and HEVC - WWDC 2017 - Videos |website=Apple Developer |access-date=2019-01-19 |archive-date=2019-01-20 |archive-url=https://web.archive.org/web/20190120194504/https://developer.apple.com/videos/play/wwdc2017/511 |url-status=live }}</ref> Apple products, for instance,<ref name="WWDC2017 513">{{cite web |url=https://developer.apple.com/videos/play/wwdc2017/513 |title=High Efficiency Image File Format |time=0:09:26 |publisher=[[Apple Inc.|Apple]] |date=June 2017 |access-date=2017-11-22 |archive-date=2017-12-01 |archive-url=https://web.archive.org/web/20171201032622/https://developer.apple.com/videos/play/wwdc2017/513 |url-status=live }}</ref> will only produce files with these extensions, which indicate clearly that the data went through HEVC encoding.<ref name="CNET">{{cite web |url=https://www.cnet.com/news/apple-ios-boosts-heif-photos-over-jpeg-wwdc |title=How Apple is squeezing more photos into your iPhone – FAQ: Apple's newest iPhone software attempts to move the world out of the JPEG era. |publisher=[[CNET]] |first=Stephen |last=Shankland |date=June 16, 2017 |access-date=2017-11-21 |archive-date=2017-11-16 |archive-url=https://web.archive.org/web/20171116181530/https://www.cnet.com/news/apple-ios-boosts-heif-photos-over-jpeg-wwdc/ |url-status=live }}</ref>
 
{{anchor|AVCI}}{{anchor|AVC}}
Line 102 ⟶ 94:
The registered MIME types are <code>image/avci</code> for still images and <code>image/avcs</code> for sequences. The format is simply known as AVCI.
 
Apple products support playback of AVC-encoded ''.avci'' still image files and ''.avcs'' image sequence files<ref name="WWDC2017 513">{{cite web |date=June 2017 |title=High Efficiency Image File Format |url=https://developer.apple.com/videos/play/wwdc2017/513 |url-status=live |archive-url=https://web.archive.org/web/20171201032622/https://developer.apple.com/videos/play/wwdc2017/513 |archive-date=2017-12-01 |access-date=2017-11-22 |publisher=[[Apple Inc.|Apple]] |time=0:09:26}}</ref> but will only generate ''.heic'' files.
 
=== AVIF: AV1 in HEIF{{Anchor|AVIF|AV1}} ===
Line 129 ⟶ 121:
== Support ==
{{See also|AVIF#Support}}
*[[Nokia]] provides an open source C++ HEIF decoder, that also has a [[Java (programming language)|Java]] API.<ref name="Nokia" />
*[[Nokia]] provides an open source [[Java (programming language)|Java]] HEIF decoder.<ref name="Nokia">{{cite web |subject=High Efficiency Image File Format |title=HEIF Technical Information |url=https://nokiatech.github.io/heif/technical.html |url-status=live |archive-url=https://web.archive.org/web/20180201152802/http://nokiatech.github.io/heif/technical.html |archive-date=2018-02-01 |access-date=2018-02-01 |website=nokiatech.github.io |publisher=Nokia |at=Images in HEIF Files}}</ref>
* The open source library "libheif" supports reading and writing HEIF files.<ref name="libheif">{{cite web |url=https://github.com/strukturag/libheif|title=strukturag/libheif |date=May 17, 2021|via=GitHub |access-date=May 2, 2018 |archive-date=June 2, 2018 |archive-url=https://web.archive.org/web/20180602131612/https://github.com/strukturag/libheif |url-status=live }}</ref><ref>{{cite web|title=strukturag/libheif|url=https://github.com/strukturag/libheif/releases|access-date=2020-06-17|website=GitHub |archive-date=2020-06-18|archive-url=https://web.archive.org/web/20200618003355/https://github.com/strukturag/libheif/releases|url-status=live}}</ref> From version 1.8.0, both reading and writing HEIC and [[AVIF]] are supported.<ref name="libheif" />
* An image codec called CopyTrans HEIC, which is free for personal use and available for Windows versions 7 through 10, supports opening HEIF files in [[Windows Photo Viewer]] without the Microsoft codec installed. (The Microsoft HEIC codec is only available for Windows 10, version 1803 and up in the Photos [[Universal Windows Platform|UWP]] app.)<ref>{{cite web|url=https://www.copytrans.net/copytransheic/|title=Open HEIC files on Windows|website=CopyTrans.net|access-date=2018-09-27|archive-date=2018-09-16|archive-url=https://web.archive.org/web/20180916121137/https://www.copytrans.net/copytransheic/|url-status=live}}</ref>
Line 143 ⟶ 135:
* [[Ubuntu]] 20.04 and later (HEIC)<ref>{{cite web|title=software recommendation - Any app on Ubuntu to open and/or convert HEIF pictures (.HEIC, High Efficiency Image File Format)?|url=https://askubuntu.com/questions/958355/any-app-on-ubuntu-to-open-and-or-convert-heif-pictures-heic-high-efficiency-i|access-date=2020-10-09|website=Ask Ubuntu|archive-date=2020-11-11|archive-url=https://web.archive.org/web/20201111221257/https://askubuntu.com/questions/958355/any-app-on-ubuntu-to-open-and-or-convert-heif-pictures-heic-high-efficiency-i|url-status=live}}</ref>
* [[Debian]] 10.13 ("buster") or later<ref>{{cite web|url=https://tracker.debian.org/pkg/libheif|title=Libheif - Debian Package Tracker }}</ref>
* [[Fedora Linux|Fedora]] 36 or later<ref>{{cite web|url=https://packages.fedoraproject.org/pkgs/libheif/libheif/|title=Libheif - Fedora Packages }}</ref>
 
=== Web browsers ===
Line 161 ⟶ 154:
* [[Digikam]] (supports 10-bit HEIC)
* [[GraphicConverter]]
* [[gThumb]]
* [[ImageMagick]]
* [[Krita]]<ref>{{Cite news |title=Krita 4.1 Release Notes |url=https://krita.org/en/krita-4-1-release-notes |url-status=live |access-date=2018-11-19 |archive-url=https://web.archive.org/web/20181119214332/https://krita.org/en/krita-4-1-release-notes/ |archive-date=2018-11-19}}</ref>
Line 169 ⟶ 163:
* [[ACDSee]] (read-only)
* [[XnView]]<ref>{{Cite web |last=Pierre |first=GOUGELET |title=Supported graphic and image formats {{!}} XnView.com |url=https://www.xnview.com/en/image_formats/ |access-date=2023-10-12 |website=XnView Software |language=en}}</ref>
 
=== Image libraries ===
* [https://github.com/strukturag/libheif libheif] – ISO/IEC 23008-12:2017 HEIF and AVIF decoder and encoder.
* [https://sail.software SAIL] – format-agnostic library with support of HEIC implemented on top of libavif.
* [[FFmpeg]]
* [https://github.com/Xelitan/AVIF-and-HEIC-for-Delphi-Lazarus AVIF and HEIC unit] - Delphi/Lazarus wrapper for libavif
* [https://www.idrsolutions.com/jdeli/ JDeli] - Java Image library with HEIC support
* [https://github.com/nokiatech/heif?tab=License-1-ov-file Nokiatech] - Nokia's HEIF library with Java wrapper
 
=== Hardware ===
Line 184 ⟶ 186:
* [[Facebook]] supports the upload of HEIC but converts to JPEG or WEBP on display.<ref>{{cite web|last=Gallagher|first=William|date=2018-11-16|title=How to work with, edit, and share HEIC images without data loss|url=https://appleinsider.com/articles/18/11/16/how-to-work-with-edit-and-share-heic-images-without-data-loss|access-date=2020-08-14|publisher=[[Apple Insider]]|archive-date=2020-08-13|archive-url=https://web.archive.org/web/20200813211920/https://appleinsider.com/articles/18/11/16/how-to-work-with-edit-and-share-heic-images-without-data-loss|url-status=live}}</ref>
* [[Discord]] does not support HEIC at all.<ref>{{cite web |last=Technik |date=2019-10-27 |title=Feature Request: Support for HEIF images - Discord Support Community |url=https://support.discord.com/hc/en-us/community/posts/360054609851-Feature-Request-Support-for-HEIF-images |website=Discord Support Community}}</ref>
* [[Wikimedia Commons]] does not support HEIC.<ref>{{Cite web |title=Commons:File types - Wikimedia Commons |url=https://commons.wikimedia.org/wiki/Commons:File_types |access-date=2025-07-12 |website=commons.wikimedia.org |language=en}}</ref>
 
== Patent licensing ==
{{Further|High Efficiency Video Coding#Patent holders}}
 
HEIF itself is a container that may not be subject to additional royalty fees for commercial ISOBMFF licensees. Note however that Nokia also grants its patents on a royalty-free basis for non-commercial purposes.<ref name="nokiaheiflicense">{{cite web |title=Nokia High-Efficiency Image File Format (HEIF) License |website=[[GitHub]] |url=https://github.com/nokiatech/heif/blob/master/LICENSE.TXT |access-date=2020-05-07 |archive-date=2020-11-12 |archive-url=https://web.archive.org/web/20201112020815/https://github.com/nokiatech/heif/blob/master/LICENSE.TXT |url-status=live }}</ref> When containing images and image sequences encoded in a particular format (e.g., HEVC or AVC) its use becomes subject to the licensing of patents on the coding format.<ref name="ars">{{cite web |title=macOS High Sierra tech preview: A quick look at the stuff you can't see |url=https://arstechnica.com/apple/2017/06/macos-high-sierra-tech-preview-a-quick-look-at-the-stuff-you-cant-see/ |access-date=2017-07-04 |date=2017-06-19 |quote=If there's one major downside to both HEVC and HEIF, it's that they're covered by patents that may need to be licensed for use in various apps and services. |archive-date=2017-06-20 |archive-url=https://web.archive.org/web/20170620001037/https://arstechnica.com/apple/2017/06/macos-high-sierra-tech-preview-a-quick-look-at-the-stuff-you-cant-see/ |url-status=live }}</ref><ref name="jpgtoheif">{{cite web |title=Converting a JPEG to the new HEIF format |url=http://jpgtoheif.com/ |access-date=2017-07-04 |quote=HEIF and HEVC are extensively covered by patents, which means there could be legal implications to implementing HEIF support, particularly in paid software or a hardware product. |archive-date=2017-07-07 |archive-url=https://web.archive.org/web/20170707214211/http://jpgtoheif.com/ |url-status=live }}</ref><ref name="beamr-on-licensing">{{cite web |title=Apple Endorses New Image Format, HEIF |url=https://www.linkedin.com/pulse/apple-endorses-new-image-format-heif-jan-ozer |access-date=2017-10-31 |date=2017-06-12 |first1=Jan |last1=Ozer |first2=Dror |last2=Gill |archive-date=2019-03-21 |archive-url=https://web.archive.org/web/20190321220755/https://www.linkedin.com/pulse/apple-endorses-new-image-format-heif-jan-ozer |url-status=live }}</ref>
 
== See also ==
* [[Better Portable Graphics]] (BPG) – another image file format using HEVC encoding, published by [[Fabrice Bellard]] in 2014
* [[JPEG XL]] – an image file format developed since 2019 (standardization completed since 2022) and based on Google {{ill|PIK (image format)|lt=PIK|WDqid=Q72886365}} and Cloudinary [[Free Lossless Image Format|FLIF]] (itself based upon {{ill|Free Universal Image Format|lt=FUIF|WDqid=Q72886200}}) claiming to outperform PNG, WebP, BPG and JPEG 2000 for lossless encoding at least
* [[WebP]] – an image file format that features both lossy (based on the [[VP8]] and [[VP9]]) and lossless (independently videodeveloped) formatscompression
 
== References ==