Windows code page: Difference between revisions

Content deleted Content added
m Typography
EBCDIC code pages: Seems to have been placed in the wrong row (when compared with Microsoft documentation)
 
(23 intermediate revisions by 14 users not shown)
Line 1:
{{Short description|Sets of characters used in the 1980s & 90s}}
'''Windows code pages''' are sets of characters or [[code pages]] (known as [[character encoding]]s in other operating systems) used in [[Microsoft Windows]] from the 1980s and 1990s. Windows code pages were gradually superseded when [[Unicode in Microsoft Windows|Unicode was implemented in Windows]],{{cn|date=October 2020}} although they are still supported both within Windows and other platforms, and still apply when [[Alt code]] shortcuts are used.
 
Current Windows versions support [[Unicode]], new Windows applications should use Unicode (UTF-8) and not 8-bit character encodings.<ref>{{cite web |url=https://learn.microsoft.com/en-us/windows/win32/intl/unicode-and-character-sets |title=Unicode and character sets |publisher=Microsoft |date=2023-06-13 |access-date=2024-05-27}}</ref>
There are two groups of system code pages in Windows systems: OEM and Windows-native ("ANSI") code pages.
 
There are two groups of system code pages in Windows systems: OEM and Windows-native ("ANSI") code pages.
(ANSI is the [[American National Standards Institute]].) Code pages in both of these groups are [[extended ASCII]] code pages. Additional code pages are supported by standard Windows conversion routines, but not used as either type of system code page.
 
==ANSI code page<span class="anchor" id="ANSI"></span>==
{{Infobox character encoding
|name = Windows-125x series
|alias = ANSI (misnomer)
|mime =
|image =
|caption =
|standard = WHATWG Encoding Standard
|extends = [[US-ASCII]]
|prev = [[ISO 8859]]
|next = [[Unicode]]<br />[[UTF-16]] (in Win32 API)<br/>[[UTF-8]] (for files)
|otherrelated =
}}
 
'''ANSI code pages''' (officially called "Windows code pages"<ref>{{Cite web
|date=2016-03-07
|title=Code Pages
|url=https://msdn.microsoft.com/en-us/goglobal/bb964653.aspx
|access-date=2021-05-26
|archive-url=https://web.archive.org/web/20160307222735/https://msdn.microsoft.com/en-us/goglobal/bb964653.aspx
|archive-date=2016-03-07}}</ref> after Microsoft accepted the former term being a misnomer<ref name="ms glossary">{{Cite web
|url=https://msdn.microsoft.com/en-us/goglobal/bb964658.aspx#a
|title=Glossary of Terms Used on this Site
|date=December 8, 2018|archive-url=https://web.archive.org/web/20181208141313/https://msdn.microsoft.com/en-us/goglobal/bb964658.aspx#a|archive-date=2018-12-08
|archive-url=https://web.archive.org/web/20181208141313/https://msdn.microsoft.com/en-us/goglobal/bb964658.aspx#a
|quote=The term "ANSI" as used to signify Windows code pages is a historical reference, but is nowadays a misnomer that continues to persist in the Windows community. The source of this comes from the fact that the Windows code page 1252 was originally based on an ANSI draft—which became International Organization for Standardization (ISO) Standard 8859-1. “ANSI applications” are usually a reference to non-Unicode or code page–based applications.}}</ref>) are used for native non-Unicode (say, [[byte oriented]]) applications using a [[graphical user interface]] on Windows systems. The term "ANSI" is a misnomer because these Windows code pages do not comply with any [[American National Standards Institute|ANSI (American National Standards Institute)]] standard; [[Code Page 1252|code page 1252]] was based on an early ANSI draft that became the international standard [[ISO 8859-1]], <ref name="ms glossary" /><!-- --> which adds a further 32 control codes and space for 96 printable characters. Among other differences, Windows code-pages allocate printable characters to the supplementary control code space, making them at best illegible to standards-compliant operating systems.)
|archive-date=2018-12-08
|quote=The term "ANSI" as used to signify Windows code pages is a historical reference, but is nowadays a misnomer that continues to persist in the Windows community. The source of this comes from the fact that the Windows code page 1252 was originally based on an ANSI draft—which became International Organization for Standardization (ISO) Standard 8859-1. “ANSI applications” are usually a reference to non-Unicode or code page–based applications.}}</ref>) are used for native non-Unicode (say, [[byte oriented]]) applications using a [[graphical user interface]] on Windows systems. The term "ANSI" is a misnomer because these Windows code pages do not comply with any [[American National Standards Institute|ANSI (American National Standards Institute)]] standard; [[Code Page 1252|code page 1252]] was based on an early ANSI draft that became the international standard [[ISO 8859-1]], <ref name="ms glossary" /><!-- --> which adds a further 32 control codes and space for 96 printable characters. Among other differences, Windows code-pages allocate printable characters to the supplementary control code space, making them at best illegible to standards-compliant operating systems.)
 
Most [[Legacy system|legacy]] "ANSI" code pages have code page numbers in the pattern 125x. However, [[windows-874|874]] (Thai) and the East Asian multi-byte "ANSI" code pages ([[Windows-31J|932]], [[windows-936|936]], [[Unified Hangul Code|949]], [[Code page 950|950]]), all of which are also used as OEM code pages, are numbered to match IBM encodings, none of which are identical to the Windows encodings (although most are similar). While [[windows-1258|code page 1258]] is also used as an OEM code page, it is original to Microsoft rather than an extension to an existing encoding. IBM have assigned their own, different numbers for Microsoft's variants, these are given for reference in the lists below where applicable.
 
All of the 125x Windows code pages, as well as 874 and 936, are labelled by [[Internet Assigned Numbers Authority]] (IANA) as "Windows-''number''", although "Windows-936" is treated as a synonym for "[[GBK (character encoding)|GBK]]". Windows code page 932 is instead labelled as "Windows-31J".<ref>{{Cite web
|title=Character Sets
|url=http://www.iana.org/assignments/character-sets/character-sets.xhtml
|access-date=2021-05-26
|website=www.iana.org
|archive-date=2021-05-25
|archive-url=https://web.archive.org/web/20210525003138/https://www.iana.org/assignments/character-sets/character-sets.xhtml
|url-status=live}}</ref>
 
ANSI Windows code pages, and especially the code page [[Windows-1252|1252]], were so called since they were purportedly based on drafts submitted or intended for ANSI. However, ANSI and [[International Organization for Standardization|ISO]] have not standardized any of these code pages. Instead they are either:<ref name="ms glossary"/>
Line 39 ⟶ 59:
 
== History ==
Initially,Early computer systems had limited storage and systemrestricted programmingthe languagesnumber didof not[[bit]]s makeavailable ato distinctionencode betweena [[character (computing)|character]]. Although earlier proprietary encodings had fewer, the [[ASCII|American Standard Code for Information Interchange]] (ASCII) settled on seven bits: this was sufficient to encode a 96 member subset of the characters used in the US. As eight-bit [[byte]]s came to predominate, Microsoft (and others) expanded the repertoire to 224, to handle a variety of other uses such a box-drawing symbols. The need to provide [[byteprecomposed character]]s for the Western European and South American markets required a different character set: Microsoft established the principle of code pages, one for each alphabet. For the [[List of writing systems#Segmental script|segmental scripts]] used in most of Africa, the Americas, southern and south-east Asia, the Middle East and Europe, a character needs just one byte, but two or more bytes are needed for the [[ideographic]] sets used in the rest of the world. ThisThe subsequentlycode-page ledmodel towas much confusion. Microsoft software and systems priorunable to the [[Windows NT]] line are examples ofhandle this, because they use the OEM and ANSI code pages that do not make the distinctionchallenge.
 
Since the late 1990s, software and systems have adopted [[Unicode]] as their preferred storage format; this trend has been improved by the widespread adoption of [[XML]] which default to [[UTF-8]] but also provides a mechanism for labelling thecharacter encoding used.<ref>{{cite web | url = httpformat://www.w3.org/TR/xml11/#charencoding |Unicode titleis =designed Extensibleto Markuphandle Languagemillions (XML)of 1characters.1 (Second Edition): Character encodings | publisher = [[W3C]] | date = 29 September 2006 | access-date = 5 October 2020 | archive-date = 19 April 2021 | archive-url = https://web.archive.org/web/20210419133700/https://www.w3.org/TR/xml11/#charencoding | url-status = live }}</ref> All current Microsoft products and [[application program interfaces]] use Unicode internally,{{cn|date=October 2020}} but some applications continue to use the default encoding{{clarify|date=October 2024}} of the computer's 'locale' when reading and writing text data to files or standard output.{{cn|date=October 2020}} Therefore, files may still be encountered that are legible and intelligible in one part of the world but unintelligible [[mojibake]] in another.
 
=== UTF-8, UTF-16 ===
Microsoft adopted a Unicode encoding (first the now-obsolete [[UCS-2]], which was then Unicode's only encoding), i.e. [[UTF-16]] for all its [[operating system]]s from Windows NT onwards, but additionally supports [[UTF-8]] (aka <code>CP_UTF8</code>) since [[Windows 10 version history#Version 1803 (April 2018 Update)|Windows 10 version 1803]].<ref>{{cite web
|url=https://srad.jp/story/17/11/14/0640253/
|title=Windows 10のInsider PreviewでシステムロケールをUTF-8にするオプションが追加される
|trans-title=The option to make UTF-8 the system locale added in Windows 10 Insider Preview
|author=hylom
|website=スラド
|language=ja
|date=2017-11-14
|access-date=2018-05-10
|archive-date=2018-05-11|archive-url=https://web.archive.org/web/20180511012606/https://srad.jp/story/17/11/14/0640253/|url-status=live}}</ref>
|archive-url=https://web.archive.org/web/20180511012606/https://srad.jp/story/17/11/14/0640253/
|url-status=live}}</ref>
UTF-16 uniquely encodes all Unicode characters in the [[Basic Multilingual Plane]] (BMP) using 16 bits but the remaining Unicode (e.g. [[emoji]]s) is encoded with a 32-bit (four byte) code{{snd}} while the rest of the industry ([[Unix-like]] systems and the web), and now Microsoft chose [[UTF-8]] (which uses one byte for the 7-bit [[ASCII]] character set, two or three bytes for other characters in the BMP, and four bytes for the remainder).
 
Line 53 ⟶ 84:
 
=== Windows-125x series ===
These nine code pages are all [[extended ASCII]] 8-bit [[SBCS]] encodings, and were designed by [[Microsoft]] for use as ANSI codepages on Windows. They are commonly known by their IANA-registered<ref name="iana31j">{{cite web
| url=https://www.iana.org/assignments/character-sets/character-sets.xhtml
| publisher=IANA
| title=Character Sets
| access-date=2019-04-07
| archive-date=2016-12-03
| archive-url=https://web.archive.org/web/20161203235506/https://www.iana.org/assignments/character-sets/character-sets.xhtml
| url-status=live }}</ref> names as <code>windows-&lt;number></code>, but are also sometimes called <code>cp&lt;number></code>, "cp" for "code page". They are all used as ANSI code pages; Windows-1258 is also used as an OEM code page.
 
The Windows-125x series includes nine of the ANSI code pages, and mostly covers scripts from [[Europe]] and [[West Asia]] with the addition of [[Vietnam]]. System encodings for [[Thai script|Thai]] and for [[East Asia]]n languages were numbered to match similar IBM code pages and are used as both ANSI and OEM code pages; these are covered in following sections.
Line 89 ⟶ 127:
* {{anchor|CP437}}[[Code page 437|437]] – IBM PC US, 8-bit [[SBCS]] [[extended ASCII]].<ref name="IBM_CP437">{{cite web|author=IBM|title=SBCS code page information document - CPGID 00437|url=http://www-01.ibm.com/software/globalization/cp/cp00437.html|access-date=2014-07-04|archive-date=2016-06-09|archive-url=https://web.archive.org/web/20160609084933/https://www-01.ibm.com/software/globalization/cp/cp00437.html|url-status=live}}</ref> Known as OEM-US, the encoding of the primary built-in font of VGA graphics cards.<!-- Windows 7 Ultimate --><!-- [[Windows 1.0|1.00]]-[[Windows ME|4.90]] -->
* [[Code page 708|708]] – Arabic, extended [[ISO 8859-6]] (ASMO 708)<!-- Windows 7 Ultimate -->
* [[Code page 720|720]] – Arabic, retaining box drawing characters in their usual locations<!-- Windows 7 Ultimate -->
* [[Code page 737|737]] – "MS-DOS Greek". Retains all box drawing characters. More popular than 869.<!-- Windows 7 Ultimate -->
* [[Code page 775|775]] – "MS-DOS Baltic Rim"<!-- Windows 7 Ultimate -->
* [[Code page 850|850]] – "MS-DOS Latin 1". Full (re-arranged) repertoire of [[ISO 8859-1]].<!-- Windows 7 Ultimate -->
* [[Code page 852|852]] – "MS-DOS Latin 2"<!-- Windows 7 Ultimate -->
* [[Code page 855|855]] – "MS-DOS Cyrillic". Mainly used for [[South Slavic languages]]. Includes (re-arranged) repertoire of [[ISO-8859-5]]. Not to be confused with cp866.<!-- Windows 7 Ultimate -->
* [[Code page 857|857]] – "MS-DOS Turkish"<!-- Windows 7 Ultimate -->
* [[Code page 858|858]] – Western European with euro sign<!-- Windows 7 Ultimate -->
* [[Code page 860|860]] – "MS-DOS Portuguese"<!-- Windows 7 Ultimate -->
* [[Code page 861|861]] – "MS-DOS Icelandic"<!-- Windows 7 Ultimate -->
* [[Code page 862|862]] – "MS-DOS Hebrew"<!-- Windows 7 Ultimate -->
Line 113 ⟶ 151:
!ID!!Language!!Encoding!! data-sort-type="number" |IBM Equivalent!!Difference from IBM CCSID of same number!!Use
|-
|[[Code page 932 (Microsoft Windows)|932]]||[[Japanese writing system|Japanese]]||[[Shift JIS]] (Microsoft variant)||943<ref name="ibm932v943">{{cite web | url=https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.nlsgdrf/ibm-943_ibm-932.htm | title=IBM-943 and IBM-932 | publisher=IBM | work=IBM Knowledge Center | access-date=2020-07-08 | archive-date=2018-08-18 | archive-url=https://web.archive.org/web/20180818214409/https://www.ibm.com/support/knowledgecenter/en/ssw_aix_71/com.ibm.aix.nlsgdrf/ibm-943_ibm-932.htm | url-status=live }}</ref>||[[IBM-932]] is also Shift JIS, has fewer extensions (but those extensions it has are in common), and swaps some [[variant Chinese characters]] (''itaiji'') for interoperability with earlier editions of [[JIS X 0208|JIS C 6226]].||ANSI/OEM (Japan)<!-- Windows 7 Ultimate -->
|-
|[[Code page 936 (Microsoft Windows)|936]]||[[Simplified Chinese characters|Chinese (simplified)]]||[[GBK (character encoding)|GBK]]||1386||[[Code page 936 (IBM)|IBM-936]] is a different Simplified Chinese encoding with a different encoding method, which has been deprecated since 1993.||ANSI/OEM (PRC, Singapore)<!-- Windows 7 Ultimate -->
Line 146 ⟶ 184:
|[[Code page 20936|20936]]||5479||Chinese (simplified)||[[GB 2312]]||Conversion
|-
|[[Code page 20949|20949]], 51949||970||Korean||Wansung (8-bit with ASCII, i.e. [[EUC-KR]])<ref>{{cite web
|url=https://source.winehq.org/git/wine.git/blob/6f68543692a7588daa581d00c475715395036b15:/tools/make_unicode#l946
|title=dump_krwansung_codepage: build Korean Wansung table from the KSX1001 file
|work=make_unicode: Generate code page .c files from ftp.unicode.org descriptions
|first=Alexandre
|last=Julliard
|date=11 March 2021
|publisher=[[Wine (software)|Wine Project]]
|access-date=2021-03-14
|archive-date=2021-05-26 |archive-url=https://web.archive.org/web/20210526173718/https://source.winehq.org/git/wine.git/blob/6f68543692a7588daa581d00c475715395036b15:/tools/make_unicode#l946 |url-status=live }}</ref>||Conversion
|archive-url=https://web.archive.org/web/20210526173718/https://source.winehq.org/git/wine.git/blob/6f68543692a7588daa581d00c475715395036b15:/tools/make_unicode#l946
|url-status=live}}</ref>||Conversion
|}
 
Line 205 ⟶ 254:
|[[Code page 20423|20423]]||423||EBCDIC Greek with Extended Latin<!-- Windows 7 Ultimate -->
|-
|[[Code page 20424|20424]]||-424||x-EBCDIC-KoreanExtended Hebrew<!-- Windows 7 Ultimate -->
|-
|[[Code page 20833|20833]]||833||Korean EBCDIC for N-Byte Hangul; {{code|x-EBCDIC-KoreanExtended}}<!-- Windows 7 Ultimate -->
|-
|[[Code page 20838|20838]]||838||EBCDIC Thai<!-- Windows 7 Ultimate -->
Line 331 ⟶ 380:
|[[Code page 20108|20108]]||1016||7-bit [[ITU T.50|IA5]] Norwegian (NS 4551-2)<ref name="MS_2014_CP201xx"/><ref name="WG_CP201xx"/><ref name="WU_CP20108">{{cite web |title=Norwegian (IA5) encoding - Windows charsets |author-first=Antonin |author-last=Foller |date=2014 |orig-year=2011 |work=WUtils.com - Online web utility and help |publisher=Motobit Software |url=http://wutils.com/encodings/x-ia5-norwegian |access-date=2016-06-20 |url-status=live |archive-url=https://web.archive.org/web/20160620183449/http://wutils.com/encodings/x-ia5-norwegian |archive-date=2016-06-20}}</ref><!-- Windows 7 Ultimate, x-IA5-Norwegian -->
|-
|[[Code page 20127|20127]]||367||7-bit [[US-ASCII]]<ref name="MS_2014_CP201xx"/><ref name="WG_CP201xx"/><ref name="WU_CP20127">{{cite web |title=US-ASCII encoding - Windows charsets |author-first=Antonin |author-last=Foller |date=2014 |orig-year=2011 |work=WUtils.com - Online web utility and help |publisher=Motobit Software |url=http://wutils.com/encodings/us-ascii |access-date=2016-06-20 |url-status=live |archive-url=https://web.archive.org/web/20160620192058/http://wutils.com/encodings/us-ascii |archive-date=2016-06-20}}</ref><!-- Windows 7 Ultimate -->
|-
|[[Code page 20261|20261]]||1036||T.61 (T.61-8bit)<!-- Windows 7 Ultimate -->
Line 355 ⟶ 404:
* The use of code pages limits the set of characters that may be used.
* Characters expressed in an unsupported code page may be converted to question marks (?) or other [[replacement character]]s, or to a simpler version (such as removing accents from a letter). In either case, the original character may be lost.
 
== See also ==
* [[AppLocale]]&nbsp;– a utility to run non-Unicode (code page-based) applications in a locale of the user's choice.
 
==References==