Standard Compression Scheme for Unicode: Difference between revisions

Revision as of 12:41, 25 August 2020

Revision as of 12:53, 25 August 2020

Line 1:

{{expand German|date=August 2013}}

The '''Standard Compression Scheme for Unicode''' (SCSU)<ref>{{cite web |url=https://www.unicode.org/reports/tr6/ |title=UTS #6: Compression Scheme for Unicode |date=2005-05-06 |quote=SCSU defines a compact encoding, which is sometimes useful. However, Unicode text is much more commonly stored and transmitted in [[UTF-8]] which is less compact (except for [[ASCII]]), much simpler, and does not present any security issues. For longer texts, general-purpose compression is effective and common.|accessdate=2008-06-13 }}</ref> is a [[Unicode]] Technical Standard for reducing the number of [[byte]]s needed to represent Unicode text (while sometimes useful, it has security risks and [[UTF-8]] encoding better usually), especially if that text uses mostly characters from one or a small number of per-language character blocks. It does so by dynamically mapping values in the range 128&ndash;255 to offsets within particular blocks of 128 characters. The initial conditions of the encoder mean that existing strings in [[ASCII]] and [[ISO-8859-1]] that do not contain C0 control codes other than NULL TAB CR and LF can be treated as SCSU strings. Since most alphabets do reside in blocks of contiguous Unicode codepoints, texts that use small alphabets and either ASCII punctuation or punctuation that fits within the window for the main alphabet can be encoded at one byte per character (plus setup overhead, which for common languages is often only 1 byte), most other punctuation can be encoded at 2 bytes per symbol through non-locking shifts. SCSU can also switch to [[UTF-16]] internally to handle non-alphabetic languages.

[[Symbian OS]], an operating system for mobile phones and other mobile devices, uses SCSU to serialiseserialize strings.

[[Reuters]], the organization that floated the first draft of SCSU, is believed to use SCSU internally.

[[Microsoft SQL Server|SQL Server 2008 R2]] uses SCSU to compress Unicode values (there meaning from strings in [[UCS-2]] encoding) stored in ''nchar(n)'' and ''nvarchar(n)'' columns, achieving space savings between 15% and 50% (while [[UTF-8]] already has this 50% reduction for [[ASCII]] subset of Unicode), depending on the language of the data.<ref>{{cite web|url=http://msdn.microsoft.com/en-us/library/ee240835(SQL.105).aspx|title=Unicode Compression Implementation (SQL Server 2008 R2 Books Online)|accessdate=2008-08-18}}</ref>

== Comparison with general-purpose plain text compression schemes ==

Line 20:

Supporting SCSU in [[HTML]] documents is prohibited by the [[W3C]]<ref>{{Cite web |url=https://www.w3.org/TR/html51/syntax.html#character-encodings |title=8.2.2.3. Character encodings |website=HTML 5.1 Standard |publisher=W3C}}</ref><ref>{{Cite web |url=https://www.w3.org/TR/html5/syntax.html#character-encodings |title=8.2.2.3. Character encodings |website=HTML 5 Standard |publisher=W3C}}</ref> and [[WHATWG]]<ref>{{Cite web |url=https://html.spec.whatwg.org/multipage/parsing.html#character-encodings |title=12.2.3.3 Character encodings |website=HTML Living Standard |publisher=WHATWG}}</ref> HTML standards, as it would present a [[cross-site scripting]] vulnerability.<ref>{{Cite web |url=https://developer.mozilla.org/en-US/docs/Web/HTML/Element/meta |title=&lt;meta> - HTML |website=MDN Web Docs |publisher=Mozilla}}</ref>

== See also ==

* [[Binary Ordered Compression for Unicode]] (BOCU-1)

* [[International Components for Unicode]] A library that can convert between SCSU and other Unicode encodings

Line 28:

{{Unicode navigation}}

{{characterCharacter encoding}}

{{DEFAULTSORT:Standard Compression Scheme For Unicode}}