Standard Compression Scheme for Unicode: Difference between revisions

Revision as of 06:57, 2 June 2008

Revision as of 19:12, 16 June 2008

Line 1:

{{Table Unicode}}

The '''Standard Compression Scheme for Unicode''' (SCSU) is a [[Unicode]] Technical Standard<ref>{{cite web |url=http://www.unicode.org/reports/tr6/

The|title=UTS '''Standard#6: Compression Scheme for Unicode''' (SCSU)|date=2005-05-06 is|accessdate=2008-06-13 a [[Unicode]] Technical Standard}}</ref> for reducing the number of [[byte]]s needed to represent Unicode text, especially if that text uses mostly characters from one or a small number of per-language character blocks. It does so by dynamically mapping values in the range 128&ndash;255 to offsets within particular blocks of 128 characters. The initial conditions of the encoder mean that existing strings in [[ASCII]] and [[ISO-8859-1]] that do not contain C0 control codes other than NULL TAB CR and LF can be treated as SCSU strings. Since most alphabets do reside in blocks of contiguous Unicode codepoints, texts that use small alphabets and either ASCII punctuation or punctuation that fits within the window for the main alphabet can be encoded at one byte per character (plus setup overhead, which for common languages is often only 1 byte), most other punctuation can be encoded at 2 bytes per symbol through non-locking shifts. SCSU can also switch to UTF-16 internally to handle non-alphabetic languages.

[[Reuters]], the organization that floated the first draft of SCSU, is believed to use SCSU internally.

Line 9 ⟶ 10:

SCSU has not been a resounding success. Few applications need to compress so much Unicode text that it's worth using a special-purpose compression scheme which does not have widespread support. Also, while it can be used as a text encoding, it can be difficult to handle internally.

Treated purely as a compression algorithm, SCSU is inferior to most commonly-used general-purpose algorithms for texts of over a few kilobytes. One of several problems with SCSU is then that the savings of SCSU versus [[UTF-16]] or [[UTF-8]] drop after external compression<ref>[{{cite web |url=http://unicode.org/notes/tn14 |title=UTN #14: A survey of Unicode compression]</ref>, often dramatically so.

|date=2004-01-30 |first=Doug |last=Ewell |accessdate=2008-06-13 |format=PDF }}</ref>, often dramatically so.

SCSU does have the advantage that it can usefully compress texts that are only a few characters long, whereas most full-scale compressors need a few kilobytes of data to break even against their own overhead.

Line 17 ⟶ 19:

* [[International Components for Unicode]] A library that can convert between SCSU and other Unicode encodings

==External linksReferences ==

<references />

* [http://www.unicode.org/reports/tr6/ UTS #6: Compression Scheme for Unicode]

* http://czyborra.com/scsu/

[[Category:Unicode]]