Standard Compression Scheme for Unicode: Difference between revisions

Revision as of 23:17, 11 November 2023

Latest revision as of 09:33, 7 May 2025

(3 intermediate revisions by 2 users not shown)

Line 6:

[[Reuters]] originally developed SCSU, then under the name RCSU for Reuters Compression Scheme for Unicode.<ref>{{Cite web|url=https://unicode.org/iuc/iuc9/Friday2.html#b3|title = Ninth International Unicode Conference - Friday - Track B}}</ref><ref>{{Cite web|url=https://unicode.org/iuc/iuc10/program.html|title=Tenth International Unicode Conference - Conference Program}}</ref><ref>{{Cite web|url=https://unicode.org/reports/tr6-10.html|title=Compression Scheme for Unicode}}</ref><ref name=Ewellic>{{Cite web|url=https://www.unicode.org/notes/tn14/UnicodeCompression.pdf|title = A survey of Unicode compression}}</ref>

At first the Unicode Consortium considered it to be a character encoding,<ref>{{Cite web|url=https://unicode.org/reports/tr17/tr17-2.html|title = UTR#17: Character Encoding Model}}</ref> but in 1999 changed its mind: although it was still considered a transfer encoding syntax, for a while it was no longer considered a character encoding because different compressors might yield different outputs for the same text.<ref>https://unicode.org/reports/tr17/tr17-3.html#Transfer Encoding Syntax</ref> However, in 2004 this decision was reverted and now SCSU is considered a ''compressing'' character encoding scheme, as opposed to a simple or compound character encoding scheme.<ref>{{Cite web|url=https://unicode.org/L2/L2004/04288-tr17-5d2.html#CharacterEncodingScheme|title=UTR#17: Character Encoding Model|date=2004-07-14}}</ref><ref>{{Cite web |title=UTR#17: Unicode Character Encoding Model |url=https://unicode.org/reports/tr17/ |access-date=2023-11-14 |website=unicode.org}}</ref>

Roman Czyborra (of [[GNU Unifont]]) wrote a decompressor.<ref>{{Cite web| title=This is a deflator to UTF-8 output for input compressed in SCSU | url=https://czyborra.com/scsu/scsu.c {{Bare| URLarchive-url=https://web.archive.org/web/19990908230458/http://czyborra.com:80/scsu/scsu.c plain text| archive-date=March 20221999-09-08}}</ref> The IBM-contributed decompressor is found in [[International Components for Unicode]], along with a compressor written in Java.<ref>{{Cite web|url=https://github.com/unicode-org/icu/blob/3f043c7693e20c8cded76035918dad104e7256e3/icu4j/main/classes/charset/src/com/ibm/icu/charset/CharsetSCSU.java|title = International Components for Unicode|website = [[GitHub]]|date = 22 October 2021}}</ref> Simpler reference codecs are available as attachments to TR6.

[[Symbian OS]], an operating system for mobile phones and other mobile devices, uses SCSU to serialize strings.

Line 21:

=== Window encoding ===

The core of SCSU lies in the windows for which the meanings of bytes 0x80-0xff are defined. There are eight static windows for simpler scripts and punctuation, and 6 types of dynamic windows (plus "half Unicode block" windows and custom Windowswindows for the supplementary planes) for scripts making use of more characters.

Both simple and dynamic windows are selected by special command characters. For individual characters that do not fit into the current block, command characters for quoting are provided.

== Comparison with general-purpose plain text compression schemes ==

Because UTF-16 or UTF-8 text might occupy more space than its equivalent in pre-Unicode encodings did, one might want to use compression such as SCSU to mitigate this problem.<ref>{{Cite web| title=Implementation Guidelines | url=https://unicode.org/versions/Unicode3.0.0/ch05.pdf {{Bare| URLarchive-url=https://web.archive.org/web/20150730234318/http://www.unicode.org/versions/Unicode3.0.0/ch05.pdf PDF| archive-date=March 20222015-07-30}}</ref> In comparison with general-purpose compressors, it is not necessarily advantageous to use SCSU.<ref name=Ewellic/> Also, while it can be used as a text encoding, because of the stateful nature of the algorithm difficulties may arise when using it as an internal text representation since basic text operations become non-trivial.

Treated purely as a compression algorithm, SCSU is inferior to most commonly used general-purpose algorithms for texts of over a few kilobytes.