Content deleted Content added
m Disambiguating links to Information processing (link changed to Data processing) using DisamAssist.
Character encodings: Get to the point
 
(6 intermediate revisions by 6 users not shown)
Line 9:
In [[communication]]s and [[Data processing|information processing]], '''code''' is a system of rules to convert [[information]]—such as a [[letter (alphabet)|letter]], [[word]], sound, image, or [[gesture]]—into another form, sometimes [[data compression|shortened]] or [[secrecy|secret]], for communication through a [[communication channel]] or storage in a [[storage medium]]. An early example is an invention of [[language]], which enabled a person, through [[speech]], to communicate what they thought, saw, heard, or felt to others. But speech limits the range of communication to the distance a voice can carry and limits the audience to those present when the speech is uttered. The invention of [[writing]], which converted spoken language into [[visual system|visual]] [[symbol]]s, extended the range of communication across space and [[time]].
 
The process of '''encoding''' converts information from a [[communication source|source]] into symbols for communication or storage. '''Decoding''' is the reverse process, converting code symbols back into a form that the recipient understands, such as English or/and, Spanish, etc.
 
One reason for coding is to enable communication in places where ordinary [[plain language]], spoken or written, is difficult or impossible. For example, [[semaphore]], where the configuration of [[Flag semaphore|flags]] held by a signaler or the arms of a [[semaphore line|semaphore tower]] encodes parts of the message, typically individual letters, and numbers. Another person standing a great distance away can interpret the flags and reproduce the words sent.
Line 27:
In this section, we consider codes that encode each source (clear text) character by a [[Code word (figure of speech)|code word]] from some dictionary, and [[concatenation]] of such code words give us an encoded string. Variable-length codes are especially useful when clear text characters have different probabilities; see also [[entropy encoding]].
 
A ''prefix code'' is a code with the "prefix property": there is no valid code word in the system that is a [[prefix (computer science)|prefix]] (start) of any other valid code word in the set. [[Huffman coding]] is the most known algorithm for deriving prefix codes. Prefix codes are widely referred to as "Huffman codes" even when the code was not produced by a Huffman algorithm. Other examples of prefix codes are [[countrytelephone callingcountry codes]], the country and publisher parts of [[ISBN]]s, and the Secondary Synchronization Codes used in the [[UMTS]] [[W-CDMA|WCDMA]] 3G Wireless Standard.
 
[[Kraft's inequality]] characterizes the sets of codeword lengths that are possible in a prefix code. Virtually any uniquely decodable one-to-many code, not necessarily a prefix one, must satisfy Kraft's inequality.
Line 44:
Codes can be used for brevity. When [[Telegraphy|telegraph]] messages were the state of the art in rapid long-distance communication, elaborate systems of [[commercial code (communications)|commercial codes]] that encoded complete phrases into single mouths (commonly five-minute groups) were developed, so that telegraphers became conversant with such "words" as ''BYOXO'' ("Are you trying to weasel out of our deal?"), ''LIOUY'' ("Why do you not answer my question?"), ''BMULD'' ("You're a skunk!"), or ''AYYLU'' ("Not clearly coded, repeat more clearly."). [[Code word (figure of speech)|Code word]]s were chosen for various reasons: [[length]], [[pronounceability]], etc. Meanings were chosen to fit perceived needs: commercial negotiations, military terms for military codes, diplomatic terms for diplomatic codes, any and all of the preceding for espionage codes. Codebooks and codebook publishers proliferated, including one run as a front for the American [[Black Chamber]] run by [[Herbert Yardley]] between the First and Second World Wars. The purpose of most of these codes was to save on cable costs. The use of data coding for [[data compression]] predates the computer era; an early example is the telegraph [[Morse code]] where more-frequently used characters have shorter representations. Techniques such as [[Huffman coding]] are now used by computer-based [[algorithm]]s to compress large data files into a more compact form for storage or transmission.
 
=== Character encodingsencoding ===
{{Main|Character encoding}}
CharacterA encodingscharacter areencoding representationsdescribes ofhow textual[[character]]-based data. A(text) givenis characterencoded. Antiquated encoding maysystems beused associateda withfixed anumber specificof character[[bit]]s, setranging (thefrom collection4 ofto characters7, whichbut itmodern cansystems represent),use thoughone someor charactermore sets8-[[bit]] have[[byte]]s multiplefor each character. encodings[[ASCII]], andthe vicedominate versa.system Characterfor encodingsdecades, mayuses beone broadlybyte groupedfor accordingeach tocharacter, theand numbertherefore, ofcan bytesencode requiredup to represent256 adifferent singlecharacters. character:To theresupport are[[natural single-bytelanguage]]s encodingswith more characters, other systems were invented that use [[Wide character|multibyte]]more (alsothan calledone wide)byte]] encodings,or anda [[Variable-width encoding|variable-width]] (also called variable-length) encodings. The earliest character encodings were single-byte, the best-known examplenumber of which is [[ASCIIbytes]]. ASCIIfor remainseach incharacter. use today, for example inA [[HTTPwriting headerssystem]]. However, single-byte encodings cannot model character sets with more than 256 characters. Scripts that requirea large character setsset such as [[CJK characters|Chinese, Japanese and Korean]] mustcan be represented with a multibyte encodingsencoding. Early multibyte encodings were fixed-length, meaning that although each character wasis represented by more than one byte, all characters used the same number of bytes ("word length"),; making them suitable for decoding withvia a [[lookup table]]. The final groupOn-the-other-hand, a variable-width encodings,encoding is amore subsetcomplex ofto multibytedecode encodings.since Theseit usecannot morebe complexdecoded encodingvia anda decodingsingle logiclookup totable efficientlyand representmust largebe characterprocessed setssequentially, whilebut keepingit thesupports representationsa more efficient representation of morea commonlylarge usedcharacter charactersset shorterby orusing maintaininga backwardsmaller compatibilityrepresentation properties.for Thismore groupcommonly includesused characters. Today, [[UTF-8]], an encoding of the [[Unicode]] character set; UTF-8, is the most common encoding of text mediaencoding used on the [[Internet]].
 
=== Genetic code ===
Line 53:
 
=== Gödel code ===
In [[mathematics]], a [[Gödel code]] wasis the basis for the proof of [[Gödel]]'s [[incompleteness theorem]]. Here, the idea wasis to map [[mathematical notation]] to a [[natural number]] (using a [[Gödel numbering]]).
 
=== Other ===
Line 76:
Other examples of encoding include:
*Encoding (in [[cognition]]) - a basic perceptual process of interpreting incoming stimuli; technically speaking, it is a complex, multi-stage process of converting relatively objective sensory input (e.g., light, sound) into a subjectively meaningful experience.
*A [[content format]] - a specific encoding format for converting a specific type of [[data]] to [[information]].
*Text encoding uses a [[markup language]] to tag the structure and other features of a text to facilitate processing by computers. (See also [[Text Encoding Initiative]].)
*[[Semantics encoding]] of formal language A informal language B is a method of representing all terms (e.g. programs or descriptions) of language A using language B.
Line 110 ⟶ 109:
== See also ==
{{Commons category|Codes}}
* [[ADDML]]
* [[Asemic writing]]
* [[Cipher]]
* [[Code (semiotics)]]
* [[Cultural code]]
* [[Equipment codes]]
* [[Quantum error correction]]