Null-terminated string: Difference between revisions

Revision as of 01:09, 31 October 2007

Latest revision as of 01:23, 25 March 2025

(384 intermediate revisions by more than 100 users not shown)

Line 1:

{{Short description|Data structure}}

{{dablink|This article is on strings in computer programming. A number of [[stringed instrument]]s, including the [[viola]] and [[cello]], have a string that, when played open, produces the [[note|note C]]. This is usually referred to as the '''C string'''.}}

{{Redirect|CString||C string (disambiguation)}}

{{see also|String (computer science)#Null-terminated}}

{{Use dmy dates|date=August 2021}}

In computing[[computer programming]], a '''Cnull-terminated string''' is a [[character string|character sequence]] stored as a one-dimensionalan [[characterArray (computing)data structure|character]] [[array]] containing the characters and terminated with a ''[[null character]] ('\0' (a character with an internal value of zero, called "NUL" in this article, not same as the [[ASCIIglyph]] zero). TheAlternative namenames are ''[[C string handling|C string]]'', which refers to the ubiquitous [[C (programming language)|C programming language]] whichand uses'''ASCIIZ'''<ref>{{Cite thisweb [[string|title=Chapter (computer15 - MIPS Assembly Language science)#Representations|stringurl=https://people.scs.carleton.ca/~sivarama/org_book/org_book_web/solution_manual/org_soln_one/arch_book_solution_ch15.pdf representation|access-date=2023-10-09 |website=Carleton University}}</ref> (although C can use encodings other than [[ASCII]]).

The length of a string is found by searching for the (first) NUL. This can be slow as it takes O(''n'') ([[linear time]]) with respect to the string length. It also means that a string cannot contain a NUL (there is a NUL in memory, but it is after the last character, not {{em|in}} the string).

In the [[C++]] [[programming language]], C strings are used in addition to another representation of character sequences, the <code>std::string</code> container found in the [[Standard Template Library]] (STL). Thus, it is important to differentiate between the traditional "C strings" and the more sophisticated "string" objects provided by the STL.

== TriviaHistory ==

The null-termination characteristic has historically created [[computer insecurity|security problems]] related to the length of the string. If the null character is not correctly accounted for, any following non-related memory area may also be processed as a part of the character sequence. This can lead to program crashes or leakage of program internal information to attackers or non-understanding users.

Null-terminated strings were produced by the <code>.ASCIZ</code> directive of the [[PDP-11]] [[assembly language]]s and the <code>ASCIZ</code> directive of the [[MACRO-10]] macro assembly language for the [[PDP-10]]. These predate the development of the C programming language, but other forms of strings were often used.

At the time C (and the languages that it was derived from) was developed, memory was extremely limited, so using only one byte of overhead to store the length of a string was attractive. The only popular alternative at that time, usually called a "Pascal string" (a more modern term is "[[String (computer science)#Length-prefixed|length-prefixed]]"), used a leading ''byte'' to store the length of the string. This allows the string to contain NUL and made finding the length need only one memory access (O(1) [[constant time|(constant) time]]), but limited string length to 255 characters. C designer [[Dennis Ritchie]] chose to follow the convention of null-termination to avoid the limitation on the length of a string and because maintaining the count seemed, in his experience, less convenient than using a terminator.<ref>{{cite book | first = Dennis M. | last = Ritchie | article = The development of the C language | title = History of Programming Languages | edition = 2 | editor-first1= Thomas J. | editor-last1= Bergin, Jr. | editor-first2 = Richard G. | editor-last2 = Gibson, Jr. | publisher = ACM Press | ___location = New York | via = Addison-Wesley (Reading, Mass) | year = 1996 | isbn = 0-201-89502-1 }}</ref>

This had some influence on CPU [[instruction set]] design. Some CPUs in the 1970s and 1980s, such as the [[Zilog Z80]] and the [[Digital Equipment Corporation|DEC]] [[VAX]], had dedicated instructions for handling length-prefixed strings. However, as the null-terminated string gained traction, CPU designers began to take it into account, as seen for example in IBM's decision to add the "Logical String Assist" instructions to the [[IBM ES/9000 family|ES/9000]] 520 in 1992 and the vector string instructions to the [[IBM z13 (microprocessor)|IBM z13]] in 2015.<ref name=pop>[http://publibfp.dhe.ibm.com/epubs/pdf/a227832c.pdf IBM z/Architecture Principles of Operation]</ref>

== C String header ==

[[FreeBSD]] developer [[Poul-Henning Kamp]], writing in ''[[ACM Queue]]'', referred to the victory of null-terminated strings over a 2-byte (not one-byte) length as "the most expensive one-byte mistake" ever.<ref>{{citation |last=Kamp |first=Poul-Henning |date=25 July 2011 |title=The Most Expensive One-byte Mistake |journal=ACM Queue |volume=9 |number=7 |pages=40–43 |doi=10.1145/2001562.2010365 |s2cid=30282393 |issn=1542-7730|doi-access=free }}</ref>

The <cstring> header and its functions are the ideal way to work with C strings. Typical operator use with C strings will frequently create problems with pointers. As an example, using the = operator between two C strings does not copy the array but copies a pointer to this array. This is incorrect usage for most typical applications.

== Limitations ==

While simple to implement, this representation has been prone to errors and performance problems.

Null-termination has historically created [[computer insecurity|security problems]].<ref>{{cite journal|url= http://insecure.org/news/P55-07.txt |author=Rain Forest Puppy |title=Perl CGI problems |journal=Phrack Magazine |publisher=artofhacking.com |date=9 September 1999 |volume=9 |issue=55 |page=7 |access-date=3 January 2016}}</ref> A NUL inserted into the middle of a string will truncate it unexpectedly.<ref>{{Cite web|url=https://security.stackexchange.com/questions/48187/null-byte-injection-on-php|title = Null byte injection on PHP?}}</ref> A common bug was to not allocate the additional space for the NUL, so it was written over adjacent memory. Another was to not write the NUL at all, which was often not detected during testing because the block of memory already contained zeros. Due to the expense of finding the length, many programs did not bother before copying a string to a fixed-size [[Data buffer|buffer]], causing a [[buffer overflow]] if it was too long.

Example:

The inability to store a zero requires that text and binary data be kept distinct and handled by different functions (with the latter requiring the length of the data to also be supplied). This can lead to code redundancy and errors when the wrong function is used.

To copy a C string correctly, the following format could be used

memcpy(CstringDst, CstringSrc, strlen(CstringSrc) + 1);

The speed problems with finding the length can usually be mitigated by combining it with another operation that is O(''n'') anyway, such as in <code>[[strlcpy]]</code>. However, this does not always result in an intuitive [[API]].

Where CstringDst is the destination C string to be copied into and CstringSrc is the source of the C string. The last argument will set the length of the destination C array. It will set this length to the length of the source plus 1 to account for the terminating character.

== Character encodings ==

Null-terminated strings require that the encoding does not use a zero byte (0x00) anywhere; therefore it is not possible to store every possible [[ASCII]] or [[UTF-8]] string.<ref>{{cite web|title=UTF-8, a transformation format of ISO 10646|date=November 2003 |url=http://tools.ietf.org/html/rfc3629#section-3|access-date=19 September 2013 |last1=Yergeau |first1=François }}</ref><ref><!-- This is the encoding table provided as a resource by the Unicode consortium: http://www.unicode.org/resources/utf8.html -->{{cite web|title=Unicode/UTF-8-character table|url=http://www.utf8-chartable.de/|access-date=13 September 2013}}</ref><ref>{{cite web|last=Kuhn|first=Markus|title=UTF-8 and Unicode FAQ|url=http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8|access-date=13 September 2013}}</ref> However, it is common to store the subset of ASCII or UTF-8 – every character except NUL – in null-terminated strings. Some systems use "[[modified UTF-8]]" which encodes NUL as two non-zero bytes (0xC0, 0x80) and thus allow all possible strings to be stored. This is not allowed by the UTF-8 standard, because it is an [[UTF-8#Overlong encodings|overlong encoding]], and it is seen as a security risk. Some other byte may be used as end of string instead, like 0xFE or 0xFF, which are not used in UTF-8.

[[UTF-16]] uses 2-byte integers and as either byte may be zero (and in fact ''every other'' byte is, when representing ASCII text), cannot be stored in a null-terminated byte string. However, some languages implement a string of 16-bit [[UTF-16]] characters, terminated by a 16-bit NUL (0x0000).

The functions included in <cstring> are as follows:

== Improvements ==

Copying:

Many attempts to make C string handling less error prone have been made. One strategy is to add safer functions such as <code>[[strdup]]</code> and <code>[[strlcpy]]</code>, whilst [[C standard library#Buffer overflow vulnerabilities|deprecating the use of unsafe functions]] such as <code>[[gets() | gets]]</code>. Another is to add an object-oriented wrapper around C strings so that only safe calls can be done. However, it is possible to call the unsafe functions anyway.

memcpy Copies block of memory

memmove Move block of memory

strcpy Copy string

strncpy Copy characters from string

Most modern libraries replace C strings with a structure containing a 32-bit or larger length value (far more than were ever considered for length-prefixed strings), and often add another pointer, a reference count, and even a NUL to speed up conversion back to a C string. Memory is far larger now, such that if the addition of 3 (or 16, or more) bytes to each string is a real problem the software will have to be dealing with so many small strings that some other storage method will save even more memory (for instance there may be so many duplicates that a [[hash table]] will use less memory). Examples include the [[C++]] [[Standard Template Library]] <code>[[String (C++)|std::string]]</code>, the [[Qt (toolkit)|Qt]] <code>QString</code>, the [[Microsoft Foundation Class Library|MFC]] <code>CString</code>, and the C-based implementation <code>CFString</code> from [[Core Foundation]] as well as its [[Objective-C]] sibling <code>NSString</code> from [[Foundation Kit|Foundation]], both by Apple. More complex structures may also be used to store strings such as the [[rope (computer science)|rope]].

Concatenation:

strcat Concatenate strings

strncat Append characters from string

==See also==

* [[characterEmpty string]]

*[[Sentinel value]]

==References==

Comparison:

{{reflist}}

memcmp Compare two blocks of memory

strcmp Compare two strings

strcoll Compare two strings using locale

strncmp Compare characters of two strings

strxfrm Transform string using locale

Searching:

memchr Locate character in block of memory

strchr Locate first occurrence of character in string

strcspn Get span until character in string

strpbrk Locate character in string

strrchr Locate last occurrence of character in string

strspn Get span of character set in string

strstr Locate substring

strtok Split string into tokens

Other:

memset Fill block of memory

strerror Get pointer to error message string

strlen Get string length

== Trivia ==

C strings are exactly equivalent to the strings created by the .ASCIZ directive implemented by the [[PDP-11]] and [[VAX]] [[Assembly language|macroassembly languages]].

==See also==

* [[character string]]

* [[strcpy]]

* [[strcat]]

* [[strlen]]

* [[strlcpy]]

{{CProLang}}

{{Data types}}

[[Category:CString programmingdata languagestructures]]

[[es:C string]]