Knuth–Morris–Pratt algorithm

English Wikisource has original text related to this article:

Knuth–Morris–Pratt algorithm

The Knuth-Morris-Pratt string searching algorithm searches for occurrences of a "pattern" string ${\displaystyle P}$ within a main string ${\displaystyle S}$ by employing the simple observation that when a mismatch occurs, we have enough knowledge simply by possessing the pattern to determine where the next match could begin, thus bypassing re-examination of previously matched characters.

The algorithm was invented by Knuth and Pratt and independently by J. H. Morris in 1977, but the three published it jointly.

The KMP algorithm

To motivate the technical details, we consider a specific instance. In this article, we will be using zero-based arrays for our strings; thus the 'C' in ${\displaystyle P}$  will be denoted ${\displaystyle P[2]}$ . Let ${\displaystyle m}$  be the start of the currently matched substring within ${\displaystyle S}$  and ${\displaystyle i}$  the position within ${\displaystyle P}$ .

m: 01234567890123456789012
S: ABC ABCDAB ABCDABCDABDE
P: ABCDABD
i: 0123456

We begin a step by matching each character one after another. Thus in the fourth step, ${\displaystyle m=0}$  and ${\displaystyle i=3}$ . But ${\displaystyle S[3]}$  is a space and ${\displaystyle P[3]='D'}$ , so we have a mismatch. Rather than beginning again at ${\displaystyle m=1}$ , we note that no 'A' occurs between positions 0 and 3 in ${\displaystyle P}$  except at 0; hence, having checked all those characters previously, we know there is no chance of finding the beginning of a match if we check them again. Therefore we move on to the next character, setting ${\displaystyle m=4}$  and ${\displaystyle i=0}$ .

m: 01234567890123456789012
S: ABC ABCDAB ABCDABCDABDE
P:     ABCDABD
i:     0123456

We quickly obtain a nearly complete match 'ABCDAB' when, at ${\displaystyle i=6}$ , we again have a discrepancy. However, just prior to the end of the current partial match, we passed an 'AB' which could be the beginning of a new match, so we must take this into consideration. As we already know that these characters match the two characters prior to the current position, we need not check them again; we simply reset ${\displaystyle m=8}$ , ${\displaystyle i=2}$  and continue matching the current character.

m: 01234567890123456789012
S: ABC ABCDAB ABCDABCDABDE
P:         ABCDABD
i:         0123456

This search fails immediately, however, as the pattern still does not contain a space, so as in the first trial, we increment ${\displaystyle m=11}$ , reset ${\displaystyle i=0}$ , and begin searching anew.

m: 01234567890123456789012
S: ABC ABCDAB ABCDABCDABDE
P:            ABCDABD
i:            0123456

Once again we immediately hit upon a match 'ABCDAB' but the next character, 'C', does not match the final character 'D' of the pattern. Reasoning as before, we set ${\displaystyle m=15}$ , to start at the two-character string 'AB' leading up to the current position, set ${\displaystyle i=2}$ , and continue matching from the current position.

m: 01234567890123456789012
S: ABC ABCDAB ABCDABCDABDE
P:                ABCDABD
i:                0123456

This time we are able to complete the match, and return the position 15 as its origin.

The search algorithm

The above example is completely instructive in this regard. We assume the existence of a "partial match" table, described below, which indicates where we need to look for the start of a new match in the event that the current one ends in a mismatch. For the moment the table, ${\displaystyle T}$ , should be taken as a black box with the property that if we have a match starting at ${\displaystyle S[m]}$  that fails when comparing ${\displaystyle S[m+i]}$  to ${\displaystyle P[i]}$ , then the next possible match will start at ${\displaystyle m+i-T[i-1]}$ . In particular, ${\displaystyle T[-1]}$  is defined and equals ${\displaystyle -1}$ . Knowing this, the algorithm is very simple:

1. Let ${\displaystyle i=m=0}$ ; say ${\displaystyle P}$  has ${\displaystyle n}$  characters, and ${\displaystyle S}$ , ${\displaystyle l}$  characters.
2. If ${\displaystyle m+i=l}$ , then we exit; there is no match. Otherwise, compare ${\displaystyle P[i]}$  versus ${\displaystyle S[m+i]}$ .
   • If they are equal, set ${\displaystyle i=i+1}$ . If ${\displaystyle i=n}$  then we have found a full match; terminate the algorithm and return ${\displaystyle m}$  as the start of the match.
   • If they are unequal, let ${\displaystyle e=T[i-1]}$ . Set ${\displaystyle m=m+i-e}$  and if ${\displaystyle i>0}$ , set ${\displaystyle i=e}$ .
3. Return to step 2.

This clearly implements the algorithm performed in the example; at each mismatch, we consult the table for the next known possible beginning of a new match and update our counters accordingly. Thus we need never backtrack; in particular, each character is matched only once (though potentially mismatched many times; an analysis of the efficiency of this algorithm is given below).

A code example of the search algorithm

Sample C code for an implementation of this algorithm is given below. In order to satisfy the natural limitations on the bounds of C arrays, ${\displaystyle T[i]}$  in the code is equivalent to ${\displaystyle T[i+1]}$  above.

int kmp_search(char *P, char *S)
{
    extern int T[];
    int m = 0;
    int i = 0;
    
    while (S[m + i] != '\0' && P[i] != '\0') {
        if (S[m + i] == P[i]) {
            ++i;
        } else {
            m += i - T[i];
            if (i > 0) i = T[i];
        }
    }
    
    if (P[i] == '\0') {
        return m;
    } else {
        return m + i;
    }
}

The efficiency of the search algorithm

Assuming the prior existence of the table ${\displaystyle T}$ , the search portion of the Knuth-Morris-Pratt algorithm has complexity O${\displaystyle (l)}$ , where ${\displaystyle l}$  is the length of ${\displaystyle S}$ . As except for the fixed overhead incurred in entering and exiting the function, all the computations are performed in the central loop, we will calculate a bound on the number of iterations of this loop; in order to do this we first make a key observation about the nature of ${\displaystyle T}$ . By definition it is constructed so that if a match which had begun at ${\displaystyle S[m]}$  fails while comparing ${\displaystyle S[m+i]}$  to ${\displaystyle P[i]}$ , then the next possible match must begin at ${\displaystyle S[m+(i-T[i])]}$ . In particular the next possible match must occur at a higher index than ${\displaystyle m}$ , so that ${\displaystyle T[i]<i}$ .

Using this fact, we will show that the loop can execute at most ${\displaystyle l}$  times. For in each iteration, it executes one of the two branches of the if statement. The first branch invariably increases ${\displaystyle i}$  and does not change ${\displaystyle m}$ , so that the index ${\displaystyle m+i}$  of the currently scrutinized character of ${\displaystyle S}$  is increased. The second branch adds ${\displaystyle i-T[i]}$  to ${\displaystyle m}$ , and as we have seen, this is always a positive number. Thus the ___location ${\displaystyle m}$  of the beginning of the current potential match is increased. Now, the loop ends if ${\displaystyle S[m+i]='\backslash 0'}$ , which following the C convention that a null character denotes the end-of-string, means that ${\displaystyle m+i=l}$ . Therefore each branch of the if statement can be reached at most ${\displaystyle l}$  times, since they respectively increase either ${\displaystyle m+i}$  or ${\displaystyle m}$ , and ${\displaystyle m\leq m+i}$  so that if ${\displaystyle m=l}$ , then certainly ${\displaystyle m+i\geq l}$ , so that since it increases by unit increments at most, we must have had ${\displaystyle m+i=l}$  at some point in the past.

Thus the loop executes at most ${\displaystyle 2l}$  times, showing that the time complexity of the search algorithm is ${\displaystyle O(l)}$ .

The "partial match" table

The goal of the table is to allow the algorithm not to check any character of the main string more than once. The key observation about the nature of a linear search that allows this to happen is that in having checked some segment of the main string against an initial segment of the pattern, we know exactly at which places a new potential match which could continue to the current position could begin prior to the current position. In other words, we "pre-search" the pattern itself and compile a list of all possible fallback positions that bypass a maximum of hopeless characters while not sacrificing any potential matches in doing so.

We want to be able to look up, for each pattern position, the length of the longest possible initial match that ends in the current position other than the full match that probably just failed. Hence ${\displaystyle T[i]}$  is exactly the length of the longest possible initial segment of ${\displaystyle P}$  which is also a terminal segment of the substring ending at ${\displaystyle P[i]}$ . We use the convention that the empty string has length 0. Since a mismatch at the very start of the pattern is a special case (there is no possibility of backtracking), we set ${\displaystyle T[-1]=-1}$ , as discussed above.

We consider the example of 'ABCDABD' first. We will see that it follows much the same pattern as the main search, and is efficient for similar reasons. We set ${\displaystyle T[-1]=0}$ . Since ${\displaystyle P[0]}$  is at the end of only the complete initial segment, we set ${\displaystyle T[0]=0}$  as well. To find ${\displaystyle T[1]}$ , we must discover a proper terminal substring of 'AB' which is also an initial substring of the pattern. But the only proper terminal substring of 'AB' is 'B', which is not an initial substring of the pattern, so we set ${\displaystyle T[1]=0}$ .

Continuing to 'C', we note that there is a shortcut to checking all terminal substrings: let us say that we discovered a terminal substring ending at 'C' with length 2; then its first character is an initial substring of an initial substring of the pattern, hence an initial substring itself...and it ends at 'B', which we already determined cannot occur. Therefore we need not even concern ourselves with substrings having length 2, and as in the previous case the sole one with length 1 fails, so ${\displaystyle T[2]=0}$ .

Similarly for 'D', giving ${\displaystyle T[3]=0}$ , so we pass to the subsequent 'A'. The same logic shows that the longest substring we need consider has length 1, and in this case 'A' does work; thus ${\displaystyle T[4]=1}$ . Considering now the following 'B', we exercise the following logic: if we were to find a subpattern beginning before the previous 'A' yet continuing to the current 'B', then in particular it would itself have a proper initial segment ending at 'A' yet beginning before 'A', which contradicts the fact that we already found that 'A' itself is the earliest occurrence of a proper subpattern ending at it. Therefore we need not look before that 'A' to find a subpattern for 'B'. In fact, checking it, we find that it continues to 'B' and that 'B' is the second entry of the substring of which 'A' is the first. Therefore the entry for 'B' in 'T' is one more than the entry for 'A', namely ${\displaystyle T[5]=2}$ .

Finally, the pattern does not continue from 'B' to 'D'. Similar reasoning as above shows that if we were to find a subpattern longer than 1 at 'D' then it must comprise a subpattern ending at 'B'; since the current one does not work, this one would have to be shorter. But the current one is an initial segment of the pattern ending at the second position, so this potential new subpattern would be as well, and we already decided there were none of those. Since 'D' itself is not a subpattern, ${\displaystyle T[6]=0}$ .

Therefore we compile the following table:

${\displaystyle i}$	-1	0	1	2	3	4	5	6
${\displaystyle P[i]}$		A	B	C	D	A	B	D
${\displaystyle T[i]}$	-1	0	0	0	0	1	2	0

The table-building algorithm

The example above illustrates the general technique for assembling the table with a minimum of fuss. The principle is that of the overall search: most of the work was already done in getting to the current position, so very little needs to be done in leaving it. Here follows the algorithm; to eliminate special cases we will use the convention that ${\displaystyle P[-1]}$  is defined and its value is unequal to any possible character in ${\displaystyle P}$ .

1. Set ${\displaystyle T[-1]=-1}$ , and let ${\displaystyle P}$  have ${\displaystyle n}$  characters.
2. Set ${\displaystyle i=0}$ , ${\displaystyle j=T[i-1]}$ .
3. If ${\displaystyle i=n}$  then we are done; terminate the algorithm. Otherwise, compare ${\displaystyle P[i]}$  with ${\displaystyle P[j]}$ .
   • If they are equal, set ${\displaystyle T[i]=j+1}$ , ${\displaystyle j=j+1}$ , and ${\displaystyle i=i+1}$ .
   • If not, and if ${\displaystyle j>0}$ , set ${\displaystyle j=T[j-1]}$ .
   • Otherwise, set ${\displaystyle T[i]=0}$ , ${\displaystyle i=i+1}$ , ${\displaystyle j=0}$ .
4. Return to step 3.

A code example of the table-building algorithm

A C code example for the table-building algorithm is given here. As for the search algorithm, the bounds of ${\displaystyle T}$  have been incremented by 1 to make the C code more natural. The additional variable ${\displaystyle c}$  is employed to simulate the existence of ${\displaystyle P[-1]}$ . It is also assumed that both this routine and the search routine are called as subroutines of a wrapper which correctly allocates memory for ${\displaystyle T}$ .

void kmp_table(char *P)
{
    extern int T[];
    int i = 0;
    int j = -1;
    char c = '\0';
    
    T[0] = j;
    while (P[i] != '\0') {
        if (P[i] == c) {
            T[i + 1] = j + 1;
            ++j;
            ++i;
        } else if (j > 0) {
            j = T[j];
        } else {
            T[i + 1] = 0;
            ++i;
            j = 0;
        }
        c = P[j];
    }
    
    return;
}

The efficiency of the table-building algorithm

The complexity of the table algorithm is ${\displaystyle O(n)}$ , where ${\displaystyle n}$  is the length of ${\displaystyle P}$ . As except for some initialization all the work is done in step 3, it is sufficient to show that step 3 executes in ${\displaystyle O(n)}$  time, which will be done by simultaneously examining the quantities ${\displaystyle i}$  and ${\displaystyle i-j}$ . In the first branch, ${\displaystyle i-j}$  is preserved, as both ${\displaystyle i}$  and ${\displaystyle j}$  are incremented simultaneously, but naturally, ${\displaystyle i}$  is increased. In the second branch, ${\displaystyle j}$  is replaced by ${\displaystyle T[j-1]}$ , which we saw above is always strictly less than ${\displaystyle j}$ , thus increasing ${\displaystyle i-j}$ . In the third branch, ${\displaystyle i}$  is incremented and ${\displaystyle j}$  is not, so both ${\displaystyle i}$  and ${\displaystyle i-j}$  increase. Since ${\displaystyle i\geq i-j}$ , this means that at each stage either ${\displaystyle i}$  or a lower bound for ${\displaystyle i}$  increases; therefore since the algorithm terminates once ${\displaystyle i=n}$ , it must terminate after at most ${\displaystyle n}$  iterations of step 3, since ${\displaystyle i-j}$  begins at ${\displaystyle 1}$ . Therefore the complexity of the table algorithm is ${\displaystyle O(n)}$ .

The efficiency of the KMP algorithm

Since the two portions of the algorithm have, respectively, complexities of ${\displaystyle O(l)}$  and ${\displaystyle O(n)}$ , the complexity of the overall algorithm is O${\displaystyle (n+l)}$ .

External link

• An explanation of the algorithm

References

• Donald Knuth, James H. Morris, Jr., and Vaughan Pratt. Fast pattern matching in strings. SIAM Journal on Computing, 6(2):323–350. 1977. Citations. Original publication.

• Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. Introduction to Algorithms, Second Edition. MIT Press and McGraw-Hill, 2001. ISBN 0262032937. Section 32.4: The Knuth-Morris-Pratt algorithm, pp.923–931.