Overlap–add method: Difference between revisions

Revision as of 11:22, 23 October 2014

Latest revision as of 05:17, 27 August 2025

(43 intermediate revisions by 19 users not shown)

Line 1:

{{Short description|Method in signal processing}}

In [[signal processing]], the '''overlap–add method (OA, OLA)''' is an efficient way to evaluate the discrete [[convolution]] of a very long signal <math>x[n]</math> with a [[finite impulse response]] (FIR) filter <math>h[n]</math>''':'''

{{about|the convolution method|the "Weight, OverLap, Add" channelization method|Discrete-time Fourier transform#Sampling the DTFT{{!}}Sampling the DTFT}}

In [[signal processing]], the '''overlap–add method''' is an efficient way to evaluate the discrete [[convolution]] of a very long signal <math>x[n]</math> with a [[finite impulse response]] (FIR) filter <math>h[n]</math>''':'''

:<math>

\begin{align}

y[n] = x[n] * h[n] \ \stackrel{\mathrm{def}}{=} \ \sum_{m=-\infty}^{\infty} h[m] \cdot x[n-m]

= \sum_{m=1}^{M} h[m] \cdot x[n-m],

\end{align}</math>

{{Equation box 1

where ''h''[''m''] = 0 for ''m'' outside the region [1, ''M''].

|indent= |cellpadding= 0 |border= 0 |background colour=white

|equation={{NumBlk|:|

<math>y[n] = x[n] * h[n]

\ \triangleq\ \sum_{m=-\infty}^{\infty} h[m] \cdot x[n - m]

= \sum_{m=1}^{M} h[m] \cdot x[n - m],</math> &nbsp; &nbsp;

|{{EquationRef|Eq.1}}}}}}

where <math>h[m] = 0</math> for <math>m</math> outside the region <math>[1,M].</math>&nbsp; This article uses common abstract notations, such as <math display="inline">y(t) = x(t) * h(t),</math> or <math display="inline">y(t) = \mathcal{H}\{x(t)\},</math> in which it is understood that the functions should be thought of in their totality, rather than at specific instants <math display="inline">t</math> (see [[Convolution#Notation]]).

The concept is to divide the problem into multiple convolutions of ''h''[''n''] with short segments of <math>x[n]</math>''':'''

[[Image:Overlap-add algorithm.svg|thumb|right|500px|Fig 1: A sequence of five plots depicts one cycle of the overlap-add convolution algorithm. The first plot is a long sequence of data to be processed with a lowpass FIR filter. The 2nd plot is one segment of the data to be processed in piecewise fashion. The 3rd plot is the filtered segment, including the filter rise and fall transients. The 4th plot indicates where the new data will be added with the result of previous segments. The 5th plot is the updated output stream. The FIR filter is a boxcar lowpass with <math>M=16</math> samples, the length of the segments is <math>L=100</math> samples and the overlap is 15 samples.]]

:<math>x_k[n] \ \stackrel{\mathrm{def}}{=}

\begin{cases}

x[n+kL] & n=1,2,\ldots,L\\

0 & \textrm{otherwise},

\end{cases}

</math>

The concept is to divide the problem into multiple convolutions of <math>h[n]</math> with short segments of <math>x[n]</math>''':'''

where ''L'' is an arbitrary segment length. Then''':'''

:<math>xx_k[n]\ =\triangleq\ \sum_begin{kcases} x_k[n-kL],\,</math>

x[n + kL], & n = 1, 2, \ldots, L\\

0, & \text{otherwise},

and ''y''[''n''] can be written as a sum of short convolutions''':'''

\end{cases}

:<math>

\begin{align}

y[n] = \left(\sum_{k} x_k[n-kL]\right) * h[n] &= \sum_{k} \left(x_k[n-kL]* h[n]\right)\\

&= \sum_{k} y_k[n-kL],

\end{align}

</math>

where <math>L</math> is an arbitrary segment length. Then''':'''

where &nbsp;<math>y_k[n] \ \stackrel{\mathrm{def}}{=} \ x_k[n]*h[n]\,</math>&nbsp; is zero outside the region [1,&nbsp;''L''&nbsp;+&nbsp;''M''&nbsp;&minus;&nbsp;1]. &nbsp;And for any parameter &nbsp;<math>N\ge L+M-1,\,</math>&nbsp; it is equivalent to the <math>N\,</math>-point [[circular convolution]] of <math>x_k[n]\,</math> with <math>h[n]\,</math>&nbsp; in the region&nbsp;[1,&nbsp;''N''].

:<math>x[n] = \sum_{k} x_k[n - kL],\,</math>

The advantage is that the [[circular convolution]] can be computed very efficiently as follows, according to the [[Discrete_Fourier_transform#Circular_convolution_theorem_and_cross-correlation_theorem|circular convolution theorem]]''':'''

and <math>y[n]</math> can be written as a sum of short convolutions''':'''<ref name=Rabiner/>

{{NumBlk|:|<math>y_k[n] = \textrm{IFFT}\left(\textrm{FFT}\left(x_k[n]\right)\cdot\textrm{FFT}\left(h[n]\right)\right)</math>|{{EquationRef|Eq.1}}}}

:<math>\begin{align}

where FFT and IFFT refer to the [[fast Fourier transform]] and inverse

y[n] = \left(\sum_{k} x_k[n - kL]\right) * h[n]

fast Fourier transform, respectively, evaluated over <math>N</math> discrete

&= \sum_{k} \left(x_k[n - kL] * h[n]\right)\\

points.

&= \sum_{k} y_k[n - kL],

\end{align}</math>

where the linear convolution <math>y_k[n]\ \triangleq\ x_k[n] * h[n]\,</math> is zero outside the region <math>[1,L+M-1].</math> And for any parameter <math>N \ge L + M - 1,\,</math>{{efn-ua

== The algorithm ==

|This condition implies that the <math>x_k</math> segment has at least <math>M-1</math> appended zeros, which prevents circular overlap of the output rise and fall transients.

}} it is equivalent to the <math>N</math>-point [[circular convolution]] of <math>x_k[n]\,</math> with <math>h[n]\,</math> in the region <math>[1,N].</math>&nbsp; The advantage is that the circular convolution can be computed more efficiently than linear convolution, according to the [[Discrete Fourier transform#Circular convolution theorem and cross-correlation theorem|circular convolution theorem]]''':'''

{{Equation box 1

[[Image:Depiction of overlap-add algorithm.png|frame|none|Figure 1: the overlap–add method]]

|indent= |cellpadding= 0 |border= 0 |background colour=white

|equation={{NumBlk|:|

<math>y_k[n]\ =\ \scriptstyle \text{IDFT}_N \displaystyle (\ \scriptstyle \text{DFT}_N \displaystyle (x_k[n])\cdot\ \scriptstyle \text{DFT}_N \displaystyle (h[n])\ ),</math> &nbsp; &nbsp;

|{{EquationRef|Eq.2}}}}}}

where''':'''

Fig. 1 sketches the idea of the overlap–add method. The

* DFT_N and IDFT_N refer to the [[Discrete Fourier transform]] and its inverse, evaluated over <math>N</math> discrete points, and

signal <math>x[n]</math> is first partitioned into non-overlapping sequences,

* <math>L</math> is customarily chosen such that <math>N=L+M-1</math> is an integer power-of-2, and the transforms are implemented with the [[Fast Fourier transform|FFT]] algorithm, for efficiency.

then the [[discrete Fourier transform]]s of the sequences <math>y_k[n]</math>

are evaluated by multiplying the FFT of <math>x_k[n]</math> with the FFT of

<math>h[n]</math>. After recovering of <math>y_k[n]</math> by inverse FFT, the resulting

output signal is reconstructed by overlapping and adding the <math>y_k[n]</math>

as shown in the figure. The overlap arises from the fact that a linear

convolution is always longer than the original sequences. In the early days of development of the fast Fourier transform, <math>L</math> was often chosen to be a power of 2 for efficiency, but further development has revealed efficient transforms for larger prime factorizations of L, reducing computational sensitivity to this parameter. A [[pseudocode]] of the algorithm is the

following:

==Pseudocode==

'''Algorithm 1''' (''OA for linear convolution'')

The following is a [[pseudocode]] of the algorithm''':'''

Evaluate the best value of N and L (L>0, N = M+L-1 nearest to power of 2).

Nx = length(x);

H = FFT(h,N) <span style="color:green;">(''zero-padded FFT'')</span>

i = 1

y = zeros(1, M+Nx-1)

'''while''' i <= Nx <span style="color:green;">(''Nx: the last index of x[n]'')</span>

il = min(i+L-1,Nx)

yt = IFFT( FFT(x(i:il),N) * H, N)

k = min(i+N-1,M+Nx-1)

y(i:k) = y(i:k) + yt(1:k-i+1) <span style="color:green;">(''add the overlapped output blocks'')</span>

i = i+L

'''end'''

<span style="color:green;">(''Overlap-add algorithm for linear convolution'')</span>

== Circular convolution with the overlap–add method ==

h = FIR_filter

M = length(h)

Nx = length(x)

N = 8 × 2^ceiling( log2(M) ) <span style="color:green;">(8 times the smallest power of two bigger than filter length M. See next section for a slightly better choice.)</span>

step_size = N - (M-1) <span style="color:green;">(L in the text above)</span>

H = DFT(h, N)

position = 0

y(1 : Nx + M-1) = 0

'''while''' position + step_size ≤ Nx '''do'''

y(position+(1:N)) = y(position+(1:N)) + IDFT(DFT(x(position+(1:step_size)), N) × H)

position = position + step_size

'''end'''

==Efficiency considerations==

When sequence ''x''[''n''] is periodic, and ''N''_''x'' is the period, then ''y''[''n''] is also periodic, with the same period. &nbsp;To compute one period of y[n], Algorithm 1 can first be used to convolve ''h''[''n''] with just one period of ''x''[''n'']. &nbsp;In the region ''M'' ≤ ''n'' ≤ ''N''_''x'', &nbsp;the resultant ''y''[''n''] sequence is correct. &nbsp;And if the next ''M''&nbsp;&minus;&nbsp;1 values are added to the first ''M''&nbsp;&minus;&nbsp;1 values, then the region 1 ≤ ''n'' ≤ ''N''_''x'' will represent the desired convolution. The modified pseudocode is''':'''

[[Image:FFT_size_vs_filter_length_for_Overlap-add_convolution.svg|thumb|400px|Fig 2: A graph of the values of N (an integer power of 2) that minimize the cost function <math>\tfrac{N\left(\log_2 N + 1\right)}{N - M + 1}</math>]]

When the DFT and IDFT are implemented by the FFT algorithm, the pseudocode above requires about {{nowrap|'''N (log₂(N) + 1)'''}} complex multiplications for the FFT, product of arrays, and IFFT.{{efn-ua

'''Algorithm 2''' (''OA for circular convolution'')

|1=Cooley–Tukey FFT algorithm for N=2^k needs (N/2) log₂(N) – see [[Fast Fourier transform#Definition|FFT – Definition and speed]]

Evaluate Algorithm 1

}} Each iteration produces {{nowrap|'''N-M+1'''}} output samples, so the number of complex multiplications per output sample is about''':'''

y(1:M-1) = y(1:M-1) + y(Nx+1:Nx+M-1)

y = y(1:Nx)

'''end'''

{{Equation box 1

== Cost of the overlap-add method ==

|indent= |cellpadding= 0 |border= 0 |background colour=white

|equation={{NumBlk|:|

<math>\frac{N (\log_2(N) + 1)}{N-M+1}.\,</math> &nbsp; &nbsp;

|{{EquationRef|Eq.3}}}}}}

For example, when <math>M=201</math> and <math>N=1024,</math> {{EquationNote|Eq.3}} equals <math>13.67,</math> whereas direct evaluation of {{EquationNote|Eq.1}} would require up to <math>201</math> complex multiplications per output sample, the worst case being when both <math>x</math> and <math>h</math> are complex-valued. Also note that for any given <math>M,</math> {{EquationNote|Eq.3}} has a minimum with respect to <math>N.</math> Figure 2 is a graph of the values of <math>N</math> that minimize {{EquationNote|Eq.3}} for a range of filter lengths (<math>M</math>).

The cost of the convolution can be associated to the number of complex

multiplications involved in the operation. The major computational

effort is due to the FFT operation, which for a radix-2 algorithm

applied to a signal of length <math>N</math> roughly calls for <math>C=\frac{N}{2}\log_2 N</math>

complex multiplications. It turns out that the number of complex multiplications

of the overlap-add method are:

Instead of {{EquationNote|Eq.1}}, we can also consider applying {{EquationNote|Eq.2}} to a long sequence of length <math>N_x</math> samples. The total number of complex multiplications would be:

:<math>C_{OA}=\left\lceil \frac{N_x}{N-M+1}\right\rceil

N\left(\log_2 N+1\right)\,</math>

:<math>N_x\cdot (\log_2(N_x) + 1).</math>

<math>C_{OA}</math> accounts for the FFT+filter multiplication+IFFT operation.

Comparatively, the number of complex multiplications required by the pseudocode algorithm is:

The additional cost of the <math>M_L</math> sections involved in the circular

version of the overlap–add method is usually very small and can be

neglected for the sake of simplicity. The best value of <math>N</math>

can be found by numerical search of the minimum of <math>C_{OA}\left(N\right)=C_{OA}\left(2^m \right)</math>

by spanning the integer <math>m</math> in the range <math>\log_2\left(M\right)\le m\le\log_2 \left(N_x\right)</math>.

Being <math>N</math> a power of two, the FFTs of the overlap–add method

are computed efficiently. Once evaluated the value of <math>N</math> it

turns out that the optimal partitioning of <math>x[n]</math> has <math>L=N-M+1</math>.

For comparison, the cost of the standard circular convolution of <math>x[n]</math>

and <math>h[n]</math> is:

:<math>C_S=N_x\leftcdot (\log_2(N) N_x+ 1\right)\,cdot \frac{N}{N-M+1}.</math>

Hence the ''cost'' of the overlap–add method scales almost as <math>O\left(N_x\log_2 N\right)</math> while the cost of a single, large circular convolution is almost <math>O\left(N_x\log_2 N_x \right)</math>. The two methods are also compared in Figure 3, created by [[MATLAB|Matlab]] simulation. The contours are lines of constant ratio of the times it takes to perform both methods. When the overlap-add method is faster, the ratio exceeds 1, and ratios as high as 3 are seen.

while the cost of the standard circular convolution method is almost

<math>O\left(N_x\log_2 N_x \right)</math>. However such functions accounts

only for the cost of the complex multiplications, regardless of the

other operations involved in the algorithm. A direct measure of the

computational time required by the algorithms is of much interest.

Fig. 2 shows the ratio of the measured time to evaluate

a standard circular convolution using &nbsp;{{EquationNote|Eq.1}} with

the time elapsed by the same convolution using the overlap–add method

in the form of Alg 2, vs. the sequence and the filter length. Both algorithms have been implemented under [[Matlab]]. The

bold line represent the boundary of the region where the overlap–add

method is faster (ratio>1) than the standard circular convolution.

Note that the overlap–add method in the tested cases can be three

times faster than the standard method.

[[Image:gain oa method.png|frame|none|FigureFig 23: RatioGain betweenof the timeoverlap-add requiredmethod bycompared &nbsp;{{EquationNote|Eqto a single, large circular convolution.1}} and theThe timeaxes requiredshow byvalues theof overlap–addsignal Alg.length 2N_x toand evaluatefilter length N_h.]]

a complex circular convolution, vs the sequence length <math>N_x</math> and

the filter length <math>M</math>.]]

== See also ==

* [[Overlap–save method]]

* [[Circular_convolution#Example]]

==Notes==

*[[Overlap–save method]]

{{notelist-ua}}

== References ==

{{reflist|refs=

<ref name=Rabiner>

{{Cite book

| author1=Rabiner, Lawrence R.

| author2=Gold, Bernard

| title=Theory and application of digital signal processing

| year=1975

| publisher=Prentice-Hall

| ___location=Englewood Cliffs, N.J.

| isbn=0-13-914101-4

| chapter=2.25

| pages=[https://archive.org/details/theoryapplicatio00rabi/page/63 63–65]

| chapter-url-access=registration

| chapter-url=https://archive.org/details/theoryapplicatio00rabi/page/63

}}</ref>

}}

==Further reading==

*{{Cite book

|author1=Oppenheim, Alan V. |author2=Schafer, Ronald W. | title=Digital signal processing

| author=Rabiner, Lawrence R.; Gold, Bernard

| authorlink=

| coauthors=

| title=Theory and application of digital signal processing

| year=1975

| publisher=Prentice-Hall

| ___location=Englewood Cliffs, N.J.

| isbn=0-13-914101-4

| pages=63–67

}}

*{{Cite book

| author=Oppenheim, Alan V.; Schafer, Ronald W.

| authorlink=

| coauthors=

| title=Digital signal processing

| year=1975

| publisher=Prentice-Hall

Line 155 ⟶ 131:

*{{Cite book

| author=Hayes, M. Horace

| authorlink=

| coauthors=

| title = Digital Signal Processing

| series = Schaum's Outline Series

Line 165 ⟶ 139:

| pages=

}}

* {{cite journal | last1=Senobari | first1=Nader Shakibay | last2=Funning | first2=Gareth J. | last3=Keogh | first3=Eamonn | last4=Zhu | first4=Yan | last5=Yeh | first5=Chin-Chia Michael | last6=Zimmerman | first6=Zachary | last7=Mueen | first7=Abdullah | title=Super-Efficient Cross-Correlation (SEC-C): A Fast Matched Filtering Code Suitable for Desktop Computers | journal=Seismological Research Letters | volume=90 | issue=1 | date=2019 | issn=0895-0695 | doi=10.1785/0220180122 | pages=322–334 | url=https://www.cs.ucr.edu/~eamonn/SuperEfficientCrossCorrelation.pdf }}

== External links ==

{{DEFAULTSORT:Overlap-Add Method}}