Pyramids or 'pyramid representation' is a type of multi-scale signal representation developed by the computer vision, image processing and signal processing communities in which a signal or an image is subject to repeated smoothing and subsampling. Historically, pyramid representation is a predecessor to scale-space representation and multiresolution analysis.
Pyramid generation
There are two main types of pyramids; lowpass pyramids and bandpass pyramids. A lowpass pyramid is generated by first smoothing the image with an appropriate smoothing filter and then subsampling the smoothed image, usually by a factor of two along each coordinate direction. As this process proceeds, the result will be a set of gradually more smoothed images, where in addition the spatial sampling density decreases level by level. If illustrated graphically, this multi-scale representation will look like a pyramid, from which the name has been obtained. A bandpass pyramid is obtained by forming the difference between adjacent levels in a pyramid, where in addition some kind of interpolation is performed between representations at adjacent levels of resolution, to enable the computation of pixelwise differences.
Pyramid generation kernels
A variety of different smoothing kernels have proposed for generating pyramids (Burt 1981; Crowley 1981; Burt and Adelson 1983; Crowley and Sanderson 1984; Meer et al 1987). . Among the suggestions that have been given, the binomial kernels arising from the binomial coefficients stand out as a particularly useful and theoretically well-founded class (Crowley 1981; Lindeberg 1990, 1994) (see the article on multi-scale approaches for a very brief review). Thus, given a two-dimensional image, we may apply the binomial filter (1, 2, 1) typically once, twice or a few times along each spatial dimension and then subsample the image by a factor of two. This operation may then proceed as many times as desired, leading to a compact and efficient multi-scale representation. If motivatived by specific requirements, intermediate scale levels may also be generated where the subsampling stage is left out, leading to an oversampled or hybrid pyramid. With the increasing computational efficiency of CPUs available today, it is in some situations also feasible to use wider support Gaussian filters as smoothing kernels in the pyramid generation steps.
Applications of pyramids
In the early days of computer vision, pyramids were used as the main type of multi-scale representation for computing multi-scale image features from real-world image data. Today, this role has been taken over by scale-space representation, motivated by the more solid theoretical foundation, the ability to decouple the subsampling stage from the multi-scale representation, the more powerful tools for theoretical analysis as well as the ability to compute a representation at any desired scale, thus avoiding the algorithmic problems of relating image representations at different levels of representation. Nevertheless, pyramids are still frequently used for expressing computationally efficient approximations to scale-space representation (Lindeberg and Bretzner 2003; Crowley and Riff 2003; Lowe 2004).
References
- Burt, P.J. "Fast filter transforms for image processing", Computer Vision, Graphics and Image Processing, vol 16, pages 20-51, 1981.
- Burt, Peter and Adelson, Ted, "The Laplacian Pyramid as a Compact Image Code", IEEE Trans. Communications, 9:4, 532–540, 1983.
- Crowley, James "A representation for visual information", PhD thesis, Carnegie-Mellon University, Robotics Institute, Pittsburgh, Pennsylvania 1981.
- Crowley, J. L. and Sanderson, A. C. "Multiple resolution representation and probabilistic matching of 2-D gray-scale shape", IEEE Transactions on Pattern Analysis and Machine Intelligence, 9(1), pp 113-121, 1987.
- Crowley, J, Riff O: Fast computation of scale normalised Gaussian receptive fields, Proc. Scale-Space'03, Isle of Skye, Scotland, Springer Lecture Notes in Computer Science, volume 2695, 2003.
- Lindeberg, T., "Scale-space for discrete signals," PAMI(12), No. 3, March 1990, pp. 234-254.
- Lindeberg, Tony, Scale-Space Theory in Computer Vision, Kluwer Academic Publishers, 1994, ISBN 0-7923-9418-6
- Lindeberg, T. and Bretzner, L.: Real-time scale selection in hybrid multi-scale representations, Proc. Scale-Space'03, Isle of Skye, Scotland, Springer Lecture Notes in Computer Science, volume 2695, pages 148-163, 2003.
- P. Meer, E. S. Baugher and A. Rosenfeld "Frequency ___domain analysis and synthesis of image generating kernels", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 9, pages 512-522, 1987.
- Lowe, D. G., “Distinctive image features from scale-invariant keypoints”, International Journal of Computer Vision, 60, 2, pp. 91-110, 2004.