Local binary patterns: Difference between revisions

Revision as of 05:41, 25 November 2017

Latest revision as of 01:22, 15 November 2024

(24 intermediate revisions by 20 users not shown)

Line 1:

{{Short description|Descriptor of computer vision}}

'''Local binary patterns''' ('''LBP)''') is a type of [[visual descriptor]] used for classification in [[computer vision]]. LBP is the particular case of the Texture Spectrum model proposed in 1990.<ref>DC. He and L. Wang (1990), "Texture Unit, Texture Spectrum, And Texture Analysis", Geoscience and Remote Sensing, IEEE Transactions on, vol. 28, pp. 509 - 512.</ref><ref>L. Wang and DC. He (1990), "Texture Classification Using Texture Spectrum", Pattern Recognition, Vol. 23, No. 8, pp. 905 - 910.</ref> LBP was first described in 1994.<ref>T. Ojala, [[Matti Pietikäinen (academic)|M. Pietikäinen]], and D. Harwood (1994), "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions", Proceedings of the 12th IAPR International Conference on Pattern Recognition (ICPR 1994), vol. 1, pp. 582 - 585.</ref><ref>T. Ojala, M. Pietikäinen, and D. Harwood (1996), "A Comparative Study of Texture Measures with Classification Based on Feature Distributions", Pattern Recognition, vol. 29, pp. 51-59.</ref> It has since been found to be a powerful feature for texture classification; it has further been determined that when LBP is combined with the [[Histogram of oriented gradients]] (HOG) descriptor, it improves the detection performance considerably on some datasets.<ref>"An HOG-LBP Human Detector with Partial Occlusion Handling", Xiaoyu Wang, Tony X. Han, Shuicheng Yan, ICCV 2009</ref> A comparison of several improvements of the original LBP in the field of background subtraction was made in 2015 by Silva et al.<ref>C. Silva, T. Bouwmans, C. Frelicot, "An eXtended Center-Symmetric Local Binary Pattern for Background Modeling and Subtraction in Videos", VISAPP 2015, Berlin, Germany, March 2015.</ref> A full survey of the different versions of LBP can be found in Bouwmans et al.<ref>T. Bouwmans, C. Silva, C. Marghes, M. Zitouni, H. Bhaskar, C. Frelicot,, "On the Role and the Importance of Features for Background Modeling and Foreground Detection”, https://arxiv.org/pdf/{{ArXiv|1611.09099v1.pdf09099}}</ref>

== Concept ==

[[File:Lbp neighbors.svg|thumbnail|upright=1.5|right|Three neighborhood examples used to define a texture and calculate a local binary pattern (LBP)]]

Line 7 ⟶ 8:

* Divide the examined window into cells (e.g. 16x16 pixels for each cell).

* For each pixel in a cell, compare the pixel to each of its [[Pixel connectivity|8 neighbors]] (on its left-top, left-middle, left-bottom, right-top, etc.). Follow the pixels along a circle, i.e. clockwise or counter-clockwise.

* Where the center pixel's value is greater than the neighbor's value, write "0". Otherwise, write "1". This gives an 8-digit binary number (which is usually converted to decimal for convenience).

* Compute the [[histogram]], over the cell, of the frequency of each "number" occurring (i.e., each combination of which pixels are smaller and which are greater than the center). This histogram can be seen as a 256-dimensional [[feature vector]].

Line 13 ⟶ 14:

* Concatenate (normalized) histograms of all cells. This gives a feature vector for the entire window.

The feature vector can now be processed using the [[Support vector machine]], [[Extreme learning machine|extreme learning machinesmachine]]s, or some other [[machine- learning]] algorithm to classify images. Such classifiers can be used for [[facial recognition system|face recognition]] or texture analysis.

A useful extension to the original operator is the so-called uniform pattern,<ref>Barkan et. al. "Fast High Dimensional Vector Multiplication Face Recognition." Proceedings of ICCV 2013</ref>, which can be used to reduce the length of the feature vector and implement a simple rotation invariant descriptor. This idea is motivated by the fact that some binary patterns occur more commonly in texture images than others. A local binary pattern is called uniform if the binary pattern contains at most two 0-1 or 1-0 transitions. For example, 00010000 (2 transitions) is a uniform pattern, but 01010100 (6 transitions) is not. In the computation of the LBP histogram, the histogram has a separate bin for every uniform pattern, and all non-uniform patterns are assigned to a single bin. Using uniform patterns, the length of the feature vector for a single cell reduces from 256 to 59. The 58 uniform binary patterns correspond to the integers 0, 1, 2, 3, 4, 6, 7, 8, 12, 14, 15, 16, 24, 28, 30, 31, 32, 48, 56, 60, 62, 63, 64, 96, 112, 120, 124, 126, 127, 128, 129, 131, 135, 143, 159, 191, 192, 193, 195, 199, 207, 223, 224, 225, 227, 231, 239, 240, 241, 243, 247, 248, 249, 251, 252, 253, 254 and 255.

== Extensions ==

* Over-Complete Local Binary Patterns (OCLBP):<ref>Barkan et. al. "Fast High Dimensional Vector Multiplication Face Recognition." Proceedings of ICCV 2013</ref> OCLBP is a variant of LBP that has been shown to improve the overall performance on face verification . Unlike LBP, OCLBP adopts overlapping to adjacent blocks. Formally, the configuration of OCLBP is denoted as S : (a, b, v, h, p, r): an image is divided into a×b blocks with vertical overlap of v and horizontal overlap of h, and then uniform patterns LBP(u2,p,r) are extracted from all the blocks. Moreover, OCLBP is composed of several different configurations. For example, in their original paper, the authors used three configurations: S : (10,10,12,12,8,1),(14,14,12,12,8,2),(18,18,12,12,8,3). The three configurations consider three block sizes: 10×10, 14×14, 18×18, and half overlap rates along the vertical and horizontal directions. These configurations are concatenated to form a 40877 dimensional feature vector for an image of size 150x80.

* Transition Local Binary Patterns(tLBP):<ref>Trefný, Jirí, and Jirí Matas."Extended set of local binary patterns for rapid object detection." Proceedings of the Computer Vision Winter Workshop. Vol. 2010. 2010.</ref> binary value of transition coded LBP is composed of neighbor pixel comparisons clockwise direction for all pixels except the central.

* Direction coded Local Binary Patterns(dLBP): the dLBP encodes the intensity variation along the four basic directions through the central pixel in two bits.

* Modified Local Binary Patterns(mLBP): the mLBP compares the values of neighboring pixels to the average of the intensity values in the 3x3 window.

* Multi-block LBP: the image is divided into many blocks, a LBP histogram is calculated for every block and concatenated as the final histogram.

* Volume Local Binary Pattern(VLBP):<ref>Zhao, Guoying, and Matti Pietikainen. "Dynamic texture recognition using local binary patterns with an application to facial expressions." IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on 29.6 (2007): 915-928.</ref> VLBP looks at dynamic texture as a set of volumes in the (X,Y,T) space where X and Y denote the spatial coordinates and T denotes the frame index. The neighborhood of a pixel is thus defined in three dimensional space, and volume textons can be extracted into histograms.

* RGB-LBP: This operator is obtained by computing LBP over all three channels of the RGB color space independently, and then concatenating the results together.

== Implementations ==

* [http://www.cse.oulu.fi/CMV CMV], includes the general LBP [http://www.cse.oulu.fi/CMV/Downloads/LBPMatlab implementation] {{Webarchive|url=https://web.archive.org/web/20141128164522/http://www.cse.oulu.fi/CMV/Downloads/LBPMatlab? |date=2014-11-28 }} and many further extensions over LBP histogram in MATLAB.

* [http://luispedro.org/software/mahotas Python mahotas], an open source computer vision package which includes an implementation of LBPs.

* [[OpenCV]]'s Cascade Classifiers support LBPs as of version 2.

Line 39:

* [[Local ternary patterns]]

* [http://www.scholarpedia.org/article/Local_Binary_Patterns Local Binary Pattern (LBP) methodology in Scholarpedia]

* [[Census transform]]

== References ==

{{Reflist}}