Structure tensor: Difference between revisions

Revision as of 02:21, 30 July 2021

Revision as of 06:59, 21 August 2021

Line 5:

J. Bigun and G. Granlund (1986), ''Optimal Orientation Detection of Linear Symmetry''. Tech. Report LiTH-ISY-I-0828, Computer Vision Laboratory, Linkoping University, Sweden 1986; Thesis Report, Linkoping studies in science and technology No. 85, 1986.

</ref><ref name=bigun87>

{{cite conference|author1=J. Bigun |author2=G. Granlund |name-list-style=amp |titlechapter=Optimal Orientation Detection of Linear Symmetry|___location=Piscataway|book-title=First int. Conf. on Computer Vision, ICCV, (London) |publisher=IEEE Computer Society Press, Piscataway|pages=433–438|year=1987 }}

</ref><ref name=knutsson89>

{{cite conference|author=H. Knutsson|titlechapter=Representing local structure using tensors|___location=Oulu|book-title=Proceedings 6th Scandinavian Conf. on Image Analysis|publisher=Oulu University|pages=244–251|year=1989}}

</ref>

Line 79:

Note that the average of the gradient <math>\nabla I</math> inside the window is '''not''' a good indicator of anisotropy. Aligned but oppositely oriented gradient vectors would cancel out in this average, whereas in the structure tensor they are properly added together.<ref>

{{cite journal|author=T. Brox, J. Weickert, B. Burgeth and P. Mrazek|titlechapter=Nonlinear Structure Tensors|book-title=Universität des Saarlandes, Tech. Report|issue=113|pages=1–32|year=2004}}

</ref> This is a reason for why <math>(\nabla I)(\nabla I)^\text{T}</math> is used in the averaging of the structure tensor to optimize the direction instead of <math>\nabla I</math>.

Line 243:

==Applications==

The eigenvalues of the structure tensor play a significant role in many image processing algorithms, for problems like [[corner detection]], [[interest point detection]], and [[feature tracking]].<ref name="Medioni">

{{cite conference|author1=M. Nicolescu |author2=G. Medioni |name-list-style=amp |titlechapter=Motion Segmentation with Accurate Boundaries – A Tensor Voting Approach|book-title=Proc. IEEE Computer Vision and Pattern Recognition|volume=1|pages=382–389|year=2003}}

</ref><ref>

{{cite journal

|author=W. Förstner|titlechapter=A Feature Based Correspondence Algorithm for Image Processing

|book-title=International Archives of Photogrammetry and Remote Sensing|volume=26|pages=150–166|year=1986}}

</ref><ref>

{{cite conference|author1=C. Harris |author2=M. Stephens |name-list-style=amp |titlechapter=A Combined Corner and Edge Detector

|book-title=Proc. of the 4th ALVEY Vision Conference|pages=147–151|year=1988}}

</ref><ref>

{{cite journal|author=K. Rohr|titlechapter=On 3D Differential Operators for Detecting Point Landmarks

|book-title=Image and Vision Computing|volume=15|issue=3|pages=219–233|year=1997}}

</ref><ref>

{{cite conference|author1=I. Laptev |author2=T. Lindeberg |name-list-style=amp |titlechapter=Space–time interest points

|book-title=International Conference on Computer Vision ICCV'03|url=http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A442088&dswid=2038 |doi=10.1109/ICCV.2003.1238378|pages=432–439|volume=I|year=2003}}

</ref><ref>

{{cite conference|author=B. Triggs|titlechapter=Detecting Keypoints with Stable Position, Orientation, and Scale under Illumination Changes

|book-title=Proc. European Conference on Computer Vision|volume=4|pages=100–113|year=2004}}

</ref><ref>

{{cite conference|author1=C. Kenney, M. Zuliani |author2=B. Manjunath |name-list-style=amp |titlechapter=An Axiomatic Approach to Corner Detection|book-title=Proc. IEEE Computer Vision and Pattern Recognition|pages=191–197|year=2005}}

</ref> The structure tensor also plays a central role in the [[Lucas–Kanade Optical Flow Method|Lucas-Kanade optical flow algorithm]], and in its extensions to estimate [[affine shape adaptation]];<ref name=lingar97/> where the magnitude of <math>\lambda_2</math> is an indicator of the reliability of the computed result. The tensor has been used for [[scale space]] analysis,<ref name=lin94book/> estimation of local surface orientation from monocular or binocular cues,<ref name=garlin96/> non-linear [[fingerprint enhancement]],<ref>

A. Almansa and T. Lindeberg (2000), ''[http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A338874&dswid=-9161 Enhancement of fingerprint images using shape-adaptated scale-space operators]''. IEEE Transactions on Image Processing, volume 9, number 12, pages 2027–2042.

</ref> [[diffusion-based image processing]],<ref>[http://www.mia.uni-saarland.de/weickert/book.html J. Weickert (1998), Anisotropic diffusion in image processing, Teuber Verlag, Stuttgart.]</ref><ref>

{{cite journal|author=D. Tschumperle and Deriche|titlechapter=Diffusion PDE's on Vector-Valued Images|book-title=IEEE Signal Processing Magazine|pages=16–25|date=September 2002}}

</ref><ref>

{{cite conference|author1=S. Arseneau |author2=J. Cooperstock |name-list-style=amp |titlechapter=An Asymmetrical Diffusion Framework for Junction Analysis|book-title=British Machine Vision Conference|volume=2|pages=689–698|date=September 2006}}

</ref><ref>

{{cite conference|author1=S. Arseneau |author2=J. Cooperstock |name-list-style=amp |titlechapter=An Improved Representation of Junctions through Asymmetric Tensor Diffusion|book-title=International Symposium on Visual Computing|date=November 2006}}

</ref> and several other image processing problems. The structure tensor can be also applied in [[geology]] to filter [[Seismology|seismic]] data.<ref>{{Cite journal|last1=Yang|first1=Shuai|last2=Chen|first2=Anqing|last3=Chen|first3=Hongde|date=2017-05-25|title=Seismic data filtering using non-local means algorithm based on structure tensor|journal=Open Geosciences|volume=9|issue=1|pages=151–160|doi=10.1515/geo-2017-0013|issn=2391-5447|bibcode=2017OGeo....9...13Y|s2cid=134392619|doi-access=free}}</ref>

Line 278:

:<math> \begin{bmatrix} x' \\ y' \\ t' \end{bmatrix} = G \begin{bmatrix} x \\ y \\ t \end{bmatrix} = \begin{bmatrix} x - v_x \, t \\ y - v_y \, t \\ t \end{bmatrix} </math>,

it is, however, from a computational viewpoint preferable to parameterize the components in the structure tensor/second-moment matrix <math>S</math> using the notion of ''Galilean diagonalization''<ref name=lin04icpr>

{{cite conference|author1=T. Lindeberg |author2=A. Akbarzadeh |author3=I. Laptev |name-list-style=amp |titlechapter=Galilean-corrected spatio-temporal interest operators|book-title=International Conference on Pattern Recognition ICPR'04|url=http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A441205&dswid=-545 |doi=10.1109/ICPR.2004.1334004|date=August 2004|volume=I| pages=57–62}}

</ref>

:<math> S' = R_\text{space}^{-\text{T}} \, G^{-\text{T}} \, S \, G^{-1} \, R_\text{space}^{-1} = \begin{bmatrix} \nu_1 & \, & \, \\ \, & \nu_2 & \, \\ \, & \, & \nu_3 \end{bmatrix} </math>

Line 285:

:<math> S'' = R_\text{spacetime}^{-\text{T}} \, S \, R_\text{spacetime}^{-1} = \begin{bmatrix} \lambda_1 & & \\ & \lambda_2 & \\ & & \lambda_3 \end{bmatrix} </math>.

To obtain true Galilean invariance, however, also the shape of the spatio-temporal window function needs to be adapted,<ref name=lin04icpr/><ref>

{{cite conference|author1=I. Laptev |author2=T. Lindeberg |name-list-style=amp |title=Velocity adaptation of space–time interest points |book-titleconference=International Conference on Pattern Recognition ICPR'04 |url=http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A451230&dswid=-7763|doi=10.1109/ICPR.2004.971|date=August 2004|volume=I| pages=52–56}}

</ref> corresponding to the transfer of [[affine shape adaptation]]<ref name=lingar97/> from spatial to spatio-temporal image data.

In combination with local spatio-temporal histogram descriptors,<ref>

{{cite conference|author1=I. Laptev |author2=T. Lindeberg |name-list-style=amp |title=Local descriptors for spatio-temporal recognition|book-titleconference=ECCV'04 Workshop on Spatial Coherence for Visual Motion Analysis (Prague, Czech Republic) Springer Lecture Notes in Computer Science|url=http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A445261&dswid=-1233|doi=10.1007/11676959|date=May 2004|volume=3667| pages=91–103}}

</ref>

these concepts together allow for Galilean invariant recognition of spatio-temporal events.<ref>

{{cite conference|author1=I. Laptev |author2=B. Caputo |author3=C. Schuldt |author4=T. Lindeberg |name-list-style=amp |titlechapter=Local velocity-adapted motion events for spatio-temporal recognition |book-title=Computer Vision and Image Understanding|url=http://kth.diva-portal.org/smash/record.jsf?pid=diva2%3A335153&dswid=7950 |doi=10.1016/j.cviu.2006.11.023|year=2007|volume=108| pages= 207–229}}</ref>

==See also==