Gradient-___domain image processing: Difference between revisions

Revision as of 15:17, 9 August 2014

Latest revision as of 14:11, 16 May 2024

(19 intermediate revisions by 11 users not shown)

Line 1:

'''Gradient ___domain image processing''', isalso acalled relatively'''Poisson newimage editing''',<ref name="Perez2003"/> is a type of [[digital image processing]] that operates directly on the differences between neighboring pixels, rather than on the pixel values directly. AnMathematically, an [[image gradient]] represents the [[derivative]] of an image, so the goal of gradient ___domain processing is to obtainconstruct ana new image by [[integral|integrating]] the gradient, which requires solving [[Poisson's equation]].<ref name="Bhat2010">Bhat,{{cite Pravin, et aljournal|doi=10. "Gradientshop1145/1731047.1731048|url=http://grail.cs.washington.edu/projects/gradientshop/demos/gs_paper_TOG_2009.pdf|title=Gradient A''Shop'' gradient-___domain|year=2010 optimization|last1=Bhat framework|first1=Pravin for|last2=Zitnick image|first2=C. andLawrence video|last3=Cohen filtering."|first3=Michael |last4=Curless |first4=Brian |journal=ACM Transactions on Graphics (TOG) |volume=29. |issue=2 (2010):|pages=1–14 10.|s2cid=3097999 }}</ref>

== BasicsOverview ==

'''Gradient ___domain image processing''' is a relatively new type of [[digital image processing]] that operates on the differences between neighboring pixels, rather than on the pixel values directly. An [[image gradient]] represents the derivative of an image, so the goal of gradient ___domain processing is to obtain an image by integrating the gradient, which requires solving [[Poisson's equation]].<ref name="Bhat2010">Bhat, Pravin, et al. "Gradientshop: A gradient-___domain optimization framework for image and video filtering." ACM Transactions on Graphics (TOG) 29.2 (2010): 10.</ref>

== Basics ==

Processing images in the gradient ___domain is a two-step process. The first step is to choose an image gradient. This is often extracted from one or more images and then modified, but it can also be obtained through other means as well. For example, some researchers have explored the advantages of users painting directly in the gradient ___domain.,<ref>McCann,{{cite James, and Nancy Sjournal|doi=10. Pollard1145/1360612. "1360692|url=http://graphics.cs.cmu.edu/nsp/papers/gradFinal.pdf|title=Real-time gradient-___domain painting |year=2008 |last1=McCann |first1=James |last2=Pollard |first2=Nancy S." |journal=ACM Transactions on Graphics (TOG).|volume=27 Vol|issue=3 |pages=1–7 }}</ref> while others have proposed sampling a gradient directly from a camera sensor.<ref>{{cite 27book|url=https://users.cs.northwestern.edu/~jet/docs/2005_1323GradCamFinal.pdf|doi=10.1109/CVPR.2005.374 |chapter=Why I Want a Gradient Camera |title=2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) |year=2005 |last1=Tumblin No|first1=J. 3|last2=Agrawal |first2=A. ACM,|last3=Raskar 2008|first3=R. |volume=1 |pages=103–110 |isbn=0-7695-2372-2 |s2cid=1821571 }}</ref> The second step is to solve Poisson's equation to find a new image that can produce the gradient from the first step. An exact solution often does not exist, sobecause anthe imagemodified gradient field is foundno thatlonger approximates[[Conservative vector field|conservative]], so an image approximating the desired gradient as closely as possible is found.

== Image Processingediting ==

For image processing purposes, theThe gradient is obtained from an existing image and modified for [[image editing]] purposes. Various methodsoperators, such as a[[finite difference]] or [[Sobel operator|Sobel]], can be used to find the gradient of a given image. This gradient can then be manipulated directly to produce a number ofseveral different effects when the resulting image is solved for. For example, if thea gradientuniform isconstant scaledscales bythe a uniform constantgradient, it results in a simple sharpening filter. A better sharpening filter can be made by only scaling the gradient in areas deemed important.<ref name="Bhat2010" />

Other uses include seamless [[image stitching]],<ref>{{cite book|doi=10.1007/978-3-540-24673-2_31|chapter-url=https://webee.technion.ac.il/people/anat.levin/papers/blendingTR.pdf|chapter=Seamless Image Stitching in the Gradient Domain |title=Computer Vision - ECCV 2004 |series=Lecture Notes in Computer Science |year=2004 |last1=Levin |first1=Anat |last2=Zomet |first2=Assaf |last3=Peleg |first3=Shmuel |last4=Weiss |first4=Yair |volume=3024 |pages=377–389 |isbn=978-3-540-21981-1 }}</ref> removal of unwanted details from an image,<ref name="Perez2003">{{cite book|doi=10.1145/1201775.882269|chapter-url=https://www.cs.jhu.edu/~misha/Fall07/Papers/Perez03.pdf|chapter=Poisson image editing |title=ACM SIGGRAPH 2003 Papers on - SIGGRAPH '03 |year=2003 |last1=Pérez |first1=Patrick |last2=Gangnet |first2=Michel |last3=Blake |first3=Andrew |page=313 |isbn=1581137095 |s2cid=6541990 }}</ref> [[non-photorealistic rendering]] filters,<ref name="Bhat2010" /> image [[deblocking]],<ref name="Bhat2010" />

the ability to seamlessly clone one part of an image onto another in ways that are difficult to achieve with conventional image-___domain techniques,<ref name="Perez2003" /> and [[high-dynamic-range imaging]]<ref>{{cite book|doi=10.1145/566570.566573|chapter-url=https://www.cs.huji.ac.il/~danix/hdr/hdrc.pdf|chapter=Gradient ___domain high dynamic range compression |title=Proceedings of the 29th annual conference on Computer graphics and interactive techniques - SIGGRAPH '02 |year=2002 |last1=Fattal |first1=Raanan |last2=Lischinski |first2=Dani |last3=Werman |first3=Michael |page=249 |isbn=1581135211 |s2cid=1650337 }}</ref>

These gradient-___domain editing techniques can also be extended to moving images by considering a video clip to be a cube of pixels and solving a 3d Poisson equation.<ref>{{cite journal|doi=10.1016/j.gmod.2006.06.002|title=Videoshop: A new framework for spatio-temporal video editing in gradient ___domain |year=2007 |last1=Wang |first1=Hongcheng |last2=Xu |first2=Ning |last3=Raskar |first3=Ramesh |last4=Ahuja |first4=Narendra |journal=Graphical Models |volume=69 |pages=57–70 }}</ref>

== Seamless image cloning ==

[[Digital compositing]] is a common task in image editing in which some or all of one photo is pasted into another. Traditionally, this is done by pasting the pixel values from one image to another. A well-trained artist can make a convincing composite using traditional techniques, but it usually requires time-consuming color correction and mask cutting to make it work. Alternatively, the pasting can be performed in the gradient ___domain: if the differences between pixels are pasted rather than the actual pixel values, there is sometimes much less user input needed to achieve a clean result. The following example demonstrates the use of gradient-___domain image processing to paste from one image to another seamlessly.

<gallery mode="traditional" widths="200" heights="180">

HandPhoto.jpg|Input image A

EyePhoto.jpg|Input image B

ModifiedGradient.jpg|Modified gradient. This is the result of pasting the gradient of B onto the gradient of A.

GradientDomainPaste.jpg|Reconstructed image. This is the result of solving Poisson's equation on the modified gradient. The seam between the two images is barely visible.

ImageDomainPaste.jpg|Traditional image ___domain paste. This is the result of pasting the pixel values directly from B onto A. There is an obvious seam.

</gallery>

Notice that the hand and the eye shifted color slightly in the image reconstructed from the modified gradient. This happened because the solver was set to find the entire image. However, it is possible to add constraints so that only the pasted section is solved, leaving the rest of the image unmodified. It is also worth noting that the gradient pictured above represents the derivative of only one [[color channel]] (red) and was rendered with colors representing the strength and direction of the gradient. In practice, two [[grayscale]] gradient images are found per color channel, one representing the change in ''x'' and the other representing the change in ''y''. Each color channel is solved independently when reconstructing the final image.

For image processing purposes, the gradient is obtained from an existing image and modified. Various methods, such as a [[Sobel operator]], can be used to find the gradient of a given image. This gradient can then be manipulated directly to produce a number of different effects when the resulting image is solved for. For example, if the gradient is scaled by a uniform constant it results in a simple sharpening filter. A better sharpening filter can be made by only scaling the gradient in areas deemed important.<ref name="Bhat2010" />

Other uses include:

* seamless [[image stitching]]<ref>Levin, Anat, et al. "Seamless image stitching in the gradient ___domain." Computer Vision-ECCV 2004. Springer Berlin Heidelberg, 2004. 377-389.</ref>

* removal of unwanted details from an image<ref name="Perez2003">Pérez, Patrick, Michel Gangnet, and Andrew Blake. "Poisson image editing." ACM Transactions on Graphics (TOG). Vol. 22. No. 3. ACM, 2003.</ref>

* [[non-photorealistic rendering]] filters<ref name="Bhat2010" />

* image [[deblocking]]<ref name="Bhat2010" />

* the ability to seamlessly clone one part of an image onto another in ways that are difficult to achieve with conventional image-___domain techniques.<ref name="Perez2003" />

* [[High-dynamic-range imaging]]<ref>Fattal, Raanan, Dani Lischinski, and Michael Werman. "Gradient ___domain high dynamic range compression." ACM Transactions on Graphics (TOG). Vol. 21. No. 3. ACM, 2002.</ref>

== References ==

{{reflist}}

[[Category:Image processing]]

{{uncategorized|date=August 2014}}