Region Based Convolutional Neural Networks: Difference between revisions

Revision as of 08:06, 20 September 2024

Revision as of 18:05, 10 October 2024

Line 1:

{{Short description|Machine learning model family}}

[[File:R-cnn.svg|thumb|272x272px|R-CNN architecture]]

'''Region-based Convolutional Neural Networks (R-CNN)''' are a family of machine learning models for [[computer vision]], and specifically [[object detection]] and localization.<ref name=":0">{{Cite book |lastlast1=Zhang |firstfirst1=Aston |title=Dive into deep learning |last2=Lipton |first2=Zachary |last3=Li |first3=Mu |last4=Smola |first4=Alexander J. |date=2024 |publisher=Cambridge University Press |isbn=978-1-009-38943-3 |___location=Cambridge New York Port Melbourne New Delhi Singapore |chapter=14.8. Region-based CNNs (R-CNNs) |chapter-url=https://d2l.ai/chapter_computer-vision/rcnn.html}}</ref> The original goal of R-CNN was to take an input image and produce a set of [[Minimum bounding box|bounding boxes]] as output, where each bounding box contains an object and also the category (e.g. car or pedestrian) of the object. In general, R-CNN architectures perform selective search<ref name=":1">{{Cite journal |lastlast1=Uijlings |firstfirst1=J. R. R. |last2=van de Sande |first2=K. E. A. |last3=Gevers |first3=T. |last4=Smeulders |first4=A. W. M. |date=2013-09-01 |title=Selective Search for Object Recognition |url=https://link.springer.com/article/10.1007/s11263-013-0620-5 |journal=International Journal of Computer Vision |volume=104 |issue=2 |pages=154–171 |doi=10.1007/s11263-013-0620-5 |issn=1573-1405}}</ref> over feature maps outputted by a CNN.

R-CNN has been extended to perform other computer vision tasks, such as: tracking objects from a drone-mounted camera,<ref>{{Cite news |last=Nene |first=Vidi |date=Aug 2, 2019 |title=Deep Learning-Based Real-Time Multiple-Object Detection and Tracking via Drone |url=https://dronebelow.com/2019/08/02/deep-learning-based-real-time-multiple-object-detection-and-tracking-via-drone/ |access-date=Mar 28, 2020 |work=Drone Below}}</ref> locating text in an image,<ref>{{Cite news |last=Ray |first=Tiernan |date=Sep 11, 2018 |title=Facebook pumps up character recognition to mine memes |url=https://www.zdnet.com/article/facebook-pumps-up-character-recognition-to-mine-memes/ |access-date=Mar 28, 2020 |publisher=[[ZDNET]]}}</ref> and enabling object detection in [[Google Lens]].<ref>{{Cite news |last=Sagar |first=Ram |date=Sep 9, 2019 |title=These machine learning methods make google lens a success |url=https://analyticsindiamag.com/these-machine-learning-techniques-make-google-lens-a-success/ |access-date=Mar 28, 2020 |work=Analytics India}}</ref>

Line 12:

* November 2013: '''R-CNN'''.<ref name=":2" />

* April 2015: '''Fast R-CNN'''.<ref name=":3">{{Cite journalbook |last=Girshick |first=Ross |date=7–13 December 2015 |titlechapter=Fast R-CNN |urldate=http://ieeexplore.ieee.org/document/7410526/7–13 December 2015 |journaltitle=2015 IEEE International Conference on Computer Vision (ICCV) |chapter-url=https://ieeexplore.ieee.org/document/7410526 |publisher=IEEE |pages=1440–1448 |doi=10.1109/ICCV.2015.169 |isbn=978-1-4673-8391-2}}</ref>

* June 2015: '''Faster R-CNN'''.<ref name=":4">{{Cite journal |lastlast1=Ren |firstfirst1=Shaoqing |last2=He |first2=Kaiming |last3=Girshick |first3=Ross |last4=Sun |first4=Jian |date=2017-06-01 |title=Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks |url=httphttps://ieeexplore.ieee.org/document/7485869/ |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=39 |issue=6 |pages=1137–1149 |doi=10.1109/TPAMI.2016.2577031 |issn=0162-8828|arxiv=1506.01497 }}</ref>

* March 2017: '''Mask R-CNN'''.<ref name=":5">{{Cite journalbook |lastlast1=He |firstfirst1=Kaiming |last2=Gkioxari |first2=Georgia |last3=Dollar |first3=Piotr |last4=Girshick |first4=Ross |chapter=Mask R-CNN |date=October 2017 |title=Mask2017 R-CNNIEEE International Conference on Computer Vision (ICCV) |chapter-url=httphttps://ieeexplore.ieee.org/document/8237584/ |publisher=IEEE |pages=2980–2988 |doi=10.1109/ICCV.2017.322 |isbn=978-1-5386-1032-9}}</ref>

* June 2019: '''Mesh R-CNN''' adds the ability to generate a 3D mesh from a 2D image.<ref>{{Cite journal |lastlast1=Gkioxari |firstfirst1=Georgia |last2=Malik |first2=Jitendra |last3=Johnson |first3=Justin |date=2019 |title=Mesh R-CNN |url=https://openaccess.thecvf.com/content_ICCV_2019/html/Gkioxari_Mesh_R-CNN_ICCV_2019_paper.html |pages=9785–9795}}</ref>

== Architecture ==

Line 21:

=== Selective search ===

Given an image (or an image-like feature map), '''selective search''' (also called Hierarchical Grouping) first segments the image by the algorithm in (Felzenszwalb and Huttenlocher, 2004),<ref>{{Cite journal |lastlast1=Felzenszwalb |firstfirst1=Pedro F. |last2=Huttenlocher |first2=Daniel P. |date=2004-09-01 |title=Efficient Graph-Based Image Segmentation |url=https://link.springer.com/article/10.1023/B:VISI.0000022288.19776.77 |journal=International Journal of Computer Vision |language=en |volume=59 |issue=2 |pages=167–181 |doi=10.1023/B:VISI.0000022288.19776.77 |issn=1573-1405}}</ref> then performs the following:<ref name=":1" />

<pre>

Input: (colour) image

Line 44:

=== R-CNN ===

[[File:R-cnn.svg|thumb|272x272px|R-CNN architecture]]

Given an input image, R-CNN begins by applying selective search to extract [[Region of interest|regions of interest]] (ROI), where each ROI is a rectangle that may represent the boundary of an object in image. Depending on the scenario, there may be as many as {{nobr|two thousand}} ROIs. After that, each ROI is fed through a neural network to produce output features. For each ROI's output features, an ensemble of [[support-vector machine]] classifiers is used to determine what type of object (if any) is contained within the ROI.<ref name=":2">{{Cite journal |lastlast1=Girshick |firstfirst1=Ross |last2=Donahue |first2=Jeff |last3=Darrell |first3=Trevor |last4=Malik |first4=Jitendra |date=2016-01-01 |title=Region-Based Convolutional Networks for Accurate Object Detection and Segmentation |url=httphttps://ieeexplore.ieee.org/document/7112511/ |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=38 |issue=1 |pages=142–158 |doi=10.1109/TPAMI.2015.2437384 |pmid=26656583 |issn=0162-8828}}</ref>

=== Fast R-CNN ===