Capsule neural network: Difference between revisions

Revision as of 22:39, 23 August 2023

Latest revision as of 22:23, 5 November 2024

(2 intermediate revisions by 2 users not shown)

Line 1:

{{Short description|Type of artificial neural network}}

A '''capsule neural network''' ('''CapsNet''') is a machine learning system that is a type of [[artificial neural network]] (ANN) that can be used to better model hierarchical relationships. The approach is an attempt to more closely mimic biological neural organization.<ref name=":1" />

Line 30 ⟶ 31:

== Pooling ==

Capsnets reject the [[convolutional neural network#Pooling layer|pooling layer]] strategy of conventional CNNs that reduces the amount of detail to be processed at the next higher layer. Pooling allows a degree of translational invariance (it can recognize the same object in a somewhat different ___location) and allows a larger number of feature types to be represented. Capsnet proponents argue that pooling:<ref name=":1"/>

* violates biological shape perception in that it has no intrinsic coordinate frame;

* provides invariance (discarding positional information) instead of equivariance (disentangling that information);

Line 159 ⟶ 160:

== Human vision ==

Human vision examines a sequence of focal points (directed by [[saccade]]s), processing only a fraction of the scene at its highest resolution. Capsnets build on inspirations from [[cortical minicolumn]]s (also called cortical microcolumns) in the [[cerebral cortex]]. A minicolumn is a structure containing 80-120 neurons, with a diameter of about 28-40&nbsp;µmμm, spanning all layers in the cerebral cortex. All neurons in the larger minicolumns have the same [[receptive field]], and they output their activations as [[action potential]]s or spikes.<ref name=":1"/> Neurons within the microcolumn receive common inputs, have common outputs, are interconnected and may constitute a fundamental computational unit of the [[cerebral cortex]].<ref>{{Cite web|url=http://www.physics.drexel.edu/~ccruz/micros/research.html|title=Microcolumns in the Brain|website=www.physics.drexel.edu|access-date=2017-12-31|archive-date=2018-05-27|archive-url=https://web.archive.org/web/20180527140322/http://www.physics.drexel.edu/%7Eccruz/micros/research.html|url-status=dead}}</ref>

Capsnets explore the intuition that the human visual system creates a [[Parse tree|tree]]-like structure for each focal point and coordinates these trees to recognize objects. However, with capsnets each tree is "carved" from a fixed network (by adjusting coefficients) rather than assembled on the fly.<ref name=":1"/>

Line 204 ⟶ 205:

* {{YouTube|Hinton and Google Brain - Capsule Networks |id=x5Vxk9twXlE}}

* {{Citation|last=Liao|first=Huadong|title=CapsNet-Tensorflow: A Tensorflow implementation of CapsNet(Capsules Net) in Hinton's paper Dynamic Routing Between Capsules|date=2017-12-08|url=https://github.com/naturomics/CapsNet-Tensorflow|access-date=2017-12-08}}

*{{Cite web|first=Fangyu|last=Cai|date=2020-12-18|title='We Can Do It' — Geoffrey Hinton and UBC, UT, Google & UVic Team Propose Unsupervised Capsule…Capsule...|url=https://medium.com/syncedreview/we-can-do-it-geoffrey-hinton-and-ubc-ut-google-uvic-team-propose-unsupervised-capsule-c1f2edb6b1e9|access-date=2021-01-18|website=Medium|language=en}}

* {{cite arXiv|last1=Sun|first1=Weiwei|last2=Tagliasacchi|first2=Andrea|last3=Deng|first3=Boyang|last4=Sabour|first4=Sara|last5=Yazdani|first5=Soroosh|last6=Hinton|first6=Geoffrey|last7=Yi|first7=Kwang Moo|date=2020-12-08|title=Canonical Capsules: Unsupervised Capsules in Canonical Pose|class=cs.CV|eprint=2012.04718}}