Content deleted Content added
GoldMiner24 (talk | contribs) No edit summary |
m redirect bypass from Conciousness to consciousness using popups |
||
| (47 intermediate revisions by 27 users not shown) | |||
Line 1:
{{short description|Method by which information is represented in the brain}}
'''Neural coding''' (or '''neural representation''')
== Overview ==
Neurons have an ability uncommon among the cells of the body to propagate signals rapidly over large distances by generating characteristic electrical pulses called [[action potentials]]: voltage spikes that can travel down axons. Sensory neurons change their activities by firing sequences of action potentials in various temporal patterns, with the presence of external sensory stimuli, such as [[light]], [[sound]], [[taste]], [[Olfaction|smell]] and [[touch]]. Information about the stimulus is encoded in this pattern of action potentials and transmitted into and around the brain. Beyond this, specialized neurons, such as those of the retina, can communicate more information through [[graded potential]]s.
Although action potentials can vary somewhat in duration, [[amplitude]] and shape, they are typically treated as identical stereotyped events in neural coding studies. If the [[Brief-spike|brief duration]] of an action potential (about 1 ms) is ignored, an action potential sequence, or spike train, can be characterized simply by a series of [[all-or-none law|all-or-none]] point events in time.<ref name="Gerstner">{{cite book|author-link1=Wulfram Gerstner |first1=Wulfram |last1=Gerstner |first2=Werner M. |last2=Kistler |title=Spiking Neuron Models: Single Neurons, Populations, Plasticity |url=https://books.google.com/books?id=Rs4oc7HfxIUC |year=2002 |publisher=Cambridge University Press |isbn=978-0-521-89079-3}}</ref> The lengths of interspike intervals ([[Temporal coding|ISI]]s) between two successive spikes in a spike train often vary, apparently randomly.<ref name="Stein">{{cite journal |vauthors=Stein RB, Gossen ER, Jones KE |title=Neuronal variability: noise or part of the signal? |journal=Nat. Rev. Neurosci. |volume=6 |issue=5 |pages=389–97 |date=May 2005 |pmid=15861181 |doi=10.1038/nrn1668 |s2cid=205500218 }}</ref> The study of neural coding involves measuring and characterizing how stimulus attributes, such as light or sound intensity, or motor actions, such as the direction of an arm movement, are represented by neuron action potentials or spikes. In order to describe and analyze neuronal firing, [[statistical methods]] and methods of [[probability theory]] and stochastic [[point process]]es have been widely applied.
With the development of large-scale neural recording and decoding technologies, researchers have begun to crack the neural code and have already provided the first glimpse into the real-time neural code as memory is formed and recalled in the hippocampus, a brain region known to be central for memory formation.<ref>The Memory Code. http://www.scientificamerican.com/article/the-memory-code/</ref><ref>{{cite journal | last1 = Chen | first1 = G | last2 = Wang | first2 = LP | last3 = Tsien | first3 = JZ | year = 2009 | title = Neural population-level memory traces in the mouse hippocampus | journal = PLOS ONE | volume = 4 | issue = 12| page = e8256 | doi = 10.1371/journal.pone.0008256 | pmid = 20016843 | pmc=2788416| bibcode = 2009PLoSO...4.8256C | doi-access = free }}</ref><ref>{{cite journal | last1 = Zhang | first1 = H | last2 = Chen | first2 = G | last3 = Kuang | first3 = H | last4 = Tsien | first4 = JZ | date = Nov 2013 | title = Mapping and deciphering neural codes of NMDA receptor-dependent fear memory engrams in the hippocampus | journal = PLOS ONE | volume = 8 | issue = 11| page = e79454 | doi = 10.1371/journal.pone.0079454 | pmid = 24302990 | pmc=3841182| bibcode = 2013PLoSO...879454Z | doi-access = free }}</ref> Neuroscientists have initiated several large-scale brain decoding projects.<ref>Brain Decoding Project. http://
== Encoding and decoding ==
The link between stimulus and response can be studied from two opposite points of view. Neural encoding refers to the map from stimulus to response. The main focus is to understand how neurons respond to a wide variety of stimuli, and to construct models that attempt to predict responses to other stimuli. [[Neural decoding]] refers to the reverse map, from response to stimulus, and the challenge is to reconstruct a stimulus, or certain aspects of that stimulus, from the spike sequences it evokes.{{Citation needed|date=January 2025}}
== Hypothesized coding schemes ==
A sequence, or 'train', of spikes may contain information based on different coding schemes. In some neurons the strength with which
Whether neurons use rate coding or temporal coding is a topic of intense debate within the neuroscience community, even though there is no clear definition of what these terms mean.<ref name=":0">{{Cite book|last=Gerstner, Wulfram.
=== Rate
The rate coding model of [[neuron]]al firing communication states that as the intensity of a stimulus increases, the [[frequency]] or rate of [[action potential]]s, or "spike firing", increases. Rate coding is sometimes called frequency coding.
Line 38:
Despite its shortcomings, the concept of a spike-count rate code is widely used not only in experiments, but also in models of [[neural networks]]. It has led to the idea that a neuron transforms information about a single input variable (the stimulus strength) into a single continuous output variable (the firing rate).
There is a growing body of evidence that in [[Purkinje neurons]], at least, information is not simply encoded in firing but also in the timing and duration of non-firing, quiescent periods.<ref>{{cite journal |author=Forrest MD |title=Intracellular Calcium Dynamics Permit a Purkinje Neuron Model to Perform Toggle and Gain Computations Upon its Inputs. |journal=Frontiers in Computational Neuroscience |volume=8 |pages=86 |year=2014 | doi=10.3389/fncom.2014.00086 |pmid=25191262 |pmc=4138505|doi-access=free }}</ref><ref>{{cite journal |author=Forrest MD |title=The sodium-potassium pump is an information processing element in brain computation |journal= Frontiers in Physiology |volume=5 |issue=472 |pages=472 | date=December 2014 |doi=10.3389/fphys.2014.00472 |pmid=25566080 |pmc=4274886 |doi-access=free }}</ref> There is also evidence from retinal cells, that information is encoded not only in the firing rate but also in spike timing.<ref name=":1">{{Cite journal|last1=Gollisch|first1=T.|last2=Meister|first2=M.|date=2008-02-22|title=Rapid Neural Coding in the Retina with Relative Spike Latencies|url=https://www.sciencemag.org/lookup/doi/10.1126/science.1149639|journal=Science|language=en|volume=319|issue=5866|pages=1108–1111|doi=10.1126/science.1149639|pmid=18292344|bibcode=2008Sci...319.1108G|s2cid=1032537|issn=0036-8075|url-access=subscription}}</ref> More generally, whenever a rapid response of an organism is required a firing rate defined as a spike-count over a few hundred milliseconds is simply too slow.<ref name=":0" />
==== Time-dependent firing rate (averaging over several trials) ====
Line 54:
When precise spike timing or high-frequency firing-rate [[Statistical fluctuations|fluctuations]] are found to carry information, the neural code is often identified as a temporal code.<ref name=":0" /><ref name="Dayan">{{cite book |first1=Peter |last1=Dayan |first2=L. F. |last2=Abbott |title=Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems |url=https://books.google.com/books?id=5GSKQgAACAAJ |year=2001 |publisher=Massachusetts Institute of Technology Press |isbn=978-0-262-04199-7}}</ref> A number of studies have found that the temporal resolution of the neural code is on a millisecond time scale, indicating that precise spike timing is a significant element in neural coding.<ref name="thorpe" /><ref name="Daniel">{{cite journal |vauthors=Butts DA, Weng C, Jin J, etal |title=Temporal precision in the neural code and the timescales of natural vision |journal=Nature |volume=449 |issue=7158 |pages=92–5 |date=September 2007 |pmid=17805296 |doi=10.1038/nature06105 |bibcode = 2007Natur.449...92B |s2cid=4402057 }}</ref><ref name=":1" /> Such codes, that communicate via the time between spikes are also referred to as interpulse interval codes, and have been supported by recent studies.<ref>Singh & Levy, [http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0180839 "A consensus layer V pyramidal neuron can sustain interpulse-interval coding "], ''PLoS ONE'', 2017</ref>
Neurons exhibit high-frequency fluctuations of firing-rates which could be noise or could carry information. Rate coding models suggest that these irregularities are noise, while temporal coding models suggest that they encode information. If the nervous system only used rate codes to convey information, a more consistent, regular firing rate would have been evolutionarily advantageous, and neurons would have utilized this code over other less robust options.<ref name="van Hemmen 2006">J. Leo van Hemmen, TJ Sejnowski. 23 Problems in Systems Neuroscience. Oxford Univ. Press, 2006. p.143-158.</ref> Temporal coding supplies an alternate explanation for the “noise," suggesting that it actually encodes information and affects neural processing. To model this idea, binary symbols can be used to mark the spikes: 1 for a spike, 0 for no spike. Temporal coding allows the sequence 000111000111 to mean something different from 001100110011, even though the mean firing rate is the same for both sequences, at 6 spikes/10 ms.<ref name="Theunissen F 1995"/
Until recently, scientists had put the most emphasis on rate encoding as an explanation for [[post-synaptic potential]] patterns. However, functions of the brain are more temporally precise than the use of only rate encoding seems to allow.<ref name=":1" /> In other words, essential information could be lost due to the inability of the rate code to capture all the available information of the spike train. In addition, responses are different enough between similar (but not identical) stimuli to suggest that the distinct patterns of spikes contain a higher volume of information than is possible to include in a rate code.<ref name="Zador, Stevens">{{cite web|last=Zador, Stevens|first=Charles, Anthony|title=The enigma of the brain|url=https://docs.google.com/a/stolaf.edu/viewer?a=v&pid=gmail&attid=0.1&thid=1369b5e1cdf273f9&mt=application/pdf&url=https://mail.google.com/mail/u/0/?ui%3D2%26ik%3D0a436eb2a7%26view%3Datt%26th%3D1369b5e1cdf273f9%26attid%3D0.1%26disp%3Dsafe%26realattid%3Df_h0ty13ea0%26zw&sig=AHIEtbQB4vngr9nDZaMTLUOcrk5DzePKqA|work=© Current Biology 1995, Vol 5 No 12|access-date=August 4, 2012}}</ref>
Temporal codes (also called [https://lcnwww.epfl.ch/gerstner/SPNM/node8.html spike codes] <ref name=":0" />), employ those features of the spiking activity that cannot be described by the firing rate. For example, '''time-to-first-spike''' after the stimulus onset, '''phase-of-firing''' with respect to background oscillations, characteristics based on the second and higher statistical [[Moment (mathematics)|moments]] of the ISI [[probability distribution]], spike randomness, or precisely timed groups of spikes ('''temporal patterns''') are candidates for temporal codes.<ref name="Kostal">{{cite journal |vauthors=Kostal L, Lansky P, Rospars JP |title=Neuronal coding and spiking randomness |journal=Eur. J. Neurosci. |volume=26 |issue=10 |pages=2693–701 |date=November 2007 |pmid=18001270 |doi=10.1111/j.1460-9568.2007.05880.x |s2cid=15367988 }}</ref> As there is no absolute time reference in the nervous system, the information is carried either in terms of the relative timing of spikes in a population of neurons (temporal patterns) or with respect to an [[neural oscillations|ongoing brain oscillation]] (phase of firing).<ref name="thorpe" /><ref name="Stein" /> One way in which temporal codes are decoded, in presence of [[neural oscillations]], is that spikes occurring at specific phases of an oscillatory cycle are more effective in depolarizing the [[Chemical synapse|post-synaptic neuron]].<ref name = "Gupta2016">{{Cite journal|last1=Gupta|first1=Nitin|last2=Singh|first2=Swikriti Saran|last3=Stopfer|first3=Mark|date=2016-12-15|title=Oscillatory integration windows in neurons|journal=Nature Communications|volume=7|doi=10.1038/ncomms13808|issn=2041-1723|pmc=5171764|pmid=27976720| The temporal structure of a spike train or firing rate evoked by a stimulus is determined both by the dynamics of the stimulus and by the nature of the neural encoding process. Stimuli that change rapidly tend to generate precisely timed spikes<ref>{{Cite journal|last1=Jolivet|first1=Renaud|last2=Rauch|first2=Alexander|last3=Lüscher|first3=Hans-Rudolf|last4=Gerstner|first4=Wulfram|date=2006-08-01|title=Predicting spike timing of neocortical pyramidal neurons by simple threshold models|url=https://doi.org/10.1007/s10827-006-7074-5|journal=Journal of Computational Neuroscience|language=en|volume=21|issue=1|pages=35–49|doi=10.1007/s10827-006-7074-5|pmid=16633938|s2cid=8911457|issn=1573-6873}}</ref> (and rapidly changing firing rates in PSTHs) no matter what neural coding strategy is being used. Temporal coding in the narrow sense refers to temporal precision in the response that does not arise solely from the dynamics of the stimulus, but that nevertheless relates to properties of the stimulus. The interplay between stimulus and encoding dynamics makes the identification of a temporal code difficult.
Line 67 ⟶ 69:
For very brief stimuli, a neuron's maximum firing rate may not be fast enough to produce more than a single spike. Due to the density of information about the abbreviated stimulus contained in this single spike, it would seem that the timing of the spike itself would have to convey more information than simply the average frequency of action potentials over a given period of time. This model is especially important for [[sound localization]], which occurs within the brain on the order of milliseconds. The brain must obtain a large quantity of information based on a relatively short neural response. Additionally, if low firing rates on the order of ten spikes per second must be distinguished from arbitrarily close rate coding for different stimuli, then a neuron trying to discriminate these two stimuli may need to wait for a second or more to accumulate enough information. This is not consistent with numerous organisms which are able to discriminate between stimuli in the time frame of milliseconds, suggesting that a rate code is not the only model at work.<ref name="Theunissen F 1995">{{cite journal | last1 = Theunissen | first1 = F | last2 = Miller | first2 = JP | year = 1995 | title = Temporal Encoding in Nervous Systems: A Rigorous Definition | journal = Journal of Computational Neuroscience | volume = 2 | issue = 2| pages = 149–162 | doi=10.1007/bf00961885| pmid = 8521284 | s2cid = 206786736 }}</ref>
To account for the fast encoding of visual stimuli, it has been suggested that neurons of the retina encode visual information in the latency time between stimulus onset and first action potential, also called latency to first spike or time-to-first-spike.<ref>{{cite journal|last=Gollisch|first=T.|author2=Meister, M.|title=Rapid Neural Coding in the Retina with Relative Spike Latencies|journal=Science|date=22 February 2008|volume=319|issue=5866|pages=1108–1111|doi=10.1126/science.1149639|pmid=18292344|bibcode=2008Sci...319.1108G|s2cid=1032537
The mammalian [[gustatory system]] is useful for studying temporal coding because of its fairly distinct stimuli and the easily discernible responses of the organism.<ref>{{cite journal | last1 = Hallock | first1 = Robert M. | last2 = Di Lorenzo | first2 = Patricia M. | year = 2006 | title = Temporal coding in the gustatory system | doi = 10.1016/j.neubiorev.2006.07.005 | pmid = 16979239 | journal = Neuroscience & Biobehavioral Reviews | volume = 30 | issue = 8| pages = 1145–1160 | s2cid = 14739301 }}</ref> Temporally encoded information may help an organism discriminate between different tastants of the same category (sweet, bitter, sour, salty, umami) that elicit very similar responses in terms of spike count. The temporal component of the pattern elicited by each tastant may be used to determine its identity (e.g., the difference between two bitter tastants, such as quinine and denatonium). In this way, both rate coding and temporal coding may be used in the gustatory system – rate for basic tastant type, temporal for more specific differentiation.<ref name="Carleton A 2010">{{cite journal | last1 = Carleton | first1 = Alan | last2 = Accolla | first2 = Riccardo | last3 = Simon | first3 = Sidney A. | year = 2010 | title = Coding in the mammalian gustatory system | doi = 10.1016/j.tins.2010.04.002 | pmid = 20493563 | journal = Trends in Neurosciences | volume = 33 | issue = 7| pages = 326–334 | pmc = 2902637 }}</ref>
As with the visual system, in [[mitral cell|mitral/tufted cells]] in the [[olfactory bulb]] of mice, first-spike latency relative to the start of a sniffing action seemed to encode much of the information about an odor. This strategy of using spike latency allows for rapid identification of and reaction to an odorant. In addition, some mitral/tufted cells have specific firing patterns for given odorants. This type of extra information could help in recognizing a certain odor, but is not completely necessary, as average spike count over the course of the animal's sniffing was also a good identifier.<ref>{{cite journal | last1 = Wilson | first1 = Rachel I | year = 2008 | title = Neural and behavioral mechanisms of olfactory perception | journal = Current Opinion in Neurobiology | volume = 18 | issue = 4| pages = 408–412 | doi=10.1016/j.conb.2008.08.015| pmid = 18809492 | pmc = 2596880 }}</ref> Along the same lines, experiments done with the olfactory system of rabbits showed distinct patterns which correlated with different subsets of odorants, and a similar result was obtained in experiments with the locust olfactory system.<ref name="Theunissen F 1995"/>
Line 83 ⟶ 87:
Phase-of-firing code is a neural coding scheme that combines the [[action potential|spike]] count code with a time reference based on [[Neural oscillations|oscillations]]. This type of code takes into account a time label for each spike according to a time reference based on phase of local ongoing oscillations at low<ref name="Montemurro" /> or high frequencies.<ref name="Gamma cycle">{{cite journal |vauthors=Fries P, Nikolić D, Singer W |title=The gamma cycle |journal=Trends Neurosci. |volume=30 |issue=7 |pages=309–16 |date=July 2007 |pmid=17555828 |doi=10.1016/j.tins.2007.05.005 |s2cid=3070167 }}</ref>
It has been shown that neurons in some cortical sensory areas encode rich naturalistic stimuli in terms of their spike times relative to the phase of ongoing network oscillatory fluctuations, rather than only in terms of their spike count.<ref name="Montemurro">{{cite journal|doi=10.1016/j.cub.2008.02.023|pmid=18328702|title=Phase-of-Firing Coding of Natural Visual Stimuli in Primary Visual Cortex|journal=Current Biology|volume=18|issue=5|pages=375–380|year=2008|last1=Montemurro|first1=Marcelo A.|last2=Rasch|first2=Malte J.|last3=Murayama|first3=Yusuke|last4=Logothetis|first4=Nikos K.|last5=Panzeri|first5=Stefano|doi-access=free|bibcode=2008CBio...18..375M }}</ref><ref>[http://pop.cerco.ups-tlse.fr/fr_vers/documents/thorpe_sj_90_91.pdf Spike arrival times: A highly efficient coding scheme for neural networks] {{webarchive|url=https://web.archive.org/web/20120215151304/http://pop.cerco.ups-tlse.fr/fr_vers/documents/thorpe_sj_90_91.pdf |date=2012-02-15 }}, SJ Thorpe - Parallel processing in neural systems, 1990</ref> The [[local field potential]] signals reflect population (network) oscillations. The phase-of-firing code is often categorized as a temporal code although the time label used for spikes (i.e. the network oscillation phase) is a low-resolution (coarse-grained) reference for time. As a result, often only four discrete values for the phase are enough to represent all the information content in this kind of code with respect to the phase of oscillations in low frequencies. Phase-of-firing code is loosely based on the [[Place cell#Phase precession|phase precession]] phenomena observed in place cells of the [[hippocampus]]. Another feature of this code is that neurons adhere to a preferred order of spiking between a group of sensory neurons, resulting in firing sequence.<ref name="Firing sequences">{{cite journal |vauthors=Havenith MN, Yu S, Biederlack J, Chen NH, Singer W, Nikolić D |title=Synchrony makes neurons fire in sequence, and stimulus properties determine who is ahead |journal=J. Neurosci. |volume=31 |issue=23 |pages=8570–84 |date=June 2011 |pmid=21653861 |pmc=6623348 |doi=10.1523/JNEUROSCI.2817-10.2011 |doi-access=free }}</ref>
Phase code has been shown in visual cortex to involve also [[High frequency oscillations|high-frequency oscillations]].<ref name="Firing sequences" /> Within a cycle of gamma oscillation, each neuron has its own preferred relative firing time. As a result, an entire population of neurons generates a firing sequence that has a duration of up to about 15 ms.<ref name="Firing sequences"/>
=== Population coding ===
Population coding is a method to represent stimuli by using the joint activities of a number of neurons. In population coding, each neuron has a distribution of responses over some set of inputs, and the responses of many neurons may be combined to determine some value about the inputs. From the theoretical point of view, population coding is one of a few mathematically well-formulated problems in neuroscience. It grasps the essential features of neural coding and yet is simple enough for theoretic analysis.<ref name="Wu">{{cite journal |vauthors=Wu S, Amari S, Nakahara H |title=Population coding and decoding in a neural field: a computational study |journal=Neural Comput |volume=14 |issue=5 |pages=999–1026 |date=May 2002 |pmid=11972905 |doi=10.1162/089976602753633367 |s2cid=1122223 }}</ref> Experimental studies have revealed that this coding paradigm is widely used in the
For example, in the visual area [[Medial temporal lobe|medial temporal]] (MT), neurons are tuned to the
Place-time population codes, termed the averaged-localized-synchronized-response (ALSR) code, have been derived for neural representation of auditory acoustic stimuli. This exploits both the place or tuning within the auditory nerve, as well as the phase-locking within each nerve fiber auditory nerve. The first ALSR representation was for steady-state vowels;<ref>{{cite journal|last1=Sachs|first1=Murray B.|last2=Young|first2=Eric D.|title=Representation of steady-state vowels in the temporal aspects of the discharge patterns of populations of auditory-nerve fibers|journal= The Journal of the Acoustical Society of America|date=November 1979|volume=66|issue=5|pages=1381–1403|doi=10.1121/1.383532|pmid=500976|bibcode=1979ASAJ...66.1381Y}}</ref> ALSR representations of pitch and formant frequencies in complex, non-steady state stimuli were later demonstrated for voiced-pitch,<ref>{{cite journal|last1=Miller|first1=M.I.|last2=Sachs|first2=M.B.|title=Representation of voice pitch in discharge patterns of auditory-nerve fibers|journal=Hearing Research|date=June 1984|volume=14|issue=3|pages=257–279|pmid=6480513|doi=10.1016/0378-5955(84)90054-6|s2cid=4704044}}</ref> and formant representations in consonant-vowel syllables.<ref>{{cite journal|last1=Miller|first1=M.I.|last2=Sachs|first2=M.B.|title=Representation of stop consonants in the discharge patterns of auditory-nerve fibrers|journal= The Journal of the Acoustical Society of America|date=1983|volume=74|issue=2|pages=502–517|doi=10.1121/1.389816|pmid=6619427|bibcode=1983ASAJ...74..502M}}</ref>
Line 143 ⟶ 147:
==== Independent-spike coding ====
The independent-spike coding model of [[neuron]]al firing claims that each individual [[action potential]], or "spike", is independent of each other spike within the [[Action potential|spike train]].<ref
==== Position coding ====
Line 150 ⟶ 154:
[[File:NoisyNeuralResponse.png|thumb|Neural responses are noisy and unreliable.]]
This type of code is used to encode continuous variables such as joint position, eye position, color, or sound frequency. Any individual neuron is too noisy to faithfully encode the variable using rate coding, but an entire population ensures greater fidelity and precision. For a population of unimodal tuning curves, i.e. with a single peak, the precision typically scales linearly with the number of neurons. Hence, for half the precision, half as many neurons are required. In contrast, when the tuning curves have multiple peaks, as in [[grid cell]]s that represent space, the precision of the population can scale exponentially with the number of neurons. This greatly reduces the number of neurons required for the same precision.<ref name="Mat">{{cite journal |vauthors=Mathis A, Herz AV, Stemmler MB |title=Resolution of nested neuronal representations can be exponential in the number of neurons |journal=Phys. Rev. Lett. |volume=109 |issue=1 |
==== Topology of population dynamics ====
[[Dimensionality reduction]] and [[topological data analysis]], have revealed that the population code is constrained to low-dimensional manifolds,<ref>{{cite journal| title=Neural population dynamics during reaching|first1=MM|last1=Churchland|first2=JP|last2=Cunningham |first3=MT|last3=Kaufmann|first4=JD|last4=Foster|first5=P|last5=Nuyujukian|first6=SI|last6=Ryu|first7=KV|last7=Shenoy|journal=Nature|issue=7405|pages=51–56|date=2012|volume=487 |doi=10.1038/nature11129|pmid=22722855 |pmc=3393826 |bibcode=2012Natur.487...51C }}</ref> sometimes also referred to as [[attractors]]. The position along the neural manifold correlates to certain behavioral conditions like head direction neurons in the anterodorsal thalamic nucleus forming a ring structure,<ref>{{cite journal |last1=Chaudhuri |first1=R |last2=Gercek |first2=B |last3=Pandey |first3=B |last4=Peyrache |first4=A |last5=Fiete |first5=I |title=The intrinsic attractor manifold and population dynamics of a canonical cognitive circuit across waking and sleep |journal=Nature Neuroscience |date=2019 |volume=22 |issue=9 |pages=1512–150 |doi=10.1038/s41593-019-0460-x}}</ref> [[grid cells]] encoding spatial position in [[entorhinal cortex]] along the surface of a [[torus]],<ref>{{cite journal |last1=Gardner |first1=RJ |last2=Hermansen |first2=E |last3=Pachitariu |first3=M |last4=Burak |first4=Y |last5=Baas |first5=NA |last6=Dunn |first6=BA |last7=Moser |first7=MB |last8=Moser |first8=EI |title=Toroidal topology of population activity in grid cells |journal=Nature |date=2022 |volume=602 |issue=7895 |pages=123–128 |doi=10.1038/s41586-021-04268-7|pmid=35022611 |hdl=11250/3023140 |hdl-access=free |pmc=8810387 |bibcode=2022Natur.602..123G }}</ref> or [[motor cortex]] neurons encoding hand movements<ref>{{cite journal |last1=Gallego |first1=JA |last2=Perich |first2=MG |last3=Miller |first3=LE |last4=Solla |first4=SA |title=Neural Manifolds for the Control of Movement |journal=Neuron |date=2017 |volume=94 |issue=5 |pages=978–984 |doi=10.1016/j.neuron.2017.05.025|pmid=28595054 |hdl=10261/151381 |hdl-access=free |pmc=6122849 }}</ref> and preparatory activity.<ref>{{cite journal |last1=Churchland |first1=MM |last2=KV |first2=Shenoy |title=Preparatory activity and the expansive null-space |journal=Nature Reviews Neuroscience |date=2024 |volume=25 |issue=4 |pages=213–236 |doi=10.1038/s41583-024-00796-z}}</ref> The low-dimensional manifolds are known to change in a state dependent manner, such as eye closure in the [[visual cortex]],<ref>{{cite journal |last1=Morales-Gregorio |first1=A |last2=Kurth |first2=AC |last3=Ito |first3=J |last4=Kleinjohann |first4=A |last5=Barthelemy |first5=FV |last6=Brochier |first6=T |last7=Gruen |first7=S |last8=van Albada |first8=S |title=Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region |journal=Cell Reports |date=2024 |volume=43 |issue=7 |page=114371 |doi=10.1016/j.celrep.2024.114371|doi-access=free |pmid=38923458 }}</ref> or breathing behavior in the [[ventral respiratory column]].<ref>{{cite journal |last1=Bush |first1=NE |last2=Ramirez |first2=JM |title=ventral respiratory column |journal=Nature Neuroscience |date=2024 |volume=27 |issue=2 |pages=259–271 |doi=10.1038/s41593-023-01520-3|pmid=38182835 |pmc=10849970 }}</ref>
=== Sparse coding ===
The sparse code is when each item is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons. In contrast to sensor-sparse coding, sensor-dense coding implies that all information from possible sensor locations is known.
As a consequence, sparseness may be focused on temporal sparseness ("a relatively small number of time periods are active") or on the sparseness in an activated population of neurons. In this latter case, this may be defined in one time period as the number of activated neurons relative to the total number of neurons in the population. This seems to be a hallmark of neural computations since compared to traditional computers, information is massively distributed across neurons. Sparse coding of natural images produces [[wavelet]]-like oriented filters that resemble the [[receptive field]]s of simple cells in the visual cortex.<ref>{{cite journal | last1 = Olshausen | first1 = Bruno A | last2 = Field | first2 = David J | year = 1996 | title = Emergence of simple-cell receptive field properties by learning a sparse code for natural images | url = http://www.cs.ubc.ca/~little/cpsc425/olshausen_field_nature_1996.pdf | journal = Nature | volume = 381 | issue = 6583 | pages = 607–609 | doi = 10.1038/381607a0 | pmid = 8637596 | bibcode = 1996Natur.381..607O | s2cid = 4358477 | access-date = 2016-03-29 | archive-url = https://web.archive.org/web/20151123113216/http://www.cs.ubc.ca/~little/cpsc425/olshausen_field_nature_1996.pdf | archive-date = 2015-11-23 | url-status = dead }}</ref> The capacity of sparse codes may be increased by simultaneous use of temporal coding, as found in the locust olfactory system.<ref>{{cite journal|last1=Gupta|first1=N|last2=Stopfer|first2=M|title=A temporal channel for information in sparse sensory coding.|journal=Current Biology|date=6 October 2014|volume=24|issue=19|pages=2247–56|pmid=25264257|doi=10.1016/j.cub.2014.08.021|pmc=4189991|bibcode=2014CBio...24.2247G}}</ref>
Given a potentially large set of input patterns, sparse coding algorithms (e.g. [[Autoencoder#Sparse autoencoder (SAE)|sparse autoencoder]]) attempt to automatically find a small number of representative patterns which, when combined in the right proportions, reproduce the original input patterns. The sparse coding for the input then consists of those representative patterns. For example, the very large set of English sentences can be encoded by a small number of symbols (i.e. letters, numbers, punctuation, and spaces) combined in a particular order for a particular sentence, and so a sparse coding for English would be those symbols.
Line 162 ⟶ 169:
Most models of sparse coding are based on the linear generative model.<ref name=Rehn>{{cite journal|first1=Martin|last1=Rehn|first2=Friedrich T.|last2=Sommer|title=A network that uses few active neurones to code visual input predicts the diverse shapes of cortical receptive fields|journal=Journal of Computational Neuroscience|year=2007|volume=22|issue=2|pages=135–146|doi=10.1007/s10827-006-0003-9|pmid=17053994|s2cid=294586|url=http://redwood.berkeley.edu/fsommer/papers/rehnsommer07jcns.pdf}}</ref> In this model, the symbols are combined in a [[Linear combination|linear fashion]] to approximate the input.
More formally, given a k-dimensional set of real-numbered input vectors <math>\vec{\xi }\in \mathbb{R}^{k}</math>, the goal of sparse coding is to determine n k-dimensional [[Basis (linear algebra)|basis vectors]] <math>\vec{b_1}, \ldots, \vec{b_n} \in \mathbb{R}^{k}</math>, corresponding to neuronal receptive fields, along with a [[Sparse vector|sparse]] n-dimensional vector of weights or coefficients <math>\vec{s} \in \mathbb{R}^{n}</math> for each input vector, so that a linear combination of the basis vectors with proportions given by the coefficients results in a close approximation to the input vector: <math>\vec{\xi} \approx \sum_{j=1}^{n} s_{j}\vec{b}_{j}</math>.<ref name=Lee>{{cite journal|last1=Lee|first1=Honglak|last2=Battle|first2=Alexis|last3=Raina|first3=Rajat|last4=Ng|first4=Andrew Y.|title=Efficient sparse coding algorithms|journal=Advances in Neural Information Processing Systems|year=2006|url=https://ai.stanford.edu/~hllee/nips06-sparsecoding.pdf}}</ref>
The codings generated by algorithms implementing a linear generative model can be classified into codings with ''soft sparseness'' and those with ''hard sparseness''.<ref name=Rehn/> These refer to the distribution of basis vector coefficients for typical inputs. A coding with soft sparseness has a smooth [[Normal distribution|Gaussian]]-like distribution, but peakier than Gaussian, with many zero values, some small absolute values, fewer larger absolute values, and very few very large absolute values. Thus, many of the basis vectors are active. Hard sparseness, on the other hand, indicates that there are many zero values, ''no'' or ''hardly any'' small absolute values, fewer larger absolute values, and very few very large absolute values, and thus few of the basis vectors are active. This is appealing from a metabolic perspective: less energy is used when fewer neurons are firing.<ref name=Rehn/>
Another measure of coding is whether it is ''critically complete'' or ''overcomplete''. If the number of basis vectors n is equal to the dimensionality k of the input set, the coding is said to be critically complete. In this case, smooth changes in the input vector result in abrupt changes in the coefficients, and the coding is not able to gracefully handle small scalings, small translations, or noise in the inputs. If, however, the number of basis vectors is larger than the dimensionality of the input set, the coding is ''overcomplete''. Overcomplete codings smoothly interpolate between input vectors and are robust under input noise.<ref name=Olshausen>{{cite journal|first1=Bruno A.|last1=Olshausen|first2=David J.|last2=Field|title=Sparse Coding with an Overcomplete Basis Set: A Strategy Employed by V1?|journal=Vision Research|year=1997|volume=37|number=23|pages=3311–3325
Other models are based on [[matching pursuit]], a [[sparse approximation]] algorithm which finds the "best matching" projections of multidimensional data, and [[Sparse dictionary learning|dictionary learning]], a representation learning method which aims to find a [[sparse matrix]] representation of the input data in the form of a linear combination of basic elements as well as those basic elements themselves.<ref>{{Cite journal|last1=Zhang|first1=Zhifeng|last2=Mallat|first2=Stephane G.|last3=Davis|first3=Geoffrey M.|date=July 1994|title=Adaptive time-frequency decompositions|journal=Optical Engineering|volume=33|issue=7|pages=2183–2192|doi=10.1117/12.173207|issn=1560-2303|bibcode=1994OptEn..33.2183D}}</ref><ref>{{Cite book|last1=Pati|first1=Y. C.|last2=Rezaiifar|first2=R.|last3=Krishnaprasad|first3=P. S.|title=Proceedings of 27th Asilomar Conference on Signals, Systems and Computers |chapter=Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition |date=November 1993|pages=40–44 vol.1|doi=10.1109/ACSSC.1993.342465|isbn=978-0-8186-4120-6|citeseerx=10.1.1.348.5735|s2cid=16513805}}</ref><ref>{{Cite journal|date=2009-05-01|title=CoSaMP: Iterative signal recovery from incomplete and inaccurate samples|journal=Applied and Computational Harmonic Analysis|volume=26|issue=3|pages=301–321|doi=10.1016/j.acha.2008.07.002|issn=1063-5203|last1=Needell|first1=D.|last2=Tropp|first2=J.A.|arxiv=0803.2392|s2cid=1642637 }}</ref>
Line 176 ⟶ 183:
In the ''[[Drosophila]]'' [[olfactory system]], sparse odor coding by the [[Kenyon cell]]s of the [[Mushroom bodies|mushroom body]] is thought to generate a large number of precisely addressable locations for the storage of odor-specific memories.<ref>A sparse memory is a precise memory. Oxford Science blog. 28 Feb 2014. http://www.ox.ac.uk/news/science-blog/sparse-memory-precise-memory</ref> Sparseness is controlled by a negative feedback circuit between Kenyon cells and [[GABAergic]] anterior paired lateral (APL) neurons. Systematic activation and blockade of each leg of this feedback circuit shows that Kenyon cells activate APL neurons and APL neurons inhibit Kenyon cells. Disrupting the Kenyon cell–APL feedback loop decreases the sparseness of Kenyon cell odor responses, increases inter-odor correlations, and prevents flies from learning to discriminate similar, but not dissimilar, odors. These results suggest that feedback inhibition suppresses Kenyon cell activity to maintain sparse, decorrelated odor coding and thus the odor-specificity of memories.<ref>Lin, Andrew C., et al. "[https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4000970/ Sparse, decorrelated odor coding in the mushroom body enhances learned odor discrimination]." Nature Neuroscience 17.4 (2014): 559-568.</ref>
== See also ==
Line 211 ⟶ 201:
* [[Sparse distributed memory]]
* [[Vector quantization]]
* [[Representational drift]]
{{div col end}}
Line 225 ⟶ 216:
[[Category:Neural coding| ]]
[[Category:Computational neuroscience]]
[[Category:Neural
| |||