Sparse distributed memory

Sparse Distributed Memory is a mathematical model of human long-term memory first introduced by Pentti Kanerva in 1988. It is used for storing and retrieving large amounts of information without focusing on the accuracy of the information. It uses patterns to serve as memory addresses, where information is retrieved based on similarities between addresses. Memory addresses are all in a list even if they are not related, but are only retrieved based on similar content between themselves.

Formula

The general formula is ${\displaystyle 2^{n}}$  where n is the number of dimensions of the space, and ${\displaystyle 2^{n}}$  is the number of feasible memory items.

Definition

Sparse Distributed Memory is a mathematical representation of human memory, and since human memory is complex, Sparse Distributed Memory uses high-dimensional space.^[1] It utilizes the Hamming distance to measure mismatched bits and read back data between the original write address and one near it.^[2] Human memory has a tendency to congregate memories based on similarities between them(although they may not be related), such as firetrucks are red and apples are red.^[3]

Example

Sparse Distributed Memory is based off of pulling in patterns between different addresses.

Imagine each line as a different memory address, an example from Manerva's book:

"Why are fire engines painted red?

Firemen's suspenders are red, too.

Two and two are four.

Four times three is twelve.

Twelve inches in a foot.

A foot is a ruler.

Queen Mary is a ruler.

Queen Mary sailed the sea.

The sea has sharks.

Sharks have fins.

The Russians conquered the Finns.

The Russians' color is red.

Fire engines are always rushin'.

So that's why they're painted red!"^[4]

Uses

At the University of Memphis, Uma Ramamurthy, Sidney K. D’Mello, and Stan Franklin created a modified version of the Sparse Distributed Memory system that mathematically represents "realizing forgetting." It uses a decay equation to better show interference in data. The Sparse Distributed Memory system distributes each pattern into approximately one hundredth of the locations, so interference can have detrimental results. ^[5]

Two possible examples of decay from this modified Sparse Distributed Memory are presented

Exponential Decay Mechanism: ${\displaystyle \!f(x)=1+e^{-ax}}$ 

 

The exponential decay function

Negated-Translated Sigmoid Decay Mechanism: ${\displaystyle f(x)=1-[{\frac {1}{1+e^{-a(x-c)}}}]}$ 

In the exponential decay function, it approaches zero as x increases, and a is a constant(usually between 3-9) and c is a counter. For the negated-translated sigmoid function, the decay is similar when a is greater than 4.^[5]

References

1. Pentti Kanerva (1993). "Sparse Distributed Memory and Related Models". Pennsylvania State University. Retrieved 3 November 2011.
2. "Sparse Distributed Memory: Principles and Operation" (PDF). Stanford University. December 1989. Retrieved 1 November 2011. {{cite web}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
3. C. George Boeree (2002). "General Psychology". Shippensburg University.
4. Kanerva, Pentti (1988). Sparse Distributed Memory. The MIT Press. ISBN 978-0262111324.
5. "Realizing Forgetting in a Modified Sparse Distributed Memory System". Computer Science Department & The Institute for Intelligent Systems. The University of Memphis. p. 1992. Archived from the original (PDF) on 2006. Retrieved 1 November 2011. {{cite web}}: Check date values in: |archivedate= (help); More than one of |pages= and |page= specified (help); Unknown parameter |coauthors= ignored (|author= suggested) (help)