Hierarchical Temporal Memory (HTM) 🧠 — PART 1

An underrated gem inspired by the functioning of Neocortex region of our brain from the company called Numenta.

Before delving into how the HTM technique works, We first need to understand about the Neocortex region of our brain.

Human Brain is the most sophisticated intelligent machine ever known starting from Big Bang Period of our universe. Though human evolution has matured enough, there are still some enigma around the functioning of our brain which makes it incredible. We can safely say that brain is the real king maker.

Regions of human brain

Brain is divided into three main regions:

1. Cerebrum — Largest part of our brain. It helps us in our day to day life from thoughts to actions. Cerebrum is responsible for managing and processing the inputs that our five senses takes in such as smell, sound, touch etc., It is also responsible for short term memory. The main important function of the cerebrum is Learning, Logic, Reasoning. They help ourselves with learning new skills, solve puzzles for a problem etc., This is the most important inspiration that lead to HTM algorithm.
2. Cerebellum — It is responsible for processing and regulating the signals from brain and body and it is responsible for coordinating actions like walking etc.,
3. Medulla Oblongata — It is a tiny part of our brain and it plays a vital role in coordinating the reflexes. It also helps to control heartbeat, blood pressure and respiration.

Neurons are the fundamental units of the brain also called as brain cell. Neurons are connected together with other neurons through synapse. A neuron may be connected to many neurons and when multiple neurons fire or send information to that neuron and the magnitude of those signals combined exceeds the threshold also called as EPSP (Excitatory Post Synaptic Potential), we call that as Neuronal Spike.

Animated Neuronal spikes

NEOCORTEX — A region in our brain that encompasses nearly half of the volume of our brain and most of the cerebral cortex region. This is very special because it takes care of the reasoning, attention, perception and episodic memory. The important fact about Neocortex is, if it is hurt, it can lead to decrease in cognitive capabilities of a person. Almost, everyone would have felt hungry, when you smell your favorite dish (for me., my Amma’s chicken gravy 😊) also because of Neocortex. Neocortex is responsible for semantic memory (Memory that degrades while aging) and episodic memory (The information that you recollect from the past).

Cross Section view of human brain

Neocortex is divided into four regions:

1. frontal lobe — Decision making, reinforcement learning
2. parietal lobe — Cognition, Spatial awareness
3. occipital lobe — Visual perception
4. temporal lobe — Emotions, Memory

Different regions of Neocortex

Apparently, from the image above, we can infer that 90% of the cerebral cortex is Neocortex. The one astonishing factor that separates human brain from all those fancy complex multi-layered deep learning systems is these systems are not capable of performing the tasks like reasoning and abstraction. Most of the systems are not flexible. The fact is that, even though they are termed as ANN (Artificial Neural Networks) that is passing of information from one layer of neurons to the other layer, most of the time, they don’t mimic the way the human brain works.

Also, we need to know something called as Neuroplasticity. It is the ability of our brain to continuously learn from experience and be flexible (structural Neuroplasticity) and the brain’s ability to move functions from a damaged area of the brain to other undamaged areas (functional Neuroplasticity).

Now, we shall delve deep into HTM:

H — Hierarchical (Higher levels of Abstraction)

T — Temporal (Change over time)

M — Memory (SDR (Sparse Distributed Representation))

HTM is based on the principles of “ Thousands Brain Theory” . HTM model is much more complex than RNNs for sequential learning. It does not contain backpropagation and depends on “Hebbian learning”.

Hebbian Learning — Neurons wire together if they fire together. Any two neurons or systems of neurons that are repeatedly active at the same time will tend to become ‘associated,’ so that activity in one facilitates activity in the other. For example, when you repeat something over and over again, such as when learning how to cook something, the connections between the activated brain cells are strengthened. Much like the common saying that “practice makes perfect”, these neural connections grow to become faster, stronger, and more efficient over time. Let’s say if neuron A persistently fires neuron B, then the connections between both of them becomes stronger.

Hebbian learning requires no other information than the activities, such as labels or error signals: it is an unsupervised learning method. Hebbian learning is not a concrete learning rule, it is a postulate on the fundamental principle of biological learning. Because of its unsupervised nature, it will rather learn frequent properties of the input statistics than task-specific properties. It is also called a correlation-based learning rule.

HTM’s are used in Pattern Recognition, Anomaly Detection and Prediction. HTM is unique in its ability to learn continuously and predict future data points based on the learned patterns.

HTM uses a combination of Temporal memory and Spatial memory. Spatial memory is learning the patterns in the data and the later is spatial learning over time. HTM are best suited for time series analysis. HTM is unique in its ability to learn continuously and predict future data points based on the learned patterns.

This is a two part series wherein we discussed about neuroscience and HTM introduction. In the next part, we will go deep into architecture of HTM with implementation.

Thanks for reading!