Download Quality – An Inherent Aspect of Agile Software Development

Hierarchical Temporal Memory
“The Application of Hierarchical Temporal
Memory to the ECG Waveforms”
May 6, 2011
Version 3.0; 05/06/2011
John M. Casarella
Proceedings of Student/Faculty Research Day
Ivan G. Seidenberg School of CSIS, Pace University
Introduction


Human Vision is a Key component of Human Learning
Learning is a process that alters the synaptic strengths
between neurons



This allows for pattern memories to form
Now, inference can take place
What appears to be a task so easily accomplished by
humans, invariant visual pattern recognition, still eludes
computational models of human intelligence or what we
refer to as Artificial Intelligence
Learning and Memory





Humans learn and form memories
Learning alters the physical structure of the brain, exhibiting
plasticity in its ability to change and adapt
Memory is not localized, but distributed across the neocortex
Learning is associated with the construction of cell
assemblies related to the formation of pattern association brain plasticity
Donald Hebb, Hebbian Learning




Briefly, if a neuron receives an input from another neuron and if
both are highly active (mathematically have the same sign), the
weight between the neurons should be strengthened.
Learning is the modification of these connections
Highly influential in the concepts behind the HTM Model
Pattern recognition learned through by changing the strength of
the connections between neurons
Intelligence and Machine Intelligence

What is Intelligence?



Machine Intelligence or AI





A uniquely human quality?
The ability to adapt to a changing environment or the ability to
change our environment for survival
Define – what is it?
Objectives - Create machines to do something which would
require intelligence if done by a human or to solve the problem
of how to solve the problem
Varieties – Expert Systems, Genetic Algorithms, Perceptrons,
Neural Nets
Key People – Shannon, von Neumann, McCarthy, Rosenblatt,
Minsky (MIT AI Lab) and Zadeh (Fuzzy Logic)
Alan Turing
Alan Turing





“Can machines Think?” and The Turing Test
Provided the foundation for Connectionism
Model digital computer like child’s mind, then “educate” to
obtain “adult”
“Unorganized Machines” : A network of neuron-like
Boolean elements randomly connected together
Proposed machines should be able to “learn by experience”
Neurons
Unorganized Machines
Neural Nets



McCulloch and Pitts

Model of Neurons of the brain

Cornerstone of neural computing and neural networks

Boolean nets of simple two-state ‘neurons’ and the concept of
‘threshold’

No mechanism for learning
Hebb - Pattern recognition learned through by changing the strength of the
connections between neurons
Rosenblatt

Perceptron Model - permitted mathematical analysis of neural
networks

Based on McCulloch and Pitts

Linear combiner followed by a hard limiter

Activation and weight training

Linear Separation - No XOR
Neuroscience







The neocortex in humans is associated with such tasks as visual
pattern recognition, understanding language and object recognition
and manipulation and inference
These are just some of the basic tasks which we engage in on a daily
basis, allowing us to function in our world, yet with all of the
advancements in computer hardware, software and computing in
general, computers are still incapable of cognitive function.
Locus of: perception, language, planned behavior, declarative
memory, imagination, planning
Invariant Pattern Recognition
Repetitive structure
Unique columnar six layer structure - Montcastle
Hierarchy of cortical regions
Electro-cardiology




Electrocardiography is the process of recording and interpreting
the electrical activity of the action of the heart, providing
physicians with the ability to study the heart's normal behavior
and allow for the diagnosing of abnormalities
The standard ECG in a clinical setting has 12 leads
Represents different phases of the cardiac cycle
A normal ECG shows a sequence of at least three waves
labeled P, QRS, and T. The baseline voltage of the
electrocardiogram is known as the isoelectric line.
ECG Computational Methods




Currently there are three computerized interpretations techniques:
(1) statistical methods (2) deterministic method and (3) the use of
artificial neural networks, with the deterministic approach the most
common, which mimics the diagnostic criteria tree used by
cardiologists
In the multitude of papers written and reviewed concerning the
computational evaluation of ECG waveforms, one of the common
methods is the application of traditional artificial neural networks.
Artificial Neural Networks have been used to correlate ECG signals
to specific pathologies, in the classification of the individual ECG
beat waveforms, for the determination of myocardial infarction to
identify but a few of the of most significant applications
The majority of these neural networks were designed using welldocumented methodologies and mathematical models, but
included the need to pre-process the data and to identify a set of
extracted features
Hierarchical Temporal Memory
The Hierarchical Temporal Memory (HTM) model is an attempt to
replicate the structural and algorithmic properties of the neocortex based
on human neurological function, specifically identified in its pattern of
learning and its ability to form pattern memories for pattern classification


The HTM model really should be viewed as a memory prediction model
The HTM can also be compared to traditional Artificial Neural Networks
(ANN) but how they learn, how they resolve the differences between
different patterns and the computational algorithms behind the HTM
model, removes any connection to ANNs

Hierarchical Temporal Memory
Overview


Each node performs similar
algorithm
Each node learns




“Names” of groups passed up



Hierarchy of memory nodes

1) Common spatial patterns
2) Common sequences of spatial
patterns
(use time to form groups of patterns
with a common cause)
- Many to one mapping, bottom to
top
- Stable patterns at top of hierarchy
Modeled as an extension of
Bayesian network with belief
propagation
Creates a hierarchical model (time
and space) of the world
Hierarchical Temporal Memory




(A) An initial node that has
not started its learning
process.
(B) The spatial pooler of the
node is in its learning phase
and has formed 2
quantization enters
(C) the spatial pooler has
finished its learning process
and is in the inference stage.
The temporal pooler is
receiving inputs and learning
the time-adjacency matrix.
(D) shows a fully learned
node where both the spatial
pooler and temporal pooler
have finished their learning
processes
Hierarchical Temporal Memory
Structure of an HTM
network for learning
invariant representations for
the binary images world.
 This network is organized
in 3 levels. Input is fed in at
the bottom level. Nodes are
shown as squares.
 The top level of the
network has one node, the
middle level has 16 nodes
and the bottom level has 64
nodes.
 The input image is of size
32 pixels by 32 pixels.
This image is divided into
adjoining patches of 4 pixels
by 4 pixels as shown.
Each bottom-level node’s
input corresponds to one
such 4x4 patch.

Hierarchical Temporal Memory
Level 2 Temporal Pooler
Level 2 Spatial Pooler
Level 1 Temporal Pooler
Level 1 Spatial Pooler
Hierarchical Temporal Memory
The fully learn ed node has 12 quantiz ation centers within its spatial po oler and 4 tempo ral
groups within its tempo ral po oler . The quantiz ation cente rs are shown as 4x4 pix el patc hes.
The tempo ral groups are shown in ter ms of their quantiz ation centers. The input to th e node
is a 4x 4 pix el patch .
Hierarchical Temporal Memory
QuickTime™ and a
decompressor
are needed to see this picture.
This figure illustrates how nodes operate in a hierarchy; we show a two-level network and its associated inputs for three
time steps. This network is constructed for illustrative purposes and is not the result of a real learning process. The outputs
of the nodes are represented using an array of rectangles. The number of rectangles in the array corresponds to the length
of the output vector. Filled rectangles represent ‘1’s and empty rectangles represent ‘0’s.
Hierarchical Temporal Memory
Research Focus



Application of the HTM model, once correctly designed
and configured, will provide a greater success rate in
the classification of complex waveforms
Abandon traditional data pre-processing and feature
extraction, applying a visual process using the actual
images
Task Description



Create an image dataset of each waveform group for
classification
Determine, through organized experiments, an optimized
HTM
Apply optimized HTM to the classification of waveforms
using images, devoid of any pre-processing or feature
extraction
HTM Design and Implementation




No set methodologies or mathematical correlations to
determine HTM structure
Key optimization parameters addressed by observation
The key parameters identified included, but were not
limited to, the number of iterations at each level, the
Euclidian distance during vector formation, the number
of images applied during learning, the maximum number
of coincidences (pattern memories) allowed and the
maximum number of groups allowed for pattern
grouping.
De facto recommendation:
 Initial settings should be based on educated guesses
 Parameters and framework to be refined by trial and
error
ECG Data






The Limb Lead II is commonly used in three lead as well as
twelve lead ECG recordings.
This lead features a prominent and upright QRS complex,
facilitating the choice of this lead.
Of the beat classifications, Normal Sinus, LBBB and RBBB beats
were used in this study.
The total number of beats used was 1160 per data set, which
consisted of learning and unknown subsets.
Each learning set was comprised on 300 images, 100 from each
group classification.
Each unknown data set contained 860 images comprising of 300
LBBB images, 360 normal sinus images and 200 RBBB images.
ECG Waveform Images
Normal Sinus
Right Bundle
Branch Block
Left Bundle
Branch Block
Individual ECG Beat Images
Left Bundle Branch Block
Normal Sinus
Right Bundle Branch Block
All images were sized to 96 x 120 pixels
Results






For the classification of the individual ECG beats, with very few exceptions, the
classification rates exceeded 98 percent and approached 100 percent for many of
the individual models in combination with the various data sets.
Once a well defined model is established, the dataset applied during learning
always returned a classification rate of 100 percent.
For the classification of the same grouped heartbeats as used in this study,
Thiagarajan obtained an average classification percentage of 91 percent for the
training set and 79 percent for the unknown set using a 1 to 1 training to
unknown ratio, each containing approximately 700 entities.
The experimental results indicate the number of images and the number of
iterations applied during learning directly influenced the performance of an HTM
model in its ability to correctly classify and differentiate the individual ECG beat
images.
When all unknown images were presented to a HTM model after learning en
masse, it was able to correctly differentiate and classify the vast majority, if not
all, of heartbeats to the correct waveform class.
The addition of background noise to the unknown image set did not have a
significant effect on the ability to classify the unknown images. Even with a high
level of noise added, the model correctly classified an average of 99.4 percent of
the images.
HTM Model Results
Mode l
p1215_4000
p1215_4000a
p1215_4000b
p1215_4000c
p1215_4000e
Averages
u n k n 1u n k n 2u n k n 3Avg
100.0 100.0 99.7 99.9
99.2 100.0 98.7 99.3
99.7 100.0 100.0 99.9
98.6 100.0 99.0 99.2
99.7 100.0 98.7 99.5
99.4 100.0 99.2 99.6
Mode l (Mas k ) u n k n 1u n k n 2u n k n 3Avg
p1215_4317a2
99.7
p1215_4317a3 100.0
p1215_4317a3b 99.4
p1215_4317a3c 98.6
p1215_4317a3d 100.0
Averages
99.5
Mode l
p1215_3317a
q1215_3317a
r1215_3317a
s1215_3317a
t 1215_3317a
u1215_3317a
v1215_3317a
x1215_3317a
Averages
100.0 99.7 99.8
100.0 99.3 99.8
100.0 99.7 99.7
100.0 100.0 99.5
100.0 99.7 99.9
100.0 99.7 99.7
u n k n 1u n k n 2u n k n 3Avg
100
100
99.7 99.9
99.3
100
97 98.8
100
100
99.2 99.7
100
100
99.7 99.9
99.2
100
98.3 99.2
98.6 97.5 95.7 97.3
99.2 99.5 99.7 99.5
99.4 99.5
100 99.6
99.5 99.6 98.7 99.2
Unkn1 = Normal Sinus
Unkn2 = Right Bundle Branch Block
Unkn3 = Left Bundle Branch Block
Results by HTM Model
Percent Classified by HTM Model
100
99.8
99.6
Percent Classified
99.4
99.2
99
98.8
98.6
98.4
98.2
98
1215_4317a7a
1215_3317a
Model Designation
1215_4000k2
Results by Dataset and Model
Percent Classified by Dataset
100.0
99.5
99.0
Percent Classified
98.5
98.0
97.5
97.0
96.5
96.0
95.5
95.0
p1215_3317a q1215_3317a
r1215_3317a
s1215_3317a
t1215_3317a u1215_3317a v1215_3317a x1215_3317a
HTM Model Applied
Results by Dataset
Percent Classification by Dataset
100
Percent Classified
99.5
99
98.5
98
97.5
97
p1215
q1215
r1215
s1215
t1215
Dataset
u1215
v1215
x1215
Effects of Occlusion on Classification
Clean
Occlusion0
Occlusion1
Occlusion2
Percent Classified with Occlusion
100.0
95.0
Category
Clean
Occlusion0
Occlusion1
Occlusion2
Percent Classified
90.0
85.0
80.0
75.0
70.0
65.0
60.0
Clean
Occlusion0
Occlusion1
Occlusion
Occlusion2
4317a 3317a
99.9
99.2
99.7
98
92.1
81.8
78.7
67.3
HTM Learning Curve
Observations
The formulation of a mathematical model to determine the number of groups,
coincidences per group, the number of iterations required to create a sufficient
number of pattern memories or any other parameter, was met with limited success.

It was determined the level of complexity of the ECG images used during learning
and the relationship (Euclidian proximity) between the graph (which could be
classified as noise) and the ECG waveform produced pattern memories of greater
complexity, diversity and Euclidian differential than encountered in previous
examples.

It was the visual inspection of the pattern memories formed, the grouping of the
patterns, the groups with many associated coincidences and the number single
coincidence groups, which was necessary to validate the results.

Creation of an Optimal image learning set can influence the learning ability of the
model.

Sufficient number of pattern memories formed during learning is necessary, which
is directly influenced by the number of iterations at each level.

Observations
There is a need for the Optimal grouping of pattern memories, based on the
number of patterns per group, the number of groups and the number of onepattern groups.

There was a distinct need to determine how often the learning image set needed
to be sampled to insure a sufficient number of coincidences or pattern memories
are created to insure the correct classification of unknown images when presented
to the model.

This model is a departure from other pattern recognition models , yet the ability of this model to lear n
and classify unkn owns is closer to that of a human, but why? The images presented to an HT M during
the learning phas e are patterns of heartbeats, each pattern having been assigned to a specific
classification. The model begins the learning process by first creating pattern memorie s for the various
unique visual aspects o f the image, breaking down the image into components before they are grouped
spatially and temporally . From the perspectiv e of human vision, as an individual scans a pattern (ima ge
saccades), the patternΥs unique components are identif ied individually and then reassembledΣ to form
the complet e image, creating the memory. When a previously unseen imag e or patter n is presented, it
too is broken d own into its components to
matchedΣ to the pattern memories to determine
identification, classification or inf erence. As the different components of the pattern memories ar e
formed for the ECG waveforms, the patterns are grouped by association , and then grouped
categorically , by associating the pattern memories to specific heartbeat pathology. Many of the images
share common artifacts, such as the underlying grid present in an ECG printout. These visual artifacts
thus become pattern memorie s across all categories , leaving the differences in the position and shape of
the waveform for differentiating the pattern m emorie s for the specific ECG waveforms. The
coincidences o r pattern memorie s formed durin g the learning phase are used to reconstruct the complete
image when applied to an unknown for classification. It does not matter if it is a complet e image or its
various c omponents .
Conclusions and Inference
The primary objective of the research presented herein was to
determine if the Hierarchical Temporal Memory model when applied
to the classification (pattern recognition) of ECG waveform patterns
(individual ECG beats), would return performance results equal to or
greater than the application of a neural network.

The results obtained provided strong evidence the Hierarchical
Temporal Memory Model provides a unique and improved approach
for the classification of ECG waveforms, with the potential of
providing a classification accuracy of 100 percent.

Additional experimental results indicate the number of images and
the number of iterations applied during learning directly influences
the performance of an HTM model in its ability to correctly classify
and differentiate the individual ECG beat images, mimicking human
learning.

References
[1] Turing, A. M. “Computing Machinery and Intelligence”. Mind, New Series, Vol. 59, No. 236. (Oct., 1950). pp 443 – 460.
[2] B. J. Copeland, "The Essential Turing," Oxford, Great Britain: Oxford University Press, 2004.
[3] B. Scassellati, "How social robots will help us to diagnose, treat, and understand autism," in 12th International Symposium of Robotics Research
San Francisco, CA. 2005.
[4] D. O. Hebb, The Organization of Behavior. New York: John Wiley & Sons, 1949.
[5] E. Besterman, and Richard Creese, "Waller - Pioneer of Electrocardiography," British Heart Journal, vol. 42, pp. 61-64, 1979
[6] W. Einthoven, G. Fahr, and A. de Waart, "On the Direction and Manifest Size of the Variations of Potential in Human Heart and on the Influence of
the Position of the Heart on the Form of the Electrocardiogram," American Heart Journal, vol. 40, pp. 163-211, 1950.
[7] Pipberger, H.V., R.J. Arms and F. W. Stallman, “Automatic Screening of Normal and Abnormal Electrocardiograms by Means of a Digital
Computer,” Proc Soc Exp Biol Med, vol. 106, pp. 130-132, 1961.
[8] L. Edenbrandt, Brian Devine, and Peter W. Macfarlane, "Neural Networks for Classification of ECG ST-T Segments," Journal of
Electrocardiology, vol. 25, pp. 167 - 173, 1992.
[9] S. Thiagarajan, "Bottom-up Design of Artificial Neural Network for Single Lead Electrocardiography Beat and Rhythm Classification," Doctoral
Dissertation; Baton Rouge: College of Engineering and Science, Louisiana Tech University, 2000
[10] A. A. Sleiman, "Simplified ECG Waveforms Analysis Utilizing Matlab-Based Artificial Neural Network (ANN) Tools," in Department of
Electrical Engineering. vol. MSc Long Beach: California State University, Long Beach, 2007.
[11] Computational Models of the Neocortex. http://www.cs.brown.edu/people/tld/projects/cortex/
[12] George, Dileep and Jaros, Bobby. The HTM Learning Algorithms. Numenta, Inc. March 1, 2007.
[13] Hawkins, Jeff. Learn like a Human. http://spectrum.ieee.org/apr07/4982.
[14] Hawkins, Jeff and Dileep, George. Hierarchical Temporal Memory, Concepts, Theory, and Terminology. Numenta, Inc. 2008.
[15] F. Rosenblatt, "The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain," Psychological Review, vol. 65, pp.
386 - 408, 1958
[16] F. Rosenblatt, Principles of Neurodynamics. Washington, DC: Spartan Books, 1962.
[17] D. E. Rumelhart, and McClelland, J.L. editors, "Parallel Distributed Processing: Explorations in the Microstructures of Cognition." vol. 2 Psychological and Biological Models Cambridge, MA: MIT Press, 1986.
[18] D. E. Rumelhart, and McClelland, J.L. editors, "Parallel Distributed Processing: Explorations in the Microstructures of Cognition." vol. 1 Foundations Cambridge, MA: MIT Press, 1986
[19] M. L. Minsky, and Papert, Seymour S., Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.
References
[20] M. Negnevitsky, Artificial Intelligence, A Guide to Intelligent Systems, 2nd ed. Essex, England: Addison Wesley (Person Education Limited), 2005.
[21] PhysioNet, "The research resource for complex physiologic signals," http://www.physionet.org/.
[22] A. L. Goldberger, L.A.N. Amaral, L. Glass, J.M. Hausdorff. P.Ch. Ivanov, R.G. Mark, J.E. Mietus, G.B. Moody, C.K. Peng, and H.E. Stanley,
"PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals," Circulation, vol. 101,
pp. 215-220, June 13 2000.
[23] P. Laguna, RG Mark, A Goldberg, and GB Moody, "A Database for Evaluation of Algorithms for Measurement of QT and Other Waveform
Intervals in the ECG," Computers in Cardiology, vol. 24, pp. 673-676, 1997.
[24] T.-W. Shen, "Biometric Identity Verification Based on Electrocardiogram (ECG)," in Biomedical Engineering. vol. PhD Madison: University of
Wisconsin - Madison, 2005, p. 178.
[25] C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006.
[26] J. Copeland, and Diane Proudfoot, "Alan Turing's Forgotten Ideas in Computer Science," Scientific American, pp. 99 - 103, 1999.
[27] Schey, N. C., Song Identification Using the Numenta Platform for Intelligent Computing. B.S. Thesis, The Ohio State University, 2008.
[28] Bonhoff, G. M., Using Hierarchical Temporal Memory for Detecting Anomalous Network Activity. Ph.D. Dissertation, Air Force Institute of
Technology, 2008.
[29] Perea, A. J., Merono, J. E., and Aguilera, M. J., “Application of Numenta Hierarchical Temporal Memory for land-use classification,” South
African Journal of Science, vol. 105, no. 9-10, pp. 370–375, 2009.
[30] Squire, Larry R. (Editor)., Encyclopedia of Learning and Memory. New York, NY. Macmillian Publishing Company, 1992.
[31] Q. Xue, Y.H. Hu, and W.J. Tompkins, "Neural-network-based adaptive matched filtering for QRS detection," IEEE Transactions on Biomedical
Engineering, vol. 39, pp. 317-329, 1992.
[32] H. Holst, Mattias Ohlsson, Carsten Peterson, and Lars Edenbrandt, "A confident decision support system for interpreting electrocardiograms,"
Clinical Physiology, vol. 19, pp. 410 - 418, 1999.
[33] B. Heden, Mattias Ohlsson, Ralf Rittner, Olle Pahlm, Wesley K. Haisty, Carsten Petterson and Lars Edenbrandt, "Agreement Between Artificial
Neural Networks and Experienced Electrocardiographer on Electrocardiographic Diagnosis of Healed Myocardial Infarction," JACC, vol. 28 No.
4, pp. 1012 - 6, October 1996.
[34] Josh Hartung, Jay McCormack and Frank Jacobus, “Support for the use of Hierarchical Temporal Memory Systems in Automated Design
Evaluation: A First Experiment,” Proc. ASME 2009 International Design Eng. Tech. Conf & Computers and Information in Eng. Conference. Aug
30 – Sept 2, 2009. San Diego, CA, USA. Pp 1 – 10.