Download Llenceu aquesta pàgina i substituïu-la per aquella que us faciliti... tat d’Informació i Projecció Institucionals (UIPI), disponible al formulari

Llenceu aquesta pàgina i substituïu-la per aquella que us faciliti la Unitat d’Informació i Projecció Institucionals (UIPI), disponible al formulari
següent
http://www.upf.edu/uii/sgrafics/formulari_tesi.htm
The hippocampus code
A computational study of the structure and
function of the hippocampus.
César Rennó Costa
Tesi Doctoral UPF / 2012
Supervisada pel
Dr. Paul Verschure
Synthetic, Perceptive, Emotive and Cognitive Systems (SPECS) and
Institució Catalana de Recerca i Estudis Avançats (ICREA).
...
By César Rennó Costa, 2012
Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported
You are free to Share – to copy, distribute and transmit the work Under the
following conditions:
• Attribution – You must attribute the work in the manner specified
by the author or licensor (but not in any way that suggests that they
endorse you or your use of the work).
• Noncommercial – You may not use this work for commercial purposes.
• No Derivative Works – You may not alter, transform, or build upon
this work.
With the understanding that:
Waiver – Any of the above conditions can be waived if you get permission
from the copyright holder.
Public Domain – Where the work or any of its elements is in the public
domain under applicable law, that status is in no way affected by the
license.
Other Rights – In no way are any of the following rights affected by the
license:
• Your fair dealing or fair use rights, or other applicable copyright
exceptions and limitations;
• The author’s moral rights;
• Rights other persons may have either in the work itself or in how
the work is used, such as publicity or privacy rights.
Notice – For any reuse or distribution, you must make clear to others the
license terms of this work. The best way to do this is with a link to
this web page.
The court’s PhD was appointed by the rector of the Universitat Pompeu
Fabra on .............................................., 2012.
Chairman
Member
Secretary
The doctoral defense was held on ......................................................., 2012,
at the Universitat Pompeu Fabra and scored as ...................................................
PRESIDENT
MEMBERS
SECRETARY
For my beloved wife
Agraïments
My biggest thanks to ...
... the city of Barcelona. Place where I had the most amazing experiences
and met the most interesting people. A special thanks to the sun, to the
beach and to the good catalan food.
... my mother, who always trusted and encouraged my decisions. Her care
and support have been truly fundamental for everything that I accomplished
so far.
... my father, my role model. Who taught me that things are just simple.
... my wife for boarding with me in this journey overseas.
... the twin brother André Luvizotto for being my wingman, from Brazil to
Thailand. And also to Sassi!
... my supervisor Paul Verschure for introducing me to the neuroscience and
guiding through the whole higher-education experience.
... my mentor John Lisman, whose patience, availability and clarity of
thought were genuinely inspiring.
... Jonatas Manzolli and Fernando Von Zuben for introducing me to science
and research and opening me so many opportunities.
... my brother and sister-in-law, for keeping the surfin’ dreams alive.
... my great culé friends Anna-Felipe and Aisha-José.
... all my Costa family, friends and in-laws in Belo Horizonte.
... all my family in Rio de Janeiro. In special to my dear goddaughter
Sophia.
iv
v
... all my Rennó family that is all around. In special to my dear grandmother.
... all the friends from Brazil who came to visit me and all of those that
couldn’t.
... my friends from the pachanga de domingo y los Pachanga All-Stars.
... The SPECS team: Anna Mura, Armin Duff, Marti Sanchez Fibla, Sylvain Le Groux, Xerxes Arsiwalla, Alberto Betella, Belén Rubio Ballester,
Encarni Marcos, Ivan Herreros Alonso (special thanks for translating my
abstract to catalan), Luis Bobo, Riccardo Zucca, Vicky Vouloutsi, Michelle
Wilson, Manuel Ebert, Jens Nirme, Fotios Balampanis, Eliza-Nefeli Tsaoussis, Diogo Pata, Daniel Pacheco Estefan, Sytse Wierenga, Pedro Omedas,
Enrique Martínez, Alex Escuredo Chimeno, Christian Moscardi, Mireia
Mora, Carme Buisan. Ulysses Bernardet, Martin Inderbitzin, Zenon Mathews, Elena Kokkinara, Fabio Rotondi, Arnau Espinosa, Cristina Campillo,
Melle Hofman, Hannu Markus Järvinen, Sergi Bermúdez, Andrea Giovannucci, Mónica Cameirão, Alexander Valjamae, Jose Maria Blanco Calvo,
Dor Konforty, Miguel Lechón, Noemí Conesa, Ana Pesquita, Anant Dhir,
Fabio Manzi, Ninuska.
... the DTIC and UPF staff.
... the PhD committee.
... Google Inc., Microsoft Gaming Division, Sony Entertainment, Ubuntu,
EA Sports, Infinity Ward, Mendeley, Mathworks and many other companies
that, for a little price, made my life much easier.
Abstract
There is no consensual understanding on what the activity of the hippocampus neurons represents. While experiments with humans foster a dominant
view of an episodic memory system, experiments with rodents promote its
role as a spatial cognitive system. Although there is abundant evidence
pointing to an overlap between these two theories, the dissociation is sustained by conflicting physiological data. This thesis proposes that the functional role of the hippocampus should be analyzed in terms of its structure
and function rather than by the correlation of neuronal activity and behavioral performance. The identification of the hippocampus code, i.e. the set
of computational principles underlying the input-output transformations of
neural activity, might ultimately provide a unifying understanding of its role.
In this thesis we present a theoretical model that quantitatively describes
and interprets the selectivity of regions of the hippocampus to spatial and
non-spatial variables observed in experiments with rats. The results suggest
that the multiple aspects of memory expressed in human and rodent data
are derived form similar principles. This approach suggests new principles
for memory, pattern completion and plasticity. In addition, by creating a
causal tie between the neural circuitry and behavior through a robotic control framework we show that the conjunctive nature of neural selectivity
observed in the hippocampus is needed for effective problem solving in realworld tasks such as foraging. Altogether, these results advance the concept
that the hippocampal code is generic to the different aspects of memory
highlighted in the literature.
viii
Resum
Actualment, no hi ha consens científic respecte a la informació representada
en la activitat de les célules del hipocamp. D’una banda, experiments amb
humans sostenen una visión de la funció de l’hipocamp com a un sistema
per l’emmagatzematge de memóries episódiques, mentre que la recerca amb
rodents enfatitza una visió com a sistema cognitiu espacial. Tot i que existeix abundant evidència experimental que indica una possible sobreposició
d’ambdues teories, aquesta dissociació també es manté en part en base a
dades fisiològiques aparentment incompatibles. Aquesta tèsi poposa que
l’hippocamp té un rol funcional que s’hauría d’analitzar en termes de la
seva estructura i funció, enlloc de mitjança estudis correlació entre activitat
neuronal i comportament. La identificació d’un codi a l’hipocamp, es a dir,
el conjunt de principis computacionals que conformen les transformacions
d’entrada i sortida de l’activitat neuronal, hauría de proporcionar un explicació unificada de la seva funció. En aquesta tèsi presentem un model
teòric que descriu quantitativament i que interpreta la selectivitat de certes regions de l’hipocamp en funció de variables espaials i no-espaials, tal
i com observada en experiments amb rates. Aquest resultat suggereix que
multiples aspectes de la memòria expressada en humans i rodents deriven
d’uns mateixos principis. Per aquest motius, proposem nous principis per
la memòria, l’auto-completat de patrons i plasticitat. A més, mitjançant
aplicacions robòtiques, creem d’un nexe causal entre el circuit neural i el
comportament amb el que demostrem la naturalesa conjuntiva de la selectivitat neuronal observada en el hipocamp es necessària per la solució
de problemes pràctics comuns, com per example la cerca d’aliments. Tot
plegat, aquests resultats avancen en l’idea general de que el codi de l’hipocamp es genèric i aplicable als diversos tipus de memòries estudiades en la
literatura.
x
Publications
Included in the thesis
Peer-reviewed
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2010a). The
mechanism of rate remapping in the dentate gyrus. Neuron, 68(6):1051–8.
ISSN 1097-4199. doi: 10.1016/j.neuron.2010.11.024
Rennó-Costa, C., Luvizotto, A., Marcos, E., Duff, A., Sanchez-Fibla, M.,
& Verschure, P. F. M. J. (2011). Integrating neuroscience-based models
towards an autonomous biomimetic Synthetic Forager. Dins 2011 IEEE International Conference on Robotics and Biomimetics, ps. 210–215. IEEE,
Phuket, Thailand. ISBN 978-1-4577-2138-0. doi: 10.1109/ROBIO.2011.
6181287
Rennó-Costa, C., Luvizotto, A., Betella, A., Sanchez Fibla, M., & Verschure,
P. F. M. J. (2012c). Internal drive regulation of sensorimotor reflexes in the
control of a catering assistant autonomous robot. Dins Lecture Notes in
Artificial Intelligence: Living Machines
In preparation
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2012a). The
mechanism of attractor dynamics in the CA3. In Preparation
Rennó-Costa, C. & Verschure, P. F. M. J. (2012). Nonspatial selectivity
of place cells supports quasi-optimal behavior in mixed spatial/nonspatial
tasks. In Preparation
xii
xiii
Other publications and abstracts
Duff, A., Rennó-Costa, C., Marcos, E., Luvizotto, A., Giovannucci, A.,
Sanchez-Fibla, M., Bernardet, U., & Verschure, P. (2010). From Motor
Learning to Interaction Learning in Robots, volum 264 de Studies in Computational Intelligence. Springer Berlin Heidelberg, Berlin, Heidelberg. ISBN
978-3-642-05180-7. doi: 10.1007/978-3-642-05181-4
Luvizotto, A., Rennó-Costa, C., Pattacini, U., & Verschure, P. F. M. J.
(2011). The encoding of complex visual stimuli by a canonical model of the
primary visual cortex: Temporal population code for face recognition on
the iCub robot. Dins 2011 IEEE International Conference on Robotics and
Biomimetics, ps. 313–318. IEEE, Phuket, Thailand. ISBN 9781457721373.
doi: 10.1109/ROBIO.2011.6181304
Luvizotto, A., Rennó-Costa, C., & Verschure, P. F. M. J. (2012). A waveletbased neural model to optimize and read out a temporal population code.
Frontiers in computational neuroscience, 6:21. ISSN 1662-5188. doi: 10.
3389/fncom.2012.00021
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2010b). The
mechanism of rate remapping in the dentate gyrus. Dins Society for Neuroscience Abstracts. San Diego, CA, USA
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2012b). The
mechanism of attractor dynamics in the CA3. Dins Society for Neuroscience
Abstracts. New Orleans, LA, USA
Contents
Agraïments
iv
Abstract
viii
Resum
x
Publications
xii
Contents
xv
List of Figures
xvii
1 Introduction
2 The
2.1
2.2
2.3
2
hippocampus
8
Place cells . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Other spatially driven cells . . . . . . . . . . . . . . . . . . . 18
Conjunctive place cells and rate remapping . . . . . . . . . . 21
3 Computational models of the medial temporal lobe
3.1 Computational models of spatial selectivity . . . . . .
3.1.1 Computational models of grid cells . . . . . . .
3.1.2 Computational models of place cells . . . . . .
3.1.3 Place cell navigation and behavior . . . . . . .
3.2 Computational models of memory . . . . . . . . . . .
3.2.1 Pattern completion and attractor dynamics . .
3.2.2 Memory sequences in the hippocampus . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
24
24
25
31
38
39
39
40
4 Mechanisms of conjunctive selectivity in the dentate gyrus 42
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
xv
xvi
CONTENTS
4.3
4.4
4.5
Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Supplemental material . . . . . . . . . . . . . . . . . . . . .
4.4.1 Alternative assumptions about how the LEC responds
to morphing . . . . . . . . . . . . . . . . . . . . . . .
4.4.2 Differences in how DG and LEC encode sensory information . . . . . . . . . . . . . . . . . . . . . . . .
4.4.3 Comparison to other models . . . . . . . . . . . . . .
Experimental procedures . . . . . . . . . . . . . . . . . . . .
. 52
. 58
. 58
. 59
. 59
. 61
5 Mechanisms of conjunctive selectivity in the CA3
72
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6 Mechanisms of hippocampal behavioral control
6.1 Theoretical study on behavior . . . . . . . . . . .
6.1.1 Introduction . . . . . . . . . . . . . . . .
6.1.2 Results . . . . . . . . . . . . . . . . . . .
6.1.3 Discussion . . . . . . . . . . . . . . . . . .
6.1.4 Methods . . . . . . . . . . . . . . . . . . .
6.2 Robot experimentation . . . . . . . . . . . . . . .
6.2.1 Introduction . . . . . . . . . . . . . . . .
6.2.2 Results . . . . . . . . . . . . . . . . . . .
6.2.3 Conclusions . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
7 Conclusion
A Internal drive regulation of sensorimotor reflexes
A.1 Introduction . . . . . . . . . . . . . . . . . . . . . .
A.2 The control architecture . . . . . . . . . . . . . . .
A.2.1 The hardware . . . . . . . . . . . . . . . . .
A.2.2 The autonomous control system . . . . . .
A.3 Results . . . . . . . . . . . . . . . . . . . . . . . . .
A.4 Discussion . . . . . . . . . . . . . . . . . . . . . . .
Bibliography
84
84
85
91
94
96
97
98
105
114
116
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
120
121
124
124
125
131
135
138
List of Figures
2.1
2.2
2.3
2.4
The medial temporal lobe in the brain. (top) Coronal
cut of whole-brain Macaca mulatta (bottom) Nissl-sagittal cut
of whole-brain Rattus Norvegicus. Highlighted areas: dentate
gyrus (DG), CA1, CA3, subiculum (Sub), entorhinal cortex (EC,
lateral LEC, medial MEC), and perirhinal cortex (36). Adapted
from brainmaps.org. . . . . . . . . . . . . . . . . . . . . . . . .
Hippocampus anatomy. Drawing of the neural circuitry of a
rodent hippocampus by Ramón y Cajal (1909). Diagram with
most relevant connections within the hippocampus and the entorhinal cortex. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Illustration of the methods for the recording of place
cells. (A) Recording setup. Rat with implanted immutable
drive with multiple tetrodes is free to move inside an arena. A
camera is used to track the position of the rat. Tracking and
neurophysiological data are time stamped. (B) Common place
cell representations. (top) Spikes (red dots) overlying the trajectory of the rat during one recording session. (down) Rate map
of the spatial activity. Frequency varies from highest (red) to
silent (blue). . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Illustration of the spatial proprieties of place cells. (A)
Rotation of dominant environmental cues (such as a white card)
causes similar transformation in the place cells. (B) Changes in
the dimensions of the environment causes similar transformation
in the place cells. (C) Different arenas (represented with black
and gray border) with the same dimensions cause place cells to
exhibit uncorrelated spatial activity. (D) Place cells are stable
in the dark if the rat had experienced the environment with light
previously. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
9
. 10
. 15
. 17
xvii
xviii
LIST OF FIGURES
2.5
Illustration of other spatially selective cells. (A) Rate map
of a grid cell with triangular grid organization. (B) Polar plot of
the angular response of a head direction cell. (C) Rate map of a
border cell in an arena with an internal wall. . . . . . . . . . . . 18
2.6
Illustration of the spatial variations of grid cells. (A)
Scale or intervertex distance: the distance between place fields
in the same triangular formation. (B) Angular phase: the angle
between place fields in the same triangular formation. (C) Spatial phase: the absolute position of a specific place field in the xand y-axis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.7
Illustration of the rate remapping phenomenon. (A) Different arena shapes that provoke gradual rate remapping and illustrative rate maps for CA3 (B) and DG (C). Red refers to high
rate while deep blue refers to silent. Graph at right is the mean
fire rate of the place fields according to the morphing phase. For
DG, upper place field (black) and bottom place field (gray) are
represented. (D) Procedure to compute the population vector
(PV). The rate of all cells at a specific position are included in
sequence in an array. The activity ensemble change is computed
by the mean correlation of the PV of two conditions computed
for every position. (E) Illustration of the PV correlation curve
for DG and CA3. CA3 present no difference between 1-step morphing and two successive recordings of the same arena. Adapted
from (Leutgeb et al., 2007). . . . . . . . . . . . . . . . . . . . . . 22
3.1
Illustration of the oscillatory interference model. (A)
Each grid cell has three dendritic subunits that receives input
from different head-direction cells with phase intervals of 60.
The dendritic subunits have an intrinsic internal oscillation that
is modulated by the integral of the speed input of the headdirection cells. (B) A grid cells exhibit theta based activity when
the three sub-units have similar frequencies that causes an oscillatory interference. The movement of the animal changes the
frequency of each dendritic subunit leading to successive active
and silent states that give rise to the grid pattern. . . . . . . . . 26
LIST OF FIGURES
xix
3.2
Illustration of the intrinsic persistent spiking model. (A)
Each grid cell receives input of three different head direction
cells with phase intervals of 60. (B) A grid cells is active when
the persistent firing of three head direction cells are synchronized. The movement of the animal changes the phase of each
head direction cell leading to successive synchronization and dissynchronization states that give rise to the grid pattern. . . . . . 28
3.3
Illustration of the continuous attractor dynamic models.
Bump of activity (black with high activity and white with low
activity) in the bidimensional topological organization of the entorhinal cortex neural network with (A) rectangular neighborhood (McNaughton et al., 2006) and (B) triangular neighborhood (Guanella & Verschure, 2006). In both models the cells
in the boundaries are interconnected allowing the formation of
periodic place fields. (C) Bump is maintained by homogeneous
lateral connectivity (shown in a linear representation for clarity).
(D) Bump of activity is moved by the modulation of the lateral
connectivity in the direction of the animal motion. . . . . . . . . 29
3.4
Illustration of the place cell model based on border cells.
(A) Example of the typical rate map of a BVC (boundary vector
cell) in three environments. (B) The activity of a place cells is
obtained by the integration (followed by a linear threshold filter)
of multiples BVC cells. . . . . . . . . . . . . . . . . . . . . . . . . 32
3.5
Illustration of how place cells are generated from grid
cells in the integrative-competitive model. (top) The integration of multiple grid cells will generate a spatial dependent
excitation for a granule cell (bottom-left) . The E%-MAX winnertake-all competition working on the basis of this excitation will
lead to the formation of the place fields (bottom-right). . . . . . . 35
xx
LIST OF FIGURES
3.6
Illustration of the E%-MAX winner-take-all mechanism.
(A) Architecture of the hippocampus network in de Almeida
et al. (2009b, 2010). The granule cells in the dentate gryrus receives major convergent input from the entorhinal cortex. The
pyramidal cells receive strong input from single granule cells and
major convergent input from the entrohinal cortex. DG and CA3
exhibit E%MAX winner-take-all competition. (B) Competition
is caused by the inhibitory interneurons that are capable of emitting global feedback inhibition (IPSP). (C) The interneurons are
triggered after 3 ms of the first spike in the network cycle. Cells
that are capable of reaching threshold during the 3 ms window
also produce spikes, the other cells are inhibited before becoming
active. Adapted from de Almeida et al. (2009a). . . . . . . . . . 36
4.1
Illustration of the process of place cell formation including LEC and MEC inputs. For each neuron, the excitement
is computed for each position (x, y). Input from LEC is added to
the input from MEC. At each specific position all cells compete
through the E%-MAX process that outputs a population inhibition level. Rate map is build from the amount of excitation that
exceeds the inhibition plane. This process is used in (RennóCosta et al., 2010a) and is analog to the one used in (de Almeida
et al., 2009b, 2010). . . . . . . . . . . . . . . . . . . . . . . . . . 43
LIST OF FIGURES
xxi
4.2
MEC and LEC inputs and estimation of model parameters. (a) Example of the 10 MEC modeled rate maps (number
is the maximum firing rate). MEC rate maps remain constant
during morphing. (b) Example of the 10 LEC rate maps from
experimental data (H, maximum rate when informed, adapted
from (Hargreaves et al., 2005) and 10 from the model for the
two environments (square and round, maximum rate in both
environments). Rate maps presented with equally distributed
spatial score (ranked from right to left). (c) Histogram of spatial
information score from LEC rate maps (H, experimental data
and square, model. correlation 0.9957, P < 0.05). (d) Ratio
(α) between the mean firing rates in MEC and LEC estimated
as 0.32 by fitting to the experimentally observed reduction on
spatial coincidence using population vector correlation as the
environment is morphed (Leutgeb et al., 2007). (e) Histogram of
the number of place fields found in DG cells (Leutgeb and square
environment). Stable high correlation between experimental and
simulated histograms during morphing indicates that modification in LEC activity do not disrupt place field formation (R =
0.98). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
4.3
Difference in spatial coincidence reduction rate for abrupt
and linear interpolated morphing of LEC spatial response.
Comparison of the mean population vector (PV) during remapping compared with (Leutgeb et al., 2007). For 17% of morphing,
the minimal correlation value is 0.82 ± 0.01 compared to 0.75 observed experimentally. . . . . . . . . . . . . . . . . . . . . . . . . 50
4.4
Simulated DG cells exhibit independent place field rate
remapping, as observed experimentally. Differential rate
changes in individual firing fields of cells from the dentate gyrus
during progressive maneuvering of the walls of the arena. (a)
Recorded cells. Adapted from (Leutgeb et al., 2007). (b) Simulated cells. Individual fields are numerically labeled to relate
to the respective line diagram of the mean field rate. The rate
curves were fitted to linear (red), quadratic (green) or sigmoid
(blue) functions and are shown when significant (p < 0.05, dotted
line). (c) Histogram of the best fit classification for recorded and
simulated curves. Correlation between histograms is of 0.9543 (P
= 0.045). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
xxii
LIST OF FIGURES
4.5
Different mechanisms for independent rate remapping
of different place fields of the same cell. (A) Rate is directly affected by changes of the input drive. For a given cell,
morphing (round to square) induces localized variation of LEC
input, changing the rate of each place field independently (At
PF1 , elevation of input drive (INPUT1 ) causes the rise of rate
(RATE1 ). At PF2 , the fall of the input level (INPUT2 ) leads
to reduction of rate (RATE2 )). In this case, remapping is only
caused by the change of the input since the global inhibition level
does not vary (dotted red line); (B) Rate is inversely affected by
changes of the inhibition. Morphing induces localized variation
of the global inhibition level, changing the rate of each place
field independently (At PF1 , the raise of the global inhibition
level (INH3 ) causes the decay of the rate (RATE3 ). At PF2 , the
fall of the global inhibition level (INH4 ) causes the rise of the
rate (RATE4 )). In this case, remapping is only caused by the
local changes on the global inhibition level since all inputs to this
cell remain in the same level during remapping. The change of
the inhibition level is caused by variations of the input drive of
the most excited cell. For each cell and wall shape a rate map is
shown with the relevant place fields indicated by a white circle
and the process values used of the computation of the rate at
these place fields: the sum of entorhinal input (light gray bar
for LEC and dark gray bar for MEC); the sum of entorhinal input of the most excited cell (red line); the global inhibition level
(dotted red line) and the rate (black bar). . . . . . . . . . . . . . 53
4.6
Distribution of the mechanism balance ratio through active place fields. For clarification see Methods. Low ratio indicates prevalence of first mechanism (Figure 4.5A) while high
ratio indicates that second mechanism (Figure 4.5B) is more effective. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
LIST OF FIGURES
xxiii
4.7
DG cells do not represent sensory and spatial information in the same way as LEC cells. (A) Plot of the mean
spatial information score on both shapes and the shape information score of LEC cells (blue, x) and DG cells (red, o). Spatial
information score measures quantitatively how the position is encoded by one spike while shape information score relates to how
much information about the shape each spike carries. (B) Box
plot of the mean spatial information score and of (C) the shape
information score for both populations. In each box, the central
mark is the median, the edges of the box are the 25th and 75th
percentiles and the whiskers extend to the most extreme data
points. (D) The relative contribution of LEC and MEC input
influences spatial properties. Histogram of the mean place field
size as function of the ratio (α) of the mean drive of MEC and
LEC onto EC. Low alpha indicates high LEC influence while low
alpha indicates stronger MEC input. . . . . . . . . . . . . . . . . 70
4.8
Experimental variance correction for simulated data. (A)
Mean population vector (PV) correlation for two successive recordings of the same morphing stage decays linearly with the increase
of frequency proportional variance. The number of cells considered for the PV influences the effect of variance in correlation:
less cells raises sensibility. To correct the simulated data to the
experimental condition of (Leutgeb et al., 2007) we used the
variance/frequency value (experimental factor β) that fits the
experimental PV correlation. (B) Fitting of β is influenced by
both number of cells and LEC/MEC mean rate ratio. . . . . . . 71
5.1
Rate remapping in the DG with spiking neurons. (A)
Sample cells from MEC and LEC in the two environments. (B)
Two sample DG cells exhibiting rate remapping. (C) PV correlation curve for the DG population compared with data from
Leutgeb et al. (2007). . . . . . . . . . . . . . . . . . . . . . . . . 76
5.2
Distribution of the number of place fields with the spiking model. Distribution for both DG and CA3 (no recurrents)
is coherent with experimental findings (Leutgeb et al., 2007). . . 78
5.3
Rate remapping in the CA3 population without recurrents don’t explain experimental data. PV correlation
curve for of the simulated CA3 population aligned to experimental CA3 (red ) and DG (blue) curves (Leutgeb et al., 2007). . 79
xxiv
5.4
5.5
5.6
6.1
6.2
LIST OF FIGURES
PV correlation curve for CA3 with recurrents batchtrained. Shown for multiple recurrent strengths (from black to
light blue). Experimental curve, normalized to mean 1-1’ correlation in simulation, is shown in dotted red (Leutgeb et al.,
2007). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
Pattern completion by recurrent excitation. Trace of the
potential of a neuron in a single gamma cycle for two different
morphing stages. (top) When there is no recurrent input, the
10% morphing changes the input in a way that the cell cannot
accumulate enough energy to release a spike. (bottom) When
a recurrent input is present, the cell gets an extra amount of
energy and spikes before the global inhibition is released. . . . . 81
Effect of LTP in the PV correlation curve. . . . . . . . . . 82
Experimental protocol. Multiple Y-maze (lef t) shown with
its graph representation (right). Decision points represented as
vertexes, spatial actions as straight arrows and nonspatial actions as angular arrows. All affordances are shown. Optimal
solution is in red. (A) Spatial task. Reward is delivered when
the agent reaches a specific location and applies a nonspatial action. Illustrated as the task in which the rat has to find and eat
a piece of cheese. (B) Mixed spatial/nonspatial task. Rewarded
action (∗) is only available at the goal location after the agent
applies a nonspatial action at a different location (§). Illustrated
as the task in which the rat has to pull a button to release water
in the fountain located elsewhere. . . . . . . . . . . . . . . . . . . 87
The DAC architecture. (A) System overview and its major connections. The reactive layer relates statically sensory information and allostatic regulation with the motor primitives.
The adaptive layer builds on top of the reactive layer with selforganized responsive units of perception, proprioception and actions. (B) Relevant connectivity in the medial entorhinal cortex. Dentate Gyrus (DG) and CA3 integrate the multimodal input from the lateral and media portions of the entorhinal cortex
(LEC and MEC). Sequencing is obtained by the interconnectivity between CA3 and DG. Output is channeled back to the cortex
through the CA1 and the Subiculum (SuB) (C) Schematic for
behavioral learning (LTM acquisition) in the contextual layer.
(D) Schematic for action recall. (E) Procedure to select next
action. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
LIST OF FIGURES
xxv
6.3
Spatial selectivity is sufficient for solving a spatial task.
(A) Performance in the spatial task (optimal solution length
2.8±1.1 actions) as a function of the memory sequence length
(median and interquartile range). (B) Estimated memory sequence length that leads to the best performance as a function
of the mean length of the optimal solution of the maze. . . . . . 92
6.4
Conjunctive spatial/nonspatial selectivity is necessary
for solving a mixed spatial/nonspatial task. (A) Percentage of successful trials of the spatial controller in the mixed task
(optimal solution length 4.8±1.0 actions) and (B) performance
relative to the shortest path in the mixed task as a function of
memory sequence length (median and interquartile) for the halfshortest/-longest solutions. (C) Estimated memory sequence
length that leads to the best performance as a function of of
the mean length of the optimal solution of the maze. . . . . . . . 93
6.5
How conjunctive place cells solve the mixed spatial/nonspatial
task. (A) 4-memory sequences of non-conjunctive place cells
that cause misleading by attracting to action ∗ before action
§being executed. The middle memory sequence is a special case
in which the action §will never be accessible if the agent is at a
locked position. (B) Conjunctive place cells solve it by establishing an independent graph for each behavioral context, causing
that the agent will not be attracted to ∗ before executing §. . . . 95
6.6
Maze samples. With 10, 30, 100 and 129 decision points (topleft, top-right, bottom-left, bottom-right). Decision points in red
and path in gray. . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.7
SF Robots. (left) Outdoor and (right) indoor units with 1.1 x
0.6 m and 0.6 x 0.6 m respectively. Both equipped with embedded computation, proximity sensors and color camera mounted
on a pan-tilt unit. . . . . . . . . . . . . . . . . . . . . . . . . . . 105
6.8
Outdoors arena. (A) Overview of the operative area. (B)
Snapshots of the robot (90o between each other). . . . . . . . . . 106
6.9
Neural representation of space. Rate maps of (A) three
visual layer cells and (B) three memory layer cells. X- and Yaxis represent position and brightness the unit activity. (C)
Population vector self-correlation for both layers. (D) Boxplot
of spatial-info score in bits/spike. . . . . . . . . . . . . . . . . . . 109
xxvi
LIST OF FIGURES
6.10 Sequence-learning task. (A) Robot goes forward until the
red mark is detected. (B) Two color options are presented to
the robot. (C) If the correct color is selected then two new
options are presented. . . . . . . . . . . . . . . . . . . . . . . . . 111
6.11 Resource-localization task. (A) Experimental space (20 x 15
meters) with the arena marked on the floor. (B) Cue gathering at
home. (C) Kidnap procedure. Robot is taken from one position
to another and successfully finds the cued rewarded location. . . 112
A.1 Robot overview. The color camera is mounted over the pantilt unit in the front segment of the robot. The control CPU is
placed over the middle segment. The load is placed over the rear
segment. The proximity sensors are placed all around the robot,
but only the ones in the front and in the back are used for the
experiment. Robot produced by Robosoft. . . . . . . . . . . . . . 125
A.2 From face perception to gaze action: the perception/action
reflex in the visuomotor loop (A) Illustration of the visual
field of the robot and (B) the associated salience map emitted
by the face detection and salience map system. (C) Through a
competitive process the most salient point in the visual field is
selected. If the most salient point is active in a zone associated
with a gaze action, a saccade is activated. In this specific case,
the action “Pan Left” is triggered, moving the detected face to
the center of the visual field. (D) Illustration of the two possible
action types ("tilt" and "pan”) in relation to the gaze direction. . 126
A.3 Visual loop organization. Perception excites action. Action
inhibits the drive. The drive inhibits perception. Action and
drive have spontaneous activity. Drives regulates the system
by allowing action spontaneous activity to take place when few
actions are triggered. . . . . . . . . . . . . . . . . . . . . . . . . . 128
A.4 Navigation loop organization. Action has spontaneous activity. Action can be inhibited by the perception and by the
drive. The drive is activated by the action. . . . . . . . . . . . . 130
A.5 Demonstration venue (right) and the robot (top-left),
chocolates and candies are placed in a plate located in the back
part of the robot body. . . . . . . . . . . . . . . . . . . . . . . . . 132
LIST OF FIGURES
xxvii
A.6 Simulated navigation data. (A) Time evolution (exponential
fit) for the percentage of the area covered by delivery stops for
five different arena sizes. (B) Sample of robot trajectories and
delivery spot spatial distributions at different time windows for
the 5921 m2 square arena. . . . . . . . . . . . . . . . . . . . . . . 133
A.7 Demonstration of the gazing behavior by the visuomotor
loop in a sequence with a moving subject. . . . . . . . . . 134
CHAPTER
Introduction
The mammalian brain is a complex structure organized into many components with different functionality and morphology. The hippocampus
is one of these components. It is a subcortical structure located in the
medial temporal lobe with abundant connectivity to many other cortical
and subcortical regions. The anatomical preeminence of this brain region
highlights its relevance for neuroscience and the study of the nervous system.
In the last 60 years, strong research effort has been directed to the study
of the hippocampus. In this context, the discovery of spatial selective
neurons had the highest impact. Popularly known as “place cells”, these
cells exhibit sustained and reliable increase in the rate of activity whenever the animal is situated in a specific and well-delimited area of the
environment (O’Keefe & Dostrovsky, 1971). Empowered by its scientific
charm and inspired by prior theories of spatial behavior in psychology
(Tolman, 1948), the observations of place cells fostered a dominant theory that the main function of the hippocampus is to provide a cognitive
map of the environment (O’Keefe & Nadel, 1978). Nowadays, with the
2
1
3
advent of many studies that were inspired by the "cognitive map" theory, there is plentiful experimental knowledge about how the "place cells"
represent the space. Examples are the recognition of the effect of many
environmental manipulations in the firing of place cells (Muller & Kubie,
1987; Fenton et al., 2000a); the observation of spatial selectivity in the
first days of newborn rat pups (Langston et al., 2010); and the identification of place cells in the hippocampus of several species of mammals,
from bats (Ulanovsky & Moss, 2007) to humans (Ekstrom et al., 2003).
The "cognitive map" theory also promoted many theoretical and modeling studies that accompanied the experimental research and provided
valuable insight in the understanding of experimental data.
Nevertheless, the "cognitive map" is not the only theory of the hippocampus. It is also associated to the hippocampus a role in memory consolidation (Nadel & Moscovitch, 1997) and declarative memory formation
(Cohen & Squire, 1980). The most influential evidence is that lesions
on the hippocampus in humans cause severe retrograde amnesia that is
not limited to spatial memory (Scoville & Milner, 1957). This famous
clinical-study by Scoville called much attention and carried weight for the
medial temporal lobe research field. In this direction, experimentation
with humans revealed that some hippocampal neurons were selective to
faces and objects (Fried et al., 1997) and to purely abstract concepts
such as a character of a TV show (Quiroga et al., 2005). In rats, cells in
the same region where place cells are recorded have shown selectivity to
other physical aspects of behavior such as time (MacDonald et al., 2011).
An experiment by Wood et al. (1999) identified hippocampal cells in rats
that were selective to events rather than to the location where they took
place in a recognition memory task. Altogether, these facts stress the
hippocampus role in highly cognitive tasks that are not necessarily related to spatial cognition and behavior.
4
CHAPTER 1. INTRODUCTION
Nowadays, the disjunction of hippocampus research is a major challenge
in the field. It is not clear to what extend memory and spatial selectivity
overlap. So far, these two lines of thought did not converge. It is unknown
whether memory and spatial cognition functions share the same neural
circuitry or whether there is a major functional dissociation in the hippocampus. Nadel, co-postulator of the cognition map theory, was aware
of this kind of debate and positioned himself by labeling the hippocampus as “a spatial memory system” rather than a simply spatial system
(Nadel, 1991). Nadel argued that the spatial aspect of memory was an
“ineliminable property of our experience in the world” and therefore the
hippocampus memory system could not be dissociated from spatial cognition.
The viewpoint from Nadel is not consensual. Indeed, there is a crescent
and already dominant view that space is one aspect (of many) of the
memory represented by place cells (Eichenbaum, 2000). Evidence come
from the observation that the hippocampus is also implicated in nonspatial relational learning in both rats and humans (Bunsey & Eichenbaum,
1996) and that place cells activity is influenced by behavior (Frank et al.,
2000). Moreover, a study by Rolls et al. (2005) identified in rats different
groups of cells that were coding for position, for objects and for objectposition combinations. These observations indicate the possibility that
memory and spatial selectivity are indeed different interpretation of the
same phenomenon.
Within this context, the studies developed in this thesis sought the identification of what we call The Hippocampus Code, i.e. the set of computational principles underlying input-output transformation of neural
activity in the hippocampus. Our axiom is that memory and space selectivity are related to the same computational process, they just differ in
the nature of the observation and not on the process itself. We propose
5
that a structural and functional description of the hippocampus and its
neural circuits is the key for the convergence of the theories of memory
and space.
To allow the definition of a generic computational principle it is essential
to show that it applies equally for different aspects of the information,
i.e. spatial and nonspatial data should be processed in the same way.
There is however a major hitch in the analysis of this process. Most
studies present the spatial and nonspatial aspects of the neural response
in a way that they do not allow quantitative comparison. That’s mostly
because of the special nature of space: whilst the "where" information is
presented in a quasi-continuous and highly structured plane, the other aspects are in general presented in binary quantification (yes/no) and lack
formal organization. This allow the distorted interpretation that space
is more important than other aspects of memory or that nonspatial selectivity is built on top of the spatial representation, which constitutes
an "annotated cognitive map".
The unraveling of this problem was produced by an ingenious setup of
Leutgeb et al. (2007). In this study, rats where placed in an arena
whose walls could be gradually morphed from one shape to another.
This experimental apparatus allowed a controlled, graded and structured
parametrization of the nonspatial variable. The experiments with place
cells revealed the "rate remapping" phenomenon in which place fields
were kept stationary whilst their peak rate was gradually modulated by
the morphing of the walls. This particular experiment allowed for the
first time a quantitative analysis of the nonspatial aspects of memory
with the same methods used for the place cells.
Inspired in the experiment by Leutgeb et al. (2007), we investigated how
the hippocampal representation is created with respect to both spatial
6
CHAPTER 1. INTRODUCTION
and nonspatial aspects of memory (Chapter 4 and 5). For that, we modeled the mechanisms underlying nonspatial selectivity of place cells by
quantitatively fitting to the spatial and nonspatial selectivity observed
in the rate remapping process. Our computational study was based on a
biologically-constrained model of how spatial selectivity emerges in the
place cells if the known proprieties of its cortical inputs are considered
(de Almeida et al., 2009a,b, 2010). By considering the nonspatial cortical
input, we could demonstrate that the same neural mechanism underlies
spatial and nonspatial selectivity in the dentate gyrus, the first stage of
the hippocampus (Rennó-Costa et al., 2010a). Our results could quantitatively explain the experimental data (Leutgeb et al., 2007). In a second
study, we based on the dynamics of the nonspatial selectivity observed
with rate remapping to provide a novel interpretation on how attractor
dynamics support pattern completion in the hippocampus. Moreover,
our analysis allowed a quantitative analysis of the dynamics of plasticity
in the formation of stable memories in the CA3 region of the hippocampus (Rennó-Costa et al., 2012a).
With the computational model of the representation structure in place,
we could study the function of the neural circuits (Chapter 6). We used
the link between hippocampal activity and behavior identified in the
biomimetic robotic-oriented cognitive architecture Distributed Adaptive
Control (DAC) (Lisman, 2007; Verschure et al., 1992, 2003) to study if
the mixed spatial and nonspatial representation is essential for the ability to solve problems in mixed spatial/nonspatial tasks (Rennó-Costa &
Verschure, 2012). For this reason, we used a virtual and mathematically
defined action-space that allowed a quantitative analysis of performance
of the hippocampal-based controllers. In a further step, we investigated
whether the same principles hold in real-world environments. For that,
we implemented the DAC architecture in an unmanned mobile vehicle
and tested it in spatial, nonspatial and mixed spatial/nonspatial tasks
7
(Rennó-Costa et al., 2011).
Altogether, the experiments included in this thesis provided valuable insights about The Hippocampus Code (Discussions in chapter 7). Our
major contribution has been the demonstration that spatial and nonspatial information are processed through the same mechanisms and that
the conjunctive representation is essential for real-world cognition and
behavior. This allowed an interpretation in which memories and place
selectivity are indeed instances of the same computational concept.
Completing this thesis, we have in the next two chapters a non-exhaustive
review of the current knowledge of the hippocampus (Chapter 2) and the
available computational models of the medial temporal lobe (Chapter 3).
CHAPTER
The hippocampus
The hippocampus is located in the medial temporal lobe along with the
parahippocampal gyrus, which include the perirhinal, parahippocampal and entorhinal cortices (Figure 2.1). It is elongated1 through the
dorsoventral axis with the synaptic junctions distributed alongside the
dorsoventral and the mediolateral axis. The entorhinal cortex is the major synaptic interface from the neocortex to the hippocampus by the projection of its superficial layers to all hippocampal parts and by receiving
projections in its deep layers mainly through the subiculum. The medial temporal lobe connects to mostly all brain regions (Bird & Burgess,
2008), from the specialized areas of the neocortex such as the visual cortex and the prefrontal cortex (Degenetais, 2003) to subcortical areas such
as the amygdala (Pitkänen et al., 2000) and the ventral striatum (Pennartz et al., 2011).
The hippocampus formation is subdivided in several regions (Figure 2.2):
the dentate gyrus (DG), the cornu ammonis (CA) areas (CA1, CA2, CA3
1
Hippocampus is the latin name of a sea monster from Greek Mythology whose elongated outline reassembles the shape of the hippocampus in the brain. Hence the name
used in biology.
8
2
9
Figure 2.1: The medial temporal lobe in the brain. (top) Coronal cut
of whole-brain Macaca mulatta (bottom) Nissl-sagittal cut of whole-brain
Rattus Norvegicus. Highlighted areas: dentate gyrus (DG), CA1, CA3,
subiculum (Sub), entorhinal cortex (EC, lateral LEC, medial MEC), and
perirhinal cortex (36). Adapted from brainmaps.org.
10
CHAPTER 2. THE HIPPOCAMPUS
Figure 2.2: Hippocampus anatomy. Drawing of the neural circuitry of a
rodent hippocampus by Ramón y Cajal (1909). Diagram with most relevant
connections within the hippocampus and the entorhinal cortex.
and CA4) and the subiculum (Sub). An extensive review about the hippocampus wiring is provided by Johnston & Amaral (1998). The dentate
gyrus is composed mainly by granule cells and by inhibitory interneurons.
The perforant pathway (PP), originated in the entorhinal cortex, and the
mossy fibers, originated in the mossy cells that receive input from the
CA3, PP and other granule cells, provides major innervation to this area.
In the CA3, pyramidal cells receive input from the PP and DG and other
CA3 pyramidal cells. It is remarkable that the information flow is not
purely sequential but rather exhibits cyclical processing with the DGCA3 links (Lisman, 1999). In the CA1, pyramidal cells receive input
from the PP and CA3 and project back to the entorhinal cortex through
the Sub. Most of the synaptic organization in the hippocampus is conserved across species (Manns & Eichenbaum, 2006).
Another salient characteristic of the hippocampus is that it is the locus of the first observation of Long-Term Potentiation (LTP) (Lø mo,
1966), a special kind of synaptic plasticity in which the signal transmis-
11
sion between two neurons is augmented following synchronous activation
(Bliss & Lø mo, 1973). The anatomical properties of the CA3 region – in
special the interconnectivity between the pyramidal cells – in addition to
the observation of LTP have inspired models of recurrent auto-associative
neural networks, in special the Hopfield network (Hopfield, 1982).
The parahippocampal gyrus is organized in a way that the entorhinal cortex is the main interface of the neocortex with the hippocampus (Witter
et al., 2000a). The entorhinal cortex has a major anatomical dissociation between its lateral and medial regions (Witter et al., 2000b). The
two parts are also distinguishable in the connectivity, having the lateral
entorhinal cortex (LEC) connected favorably with sensory driven areas
such as the olfactory, insular and perirhinal cortices while the medial entorhinal cortex (MEC) is mainly connected with visual-spatial occipital,
parietal and postrhinal cortices and the pre-parasubiculum (Canto et al.,
2008). There is no noticeable difference in how LEC and MEC project
to the dentate gyrus in rats while some topological organization can be
observed in monkeys (Witter et al., 1989).
One important aspect of the hippocampus morphology is that it remains
fairly stable through many species, from rodents to primates. This evolutionary constancy evidences the fundamental role of the hippocampus
in the brain on the support of cognition and behavior. The implication
of this observation is that conclusions from animal experimentation can
in many cases be very influential on the study of the human brain.
Regarding its function, the medial temporal lobe is mainly associated
with three purposes: inhibition, memory and space. The inhibition function is related with the fact that animals with hippocampal lesions exhibit
motor hyperactivity, which allowed the link to anxiety disorder (Gray &
McNaughton, 2000).
12
CHAPTER 2. THE HIPPOCAMPUS
The first link between memory and the hippocampus was originated on
the reports from the clinical studies of the patient Henry Gustav Molaison
(H. M.) by Scoville & Milner (1957). H. M. had most of his medial temporal lobe surgically removed as a treatment to epilepsy, which resulted in
partial retrograde amnesia - the patient could remember memories from
many years before the surgery but was not able to recall facts and events
that happened a few years before the surgery – and severe anterograde
amnesia – the patient could not remember events that have just happened. This outcome reveled an important role of the medial temporal
lobe in the formation of new memories. Following studies have related
the hippocampus and the medial temporal lobe with declarative and
episodic memory functions, with the specific functions of memory consolidation, relational cognition and the link between memory and space
(Eichenbaum, 2001). The parahippocampal gyrus also has some specific
functionality regarding memory cognition. For instance, the entorhinal
cortex is associated with working memory of novel objects (Hasselmo &
Stern, 2006) while the perirhinal cortex is related with familiarity-based
object recognition (Murray et al., 2007).
The latest function attributed to the hippocampus is spatial cognition.
The major finding supporting this functionality was the observation of
the place cells by O’Keefe & Dostrovsky (1971) in the CA1 region. Place
cells were also found in the dentate gyrus and CA3 (Jung & McNaughton,
1993; Leutgeb et al., 2007) and in the subiculum (Brotons-Mas et al.,
2010; Sharp, 2006). Regarding the parahippocampal gyrus, spatial selectivity is also observed in the medial entorhinal cortex (Fyhn et al., 2004)
but not in the lateral entorhinal cortex (Hargreaves et al., 2005). Evidence that place cells is effectively associated with behavior is the fact
that performance in spatial tasks is impaired after hippocampal damage
in both rats (Morris et al., 1982) and humans (Astur et al., 2002). More-
13
over, patients with hippocampal damage have impaired spatial memory
recall (Bohbot et al., 1998; Bartsch et al., 2010).
While the three cognitive functions associated with the hippocampus
might seem uncorrelated, there is evidence for functional overlapping
and neural circuitry sharing between them. For instance, place cells firing is modulated by changes in nonspatial features of the environment
(O’Keefe & Conway, 1978) and by behavioral context, which can be
related to episodic memory encoding (Wood et al., 2000). Moreover,
inhibition might affect spatial cognition as a process of attention with
implications in place cells properties (Fenton et al., 2010). However,
these finding does not overrule neural circuitry specialization, as there
is evidence for functional dissociation in the septotemporal axis of the
hippocampus in spatial tasks (Bannerman et al., 1999) and inhibitory
learning tasks (McDonald et al., 2006).
The available evidence also delimits the functional boundaries of processes in which the hippocampus and the medial temporal role are not
involved or, at least, do not play a fundamental role. One example comes
from the observations that H. M. was capable of learning new motor skills
such as drawing (Corkin, 2002), rotary pursuit, bimanual tracking, and
tapping (Corkin, 1968) and the ability to learn certain problem-solving
procedures (Cohen & Corkin, 1981). The same effect is also observed in
rat in regards to spatial cognition (Morris et al., 1982). In the Morris
water maze, rats with hippocampal lesion are initially impaired of finding the location of a hidden platform. However, after a long period of
learning they succeed in accomplish the task. Indeed, H. M. “was able
to construct a cognitive map of the spatial layout of his house as the
result of daily locomotion from room to room” (Corkin, 2002). These
are evidence of dissociation between procedural non-conscious memory,
whose learning is not dependent on the medial temporal lobe, and declar-
14
CHAPTER 2. THE HIPPOCAMPUS
ative memory, which is dependent on the medial temporal lobe (Cohen
& Squire, 1980).
2.1
Place cells
The recording of place cells was made possible by the development of
single-unit (one neuron) recording technics. The technological breakthrough was the invention of immovable implantable electrodes that
could reach specific brain areas and its further evolution to stereotrodes
(McNaughton et al., 1983) and tetrodes (Recce & O’Keefe, 1989). Singleunit spikes trains can be identified in a multi-unit recording by their differential amplitude and waveform. From the same recording is possible
to obtain the local field potential (LFP), which is the electrical current
flowing within a certain volume of tissue and that, thus, includes the
activity of multi-units.
The identification of the place cells followed the simultaneous recording
of the activity of single neurons and the position of the animal (Figure 2.3A). By plotting the spikes of a neuron on top of the trajectory
of the rat (Figure 2.3B), O’Keefe & Dostrovsky (1971) observed that
some CA1 neurons were only producing spikes at a specific region of the
arena. This discovery was highly influential in neuroscience and after
many years much is known about the spatial properties of the hippocampal neurons. Place cells were also found in the dentate gyrus and CA3
(Jung & McNaughton, 1993; Leutgeb et al., 2007) and in the subiculum
(Brotons-Mas et al., 2010; Sharp, 2006). Moreover, place cells were also
found in the hippocampus of humans (Ekstrom et al., 2003), monkeys
(Hori et al., 2005), bats (Tsoar et al., 2011) and birds (Hough & Bingman, 2004).
2.1. PLACE CELLS
A
15
B
CAMERA
ARENA
Figure 2.3: Illustration of the methods for the recording of place
cells. (A) Recording setup. Rat with implanted immutable drive with
multiple tetrodes is free to move inside an arena. A camera is used to
track the position of the rat. Tracking and neurophysiological data are time
stamped. (B) Common place cell representations. (top) Spikes (red dots)
overlying the trajectory of the rat during one recording session. (down)
Rate map of the spatial activity. Frequency varies from highest (red) to
silent (blue).
To better understand what is the nature of the place cell representation, many studies have used environmental manipulations as a study
paradigm. Muller & Kubie (1987) found that rescaling and rotating a
circular arena evoked similar transformations in the rate maps of the
place cells (Figure 2.4AB). They could predict the activity of the hippocampal cells based on the transformations. They also observed that
some cells were linked to distal cues (the walls of the lab where the arena
was placed) while some other cells were linked to proximal cues (a white
card placed inside of the arena). Also, they found that place cells do
not show direction selectivity in open field experiment but they exhibit
strong polarization in linear tracks in maze experiments (Muller et al.,
16
CHAPTER 2. THE HIPPOCAMPUS
1994).
Their major finding however was that place cells remap. When the rat is
placed in a different arena – even though it has the same characteristic
than the original – the activity of the place cells cannot be predicted
from the previous experiment (Figure 2.4C). In other words, the place
fields are morphed to a random position. Moreover, they observed that
many cells are active in both environments but some are only active at
one of the environments. When returned to the original environment the
place cells returned to their initial firing profile. These findings allow
an interpretation that place cells have not only a general and relative
spatial representation of the space but it spatial representation is related
to specific locations.
Another interesting observation is that place cells can still exist when
the rat is in the dark (Figure 2.4D), although with a lower reliability and
in minor number (Markus et al., 1994). Indeed, the firing of hippocampal place cells in the dark depends on the rat’s recent experience (Quirk
et al., 1990), which might suggest some kind of spatial memory recall.
These results point out that the rat is capable of building a fairly reliable
representation of the space when deprived from sensory stimuli. Together
with the fact that such representation depends on previous anchoring on
environmental cues, the evidences suggest the existence of some kind of
relative path integration system underlying the formation of the place
cells. This could be clearly observed in an ingenious study by Gothard
et al. (1996) in which the initial position of a linear track could be manipulated. Place cells were dominantly driven by the environmental cues
when these were available (light condition) since the place fields were
stable in relation to the endpoint of the linear track regardless of the
initial position. However, when sensory deprived (dark condition) the
place fields were stable in relation to the initial position and not to the
2.1. PLACE CELLS
17
A
C
B
D
Figure 2.4: Illustration of the spatial proprieties of place cells. (A)
Rotation of dominant environmental cues (such as a white card) causes
similar transformation in the place cells. (B) Changes in the dimensions
of the environment causes similar transformation in the place cells. (C)
Different arenas (represented with black and gray border) with the same
dimensions cause place cells to exhibit uncorrelated spatial activity. (D)
Place cells are stable in the dark if the rat had experienced the environment
with light previously.
endpoint, suggesting that the rat was somehow “counting steps”.
It is also important to mention the time properties of the spike train of
the place cells. The hippocampus activity is strongly modulated in two
frequency ranges: theta (∼ 8 Hz) (Green & Arduini, 1954) and gamma
(∼ 40 Hz) (Soltesz & Deschênes, 1993). The time of a spike of hippocampal cells tend to be confined to a specific phase of the gamma cycle
(Bragin et al., 1995), which suggests that the mechanism that evokes the
spike is also responsible for this oscillation (de Almeida et al., 2009a).
Regarding the theta rhythm, it is present during motor activation and
REM sleep (Vanderwolf, 1969). The most relevant propriety regarding
18
CHAPTER 2. THE HIPPOCAMPUS
spatial activity is a phenomenon named phase precession: the position
of the rat inside of the place field (how close it is to the center) can be
predicted from the phase of the firing within the theta cycle (O’Keefe
& Recce, 1993). The relation of theta and gamma rhythm is of major
relevance for the understanding of the functionality of the neural circuitry in the hippocampus. For instance, it allows the organization of
time-compressed discrete sequences which might be related to trajectories (Skaggs et al., 1996) or even as a forward path for a future trajectory
(Johnson & Redish, 2007). The observation of replays of sequences during sleep is an indication of memory consolidation in the hippocampus
(Girardeau & Zugaro, 2011). The organization in sequences seems to be
a computational principle of how the hippocampus deal with memory
(Jensen & Lisman, 1996) and plays a major role in how it affects behavior (Lisman, 2007).
2.2
Other spatially driven cells
A
B
C
90º
180º
0º
270º
Figure 2.5: Illustration of other spatially selective cells. (A) Rate
map of a grid cell with triangular grid organization. (B) Polar plot of the
angular response of a head direction cell. (C) Rate map of a border cell in
an arena with an internal wall.
Place cells are not the only spatially driven cells in the brain. The same
technic used to identify place cells when applied to other brain regions
allowed the identification of additional spatial selective cell types such
2.2. OTHER SPATIALLY DRIVEN CELLS
19
as the grid cells (Figure 2.5A) in the medial entorhinal cortex (Fyhn
et al., 2004; Hafting et al., 2005), the head-direction cells (Figure 2.5B)
found in the post-subiculum (Taube et al., 1990b,a) and the border cells
(Figure 2.5C) also found in the medial entorhinal cortex and also in the
parasubiculum (Solstad et al., 2008).
The grid cells exhibit multiple place fields organized in a triangular grid
that spans throughout the whole environment (Figure 2.5A). The spatial
pattern of grid cells can vary the spatial phase, angular phase and scale
(Figure 2.6) (Hafting et al., 2005). Moreover, cells located in the same
micro-region exhibit the same angular phase and scale but not the same
spatial phase, suggesting a network property. Grid cells are topographically organized in the dorsoventral axis of the medial entorhinal cortex
having the intervertex distance varying from 25 cm to 8 meters (Brun
et al., 2008b). The scale span is not continuous but discrete (Barry et al.,
2007), which supports the anatomical evidence of the existence of multiples network cores (Witter & Moser, 2006). Grid cells are found on
the superficial layers II and III and in the deep layer V and the parasubiculum (Sargolini et al., 2006). In the layer III there can be found
conjunctive grid cells that the response is also selective to orientation
as in the case of head-direction cells. This propriety allow the indirect
observation of grid cells in humans using fMRI data (Doeller et al., 2010).
Grid cells are believed to be the main mechanism of path integration in
the medial temporal lobe (McNaughton et al., 2006). Major evidence
come from the fact that the grid cells remain stable whenever CA3 cells
undergo rate remapping (Fyhn et al., 2007). There are two kinds of models that explain the grid cell formation: one based on continuous attractor networks (Samsonovich & McNaughton, 1997; Guanella & Verschure,
2006) and the second based on oscillatory interference (Burgess et al.,
2007; Hasselmo et al., 2007). Both models agree in the fact that position
20
CHAPTER 2. THE HIPPOCAMPUS
A
B
C
Figure 2.6: Illustration of the spatial variations of grid cells. (A)
Scale or intervertex distance: the distance between place fields in the same
triangular formation. (B) Angular phase: the angle between place fields in
the same triangular formation. (C) Spatial phase: the absolute position of
a specific place field in the x- and y-axis.
is integrated based on odometry values such as speed and head-direction.
Although there is evidence for independence of the head direction and
grid cells systems (Whitlock & Derdikman, 2012), the fact that they are
sensible to the same kind of environmental manipulations (Taube et al.,
1990b) suggests that the head direction system is situated upstream, affecting the grid formation. These models are reviewed in the Section
3.1.1.
Path integration in rats is computed on the basis of purely internal signals, such as vestibular or proprioceptive afferences (Etienne et al., 1998).
However, visual cues are also is used to calibrate the path integration system to known landmarks. Evidence come from one experiment in which
2.3. CONJUNCTIVE PLACE CELLS AND RATE REMAPPING
21
rats were trained to explore and return to the nest Etienne et al. (2004).
In both dark and light conditions the rats were able to perform. However,
when the position of the nest (and the visual landmarks) were rotated,
the rats returned to the original nest position in the dark condition and
to the adapted nest position in the light condition.
The border cell is the latest kind of spatial selectivity observed in the
medial temporal lobe. Not much is known about its implications in the
formation of grid cells, although it is diversely distributed along the entorhinal cortex, along with the grid cells (Solstad et al., 2008). Moreover,
they can have strong implication in the formation of place cells as anticipated by Hartley et al. (2000).
2.3
Conjunctive place cells and rate remapping
There are many evidences that place cells are not only spatially driven.
For instance, Wiener et al. (1995) have shown that some place cells were
not fixed to the absolute position in an arena but to task specific locations. This was found by rotating the whole arena but the task specific places. Further work by Deadwyler et al. (1996) have shown that
many CA1 and CA3 cells were coding for distinct task stages in a spatial
delayed-nonmatch-to-sample task. These evidences, reviewed by Muller
(1996), suggest that the location of firing is defined by other aspects and
not purely by the spatial representation.
The observation of rate remapping by Leutgeb et al. (2005) revealed that
place cells are conjointly selective to spatial and nonspatial information.
After making subtle changes in the environment, the researchers observed
that the place cells kept stable place field location but exhibited different
place field peak rate. The authors suggest that this phenomenon might
22
CHAPTER 2. THE HIPPOCAMPUS
A
B
Rate
12
10
8
6
4
2
0
C
E
1.0
PV correlation
D
Population Vector (PV)
Rate
12
10
8
6
4
2
0
CA3
0.8
DG
0.6
0.4
0.2
Morph progression
Figure 2.7: Illustration of the rate remapping phenomenon. (A)
Different arena shapes that provoke gradual rate remapping and illustrative
rate maps for CA3 (B) and DG (C). Red refers to high rate while deep
blue refers to silent. Graph at right is the mean fire rate of the place
fields according to the morphing phase. For DG, upper place field (black)
and bottom place field (gray) are represented. (D) Procedure to compute
the population vector (PV). The rate of all cells at a specific position are
included in sequence in an array. The activity ensemble change is computed
by the mean correlation of the PV of two conditions computed for every
position. (E) Illustration of the PV correlation curve for DG and CA3. CA3
present no difference between 1-step morphing and two successive recordings
of the same arena. Adapted from (Leutgeb et al., 2007).
be the cause of single-cell remapping in task-specific, direction-specific
and trajectory-specific changes in population activity.
Further work by the same group (Leutgeb et al., 2007) has shown that the
change in rate in an ensemble level is coherent with the gradual change
in the environment (Figure 2.7). This shows that place cells are not only
coding for nonspatial information but that they also do it in a gradual
2.3. CONJUNCTIVE PLACE CELLS AND RATE REMAPPING
23
fashion, as the coding for space. Moreover, the change profile of each
place field is independent signifying that nonspatial selectivity is place
and cell specific.
CHAPTER
Computational models of the
medial temporal lobe
In this chapter we will review (not exhaustively) the most relevant models of the medial temporal lobe. In this scope we include models of the
hippocampus, the entorhinal cortex and the interaction between them.
Most of these models are limited to one of the theoretical streams: spatial
selectivity and memory. Although it is not rare to observe some overlapping.
3.1
Computational models of spatial selectivity
Following the discovery of the place cells, there have been many attempts
to model the formation of spatial selectivity in the hippocampus: from
robotic-based systems that use extensive sensory information, path integration and reinforcement learning to theoretical models based on properties of cortical activity. With the discovery of the grid cells, the models
started to link its activity to the place cells which also fostered the cre24
3
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
25
ation of grid cells models.
3.1.1
Computational models of grid cells
There are two main groups of computational models of grid cells: the
oscillatory-interference models (Burgess et al., 2007; Hasselmo, 2008)
predict that the hexagonal pattern emerges in single cells from the interaction of multiple oscillatory inputs; the continuous dynamics models
(McNaughton et al., 2006; Guanella & Verschure, 2006) predict that the
grid patterns emerge from network interaction. These models were extensively reviewed by Giocomo et al. (2011) and more recently by Zilli
(2012).
Oscillatory interference model
The oscillatory interference model is a work by Burgess et al. (2007). It
predicts that the grid pattern is a cellular feature. The model is based
on an oscillatory interference phenomena in which the sum of activity
of high-frequency oscillators creates a low-frequency pattern. The model
predicts that the activity of the grid cell is a product of the synchronization of the intrinsic oscillation of three dendritic subunits (Figure 3.1).
The frequency of each dendritic subunit is modulated periodically by the
integral of the speed in a preferred direction. Speed information is available through the head direction cells (O’Keefe et al., 1998). When the
frequency of the three dendritic subunits is in a high-state it give rise to
the interference phenomena at the theta frequency, generating the place
fields.
In this model, the inter-vertex distance is dependent on the strength of
the modulation of the oscillatory frequency by the speed of the animal.
This variable can be dependent on cellular proprieties such as synaptic
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
26
A
HD CELL 0º
HD CELL 60º
Dendritic
subunit
GRID CELL
HD CELL 120º
B
Dendritic
subunit 0º
Dendritic
subunit 60º
Dendritic
subunit120º
GRID CELL
Figure 3.1: Illustration of the oscillatory interference model. (A)
Each grid cell has three dendritic subunits that receives input from different
head-direction cells with phase intervals of 60. The dendritic subunits have
an intrinsic internal oscillation that is modulated by the integral of the
speed input of the head-direction cells. (B) A grid cells exhibit theta based
activity when the three sub-units have similar frequencies that causes an
oscillatory interference. The movement of the animal changes the frequency
of each dendritic subunit leading to successive active and silent states that
give rise to the grid pattern.
weight that can vary dependent on the region of the medial entorhinal
cortex. Angular offset is determined by the preferred orientation of the
head-direction cells. Position offset can be set by a feedback phase-reset
signal from the downstream place cells.
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
27
Intrinsic persistent spiking model
The intrinsic persistent spiking model is a work of Hasselmo (2008) and
is a slight variation of the oscillatory interference model (Burgess et al.,
2007). In this model the oscillation is produced by the head-direction cells
persistent firing and not on the sub-threshold firing of the entorhinal cells.
The bases on the idea that the activity of the grid cells is determined by
the synchronicity of the head-direction cells upstream (Figure 3.2). The
considered head-direction cells exhibit intrinsic persistent activity with
fixed and uniform frequency. The grid pattern rise from the fact that the
phase of each head-direction cell is modulated by the projection of the
speed of the animal in the preference angle. This speed input does not
change the intrinsic frequency oscillation but only the firing phase. The
total phase shift is proportional to the integral of the projected speed.
The grid cells receive input of three different head direction cells with
phase intervals of 60 which allow the two-dimensional periodic pattern.
In this model, the inter-vertex distance is dependent on the strength by
which the oscillatory phase in the head-direction cells is modulated by
the speed of the rat. As in the oscillatory interference model, this variable can be dependent on cellular proprieties that can vary dependent on
the region of the medial entorhinal cortex. Angular offset is determined
by the preferred orientation of the head-direction cells. Position offset
can be set by a feedback phase-reset signal from the downstream place
cells.
Continuous attractor dynamic models
The second class of grid cells models is based on continuous attractor dynamics. These models predict that the grid pattern is a network feature.
The principle was created by Samsonovich & McNaughton (1997) and
extended by McNaughton et al. (2006) to add spatial periodicity and by
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
28
A
HD CELL 0º
HD CELL 60º
GRID CELL
HD CELL 120º
B
HD CELL 0º
HD CELL 60º
HD CELL 120º
GRID CELL
Figure 3.2: Illustration of the intrinsic persistent spiking model. (A)
Each grid cell receives input of three different head direction cells with phase
intervals of 60. (B) A grid cells is active when the persistent firing of three
head direction cells are synchronized. The movement of the animal changes
the phase of each head direction cell leading to successive synchronization
and dis-synchronization states that give rise to the grid pattern.
Guanella & Verschure (2006) to allow the hexagonal grid pattern.
The model consists of a neural network in which the neurons are organized in a bidimensional topology (Figure 3.3AB). Each neuron is connected to its closest neighbors through an excitatory connection and to
distal neighbors through an inhibitory connection (Figure 3.3C). With
this setup the system always converges into a state in which a stable
bump of energy if formed. The bump of energy can be maneuvered towards a certain direction if the weights of the neighborhood connections
are modulated at this direction (Figure 3.3C). By linking the speed and
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
A
B
C
D
29
Figure 3.3: Illustration of the continuous attractor dynamic models.
Bump of activity (black with high activity and white with low activity) in
the bidimensional topological organization of the entorhinal cortex neural
network with (A) rectangular neighborhood (McNaughton et al., 2006) and
(B) triangular neighborhood (Guanella & Verschure, 2006). In both models the cells in the boundaries are interconnected allowing the formation of
periodic place fields. (C) Bump is maintained by homogeneous lateral connectivity (shown in a linear representation for clarity). (D) Bump of activity
is moved by the modulation of the lateral connectivity in the direction of
the animal motion.
the direction of the movement to the modulation of the weights it is possible to integrate the path by mapping the position of the animal to the
position of the bump.
The hexagonal grid pattern is obtained by the wrapping of the neural
network. If the bump reaches a boundary, it appears in the other side of
30
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
the network. The organization as a twisted torus proposed by Guanella
& Verschure (2006) allowed the formation of the triangular tessellation.
The inter-vertex distance can be linked to the level of modulation of the
weights by the speed vector. Every cell in the same population share
the same inter-vertex distance and angular phase, what confirms experimental observation (Sargolini et al., 2006). The fact that the spatial
frequency of the grid cells changes along the dorsoventral axis of the
medial entorhinal cortex (Hafting et al., 2005) can be explained by the
existence of multiple networks with different gains. Further evidence
comes from the fact that the distribution of spatial frequencies is not
continuous (Barry et al., 2007).
Regarding the spatial phase, the bidimensional organization makes that
each cell have a different spatial phase and that the set of cells is able to
cover the whole phase-space in an uniform fashion.
Another interesting point is that the bump can be set to a specific position by specific excitatory input patterns. Such patterns can be learned
by hebbian rules when interacting with a place cell population. With this
mechanisms the grid cells can be calibrated by the place cells, allowing
the reset of the integrative error.
Mixed cellular-network model
A recent model (Navratilova et al., 2011) was able to merge the cellular
features of the oscillatory-interference models with the network features
of the continuous attractor dynamics model. In its model, a population
of conjunctive grid cells (grid cells whose firing has a preferred angle like
the head-direction cells) is build implementing cellular cellular mechanisms for grid formation. The bump in the continuous attractor network
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
31
is driven by the connections of the population of conjunctive grid cells.
This solves the issue of how the weights could be modulated by speed
and also allow the emergence of phase precession. A similar approach is
provided by Hasselmo & Brandon (2012).
3.1.2
Computational models of place cells
Through time, many groups tried to model the spatial selectivity of the
place cells. Most of these attempts followed two approaches: some use
mathematical modeling to try to predict the place cell activity from
known features of the cortical inputs to the hippocampus; others had
a more technological approach using robots, cameras or graphical simulations of three-dimensional environments to emerge spatial selectivity
from visual input. Some models also produce a synergy between the two
approaches by studying how the two inputs interacts.
There is still a third class of models that is not concerned about the formation of place cells but deals with its implication in behavior. Most of
the models come with navigation strategies that use the cognitive map
to plan the action sequence.
Place cells from border cells
A model of place cells based on the activity of border cells was proposed
by Hartley et al. (2000) from the group of Neil Burgess. Interestingly,
this model was published before the discovery of the border cells (Solstad et al., 2008), which was one of the predicitions of their model. They
defined a special kind of cell named "boundary vector cell" (BVC) whose
firing is selective to the borders in a specific direction (Figure 3.4A). In
the text they even predicted that the border cells would be find in the
32
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
Figure 3.4: Illustration of the place cell model based on border
cells. (A) Example of the typical rate map of a BVC (boundary vector
cell) in three environments. (B) The activity of a place cells is obtained by
the integration (followed by a linear threshold filter) of multiples BVC cells.
entorhinal cortex.
To build the place cell activity from the BVCs, each hippocampal cell
receives input from multiple BVCs and apply a linear threshold filter
(Figure 3.4B). The model could make interesting predictions regarding
the population of place cells, including the transformations in the rate
maps caused by the inclusion of barriers and by geometrical manipula-
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
33
tions of the walls.
In a latter work they also included plasticity in the connection between
the BVCs and the place cells (Barry & Burgess, 2007). Plasticity allowed the reproduction of some experimental findings such as: the slow
disappearance of duplicate place fields produced when a barrier is placed
into a familiar environment (Barry et al., 2006); the fact that place cells
tend to over-represent barriers placed inside of the arena (Rivard et al.,
2004); and the parametric prediction on the change of the place field
distribution after non coherent rotation of multiple visual cues (Fenton
et al., 2000a,b).
Place cells from grid cells
A major breakthrough on the place cells models was the discovery of the
grid cells (Hafting et al., 2005). That’s because the major input to the
hippocampus is the innervation from the entorhinal cortex and, therefore,
the activity of its cells is likely to be essential to place cell formation. Indeed, place cells are less stable after EC lesions (Van Cauter et al., 2008)
and CA1 place fields are more sparse after MEC layer III lesions (Brun
et al., 2008a). All models share the same principles that each place cell
integrates input from multiple grid cells. The models differ in the strategy by which the activity at single place fields is isolated, which might
include different learning rules and network inhibition.
A first model by Solstad et al. (2006) identified a phenomena in which
the overlap of multiple grid cells with different configurations leaded to
an irregular rate map landscape in which single place fields could be
observed. Based on this principle, the place cells could be formed by
the sum of multiple grid cells followed by a linear threshold filter. The
drawback of this model is that the difference between the peaks is highly
34
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
variable, making it hard to settle a threshold value with good overall
fitness. This effect can be reduced by an efficient selection of synaptic
projections, which can be accomplished by means of competitive learning
(Rolls et al., 2006; Si & Treves, 2009). This effect is also present on an
integrate and fire model of place cells with hebbian-learning and competition (Savelli & Knierim, 2010).
The phenomena observed by Solstad et al. (2006) was latter formalized
as part of a moiré interference effect (Blair et al., 2007). It was show the
interference between the grid cells can cause the formation of place fields
of a multiple scales and rotations.
Another interesting group of models is provided by Jeffery (2011). To
explain remapping on the place cells they assume that different grid cell
patterns are created for each dendritic subunit of the hippocampal cells.
By means of contextual gating it is possible that different grid cells drive
the activity of the place cell at different contexts.
Integrative-competitive model
In their recent work, de Almeida and colleagues have identified two mechanisms underlying place cell emergence from grid cells in the dentate
gyrus (de Almeida et al., 2009b) and CA3 (de Almeida et al., 2010): (1)
the integration of multiple inputs with specific synaptic weight distribution (Figure 3.5 top) and (2) a competitive process based on a networkwide feedback inhibition named E%-MAX winner-take-all (de Almeida
et al., 2009a) (Figure 3.5 bottom and 3.6A). The competition is ruled
by a network on inhibitory interneurons that is activated by the hippocampus neurons (Figure 3.6B). Whenever the first spike is detected,
the interneuron network inhibits the hippocampal neurons after a short
activation delay (Figure 3.6C). This latency gives enough time for other
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
+
=
+
35
+
E%-MAX WTA
Figure 3.5: Illustration of how place cells are generated from grid
cells in the integrative-competitive model. (top) The integration of
multiple grid cells will generate a spatial dependent excitation for a granule
cell (bottom-left) . The E%-MAX winner-take-all competition working on
the basis of this excitation will lead to the formation of the place fields
(bottom-right).
neurons with high accumulated excitation to also fire.
The E%-MAX principle defines an energy bottom-limit (measured as the
maximum energy minus 10% (de Almeida et al., 2009a)) that separates
the cells that will be active from the cells that will not. The translation
into rate is due the dynamic nature of the neurons that forces the competitive cycle to run in a defined frequency in the range of gamma. By
assuming a variable initial condition for each cell in each cycle is possible to assume a gradual mapping between the excitation and rate in the
range above the bottom-limit. This mapping supports the emergence of
the place field (de Almeida et al., 2009b).
This approach is follow, at some point, the same integrative principle of
Solstad et al. (2006). The major difference is that the E%-MAX mecha-
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
36
A
CA3
E%-MAX WTA
B
pyramidal/granule cells
-
Dentate Gyrus
+
E%-MAX WTA
inhibitory interneuron
3ms
C
Entorhinal Cortex
potential
cell 1
cell 2
cell 3
IPSP
Figure 3.6: Illustration of the E%-MAX winner-take-all mechanism.
(A) Architecture of the hippocampus network in de Almeida et al. (2009b,
2010). The granule cells in the dentate gryrus receives major convergent
input from the entorhinal cortex. The pyramidal cells receive strong input
from single granule cells and major convergent input from the entrohinal
cortex. DG and CA3 exhibit E%MAX winner-take-all competition. (B)
Competition is caused by the inhibitory interneurons that are capable of
emitting global feedback inhibition (IPSP). (C) The interneurons are triggered after 3 ms of the first spike in the network cycle. Cells that are capable
of reaching threshold during the 3 ms window also produce spikes, the other
cells are inhibited before becoming active. Adapted from de Almeida et al.
(2009a).
nism solves the problem with the fixed linear threshold.
Place cells from sensory input
There are many models that try to explain the formation of place cells
from sensory input. Many of these models are not constraint on biology
but provide a robotic interpretation for the place cells phenomena. For
instance, a computational model of context processing by Balkenius &
Morén (2000) can produce spatial selectivity by considering location as a
specific type of context. Rules of conditioning are used to associate the
3.1. COMPUTATIONAL MODELS OF SPATIAL SELECTIVITY
37
processed stimuli to position in a simulated robot. Further recall of the
location of the robot is possible by comparing the current input vector
with the contexts stored in memory.
Other models try to associate sensory input with the perception of the
motor activity. For instance, in the work of Burgess et al. (1997) a small
robot with an ontop camera is used to associate local visual landmarks
together with a path integration system. Learning is accomplished by
hebbian rules. With this strategy they were able to emerge place cells
like spatial selectivity. Other similar systems obtained the same results
with other strategies for reinforcement learning (Arleo & Gerstner, 2000;
Arleo et al., 2001).
Although vision is the main source of sensory input, other modalities are
also relevante. For instance, the model of Kulvicius et al. (2008) use odor
cues to build a place cell network.
One interesting approach is provided by the model of Verschure et al.
(2006). They used a small robot with a ontop camera to explore a square
area. It is different to the model of Burgess et al. (1997) because no motor
information is used. The robot is steered by a simple random controller
that alternates rotation with forward moves. They specified a neural
network with 5 layers which are hierarchically organized. The layers are
connected by convergent synapses. The layers are also provided with
intra-area lateral connectivity. Upstream layers have more cells which
are more sparsely connected. Downstream layers have less cells which
are fully connected. The neurons are projected as mean-field units with
local memory, i.e. activity is not only defined by the instantaneous input,
it also includes a short trace from the recent input history. This network
receives as its input a retinotopic projection of the camera input. The
network is plastic and follows a learning rule that tries to optimize the
38
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
stability of single units (i.e. minimize the variability of the activity)
and the diversity of activity in the same layer. This setup was able to
emerge place cell like spatial selectivity in the highest layer after training.
3.1.3
Place cell navigation and behavior
There exist many models of robot navigation based on hippocampal place
cells. In general they are used to control small robots or simulated virtual
agents. The key feature of most of these systems is the strategy used to
link action to the activity of place cells.
In the model of Gaussier et al. (2002), place cells are learned with supervision (to the neurons the precise location information is available and
thus it just need to associate the sensory input). A cognitive map is build
by linking the place cells using the delay between their activation during
the learning phase. To allow action decision the system also defines a
"transition cell" that is associated to the action that once took the robot
from one position to the other. With this system in place, navigation
can be obtained by chaining the place cells until the goal is reached and
the actions can be decided on the basis of the needed transitions.
The model of Foster et al. (2000) uses a temporal difference learning
method in two distinct components: an actor-critic strategy and a network that uses temporal difference learning and self-motion information
to acquire consistent spatial coordinates in the environment. The method
was used to solve two navigation tasks: reference memory in the watermaze and a delayed matching-to-place task.
Other models such as (Arleo et al., 2001) are able to learn action-value
functions over a continuous location space, allowing the construction of
3.2. COMPUTATIONAL MODELS OF MEMORY
39
a navigation map.Using a similar strategy and the same Q-Learning algorithm, the model of Kulvicius et al. (2008) used odor-based place cells
to guide navigation.
3.2
3.2.1
Computational models of memory
Pattern completion and attractor dynamics
The CA3 is a major inspiration for the formation of theoretical attractor
dynamics based memory systems. The classic network from (Hopfield,
1982) was inspired on the massive lateral connectivity between the CA3
pyramidal cells and the observation of LTP in these synapses.
The concept behind the attractor dynamics network is that the activity
in the network will always converge to a stable state. There could be
multiple stable states for a single system in a way that depending on the
initial condition of the network the activity will converge to a different
stable pattern, a.k.a. attractor. These stable states can be interpreted as
memories (Marr, 1971). In the case that an incomplete input related to a
memory is presented, the system will converge to the activity related to
the perfect memory input. This process is known as pattern completion.
There are however some requirements to a network system to work like
the theoretical computational models. For instance, the time dynamics
until the convergence is relevant. Also, it is predicted that the activity is
persistent in the neurons. The real CA3 network does not have the same
properties as the theoretical models. For instance, connectivity is far
more sparse and there is no persistent firing given the intrinsic gamma
modulation. However, some of the the theoretical features apply. A series of hippocampal models based on attractor dynamics is presented by
Rolls (2007).
40
CHAPTER 3. COMPUTATIONAL MODELS OF THE MEDIAL
TEMPORAL LOBE
A work by de Almeida et al. (2007) has shown that, when considering the
anatomical aspects of the CA3 network, it is possible to recall memories
within a single gamma cycle. By means of computer simulation they
could also estimate the size of the CA3 memory as 20.000 items.
3.2.2
Memory sequences in the hippocampus
Major evidence for the existence of sequences in the hippocampus is the
replay of previous behavior experience sequences during sleep (Louie &
Wilson, 2001; Lee & Wilson, 2002) and awake states (Foster & Wilson,
2006) .
A model by Lisman (1999) was able to associate sequences with the
hippocampus structure. According to this model, sequences can be produced by the cyclic connectivity between the dentate gyrus and the CA3.
Memories are formed in a single gamma cycle with the pattern completion process in the CA3. The activity of the CA3 neurons is projected
back to the DG, which will cycle back to the CA3 as input to the next
gamma cycle forming the next memory. The initial memory cue is obtained from the entorhinal cortex in the beginning of the theta cycle.
This chaining of theta and gamma cycles allow storage of 7 ± 2 memories
as observed on psychological studies Lisman & Idiart (1995).
CHAPTER
Mechanisms of conjunctive
selectivity in the dentate
gyrus
Our starting point to better understand how place cells become conjunctive starts from the identification of the mechanisms that allow place
selectivity. Much of the work presented in this chapter have major support on a series of papers published by Licurgo de Almeida, Marco Idart
and John Lisman (de Almeida et al., 2009a,b) in which they identified
many of the underlying mechanisms of place field formation in the dentate
gyrus. Our major contribution was to use (and extend) these models to
consider the nonspatial input of the Lateral Entorhinal Cortex. Supporting behavioral evidence comes from the fact that lesion of MEC impair
spatial task and lesion of LEC impair spatial/nonspatial mixed task (Van
Cauter et al., 2012). With this approach we could explain the phenomena
of rate remapping (Leutgeb et al., 2007) and therefore provide a mechanistic description for the conjunctive response of place cells.
This chapter reproduces the paper "The Mechanism of Rate Remapping
42
4
43
Rate map
Population inhibition level
Sum of MEC input
Sum of LEC input
position (x)
position (y)
Figure 4.1: Illustration of the process of place cell formation including LEC and MEC inputs. For each neuron, the excitement is computed
for each position (x, y). Input from LEC is added to the input from MEC.
At each specific position all cells compete through the E%-MAX process
that outputs a population inhibition level. Rate map is build from the
amount of excitation that exceeds the inhibition plane. This process is used
in (Rennó-Costa et al., 2010a) and is analog to the one used in (de Almeida
et al., 2009b, 2010).
in the Dentate Gyrus", which was published at Neuron (Rennó-Costa
et al., 2010a). The abstract reads:
Rate remapping is a recently revealed neural code in which
sensory information modulates the firing rate of hippocampal place cells. The mechanism underlying rate remapping is
unknown. Its characteristic modulation, however, must arise
from the interaction of the two major inputs to the hippocampus, the medial entorhinal cortex (MEC), in which grid cells
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
44
THE DENTATE GYRUS
represent the spatial position of the rat, and the lateral entorhinal cortex (LEC), in which cells represent the sensory
properties of the environment. We have used computational
methods to elucidate the mechanism by which this interaction produces rate remapping. We show that the convergence
of LEC and MEC inputs, in conjunction with a competitive
network process mediated by feedback inhibition, can account
quantitatively for this phenomenon. The same principle accounts for the independent variation of the place fields of the
same cell as its sensory information is altered. Our results
show that rate remapping is a novel code that can be explained in terms of known mechanisms.
4.1
Introduction
Early work on the receptive field properties of rat hippocampal cells
showed that their firing depends strongly on the rat’s location (O’Keefe
& Dostrovsky, 1971). Indeed, their activity is generally restricted to one
or several small regions of the environment called place fields. However,
the hippocampus is also a storage site for non-spatial information (Wood
et al., 1999; Rolls et al., 2005) so such information must somehow be
represented. The fact that the spatial properties of hippocampal firing is
modulated by manipulations of sensory cues (O’Keefe & Conway, 1978;
Muller et al., 1991) and behavioral context (Wood et al., 2000) indicates
that both spatial and non-spatial information are sharing the same neural
structures and are likely to use a single common coding scheme. Recent
work explored this question using a procedure in which the shape of the
environment’s walls were slowly morphed from square to round (or vice
versa), thereby changing their sensory qualities. It was found that such
morphing changed the rate of firing of individual place cells, either upwards or downwards, a phenomenon called “rate remapping” (Leutgeb
4.1. INTRODUCTION
45
et al., 2005, 2007). Moreover, different place fields of the same cell can
change upwards and downward independently. Thus, coding is not a
cellular property, but the property of individual fields, each of which represents a separate conjunction of spatial and sensory information. This
form of coding has not been previously observed in the brain and is very
different from how sensory information is encoded in inferotemporal cortex, where cells represent specific sensory constructs, largely independent
of their spatial position (Hung et al., 2005). Rate remapping, in contrast,
permits the distinct representation of sensory events while maintaining
the integrity of a code for spatial location. The mechanism underlying
rate remapping has not been previously addressed.
The hippocampus receives inputs from two regions of the entorhinal cortex (EC). One input is the medial entorhinal cortex (MEC), a region that
contains grid cells of varying spatial frequency, orientation and phase
(Hafting et al., 2005). The axons of many such cells converge on the
dendrites of the granule cells of the dentate gyrus (DG), the first-order
processing stage of the hippocampus. These granule cells show one or
more place fields (Leutgeb et al., 2007). A previous computational study
indicates that the summation of excitatory input from MEC grid cells,
in conjunction with feedback inhibition from the dentate network, is sufficient to account for the spatially specific firing pattern of granule cells
(de Almeida et al., 2009b). Moreover, this study showed that the realignment of the MEC grid cell population automatically makes the granule
cells globally remap, as observed experimentally (Leutgeb et al., 2005,
2007). However, this mechanism alone cannot account for rate remapping because the MEC input itself does not change during environmental
morphing (Leutgeb et al., 2007; Fyhn et al., 2007). Several lines of evidence indicate that sensory information about the environment is brought
to the hippocampus by input from the lateral entorhinal cortex (LEC):
in rodents, this region is itself driven by sensory related areas including
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
46
THE DENTATE GYRUS
almost exclusive direct inputs from the ventral visual processing pathways of the occipitotemporal cortex (McDonald & Mascagni, 1996), the
olfactory bulb (Carlsen et al., 1982) and indirect sensory input from area
35 of the perirhinal cortex (Burwell & Amaral, 1998; Burwell, 2000).
Consistent with the sensory role of LEC, lesion of this region produces
decreased investigation of novel objects (Myhrer, 1988). Furthermore,
direct recordings from the LEC exhibit a spatial response with low selectivity, indicating the influence of the sensory (non-spatial) drive (Hargreaves et al., 2005). The inputs from the LEC converge with those from
the MEC onto all granule cells of the DG. Since the LEC and MEC constitute the main source of the extra hippocampal input to the DG, it is
this convergence that must somehow account for the rate remapping of
DG cells. We have used computational methods to study the effects of
these inputs from the EC onto the DG and have sought to answer two
main questions: (1) What is the mechanism of rate remapping? (2) Why
do different place fields of the same DG cell display independent rate
remapping?
4.2
Results
We simulated the response of DG cells to inputs from MEC and LEC in
the following way. The spatial response (rate maps) of the grid cells were
modeled as previously described (Blair et al., 2007; de Almeida et al.,
2009b) and, in accord with data (Leutgeb et al., 2007), were made insensitive to morphing. 10 examples of such cells are shown in Figure
4.2a. LEC cells were modeled to be consistent with the finding (Hargreaves et al., 2005) that the firing rate of these cells carries little, but
not zero, information about the position of the rat (Figure 4.2c, t =
0.9957, P = 2e−7 ). To account for the sensory consequences of morphing
on LEC, we assumed that the spatial response of each cell is switched
from one map to an independently generated one at some random point
4.2. RESULTS
47
during morphing (different assumptions are examined in Supplemental
Text, Figure 4.3). The resulting receptive fields are shown for 10 LEC
cells in Figure 4.2b. In order to approximate the response dynamics of
the EC during environment morphing we generated the rate maps for
both LEC and MEC (10,000 neurons each). To compute the excitatory
input to each individual DG neuron we used a realistic number of inputs
(1200 from the MEC and 1500 from LEC; see Methods) and summed
them. Each synaptic input to the DG was taken from a population of
randomly chosen entorhinal neurons, with the synaptic weight randomly
assigned according to the synaptic weight distribution derived from the
distribution of synapse sizes (de Almeida et al., 2009b) as determined
by serial EM (Trommald & Hulleberg, 1997). The spatial distribution
of firing of 10,000 DG granule cells was computed by applying, at each
position, a winner-take-all interaction over the sum of excitation input.
This winner-take-all process is governed by the so called, E%-max principle (de Almeida et al., 2009a) derived from the interaction of excitation
with gamma frequency feedback inhibition, a form of inhibition known
to exist in this brain (Bragin et al., 1995; Towers et al., 2002; Pöschel
et al., 2002) and that synchronizes the firing of DG cells (reviewed by
Bartos et al. (2007). According to this principle, the level of inhibition is
set such that cells will fire provided their excitation is within 10% of the
cell with maximum excitation. For these cells, their rate is proportional
to where they fall in this 10% range. The value of 10% is computed from
d/τm (de Almeida et al., 2009a), where d = delay of feedback inhibition
and τm = membrane time constant, both of which have been experimentally determined. A previous study showed that the interaction of MEC
input with this form of inhibition is able to quantitatively account for the
size and number of place fields exhibited by active DG cells (de Almeida
et al., 2009b). In our simulations, we also take into consideration the
LEC. The interaction of the two inputs depends on the ratio (α) of the
mean drive of MEC and LEC onto EC. No data is available that would
allow us to directly estimate α. However, our results provide for a quan-
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
48
THE DENTATE GYRUS
titative estimate of its value (see below).
With this simulation framework in place, we investigated whether the
cumulative decorrelation of population output from the dentate gyrus
observed during progressive morphing of the arena shape (Leutgeb et al.,
2007) (PV correlation curve, Figure 3a) could be explained by the changes
of the LEC spatial response. We computed the correlation between composite population vectors (see Methods) as a function of morphing stage
throughout over a range of α and compared this with the correlations
reported by Leutgeb et al. (2007) (Fig1d). To account for the variability of the firing rate in consecutive recordings under the same conditions
(Hargreaves et al., 2005; Leutgeb et al., 2007; Fyhn et al., 2007), we emulated the effect of under-sampling of the space, an unavoidable condition
given the experimental protocols. To account for the effect of undersampling, we introduced a stochastic factor in every comparison with a
variance dependent on the rate (see Methods). The level of the correction
was obtained by fitting to the experimental data (PV correlation) of two
subsequent recordings obtained under the same condition (Figure 4.8).
We observed an exponential-like decay shape for the correlation curves
with the global level of decorrelation monotonically and positively affected by the level of influence of the LEC input (regulated by α). A
value of (α=0.32, Figure 4.2) gave the best fit. With the value of α determined, we could then examine how morphing affected rate remapping.
First, we investigated whether the simulated place fields have properties that match those experimentally observed. We found that simulated
granule cells have multiple place fields (average of 2.2 place fields) and
have a mean place field size of 943cm2 . The distribution of the number
of place fields in each active cell was similar to experimental measurements (Figure 4.2e, t = 0.98, P < 0.0005). The place field size is also
in accord with data (analysis of (Leutgeb et al., 2007) by de Almeida
4.2. RESULTS
49
Figure 4.2: MEC and LEC inputs and estimation of model parameters. (a) Example of the 10 MEC modeled rate maps (number is the
maximum firing rate). MEC rate maps remain constant during morphing.
(b) Example of the 10 LEC rate maps from experimental data (H, maximum rate when informed, adapted from (Hargreaves et al., 2005) and 10
from the model for the two environments (square and round, maximum rate
in both environments). Rate maps presented with equally distributed spatial score (ranked from right to left). (c) Histogram of spatial information
score from LEC rate maps (H, experimental data and square, model. correlation 0.9957, P < 0.05). (d) Ratio (α) between the mean firing rates in
MEC and LEC estimated as 0.32 by fitting to the experimentally observed
reduction on spatial coincidence using population vector correlation as the
environment is morphed (Leutgeb et al., 2007). (e) Histogram of the number
of place fields found in DG cells (Leutgeb and square environment). Stable high correlation between experimental and simulated histograms during
morphing indicates that modification in LEC activity do not disrupt place
field formation (R = 0.98).
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
50
THE DENTATE GYRUS
α
PV Correlation
0.9
0.9 MEC
0.7
0.7
0.5
0.5
0.3
0%
Interpolated
Abrupt (a = 0.32)
[Leutgeb 2007]
0.3
0.2
0.0 LEC
17% 33% 50% 67% 83% 100%
Morphing progression
Figure 4.3: Difference in spatial coincidence reduction rate for
abrupt and linear interpolated morphing of LEC spatial response.
Comparison of the mean population vector (PV) during remapping compared with (Leutgeb et al., 2007). For 17% of morphing, the minimal correlation value is 0.82 ± 0.01 compared to 0.75 observed experimentally.
et al. (2009b)). We also tested whether the observed restricted diversity
of grid cell activity (Barry et al., 2007) affects the results of our simulation. When the grid cell proprieties were limited to one orientation
and three grades of spacing, no significant difference in the distribution
of the number of place fields (Wilcoxon, p = 0.65) or the PV correlation
(Student t-test two-tailed, p = 0.31) was found. These results are not
unexpected given previous work showing that MEC input alone can account for these properties; what is added here is the demonstration that
the LEC inputs, when included in the model, do not interfere with place
cell formation in the DG by the MEC inputs.
We next directly compared the remapping of individual place fields of our
simulation of morphing with the results obtained by Leutgeb et al. (2007)
4.2. RESULTS
51
(Figure 4.4a). The experimental results show that all place fields of the
same cell remap and do so independently; thus one field may increase its
firing rate during morphing while the other decreases its rate. Figure 4.4b
shows this to be similarly true in our simulated place fields. Moreover,
the relative proportion of remapping patterns (linear, quadratic and sigmoidal) could not be distinguished from the experimental observations
(Figure 4.4c, t = 0.93, not significant (n.s.)).
To obtain insight into why remapping is independent for different place
fields of the same cell, we analyzed the changes during morphing (Figure
4.5). We identified two processes that cause independent place field rate
remapping: (A) the effect of morphing on LEC cells changes the direct
excitation of the granule cells (Figure 4.5A). Since the rate change of
LEC cells due to morphing is a function of position, the variation on the
integration of the LEC excitatory input is independent for each place
field; (B) the change of the excitation of other cells will determine which
cell is most activate at a given position. This determines the E% max
level and thereby indirectly, via inhibition, alters the rate of other cells
(Figure 4.5B). This process is localized and therefore independent for
each place field.
To determine which mechanism prevails in controlling rate remapping,
i.e. excitatory drive versus inhibitory competition, we looked for the ratio between the levels of remapping accounted for by each mechanism
(see Methods). We observed that both mechanisms contribute to almost
all place fields, with a slight dominance of mechanism A (Figure 4.6).
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
52
THE DENTATE GYRUS
Figure 4.4: Simulated DG cells exhibit independent place field rate
remapping, as observed experimentally. Differential rate changes in
individual firing fields of cells from the dentate gyrus during progressive
maneuvering of the walls of the arena. (a) Recorded cells. Adapted from
(Leutgeb et al., 2007). (b) Simulated cells. Individual fields are numerically labeled to relate to the respective line diagram of the mean field rate.
The rate curves were fitted to linear (red), quadratic (green) or sigmoid
(blue) functions and are shown when significant (p < 0.05, dotted line). (c)
Histogram of the best fit classification for recorded and simulated curves.
Correlation between histograms is of 0.9543 (P = 0.045).
4.3
Discussion
Rate remapping is a new form of coding, the mechanism of which has
been unclear. We have found that it can be explained in terms of simple processes: the summation of several thousand LEC and MEC inputs
to DG cells, in conjunction with a network process that produces com-
4.3. DISCUSSION
53
Figure 4.5: Different mechanisms for independent rate remapping
of different place fields of the same cell. (A) Rate is directly affected
by changes of the input drive. For a given cell, morphing (round to square)
induces localized variation of LEC input, changing the rate of each place
field independently (At PF1 , elevation of input drive (INPUT1 ) causes the
rise of rate (RATE1 ). At PF2 , the fall of the input level (INPUT2 ) leads to
reduction of rate (RATE2 )). In this case, remapping is only caused by the
change of the input since the global inhibition level does not vary (dotted red
line); (B) Rate is inversely affected by changes of the inhibition. Morphing
induces localized variation of the global inhibition level, changing the rate
of each place field independently (At PF1 , the raise of the global inhibition
level (INH3 ) causes the decay of the rate (RATE3 ). At PF2 , the fall of the
global inhibition level (INH4 ) causes the rise of the rate (RATE4 )). In this
case, remapping is only caused by the local changes on the global inhibition
level since all inputs to this cell remain in the same level during remapping.
The change of the inhibition level is caused by variations of the input drive
of the most excited cell. For each cell and wall shape a rate map is shown
with the relevant place fields indicated by a white circle and the process
values used of the computation of the rate at these place fields: the sum
of entorhinal input (light gray bar for LEC and dark gray bar for MEC);
the sum of entorhinal input of the most excited cell (red line); the global
inhibition level (dotted red line) and the rate (black bar).
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
54
THE DENTATE GYRUS
Figure 4.6: Distribution of the mechanism balance ratio through
active place fields. For clarification see Methods. Low ratio indicates
prevalence of first mechanism (Figure 4.5A) while high ratio indicates that
second mechanism (Figure 4.5B) is more effective.
petitive inhibition. These mechanisms are sufficient to explain the key
observation, that even though the LEC input to the DG is not restricted
to specific positions, virtually all DG cells have place fields. Our simulations show that the spatial firing pattern of DG cells is determined
primarily by the MEC inputs; the role of the LEC is to determine the
specific rate at which place cells fire. In addition to accounting for these
findings, our model elucidates several other properties, notably the size
of place fields, the average number of place fields, and the fact that if DG
cells have multiple place fields, these vary independently during morphing of the environment. Other models have investigated the integration
of input from LEC and MEC in the DG (Hayman & Jeffery, 2008; Si &
Treves, 2009) and provided some insights that are consistent with our results. However our model is the first to attempt to quantitatively account
for rate remapping (for a comparison of models, see Supplementary Text).
The mechanism of rate remapping can be understood intuitively in terms
of the summation of LEC and MEC inputs and the strong competition
for firing in DG produced by the DG inhibitory network (Figure 4.5).
4.3. DISCUSSION
55
In this context, the strength of an input is defined by the presynaptic
activity of the neurons of the entorhinal cortex and the strength of the
synapses they form onto granule cells. If only the most excited cells can
fire, then cells with both strong LEC and MEC input will have great
advantage in this competition. Thus, only cells that have strong MEC
inputs, and are thus “successful” place cells, can express the additional
input from the LEC. Conversely, cells that have strong LEC input, but
weak MEC input, and which could therefore express properties of the
sensory world largely independent of place, are unlikely to be winners.
This explains why cells that solely code sensory information, like those
in the LEC and IT cortex, are very rare in the DG. This implies that the
representation of the environment, as conveyed by LEC, is mixed in the
DG with the spatial metric imposed by MEC.
Although convergence and competition are keys to understanding the
mechanism of rate remapping, two additional factors should be noted.
First, the number of inputs into a single DG cell from both LEC and
MEC are large (>1000) and therefore not subject to large statistical
fluctuations. If the number were much smaller, it might often arise by
chance that significant numbers of DG cells received strong enough LEC
input to win the competition even with negligible MEC input, contrary to
what is observed (see Supplemental Text, Figure 4.7). Second, spatial encoding is unique because the organism is always at a place; i.e. the MEC
is always active and formation of grid cells is not impaired by darkness
(Hafting et al., 2005). In contrast, information from any specific sensory
modality in the LEC may be present or not at any point in time. Because
place is always present, other sensory information can never compete by
itself for influence over the DG; the competition is always influenced by
MEC input. It may happen that sensory input affects the properties of
the grid cells when grids realign to distal cues (Sargolini et al., 2006),
but such changes only occur during global remapping, which is outside
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
56
THE DENTATE GYRUS
the scope of this study.
The mechanism we propose for rate remapping depends on the interaction of the LEC and MEC. This interaction depends quantitatively on the
relative magnitude of the two inputs (α), which according to our analysis
should be in the range of 0.2 − 0.3. Importantly, modification of α provides a way of testing the proposed model of rate remapping. Specifically,
(1) the mean population vector correlation produced by morphing should
monotonically increase with α (Figure 4.2d) and (2) the mean place field
size should monotonically decrease with α (Figure 4.7d). With the advent of molecular methods for altering firing rates or synaptic strengths
in a region specific manner, it should become possible to directly test
these predictions.
Previous studies have shown that multiple place fields of single DG neurons emerge from the mechanism considered here using inputs from MEC
only (de Almeida et al., 2009b). Our simulations show that this phenomenon still holds when inputs from both MEC and LEC are considered. What emerges from our analysis is that simple random summation
of the inputs and competition among DG cells is sufficient to form place
fields, but not selective enough to form only one; i.e. multiple place fields
is the best the system can do in decoding the highly distributed grid cell
input. The emergence of cells with single place fields, as occurs in CA3,
requires an additional processing step (de Almeida et al., 2010).
The independence of the rate remapping observed in the multiple place
fields of single DG cells (Leutgeb et al., 2007) constitutes a novel form
of neural code. In this code the DG neuron multiplexes multiple independent features that are selected on the basis of a spatial metric. The
independence emerges because both excitation and inhibition vary with
spatial location. Rate remapping is different from other rate codes in
4.3. DISCUSSION
57
the brain that are selective for multiple features, as for instance, the
combined spatial frequency and orientation tuning curves found in single
neurons of the primary visual cortex (De Valois & De Valois, 1990). The
overall response of these V1 cells can be explained by the multiplication
of tuning curves that, in contrast to the rate remapping in the DG, are
fixed and invariant to any other feature change (Mazer et al., 2002). The
independent (nonmultiplicative) modulation of the place fields of single
DG neurons promotes orthogonalization of the encoding that is required
to generate the highly specific responses to single locations found in CA3
(Leutgeb et al., 2007).
Our results answer some questions about this code, but other important
questions remain. A defining feature of this code is that the firing rate is
not binary. Thus, a particular memory is represented not only by which
cells fire, but also by the firing rates. Now consider the process of pattern
completion for n cells with rates R1, R2. . . Rn. Suppose a partial cue is
presented, say R1 to R5. This should lead to the firing of unstimulated
cells at their appropriate graded rates. Indeed, there are attractor network models that use graded rather than binary rates (Rolls, 2007), and
it will be interesting to see if these can account quantitatively for pattern
completion in CA3. Another unanswered question is where and how rate
remapping is decoded so that cortical cells, which do not code sensory
information using spatially specific cells, can decode information (such
as during replay) that they receive from the hippocampus.
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
58
THE DENTATE GYRUS
4.4
4.4.1
Supplemental material
Alternative assumptions about how the LEC
responds to morphing
In the main text we assumed that the spatial response of LEC cells in
two arenas with different shapes is uncorrelated. We also assumed that
during morphing, the spatial response is abruptly changed at some random point during morphing. To test whether other assumptions would
lead to similar results, we analyzed two alternatives: (1) the spatial response of LEC cells is highly correlated in the two arena shapes, having
only the mean fire rate varying with the sensory condition; (2) the spatial
response of LEC varies linearly with morphing by means of interpolation.
In the first alternative, a base rate map was created invariant to morphing for each cell (see Methods). To the base spatial response was added
a positive bias that linearly relates to the morphing degree, having its
value at the extremities of morphing defined randomly. This preserves
the spatial correlation of the spatial map but changes the mean drive of
each cell. With this form of LEC encoding, the minimum PV correlation
value obtained (0.71 ± 0.01) was significantly higher than the minimum
experimental value (0.32), rejecting this class of information coding in
LEC.
In the second alternative, two base rate maps were created for each cell
exactly as in the main text. However, instead of changing abruptly which
rate map is effective at a random point of morphing, the effective rate
map was a linear interpolation of the two base rate maps following the
morphing stage. The change in rate was proportional to the degree of
morphing, making this a form of rate remapping in the LEC. Although
the PV correlation values were within the experimental range of the two
extremes of the morphing, the inclination of the curve was significantly
4.4. SUPPLEMENTAL MATERIAL
59
different; it showed a smaller decay of the PV correlation of the first stage
of the morphing (Figure 4.3).
4.4.2
Differences in how DG and LEC encode sensory
information
To test whether DG cells have a similar representation than LEC cells
we examined how the LEC and DG cells code sensory and spatial information. Using as a metric for the amount of information that each
spike carries about the position and the shape of the arena (Skaggs et al.,
1993) we observed two clusters (Figure 4.7a). DG cells presented significantly higher spatial information score (Figure 4.7b, DG, median = 2.02
bits/spike, LEC, median = 0.16 bit/spike, Mann-Whitney P < 0.05 twotailed) and higher shape information score (Figure 4.7c, DG, median =
0.14 bit/spike, LEC, median = 0.09 bit/spike, Mann-Whitney P < 0.05
two-tailed). This illustrates that LEC cells do not impose their character
on hippocampal cells (there are no “round” or “square” cells that fire over
a large fraction of the environment). Rather the firing of hippocampal
cells represents sensory information in a very spatially restricted fashion.
4.4.3
Comparison to other models
The integration of LEC and MEC and both types of remapping that occur in the hippocampus have been the target of previous modeling efforts.
Si & Treves (2009) presented a model in which place fields are formed
by applying a competitive Hebbian learning strategy to a postsynaptic
target driven by a population of simulated grid cells. They successively
produce place fields with this strategy and, by including the LEC input
as an overall rate bias, both papers reached similar conclusions conclusions on how the proprieties of place fields are affected: larger place fields
(Figure 4.7d) and a higher number of place fields (data not shown). This
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
60
THE DENTATE GYRUS
model, unlike the one presented here, however, does not consider any specific anatomical constraints and requires learning. Moreover, the authors
do not mention how remapping affects their model, but they probably
can simulate some aspects of rate remapping by changing the values of
their LEC input in a mechanism that is close to our mechanism A. Their
model cannot reproduce the independent remapping of different place
fields of the same cell because their LEC input is uniform with respect to
position and thus cannot account for the data of Hargreaves et al. (2005).
The model of Savelli & Knierim (2010) is also based on competitive Hebbian learning. It, like our previous work (de Almeida et al., 2009b) shows
how global remapping could be produced by changing the alignment of
the MEC grid cells, as observed by Fyhn et al. (2007). The drawback
of the model of Savelli & Knierim (2010) is that, whenever the MEC
grid cells realign, the plasticity mechanism changes the synaptic weights.
This causes that when the rat is placed back to a previously known environment, “even if the grid cells fire in the same locations as in the first
exploration”, “the model would produce a different set of place fields”,
which contradicts the experimental observations that the original set of
place fields is recovered once the rat is placed back in the familiar environment (Leutgeb et al., 2005; Fyhn et al., 2007). Since no plasticity is
involved in our model, one specific grid cell alignment will always generate the same set of place cells.
Hayman & Jeffery (2008) suggested that LEC and MEC inputs could
be clustered on separate branches of the dendritic trees of the granule
cells of the DG. In this model, each branch would integrate the activity of some subset of aligned grid cells, generating a place cell response.
Subsequently the input from LEC would gate which branches are active.
This model succeeds qualitatively in explaining some aspect of (partial)
rate remapping. The gradual change of the LEC input could perform an
4.5. EXPERIMENTAL PROCEDURES
61
interpolation of several rate maps associated with a single cell, creating
the rate remapping effect. However, such models have some limitations:
(1) the diversity of contexts that evoke global remapping represented by
each cell is limited by the number of clusters it can have; (2) the number
of clusters is limited by the number of branches in the dendritic tree; (3)
in the advent of a new environment, it might be required that one cell
forgets previous clusters or that new cells are recruited. The first case
will lead to the same dilemma as for the model discussed in the previous
paragraph, while the second case has no experimental support (Leutgeb
et al., 2007). In contrast, in our model there is practically no limitation
to the diversity of contexts and the recruitment of new cells is in accordance to the literature (de Almeida et al., 2009b).
4.5
Experimental procedures
Spatial response representation
All data was simulated for a 1 m square enclosure with a resolution of
1 cm2 , compromising 10000 square bins organized in a 100 x 100 rectangular grid. The spatial response for each cell of all considered cortical
regions was composed by rate values assigned for each bin, defining a
rate map.
Simulation of spatial response from entorhinal cortex
MEC spatial response was set invariant to the morphing of the environment, being simulated only once. The rate (λ) of each MEC cell follows
the equation defined by Blair et al. (2007) and is a function of the Cartesian position (r = (x, y)) and subject to the following parameters: the
place field decay constant (a, normally distributed with 0.55 ± 0.03), the
inter-vertex distance (d, ranging from 30 to 100 cm), the spatial offset (c
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
62
THE DENTATE GYRUS
= (x0 , y0 ), ranging from (0, 0) to (d, d)) and the angular offset (θ, from
0o to 60o ):
4π
√ (cos(θk +θ),sin(θk +θ))·(r−c)
k=1
3d
!
a·
λ(r, a, d, c, θ) = e
Pa
−
3
2
!
− 1 (4.1)
The vertex angles (θ1 = −30o , θ2 = +30o and θ3 = +90o ) define a honeycomb grid that bases the formation of the grid cell firing. We simulated
the spatial response of 10000 MEC cells, each of them with a random
parametric set within the range specified above.
LEC spatial response was set dependent to the degree of morphing (v).
Indeed, morphing was incorporated in the model by changing the spatial
response of LEC cells. For each LEC cell there were assigned one rate
map for the beginning and another for the end of the morphing, each
of them generated independently (following the methods below). For
the intermediate morphing steps, it was defined a random (uniformly
distributed) transition morphing degree for each cell in a way that the
spatial response of the cell is invariant from the beginning to this point
and from this point to the end.
To synthesize the LEC rate maps, the arena was divided in a 5x5 grid that
covers the whole arena. For each rate map, these regions were randomly
separated in two groups (active or inactive) according to the expected
spatial information score (high spatial specificity renders less active regions). A base rate map is built by assigning a random rate value within
the range [0, 0.5] for non-active regions and [0.5, 1] for active regions.
To obtain the final map of LEC responses we convolved the base map
with a Gaussian kernel with standard deviation of 17 bins. We simulated
4.5. EXPERIMENTAL PROCEDURES
63
the spatial response of 10,000 LEC cells by using the number of active
regions to fit to the experimental spatial information score (Hargreaves
et al., 2005). Samples of LEC rate maps and the spatial information
score histogram are shown in Figure 4.2b and Figure 4.2c respectively.
LEC and MEC spatial responses had the population mean average rate
normalized. Since we could not obtain information about the relative
mean fire rate of MEC and LEC populations, we had the ratio parameterized by α in the range [0, 1] when the rates were integrated in the
computation of the excitatory input of the granule cell.
Granule cells
Each granule cell integrates the excitatory input received from a random
group of MEC and LEC cells following the estimated convergence (see
below). The sum of entorhinal input of each granule cell (I) is specific for
each position, which allow a map representation. The excitatory input is
the product of the rate (λ) of the afferent cell with the specific synaptic
weight (W , see below):
Iiv (r) = α
M
EC
X
j
λj (r) · Wij + (1 − α)
LEC
X
λvk (r) · Wik
(4.2)
k
The rate of granule cells is defined by competition of the sum of the
entorhinal input within the population ruled by a percentage of maximal
suprathreshold excitation (E%-max) winner-take-all process (de Almeida
et al., 2009a), measured as 10%. At a specific position and arena shape,
the amount of inhibition is equal to 90% of the sum of the entorhinal
input of the most excited cell in the population. Whenever this global
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
64
THE DENTATE GYRUS
inhibition is higher than the sum of entorhinal inputs of a specific cell,
this cell remains silent. Otherwise, the rate of the cell is the difference
between excitation and inhibition.
v
v
DG
v
λvi (r) = Iiv (r) − 0.9 · maxDG
I
(r)
·H
I
(r)
−
0.9
·
max
I
(r)
j
j
i
j
j
(4.3)
, where H is a Heaviside function.
Figure 4.3 gives an insight of how granule cells rate map is obtained from
grid cells and LEC cells and how rate is influenced by both the input of
entorhinal input of the cell and by the population inhibition.
Convergence from entorhinal cortex
The convergence of the entorhinal cortex input onto granule cells was
estimated by the number of synapses as 1200 for grid cells (de Almeida
et al., 2009b) and following the same procedure as 1500 for LEC inputs
(see Supplementary Methods).
Synaptic weight
Synaptic weight (W ) is defined by the synaptic size (s) (de Almeida et al.,
2009b):
W (s) =
s
s
0.2 s + 0.0314
(4.4)
The synaptic size distribution was defined by the measured size distribution of excitatory synapses onto granules cells (Trommald & Hulleberg,
1997):
4.5. EXPERIMENTAL PROCEDURES

−
P (s) = 100.7 1 − e
65
s 
s   s −
−
0.022  · e 0.018 + 0.02 · e 0.15 
(4.5)
, s ranges from 0 to 0.2.
Data analysis
Cells with average firing rate above 10% of the mean average firing rate
of cell population were considered active.
Composite population vector correlation
Composite population vector (PV) correlation has been used in the analysis of experimental data to observe the reduction of rate coincidence at
the same position in the dentate gyrus when the shape of the arena is
morphed (Leutgeb et al., 2007). PVs are obtained by storing in a vector
the rate at a certain position bin of each cell of a population. The correlation between the PV of the same group on two different conditions give
a measure of how the condition affects the overall population activity.
The PV correlation value is the mean correlation value considering all
positions bins.
Place field analysis
The number of place fields was estimated from the rate map for active
cells in each stage of the morphing. Rate maps were smoothed by a
Gaussian kernel with 9 pixels radius. Pixels with firing rate above 20%
of the peak rate were considered active. Groups of contiguous active
pixels (> 200 and < 2500pixels) with average rate exceeding the mean
population fire rate and with peak activity above two times the mean
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
66
THE DENTATE GYRUS
population fire rate were considered to be a firing field.
Curve fit
Persistent place fields were obtained by applying place field analysis on
the average rate map for all morphing shapes (Leutgeb et al., 2007).
There different curves were fit to the in-field rate for each persistent
place field following the morphing: (a) linear regression, (b) quadratic
regression and (c) sigmoid function. Fits with p values < 0.05 were considered significant, and each place field was assigned to the category with
the highest explained variance (F values).
Rate remapping measures
The level of the rate remapping effect is measured for each persistent
place field (p) whose average mean rate for the two extreme shapes of
the morphing (λSR ) is above 10% of the mean average firing rate of the
cell population. The rate remapping level (ηR ) is defined as the absolute
difference in firing rate normalized by λSR . The level of rate remapping
due to mechanism A (ηA ), which is based on the change of the sum of
direct excitatory inputs, is the absolute difference in the mean sum of the
input at the positions of the place field normalized by λSR . The level of
rate remapping due to mechanism B (ηB ), which is based on the change
in the level of inhibition, is the absolute difference in the mean global
inhibition level at the positions of the place field, normalized by λSR .
The ratio of the impact of the two mechanisms (γ) is ηB divided by ηA
+ ηB .
Pr⊂p
ηR (p) =
r
|λ1i (r) − λ0i (r)|
Pr⊂p
λSR
r
(4.6)
4.5. EXPERIMENTAL PROCEDURES
67
Pr⊂p
ηA (p) =
Pr⊂p
ηB (p) =
r
r
|Ii1 (r) − Ii0 (r)|
Pr⊂p
λSR
r
1
DG 0
|0.9 · maxDG
j (Ij (r)) − 0.9 · maxj (Ij (r))|
Pr⊂p SR
λ
r
γ(p) =
ηB
ηA + ηB
(4.7)
(4.8)
(4.9)
Estimation of the convergence from entorhinal cortex
For the projection from the grid cells the number of synapses has been
previously estimated as 1200 (de Almeida et al., 2009b). The number of
synapses made by LEC cells onto granule cells can be estimated following the same methods: granule cells have 3000 µm (Johnston & Amaral,
1998) of dendrite and spine density of 2.3 spines/µm (Johnston et al.
1998); Each spine has one synapse; There are thus 6840 synapses; The
middle molecular layer has 3̃0% of the dendrite area while the distal layer
has 25% (Hama et al. 1989); 85% of the synapses receive input from
the layer 2 of the entorhinal cortex (Nafstad, 1967); Since the fraction of
silent synapses is small in this cell type (Min et al., 1998) there are 1500
LEC inputs.
Emulation of experimental condition
Simulated rate maps take into consideration mean fire rates that are obtained by a deterministic mechanism and therefore are always the same.
Under experimental conditions, however, the observed rate is subject to
a variance caused by non-uniform sampling of the space and the non
deterministic nature of spiking neurons. This is the most straightforward explanation for why two successive rate maps obtained under the
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
68
THE DENTATE GYRUS
same environmental conditions are not always the same, as observed for
LEC and MEC rate maps (Hargreaves et al., 2005) and for the rate of
grid fields (Fyhn et al., 2007). In Leutgeb et al. (2007), two successive
recordings of the same neural population showed a PV correlation vector
of 0.87. To emulate this experimental factor in the calculation of the
PV correlation of simulated data, we used random rates from a normal
distribution with the model rate value as mean and a specific variance.
The variance (var(r)) of a certain bin was set to be proportional to the
mean rate (< λ(r) >) at this position, since the non-deterministic event
observed is spikes and therefore the number of probabilistic events is
proportional to the rate, and to an experimental factor (β):
var(r) = β · hλ(r)i
(4.10)
To determine β for the experimental conditions of Leutgeb et al. (2007),
we fit the PV correlation value for two rate map groups for the same
morphing stage (Figure 4.8). We observed that β varied with the number of cells used, with the PV correlation and with α.
Spatial and shape information score
The spatial information score is a reliable measure of the sharpness of
spatial tuning of a spike train (Skaggs et al., 1996). The spatial information score has been derived from communication theory by considering
that a cell is a communication channel with the rat’s location as input
and cell’s spike activity as output, assuming that all information about
position is encoded by the fire rate (Skaggs et al., 1993). The spatial information score of rate map depends on the occupancy probability (p(r),
sampling time at position r relative to the total sampling time), the rate
at each position (λ(r)) and the mean rate (< λ >):
4.5. EXPERIMENTAL PROCEDURES
SpatialInf ormation =
bins
X
69
p(r)
r
λ(r)
λ(r)
log2
hλi
hλi
(4.11)
Shape information score follows the same method above by considering
as input the shape of the environment and the output the cell’s spike
train, assuming that all information about the shape of the arena is encoded by the fire rate. It is a measure of the sensory tuning of the spike
train independent of the position of the rat.
CHAPTER 4. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
70
THE DENTATE GYRUS
A
B
5
4.5
DG
LEC
Spatial info (bits/spike)
Shape info (bits/spike)
1
0.8
0.6
0.4
4
3.5
3
2.5
2
1.5
1
0.2
0.5
0
0
1
2
3
4
C
0
5
Mean spatial info (bits/spike)
D
LEC
DG
1200
0.9
1000
0.8
2
Mean place field size (cm )
Shape info score (bit/spike)
1
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
LEC
DG
800
600
400
200
0
0.0 0.1
LEC
0.2
0.3
0.4
0.5
α
0.6
0.7
0.8
0.9
1.0
MEC
Figure 4.7: DG cells do not represent sensory and spatial information in the same way as LEC cells. (A) Plot of the mean spatial
information score on both shapes and the shape information score of LEC
cells (blue, x) and DG cells (red, o). Spatial information score measures
quantitatively how the position is encoded by one spike while shape information score relates to how much information about the shape each spike
carries. (B) Box plot of the mean spatial information score and of (C) the
shape information score for both populations. In each box, the central mark
is the median, the edges of the box are the 25th and 75th percentiles and
the whiskers extend to the most extreme data points. (D) The relative contribution of LEC and MEC input influences spatial properties. Histogram
of the mean place field size as function of the ratio (α) of the mean drive of
MEC and LEC onto EC. Low alpha indicates high LEC influence while low
alpha indicates stronger MEC input.
4.5. EXPERIMENTAL PROCEDURES
A
71
B
1
66 Cells
3
[Leutgeb et al., 2007]
Experimental factor (β)
PV correlation
0.9
0.8
0.7
10000 Cells
3
2
2
1
1
66 Cells
0.6
10000 Cells
0.5
1
1.5
2
Variance/Frequency
2.5
3
0
0
LEC
0.5
α
1
MEC
0
0
LEC
0.5
α
1
MEC
Figure 4.8: Experimental variance correction for simulated data.
(A) Mean population vector (PV) correlation for two successive recordings
of the same morphing stage decays linearly with the increase of frequency
proportional variance. The number of cells considered for the PV influences
the effect of variance in correlation: less cells raises sensibility. To correct
the simulated data to the experimental condition of (Leutgeb et al., 2007)
we used the variance/frequency value (experimental factor β) that fits the
experimental PV correlation. (B) Fitting of β is influenced by both number
of cells and LEC/MEC mean rate ratio.
CHAPTER
Mechanisms of conjunctive
selectivity in the CA3
The content of this section has not yet been published and is part of
a manuscript in preparation with the cooperation of John Lisman and
Paul Verschure. The manuscript is named: "The mechanism of attractor
dynamics in the CA3". The aim of this study is to explain the mechanism of rate remapping in the CA3 and expose its implications in the
understanding of how attractor dynamics work in the CA3 network. Our
approach was to extend the model of de Almeida et al. (2010) by incorporating the LEC input and by adding the time-constraints of the
recurrent synapses in the CA3 (Figure 4.1). The abstract has been published as a Society for Neuroscience abstract (Rennó-Costa et al., 2012b):
Attractor networks are thought to play a fundamental role
in memory by producing pattern completion. A characteristic
feature of such networks is their recurrent connectivity. Given
its specific connectivity the hippocampal CA3 region has been
suspected to follow attractor dynamics. However the exper72
5
5.1. INTRODUCTION
73
imental evidence supporting this notion has not been conclusive. Here we analyze the phenomenon of rate remapping
that occurs as the environment is gradually morphed (Leutgeb
et al., 2007). During such morphing, the grid cells that provide
spatial localization input to the hippocampus are unchanged,
explaining the constancy of place field location. Other inputs,
however, provide sensory input affected by the morphing and
combine with the spatial information to produce rate remapping in the firing rate of the place fields. A key finding is that
CA3 place cells are less variant to small changes in the sensory
input than those found in the Dentate Gyrus (DG). This is
seen as an attractor-like effect. We have used computational
modeling (Rennó-Costa et al., 2010a; de Almeida et al., 2010)
to elucidate the origin of this difference in the physiology of
DG and CA3. We find that the difference can be accounted
for quantitatively by the fact that CA3 has recurrent connections, whereas the dentate gyrus does not. Furthermore, we
show that the observed hysteresis in CA3 place cells suggests
that changes in the attractor dynamics can be produced by
continuous plasticity during the experiments. The ability to
analyze the responses to morphing in the hippocampus thus
provides a rather direct view of how attractors function in a
biological memory network.
5.1
Introduction
The major difference between rate remapping in the DG and in CA3 is
illustrated by the PV correlation curve (Leutgeb et al., 2007, Figure 2A
or in this text, Figure 2.7E). In the DG, the ensemble population change
in PV correlation in relation to the recording in the first arena (1) is
significantly higher for the first morphing stage (2) than the change to
a second recording in the first arena after the whole morphing protocol
CHAPTER 5. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
74
THE CA3
(1’). In contrast, in the CA3 there is no significant difference between the
the 1-1’ and 1-2 PV correlations. This can be interpreted as an evidence
of pattern completion, a typical feature of attractor dynamics and associative memory systems that is commonly attributed as an hippocampus
function (Rolls, 2007; de Almeida et al., 2007). Pattern completion is the
ability of a network to exhibit a stable output in the case of an incomplete input. It is a feature of attractor networks that are able to converge
the output to a stable attractor if the input is within the attractor neighborhood Amit (1992). The matching of 1-1’ and 1-2 correlations might
suggest that the CA3 interprets that the first two shapes (1 and 2) are
indeed the same shape. Thus, understanding the differences of dynamics
in the rate remapping phenomenon might ultimately explain how pattern
completion is accomplished in the hippocampus.
The mechanism of pattern completion is likely to be within the CA3 network given that its major input, the DG, is able to distinguish the two
arenas. The major difference between DG and CA3 principal cells is that
CA3 cells present recurrent connectivity, a major feature of the neural
networks models of attractor dynamics Hopfield (1982). Recurrent connectivity can pattern complete by setting ensembles of cells. Each cell of
the ensemble do not only receive input from upstream regions but also
from other cells of the same the ensemble. In the case that a noisy version
of a memory is presented to the network, it is likely that some cells of
the ensemble will not receive the expected input. However, the excitation
originated in the other cells of the ensemble will be sufficient to activate
these cells, allowing the whole ensemble to be active (de Almeida et al.,
2007). Ensembles could be formed to represent a specific input pattern
by auto-associative learning rules (Hebb, 1932).
Although the ensembles might allow pattern completion, the nature of
the time dynamics of the CA3 networks creates very restrictive constrains
5.2. RESULTS
75
when compared tom theoretical models. For instance, the recurrent excitation must occur within the feedback inhibition delay of 3 ms. Moreover,
given that inhibition is evoked by the network, it is likely that competition modulates the pattern completion process. This allows very low
bandwidth for the ensemble process forcing the memory recall to happen
in a single interaction and not by sustained activity (de Almeida et al.,
2007).
To better understand the dynamics of the recurrent connections we implemented a time sensitive spike-based neural network implementing the
same principles of previous rate model: massive input convergence and
feedback inhibition. We base our implementation in standard integrate
and fire neurons with parameters extracted from CA3 physiology (de Almeida
et al., 2007) and in the network connectivity of the CA3 (de Almeida
et al., 2010). With the same experimental protocol as previous session,
we observed how the change in LEC population affected the overall population activity in the CA3.
5.2
Results
In a first step we simulated the response of DG cells to the input from
LEC and MEC by adapting methods from previous section (Figure 5.1A).
The spatial response (rate maps) of 10,000 grid cells were made insensitive to the shape of the arena (Leutgeb et al., 2007), whilst 10,000 LEC
cells had their spatial response switched from one map to an independent one at some random morphing progression, specific for each cell.
To compute the excitatory input to each individual DG cell we used a
realistic number of inputs and summed them. Each synaptic input was
taken from a population of randomly chosen entorhinal neurons, with
the synaptic weight randomly assigned according to the synaptic weight
distribution derived from the distribution of synaptic sizes (de Almeida
CHAPTER 5. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
76
THE CA3
et al., 2009a) as determined by serial EM (Trommald & Hulleberg, 1997).
Figure 5.1: Rate remapping in the DG with spiking neurons. (A)
Sample cells from MEC and LEC in the two environments. (B) Two sample
DG cells exhibiting rate remapping. (C) PV correlation curve for the DG
population compared with data from Leutgeb et al. (2007).
The spatial response of 10,000 DG neurons was constructed from the
spiking activity obtained from the simulation of individual gamma cycles
in an integrate-and-fire network with delayed feedback inhibition over
5.2. RESULTS
77
real rat trajectories in open field protocols (Hafting et al., 2005) (Figure
5.1B). Temporal resolution was set to 0.1 ms. Samples from different
trajectories were used when rate maps were compared. Overall excitatory input gain was selected in order to evoke gamma oscillation (37 ± 2
Hz). Relative MEC/LEC gain was set as 0.32 as in previous section.
Most of the simulated DG neurons exhibited place fields with an average
of 2.0 ± 0.8 per neuron (Figure 5.2A). Rate remapping was comparable
to the real rat observation of the decorrelation of the population rate
vector due morphing (Figure 5.1C). These results are in conformity with
previous results obtained with a rate based model. In the simulated data
the PV correlation curve is not affected by the direction of the morphing. This lack of hysteresis in the DG population has also been observed
experimentally (Leutgeb et al., 2007).
Following a similar method the response of 5.000 CA3 neurons was computed. The input originated in the DG was calculated from the rate
maps built previously. LEC and MEC input were taken from the same
population as the DG simulation. In a first simulation there were not
considered recurrent synapses. Overall excitatory input gain was selected
in order to evoke gamma oscillation (38 ± 1 Hz). Most of the simulated
DG neurons exhibited place fields with an average of 2.0 ± 0.8 per neuron
(Figure 5.2A). Most of the simulated CA3 neurons exhibited place fields
with an average of 1.4 ± 0.3 per neuron (Figure 5.2B), confirming results
from rate model (de Almeida et al., 2010).
Next we verified whether the observed CA3 cells exhibit rate remapping.
The PV correlation curve revealed that it does but not with the expected
properties (Figure 5.3). For instance, the simulated curve exhibit significant difference between the correlation from the first arena to the second
(1-2) and from the first arena to a subsequent recording to the same arena
(1-1’). Moreover, the format of the curve was more similar to the one
CHAPTER 5. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
78
THE CA3
Figure 5.2: Distribution of the number of place fields with the spiking model. Distribution for both DG and CA3 (no recurrents) is coherent
with experimental findings (Leutgeb et al., 2007).
observed in the DG. This is evidence that the recurrent input is essential
for the changes in the PV correlation curve.
Next we implemented the recurrent connections in the CA3 population.
Following experimental procedure of Leutgeb et al. (2007) we trained
the recurrents according to the population activity in the two extreme
arena shapes using a hebbian-like function (de Almeida et al., 2007). The
synaptic weight from cells i to j (W (i, j)) was defined based on the rate
(r(x, y, s)) in every position (x, y) and arena shape (s). W (i, j) was calculated as follows:
R(cell, r) = min(r, 10Hz);
∀(x,y,s)
f1 (i, j) =
∀(x,y,s)
f2 (i, j) =
X
r
W (i, j) =
R(i, r) ∗ R(j, r)
100
(5.2)
R(i, r) ∗ (10 − R(j, r))
100
(5.3)
X
r
(5.1)
f1 (i, j)
1.4 ∗ f1 (i, j) + 0.21 ∗ f2 (i, j) + 0.22 ∗ f1 (j, i)
(5.4)
5.2. RESULTS
79
Figure 5.3: Rate remapping in the CA3 population without recurrents don’t explain experimental data. PV correlation curve for of
the simulated CA3 population aligned to experimental CA3 (red ) and DG
(blue) curves (Leutgeb et al., 2007).
Once set the recurrent weights, we tested the effect of the synaptic
strength in the PV correlation curve (Figure 5.4). The experimental
curve could not be explained. Increase of recurrent strength caused an
overall increase in PV correlation in all morphing stages. Although it
was barely possible to obtain a flat correlation between 1-1’ and 1-2, in
such situation the difference of correlation between 1-1’ and 1-7 (first and
last shapes) was dramatically higher than the one observed by Leutgeb
et al. (2007). A close look to the time dynamics of the neurons potentials
shown that indeed the recurrents were able to cause pattern completion
(Figure 5.5) but at a cost that the two extreme memories would be almost indistinguishable.
CHAPTER 5. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
80
THE CA3
Figure 5.4: PV correlation curve for CA3 with recurrents batchtrained. Shown for multiple recurrent strengths (from black to light blue).
Experimental curve, normalized to mean 1-1’ correlation in simulation, is
shown in dotted red (Leutgeb et al., 2007).
One important aspect of this simulation is that learning is applied before
the trials. Thus, no hysteresis can be observed. Next we tested the
effect of learning in rate remapping. Recurrent synaptic weight was set
as before. Once started the task, it was computed a reference synaptic
weight matrix (W temp (i, j)) based on the non-recurrent output of the
CA3 population to the specific shape of the arena. Given a learning rate
ψ defined in the range [0, 1], the synaptic weight matrix was updated as
follow:
W (i, j) = (1 − ψ) ∗ W (i, j) + ψ ∗ W temp (i, j)
(5.5)
The constant learning had strong impact in the 1-1’ correlation (Figure
5.2. RESULTS
81
Figure 5.5: Pattern completion by recurrent excitation. Trace of the
potential of a neuron in a single gamma cycle for two different morphing
stages. (top) When there is no recurrent input, the 10% morphing changes
the input in a way that the cell cannot accumulate enough energy to release
a spike. (bottom) When a recurrent input is present, the cell gets an extra
amount of energy and spikes before the global inhibition is released.
5.6). Indeed, with ψ set to 10% the correlation in 1-1’ was much lower
than in 1-2. This allowed a PV correlation curve similar to experimental
data with ψ set to 5%.
These results suggests that the experimental observation has been misinterpreted as if the output of the CA3 population in the morphing stage
2 is the same as the morphing stage 1. It seems that 1-1’ is not the same
as 1-1 since the recurrent synapses has altered during morphing. This
is an indication that memory is not static but it drifts with new experiences. Another indication of this phenomenon is that the correlation of
two subsequent recordings is below the expected by sampling uncertainty
(as observed by the 1-1’ correlation in Figure 5.4) which opens run for
CHAPTER 5. MECHANISMS OF CONJUNCTIVE SELECTIVITY IN
82
THE CA3
Figure 5.6: Effect of LTP in the PV correlation curve.
an extra source of variability in the signal.
CHAPTER
Mechanisms of hippocampal
behavioral control
As part of our effort to understand why place cells become conjunctive
we studied how place cells can be used to support complex behavior. Our
approach was to first relate the hippocampal mechanisms to behavior in a
theoretical framework through computer simulations. In the subsequent
study we used a brain-based robotic control architecture to experiment
with robots in real-world tasks. Both studies present arguments of why
conjunctive hippocampal representation is relevant to support complex
task solving in theoretical and embodied grounds.
6.1
Theoretical study on behavior
The first study is presented as the manuscript "Nonspatial selectivity of
place cells supports quasi-optimal behavior in mixed spatial/nonspatial
tasks" which is in preparation (Rennó-Costa & Verschure, 2012). The
abstract reads:
84
6
6.1. THEORETICAL STUDY ON BEHAVIOR
85
The spatial selectiveness of hippocampal place cells is belief to be the computational basis of cognition for navigation
in rodents. However, place cells are not only selective to position. Other nonspatial variables such as sensorial perception,
behavioral context and emotional state also affect the firing
of place cells. Based on the computational principles of convergent information integration, population feedback inhibition and sequencing, all observed in the medial temporal lobe,
we used a computational model of the hippocampus and the
entorhinal cortex to show that spatial selectivity is not sufficiently to solve mixed spatial/nonspatial tasks. Rather, the
conjunctive representation of spatial and nonspatial information by the place cells supports quasi-optimal performance.
From this principle is possible to predict behavioral performance given the complexity of any general task. These results attest for the behavioral importance of phenomena such
as remapping and the joint codding of spatial and nonspatial
information on place cells.
6.1.1
Introduction
The observation of place cells is a major evidence of the existence of a
cognitive map in the hippocampus (O’Keefe & Conway, 1978). Lesion of
this brain region reveals behavioral impairment in spatial tasks (Morris
et al., 1982), supporting that place cells play an important role in the
cognition of navigation. This fact inspired computational models that
based on place cells units to solve simple spatial tasks (Erdem & Hasselmo, 2012; Burgess et al., 1997). However, rodents are also able to
perform spatial tasks that are dependent on nonspatial cues (Harrison
et al., 2006). Indeed, lesions of the hippocampus impair object-location
memory and the behavior in tasks that depend on objects or nonspa-
86
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
tial features (Gaskin et al., 2003; Mumby et al., 2002; Ennaceur et al.,
1997). This raises the question of whether strict spatial selectivity is
enough to support the behavior observed in rodents. That’s unlikely
given that place cells are not only spatially selective. The granule cells
of the dentate gyrus and the pyramidal cells of the CA3 region present
place fields whose peak rates are modulated by nonspatial variables such
as the shape and the color of the environment (Leutgeb et al., 2007).
This modulation, known as rate remapping, is caused by the integration
in the hippocampus of two separate spatial and nonspatial input channels from the entorhinal cortex (Hargreaves et al., 2005; Rennó-Costa
et al., 2010a). While localized lesion of the medial entorhinal cortex, the
source of spatial information (Sargolini et al., 2006), impairs exploratory
behavior (Schenk et al., 1983), localized lesion of the lateral entorhinal
cortex impairs object exploration but not pure navigation skills (Van
Cauter et al., 2012). This suggests that the conjunctive representation
of spatial and nonspatial information by the place cells is necessary for
successful behavior in tasks that demand spatial and nonspatial reasoning. It remains unclear however the mechanism by which the nonspatial
selectivity of place cells ultimately support exploratory behavior.
Our approach to elucidate how the brain accomplishes exploratory behavior using conjunctive spatial/nonspatial representation was to use a
virtual computational agent to perform two tasks in a virtual multiple
Y-maze (Figure 6.1). In the navigation task the agent is rewarded whenever it performs a nonspatial action only available at a specific arm of the
maze (Figure 6.1A). In the mixed task, the rewarded nonspatial action is
only available after an intermediate nonspatial action is executed in another specific arm of the maze (Figure 6.1B). By analytically confronting
the performance of a neural controller implementing place cells units with
strictly spatial response profile (spatial controller) against another neural
controller enhanced with conjunctive spatial/nonspatial selectivity (con-
6.1. THEORETICAL STUDY ON BEHAVIOR
87
A
1
2
3
4
R
B
4
1
7
*8
decision point
R rewarded action
3
2
§
5
6
R
spatial action
nonspatial action
n optimal solution
* avaiable after §
Figure 6.1: Experimental protocol. Multiple Y-maze (lef t) shown with
its graph representation (right). Decision points represented as vertexes,
spatial actions as straight arrows and nonspatial actions as angular arrows.
All affordances are shown. Optimal solution is in red. (A) Spatial task.
Reward is delivered when the agent reaches a specific location and applies
a nonspatial action. Illustrated as the task in which the rat has to find and
eat a piece of cheese. (B) Mixed spatial/nonspatial task. Rewarded action
(∗) is only available at the goal location after the agent applies a nonspatial
action at a different location (§). Illustrated as the task in which the rat
has to pull a button to release water in the fountain located elsewhere.
junctive controller) it is possible to identify the functional role of the
conjunctive coding in the place cells.
The design of a neural controller that could, concurrently, be related to
88
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
the hippocampus functioning and be able to perform the maze tasks is facilitated by the fact that many of the underlying computational principles
of the medial temporal lobe have already been identified: the interplay
between gamma-modulated feedback inhibition (de Almeida et al., 2009a)
and massive convergence of entorhinal input have shown how place cells
emerge from grid cells (de Almeida et al., 2009b, 2010). The same principle accounts for rate remapping when considering nonspatial inputs from
the lateral entorhinal cortex (Rennó-Costa et al., 2010a); the sequencing
of episodic memory can be achieved in the entorhinal cortex (Hasselmo
et al., 2000; Koene & Hasselmo, 2007) and stored and recalled in the
DG-CA3 loop (Lisman et al., 2005; Lisman, 1999); pattern completion of
hazy input in the CA3 (Leutgeb et al., 2007) can be explained by autoassociative synapses (Rolls, 2007; de Almeida et al., 2007). Some of these
principles such as immediate and persistent memory sequencing and integrative memory formation have been identified in a specific neural model
(Lisman, 2007). Called distributed adaptive control (DAC) (Verschure
et al., 1992, 2003), it includes mechanisms for perceptual and behavioral
learning which allow the self-organization of control rules from the interaction with the environment. DAC’s emerging features extrapolate
the functioning of the hippocampus. This suits the need of closing the
perceptual/behavioral loop in an embodied agent and at the same time
allows scoping specifically the hippocampal related computation. Moreover, other relevant aspects might not be computed in the hippocampus
and therefore can be considered available. For instance, self-location
(Fyhn et al., 2004; Hafting et al., 2005) and context information (Burwell et al., 2004) is provided by the entorhinal cortex. Performed action
information is provided by corollary discharge (Crapse & Sommer, 2008).
Decision-making is performed by the medial frontal cortex, which integrates information from the hippocampus (Walton et al., 2002).
DAC comprises three hierarchically coupled layers (Figure 6.2A). Se-
6.1. THEORETICAL STUDY ON BEHAVIOR
89
quential memories are formed at the ‘contextual’ layer from sensorimotor
contingencies of associative perception and actions. These are learnt at
the ‘adaptive’ layer from raw sensory input, motor primitives and predefined reflexes from the ‘reactive’ layer. In this study we bypassed the
perceptive learning of the ‘adaptive’ control layer since this mechanism
does not strongly depend on the hippocampus (Manns & Squire, 2001).
This bypass facilitates observation and avoids performance artifacts due
to behavioral feedback (Verschure et al., 2003). The defined associative
perception included the position of the agent in the maze and whether
the agent performed the intermediate nonspatial action or not. The associative action set includes the move from one location to another and
the nonspatial actions. Affordances are available since they naturally
emerge from the “adaptive” layer interactions with the environmental
set (Duff et al., 2010), including point-to-point displacement in map-less
navigation (Mathews et al., 2009). We assume salient events when the
agent moves from one junction to another or after box-related actions.
Salient events trigger the formation of perception/action couplets that
are sequentially stored in the short-term memory (STM). STM is stored
persistently in the long-term memory (LTM) when reward is delivered
(Figure 6.2C). STM and LTM are, respectively, analog to the sequencing
of memories in the entorhinal cortex (Koene & Hasselmo, 2007) and in
the hippocampus (Lisman, 1999) (Figure 6.2B). Behavior is dominated
by a specific LTM sequence when the STM match coefficient is the highest among all sequences and is above an activation threshold (Figure
6.2DE). If no LTM sequence is active, than the adaptive controller will
trigger a random action among the affordances.
As the perceptual input for the memory couplets we assume that precise
self-localization information is available for computation, being provided
through the outcome of a path integration mechanism in the medial entorhinal cortex (McNaughton et al., 2006). Although path integration
90
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
A
Adaptive
Perception
Proprioception
Action
Reactive
Sensory
information
Allostatic
regulation
Motor
primitives
External
stimulus
C
D
LTM
CA3
DG
LEC
+
STM
Spatial
Information
=
max
STM
Couplet
Action
Nonspatial
Information
MEC
Cortex
perception
ABC
éê ?
E
Recall
LTM
Reward
Nonspatial
Information
CA1/SuB
Response
Behavioral Learning
STM
B
Long-Term
Memory
Short-Term
Memory
Contextual
Couplet
Spatial
Information
Action
LTM
action
CABCABCZ
é êê êéêé é
1 0 0 2 0 0 3 0
max score
next action:
é
Figure 6.2: The DAC architecture. (A) System overview and its major
connections. The reactive layer relates statically sensory information and
allostatic regulation with the motor primitives. The adaptive layer builds
on top of the reactive layer with self-organized responsive units of perception, proprioception and actions. (B) Relevant connectivity in the medial
entorhinal cortex. Dentate Gyrus (DG) and CA3 integrate the multimodal
input from the lateral and media portions of the entorhinal cortex (LEC
and MEC). Sequencing is obtained by the interconnectivity between CA3
and DG. Output is channeled back to the cortex through the CA1 and the
Subiculum (SuB) (C) Schematic for behavioral learning (LTM acquisition)
in the contextual layer. (D) Schematic for action recall. (E) Procedure to
select next action.
6.1. THEORETICAL STUDY ON BEHAVIOR
91
is error prone in rats (Etienne et al., 1998), it can be corrected by the
existence of stable environmental cues (Verschure et al., 2006). Also,
sensory information or behavioral context is provided through the lateral entorhinal cortex. This area of the brain is highly innervated by
sensory related areas such as the occipitotemporal cortex (McDonald &
Mascagni, 1996) and the olfactory bulb (Carlsen et al., 1982). Moreover,
behavioral context information might be provided by the prefrontal cortex through the entorhinal cortex (Hyman et al., 2005) and emotional
context might be provided by the ventral striatum (Lansink et al., 2009).
The computation of the memory units in the computational controller is
analog to the integrative/competitive process that rise the formation of
place fields in the dentate gyrus (de Almeida et al., 2009b) and in the
CA3 (de Almeida et al., 2010). The two designated neural controllers differ by the information that is integrated in the formation of the memory
couplets: the spatial controller only associates position to action while
the conjunctive controller associate conjoinedly position and behavioral
context.
6.1.2
Results
The experimental protocol consisted of a learning phase and a performing
phase. In the learning phase, the agent explored the maze by applying
random actions within the acquired affordances. The phase was completed when LTM was full. In the performing phase behavior was dominated by the LTM sequences. To establish the upper and lower bounds
of the performance we also considered a random controller with no LTM
and an optimal controller that uses a shortest path algorithm (Dijkstra,
1959). The random controller succeeded in delivering a solution in every
trial of each task. Time-out was set as 2 times the longest random solution of each experimental condition. Further trials that reach time-out
were considered a failure.
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
A
spatial place cells
random
Performance
relative to the shortest path
optimal
2
60
50
3
40
4
5
30
6
20
7
8
10
9
10
100
B
Optimal memory length
92
101
102
Length of memory sequence
0
2
3
4
5
6
7
8
9
10
11
Mean optimal solution length
12
Figure 6.3: Spatial selectivity is sufficient for solving a spatial task.
(A) Performance in the spatial task (optimal solution length 2.8±1.1 actions) as a function of the memory sequence length (median and interquartile range). (B) Estimated memory sequence length that leads to the best
performance as a function of the mean length of the optimal solution of the
maze.
In the navigation task, the neural controller succeeded in taking the agent
to the goal position in every trial and for every memory sequence length.
Memory sequence length affected performance (Figure 6.3A). Performance peaked quasi-optimally with mid-lengthened memory sequences
and deteriorated asymptotically to chance with short and long memory
sequence lengths. This observation extended to different mazes complexities with maximum sequence length expanding exponentially with maze
complexity (Figure 6.3B).
In the mixed task, the controller based on purely spatial place cells failed
to produce a solution in some task conditions with mid-lengthened memory sequences (Figure 6.4A). The initial position of the agent and the
length of the optimal solution affected failure level. The controller based
on conjunctive place cells succeeded in every trial and for every memory
sequence length. Memory sequence length affected performance for both
controllers (Figure 6.4B). Performance peaked quasi-optimally with mid-
6.1. THEORETICAL STUDY ON BEHAVIOR
A
Long optimal solution
Short optimal solution
100%
Conjuctive place cells
Spatial place cells
Random
relative to the shortest path
Successful trials
optimal
Performance
75%
50%
25%
0
10
1
10
Length of memory sequence
C
2
3
4
5
6
7
8
9
10 0
10
2
10
Optimal memory length
0%
B
93
1
10
2
10
Length of memory sequence
90
80
70
60
50
40
30
20
10
0
4
5
6
7
8
9
10
Mean optimal solution length
11
Figure 6.4: Conjunctive spatial/nonspatial selectivity is necessary
for solving a mixed spatial/nonspatial task. (A) Percentage of successful trials of the spatial controller in the mixed task (optimal solution
length 4.8±1.0 actions) and (B) performance relative to the shortest path
in the mixed task as a function of memory sequence length (median and interquartile) for the half-shortest/-longest solutions. (C) Estimated memory
sequence length that leads to the best performance as a function of of the
mean length of the optimal solution of the maze.
lengthened memory sequences and deteriorated asymptotically to chance
with short and long memory sequence lengths. The conjunctive controller outperformed the spatial controller in the memory long sequence
length range in which the spatial controller could produce valid solutions
in more than 75% of the trials. This observation extended to different
mazes complexities with maximum sequence length expanding exponentially with maze complexity (Figure 6.4C).
One interesting aspect of the simulations in both tasks is that the size
of memory sequences in rats is estimated as 7±2 memories (Lisman &
94
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
Idiart, 1995; Jensen & Lisman, 1996) which allow the use of the computational model to relate to the approximate performance of rats in mazes
of specific complexity.
The failure of the spatial controller to produce successful solutions with
short memory sequences can be explained by the lack of behavioral context information in the recall of memories. In this situation, the sequences
that lead to the rewarded action dominate the behavior whenever the
agent is close to the ultimate goal location (Figure 6.5A). Evidence of
this mechanism is that failure level is dependent on the initial position
of the agent (Figure 6.4A). Conjunctive place cells solve this problem by
disambiguating the position and the behavioral context so that behavior
will only be dominated by the memories after the agent triggered the intermediate nonspatial action (Figure 6.5B). This mechanism is conform
to experimental finding that place cells acquire directional selectivity
when passing the same location in two different behavioral contexts of
the same task (Navratilova et al., 2012).
6.1.3
Discussion
These results evidence that neural computation performed by the hippocampus circuitry and based on conjunctive place cells is sufficient to
solve mixed spatial/nonspatial tasks in a quasi-optimal fashion. Moreover, the hippocampus alone is not capable of solving the task relying
solely in the spatial selectivity of place cells. Although other brain areas
such as the medial prefrontal cortex (Erdem & Hasselmo, 2012) and the
ventral striatum (Lansink et al., 2009) can possible provide the needed
disambiguation tools for the spatial information of place cells, the theta
synchronization between these areas and the hippocampus suggest a
lower frequency information transfer than what is observed in the internal gamma modulated microcircuit responsible for sequencing, auto-
6.1. THEORETICAL STUDY ON BEHAVIOR
A
decision point
n
R rewarded action
spatial action
memory sequence
§
-3
§
-2
R
nonspatial action
avaiable after §
*
§
-3
-1
*0
95
-1
*0
-2
-1
*0
R
-2
-3
R
locked position
B
behavioral context 1 behavioral context 2
-3
-2
-1
0
-3
-2
-2
-3
R
Figure 6.5: How conjunctive place cells solve the mixed spatial/nonspatial task. (A) 4-memory sequences of non-conjunctive place
cells that cause misleading by attracting to action ∗ before action §being
executed. The middle memory sequence is a special case in which the action
§will never be accessible if the agent is at a locked position. (B) Conjunctive
place cells solve it by establishing an independent graph for each behavioral
context, causing that the agent will not be attracted to ∗ before executing
§.
96
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
associative memory and phase-precession (Skaggs et al., 1996; Jensen &
Lisman, 1996). This suggests that these areas provide important information that is integrated in the conjunctive representation of the place cells
but that the computation itself is performed locally in the hippocampus.
6.1.4
Methods
The multiple Y-mazes were built using a random Prim algorithm (Prim,
1957). Locations with 1 or 3 adjacent locations were considered as decision points (Figure 6.6). Nonspatial actions were set only on maze arms
limiting to 3 the maximum number of actions available at a single position. The complexity of a specific maze was measure as the mean length
of the minimal path to complete the task given a set of random initial
positions.
For the spatial controller, the memory couplet was set as <R,A> where
R is the current agent location and A is an action. For the conjunctive
controller, the memory couplet was set as <R,B,A> where B is the behavioral context. STM was set as FIFO queue of memory couplets of
maximum size M. Memory couplets were pushed at the STM whenever
an action was successfully applied. LTM was set as a list of memory
couplet sequences of maximum size M with a maximum number of sequences N (N=32). Reward made the STM queue to be pushed at the
LTM if LTM was not full. A persistent fitness value was associated to every memory couplet in LTM. Chaining was accomplished by transferring
fitness value + 1 to the following memory when there is a hit. Values not
transferred are zeroed. Action is selected as the one with highest fitness
values among the affordances. If multiple actions have the same fitness,
the action is selected randomly. If no action has fitness above threshold
(T=1) than action is selected randomly among affordances.
6.2. ROBOT EXPERIMENTATION
97
Figure 6.6: Maze samples. With 10, 30, 100 and 129 decision points (topleft, top-right, bottom-left, bottom-right). Decision points in red and path in
gray.
One each task the agent explores randomly the maze until LTM is full.
After, the agent preforms several trials relying on memory (500 trials per
simulation). Each task condition (5 goals for each maze) was simulated
5 times.
6.2
Robot experimentation
This work was published as the paper "Integrating neuroscience-based
models towards an autonomous biomimetic synthetic forager", presented
in the conference IEEE ROBIO 2011 in Phuket, Thailand (Rennó-Costa
et al., 2011). As an extension of this robot setup we also developed a
reactive controller designed to provide basic behavior (Appendix A) The
98
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
abstract reads:
Foraging can be described as goal-oriented exploration for
resources. It exemplifies how animals coordinate complex sensory and effector systems under varying environmental conditions. To emulate the foraging capabilities of natural systems is a major goal for robotics.
Therefore, foraging is
an excellent paradigm to benchmark novel autonomous control strategies. Here we describe the biomimetic control architecture of the Synthetic Forager (SF), an effort to integrate multiple biologically constrained models of specific perceptual and cognitive processes pertaining to foraging into
one general autonomous robot controller. This proposal is
built upon the well-established Distributed Adaptive Control (DAC) framework and brings together neuroscience-based
models of decision-making, multi-modal sensory processing,
localization and mapping and allostatic behavioral control.
To show the potential of the SF model we used it to control
a high-mobility wheeled robotic platform in three behavioral
tasks similar to experimental protocols applied to rodents. We
show that the robot can reliably perform cue detection, rule
learning and goal-oriented navigation in open environments.
We propose that this approach to robotics allows both the
study of embodied neuroscience models and the transfer of
brain based principles to robotic systems.
6.2.1
Introduction
The central challenge of autonomous robotic technologies is the coordination of complex sensory and effector systems under varying task conditions. Successful systems have to account for rapidly changing demands
from both the behaving agent itself and its environment. Additionally, all
6.2. ROBOT EXPERIMENTATION
99
this is to be accomplished under austere restrictions placed upon available
computing resources and time, and upon the basis of incomplete and only
partly reliable information. In nature, abundant examples of biological
systems exist that fulfill all these requirements, sometimes in the most
unpredictable and challenging ecosystems (Calhoun, 1963). Engineering
solutions might profit if grounded in successful biological systems. An
animal behavior of particular interest in this context is foraging, i.e. the
ability of animals to optimally explore and environment and exploit its
resources (Stephens et al., 2007). The concept of foraging is a step towards integrated situated behaving systems that further generalizes from
biomimetic robotics examples such as legged dog (Song & Waldron, 1989)
and fish robots (Chen & Zhu, 2005); chemical sensing (Mathews et al.,
2009) to haptics using whiskers (Lepora et al., 2010); and evolutionary
morphogenetic approaches (Jin & Meng, 2011) to name a few.
Foraging is defined in behavioral ecology as the exploration for resources,
usually motivated by deprivation, e.g., energetic and reproductive needs
(Stephens et al., 2007). The concept encompasses goal-oriented behavior in complex environments where prior knowledge and action strategies
must be matched to the novelty and hazards of a dynamic world and
the varying requirements of the system itself. Successful foragers must
simultaneously satisfy a wide range of constraints in varying spatial and
temporal scales such as avoiding obstacles while maintaining an energyefficient trajectory back to a known feeding or nest site. In addition,
due to natural selection and intense competition for resources, animals
must perform in a near-optimal fashion. Indeed, this can be observed
in a range of animal species from humans and monkeys to small insects
(MacDonall et al., 2006; Davis, 1996). For example, it has been shown
that rats choose their strategies based on the expected gain and its magnitude (Herrnstein, 1970). In a more specific case, rats have been shown
to develop optimal adaptive foraging strategies with respect to travel
100
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
time, probability of food appearance and amount of food pellets in a radial arm maze, where food is delivered in different arms under varying
conditions (Roberts, 1992).
Although descriptive models exist for the behavioral aspects of foraging (Stephens et al., 2007; Giraldeau & Caraco, 2000), the underlying
functional and neural organization have not yet been integrated into a
unified theory. Given the complexity of foraging, such a theory must
involve different levels of organization, from basic reflexes to social interaction. An inclusive understanding can only be reached if the individual
aspects are combined in an integrated framework. Here we follow the
Distributed Adaptive Control (DAC) architecture, a robot-based model
of perception, cognition and behavior (Verschure et al., 1992). DAC has
been successfully used in general tasks from mobile robotics (Mathews
et al., 2009), interactive spaces (Eng et al., 2003) to synthetic music composition (Manzolli & Verschure, 2005). In addition, DAC has supported
several biological findings such as behavioral feedback (Verschure et al.,
2003), an integrated theory of classical conditioning (Verschure et al.,
1992) and the principles of animal behavioral regulation (Sanchez-Fibla
et al., 2010).
In this study we present an integrated neutrally constrained model of foraging based on the DAC architecture. The Synthetic Forager (SF) is an
integrated robotic mobile platform and biomimetic control architecture
that realizes a foraging agent with a continuous duty cycle. SF is formulated against the benchmark of rodent foraging. SF integrates several
neuroscience-based models including: decision-making and rule learning
in the prefrontal cortex (Duff & Verschure, 2010), multi-modal sensory
processing in the ventral visual stream (Wyss et al., 2003), localization
and mapping in the hippocampus (Verschure et al., 2006) and allostatic
behavioral control (Sanchez-Fibla et al., 2010). Here we describe how
6.2. ROBOT EXPERIMENTATION
101
the SF framework, the high-mobility platform to which it is applied, is
used to control a wheeled robot is applied to three experimental tasks
that capture protocols developed for studying foraging in rodents: (1)
self-localization, (2) sequence learning and (3) resource localization. We
demonstrate that the robot is able to succeed in these tasks suggesting
that the integration of several specific biologically constrained models
under the DAC framework will ultimately support optimal foraging behavior in robots.
The synthetic forager framework
SF is an integrated model of foraging based on the DAC architecture.
It includes a series of sub models for the different requirements of foraging: allostatic control allows the management of different drives and
associated behavior systems; the visual processing hierarchy provides for
the transformation of high-dimensional input into a compact and usable
representation; the egocentric and allocentric spatial processing hierarchy supports localization and mapping; the memory and decision-making
systems ties the available perceptual information into causal relations to
behavioral outcomes in the form of goal oriented sequences. These components are integrated in the DAC framework, rendering the SF control
model.
Distributed Adaptive Control (DAC)
DAC (Verschure et al., 1992, 2003; Verschure & Althaus, 2003; Verschure
et al., 2006; Duff et al., 2010) is a robot-based neuronal model of perception, cognition and behavior that is a standard in the domains of new
artificial intelligence and behavior-based robotics. It is constraint by biology and fully grounded since it autonomously generates representations
of its primary sensory input. This facilitates a direct comparison to bi-
102
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
ological systems and the thorough investigation of complex phenomena,
such as foraging, and the generation of internal states that are related to
sensory, motor, cognitive or motivational states and the influence of the
environment and behavior on this process (Duff et al., 2011).
DAC is organized around three tightly coupled layers: reactive, adaptive
and contextual. In the reactive layer, a prewired set of reflexes enables
the behaving system to exhibit simple reactive behaviors. The adaptive layer uses the cues provided by the reactive layer to acquire sensory
representations and their associated behaviors. This provides the mechanisms for the adaptive grouping of sensory events and the reshaping of
responses, as observed in classical conditioning (Duff et al., 2010). The
contextual layer acquires, retains, and expresses sequential sensory-motor
contingencies, e.g. perception-action tuples (Duff et al., 2011), provided
by the adaptive control layer, using mechanism for short- and long-term
memory. This mechanism describes goal-oriented learning as in operant
conditioning. The dynamics of these memory structures during a foraging task are equivalent to a Bayesian description of foraging (Verschure
& Althaus, 2003).
Allostatic control
The allostatic control system of the reactive layer deals with specific
drives, e.g., hunger and safety, and modulates behavior by establishing
specific reactive goals. In SF, the allostatic system is composed of actively updated state variables that are connected to reactive behaviors
and/or are available to the decision-making system. For example, state
variables might be described as internal needs such as “hunger” which
would initiate and sustain a “seek for food” behavior. The allostatic
model regulates multiple systems that collectively generate complex task
dependent behavior.
6.2. ROBOT EXPERIMENTATION
103
Visual processing hierarchy
In SF, vision is the main distal sensory input. The aim of the visual processing hierarchy is to provide a robust representation of salient elements
of the world that support navigation and goal oriented behavior. In SF
we use the SIFT algorithm (Lowe, 1999).
Spatial processing hierarchy
Spatial localization and mapping is a major element of foraging. Some
important notions as home/nest and resource disposal depend on the
abilities of acquiring the knowledge of where the system itself is and how
this information spatially relates to relevant sites. In the DAC architecture, memories are perception-action tuples formed in the adaptive
layer. There is functional evidence for this spatial/non-spatial association in the hippocampus (Lisman, 2007), which is strengthened by the
fact that the spatial code is modulated by manipulations of non-spatial
variables, in a phenomenon called rate remapping (Leutgeb et al., 2007).
This aspect of the the adaptive layer is based on a recent model of the
hippocampus where place cells are formed from the principles of convergent signal accumulation and population derived feedback inhibition
(de Almeida et al., 2009a), which explains the formation of place cells
from grid cells (de Almeida et al., 2009b) and rate remapping (RennóCosta et al., 2010a). In the model, the hippocampus receives as input:
the response of a path integration system, which is updated by odometry
and regulated by place cell activity (Guanella & Verschure, 2006); the
output of the visual processing system, which provides landmark information (Verschure et al., 2006); and motivational information provided
by the allostatic control (Sanchez-Fibla et al., 2010). The output of this
system provides input to the decision making and memory system in spa-
104
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
tial tasks.
Memory and decision making
In DAC, memory is organized in the contextual layer by means of sequencing, e.g., active memories trigger subsequent memories through
lateral interaction (Marcos et al., 2010). Memory sequences are built
from the sequential occurrence of single memories, i.e. sensory-motor
tuplets, and are stored in a short-term memory buffer, thus sequencing
depends on their occurrence in time. Whenever a goal relevant event is
detected - when a cue is detected, the robot arrives home or food is found,
among others – the full content of the short-term memory is stored in
the long-term memory. Recall is based on the matching of the elements
of these stored sequences to ongoing sensory events. When a memory
element is active and contributes to action it will bias the next memory
element in its sequence increasing the probability that it contributes to
action in the future.
Robots
For the experiments described here we used two wheeled robots designed
and constructed by Robosoft (Bidart, France). The outdoor unit is a
prototype in development (Figure 6.7 left) whereas the indoor unit is a
standard robuLab 10 (Figure 6.7 right). Each unit is capable of doing
on-spot turning and is equipped with infrared and ultra-sound proximity sensors covering a range of 360o . Both robots are equipped with a
color CCD camera (Imaging Source, Bremen, Germany) mounted on a
pan-tilt unit (Direct Perception, Burlingame, USA). Both robots include
an embedded computer system running Linux Ubuntu. Control software
was implemented as a stand-alone application in ANSI C++.
6.2. ROBOT EXPERIMENTATION
105
Figure 6.7: SF Robots. (left) Outdoor and (right) indoor units with
1.1 x 0.6 m and 0.6 x 0.6 m respectively. Both equipped with embedded
computation, proximity sensors and color camera mounted on a pan-tilt
unit.
6.2.2
Results
To test the ability of DAC to generate foraging we applied the SF system
to three tasks critically involved in foraging: (A) in the self-localization
task we seek the ability of creating a robust internal representation of
the position of the robot; (B) in the sequence-learning task the objective
is to learn relations among causal events that can be expressed in rule
based goal oriented behavioral sequences; (C) in the resource-localization
task the robot is expected to first associate the position of a resource to
a variable visual cue and second to be able to reach the acquired target
location.
For each task we predefined perception primitives, action sets and allostatic values or drives. The connection between perception, action and
allostatic values of the reactive layer is predefined in the form of causal
rules (cause→consequence). In the adaptive layer, complex perceptual
states are formed, as for example in position specific memories and these
are associated with both allostatic values and actions. In the contextual
106
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
Figure 6.8: Outdoors arena. (A) Overview of the operative area. (B)
Snapshots of the robot (90o between each other).
layer, there are predefined events - such as achieving specific goal states
- that trigger the formation of contextual memories in the form of sequences.
Self-localization task
The self-localization task is designed to test the adaptive formation of
position memories. It is a protocol in which the robot moves semi randomly in an open field arena with the objective of developing a neural
representation of its position. It is comparable to the standard open field
task used in the description of the, so called, place cells in which the rat
explores the arena without explicit goals (O’Keefe & Dostrovsky, 1971).
For the robot version, it was performed outdoors in a campus square surrounded by buildings of different shapes and colors (Figure 6.8A). The
operational area was limited to a rectangle of 8.8 x 6.4 meters. Tracking
applied in the analysis was performed by referring to positioning marks
placed on the floor with a resolution of 0.8 x 0.8 meters.
6.2. ROBOT EXPERIMENTATION
107
The reactive behaviors in this task were the following three reactive actions: (A1.1) “random walk”, in which the robot performs a small random
rotation (−90o to +90o ) followed by a forward movement (0.8 meters);
(A1.2) “orienting”, in which the robot takes snapshots from −135o to
+135o with 45o steps (Figure 6.8B); and (A1.3) “homing” in which the
robot is manually oriented to the center of the arena using joystick control.
We defined three allostatic drives: (D1.1) “exploration”, (D1.2) “novelty”
and (D1.3) “out of arena”. D1.1 is always active while D1.3 is set externally using a joystick whenever the robot is located outside of the arena.
No reactive perceptual states were defined. The reactive rules are: D1.1
+ not D1.2 → A1.1, not a novel place leads to exploration; A1.1 → D1.2,
exploratory action leads to a novel place; D1.1 + D1.2 → A1.2, a novel
place leads to cue gathering; A1.2 → not D1.2, cue gathering reduces
novelty; D1.3 → A1.3, escape of the arena leads to control by the user.
The visual processing system was based on SIFT. In each picture we
computed a collection of local salient points using the SIFT algorithm
(Lowe, 1999). Snapshots taken at each corner with an angle step of 90o
covering a 360o total view served as a reference. Whenever A1.2 was executed, the orientation of the robot was measured by comparing 3 pictures
with 90o angle step to the all reference pictures sequential combinations.
The robot orientation was recalled as the one with the combination of
reference pictures that maximizes the number of matching salient points.
Once the orientation was set, the mean change in scale from the corresponding images to the 3 reference pictures sharing the same alignment
define the output of the visual layer (V O). The V O comprises 16 cells
(4 corners x 4 orientations). In a map like representation, the output of
the visual layer shows strong position specific modulation (Figure 6.9A).
108
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
The memory layer activity uses the visual layer output as input. Computation is performed by a model of the input-output transformations of
the dentate gyrus (de Almeida et al., 2009b; Rennó-Costa et al., 2010a).
The input M Ii of a memory cell i at position r is defined as:
M Ii (r) =
16
X
e−2∗|V Oj (r)−αij (r)|
(6.1)
j=1
Where for each cell i in the memory layer we defined a preferred value
αij for the visual cell j. αij was set randomly in the range [0, 2]. The
output M Oi of the memory cell i is a result of a competition process in
the form of a E%-MAX winner-take-all mechanism (de Almeida et al.,
2009b,a, 2010; Rennó-Costa et al., 2010a):
M Oi (r) = max (0|M Ii (r) − (max1≤k≤1000 M Ik (r)) ∗ .9)
(6.2)
Through this process the memory population exhibits place cell like responses, e.g. the activity of the cells is limited to a certain region of the
arena (Figure 6.9B), in contrast to typical non-spatially specific response
found for the visual layer.
In order to access the precision of the position information, standard
Bayesian reconstruction (Guanella & Verschure, 2007) was applied to the
readout of the self-localization data revealing a mean normalized error
of 1.1 meters in both layers (equivalent to 2.4% of the arena). Moreover,
the population vector autocorrelation analysis (methods in Leutgeb et al.
6.2. ROBOT EXPERIMENTATION
109
Figure 6.9: Neural representation of space. Rate maps of (A) three
visual layer cells and (B) three memory layer cells. X- and Y-axis represent position and brightness the unit activity. (C) Population vector selfcorrelation for both layers. (D) Boxplot of spatial-info score in bits/spike.
(2007)) showed a more stressed incremental disparity in the population
activity as a function of the distance for the memory layer (Figure 6.9C).
This is an expected property of pattern separation in the dentate gyrus
(Leutgeb et al., 2007). Furthermore, the activity of single memory units
carries more information about the position of the animal than single
visual cells (Figure 6.9D, methods in Skaggs et al. (1993)). These results
together show that the visual model is successfully capturing the position
of the robot and that the memory model is capable of representing this
information in a compact and differentiable way supporting a symbolic-
110
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
like computation in a fully grounded manner.
Sequence-learning task
The sequence-learning task is a protocol in which the robot has to learn
a color sequence. In the task, the robot has to move to a specific position
indicated by a red light (Figure 6.10A) where two colors, one in each side
of the robot, are displayed on the floor (Figure 6.10B). The robot has
to gather the color cues and after point to the correct color. If it points
to the correct side two new colors are presented, following the sequence
(Figure 6.10C). If the robot makes the wrong decision the floor blinks
in red and the robot has to return home to restart the procedure. This
procedure is inspired by sequential water tank visual discrimination task
(Aggleton et al., 2010).
The cognitive system was programed with five actions: (A2.1) go forward; (A2.2) go backwards for 3 meters; (A2.3) visual orienting (set pan
tilt −90o , gather color A, set pan +90o gather color B); (A2.4) point
left; (A2.5) point right; Four drive states were defined: (D2.1) “sequence
on”; (D2.2) “at position”; (D2.3) “memory full”; (D2.4) “wait answer”; A
further reactive perception event was defined (P2.1) when a red color
patch was detected on the floor and a further internal state (P2.2) when
a correct selection was made. The reactive rules defined were: D2.1 +
not D2.2 → A2.1, if sequence is on and not in position, go forward; D2.1
+ not D2.2 + P2.1 → D2.3, when red is detected, it is in position; D2.1 +
D2.2 + not D2.3 → A2.3, if no memory check for options; A2.3 → D2.3,
if color checked then memory is full; A2.4 or A2.5 → D2.4, if selection
made, wait answer; D2.4 + P2.4 → not D2.1 + A2.2, if red is detected
when a sequence is completed with a wrong answer, go home; D2.4 +
(delayed) not P2.4 → not D2.3 + P2.2, if no red is detected, continue
the sequence. To realize this task, the vision system detected saturation
6.2. ROBOT EXPERIMENTATION
111
Figure 6.10: Sequence-learning task. (A) Robot goes forward until the
red mark is detected. (B) Two color options are presented to the robot.
(C) If the correct color is selected then two new options are presented.
and hue divided in 12 hue bands. The perceptual segment of memory
units stored hue value. Memory sequences were recorded whenever P2.2
was triggered. Through perceptual matching, stored long-term memory
elements can be activated and the location of the next cue and the direction in which to move the camera is selected according to the predicted
hue. If no memory is activated then a color location is selected randomly.
The robot could correctly acquire and perform the whole sequence (depth
= 4) in all runs (n=10). Starting with an empty memory it took a mean
of 2.8 trials to accomplish the whole sequence. The robot performed the
task successfully for a public demonstration performed in the XIM space
(Bernardet et al., 2007).
112
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
Figure 6.11: Resource-localization task. (A) Experimental space (20 x
15 meters) with the arena marked on the floor. (B) Cue gathering at home.
(C) Kidnap procedure. Robot is taken from one position to another and
successfully finds the cued rewarded location.
Resource-localization task
The resource-localization task is a behaviorally active protocol in which
the robot has to recognize a conditioned stimulus presented at his home/nest
and associate it to the expected position of the reward (Figure 6.11B).
The reward location is assigned to a specific position deterministically
according to the cue. The task is conceptually similar to an experimental protocol used to study decision-making in rodents. In this T-maze
task rata could either choose to climb a barrier to obtain a high reward
in one arm or could obtain a small reward in the other with no barrier
present (Walton et al., 2002).
The robot T-maze task was developed indoors (Figure 6.11A). The operational area was restricted to a 4.8 meters wide square in the middle of
6.2. ROBOT EXPERIMENTATION
113
the experimental room. Cues were placed on all walls to provide distal
landmark references that can be captured by the visual system. One wall
of the arena was marked as home while the three others were marked as
target/food disposal localities.
The reactive layer of the robot was predefined with the drives: (D3.1)
“at home”; (D3.2) “food available”; (D3.3) “at food position”; (D3.4) “outside of the arena”. The specific actions were defined as: (A3.1) “orient
towards cue”, set pan-tilt up and rotate to search for a visual cue; (A3.2)
“go to specific position”; (A3.3) “check current position”, which is based
on the self-localization procedure defined previously; (A3.4) “go home”;
The rules were: D3.2 → A3.2, whenever there is food, go towards it; not
D3.2 + not D3.1 → A3.4, if no food is available then go home; not D3.2
+ D3.1 → A3.1, if no food available and at home, seek for a cue; D3.3
→ not D3.2, whenever food is found, consume it; D3.4 → A3.3, leaving
the arena initiates a check for the current position.
Cue detection is provided by the visual system by segmenting the cue carrier (fluorescent yellow card) and applying matching on the salient points
obtained with SIFT. The specific cue is selected based on the class with
the highest scale and rotation correlation among all other candidates
for four sequential frames. The goal-oriented memory, is represented by
a sensorimotor tuple: <cue>/<position>. Memory is consolidated in
LTM whenever the robot finds a novel rewards site without using the
long-term memory. When a cue is detected but no memory unit is associated with it, the robot will do a random search following the procedure
described in the self-localization task to establish the relationship of the
novel cue with a specific position.
The robot successfully performed the task in a live demo for an audience
of 20 people under three conditions: (1) new cue / food location; (2)
114
CHAPTER 6. MECHANISMS OF HIPPOCAMPAL BEHAVIORAL
CONTROL
known cue / food location; and (3) know cue / food location with kidnap, e.g., the robot is stopped and transferred to another place during
the execution of the task (Figure 6.11C). When including all performance
sessions (n=8): cue detection was successful in 91% of the trials (n=78);
the robot always could find the correct position, relying only on path
integration in 60% of the trials and requiring one or more position reconstruction procedures using visual information in the other 40% trials.
6.2.3
Conclusions
Here we presented the SF control architecture, based on DAC, which
solves behavioral tasks similar to those applied in the study of animal
behavior. We use foraging as a main benchmark since it comprises a
range of motivational, perceptual, cognitive and behavioral systems that
well summarize the main aims of autonomous robotics while representing
a major area in the study of animal behavior. The current implementation of SF shows how the multi-layer organization of actions, perception
and drives combined with mechanisms of learning and memory can be
used to express foraging behavior. Moreover, our approach not only allows the performance of behavioral tasks using robots but also provides
methods to validate specific neuroscience-based models. As an example,
the SF system for localization and mapping provides a biologically based
system comparable to the SLAM paradigm (Leonard & Durrant-Whyte,
1991).
As a drawback, the current model still depends on dedicated task definitions for perception, action and drives. Although some of these systems
will rely on strong genetic pre-specification the role of ontogenesis in the
generation of these systems in these systems will be expanded in future
instantiations of the SF-DAC model. Further steps include the development of a basic set of actions, perception and drives that can generalize
6.2. ROBOT EXPERIMENTATION
115
to a larger range of tasks. Such progress is already considered in recent
DAC related work (Duff et al., 2010) and might support what could be
defined as a general synthetic foraging agent.
CHAPTER
Conclusion
This dissertation is an effort towards a definition of The Hippocampus
Code, the underlying computational structure underlying hippocampal
function. In this direction, the set of studies herein presented is attentive
to the ability of hippocampal neurons to be conjunctively selective to
spatial and nonspatial aspects of memory.
In a first stage (Chapter 4 and 5), we presented a systematic explanation
of how the neural network in the hippocampus evokes such conjunctive
selectivity. Further (Chapter 6), we used a system-wide robotic control
architecture to determine, both theoretically and experimentally, why
conjunctive selectivity is relevant for behavior and efficient task-solving.
Hence, our major contributions are the demonstrations of how and why
hippocampal cells become conjunctive, a characteristic that encompasses
both memory and space theories for the hippocampus.
The implications of our findings span throughout a wide range of scientific domains, from biology and neuroscience, behavioral psychology,
cognitive and computer science and robotics. This is illustrated by the di116
7
117
versity of symposia and conferences in which some parts of this work have
been included. Some of these events are: the Society for Neuroscience
meetings in San Diego and New Orleans; the congress of biomimetic
robotics (ROBIO) organized by IEEE in Phuket, Thailand; the Living
Machines robotics conference in Barcelona; the Federation of European
Neurosciences (FENS) meetings in Amsterdam (in which a grant was
guaranteed by the Spanish Society for Neuroscience) and Barcelona; and
the Cognitive Systems (CogSys) conference sponsored by the EUCog network in Vienna.
In what regards to biology and neuroscience we advanced the knowledge
of the rate remapping effect exhibited by the hippocampal place cells.
We posed specific predictions about the importance of brain areas such
as the lateral entorhinal cortex and provided new interpretations of neurophysiological data concerning attractor dynamics, pattern completion
and long term potentiation in the hippocampus. A prove of relevance of
the work is the fact that it was published in a high impact peer reviewed
journal in the neuroscience field (Rennó-Costa et al., 2010a), which included an introductory article by Dupret et al. (2010).
As a natural follow up of these studies remains some open questions regarding the time dynamics of the rate remapping effect. For instance,
how would phase precession affect rate remapping? How does the dynamics of theta cycling changes the internal competition? How is the
CA1 interaction with the rate remapping mechanism? The tools we used
on this work allow the study of other relevant mechanisms of the hippocampus.
The results of the behavioral studies also have they share of relevance
in these fields since it allows a causal link between physiology and behavior. Yet, to accomplish higher impact it requires some refinement
118
CHAPTER 7. CONCLUSION
on the methods to allow a better match to existing behavioral data or
the realization of behavioral studies following the proposed experimental
protocol. Still, the possibilities are very auspicious given that nowadays
causality between behavior and physiology of the hippocampus is only
accomplished indirectly by means of lesion studies.
Far beyond, the ability of using robots will very likely be the ultimate
benchmark for biology models by allowing the closed cycle including perception, cognition and behavior in realistic and natural environments. A
previous study using the same framework and a very simplistic robotic
setup already allowed the identification of the novel biological principle
of environmental feedback (Verschure 2003). With the advance of computational models and the use of integration architectures, as the one
presented in Chapter 6, the robots might soon become an essential tool
to study the hippocampus and the formation of memory by grounding in
behavior. Moreover, this method can potentially affect the study of behavior itself by fostering the identification of its underlying mechanisms.
It is important to phrase that the establishment of this scenario requires
detailed modeling of many parts of the brain and nervous systems. The
current state-of-art already supports simple biomimetic behavior like the
one presented in Chapter 6, but there is high demand for complex and
detailed computational models of the brain.
Not limited to biology, this work also has implications in applied sciences
such as robotics and automation, in computer science and in cognitive
science. The identification of computational mechanisms in biology might
allow the development of biomimetic solutions for real-world engineering
problems. For example, rodents excel in foraging tasks and suit as a
reference model for robots to be designed for tasks that require the same
set of skills such as resource-finding in unfriendly environments (e.g.,
underwater oil exploration) and automated rescue missions in harsh con-
119
ditions (e.g., earthquake and radiation sites). Furthermore, our findings
might also allow the identification of novel algorithms, specially taken in
consideration the advance of neuromorphic computational architectures.
For instance, models of the hippocampus will be able to provide compact
memory sub-systems for non-symbolic architectures.
Although much of these scenarios are already tangible, there’s still a long
road to be trailed before being effective. It rests no doubt however that
the advance of many of the listed fields will be supported by interdisciplinary studies like this dissertation.
APPENDIX
Internal drive regulation of
sensorimotor reflexes
This appendix chapter reproduces the paper "Internal drive regulation
of sensorimotor reflexes in the control of a catering assistant autonomous
robot" published in the proceedings of the Living Machines conference
(Rennó-Costa et al., 2012c). The abstract reads:
We present an autonomous waiter robot control system
based on the reactive layer of the Distributed Adaptive Control (DAC) architecture. The waiterbot has to explore the
space where catering is set and invite the guests to serve themselves with chocolate or candies. The control model is taking
advantage of DAC’s allostatic control system that allows the
selection of actions through the modulation of drive states. In
the robot’s control system two independent behavioral loops
are implemented serving specific goals: a navigation system
to explore the space and a gazing behavior that invites human users to serve themselves. By approaching and gazing
120
A
A.1. INTRODUCTION
121
at a potential consumer the robot performs its serving behavior. The system was tested in a simulated environ-ment and
during a public event where it successfully delivered its wares.
From the observed interactions the effect of drive based selfregulated action in living machines is discussed.
A.1
Introduction
Many day-by-day human tasks require the ability to interact with others.
That is the case of serving chocolate and candies in a social event. The
waiter has to navigate through the hall, approach the guests and invite
them to try what is on his plate. An optimal waiter would be able to
map the space and program the visit to every guest remembering the
time constraints involved. Those are not simple tasks for an autonomous
robot. In this paper, we propose a purely reactive controller aimed to
allow robots to assist catering services. Although it does not envisage a
performance comparable to a human waiter, we suggest that the complexity added to the agent’s behavior by the internal regulation of the
sensorimotor reflexes can establish a non-verbal communication channel
and create a rich and effective serving experience to the user.
The approach taken is to communicate with the guest through gaze
(Knapp & Hall, 2009) and smooth reduction of interpersonal distance
(Lawson, 2001; Inderbitzin et al., 2009). This is accomplished by the
control of a camera mounted on a pan-tilt unit and the navigation of
a mobile robot. The theory we pursue is that complex behavior, such
as effective serving of food might emerge from simple and limited constructions without relying on complex processes such as representations,
memory or inference skills (Braitenberg, 1984). The apparent complexity
of behavior over time is a reflection of the complexity of the environment
in which the agent finds itself (Simon, 1969). By allowing basic behaviors
of gaze and motion to be coordinated through the interaction with the en-
122
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
vironment we aim to achieve emergent behavioral regularities: designing
for emergence. Our approach is essentially different from the traditional
top-down robot design methodology in which the environment and the
possible interactions are parameterized and behavior follows specific and
declarative predefined schemes (Pfeifer & Verschure, 1994; Pfeifer & Bongard, 2006). Designing for emergence might appear more of an art than
a science but we base our methods on the fact that the synergy between
perception and behavior is mediated by the environment in controllable
ways (Verschure et al., 2003). This supports a bottom-up robot design
methodology grounded on a more generic and also simple control system.
The complexity of the interaction is expected to emerge naturally from
the immersion in the environment and from the contact with other agents
such as animals (Lund, 1997), humans (Eng et al., 2003) or other robots
(Asama et al., 1994).
Robots that can establish a closed loop in the sensorimotor interaction
with the environment and generate emergent complex behavior have been
called living machines (Hasslacher & Tilden, 1995), which established
a significant field inside the autonomous robotics community (Bekey,
2005). The objective of the proposed control system is to implement a
living machine with behavioral complexity that matches the environmental requirements and human expectations of the catering task, without
the need for explicitly and centrally declaring the task in the robot control system.
The main source of behavioral complexity in the control system we propose is the dynamic regulation of the sensorimotor loops through internal
drives. Drives are defined as internal states that are defined by the fundamental homeostatic needs of the agent and that adjust the link between
perception and action. Drives are set dynamically by the activity of the
system and may follow different time scales than the time course of a
A.1. INTRODUCTION
123
motor action or the sensorimotor cycle. These two properties together
allow a repertoire of multiple responses to the same stimuli with a varying time frame that might not be perceived as a change in perceptual
reaction (Simons & Levin, 1998).
The implementation of the control system is based on the Distributed
Adap-tive Control (DAC) architecture (Verschure et al., 1992, 2003; Verschure & Althaus, 2003; Duff et al., 2010). DAC is a sensory-action closed
loop embodied control system based on the theories of conditioning and
grounded on its neurophysiological constraints. Its vertical multi-layered
scheme allows the acquisition of internal representations of conditional
stimuli with crescent complexity sup-porting adaptive environmentalmediated behavior (Verschure et al., 2003). The implementation of internal drive in DAC’s reactive layer allows the low-level regulation of vital
needs in an allostatic control framework (Sanchez-Fibla et al., 2010) and
supports goal-oriented behavior by acquiring higher cognitive representations such as rules and plans (Duff et al., 2011).
Although DAC provides the tools for learning and adaptation, we limit
the robot’s controller to an implementation of the reactive layer. The
reactive layer offers a basic repertoire of actions, sensations, reflexes and
drives. Through this basic control system only non-adaptive behavior is
generated since no persistent memory of any kind is implemented. The
emerging sensorimotor dynamics might ultimately form the basis for the
acquisition of internal representations and adaptive behavior (Duff et al.,
2010), so this structure might work as a base for adaptive behaving system working in the same environmental context.
Another important aspect is that DAC has already been used in the control of living machines that interact with humans. It is the case of the
Ada interactive space (Eng et al., 2003). Ada is a room that interacts
124
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
with the user using light and dynamically composed music (Manzolli &
Verschure, 2005). A key element for evoking the sense of interaction in
humans is the ability to generate complex and unpredictable behavior,
which does not look (or sound) like a pure reactive response (Michaud
et al., 2005). In this respect, as a benchmark the control system, it was
used in the control of an unmanned mobile vehicle in a real catering situation during the Future and Emergent Technologies (FET’11) meeting
in Warsaw.
In the following sections we describe the control architecture, the results
of computer simulations and provide a brief report on the public demonstration of the system. We further discuss the effect of drive regulation
and the limitations imposed by the use of purely reactive systems.
A.2
A.2.1
The control architecture
The hardware
The robotic platform used to implement the living machine is a 50 kg
unmanned mo-bile vehicle (124 x 80 x 67 cm) with 6 wheels and an articulated body with 3 subsections (Figure A.1). Its sensory system is
composed by an array of 16 ultrasound and 16 infrared proximity sensors and a Firewire color camera mounted over a pan-tilt unit in the front
body subsection. A notebook (MacBook Pro running custom application
under Linux Ubuntu) that is docked in top of the middle body subsection
performs computer processing. The load to be distributed is placed in
the back body subsection. Robosoft (Bidart, France) developed the platform in cooperation with Universitat Pompeu Fabra (Barcelona, Spain).
As an external interface, an iPad with a custom application allowed the
visualization of all variables in real time, the override of the navigation
loop and emergency locking. The simulation environment used to opti-
A.2. THE CONTROL ARCHITECTURE
125
Figure A.1: Robot overview. The color camera is mounted over the pantilt unit in the front segment of the robot. The control CPU is placed
over the middle segment. The load is placed over the rear segment. The
proximity sensors are placed all around the robot, but only the ones in the
front and in the back are used for the experiment. Robot produced by
Robosoft.
mize the parameters of the reactive controller (Sanchez-Fibla et al., 2010)
allows a control that is similar to that of the real robot. One of the main
features of the simulator is the possibility to customize different virtual
environments and realize experiments in computational time rather than
real time.
A.2.2
The autonomous control system
The autonomous control system is divided in two main sensorimotor
loops: the vision loop and the navigation loop. This is an approximation of the subcortical loops formed by the basal ganglia and brainstem
sensorimotor structures comprise sensing, internal motivational states
and action (McHaffie et al., 2005). While the navigation loop allows
the robot to explore the space searching for unattended guests, the vision loop will permit “eye contact” through the gaze system, showing
to the user the “intention” of the robot to serve that specific person.
126
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
Figure A.2: From face perception to gaze action: the perception/action reflex in the visuomotor loop (A) Illustration of the visual
field of the robot and (B) the associated salience map emitted by the face
detection and salience map system. (C) Through a competitive process the
most salient point in the visual field is selected. If the most salient point
is active in a zone associated with a gaze action, a saccade is activated. In
this specific case, the action “Pan Left” is triggered, moving the detected
face to the center of the visual field. (D) Illustration of the two possible
action types ("tilt" and "pan”) in relation to the gaze direction.
Both sensorimotor loops conform to structures with the same components: perception, action, drive and reflex-es. Perception and action are
the sensor/actuator interfaces to the environment. Drive is the internal
state of the agent. Reflexes are the hardwired connections among these
components. Each control loop is independent and no information or
signals are inter-changed. Moreover, they differ with respect to the types
of perception, action, drive and the specific organization of the reflexes.
A.2. THE CONTROL ARCHITECTURE
127
The vision loop receives sensory input from a color camera and acts
through the motorized pan-tilt unit. The expected behavior is that the
robot tracks faces using the gaze system. Faces are detected by a cascade of boosted classifiers working with Haar-like features and trained
with face samples (Lienhart et al., 2003). The output of the visual processing is a salience map that is aligned to the Cartesian representation of
the retinotopic input (Figure A.2AB) and can be seen as an approximation of the sensori-motor mappings found in the superior colliculus (Song
et al., 2010; Gandhi & Katnani, 2011). We interpret the active control
of the gaze unit in terms of a cerebellum-like saccadic control system
acquired through conditioning-like learning mechanisms (Schweighofer
et al., 1996; Hofstötter et al., 2002; Blazquez et al., 2003). From this
salience map it is possible to determine whether there are relevant eye
movement targets, i.e. faces, in specific areas of the visual field. An
attention mechanism based on competition and predictive anticipation
from the recent response history is used to select one single salient face
in the visual field. The algorithm consists of searching for a peak in the
neighboring area of the last salient point. If no peak is found (highest
value in the neighborhood is below the salience threshold, Ω), it searches
for the highest peak in the whole visual field. The neighborhood is defined as the predicted area of the highest likeness of finding new salient
point given recent history, or an anticipatory gate (Mathews et al., 2009).
The algorithm that define the salient points follows:
neighbor ← computeN eighbor(currentSalientP oint)
if maxSalience(neighbor) > salienceT hreshold then
currentSalientP oint ← maxSalience(neighbor)
else
if maxSalience(visualF ield) > salienceT hreshold then
currentSalientP oint ← maxSalience(visualF ield)
else
currentSalientP oint ← center(visualF ield)
end if
128
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
end if
Reflexes were set connecting the face detection output to the movements
of the pan-tilt unit (Figure A.2CD). The visual field was divided in five
horizontal and five vertical zones. To each zone two reflexes are assigned,
one horizontal and another vertical. The movement triggered by a certain reflex is capable of centering the camera in the active zones.
In the vision sensorimotor loop, the drive variable is analog to the concept of “curiosity” or “novelty” (Figure A.3). Higher levels of curiosity
make the gaze system search for new faces, whilst lower levels of curiosity
make the gaze system stick to a certain face or remain still. The level
of motor action regulates the curiosity level. This is an indirect measure
of the variability of the visual input, which allows an environmental mediation of the drive regulation. This is an visual analog of exploration
behavior (Sanchez-Fibla et al., 2010).
Figure A.3: Visual loop organization. Perception excites action. Action inhibits the drive. The drive inhibits perception. Action and drive
have spontaneous activity. Drives regulates the system by allowing action
spontaneous activity to take place when few actions are triggered.
The perceptual activity is modulated by the curiosity drive through an
inhibitory reflex (Equation A.1). The curiosity drive is set accordingly to
A.2. THE CONTROL ARCHITECTURE
129
the number of actions triggered in a specific time window (δ) in relation to
an activation threshold (TAction ) implemented as a spontaneous activity
(Equation A.2). Positive curiosity levels, caused by low action activity,
lead to the inhibition of perception by a combination of spontaneous activity and an inhibitory reflex from the action set. The action network
also has spontaneous activity, so that when all input is extinguished the
camera will make some random movement. When perception is active it
overrides the spontaneous activity (Equation A.3)). This intrinsic action
by itself constitutes a kind of exploratory searching behavior set when
no perception is available to lead the action. Random search is also regulated by the curiosity drive since the intrinsic actions will lower the
curiosity level, which will allow perception to rule again. This system
allows the gaze control to continuously track for faces and avoids getting
stuck in missclassified points. In pseudo code the visualmotor gaze system could be described as:

Perception
Perception =
0
Drive = TAction −
now−δ
X
Drive ≤ 0,
(A.1)
Drive > 0
Action
(A.2)
t=now

Perception control
Action =
Random control
Perception > 0,
Perception = 0
(A.3)
130
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
Figure A.4: Navigation loop organization. Action has spontaneous
activity. Action can be inhibited by the perception and by the drive. The
drive is activated by the action.
The navigation loop follows a different architecture when compared to
the visual loop (Figure A.4). It receives sensory input from an array of
proximity sensors positioned around the robot body and acts through
motor commands that spin the wheels. The expected behavior is that
the robot runs around the location, avoiding collisions of any kind. Differently to the vision loop, in the navigation loop the reflex from the
perception to the action in inhibitory. Whenever an obstacle is perceived
the motor control is inhibited. Motor action has spontaneous activity,
which when is not inhibit-ed causes the robot to follow a random direction (Equation A.4). Direction is changed randomly and periodically
with a time constant (σ) defined as 10 seconds.
The drive in the navigation task works as a negative feedback. It is set in
a way that the robot periodically stops at a certain position so that the
guests have time to collect the load. The combination between the time
constant (δ) and the drive activation threshold (TAction ) sets the time
the robot takes in the stop phase and the time it stays in the moving
phase (Equation A.5).
A.3. RESULTS
131

0
Action =
Random control
Drive =
now−δ
X
Perception or Drive > 0,
(A.4)
Otherwise
Action − TAction
(A.5)
t=now
A.3
Results
A full demonstration of the living machine was performed during the
Future and Emergent Technologies (FET’11) at the Polytechnic University of Warsaw in November of 2011 (Figure A.5). The robot was used
to distribute chocolates to the attendees of the event during the coffee
breaks. The venue consisted of a large hall of approximately 4000 square
meter filled with tables and other sorts of obstacles.
Before the demonstration, in order to profile the behavioral capability
of the navigation loop and validate the parameters we used the control
mechanism to steer a simulated robot in a virtual environment. For this
experiment we used a drive time constant of 20 seconds and a TAction
of on average 2.5 kilometers/hour or 5 degrees per second. The aim of
the simulations was to verify the ability of the robot to cover the whole
catering space and quantify the quality of its delivery service. We run
five trials of three hours with different areas from 1600 m2 to 7000 m2
(Figure A.6). A specific area was considered “visited” when it was closer
than 3.75 meters from a position where the robot was stationary (drive
above zero). The control paradigm was able to deliver to more than 90%
of the locations in all the arena sizes. Moreover, after 1 hour it was able
132
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
Figure A.5: Demonstration venue (right) and the robot (top-left),
chocolates and candies are placed in a plate located in the back part of the
robot body.
to visit more than 60% of the arena with an area similar to the experimental site. The gaze system was also verified in lab conditions. TAction
was set to .1 movements/second in average and the time constant to 10
seconds. This would imply that if no movement was done in 10 seconds,
the drive would be active and gaze would follow a random direction.
With controlled conditions - single face with white background – the vision loop was able to fix the gaze correctly in all the trials (n=10) and
with the subject stationary it took an average time of 10.5 seconds to
gaze in an-other direction. With the subject moving slowly (< 1 steps
per second or 0.76 m/sec), the gaze system never lost track of the subject
(n=10). In the case in which the subject was moving fast (> 1 steps per
second), the gaze system always lost track of the user when the movement followed continuously in the same direction for more than 2 seconds
(n=10). Movements confined to the view range were successfully tracked
by the vision loop. Thus, the system is highly reliable in localizing stationary or slowly moving human faces with a speed of up to 3 km/h.
A.3. RESULTS
133
Figure A.6: Simulated navigation data. (A) Time evolution (exponential fit) for the percentage of the area covered by delivery stops for five different arena sizes. (B) Sample of robot trajectories and delivery spot spatial
distributions at different time windows for the 5921 m2 square arena.
In the real demonstration, the robot was successful in the task of distributing chocolates. In total, 213 pieces were distributed in a time course
of 10 hours or on the average about 1 piece every 3 minutes. The gaze
system worked for the whole time course of the demonstration and was
effective in calling attention and in creating a connection with the guests
(Figure A.7).
The robot was able to cover autonomously a large part of the hall’s sur-
134
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
Figure A.7: Demonstration of the gazing behavior by the visuomotor loop in a sequence with a moving subject.
face, confirming the simulation results. The variability in the movements
caused by the drive regulation showed to be effective since most of the
guest would only approach the robot when it stopped or was moving
slowly.
We could observe situations in which it got stuck (N=19 in 10 hours).
This would happen mostly in corners and in places with high desk and
tables density. In these cases the navigation system was overridden by a
remote control. Moreover, due to the different kind of materials used in
the room, the proximity sensor could not detect a few obstacles causing
the robot to crash. In moments with high guest density the navigation
system was overridden for security reasons.
Although both control loops were set independently and could not exchange any kind of messages through the computer, the influence of navigation loop over the gaze loop is striking. When the robot was in movement, gaze would keep directed to the same face for a longer time if
compared to when the robot is still. The reason is that the movement of
the robot forced the gaze system to activate the saccade movements more
frequently, extending the time of drive integration. With the parameters
used in the demonstration, the robot would keep the camera focused for
about 10 seconds if the target face remained still. With the robot in
A.4. DISCUSSION
135
movement (> 0.5 meters per second) the robot would only change the
target face in case it lost the target face for a period greater than 1 second (caused mainly by visual obstruction).
A.4
Discussion
We addressed the question whether designing a controller that achieves
its coherence through dynamic interaction with the environment can render robust and effective behavior. In this example of designing for emergence we targeted a delivery service task where a mobile robot had to
deliver chocolate to a naive audience in a public event. The public demonstration of the presented control system is an example of how enhanced
drive based reactive control can lead to emergent behavioral skills sufficient for permitting effective human-robots interaction. More specifically
to the waiter task, it generates gaze and interpersonal distance regulation
behaviors. The approach used is grounded on the regulation of reactive
sensorimotor loops through internal drives that adds complexity to the
performed activity and modulates the reactive response on an uncorrelated time-scale. Most importantly, the robot accomplishes its mission
without the need of any declarative representation of the task or the
other agents involved.
Nevertheless, the demonstration showed that the system is sufficient but
not optimal to reproduce a waiter performance. This is somehow expected since the sys-tem can be enhanced in many ways. But it is important to highlight the sufficiency of the reactive control since it gives
a safe behaving procedure for any system eventually built on top of it.
This form of environmentally mediated allostatic control can be seen as
an hypothesis on how biological systems ultimately support survival in
potentially harmful situations or to satisfy a range of needs (SanchezFibla et al., 2010). Moreover, it would be interesting to observe how
136
APPENDIX A. INTERNAL DRIVE REGULATION OF
SENSORIMOTOR REFLEXES
multiple robots would interact in the performance of the task. Since the
covering of space tended to be confined to a certain region for a limited
time, a simple communication channel in which robots avoid other machines might help to establish zones of action, reducing the active area of
each robot and diminishing the time needed to cover the space through
emergent collaboration.
Regarding the extension of the control system, it is possible to keep the
reactive idea by establishing direct communication between the two control loops. One example would be to use the face detection component
to modulate the drive of the navigation, enhancing the sensation of approaching a guest. However, the clearest possibility is the addition of
more cognitive skills such as memory and planning. The reactive system
can support the construction of highly structured representations when
considering the other two layers of the DAC architecture (Duff et al.,
2010; Duff & Verschure, 2010). The same platform has been controlled
by a full DAC architecture in non-interactive tasks in which we included
features such as mapping, sequence learning, object recognition and spatial memory (Rennó-Costa et al., 2011).
More specifically to the control loops, the visual-motor loop can be enhanced by face recognition itself (Luvizotto et al., 2011). This would
allow the robot to focus on guests who have not yet been served. Regarding the navigation loop, the use of a spatial memory system would
allow a homogeneous covering of the space since it would be possible to
remember where it was before and avoid recently visited locations. Another possibility is to integrate both loops in the mapping of the space
(Verschure et al., 2006).
Bibliography
Aggleton, J. P., Albasser, M. M., Aggleton, D. J., Poirier, G. L., &
Pearce, J. M. (2010). Lesions of the rat perirhinal cortex spare the
acquisition of a complex configural visual discrimination yet impair
object recognition. Behavioral neuroscience, 124(1):55–68. ISSN 19390084. doi: 10.1037/a0018320.
Amit, D. J. (1992). Modeling Brain Function: The World of Attractor
Neural Networks. Cambridge University Press. ISBN 0521421241.
Arleo, A. & Gerstner, W. (2000). Spatial cognition and neuro-mimetic
navigation:
a model of hippocampal place cell activity.
ical Cybernetics, 83(3):287–299.
ISSN 0340-1200.
Biolog-
doi: 10.1007/
s004220000171.
Arleo, A., Smeraldi, F., Hug, S., & Gerstner, W. (2001). Place Cells
and Spatial Navigation based on Vision, Path Integration, and Reinforcement Learning. Dins T. K. Leen, T. G. Dietterich, & V. Tresp,
editors, Advances in Neural Information Processing Systems 13, ps.
89–95. Path Integration, and Reinforcement Learning.&amp;quot; Advances in Neural Information Processing Systems.
Asama, H., Fukuda, T., & Arai, T. (1994). Distributed Autonomous
Robotic Systems. Springer-Verlag. ISBN 0387701478.
Astur, R. S., Taylor, L. B., Mamelak, A. N., Philpott, L., & Sutherland,
R. J. (2002). Humans with hippocampus damage display severe spatial
138
BIBLIOGRAPHY
139
memory impairments in a virtual Morris water task. Behavioural Brain
Research, 132(1):77–84. ISSN 01664328. doi: 10.1016/S0166-4328(01)
00399-0.
Balkenius, C. & Morén, J. (2000). A Computational Model of Context
Processing. Dins J.-A. Meyer, A. Berthoz, D. Floreano, H. L. Roitblat,
& S. W. Wilson, editors, From Animals to Animats 6: Proceedings of
the 6th International Conference on the Simulation of Adaptive Behaviour. The MIT Press, Cambridge, MA, USA.
Bannerman, D. M., Yee, B. K., Good, M. A., Heupel, M. J., Iversen,
S. D., & Rawlins, J. N. (1999). Double dissociation of function within
the hippocampus: a comparison of dorsal, ventral, and complete hippocampal cytotoxic lesions. Behavioral neuroscience, 113(6):1170–88.
ISSN 0735-7044.
Barry, C. & Burgess, N. (2007). Learning in a geometric model of place
cell firing. Hippocampus, 17(9):786–800. ISSN 1050-9631. doi: 10.
1002/hipo.20324.
Barry, C., Hayman, R., Burgess, N., & Jeffery, K. J. (2007).
Experience-dependent rescaling of entorhinal grids.
Nature neuro-
science, 10(6):682–4. ISSN 1097-6256. doi: 10.1038/nn1905.
Barry, C., Lever, C., Hayman, R., Hartley, T., Burton, S., O’Keefe, J.,
Jeffery, K., & Burgess, N. (2006). The boundary vector cell model
of place cell firing and spatial memory. Reviews in the neurosciences,
17(1-2):71–97. ISSN 0334-1763.
Bartos, M., Vida, I., & Jonas, P. (2007). Synaptic mechanisms of
synchronized gamma oscillations in inhibitory interneuron networks.
Nature reviews. Neuroscience, 8(1):45–56.
ISSN 1471-003X.
doi:
10.1038/nrn2044.
Bartsch, T., Schönfeld, R., Müller, F. J., Alfke, K., Leplow, B., Aldenhoff, J., Deuschl, G., & Koch, J. M. (2010). Focal lesions of human
140
BIBLIOGRAPHY
hippocampal CA1 neurons in transient global amnesia impair place
memory. Science (New York, N.Y.), 328(5984):1412–5. ISSN 10959203. doi: 10.1126/science.1188160.
Bekey, G. A. (2005). Autonomous Robots: From Biological Inspiration
to Implementation and Control. MIT Press. ISBN 9780262025782.
Bernardet, U., i Badia, S. B., & Verschure, P. F. M. J. (2007). The eXperience Induction Machine and its Role in the Research on Presence.
Dins The 10th Annual International Workshop on Presence.
Bird, C. M. & Burgess, N. (2008). The hippocampus and memory: insights from spatial processing. Nature reviews. Neuroscience, 9(3):182–
94. ISSN 1471-0048. doi: 10.1038/nrn2335.
Blair, H. T., Welday, A. C., & Zhang, K. (2007).
Scale-invariant
memory representations emerge from moiré interference between grid
fields that produce theta oscillations: a computational model. The
Journal of Neuroscience, 27(12):3211–29.
ISSN 1529-2401.
doi:
10.1523/JNEUROSCI.4724-06.2007.
Blazquez, P. M., Hirata, Y., Heiney, S. A., Green, A. M., & Highstein,
S. M. (2003). Cerebellar signatures of vestibulo-ocular reflex motor
learning. The Journal of neuroscience : the official journal of the
Society for Neuroscience, 23(30):9742–51. ISSN 1529-2401.
Bliss, T. V. & Lø mo, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. The Journal of physiology,
232(2):331–56. ISSN 0022-3751.
Bohbot, V. D., Kalina, M., Stepankova, K., Spackova, N., Petrides, M.,
& Nadel, L. (1998). Spatial memory deficits in patients with lesions
to the right hippocampus and to the right parahippocampal cortex.
Neuropsychologia, 36(11):1217–38. ISSN 0028-3932.
BIBLIOGRAPHY
141
Bragin, A., Jandó, G., Nádasdy, Z., Hetke, J., Wise, K., & Buzsáki,
G. (1995). Gamma (40-100 Hz) oscillation in the hippocampus of the
behaving rat. The Journal of neuroscience : the official journal of the
Society for Neuroscience, 15(1 Pt 1):47–60. ISSN 0270-6474.
Braitenberg, V. (1984). Vehicles: Experiments in synthetic psychology.
MIT Press, Cambridge, MA.
Brotons-Mas, J. R., Montejo, N., O’Mara, S. M., & Sanchez-Vives, M. V.
(2010). Stability of subicular place fields across multiple light and dark
transitions. The European journal of neuroscience, 32(4):648–58. ISSN
1460-9568. doi: 10.1111/j.1460-9568.2010.07308.x.
Brun, V. H., Leutgeb, S., Wu, H.-Q., Schwarcz, R., Witter, M. P., Moser,
E. I., & Moser, M.-B. (2008a). Impaired Spatial Representation in CA1
after Lesion of Direct Input from Entorhinal Cortex. Neuron, 3(2):290–
302. ISSN 0896-6273. doi: 10.1016/j.neuron.2007.11.034.
Brun, V. H., Solstad, T., Kjelstrup, K. B., Fyhn, M., Witter, M. P.,
Moser, E. I., & Moser, M.-B. (2008b). Progressive increase in grid
scale from dorsal to ventral medial entorhinal cortex. Hippocampus,
18(12):1200–12. ISSN 1098-1063. doi: 10.1002/hipo.20504.
Bunsey, M. & Eichenbaum, H. (1996). Conservation of hippocampal
memory function in rats and humans. Nature, 379(6562):255–7. ISSN
0028-0836. doi: 10.1038/379255a0.
Burgess, N., Barry, C., & O’Keefe, J. (2007). An oscillatory interference
model of grid cell firing. Hippocampus, 17(9):801–12. ISSN 1050-9631.
doi: 10.1002/hipo.20327.
Burgess, N., Donnett, J. G., Jeffery, K. J., & O’Keefe, J. (1997).
Robotic and neuronal simulation of the hippocampus and rat navigation. Philosophical transactions of the Royal Society of London.
Series B, Biological sciences, 352(1360):1535–43. ISSN 0962-8436. doi:
10.1098/rstb.1997.0140.
142
BIBLIOGRAPHY
Burwell, R. D. (2000). The parahippocampal region: corticocortical connectivity. Annals of the New York Academy of Sciences, 911:25–42.
ISSN 0077-8923.
Burwell, R. D. & Amaral, D. G. (1998). Perirhinal and postrhinal cortices of the rat: interconnectivity and connections with the entorhinal
cortex. The Journal of comparative neurology, 391(3):293–321. ISSN
0021-9967.
Burwell, R. D., Saddoris, M. P., Bucci, D. J., & Wiig, K. A. (2004). Corticohippocampal contributions to spatial and contextual learning. The
Journal of neuroscience : the official journal of the Society for Neuroscience, 24(15):3826–36. ISSN 1529-2401. doi: 10.1523/JNEUROSCI.
0410-04.2004.
Calhoun, J. B. (1963). The ecology and sociology of the Norway rat. U.S.
Dept. of Health, Education, and Welfare, Public Health Service.
Canto, C. B., Wouterlood, F. G., & Witter, M. P. (2008). What does
the anatomical organization of the entorhinal cortex tell us? Neural
plasticity, 2008:381243. ISSN 1687-5443. doi: 10.1155/2008/381243.
Carlsen, J., De Olmos, J., & Heimer, L. (1982). Tracing of two-neuron
pathways in the olfactory system by the aid of transneuronal degeneration: projections to the amygdaloid body and hippocampal formation.
The Journal of comparative neurology, 208(2):196–208. ISSN 00219967. doi: 10.1002/cne.902080208.
Chen, H. & Zhu, C.-a. (2005). Modeling the dynamics of biomimetic underwater robot fish. Dins IEEE International Conference on Robotics
and Biomimetics - ROBIO, ps. 478–483. IEEE. ISBN 0-7803-9315-5.
doi: 10.1109/ROBIO.2005.246314.
Cohen, N. J. & Corkin, S. (1981). The amnesic patient H. M.: Learning
and retention of cognitive skill. Society for Neuroscience Abstracts,
7:517–518.
BIBLIOGRAPHY
143
Cohen, N. J. & Squire, L. R. (1980). Preserved learning and retention
of pattern-analyzing skill in amnesia: dissociation of knowing how and
knowing that. Science (New York, N.Y.), 210(4466):207–10. ISSN
0036-8075.
Corkin, S. (1968).
Acquisition of motor skill after bilateral me-
dial temporal-lobe excision. Neuropsychologia, 6(3):255–265. ISSN
00283932. doi: 10.1016/0028-3932(68)90024-9.
Corkin, S. (2002). What’s new with the amnesic patient H.M.? Nature
reviews. Neuroscience, 3(2):153–60. ISSN 1471-003X. doi: 10.1038/
nrn726.
Crapse, T. B. & Sommer, M. A. (2008). Corollary discharge across the
animal kingdom. Nature reviews. Neuroscience, 9(8):587–600. ISSN
1471-0048. doi: 10.1038/nrn2457.
Davis, H. (1996). Underestimating the rat’s intelligence. Brain research.
Cognitive brain research, 3(3-4):291–8. ISSN 0926-6410.
de Almeida, L., Idiart, M., & Lisman, J. E. (2007). Memory retrieval time
and memory capacity of the CA3 network: role of gamma frequency
oscillations. Learning & Memory, 14(11):795–806. ISSN 1549-5485.
doi: 10.1101/lm.730207.
de Almeida, L., Idiart, M., & Lisman, J. E. (2009a). A second function of
gamma frequency oscillations: an E%-max winner-take-all mechanism
selects which cells fire. The Journal of Neuroscience, 29(23):7497–7503.
ISSN 1529-2401. doi: 10.1523/JNEUROSCI.6044-08.2009.
de Almeida, L., Idiart, M., & Lisman, J. E. (2009b). The input-output
transformation of the hippocampal granule cells: from grid cells to
place fields. The Journal of Neuroscience, 29(23):7504–7512. ISSN
1529-2401. doi: 10.1523/JNEUROSCI.6048-08.2009.
144
BIBLIOGRAPHY
de Almeida, L., Idiart, M., & Lisman, J. E. (2010). The single place fields
of CA3 cells: A two-stage transformation from grid cells. Hippocampus,
22(2):200–8. ISSN 1098-1063. doi: 10.1002/hipo.20882.
De Valois, R. L. & De Valois, K. K. (1990). Spatial Vision. Oxford Univ.
Press, New York.
Deadwyler, S. A., Bunn, T., & Hampson, R. E. (1996). Hippocampal
ensemble activity during spatial delayed-nonmatch-to-sample performance in rats. Journal of Neuroscience, 16(1):354–372.
Degenetais, E. (2003). Synaptic Influence of Hippocampus on Pyramidal
Cells of the Rat Prefrontal Cortex: An In Vivo Intracellular Recording
Study. Cerebral Cortex, 13(7):782–792. ISSN 1460-2199. doi: 10.1093/
cercor/13.7.782.
Dijkstra, E. W. (1959). A note on two problems in connexion with graphs.
Numerische Mathematik, 1(1):269–271. ISSN 0029-599X. doi: 10.1007/
BF01386390.
Doeller, C. F., Barry, C., & Burgess, N. (2010). Evidence for grid cells in
a human memory network. Nature, 463(7281):657–61. ISSN 1476-4687.
doi: 10.1038/nature08704.
Duff, A., Rennó-Costa, C., Marcos, E., Luvizotto, A., Giovannucci, A.,
Sanchez-Fibla, M., Bernardet, U., & Verschure, P. (2010). From Motor
Learning to Interaction Learning in Robots, volum 264 de Studies in
Computational Intelligence. Springer Berlin Heidelberg, Berlin, Heidelberg. ISBN 978-3-642-05180-7. doi: 10.1007/978-3-642-05181-4.
Duff, A., Sanchez Fibla, M., & Verschure, P. F. M. J. (2011). A biologically based model for the integration of sensory-motor contingencies in
rules and plans: A prefrontal cortex based extension of the Distributed
Adaptive Control architecture. Brain research bulletin, 85(5):289–304.
ISSN 1873-2747. doi: 10.1016/j.brainresbull.2010.11.008.
BIBLIOGRAPHY
145
Duff, A. & Verschure, P. F. M. J. (2010). Unifying perceptual and behavioral learning with a correlative subspace learning rule. Neurocomputing, 73(10-12):1818–1830. ISSN 09252312. doi: 10.1016/j.neucom.
2009.11.048.
Dupret, D., Pleydell-Bouverie, B., & Csicsvari, J. (2010). Rate remapping: when the code goes beyond space. Neuron, 68(6):1015–6. ISSN
1097-4199. doi: 10.1016/j.neuron.2010.12.011.
Eichenbaum, H. (2000). Hippocampus: Mapping or memory?
rent Biology, 10(21):R785–R787.
ISSN 09609822.
Cur-
doi: 10.1016/
S0960-9822(00)00763-6.
Eichenbaum, H. (2001). The hippocampus and declarative memory: cognitive mechanisms and neural codes. Behavioural brain research, 127(12):199–207. ISSN 0166-4328.
Ekstrom, A. D., Kahana, M. J., Caplan, J. B., Fields, T. A., Isham,
E. A., Newman, E. L., & Fried, I. (2003). Cellular networks underlying
human spatial navigation. Nature, 425(6954):184–8. ISSN 1476-4687.
doi: 10.1038/nature01964.
Eng, K., Babler, A., Bernardet, U., Blanchard, M., Costa, M., Delbruck,
T., Douglas, R., Hepp, K., Klein, D., Manzolli, J., Mintz, M., Roth, F.,
Rutishauser, U., Wassermann, K., Whatley, A., Wittmann, A., Wyss,
R., & Verschure, P. F. (2003). Ada - intelligent space: an artificial
creature for the SwissExpo.02. Dins IEEE International Conference
on Robotics and Automation (ICRA), volum 3, ps. 4154–4159. IEEE.
ISBN 0-7803-7736-2. doi: 10.1109/ROBOT.2003.1242236.
Ennaceur, A., Neave, N., & Aggleton, J. P. (1997). Spontaneous object
recognition and object location memory in rats: the effects of lesions
in the cingulate cortices, the medial prefrontal cortex, the cingulum
bundle and the fornix. Experimental brain research. Experimentelle
146
BIBLIOGRAPHY
Hirnforschung. Expérimentation cérébrale, 113(3):509–19. ISSN 00144819.
Erdem, U. M. & Hasselmo, M. (2012). A goal-directed spatial navigation model using forward trajectory planning based on grid cells. The
European journal of neuroscience. ISSN 1460-9568. doi: 10.1111/j.
1460-9568.2012.08015.x.
Etienne, A. S., Maurer, R., Berlie, J., Reverdin, B., Rowe, T., Georgakopoulos, J., & Séguinot, V. (1998). Navigation through vector addition. Nature, 396(6707):161–4. ISSN 0028-0836. doi: 10.1038/24151.
Etienne, A. S., Maurer, R., Boulens, V., Levy, A., & Rowe, T. (2004).
Resetting the path integrator: a basic condition for route-based navigation. The Journal of experimental biology, 207(Pt 9):1491–508. ISSN
0022-0949.
Fenton, A. A., Csizmadia, G., & Muller, R. U. (2000a). Conjoint control
of hippocampal place cell firing by two visual stimuli. I. The effects
of moving the stimuli on firing field positions. The Journal of general
physiology, 116(2):191–209. ISSN 0022-1295.
Fenton, A. A., Csizmadia, G., & Muller, R. U. (2000b). Conjoint control
of hippocampal place cell firing by two visual stimuli. II. A vectorfield theory that predicts modifications of the representation of the
environment. The Journal of general physiology, 116(2):211–21. ISSN
0022-1295.
Fenton, A. A., Lytton, W. W., Barry, J. M., Lenck-Santini, P.-P., Zinyuk,
L. E., Kubík, S., Bures, J., Poucet, B., Muller, R. U., & Olypher,
A. V. (2010). Attention-like modulation of hippocampus place cell
discharge. The Journal of neuroscience : the official journal of the
Society for Neuroscience, 30(13):4613–25. ISSN 1529-2401. doi: 10.
1523/JNEUROSCI.5576-09.2010.
BIBLIOGRAPHY
147
Foster, D. J., Morris, R. G., & Dayan, P. (2000). A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus, 10(1):1–16. ISSN 1050-9631. doi: 10.1002/
(SICI)1098-1063(2000)10:1<1::AID-HIPO1>3.0.CO;2-1.
Foster, D. J. & Wilson, M. A. (2006). Reverse replay of behavioural
sequences in hippocampal place cells during the awake state. Nature,
440(7084):680–3. ISSN 1476-4687. doi: 10.1038/nature04587.
Frank, L. M., Brown, E. N., & Wilson, M. (2000). Trajectory Encoding
in the Hippocampus and Entorhinal Cortex. Neuron, 27(1):169–178.
ISSN 08966273. doi: 10.1016/S0896-6273(00)00018-0.
Fried, I., MacDonald, K. A., & Wilson, C. L. (1997). Single Neuron
Activity in Human Hippocampus and Amygdala during Recognition
of Faces and Objects. Neuron, 18(5):753–765. ISSN 08966273. doi:
10.1016/S0896-6273(00)80315-3.
Fyhn, M., Hafting, T., Treves, A., Moser, M.-b., & Moser, E. I. (2007).
Hippocampal remapping and grid realignment in entorhinal cortex.
Nature, 446(7489):190–194. doi: 10.1038/nature05601.
Fyhn, M., Molden, S., Witter, M. P., Moser, E. I., & Moser, M.B. (2004).
Spatial representation in the entorhinal cortex.
Sci-
ence (New York, N.Y.), 305(5688):1258–64. ISSN 1095-9203. doi:
10.1126/science.1099901.
Gandhi, N. J. & Katnani, H. A. (2011). Motor functions of the superior
colliculus. Annual review of neuroscience, 34:205–31. ISSN 1545-4126.
doi: 10.1146/annurev-neuro-061010-113728.
Gaskin, S., Tremblay, A., & Mumby, D. G. (2003). Retrograde and
anterograde object recognition in rats with hippocampal lesions. Hippocampus, 13(8):962–9. ISSN 1050-9631. doi: 10.1002/hipo.10154.
148
BIBLIOGRAPHY
Gaussier, P., Revel, A., Banquet, J. P., & Babeau, V. (2002). From
view cells and place cells to cognitive map learning: processing stages
of the hippocampal system. Biological cybernetics, 86(1):15–28. ISSN
0340-1200.
Giocomo, L. M., Moser, M.-B., & Moser, E. I. (2011). Computational
models of grid cells. Neuron, 71(4):589–603. ISSN 1097-4199. doi:
10.1016/j.neuron.2011.07.023.
Giraldeau, L.-A. & Caraco, T. (2000). Social Foraging Theory. Princeton
University Press. ISBN 0691048762.
Girardeau, G. & Zugaro, M. (2011). Hippocampal ripples and memory
consolidation. Current opinion in neurobiology, 21(3):452–9. ISSN
1873-6882. doi: 10.1016/j.conb.2011.02.005.
Gothard, K. M., Skaggs, W. E., & McNaughton, B. L. (1996). Dynamics
of mismatch correction in the hippocampal ensemble code for space:
interaction between path integration and environmental cues. Journal
of Neuroscience, 16(24):8027–8040.
Gray, J. A. & McNaughton, N. (2000). The Neuropsychology of Anxiety: An Enquiry into the Functions of the Septo-Hippocampal System
(Oxford Science Publications). Oxford University Press, USA. ISBN
0198522703.
Green, J. D. & Arduini, A. A. (1954). Hippocampal activity in arousal.
J Neurophysiol, 17(6):533–557.
Guanella, A. & Verschure, P. (2006). A Model of Grid Cells Based on
a Path Integration Mechanism. Dins S. D. Kollias, A. Stafylopatis,
W. Duch, & E. Oja, editors, Artificial Neural Networks – ICANN
2006, volum 4131 de Lecture Notes in Computer Science, ps. 740–
749. Springer Berlin Heidelberg, Berlin, Heidelberg, 4131 edició. ISBN
978-3-540-38625-4. doi: 10.1007/11840817.
BIBLIOGRAPHY
149
Guanella, A. & Verschure, P. F. M. J. (2007). Prediction of the position
of an animal based on populations of grid and place cells: a comparative simulation study. Journal of integrative neuroscience, 6(3):433–46.
ISSN 0219-6352.
Hafting, T., Fyhn, M., Molden, S., Moser, M.-B., & Moser, E. I. (2005).
Microstructure of a spatial map in the entorhinal cortex. Nature,
436:801–806.
Hargreaves, E. L., Rao, G., Lee, I., & Knierim, J. J. (2005). Major
Dissociation Between Medial and Lateral Entorhinal Input to Dorsal
Hippocampus. Science, 1792(5729):1792–1794. doi: 10.1126/science.
1110449.
Harrison, F. E., Reiserer, R. S., Tomarken, A. J., & McDonald, M. P.
(2006). Spatial and nonspatial escape strategies in the Barnes maze.
Learning & memory (Cold Spring Harbor, N.Y.), 13(6):809–19. ISSN
1072-0502. doi: 10.1101/lm.334306.
Hartley, T., Burgess, N., O’Keefe, J., Lever, C., & Cacucci, F. (2000).
Modeling Place Fields in Terms of the Cortical Inputs to the Hippocampus. Hippocampus, 10(4):369–79. ISSN 1050-9631. doi: 10.
1002/1098-1063(2000)10:4<369::AID-HIPO3>3.0.CO;2-0.
Hasselmo, M. E. (2008). Grid cell mechanisms and function: contributions of entorhinal persistent spiking and phase resetting. Hippocampus, 18(12):1213–29. ISSN 1098-1063. doi: 10.1002/hipo.20512.
Hasselmo, M. E. & Brandon, M. P. (2012). A model combining oscillations and attractor dynamics for generation of grid cell firing. Frontiers
in Neural Circuits, 6(30).
Hasselmo, M. E., Fransen, E., Dickson, C., & Alonso, A. A. (2000).
Computational modeling of entorhinal cortex. Annals of the New York
Academy of Sciences, 911:418–46. ISSN 0077-8923.
150
BIBLIOGRAPHY
Hasselmo, M. E., Giocomo, L. M., & Zilli, E. A. (2007). Grid cell firing
may arise from interference of theta frequency membrane potential
oscillations in single neurons. Hippocampus, 17(12):1252–1271. ISSN
10509631. doi: 10.1002/hipo.20374.
Hasselmo, M. E. & Stern, C. E. (2006). Mechanisms underlying working memory for novel information.
Trends in cognitive sciences,
10(11):487–93. ISSN 1364-6613. doi: 10.1016/j.tics.2006.09.005.
Hasslacher, B. & Tilden, M. W. (1995). Living machines. Robotics and
Autonomous Systems, 15(1-2):143–169. ISSN 09218890. doi: 10.1016/
0921-8890(95)00019-C.
Hayman, R. M. & Jeffery, K. J. (2008).
How heterogeneous place
cell responding arises from homogeneous grids–a contextual gating
hypothesis.
Hippocampus, 18(12):1301–13.
ISSN 1098-1063.
doi:
10.1002/hipo.20513.
Hebb, D. O. (1932). Studies of the organization of behavior. I. Behavior
of the rat in a field orientation. Journal of Comparative Psychology,
25:333–353.
Herrnstein, R. J. (1970). On the law of effect. Journal of the experimental
analysis of behavior, 13(2):243–66. ISSN 0022-5002.
Hofstötter, C., Mintz, M., & Verschure, P. F. M. J. (2002). The cerebellum in action: a simulation and robotics study. The European journal
of neuroscience, 16(7):1361–76. ISSN 0953-816X.
Hopfield, J. J. (1982).
Neural Networks and Physical Systems with
Emergent Collective Computational Abilities. Proceedings of the National Academy of Sciences, 79(8):2554–2558. ISSN 0027-8424. doi:
10.1073/pnas.79.8.2554.
Hori, E., Nishio, Y., Kazui, K., Umeno, K., Tabuchi, E., Sasaki, K.,
Endo, S., Ono, T., & Nishijo, H. (2005). Place-related neural responses
BIBLIOGRAPHY
151
in the monkey hippocampal formation in a virtual space. Hippocampus,
15(8):991–6. ISSN 1050-9631. doi: 10.1002/hipo.20108.
Hough, G. E. & Bingman, V. P. (2004). Spatial response properties of
homing pigeon hippocampal neurons: correlations with goal locations,
movement between goals, and environmental context in a radial-arm
arena. Journal of comparative physiology. A, Neuroethology, sensory,
neural, and behavioral physiology, 190(12):1047–62. ISSN 0340-7594.
doi: 10.1007/s00359-004-0562-z.
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005). Fast
readout of object identity from macaque inferior temporal cortex.
Science (New York, N.Y.), 310(5749):863–6. ISSN 1095-9203. doi:
10.1126/science.1117593.
Hyman, J. M., Zilli, E. A., Paley, A. M., & Hasselmo, M. E. (2005).
Medial prefrontal cortex cells show dynamic modulation with the hippocampal theta rhythm dependent on behavior. Hippocampus 15:
739–749.
Inderbitzin, M., Wierenga, S., Väljamäe, A., Bernardet, U., & Verschure,
P. F. M. J. (2009). Social cooperation and competition in the mixed
reality space eXperience Induction Machine XIM. Virtual Reality,
13(3):153–158. ISSN 1359-4338. doi: 10.1007/s10055-009-0119-0.
Jeffery, K. J. (2011). Place Cells, Grid Cells, Attractors, and Remapping.
Neural Plasticity. doi: 10.1155/2011/182602.
Jensen, O. & Lisman, J. E. (1996). Novel lists of 7 +/- 2 known items
can be reliably stored in an oscillatory short-term memory network:
interaction with long-term memory. Learning & memory (Cold Spring
Harbor, N.Y.), 3(2-3):257–63. ISSN 1072-0502.
Jin, Y. & Meng, Y. (2011). Morphogenetic Robotics: An Emerging
New Field in Developmental Robotics. IEEE Transactions on Systems,
152
BIBLIOGRAPHY
Man, and Cybernetics, Part C (Applications and Reviews), 41(2):145–
160. ISSN 1094-6977. doi: 10.1109/TSMCC.2010.2057424.
Johnson, A. & Redish, A. D. (2007). Neural ensembles in CA3 transiently
encode paths forward of the animal at a decision point. The Journal
of neuroscience : the official journal of the Society for Neuroscience,
27(45):12176–89. ISSN 1529-2401. doi: 10.1523/JNEUROSCI.3761-07.
2007.
Johnston, D. & Amaral, D. G. (1998). Hippocampus. Dins G. M. Shepherd, editor, The synaptic organization of the brain, ps. 417–458. Oxford UP, New York, 4 edició.
Jung, M. W. & McNaughton, B. L. (1993). Spatial selectivity of unit
activity in the hippocampal granular layer. Hippocampus, 3(2):165–82.
ISSN 1050-9631. doi: 10.1002/hipo.450030209.
Knapp, M. L. & Hall, J. A. (2009). Nonverbal Communication in Human
Interaction. Cengage Learning. ISBN 0495568694.
Koene, R. A. & Hasselmo, M. E. (2007). First-in-first-out item replacement in a model of short-term memory based on persistent spiking.
Cerebral cortex (New York, N.Y. : 1991), 17(8):1766–81. ISSN 10473211. doi: 10.1093/cercor/bhl088.
Kulvicius, T., Tamosiunaite, M., Ainge, J., Dudchenko, P., & Wörgötter,
F. (2008). Odor supported place cell model and goal navigation in
rodents. Journal of computational neuroscience, 25(3):481–500. ISSN
1573-6873. doi: 10.1007/s10827-008-0090-x.
Langston, R. F., Ainge, J. A., Couey, J. J., Canto, C. B., Bjerknes, T. L.,
Witter, M. P., Moser, E. I., & Moser, M.-B. (2010). Development of
the spatial representation system in the rat. Science (New York, N.Y.),
328(5985):1576–80. ISSN 1095-9203. doi: 10.1126/science.1188210.
BIBLIOGRAPHY
153
Lansink, C. S., Goltstein, P. M., Lankelma, J. V., McNaughton, B. L.,
& Pennartz, C. M. A. (2009). Hippocampus leads ventral striatum in
replay of place-reward information. PLoS biology, 7(8):e1000173. ISSN
1545-7885. doi: 10.1371/journal.pbio.1000173.
Lawson, B. (2001). The Language of Space. Routledge Chapman & Hall.
ISBN 0750652462.
Lee, A. K. & Wilson, M. A. (2002). Memory of sequential experience
in the hippocampus during slow wave sleep. Neuron, 36(6):1183–94.
ISSN 0896-6273.
Leonard, J. & Durrant-Whyte, H. (1991). Simultaneous map building
and localization for an autonomous mobile robot. Dins Proceedings
IROS ’91:IEEE/RSJ International Workshop on Intelligent Robots and
Systems ’91, ps. 1442–1447. IEEE. ISBN 0-7803-0067-X. doi: 10.1109/
IROS.1991.174711.
Lepora, N. F., Pearson, M. J., Mitchinson, B., Evans, M., Fox, C., Pipe,
A., Gurney, K., & Prescott, T. J. (2010). Naive Bayes novelty detection
for a moving robot with whiskers. Dins IEEE International Conference
on Robotics and Biomimetics, ps. 131–136. IEEE.
Leutgeb, J. K., Leutgeb, S., Moser, M.-B., & Moser, E. I. (2007). Pattern
separation in the dentate gyrus and CA3 of the hippocampus. Science,
315(5814):961–966. ISSN 1095-9203. doi: 10.1126/science.1135801.
Leutgeb, S., Leutgeb, J. K., Barnes, C. A., Moser, E. I., McNaughton,
B. L., & Moser, M.-B. (2005).
Independent codes for spatial
and episodic memory in hippocampal neuronal ensembles. Science,
309(5734):619–23. ISSN 1095-9203. doi: 10.1126/science.1114037.
Lienhart, R., Kuranov, E., & Pisarevsky, V. (2003). Empirical Analysis of
Detection Cascades of Boosted Classifiers for Rapid Object Detection.
IN DAGM 25TH PATTERN RECOGNITION SYMPOSIUM, ps. 297
– 304.
154
BIBLIOGRAPHY
Lisman, J. & Idiart, M. (1995). Storage of 7 +/- 2 short-term memories in
oscillatory subcycles. Science, 267(5203):1512–1515. ISSN 0036-8075.
doi: 10.1126/science.7878473.
Lisman, J. E. (1999). Relating hippocampal circuitry to function: Recall
of memory sequences by reciprocal dentate-CA3 interactions. Neuron,
22(2):233–242. ISSN 0896-6273.
Lisman, J. E. (2007).
pocampus:
Role of the dual entorhinal inputs to hip-
a hypothesis based on cue/action (non-self/self) cou-
plets. Progress in Brain Research, 163:615–626. ISSN 00796123. doi:
10.1016/S0079-6123(07)63033-7.
Lisman, J. E., Talamini, L. M., & Raffone, A. (2005). Recall of memory
sequences by interaction of the dentate and CA3: a revised model of
the phase precession. Neural networks : the official journal of the International Neural Network Society, 18(9):1191–201. ISSN 0893-6080.
doi: 10.1016/j.neunet.2005.08.008.
Lø mo, T. (1966). Frequency potentiation of excitatory synaptic activity
in the dentate area of the hippocampal formation. Acta Physiologica
Scandinavica, 68(128).
Louie, K. & Wilson, M. A. (2001). Temporally structured replay of awake
hippocampal ensemble activity during rapid eye movement sleep. Neuron, 29(1):145–56. ISSN 0896-6273.
Lowe, D. (1999). Object recognition from local scale-invariant features.
Dins Proceedings of the Seventh IEEE International Conference on
Computer Vision, ps. 1150–1157 vol.2. IEEE. ISBN 0-7695-0164-8.
doi: 10.1109/ICCV.1999.790410.
Lund, H. H. (1997). Robot-Animal Interaction. Dins W. Gerstner,
A. Germond, M. Hasler, & J.-D. Nicoud, editors, Artificial Neural
Networks — ICANN’97, volum 1327 de Lecture Notes in Computer
BIBLIOGRAPHY
155
Science, ps. 745–750. Springer-Verlag, Berlin/Heidelberg. ISBN 3-54063631-5. doi: 10.1007/BFb0020124.
Luvizotto, A., Rennó-Costa, C., Pattacini, U., & Verschure, P. F. M. J.
(2011). The encoding of complex visual stimuli by a canonical model
of the primary visual cortex: Temporal population code for face recognition on the iCub robot. Dins 2011 IEEE International Conference
on Robotics and Biomimetics, ps. 313–318. IEEE, Phuket, Thailand.
ISBN 9781457721373. doi: 10.1109/ROBIO.2011.6181304.
Luvizotto, A., Rennó-Costa, C., & Verschure, P. F. M. J. (2012). A
wavelet-based neural model to optimize and read out a temporal population code. Frontiers in computational neuroscience, 6:21. ISSN
1662-5188. doi: 10.3389/fncom.2012.00021.
MacDonald, C. J., Lepage, K. Q., Eden, U. T., & Eichenbaum, H. (2011).
Hippocampal "time cells" bridge the gap in memory for discontiguous
events. Neuron, 71(4):737–49. ISSN 1097-4199. doi: 10.1016/j.neuron.
2011.07.012.
MacDonall, J. S., Goodell, J., & Juliano, A. (2006). Momentary maximizing and optimal foraging theories of performance on concurrent VR
schedules. Behavioural Processes, 72(3):283–299.
Manns, J. R. & Eichenbaum, H. (2006). Evolution of Declarative Memory. Hippocampus, 16(9):795–808.
Manns, J. R. & Squire, L. R. (2001). Perceptual learning, awareness, and
the hippocampus. Hippocampus, 11(6):776–82. ISSN 1050-9631. doi:
10.1002/hipo.1093.
Manzolli, J. & Verschure, P. F. M. J. (2005). Roboser: A Real-World
Composition System. Computer Music Journal, 29(3):55–74. ISSN
0148-9267. doi: 10.1162/0148926054798133.
156
BIBLIOGRAPHY
Marcos, E., Duff, A., Sanchez-Fibla, M., & Verschure, P. F. M. J. (2010).
The neuronal substrate underlying order and interval representations in
sequential tasks: A biologically based robot study. Dins IEEE/IJCNN
International Joint Conference on Neural Networks, ps. 1–8. IEEE.
ISBN 978-1-4244-6916-1. doi: 10.1109/IJCNN.2010.5596919.
Markus, E. J., Barnes, C. A., McNaughton, B. L., Gladden, V. L.,
& Skaggs, W. E. (1994). Spatial information content and reliability
of hippocampal CA1 neurons: effects of visual input. Hippocampus,
4(4):410–21. ISSN 1050-9631. doi: 10.1002/hipo.450040404.
Marr, D. (1971). Simple Memory: A Theory for Archicortex. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences (1934-1990), 262(841):23–81. ISSN 0080-4622. doi:
10.1098/rstb.1971.0078.
Mathews, Z., Lechon, M., Calvo, J. B., Dhir, A., Duff, A., Bermudez i
Badia, S., & Verschure, P. F. (2009). Insect-Like mapless navigation
based on head direction cells and contextual learning using chemovisual sensors. Dins IEEE/RSJ International Conference on Intelligent
Robots and Systems, ps. 2243–2250. IEEE. ISBN 978-1-4244-3803-7.
doi: 10.1109/IROS.2009.5354264.
Mazer, J. A., Vinje, W. E., McDermott, J., Schiller, P. H., & Gallant,
J. L. (2002). Spatial frequency and orientation tuning dynamics in area
V1. Proceedings of the National Academy of Sciences of the United
States of America, 99(3):1645–50. ISSN 0027-8424. doi: 10.1073/pnas.
022638499.
McDonald, A. J. & Mascagni, F. (1996). Cortico-cortical and corticoamygdaloid projections of the rat occipital cortex: a Phaseolus vulgaris
leucoagglutinin study. Neuroscience, 71(1):37–54. ISSN 0306-4522.
McDonald, R. J., Jones, J., Richards, B., & Hong, N. S. (2006). A
double dissociation of dorsal and ventral hippocampal function on a
BIBLIOGRAPHY
157
learning and memory task mediated by the dorso-lateral striatum. The
European journal of neuroscience, 24(6):1789–801. ISSN 0953-816X.
doi: 10.1111/j.1460-9568.2006.05064.x.
McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., & Redgrave,
P. (2005). Subcortical loops through the basal ganglia. Trends in
neurosciences, 28(8):401–7. ISSN 0166-2236. doi: 10.1016/j.tins.2005.
06.006.
McNaughton, B. L., Battaglia, F. P., Jensen, O., Moser, E. I., & Moser,
M.-B. (2006). Path integration and the neural basis of the ’cognitive
map’. Nature reviews. Neuroscience, 7(8):663–78. ISSN 1471-003X.
doi: 10.1038/nrn1932.
McNaughton, B. L., O’Keefe, J., & Barnes, C. A. (1983). The stereotrode:
a new technique for simultaneous isolation of several single units in
the central nervous system from multiple unit records. Journal of
neuroscience methods, 8(4):391–7. ISSN 0165-0270.
Michaud, F., Brosseau, Y., Cote, C., Letourneau, D., Moisan, P., Ponchon, A., Raievsky, C., Valin, J.-M., Beaudryy, E., & Kabanza, F.
(2005). Modularity and integration in the design of a socially interactive robot. Dins IEEE International Workshop on Robot and Human
Interactive Communication., ps. 172–177. IEEE. ISBN 0-7803-9274-4.
doi: 10.1109/ROMAN.2005.1513775.
Min, M.-Y., Asztely, F., Kokaia, M., & Kullmann, D. M. (1998). Longterm potentiation and dual-component quantal signaling in the dentate
gyrus. Proceedings of the National Academy of Sciences of the United
States of America, 95(April):4702–4707.
Morris, R. G. M., Garrud, P., Rawlins, J. N. P., & O’Keefe, J. (1982).
Place navigation impaired in rats with hippocampal lesions. Nature,
297(5868):681–683. ISSN 0028-0836. doi: 10.1038/297681a0.
158
BIBLIOGRAPHY
Muller, R. (1996). A quarter of a century of place cells.
17(5):813–22. ISSN 0896-6273.
Neuron,
Muller, R., Bostock, E., Taube, J., & Kubie, J. (1994). On the directional
firing properties of hippocampal place cells. J. Neurosci., 14(12):7235–
7251.
Muller, R. & Kubie, J. (1987). The effects of changes in the environment
on the spatial firing of hippocampal complex-spike cells. J. Neurosci.,
7(7):1951–1968.
Muller, R. U., Kubie, J. L., Bostock, E. M., Taube, J. S., & Quirk,
G. J. (1991). Spatial firing correlates of neurons in the hippocampal
formation of freely moving rats. Dins J. Paillard, editor, Brain and
Space, ps. 296–333. Oxford University Press. ISBN 0198542844.
Mumby, D. G., Gaskin, S., Glenn, M. J., Schramek, T. E., & Lehmann,
H. (2002). Hippocampal damage and exploratory preferences in rats:
memory for objects, places, and contexts. Learning & memory (Cold
Spring Harbor, N.Y.), 9(2):49–57. ISSN 1072-0502. doi: 10.1101/lm.
41302.
Murray, E. A., Bussey, T. J., & Saksida, L. M. (2007). Visual perception
and memory: a new view of medial temporal lobe function in primates
and rodents. Annual review of neuroscience, 30:99–122. ISSN 0147006X. doi: 10.1146/annurev.neuro.29.051605.113046.
Myhrer, T. (1988). Exploratory behavior and reaction to novelty in
rats with hippocampal perforant path systems disrupted. Behavioral
neuroscience, 102(3):356–62. ISSN 0735-7044.
Nadel, L. (1991). The hippocampus and space revisited. Hippocampus,
1(3):221–9. ISSN 1050-9631. doi: 10.1002/hipo.450010302.
BIBLIOGRAPHY
159
Nadel, L. & Moscovitch, M. (1997). Memory consolidation, retrograde
amnesia and the hippocampal complex. Current opinion in neurobiology, 7(2):217–27. ISSN 0959-4388.
Nafstad, P. H. J. (1967). An electron microscope study on the termination
of the perforant path fibres in the hippocampus and the fascia dentata.
Zeitschrift für Zellforschung und Mikroskopische Anatomie, 76(4):532–
542. ISSN 0302-766X. doi: 10.1007/BF00339754.
Navratilova, Z., Giocomo, L. M., Fellous, J.-M., Hasselmo, M. E., &
McNaughton, B. L. (2011). Phase precession and variable spatial scaling in a periodic attractor map model of medial entorhinal grid cells
with realistic after-spike dynamics. Hippocampus, 22(4):772–89. ISSN
1098-1063. doi: 10.1002/hipo.20939.
Navratilova, Z., Hoang, L. T., Schwindel, C. D., Tatsuno, M., & McNaughton, B. L. (2012). Experience-dependent firing rate remapping
generates directional selectivity in hippocampal place cells. Frontiers
in neural circuits, 6:6. ISSN 1662-5110. doi: 10.3389/fncir.2012.00006.
O’Keefe, J., Burgess, N., Donnett, J. G., Jeffery, K. J., & Maguire, E. A.
(1998). Place cells, navigational accuracy, and the human hippocampus. Philosophical transactions of the Royal Society of London. Series B, Biological sciences, 353(1373):1333–40. ISSN 0962-8436. doi:
10.1098/rstb.1998.0287.
O’Keefe, J. & Conway, D. (1978). Hippocampal place units in the freely
moving rat: Why they fire where they fire. Experimental Brain Research, 31(4):573–590. ISSN 0014-4819. doi: 10.1007/BF00239813.
O’Keefe, J. & Dostrovsky, J. (1971). The hippocampus as a spatial map.
Preliminary evidence from unit activity in the freely-moving rat. Brain
research, 34(1):171–5. ISSN 0006-8993.
O’Keefe, J. & Nadel, L. (1978). The Hippocampus as a Cognitive Map.
Oxford University Press.
160
BIBLIOGRAPHY
O’Keefe, J. & Recce, M. L. (1993). Phase relationship between hippocampal place units and the EEG theta rhythm. Hippocampus, 3(3):317–30.
ISSN 1050-9631. doi: 10.1002/hipo.450030307.
Pennartz, C. M. A., Ito, R., Verschure, P. F. M. J., Battaglia, F. P.,
& Robbins, T. W. (2011). The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends in neurosciences,
34(10):548–59. ISSN 1878-108X. doi: 10.1016/j.tins.2011.08.001.
Pfeifer, R. & Bongard, J. (2006). How the Body Shapes the Way We
Think: A New View of Intelligence (Bradford Books). A Bradford
Book. ISBN 0262162393.
Pfeifer, R. & Verschure, P. (1994). The challenge of autonomous agents:
Pitfalls and how to avoid them. Autonomous Agents, ps. 237 – 263.
Pitkänen, A., Pikkarainen, M., Nurminen, N., & Ylinen, A. (2000). Reciprocal connections between the amygdala and the hippocampal formation, perirhinal cortex, and postrhinal cortex in rat. A review. Annals of the New York Academy of Sciences, 911:369–91. ISSN 00778923.
Pöschel, B., Draguhn, A., & Heinemann, U. (2002).
Glutamate-
induced gamma oscillations in the dentate gyrus of rat hippocampal slices.
Brain Research, 938(1-2):22–28.
ISSN 00068993.
doi:
10.1016/S0006-8993(02)02477-0.
Prim, R. C. (1957). Shortest connection networks and some generalizations. Bell System Technical Journal, 36:1389–1401.
Quirk, G., Muller, R., & Kubie, J. (1990). The firing of hippocampal
place cells in the dark depends on the rat’s recent experience. J. Neurosci., 10(6):2008–2017.
BIBLIOGRAPHY
161
Quiroga, R. Q., Reddy, L., Kreiman, G., Koch, C., & Fried, I. (2005).
Invariant visual representation by single neurons in the human brain.
Nature, 435(7045):1102–7. ISSN 1476-4687. doi: 10.1038/nature03687.
Ramón y Cajal, S. (1909). Histologie du système nerveux de l’homme &
des vertébrés. Paris : Maloine.
Recce, M. & O’Keefe, J. (1989). The tetrode: a new technique for
multi-unit extracellular recording. Society for Neuroscience Abstracts,
15:1250.
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2010a).
The mechanism of rate remapping in the dentate gyrus. Neuron,
68(6):1051–8. ISSN 1097-4199. doi: 10.1016/j.neuron.2010.11.024.
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2010b). The
mechanism of rate remapping in the dentate gyrus. Dins Society for
Neuroscience Abstracts. San Diego, CA, USA.
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2012a). The
mechanism of attractor dynamics in the CA3. In Preparation.
Rennó-Costa, C., Lisman, J. E., & Verschure, P. F. M. J. (2012b). The
mechanism of attractor dynamics in the CA3. Dins Society for Neuroscience Abstracts. New Orleans, LA, USA.
Rennó-Costa, C., Luvizotto, A., Betella, A., Sanchez Fibla, M., & Verschure, P. F. M. J. (2012c). Internal drive regulation of sensorimotor
reflexes in the control of a catering assistant autonomous robot. Dins
Lecture Notes in Artificial Intelligence: Living Machines.
Rennó-Costa, C., Luvizotto, A., Marcos, E., Duff, A., Sanchez-Fibla,
M., & Verschure, P. F. M. J. (2011). Integrating neuroscience-based
models towards an autonomous biomimetic Synthetic Forager. Dins
2011 IEEE International Conference on Robotics and Biomimetics,
162
BIBLIOGRAPHY
ps. 210–215. IEEE, Phuket, Thailand. ISBN 978-1-4577-2138-0. doi:
10.1109/ROBIO.2011.6181287.
Rennó-Costa, C. & Verschure, P. F. M. J. (2012). Nonspatial selectivity of
place cells supports quasi-optimal behavior in mixed spatial/nonspatial
tasks. In Preparation.
Rivard, B., Li, Y., Lenck-Santini, P.-P., Poucet, B., & Muller, R. U.
(2004). Representation of objects in space by two classes of hippocampal pyramidal cells. The Journal of general physiology, 124(1):9–25.
ISSN 0022-1295. doi: 10.1085/jgp.200409015.
Roberts, W. (1992). Foraging by rats on a radial maze:Learning, memory, and decision rules. Dins I. Gormezano & E. Wasserman, editors,
Learning and memory: The behavioral and biological substrates, ps.
7–24. Erlbaum, Hillsdale, NJ. ISBN 0805808884.
Rolls, E. T. (2007). An attractor network in the hippocampus: theory
and neurophysiology. Learning & memory (Cold Spring Harbor, N.Y.),
14(11):714–31. ISSN 1549-5485. doi: 10.1101/lm.631207.
Rolls, E. T., Stringer, S. M., & Elliot, T. (2006). Entorhinal cortex
grid cells can map to hippocampal place cells by competitive learning.
Network (Bristol, England), 17(4):447–65. ISSN 0954-898X. doi: 10.
1080/09548980601064846.
Rolls, E. T., Xiang, J., & Franco, L. (2005). Object, space, and objectspace representations in the primate hippocampus. Journal of neurophysiology, 94(1):833–44. ISSN 0022-3077. doi: 10.1152/jn.01063.2004.
Samsonovich, A. & McNaughton, B. L. (1997). Path Integration and
Cognitive Mapping in a Continuous Attractor Neural Network Model.
J. Neurosci., 17(15):5900–5920.
Sanchez-Fibla, M., Bernardet, U., Wasserman, E., Pelc, T., Mintz, M.,
Jackson, J. C., Lansink, C., Pennartz, C., & Verschure, P. F. M. J.
BIBLIOGRAPHY
163
(2010). Allostatic Control for Robot Behavior Regulation: a Comparative Rodent-Robot Study. Advances in Complex Systems, 13(3):377.
ISSN 02195259. doi: 10.1142/S0219525910002621.
Sargolini, F., Fyhn, M., Hafting, T., McNaughton, B. L., Witter, M. P.,
Moser, M.-B., & Moser, E. I. (2006). Conjunctive representation of
position, direction, and velocity in entorhinal cortex. Science (New
York, N.Y.), 312(5774):758–62. ISSN 1095-9203. doi: 10.1126/science.
1125572.
Savelli, F. & Knierim, J. J. (2010). Hebbian analysis of the transformation of medial entorhinal grid-cell inputs to hippocampal place fields.
Journal of neurophysiology, 103(6):3167–3183. ISSN 1522-1598. doi:
10.1152/jn.00932.2009.
Schenk, F., Inglin, F., & Gyger, M. (1983). Activity and exploratory
behavior after lesions of the medial entorhinal cortex in the woodmouse
(Apodemus sylvaticus). Behavioral and neural biology, 37(1):89–107.
ISSN 0163-1047.
Schweighofer, N., Arbib, M. A., & Dominey, P. F. (1996). A model
of the cerebellum in adaptive control of saccadic gain. I. The model
and its biological substrate. Biological cybernetics, 75(1):19–28. ISSN
0340-1200.
Scoville, W. B. & Milner, B. (1957). Loss of recent memory after bilateral hippocampal lesions. Journal of neurology, neurosurgery, and
psychiatry, 20(1):11–21. ISSN 0022-3050.
Sharp, P. E. (2006). Subicular place cells generate the same "map" for different environments: comparison with hippocampal cells. Behavioural
brain research, 174(2):206–14. ISSN 0166-4328. doi: 10.1016/j.bbr.
2006.05.034.
164
BIBLIOGRAPHY
Si, B. & Treves, A. (2009). The role of competitive learning in the
generation of DG fields from EC inputs. Cognitive neurodynamics,
3(2):177–87. ISSN 1871-4080. doi: 10.1007/s11571-009-9079-z.
Simon, H. A. (1969). The architecture of complexity. Dins The Sciences
of the Artificial, ps. 192–229. MIT Press, Cambridge, MA.
Simons, D. J. & Levin, D. T. (1998). Failure to detect changes to people during a real-world interaction. Psychonomic Bulletin & Review,
5(4):644–649. ISSN 1069-9384. doi: 10.3758/BF03208840.
Skaggs, W. E., McNaughton, B. L., Gothard, K. M., & Markus, E. J.
(1993). An Information-Theoretic Approach to Deciphering the Hippocampal Code. Dins In Advances in Neural Information Processing
Systems 5, [NIPS Conference], 1990, ps. 1030–1037. Morgan Kaufmann. doi: 10.1.1.52.5231.
Skaggs, W. E., McNaughton, B. L., Wilson, M. A., & Barnes, C. A.
(1996). Theta phase precession in hippocampal neuronal populations
and the compression of temporal sequences. Hippocampus, 6(2):149–
172. ISSN 1050-9631. doi: 10.1002/(SICI)1098-1063(1996)6:2<149::
AID-HIPO6>3.0.CO;2-K.
Solstad, T., Boccara, C. N., Kropff, E., Moser, M.-B., & Moser, E. I.
(2008). Representation of geometric borders in the entorhinal cortex.
Science (New York, N.Y.), 322(5909):1865–8. ISSN 1095-9203. doi:
10.1126/science.1166466.
Solstad, T., Moser, E. I., & Einevoll, G. T. (2006). From grid cells
to place cells: a mathematical model. Hippocampus, 16(12):1026–31.
ISSN 1050-9631. doi: 10.1002/hipo.20244.
Soltesz, I. & Deschênes, M. (1993). Low- and high-frequency membrane
potential oscillations during theta activity in CA1 and CA3 pyramidal
neurons of the rat hippocampus under ketamine-xylazine anesthesia.
Journal of neurophysiology, 70(1):97–116. ISSN 0022-3077.
BIBLIOGRAPHY
165
Song, J.-H., Rafal, R., & McPeek, R. (2010). Neural substrates of target
selection for reaching movements in superior colliculus. Journal of
Vision, 10(7):1082–1082. ISSN 1534-7362. doi: 10.1167/10.7.1082.
Song, S. & Waldron, K. J. (1989). Machines that walk: The adaptive
suspension vehicle. MIT Press, Cambridge, MA.
Stephens, D. W., Brown, J. S., & Ydenberg, R. C. (2007). Foraging:
Behavior and Ecology. University of Chicago Press, Chicago.
Taube, J., Muller, R., & Ranck, J. (1990a). Head-direction cells recorded
from the postsubiculum in freely moving rats. I. Description and quantitative analysis. J. Neurosci., 10(2):420–435.
Taube, J. S., Muller, R. U., & Ranck, J. B. (1990b). Head-direction
cells recorded from the postsubiculum in freely moving rats. II. Effects
of environmental manipulations. The Journal of neuroscience : the
official journal of the Society for Neuroscience, 10(2):436–47. ISSN
0270-6474.
Tolman, E. C. (1948). Cognitive Maps in Rats and Men. The Psychological Review, 55(4):189–208.
Towers, S. K., LeBeau, F. E. N., Gloveli, T., Traub, R. D., Whittington,
M. A., & Buhl, E. H. (2002). Fast Network Oscillations in the Rat
Dentate Gyrus In Vitro. J Neurophysiol, 87(2):1165–1168.
Trommald, M. & Hulleberg, G. (1997). Dimensions and density of dendritic spines from rat dentate granule cells based on reconstructions
from serial electron micrographs. The Journal of comparative neurology, 377(1):15–28. ISSN 0021-9967.
Tsoar, A., Nathan, R., Bartan, Y., Vyssotski, A., Dell’Omo, G., &
Ulanovsky, N. (2011). Large-scale navigational map in a mammal.
Proceedings of the National Academy of Sciences of the United States
166
BIBLIOGRAPHY
of America, 108(37):E718–24. ISSN 1091-6490. doi: 10.1073/pnas.
1107365108.
Ulanovsky, N. & Moss, C. F. (2007). Hippocampal cellular and network activity in freely moving echolocating bats. Nature neuroscience,
10(2):224–33. ISSN 1097-6256. doi: 10.1038/nn1829.
Van Cauter, T., Camon, J., Alvernhe, A., Elduayen, C., Sargolini, F.,
& Save, E. (2012). Distinct Roles of Medial and Lateral Entorhinal
Cortex in Spatial Cognition. Cerebral cortex (New York, N.Y. : 1991),
ps. bhs033–. ISSN 1460-2199. doi: 10.1093/cercor/bhs033.
Van Cauter, T., Poucet, B., & Save, E. (2008). Unstable CA1 place cell
representation in rats with entorhinal cortex lesions. The European
journal of neuroscience, 27(8):1933–46. ISSN 1460-9568. doi: 10.1111/
j.1460-9568.2008.06158.x.
Vanderwolf, C. H. (1969). Hippocampal electrical activity and voluntary
movement in the rat. Electroencephalography and clinical neurophysiology, 26(4):407–18. ISSN 0013-4694.
Verschure, P. F. M. J. & Althaus, P. (2003). A real-world rational agent:
unifying old and new AI. Cognitive Science, 27(4):30. doi: 10.1016/
S0364-0213(03)00034-X.
Verschure, P. F. M. J., Kröse, B. J. A., & Pfeifer, R. (1992). Distributed Adaptive Control: The self-organization of structured behavior. Robotics and Autonomous Systems, 9(3):181–196. ISSN 09218890.
doi: 10.1016/0921-8890(92)90054-3.
Verschure, P. F. M. J., Voegtlin, T., & Douglas, R. J. (2003). Environmentally mediated synergy between perception and behaviour in
mobile robots. Nature, 425:620–624.
BIBLIOGRAPHY
167
Verschure, P. F. M. J., Wyss, R., & König, P. (2006). A Model of the
Ventral Visual System Based on Temporal Stability and Local Memory.
PLOS Biology, 4(5):1–8. doi: 10.1371/Citation.
Walton, M. E., Bannerman, D. M., & Rushworth, M. F. S. (2002). The
Role of Rat Medial Frontal Cortex in Effort-Based Decision Making.
J. Neurosci., 22(24):10996–11003.
Whitlock, J. R. & Derdikman, D. (2012). Head direction maps remain
stable despite grid map fragmentation. Frontiers in Neural Circuits,
6. ISSN 1662-5110. doi: 10.3389/fncir.2012.00009.
Wiener, S. I., Korshunov, V. A., Garcia, R., & Berthoz, A. (1995). Inertial, substratal and landmark cue control of hippocampal CA1 place
cell activity. The European journal of neuroscience, 7(11):2206–19.
ISSN 0953-816X.
Witter, M. P. & Moser, E. I. (2006). Spatial representation and the architecture of the entorhinal cortex. Trends in neurosciences, 29(12):671–8.
ISSN 0166-2236. doi: 10.1016/j.tins.2006.10.003.
Witter, M. P., Naber, P. A., van Haeften, T., Machielsen, W. C.,
Rombouts, S. A., Barkhof, F., Scheltens, P., & Lopes da Silva,
F. H. (2000a). Cortico-hippocampal communication by way of parallel parahippocampal-subicular pathways. Hippocampus, 10(4):398–
410.
ISSN 1050-9631.
doi:
10.1002/1098-1063(2000)10:4<398::
AID-HIPO6>3.0.CO;2-K.
Witter, M. P., Van Hoesen, G. W., & Amaral, D. G. (1989). Topographical organization of the entorhinal projection to the dentate gyrus of
the monkey. The Journal of neuroscience : the official journal of the
Society for Neuroscience, 9(1):216–28. ISSN 0270-6474.
Witter, M. P., Wouterlood, F. G., Naber, P. A., & Van Haeften,
T. (2000b).
Anatomical organization of the parahippocampal-
168
BIBLIOGRAPHY
hippocampal network. Annals of the New York Academy of Sciences,
911:1–24. ISSN 0077-8923.
Wood, E. R., Dudchenko, P. A., & Eichenbaum, H. (1999).
The
global record of memory in hippocampal neuronal activity. Nature,
397(6720):613–6. ISSN 0028-0836. doi: 10.1038/17605.
Wood, E. R., Dudchenko, P. A., Robitsek, R. J., & Eichenbaum, H.
(2000). Hippocampal Neurons Encode Information about Different
Types of Memory Episodes Occurring in the Same Location. Neuron,
27(3):623–633.
Wyss, R., Konig, P., & Verschure, P. F. M. J. (2003). Invariant representations of visual patterns in a temporal population code. Proceedings
of the National Academy of Sciences of the United States of America,
100(1):324–9. ISSN 0027-8424. doi: 10.1073/pnas.0136977100.
Zilli, E. A. (2012). Models of grid cell spatial firing published 2005–2011.
Frontiers in Neural Circuits, 6(16).