Download writeup - Brandon Rayhaun

Computationally Modelling the Somatosensory System - Bensmaia Lab
1
Introduction
Brandon Rayhaun
to each component separately, though this has not been tested
experimentally for propagation speeds.
Perception begins at the peripheral nervous system, where sensory neurons translate stimuli from the outside world into discrete
electrical pulses that the brain can interpret. This electrical information is relayed to the central nervous system after undergoing
subsequent levels of processing. Part of the study of perception is
devoted to discovering how exactly a stimulus is stored internally
at different levels of processing. In other words, scientists seek
to unveil the neural code with which the brain stores information
about the world.
Here we concern ourselves with tacticle perception. In particular,
we wish to investigate roughness, one of the several dimensions of
texture perception. The goal of this study is an ambitious one–
we wish to predict the electrical activity of every neuron in the
hand and then use this information to model perceptual consequences.
2
Theory and Models
Research suggests that single neurons are not responsible for carrying all the information necessary for perceiving a feature of a
texture, such as roughness. The hand is densely populated with
different populations of mechanoreceptors, which respond primarily to deformations of the skin. These populations of neurons are
implicated as being information carriers.
Figure 1: The left plot shows a diharmonic stimulus experiencing successive attenuation as it is propagated to farther and farther distances
across the surface of the skin. A phase shift is introduced as the amount
of time required for the wave to reach farther points is larger. The plot
on the right shows the value of the power spectral densities at the 25
and 250Hz bins plotted against distance along with the lines of best
fit. These both follow a similar power to that of the amplitude. Note
how the differential attenuation between different frequency components
causes a morphing of the wave as it is propagated.
Moreover, it is observed that, even when stimulating only a small
region of the surface of the skin, afferents far away from the locus of
stimulation still have the tendency to fire. Thus it is important to
first characterize how the skin is responding spatially to different
modes of stimulation. In essence, the problem we are trying to Computational implementation of the propagation model is
solve is: how does a stimulus presented at position r deform the straightforward. The fourier transform of the input trace is taken
skin at position r + d?
and each frequency coefficient is attenuated by its appropriate attenuation factor, after which the inverse fourier transform is taken.
2.1 Wave Propagation in Skin
I.e. to produce our propagated trace X 0 from our input trace X,
we
used the following procedure:
Our model suggests that attenuation of waves as a function of
distance is itself a function of the frequency of the wave (γ = γ(f )
where |d|1 γ is the attenuation factor). Additionally, the waves’
X̃(f ) = F.T.{X(t)}
propagation speeds are implicated to be functions of frequency as
well (v = v(f )). If a sinusoid of frequency f is presented to the
X̃ 0 (f ) = r−γ(f ) X̃(f )
skin
X 0 (t) = I.F.T.{X̃ 0 (f )}
X(t) = X0 sin(2πf t),
then, at a distance r away, the wave will be given by
Fast fourier transforms are relatively quick operations, but implementing a frequency dependent phase shifting of the wave would
X (r, t) =
X(t + φ)
rγ(f )
require a custom inverse fourier transform method. A combination
where φ is a phase acquired because it takes a non-zero amount of of the fact that it is not know whether or not propagation speed
extends to arbitrary superpositions of sinusoids, the fact that all
time for the waves to travel. The phase is given by
current propagation speed data possesses high uncertainties for
r
lower frequencies, and the fact that effects of propagation speeds
φ=
2πf v(f )
on spike trains to begin with are negligible, we defaulted to assuming a uniform propagation speed for the entire wave, regardless of
To extend this model to an arbitrary temporal-spatial stimulus, its spectral properties. We then shifted the entire signal tempowe use the fact that periodic functions can be separated into in- rally by the amount of time it required to travel to the new point,
dividual frequency components (per Fourier) and apply the model and this was our source of phase. In symbols, our final trace X 00 (t)
0
1
1
Computationally Modelling the Somatosensory System - Bensmaia Lab
is given by
X 00 (t)
=
=
2.2
Brandon Rayhaun
respectively. C, Vrest , Θ∞ , and b are constants representing
membrane capacitance, resting membrane potential, steady state
threshold, and a threshold rebound time constant of the neuron.
The remaining constants, a and τ are free constants and represent
the threshold adaptation and membrane time constants.
r
v
r
r
0
X (t − ), t ≥
v
v
0, t <
Modeling the threshold voltage as a function of time allows us
to capture non-negligible adaptation effects. For example, many
afferents are known to decrease in sensitivity (increase their spiking
threshold) when exposed to prolonged stimulation.
Texture to Spikes
Once we have characterized the skin dynamics of the entire hand,
we wish to model the neural response. The generation of spike
trains takes place in two steps.
Membrane voltage and adaptive threshold obey these equations
until V (t) = Θ(t) for some t > 0. Immediately afterwards, a
2.2.1 Texture to Current
spike is recorded and the membrane voltage is reset to the cells
The first step is a model that converts the mechanical stimulation resting potential (V (t+ ) = Vrest ) and the adaptive threshold is
to electrical currents. The vibratory stimulus is provided to the either left alone or reset to Θ∞ , whichever is larger (Θ(t+ ) =
model as position of the surface of the skin as a function of time, max(Θ(t+ ), Θ∞ )).
X(t). This trace is then numerically derived twice to obtain velocity and acceleration, V (t) and A(t) (sometimes derived again 3
Methods and Work
to obtain jerk) and each of these 3 traces is then split up into its
positive and negative components,
3.1 Data
X + (t)
=
max(X(t), 0)
X − (t)
=
..
.
− min(X(t), 0)
In order to assess the predictive power of our model, we needed
experimentally collected data measuring the aspects of the somatosensory system that we were trying to replicate. Moreover,
we wished to push the model to domains it had not yet been used
in. We opted to assess its performance in texture perception. This
A+ (t) = max(A(t), 0)
is possible because the act of running our fingers across a texture
A− (t) = − min(A(t), 0).
elicits vibration, the spectral properties of which correspond (in a
manner related to the scanning speed) to the spatial periodicities
In practice, numerical differentiation introduces spurious high fre- of the texture itself.
quency components (typically above 300Hz), so these are removed
using a lowpass filter. These are then subsequently weighted by
free parameters in the model. The weighted components are then
summed to produce a current,
s(t) = ω1 X + (t) + · · · + ω6 A− (t).
Finally, we should expect that for large enough inputs the neurons
will saturate: we can model this with a filter, on the free parameter
I0 ,
I0 s(t)
I(t; I0 ) =
I0 + |s(t)|
to get our final output, I(t).
2.2.2
Current to Spikes
In order to transform the current obtained in the previous step
Figure 2: Plot taken from Sliman Bensmaia and Louise Manfredi. The
into a spike train, a generalized integrate and fire model is impleimages on the left depict pictures of the surfaces of selected textures
mented.
which are known to possess very regular spatial properties. The two
The model consists of two time dependent variables, the membrane plots on the righ of each texture display the power spectral densities of
voltage and the adaptive threshold voltage, whose equations are the vibrations elicited at 80mm/s and 120mm/s (in blue and orange)
along with the spatial power spectral density of the image itself (in
given by
dV
dt
dΘ
dt
black). We note that the peaks of the spatial PSD line up with the
temporal peaks of its corresponding vibration’s PSD and moreover that
the peaks shift as expected with scanning speed. We can conclude
that most of the relevant information about a texture is stored in the
vibrations it elicits in the skin.
1
I(t) + Iind (t)
= − [V (t) − Vrest ] +
τ
C
=
a[V (t) − Vrest ] − b[Θ(t) − Θ∞ ]
2
Computationally Modelling the Somatosensory System - Bensmaia Lab
Brandon Rayhaun
To gather peripheral data, primate subjects had a set of 55 tex- tion which takes a trace, Xt , lowpass cutoff frequency, l, highpass
tures scanned across their fingers using a rotating drum. During cutoff frequency, h, and a multiplicative constant a and returns a
stimulation, multiple RA, SA1, and PC afferents were recorded computed firing rate.
using electrodes.
First, given the computational intensity of the programs being
Human subjects had the same set of textures run across their used, and the fact that optimizing a model necessarily involves
fingers. Using a laser doppler vibrometer, the lateral vibrations iterating it potentially thousands of times, several procedures were
of the surface of the skin in response to each of these textures performed to produce efficient code. The code was first vigorously
were recorded. Additionally, the subjects were asked to assess how parallelized to make efficient use of idling cores in the processor.
rough each texture felt. This data was normalized with respect to On only a 3 core machine, runtime speed decreased by as much as
each subject’s mean rating for roughness.
5 times. The final computations were run on a custom cluster of
machines in the lab which possessed more than 17 cores.
Next, the model was reimplimented to make it compilable in C.
With the MEX library, this code was integrated with MATLAB,
and enjoyed another 10 times increase in speed.
With our computations now taking 2 hours instead of 2 weeks, we
first attemped to use lsqcurvefit to solve our optimization problem,
however, due to a combination of the complexity of the model, a
large presence of local minimums and an overall lack of smoothness
of the function, these attempts were largely unsuccessful.
So, in an attempt to ascertain the complicated shape of the function we were investigating, the sum squared differences between
experimental and predicted rates were plotted as a function of lowpass/highpass cutoff frequency and multiplier over a grid of values.
This produced the desired parameters for several of our data sets.
Creation of custom filters for the data were investigated, but were
Figure 3: Textures were scanned across the finger using a rotating drum.
Simultaneously, a laser doppler vibrometer was used to recorded the
velocity of the surface of the skin as a function of time.
The skin vibration data was used as input for the neural model,
and spike trains for different sample neurons were generated. The
model was reoptimized so that the sum squared difference between
the experimentally gathered firing rates and the predicted firing
rates was minimized. Other optimization schemes, such as optimizing with respect to timing (inter-spike interval distributions)
were explored as well. The next section outlines some of the measures taken in attempting to optimize the model.
3.2
Model Optimization and Computational Efficiency
Figure 4: Heatmaps visualizing the sum squared differences and correlations between measured and predicted RA firing rates. For each
bandpass, the minimum and maximum results respectively were taken
over all multiplicative constants used.
In order to test whether or not the model could produce accurate
neural responses to skin vibrations elicited by textures, it needed
to be optimized. The traces collected were unfiltered and moreover were not calibrated multiplicatively. Lowpass/highpass filter
cutoff frequencies and a multiplicative constant were used as free
parameters to optimize the model. I.e., we sought to solve
arg min
l,h,a
ultimately found not to have affected results too much. They also
required considerable amounts of time to compute, and so all attempts to make progress down that path were abandoned. We
settled on using a built-in butterworth filter.
X
(M (Xt , l, h, a) − Rt )2
3.3
t∈T
where T is our set of textures, Rt are the experimentally collected
firing rates in response to texture t, and M is our model func-
Results
We ran the the vibrometry traces for 3 subjects, 10 trials, and 55
textures through the model and compared the rates of the pre3
Computationally Modelling the Somatosensory System - Bensmaia Lab
Brandon Rayhaun
dicted spikes to the rate of the spikes measured in the periphery
of monkeys.
(a) Neuron locations on hand
(a) RA rates
(b) SA rates
(b) Raster plot of population response
Figure 6: Sample schematic of hand tiled with receptors. In a computation of the electrical activity of the hand, each cross would have a spike
train associated with it, which is depicted in the population raster plot
on the right. Each row is the spike train of a different RA neuron in the
hand in response to satin being scanned across the finger tip at 80mm/s.
The plot is arranged from bottom to top by distance from the locus of
stimulation. The vibration trace of the texture is displayed below the
raster plot. We notice, as expected, that neurons farther away from the
locus of stimulation fire at a lower rate.
(c) SA rates
Figure 5: Plots displaying measured firing rates versus predicted firing
rates against a line of unity.
The correlations between firing rates are uniformly high, and our
data adheres reasonably well to the unity line, indicating that the
model, at least with respect to firing rates, predicts spike trains
accurately.
The objective then became to extend the neural model to the
entire hand. This was accomplished by tiling a virtual hand with Firing rates are a reasonable place to begin in terms of predicting
receptors, propagating the stimulus to each of these points, and roughness from afferent response. However, other neural encodings
have been implicated to correlate linearly with roughness with
then computing the neural response at each location.
much better predictions.
First, functions were created to simulate the responses of afferents
over a finitely large receptor sheet. In this case, a grid was inter- One such code is a temporal variation code. This can be modeled
polated over a square with the specified innervation density and by performing a rectified convolution between the histogram of a
the responses of afferents at each point were computed using the spike train and a Gabor filter. Our Gabor filter is given by
single neuron model in response to a propagated wave.
2πt
−t2
g(t) = sin(
+ φ) exp{ 2 }
To make this a little more realistic, we instead implemented a
λ
2σ
version of the model where grids of afferents were interpolated over
where λ, φ, and σ are adjustable parameters. Then our temporal
parts of a hand and finger instead of a square. Moreover, these
variation can be computed as
grids had different innervation densities in the tip of the finger
versus the length of the finger versus the palm (Figure 6.)
ν = max{0, (H ◦ s)(t) ? g(t)}
4
4.1
Extension to Roughness
where ν is our temporal variation, s is our spike train, H is a
function which divides our spikes into bins, and g is our filter
(note that this must be a discrete convolution).
Methods
Now that we possessed simulations of the electrical activity of the
entire hand, we wished to examine how well we could predict sensation. We opted to examine roughness, one of the several dimensions of texture perception.
The idea here is that if a spike train is varying wildly with respect
to time (for instance, pulsing/bursting or experiencing oscillations
between high and low firing rates) then the temporal variation will
be a larger number. Note that this model offers predictions that
Roughness rating is a psychophysical measure. In general, there do not converge with those of a simple rate encoding model. In
are many different ways to assess roughness. Our participants particular, a constant steady stream of spikes, no matter how high
were asked to assign a number value to how rough a texture felt the rate, will have a temporal variation of near 0.
without a scale necessarily selected apriori. These results were We will have different spike trains for different afferents (RAs,
then normalized by the mean rating and pooled together. In other SA1s, and PCs) so we can model roughness rating as
words, if T is the set of textures presented to a subject and Rt
R = α0 + α1 νra + α2 νsa1 + α3 νpc
is his or her rating of the roughness of texture t ∈ T , then the
normalized ratings are given by
where αi , i = 0, . . . , 3 are adjustable parameters (note also that the
Rt
1 X
temporal variation of RAs, SAs, and PCs will in general not use
R̃t =
, R̄ =
Rt
N
R̄
the same Gabor filter). Moreover, since we have the spike trains
t∈T
4
Computationally Modelling the Somatosensory System - Bensmaia Lab
6
of every single RA, SA1, and PC in the hand, we can sum each
neuron’s contribution to get
R = α0 + α1
X
n∈RAs
4.2
νn + α2
X
n∈SAs
νn + α3
X
Brandon Rayhaun
Visualization
I’ve worked closely with Ezra Zigmond on completing a GUI application for visualizing how the entire hand responds when presented
with a texture. This should be available online in months to come.
νn
n∈P Cs
Results
Using these methods, we should in theory be able to predict perceived roughness very well simply using the position traces of the
textures, computing the electrical responses of the populations of
neurons in the hand, and then performing temporal variation computations on each of these spike trains.
Figure 8: Beta version of GUI. Artificially low innervation densities were
used in this frame for aesthetic purposes. This image also demonstrates
how populations were modeled, where triangles represent the location
of an afferent.
Figure 7: This plot depicts roughness rating against regressed population temporal variation responses. It enjoys a correlation of .91, which
is close to the correlation achieved when using recorded peripheral responses to predict roughness.
7
Acknowledgements
We do relatively well, especially given the fact that roughness to
I’d like to thank Sliman Bensmaia for mentoring me and allowing
begin with is a psychophysical measure. These results have imporme to work in his lab. Additionally, I’d like to thank Hannes Saal
tant consequences for the future direction of our research.
and Justin Lieber for working closely with me on my project and
offering their experience and insight throughout my stay. Finally,
5 Conclusion and Future Direction
without Barry Aprison and Ravi Kumar, I would not have been
We have demonstrated, with highly correlated computed versus able to have this experience at all.
measured firing rates and highly correlated computed versus experimentally gathered roughness ratings, that our model scheme
possesses strong predictive power. However, we have only begun
to flex the model’s muscles. There is still work to be done.
Several models and results were taken and/or expanded upon from
the following papers:
A simple model of mechanotransduction in primate glabrous skin
- Dong et. al.
An example of an intended application of the model is ascertaining
The effect of surface wave propagation on neural responses to vihow a signal in the periphery is processed and transformed before
bration in primate glabrous skin - Manfredi et. al.
reaching the cortex. In particular, this would ordinarily require
recording from both cortex and periphery, which can be a daunting task. Our lab possesses data recorded from the cortex along
with information about the stimuli that invoked the responses in
the cortex. We can use the stimuli information to computationally
produce the peripheral response, and then use the generated peripheral data in conjunction with data from the cortex to conduct
research on processing in the somatosensory system.
Another intended application of the model lies in upper limb
neuroprosthetics. With researchers at other universities creating
robotic appendages equipped with the ability to stimulate the peripheral nervous system, we intend to use our model to investigate
how reliably it can replicate tactile perception.
5