Download A6523 Signal Modeling, Statistical Inference and Data Mining in

2/3/15 A6523
Signal Modeling, Statistical Inference and
Data Mining in Astrophysics
Spring 2015
http://www.astro.cornell.edu/~cordes/A6523
Lecture 4
See web page later tomorrow
Searching for Monochromatic Signals in Noise
•  We derived the spectrum of a time series containing a
complex exponential and additive noise
•  The shape of the spectral line is a ‘sinc’ function.
–  For continuous time and frequency this is
sinc(x) = sin(πx)/πx
–  For the discrete case it is slightly different
•  The sinc function underlies many of the problems
associated with spectral analyis based on the Fourier
transform
•  The sinc function is the response of the Fourier
transform to a sinusoid. Any function or stochastic
process can be represented as a sum of sinusoids à
its power spectrum is convolved with the appropriate
sinc function.
1 sinc(x)
sinc(x)
2/3/15 1.2
1.0
0.8
0.6
0.4
0.2
0.0
−0.2
−0.4
−10
1.2
1.0
0.8
0.6
0.4
0.2
0.0
−0.2
−0.4
−10
main lobe sidelobes −5
0
5
10
Sinc func-on aligned so that its zeros fall on integer values of x. If we plot only the black dots, we get a Kronecker delta func-on Sinc func-on mis-­‐
aligned from integer x values. We no longer get a Kronecker delta −5
0
5
10
x
So what? Generally a monochroma-c signal will not be in integer mul-ple of the frequency resolu-on δf = 1 / T so power in |sinc|2 will leak into nearby (main lobe) and distant (sidelobe) frequencies. The envelope of sidelobe amplitudes ~ 1 / f2 2 2/3/15 Searching for Monochromatic Signals in Noise
•  We derived the spectrum of a time series
containing a complex exponential and
additive noise
•  In the no-signal limit:
•  The PDF of the spectrum is exponential (one sided)
•  The false-alarm probability is e-η for a threshold for
detection of η x spectral mean
•  The spectral mean = spectral rms for an exponential
PDF
•  If we find a spectral line that exceeds the
threshold, we would say that the line is real at
100e-η% confidence.
Relevant PDFs
•  Gaussian or Normal: N(μ, σ2)
fX (x) = √
argument 2
2
1
e−(x−µ) /2σ
2πσ
Random variable •  Exponential: fX (x) = �X�−1 e−x/�X� H(x)
N
�
•  Chi2:
2
xj
X=
with xj = = i.i.d GRV: N(0,1)
j=1
fX (x) =
1
x(N −2)/2 e−x/2
N/2
Γ(N/2)2
3 2/3/15 hSp://en.wikipedia.org/wiki/Normal_distribu-on hSp://en.wikipedia.org/wiki/Exponen-al_distribu-on hSp://en.wikipedia.org/wiki/Chi-­‐squared_distribu-on hSp://upload.wikimedia.org/wikipedia/commons/a/a9/Empirical_Rule.PNG 4 2/3/15 Properi-es of a Gaussian or Normal RV χ2 5 2/3/15 Detection Probability
•  The exponential PDF applies to the no-signal
case
•  But for the frequency bin in the spectrum that
has a signal the PDF is different:
–  What is the relevant PDF?
–  Need to consider the PDF of phasor + noise
–  From the PDF we can calculate the probability of
detection (true positive) and false negatives.
6 2/3/15 PDF of Phasor Magnitude
s = 0, 3, 5, 10
sigma_n = 1
PDF of Intensity
s = 0, 1, 3, 5
sigma_n = 1
Detec-on probability �
Pdet (Imin ) =
∞
dI fI (I)
Imin
7 2/3/15 ROC Curves
“Receiver operating characteristic”
“Relative operating characteristic”
•  In a so-called detection problem, we try to
establish whether a signal of some assumed
type is present in data that include “noise”
•  This is a universal problem that applies to many
laboratory and observational contexts.
•  In astronomy, ROC curves apply to finding
sources/signals in images, spectra, time series,
etc.
•  An ROC curve = Pd vs Pfa (detection vs falsealarm probability)
•  Binary classifier used in physics, biometrics,
machine learning, data mining, …
hSp://en.wikipedia.org/wiki/Receiver_opera-ng_characteris-c 8 2/3/15 Accuracy of Ground Hog Day Predictions
Meteorological accuracy (from wikipedia):
According to Groundhog Day organizers, the rodents' forecasts are
accurate 75% to 90% of the time. However, a Canadian study for 13
cities in the past 30 to 40 years found that the weather patterns
predicted on Groundhog Day were only 37% accurate over that time
period. According to the StormFax Weather Almanac and records
kept since 1887, Punxsutawney Phil's weather predictions have been
correct 39% of the time. The National Climatic Data Center has
described the forecasts as “on average, inaccurate” and stated that
the groundhog has shown no talent for predicting the arrival of
spring, especially in recent years.”
And what about the “superbowl” predictor for whether the stock market will be up or down? Etc. etc. 9 2/3/15 Estimation Error:
For any estimation procedure, we are interested in the estimation error, which we quantify with the
variance of the estimator:
Var{Sk } ≡ �Sk2� − �Sk �2.
This requires that we calculate the fourth moment of the DFT:
�|X̃k |4� = �|A δkk0 + Ñk |4�
(3)
= A4 δkk0 + A2 δkk0 �|Ñk + Ñk∗|2�
(4)
+ �|Ñk |4�
(5)
+ 2A2 δkk0 �|Ñk |2�
(6)
+ 2A3 δkk0 �(Nk + Nk∗)�
(7)
+ 2A δkk0 �|Ñk |2 (Ñk + Ñk∗)�.
The last two terms vanish because they involve odd order moments. The third term is �|Ñk |4� =
2 �|Ñk |2�2 because Ñk is complex Gaussian noise by the Central Limit Theorem.
Thus, the first and fourth terms and half of the third terms are just the square of �|X̃k |2�, so
�|X̃k |4� = �|X̃k |2�2 + �|Ñk |2�2 + 2A2 δkk0 �|Ñk |2�
8
or
Var{|X̃k |2} = �|X̃k |4� − �|X̃k |2�2
(8)
= �|Ñk |2�2 + 2A2 δkk0 �|Ñk |2�
= �|Ñk |2�2
= (σn2 /N )2
The fractional error in the spectrum is thus
�k ≡
�
1+
�
1+
2A2 δkk0
�|Ñk |2�
�
2A2 N δkk0
σn2
(9)
(10)
�
[Var {|X̃k |2}]1/2 (1 + 2A2 N δkk0 /σn2 )1/2
=
.
1 + A2 N δkk0 /σn2
�|X̃k |2�
Thus, for frequency bins off the line (k �= k0) we have �k ≡ 1. On the line we have


1
A2 N/σn2 → 0






� 2 �2
(1 + 2A2 N/σn2 )1/2 
�k =
= 1 − 12 Aσ2N
A2 N/σn2 � 1
n

1 + A2 N/σn2





√

 2 σn
A2 N/σn2 � 1
N A
Thus, as the signal-to-noise A/σn gets very large, the error in the spectral estimate −→ 0, as expected.
9
10 2/3/15 Frequentist à Bayesian
•  The approach we have taken is classic
frequentism:
•  it appeals to the notion of repeated trials and frequency of
occurrence; also to an underlying ensemble.
•  Point estimates are given of e.g. the signal strength and the
frequency.
•  The Bayesian alternative: using the one
realization of data in hand, what is the PDF of
the frequency and amplitude?
•  The relevant PDF is the posterior PDF and it is derived from
the product of a prior PDF and a likelihood function.
•  Instead of a power spectrum one gets a PDF.
•  The PDF ends up depending on the periodogram.
Basic Probability Tools
Random variables, event space: ζ = event à X = random var.
PDF, CDF, characteristic function
Median, mode, mean
Conditional probabilities and PDFs
Bayes theorem
Comparing PDFs
Moments and moment tests
Sums of random variables and convolution theorem
Central Limit Theorem
Changes of variable
Functions of random variables
Sequences of random variables
Stochastic processes = sequences of random variables vs. t, f, etc.
Power spectrum, autocorrelation, autocovariance, and structure
functions
•  Bispectra
•  Random walks, shot noise, autoregressive, moving average,
Markov processes
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
11 2/3/15 I. Ensemble vs. Time Averages
•  Experimentally/observationally we are forced
to use sample averages of various types
•  Our goal is often, however, to learn about the
parent population or statistical ensemble from
which the data are conceptually drawn
•  In some circumstances time averages
converge to good estimates of ensemble
averages
•  In others, convergence can be very slow or
can fail (e.g. red-noise processes)
I(t, ζ)
12 2/3/15 I(t, ζ)
I(t, ζ)
As data span length T à ∞ time average à ensemble average
“Ergodic”
13 2/3/15 Types of Random Processes
Goodman Sta$s$cal Op$cs Example: the Universe
•  Measurements of the CMB and large-scale structure
are on a single realization
•  The goal of cosmology is to learn about the
(notional) ensemble of conditions that lead to what
we see
•  Quantitatively these are cast in questions like “what
was the primordial spectrum of density fluctuations?”
and that spectrum is usually parameterized as a
power law
•  Perhaps the multiverse = the ensemble
•  Are all universes the same (statistically)?
•  Do measurements on our universe typify all
universes? (Conventional wisdom says no)
14 2/3/15 Basis functions:
spherical
harmonics
TCMB = 2.7 K
ΔT/TCMB ~ 10-5
Wilkinson
Microwave
Anisotropy
Probe
15 2/3/15 Nonstationary Case
I(t, ζ)
I(t, ζ)
16 2/3/15 I(t, ζ)
Random walk
in spin phase
I(t, ζ)
Random walk in
spin frequency
17 2/3/15 I(t, ζ)
Random walk
in spin
frequency
derivative
White
noise
RW1
RW2
RW3
18 2/3/15 19 2/3/15 Random Walk Examples
•  Spinning objects: Earth, neutron stars
•  Steps in torque or spin rate
•  Observable = spin phase
•  Scattered photon propagation (diffusion)
•  Step = mean-free path
•  Observable = propagation time
•  Cosmic-ray propagation in the Galaxy
•  Step = scattering off of small-scale magnetic field variations
•  Observable = `grammage’ of interaction based on isotopic
content (typically ~ 5 g cm-2)
•  Orbital perturbations
•  Asteroid belt objects à Near Earth Objects
•  Motions of planetesimals in protoplanetary disks
•  Galactic orbits of stars from gravitational potential granularity
(molecular clouds, spiral arms) à diffusion of stellar
populations
20 2/3/15 Other Random Walk Examples
• 
• 
• 
• 
• 
MCMC: random walk in parameter space
Brownian motion of a dust particle
Molecular diffusion
Diffusion of biological populations
Options pricing in financial markets
•  Step = transaction
•  Observable = price
•  Black-Scholes equation = Fokker-Planck equation
21