Download Normal-distribution

Basic statistical concepts and least-squares. Sat05_61.ppt, 2005-11-28.
1. Statistical concepts
2.
•
Distributions
Normal-distribution
2. Linearizing
3. Least-squares
•
•
The overdetermined problem
The underdetermined problem
1
Histogram.
Describes distribution of repeated observations
At different times or places !
Distribution of global 10 mean gravity anomalies
2
Statistic:
Distributions describes not only random
events !
We use statistiscal description for
”deterministic” quantities as well as on
random quantities.
Deterministic quantities may look as if they
have a normal distribution !
3
”Event”
Basic concept:
Measured distance, temperature, gravity, …
Mapping:
X :H R
Stocastisc variabel .
In mathematics: functional, H – function-space – maybe
Hilbertspace.
Gravity acceleration in point P: mapping of the space of all
possible gravity-potentials to the real axis.
4
Probability-density , f(x):
What is the probability P that the value is in a specific interval:
b
P(a  x  b) =  f(x) dx
a
5
Mean and variance, Estimation-operator E:
Mean - value :

x   x  f ( x)dx  E ( x)
-
Variance

   ( x  x )  f ( x)dx  E (( x  x ) )
2
x
2
2
-
n' th moment : E(( x  x ) )
n
6
Variance-covariance in space of several dimensions:
Mean value and variances:
 x     ij 
2
x
 ij 



(
x

x
)(
x

x
)
dx
dx
i
i
j
j
i
j

 
7
Correlation and covariance-propagation:
Correlation between two quantities: = 0: independent.
ij 
 ij
 ii jj
  1,1
Due to linearity
E (a  X  b  Y )  aE ( x)  bE ( y )
8
Mean-value and variance of vector
If X and A0 are vectors of dimension n and A is an n
x m matrix, then
Y  A0  A  X E (Y )  A0  A  E ( X )
Y  E (( y  E (Y ))  (Y  E (Y )) )  A   X  A
T
T
The inverse P = generally denoted the weightmatrix
9
Distribution of the sum of 2 numbers:
• Exampel Here n = 2 and m = 1. We regard the
sum of 2 observations:
0
  11

A0 = {0}, A = {1 , 1},  X = 

0
 22 

Y = X 1 + X 2 ,  YY =  11 +  22
• What is the variance, if we regard the difference
between two observations ?
10
Normal-distribution
1-dimensional quantity has a normal distribution if
1
-(x- E(X))2 /(  xx 2)
f(x)=
e
2   xx
• Vektor of simultaneously normal-distributed
quantities if
F( x1 ,..., xn ) =
1
-(X - E(X))T P (X - E(X))/2
(2 )  ( det  X )
n/2
1/2
e
• n-dimensional normal distribution.
11
Covarians-propagation in several dimensions:
X: n-dimensional, normally distributed,
D nxm matrix , then
Z=DZ
also normal distributed,
E(Z)=D E(X)
E(Z )   Z  D   X  D
2
T
12
Estimate of mean, variance etc
n
Mean : xˆ   xi / n
n 1
( xi  xˆ )
Standard - deviation :  x  
n 1
i 1
n
2
n
Covariance : cov( x, y )   ( xi  xˆ )( yi  yˆ ) / n
i 1
n  number of products
13
Covarians function
If the covariance COV(x,y) is a function of x,y then we
have a
Covarians-function
May be a function of
• Time-difference (stationary)
• Spherical Distance, ψ on the unit-sphere (isotrope)
14
Normally distributed data and resultat.
If data are normaly dsitributed, then the resultats are
also normaly distributed
If they are linearily related !
We must linearize –
TAYLOR-Development with only 0 and 1. order terms.
Advantage: we may interprete error-distributions.
15
Distributions in infinite-dimensional spaces
V( P) element in separable Hilbert-space:

i
V ( P)    GM CijVij ( P),
i 0 j   i
Vij orthogonal base - functions
Stochastic variable (example) :
X ij (V )  GMCij
Normal distributed with sum of variances finite !
16
Stochastic process.
What is the probability P for the event is located in a
specific interval
P(a  X1  b, c  X 2  d )
Exampel: What is the probability that gravity in
Buddinge lies in between -20 and 20 mgal and that
gravity in Rockefeller lies in the same interval
17
Stokastisc process in Hilbertspace
What is the mean value and variance of ”the Evaluationfunctional”,
EvP (T )  T ( P),
with E ( X i )  0, E ( X i2 )   i2 , E ( X i X j )  0
E ( EvP )  i 0 E ( X i )Vi ( P)  0

E ( Ev p  Evq )  i 0  V ( P)Vi (Q)

2
i i
Variance : E ( Ev )  i 0 
2
P

2
i
18
Covariance function of stationary time-series.
2i
2i
f(x)=  ( ai cos(
x)+ bi sin (
x)), E(( ai )2 ) = E(( bi )2 ) =  i2
N
N
i=0
N
Covariance-function depends only on |x-y|
N
COV(x, y) = E(f(x) f(y)) =   i2 ( cos(i
x)cos(i y) + sin(i x)sin(i y))=
i=0
N

2
i
cos(i(x - y))
i=0
Variances called ”Power-spectrum”.
19
Covariance function – gravity-potential.

i
GM
R
T ( P)  T ( ,  , r )   
 Cij   Vij ( ,  ),
r
i 2 j i r
i
  latitude,   longitude, R Earth' s mean radius,
Cij fully normalized coefficien ts.
• Suppose Xij normal-distributed with the same
variance for constant ”i”.
GM
2
E( (
 C ij ) ) =  ij2 =  i2 /(2i + 1)
R
20
Isotropic Covariance-function for Gravity potential
COV ( P, Q)  E (T ( P), T (Q)) 
i 1
R 
 /( 2i  1)  Vij ( ,  )Vij ( ' ,  ') 


i 2 j i
 rr ' 

2
i
2
i
i 1
R 
   Pi (cos ), Pi Legendre polynomial s

i 2
 rr ' 
 spherical distance.

2
2
i
21
Linearizering: why ?
We want to find best estimate (X) for m
quantities from n observations (L).
Data normal-distributed, implies result normaly
distributed, if there is a linear relationship.
If m > n there exist an optimal metode for
estimating X:
Metode of Least-Squares
22
Linearizing – Taylor-development.
If non-linear: L    ( X ) or
observatio ns  noise  Function of parameters .
Start-værdi (skøn) for X kaldes X1
L0  ( X 1 ), y  L  L0 , x  X  X 1
Taylor-development with 0 og 1. order terms after
changing the order

y + v = A x, A = {
}|X 1
X
23
Covariance-matrix for linearizered quantities
If measurements independently normal distributed with
varians-covariance
 ij
Then the resultatet y normal-dsitributed with variancecovarians:
y
 A    A
T
ij
24
Linearizing the distance-equation.
( X , X 0 )  ( X1  X 01) 2  ( X 2  X 02 ) 2  ( X 3  X 03 ) 2
Linearized based on coordinates
( X 11, X 12 , X 13 )

(X, X 0 ) = ( X 1 , X 0 ) + 
|X 1  ( X i - X 1i ) + led af 2 - orden.

X 1i
i=1

X 1i - X 0i
|X 1 =
 X 1i
( X 1 , X 0 )
3
25
On Matrix form:
If
dX i  ( X i  X 0i )
T
  ( X , X 0 )

 dX i    ( X , X 0 )   ( X 1 , X 0 ) 


X0

X
i


observed  computed, or A  x  y
3 equations with 3 un-knowns !
26
Numerical-example
If (X11, X12,X13) = ( 3496719 m, 743242 m, 5264456 m).
Satellite: (19882818.3, -4007732.6 , 17137390.1)
Computed distance: 20785633.8 m
Measured distance: 20785631.1 m
((3496719.0-19882818.3)dX1 + (743242.0-4007732.6)
dX2+(5264456 .0-17137390.1) dX3)/20785633.8 =
( 20785631.1 - 20785633.8) or:
-0.7883 dX1 -0.1571 dX2 + 1.7083 dX3 = -2.7
27
Linearizing in Physical Geodesy based on T=W-U
In function-spaces the Normal-potential may be
regarded as a 0-order term in a Taylordevelopment.We may differentiate in Metric space
(Frechet-derivative).
  T /  height - anomaly
dT 2
g  
 T (gravity anomaly)
dr r
28
Method of Least-Square. Over-determined problem.
More observations than parameters or quantities
which must be estimated:
Examples:
GPS-observations, where we stay at the same
place (static)
We want coordinates of one or more points.
Now we suppose that the unknowns are m
linearily independent quantities !
29
Least-squares = Adjustment.
• Observation-equations:
• We want a solution so that
y   A  x
v -1y vT = (y - A x) -1y (y - A x )T = minimum(x)
Differentiation:
d
(y - Ax) -y1 (y - Ax )T = 0
d xi
•
2  y i -y1  (Ai ) - 2  ( Ai )T  -y1  ( Ai )  xi = 0
T
-1
-1
A y A x = A y y
x = ( AT -y1 A )-1 AT -y1 y (Normallig ningerne)
30
Metod of Last-Squares. Variance-covariance.
x =
-1
-1
-1 T
y
( A  A ) A  y ( A  A ) A  ) =
T
-1
y
T
-1
y
T
-1
y
T
-1
(A  A)
T
-1
y
31
Metod of Least-Squares. Linear problem.
Gravity observations:
H, g=981600.15 +/-0.02 mgal
12.11+/-0.03
-22.7+/-0.03
G
10.52+/-0.03
I
32
Observations-equations..
 1 0 0   g   981600.15 

  G 

 -110   
12.11

  gH = 

10.52 
 0 - 1 1   

  gI  

- 22.70 
 1 0 - 1

-1
 0.02 0.00 0.00 0.00 

  10 0


 1 - 1 0 1 
2


 0.00 0.03 0.00 0.00
 - 110
-1



x =  0 1 - 1 0  



2

 0.00 0.00 0.03 0.00
0 - 1 1

 
 0 0 1 - 1 



2
 1 0 - 1
 0.00 0.00 0.00 0.03 
2
33
Method of Least-Squares. Over-determined problem.
Compute the varianc-covariance-matrix
34
Method of Least-Squares.
Optimal if observations are normaly distributed +
Linear relationship !
Works anyway if they are not normally distributed !
And the linear relationship may be improved using
iteration.
Last resultat used as a new Taylor-point.
Exampel: A GPS receiver at start.
35
Metode of Least-Squares. Under-determined problem.
We have fewer observations than parameters: gravityfield, magnetic field, global temperature or pressure
distribution.
We chose a finite dimensional sub-space, dimension
equal to or smaller than number of observations.
Two possibilities (may be combined):
• We want ”smoothest solution” = minimum norm
• We want solution, which agree as best as possible with
data, considering the noise in the data
36
Method of Least-Squares. Under-determined problem.
Initially we look for finite-dimensional space so the
solution in a variable point Pi becomes a linearcombination of the observations yj:
~
x i =  ij  y j
n
j=1
If stocastisk process, we want the ”interpolation-error”
minimalized
n
E(x - ~
x )2 = E(x -  i  yi )2
i=1
= E( x ) - 2  i  E(x  yi ) +    i  j E( yi  y j ) = minimum(  i )
n
n
n
2
i=1
i=1 j=1
37
Method of Least-Squares. Under-determined problem.
2
=
E(

),
=
E(x

),
=
E(
y
y
y
Covariances: Cij
x )
i
j C Pi
i C0
Using differentiation:
Error-variance:
-1
~
x = ( C Pi )  ( Cij )  y
-1
E((x - ~
x )2 ) = E( x2 ) - 2 C Pi T  C ij  E(x  y)
+ C Pi  C ij  E( y 2 ) C ij  C Pj  =
-1
-1
T
T


 C Pi C ij  C Pj 
2
x
-1
med E(x)= 0, E(y) = 0, E(( yi )  ( y j )T ) = C ij , E(xy)= C Pj 
38
Method of Least-Squares. Gravity-prediction.
R
Example:
8 km
P
6 mgal
10 km
4 km
Q
10 mgal
Covarianses: COV(0 km)= 100 mgal2
COV(10 km)= 60 mgal2
COV(8 km)= 80 mgal2
COV(4 km)= 90 mgal2
39
Method of Least-Squares. Gravity prediction.
Continued:
-1
 100 60 
~

 g R = 90 80  
 60 100 
 10 
  = 9 mgal
 6
Compute the error-estimate for the anomaly in R.
40
Least-Squares Collocation.
Also called: optimal linear estimation
For gravity field: name has origin from solution of
differential-equations, where initial values are
maintained.
Functional-analytic version by Krarup (1969)
Kriging, where variogram is used closely connected to
collocation.
41
Least-Squares collocation.
We need covariances – but we only have one Earth.
Rotate Earth around gravity centre and we get
(conceptually) a new Earth.
Covariance-function supposed only to be dependent on
spherical distance and distance from centre.
For each distance-interval one finds pair of points, of
which the product of the associated observations is
formed and accumulated. The covariance is the mean
value of the product-sum.
42
Covarians-function for gravity anomalies, r=R.

i
R
Cij   Vij ( ,  )


r
i 2 j i
GM
g  2
r
C ( )  
Earth
i
 g ( ,  , R)g ( ' ,  ' , R)dd ,
azimuth
 fixed spherical distance.
43
Covariances function for gravity-anomalies:

C( ) =   i2  Pi ( cos ),
i= 2
2
j
GM


2
2
=

(i
1
) C ij , (tyngdeanomali gradvarianser)
i  2  
 R  j=-i
Different models for degree-variances (Power-spectrum):
Kaula, 1959, (but gravity get infinite variance)
Tscherning & Rapp, 1974 (variance finite).
44