Download S13_Sensitivity - Livestock Genomics

A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
A Simple Method for Computationally
Inferring Microarray Sensitivity
Reverter & Dalrymple
BioInfoSummer 2003, AMSI, ANU, Canberra
“Best Talk”
A Rapid Method for Computationally Inferring
Transcriptome Coverage and Microarray Sensitivity
Reverter et al. 2005
Bioinformatics 21:80-89
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Motivation
Empirical Distribution of Tags
MPSS Paper, Jongeneel et al.
PNAS 03, 100:4702
tpm
>
1
5
10
50
100
500
1,000
5,000
10,000
N Tags
(0.0)
(0.7)
(1.0)
(1.7)
(2.0)
(2.7)
(3.0)
(3.7)
(4.0)
27,965
15,145
10,519
3,261
1,719
298
154
26
7
%
100.00
54.16
37.61
11.66
6.15
1.07
0.55
0.09
0.02
MPSS Test Data
No Tags = 25,503
cDNA Noise Paper
PNAS 02, 99:14031
S1
S2
 2x2 

f ( x)  exp  
 1 x 
100.00
57.14
36.11
10.89
5.73
1.21
0.57
0.15
0.05
100.00
49.87
33.66
10.74
5.67
1.13
0.55
0.11
0.05
100.00
56.19
36.79
11.76
6.95
1.94
1.11
0.29
0.16
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Motivation
Empirical Distribution of Tags
1.
Universal distribution associated
with stochastic processes of gene
expression (Kuznetsov, 2002)
2.
Framework for a mapping function:
Concentration  Signal
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Motivation
Mapping: Concentration  Signal
f ( x)  e
x
0.0
0.7
1.0
1.7
2.0
2.7
3.0
3.7
4.0
2 x2

1 x
%
100.00
56.19
36.79
11.76
6.95
1.94
1.11
0.29
0.16
Arrays 97
Signals 3,544,000
Mean 1,724
Intensity
%
>
100.0
56.4
36.6
12.1
6.7
0.9
0.4
0.2
0.1
1
280
560
2,800
5,600
28,000
40,000
55,000
65,000
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Definition of Sensitivity
References:
• Not from Confidence (1 – )
• Not from Formulae: Sn 
Kane et al. 2000
TP
1

TP  FN 1     
Lemon et al. 2003
Zien et al. 2003
• More like Minimum Detectable Concentration/Activity
Brown et al. 1996
“The smallest concentration of radioactivity in a sample that O’Malley & Deely, 2003
can be detected with a 5% Probability of erroneously
detecting radioactivity, when in fact none was present (Type I
Error) and also, a 5% Probability of not detecting
radioactivity when in fact it is present (Type II Error).”
• If  = , then Sensitivity = Confidence
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Inspiration
Economics 101
Quantity
Supply
Demand
Market
Equilibrium !
$ Price
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Process
…for a given microarray experiment:
1.
2.
3.
2 x2

1 x
From all the genes, find the intensity thresholds that define f ( x)  e
Apply these same threshold to the set of Differentially Expressed Genes.
The ratio of 2./1. Meets at the Equilibrium defining Sensitivity.
…example:
164,318 Records
6,051 Total Genes
183 Diff. Expressed Genes
x
0.0
0.7
1.0
1.7
2.0
2.7
3.0
3.7
4.0
Threshold
1
312
566
3,417
5,414
13,936
17,096
26,477
30,378
All Genes
100.00
54.16
37.61
11.66
6.15
1.07
0.55
0.09
0.02
DE
100.00
99.45
97.81
46.45
27.32
5.46
3.83
0.00
0.00
% DE
3.02
5.55
7.87
12.05
13.44
15.45
21.03
0.00
0.00
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Process
All Genes
6051 x 0.0615 = 372
183 x 0.2752 = 50
50/372 = 13.44%
Cat_1 (1)
Cat_2 (5)
Cat_3 (10)
Cat_4 (50)
Cat_5 (100)
Cat_6 (500)
Cat_7 (1000)
Cat_8 (5000)
Cat_9 (10000)
100.00
54.16
37.61
11.66
6.15
1.07
0.55
0.09
0.02
DE
% DE
100.00
99.45
97.81
46.45
27.32
5.46
3.83
0.00
0.00
3.02
5.55
7.87
12.05
13.44
15.45
21.03
0.00
0.00
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Inferential Validity
Let
NT = N of “Total” Genes
ND = N of “Differentially Expressed” Genes (ND  NT)

nt  xi 
f (x ) 
e
%
t
N D f ( xd )
NT f ( xt )
ND
NT
 f ( xd

nd  xi 
)
ND
ND

 f ( xd )  f ( xt )
NT
Flat line (except Upper Bound)
x
1.
2.
NT
2 x2

1 x
N D f ( xd )
 f ( xt )
NT f ( xt )

nd  xi 
 xi  f ( xt ) 
nt  xi 
The relevance of f(xi) is limited to the Concentration  Signal mapping.
At equilibrium the probability of an error either way equals.
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Mechanism
INPUT: (1) Gene ID – (2) Avg Intensity – (3) DE Flag
i=1
cat_nde(i) = nde
! For each category compute
cat_pde(i) = 100.0 * nde/ntot
! N and Prop of DE Genes
DO i = 2, 9
j = ntot - int(ntot*cat(i)/100.00) ! Pointer Location of threshold
m=0
! Counter for DE genes found so far
DO k = 1, ntot
IF( gene(k)%deflag > 0 )THEN
m=m+1
IF( gene(k)%intens > int(gene(j)%intens) )THEN
cat_nde(i) = nde-m+1
cat_pde(i) = 100.0*(cat_nde(i)/(ntot*(cat(i)/100.0)))
EXIT
ENDIF
ENDIF
ENDDO
WRITE(10,1000)i,cat(i),100.0*cat_nde(i)/nde,cat_pde(i)
ENDDO
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Application Examples (validation?)
…from CSIRO Livestock Industries:
ARRAYS
1.
2.
3.
4.
Wool Follicles
Beef Cattle Diets
Pigs Pneumonia
M Avium ss avium
10
14
16
13
GENES
Total
DE
6,051
6,816
6,456
132
183
450
307
47
…from Non-CSIRO Livestock Industries:
5.
6.
7.
Callow et al. (2000)
Lin et al. (2002)
Lynx MPSS test data
16
2
2
6,384
320
27,007 1,350
25,503 8,284
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Application Examples (validation?)
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Application Examples (validation?)
130 tpm …..I’ve seen them worse
80 tpm …..a ball-park figure
40 tpm …..possibly real
25 tpm …..possibly optimistic
5 tpm …..as Lynx claims
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Inferential Validity
 < 
Not many DE genes
High Confidence
Few False +ve
 = 
 > 
Lots of DE genes
High Power
Few False -ve
Armidale Animal Breeding Summer Course, UNE, Feb. 2006
A Quantitative Overview to Gene Expression Profiling in Animal Genetics
Sensitivity
Conclusions
1. We are looking at the Sensitivity of the Experiment,
not the Sensitivity of the Microarray Technology.
2. The proposed method is Very Simple and Very Fast.
3. Results acceptable but could be affected by:
a.
b.
c.
d.
e.
N Arrays in a given experiment
Quality of the Arrays themselves
Quality of the RNA extracted
Statistical approach to identify DE
Degree of Dissimilarity between samples
4. The impact of (3.a … 3.e) is not necessarily bad.
Armidale Animal Breeding Summer Course, UNE, Feb. 2006