* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Comparative Analysis Using DNA Microarrays: Sensitivity
Short interspersed nuclear elements (SINEs) wikipedia , lookup
Heritability of IQ wikipedia , lookup
Epigenetics in learning and memory wikipedia , lookup
Epigenetics of diabetes Type 2 wikipedia , lookup
Gene desert wikipedia , lookup
Therapeutic gene modulation wikipedia , lookup
X-inactivation wikipedia , lookup
Oncogenomics wikipedia , lookup
Site-specific recombinase technology wikipedia , lookup
Long non-coding RNA wikipedia , lookup
Pathogenomics wikipedia , lookup
Epigenetics of neurodegenerative diseases wikipedia , lookup
Public health genomics wikipedia , lookup
History of genetic engineering wikipedia , lookup
Polycomb Group Proteins and Cancer wikipedia , lookup
Nutriepigenomics wikipedia , lookup
Quantitative trait locus wikipedia , lookup
Essential gene wikipedia , lookup
Gene expression programming wikipedia , lookup
Microevolution wikipedia , lookup
Artificial gene synthesis wikipedia , lookup
Genome evolution wikipedia , lookup
Designer baby wikipedia , lookup
Genome (book) wikipedia , lookup
Minimal genome wikipedia , lookup
Epigenetics of human development wikipedia , lookup
Ridge (biology) wikipedia , lookup
Biology and consumer behaviour wikipedia , lookup
Analysis of DNA Microarray Data: Sensitivity, Specificity, and Other Real-World Issues 1. Definitions and basic considerations DNA microarrays Major advantage Simultaneous measurement of level of expression for nearly all transcribed genes within given cell or tissue Major disadvantage Cost Therefore, to get the most bang for the buck, it is imperative to understand the role of uncertainty in measurement… Categorical tests (yes/no, based upon threshold) Gene arrays Is gene expressed or not? Is gene differentially expressed under two different experimental conditions? Medical tests Does patient have disease or not? Key concepts for categorical tests Specificity true negative rate 1 – FPR (false positive rate) Sensitivity TPR (true positive rate) Specificity provides the answer to questions like… What fraction of patients who are disease-free are correctly classified as disease-free? What fraction of genes that are not differentially expressed are correctly classified as being non-differentially expressed? Specificity Specificity is defined as true negative rate Probability that disease-free patient will be correctly categorized as disease-free False positive rate (FPR) = 1 – specificity Probability that disease-free patient will be incorrectly categorized as having disease Sensitivity and specificity deal with distinct sets of patients or genes Specificity Healthy patients lacking the disease Non-expressed genes Non-differentially expressed genes Sensitivity Sick patients having the disease Expressed genes Differentially expressed genes Sensitivity provides the answer to questions like… What fraction of patients who have a given disease are correctly classified as diseased? What fraction of genes that are differentially expressed are correctly classified as being differentially expressed? Sensitivity Sensitivity is defined as true positive rate Probability that diseased patient will be correctly categorized as having the disease Yin and yang of sensitivity and specificity Improving specificity always worsens sensitivity Improving sensitivity always worsens specificity Since when is the world ever ideal? Frequency 0.4 Non-differentially expressed Differentially expressed 0.3 0.2 0.1 0 -2 0 2 SLR 4 6 If we choose a threshold l of 1.5, then... Frequency 0.4 Non-differentially expressed Differentially expressed 0.3 0.2 0.1 0 -2 0 2 SLR 4 6 And if we choose a threshold l of 0.5, then... Frequency 0.4 Non-differentially expressed Differentially expressed 0.3 0.2 0.1 0 -2 0 2 SLR 4 6 2. Sources of uncertainty in categorical tests SMEASURE STRUE N SMEASURE = measured signal STRUE = true signal N = noise (error) Noise-to-Signal (N:S) Ratio N : S << 1 reliable and trustworthy measurement N~S unreliable measurement N>S highly unreliable measurement Sources of uncertainty in categorical measurements Measurement uncertainty SMEASURE does not necessarily equal STRUE N ~ S or N > S “Overlap” uncertainty Some patients with disease truly have positive test values Some patients without disease truly have negative test values Gene arrays and medical tests have distinct and different sources of uncertainty Variability in medical tests is mostly “overlap” Measurement variability Essentially none (error is of no clinical significance) N : S << 1 Hence, perform test once and only once “Overlap” variability Ubiquitous and essentially unavoidable Feature of all medical tests to one degree or another So what’s the solution? Search for a better test Variability in DNA microarrays is mostly measurement uncertainty Measurement variability Ever-present N > S for many genes “Overlap” variability None Absent gene has expression level of zero, whereas present gene has expression level of non-zero Differentially expressed gene… So what’s the solution? Repeated measurements So how do we improve the N:S ratio? Take mean of repeated measurements... S ME ASUR E 1 = [ SME ASUR E(1) + ... + SME ASUR E(n) ] n 1 = [( STRU E + N1 ) + ... + ( STRU E + Nn )] n = STRU E + N Benefits of repeated measurements Assuming that noise N has a normal (Gaussian) distribution, then the error decreases with square root of number n of measurements Example: to reduce N : S by half, take mean of 4 measurements 3. Measurements using Affymetrix (MSV 5.0) Affymetrix Microsoft Suite Version 5.0 (MSV 5.0) Single array Two arrays (absolute) (comparative) Qualitative present vs. absent increased vs. decreased Quantitative log signal signal log ratio (SLR) For our analysis, we used... Single array Two arrays (absolute) (comparative) Qualitative Quantitative present vs. absent signal log ratio (SLR) Signal Log Ratio (SLR) SLR = logarithm to base 2 of the ratio of the signal for gene under experimental condition A (SA1) to that for the same gene under experimental condition N (SN1) SLRA 1N1 SA1 = log 2 = log 2 SA 1 – log 2 SN1 SN1 Examples of SLR SA1 = 4000 SN1 = 1000 SA1 = 2 SN1 = 16 SLR = log2 (4) = 2 SLR =log2 (1/8) = –3 4. Specificity of MSV 5.0 To get a handle on specificity, perform same-versus-same comparisons SLRTRUE must be zero log2 (1) = 0 Hence, SLRMEASURE is all noise Perform separate analyses for “present” and “absent” genes Present genes N : S << 1 Absent genes N : S ≥ 1 Experimental system Primary cultures of peritoneal macrophages from mice of 3 strains BALB/c (normal) MRL/+ (autoimmune lupus) MRL/lpr (autoimmune lupus) Each array represents mRNA pooled from distinct sets of ~ 6 mice harvested on separate days Macrophages were stimulated with bacterial endotoxin (lipopolysaccharide, LPS) for 8 or 24 hours Present genes: same-vs.-same comparison (single array) 0.5 0.4 Frequency Experimental BALB/c vs. BALB/c (n=5141) Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Present genes: same-vs.-same comparison (single array) 0.5 Frequency 0.4 MRL/+ vs. MRL/+ (n=5102) Experimental Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Present genes: same-vs.-same comparison (single array) 0.5 0.4 Frequency Experimental MRL/lpr vs. MRL/lpr (n=5222) Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Table 1. Experimental distributions for "present" versus "absent" genes in single array-based same-versus-same comparisons. "P RESENT GENES" No. genes Mean SLR N BALB/c vs. BALB/c 4022 –0.1 1.1 5133 0.0 0.3 4118 –0.1 1.1 5141 0.0 0.6 5140 0.0 0.9 5461 0.0 0.3 5040 0.0 1.0 5628 0.0 0.6 MRL/+ vs. MRL/+ MRL/lpr vs. MRL/lpr Mean ± SD 4512 5104 0.0 0.0 0.9 0.6 5224 5364 0.0 0.0 0.5 0.4 4990 ± 507 0.02 ± 0.04* † 0.69 ± 0.30 § Present genes: Same-vs.-same comparison (single array) Average SLR = ~ 0.02 + 0.04 (~ 1.014-fold) not different from zero that’s good! Standard deviation = ~ 0.69 + 0.30 ~ 32% genes have SLR > 0.69 (1.61-fold induction) ~ 4% genes have SLR > 1.38 (2.60-fold induction) that’s not good Present genes: Statistical distribution of SLR Entire distribution Not normal (p < 0.01, by D statistic) Central 95% Normal (p > 0.2, by D statistic) Highly noteworthy, since D statistic detects tiny tiny deviations from normality 5% at tails overestimate the SLR Present genes: same-vs.-same comparison (single array) 0.5 Frequency 0.4 MRL/+ vs. MRL/+ (n=5102) Experimental Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 If we compare genes in central 95% versus genes in 5% tails… Center (95% genes) Mean signal intensity = 1493 Tails (5% genes) Mean signal intensity = 620 (p < 10-19, t-test) • Consistent with intuitive idea that measurement variability is inversely related to level of gene’s expression Absent genes: same-vs.-same comparison (single array) 0.5 BALB/c vs. BALB/c (n=7347) Frequency 0.4 Experimental Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Absent genes: same-vs.-same comparison (single array) 0.5 MRL/+ vs. MRL/+ (n=7385) Frequency 0.4 Experimental Normal Dist 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Absent genes: same-vs.-same comparison (single array) 0.6 MRL/lpr vs. MRL/lpr (7264) 0.5 Experimental Normal Dist Frequency 0.4 0.3 0.2 0.1 0 -4 -3 -2 -1 0 SLR 1 2 3 4 Table 1. Experimental distributions for "present" versus "absent" genes in single array-based same-versus-same comparisons. "ABSENT GENES" No. genes Mean SLR N BALB/c vs. BALB/c 8466 +0.9 1.4 7355 0.0 0.8 8370 +0.9 1.3 7347 +0.1 1.1 7348 +0.5 1.4 7027 +0.0 0.8 7448 +0.4 1.3 6860 +0.1 1.1 MRL/+ vs. MRL/+ MRL/lpr vs. MRL/lpr Mean ± SD 7976 7384 +0.2 +0.1 1.3 1.2 7264 7124 +0.4 +0.4 0.9 0.8 7497 ± 507 0.33 ± 0.31* ‡ 1.12 ± 0.24 § Absent genes: Same-vs.-same comparison (single array) Average SLR = ~ 0.33 + 0.31 (~ 1.26-fold induction) definitely not good Standard deviation = ~ 1.12 + 0.24 > 35% genes have SLR > 1.0 (2-fold induction) > 5% genes have SLR > 2.0 (4-fold induction) even worse! Absent genes: Statistical distribution of SLR Entire distribution Not normal (p < 0.01, by D statistic) Central 95% Not normal (p < 0.01, by D statistic) Central 60% Not normal (p < 0.01, by D statistic) Summary of same-vs.-same comparisons (single array) Use SLR only for genes that are actually expressed (i.e., “present” genes) Central 95% normally distributed with standard deviation of ~ 0.69 2.5% at each tail exceeds normal distribution Do not use SLR for genes that are marginally, if at all, expressed (i.e., “absent” genes) Most of measured signal is noise SLR is therefore ratio of two small randomly distributed values Specificity of single array comparisons SLR Threshold (l) "Present" genes "Absent" genes l = 1.0 FPR = ~14% FPR = >35% l = 2.0 FPR = 0.4% FPR = >5% 5. Sensitivity of MSV 5.0 Experimental evaluation of sensitivity of gene arrays is not nearly so simple as specificity With same-vs.-same, we had a large set of equivalently expressed genes whose SLRTRUE was, by definition, equal to zero But what to do for differentially expressed genes? which genes are they? what is their SLRTRUE? Experimental approach to sensitivity Step #1: Compare gene expression by macrophages from BALB/c vs. MRL/lpr mice Step #2: Use strategy of recurrence (n > 2) on “present” genes to determine those that are differentially expressed at threshold of 0.8 (1.74-fold induction) strategy of recurrence is more specific 218 differentially expressed genes identified Step #3: Determine the mean measured SLR for each of these 218 genes Experimental approach to sensitivity Step #4: Group genes according to mean measured SLR in increments of 0.1 Step #5: Assess normality of distribution by D statistic for 3 largest gene groups Mean measured SLR = -1.5 (n = 11 genes) Mean measured SLR = -1.6 (n = 10 genes) Mean measured SLR = +1.2 (n = 10 genes • using D statistic, none of 3 differed from normality (p > 0.20) Experimental approach to sensitivity Step #6: Determine standard deviation of the distribution of measured SLR within each mean measured SLR group running average (e.g., standard deviation for mean measured SLR of 1.4 combines genes whose mean measured SLR was 1.3, 1.4, or 1.5) Mean variability ( N ) 2 Unsmoothed 1.5 Moving average SLRTRUE = 0 1 0.5 0 -3 -2 -1 0 Mean SLR(MEASURED) 1 2 3 Properties of differentially expressed genes Distribution is also normal (Gaussian) Standard deviation (SD) appears to depend on SLRTRUE SD increases roughly linearly with increasing SLRTRUE SD ~ 1.0 for SLRTRUE = 3.0 Sensitivity = TPR Sensitivity of single arraybased comparisons SLRTRUE = 1.0 1 SLRTRUE = 1.5 SLRTRUE = 2.0 0.5 SLRTRUE = 2.5 SLRTRUE = 3.0 0 0 1 2 Threshold ( l) 3 Sensitivity as a function of threshold l for single array comparisons l = 1.0 (2-fold l = 2.0 (4-fold induction) induction) SLRTRUE = 1.0 TPR = 50% TPR = <5% SLRTRUE = 3.0 TPR = ~98% TPR = >85% Specificity of single array comparisons SLR Threshold (l) "Present" genes "Absent" genes l = 1.0 FPR = ~14% FPR = >35% l = 2.0 FPR = 0.4% FPR = >5% 6. Strategies for combining multiple measurements Strategies for combining data from replicate array-based comparisons Strategy of means Mean SLR from n individual comparisons must exceed threshold l Standard deviation decreases as square root of number of replicates Strategy of recurrence SLR for all n comparisons must exceed threshold l Example Set threshold SLR at l = 0.9 4 replicate measurements: 0.7, 1.1, 1.6, 1.0 Strategy of means Mean SLR = 1.1 => include gene Strategy of recurrence 1 of the 4 SLR (0.7) does not exceed 0.9 => exclude gene Strategies of recurrence vs. means: Effect of replicates on specificity Strategy of recurrence is more specific than strategy of means (i.e., fewer false positives) Strategy of recurrence: benefit of replicates Keep multiplying FPR Example: FPR (n = 4) ~ [ FPR (n = 1) ]4 Strategy of means: benefit of replicates Standard deviation reduced by square route of n Example: SD for FPR (n = 4) ~ 1/2 SD for FPR (n = 1) False Positive Rates (FPR) using strategy of recurrence and means n=2 0 1 2 3 FPR = 1 - specificity SLR threshold ( l) 1 0.1 0.01 0.001 0.0001 1E-05 1E-06 1E-07 1E-08 n=4 0 1 2 SLR threshold ( l) 3 FPR = 1 – Specificity 1 0.1 0.01 0.001 0.0001 1E-05 1E-06 1E-07 1E-08 Means FPR = 1 – Specificity FPR = 1 - specificity Recurrence 1 0.1 0.01 0.001 0.0001 1E-05 1E-06 1E-07 1E-08 n=2 0 1 2 3 SLR threshold ( l) 1 0.1 0.01 0.001 0.0001 1E-05 1E-06 1E-07 1E-08 n=4 0 1 2 SLR threshold ( l) 3 FPR using strategy of recurrence or means (threshold l = 1.0) # Replicates Recurrence Means n=1 FPR = ~ 14.0% FPR = ~ 14.0% n=2 FPR = ~ 2.0% FPR = ~ 4.0% n=4 FPR = ~ 0.04% FPR = ~ 0.4% Sensitivity (TPR) using strategy of recurrence and means Means n =2 1 Sensitivity = TPR Sensitivity = TPR Recurrence 0.5 0 0 1 2 3 n=2 1 0.5 0 0 1 Sensitivity = TPR Sensitivity = TPR n=4 1 0.5 0 1 2 Threshold ( l) 3 Threshold ( l) Threshold ( l) 0 2 3 4 n=4 1 0.5 0 0 1 2 Threshold ( l) 3 Strategies of recurrence vs. means: Effect of replicates on sensitivity Strategy of recurrence is more specific than strategy of means BUT strategy of recurrence is also less sensitive than strategy of means The inevitable trade-off between sensitivity and specificity The benefit of multiple measurements… By increasing the number of replicates n, one can achieve the same level of specificity at a lower threshold l By using a lower threshold l, one can achieve a higher level of sensitivity 7. Receiver-operator characteristic (ROC) curves Receiver-operator characteristic (ROC) curves Recall that increasing the threshold l always increases specificity and decreases sensitivity ROC curves are means of depicting overall performance of a test as the threshold l is varied ROC for single array-based comparisons 1 TPR 0.75 0.5 SLRTRUE = 1.0 SLRTRUE = 1.5 0.25 SLRT RUE = 2.0 0 0 0.25 0.5 0.75 FPR = 1 – Specificity 1 In an ideal world…. 0.4 Frequency Non-differentially expressed 0.3 Differentially expressed 0.2 0.1 0 -2 0 2 4 SLR 6 8 10 ROC for single array-based comparisons 1 TPR 0.75 0.5 SLRTRUE = 1.0 SLRTRUE = 1.5 0.25 SLRT RUE = 2.0 0 0 0.25 0.5 0.75 FPR = 1 – Specificity 1 If we choose a threshold l of 1.5, then... Frequency 0.4 Non-differentially expressed Differentially expressed 0.3 0.2 0.1 0 -2 0 2 SLR 4 6 ROC for single array-based comparisons 1 TPR 0.75 0.5 SLRTRUE = 1.0 SLRTRUE = 1.5 0.25 SLRT RUE = 2.0 0 0 0.25 0.5 0.75 FPR = 1 – Specificity 1 So how do strategies of means and recurrence compare? For any given threshold l, the strategy of recurrence is more specific than strategy of means, but it is also unfortunately less sensitive As one increases the number n of replicates, strategy of recurrence gains more in specificity, but also loses more in sensitivity ROC curves show that strategies of recurrence and means may perform nearly equivalently n =2 1 TPR 0.75 0.5 Recurrence, SLRTRUE = 1.0 Mean, SLRTRUE = 1.0 Recurrence, SLRTRUE = 1.5 0.25 Mean, SLRTRUE = 1.5 0 0 0.25 0.5 0.75 FPR = 1 – Specificity 1 Strategy of means versus strategy of recurrence As assessed by ROC curves, these two strategies seem to perform more or less equivalently To achieve this equivalence in performance, one need only choose a lower threshold l for a strategy of recurrence Example: Gene whose SLRTRUE is 1.5 (~3-fold induction) with n = 2 replicates Sensitivity 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 0.95 0.99 n = 2 replicates Strategy of Recurrence Threshold ( l )* Specificity 1.55 >0.9999 1.45 >0.9999 1.35 >0.9999 1.30 >0.9999 1.20 >0.9999 1.10 >0.9999 1.00 >0.9999 0.85 0.9993 0.70 0.9968 0.45 0.9640 Strategy of Means Threshold ( l ) Specificity 1.75 >0.9999 1.65 >0.9999 1.60 >0.9999 1.50 >0.9999 1.40 >0.9999 1.35 >0.9999 1.25 >0.9999 1.10 0.9998 1.00 0.9996 0.80 0.9954 8. Strategy of recurrence: Getting the most bang for your buck The dark side of gene arrays... Gain in certainty by performing multiple measurements (i.e., experimental repeats) is counterbalanced by high cost of each experiment Therefore, imperative that you extract maximal information from each set of experiments The mathematical problem... For any given gene, we combine 2 independent measurements (i.e., data from 2 separate arrays) to obtain a single comparative index Thus, although we use 2n arrays and therefore perform 2n independent measurements, we obtain only n comparative measurements The mathematical problem… So, can one extract additional independent comparative measurements? How many of the n2 possible pair-wise comparisons are linearly independent? Using linear algebra, one can prove that the maximal number of linearly independent comparative measurements obtainable from n individual comparisons (2n arrays) is equal to 2n–1 Heuristic proof Number of linearly independent pair-wise comparisons cannot exceed number of independent measurements or variables therefore, number of independent pair-wise comparisons can not exceed 2n System has one degree of freedom i.e., our pair-wise comparisons tell us nothing about absolute level of expression once fix absolute level of expression for any one of 2n arrays, all others fall out therefore, 2n – 1 independent pair-wise comparisons Traditional approach n=2 N1 N2 Ñ1 Ñ2 {SLRN1Ñ1, SLRN2Ñ2 } 4 arrays, 2 SLR comparisons Maximal utilization of available resources N1 N2 Ñ1 Ñ2 {SLRN1Ñ1, SLRN2Ñ2, SLRN1Ñ2, SLRÑ1N2} 4 arrays, 4 possible SLR comparisons Which 2n-1 should we use? 2n -1 = 3 N1 N2 Ñ1 Ñ2 {SLRN1Ñ1, SLRN2Ñ2, SLRN1Ñ2, SLRÑ1N2} To avoid weighting the arrays unequally, we should use all 2n comparisons Application within strategy of recurrence (first approximation) Let p denote probability that SLRMEASURE for a given gene will exceed a threshold l on a single comparison Then the probability that SLRMEASURE will exceed l on all 2n-1 independent measurements will be ~ p2n–1 Experimental test for strategy of recurrence in same-vs.-same comparisons 500 B # false positive genes 400 n = 2, experimental 2 n -1 = 3, experimental 300 200 p 2 , predicted p 3 , predicted 100 0 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 Threshold SLR Nothing is gained within a strategy of means To see this, let us take the following example: SLRMEASURE (1) = log2 (4000/1000) = log2 (4) = 2 SLRMEASURE (2) = log2 (5000/2500) = log2 (2) = 1 » mean = 1.5 SLRMEASURE (3) = log2 (4000/2500) = log2 (1.6) = 0.68 SLRMEASURE (4) = log2 (5000/1000) = log2 (5) = 2.32 » mean = 1.5 Experimental lack of benefit within strategy of means 1000 A n = 2, strategy of means #false positive genes 750 2 n -1 = 3, strategy of means n = 2, strategy of recurrence 500 2 n -1 = 3, strategy of recurrence 250 0 0.5 0.6 0.7 0.8 0.9 1 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 T hreshold SLR The following mathematical features can be proven… Assuming you choose wisely the order of comparisons, then the plot of FPR is composed of 3 distinct segments initial n comparisons (i.e., n individual experiments) n – 1 additional linearly independent comparisons remaining (n – 1)2 non-independent comparisons The following mathematical features can be proven… 1st two segments are linear (i.e., probability that SLR will exceed threshold is constant for all comparisons within each of 1st two segments) probability of surviving any given comparison within each of 1st two segments is function solely of the chosen threshold and the underlying probability distribution of SLR probability of surviving any given comparison is greater for 2nd segment (i.e., comparisons in 2nd segment are less effective in eliminating FPR) Can prove following mathematical features 3rd segment not linear comparisons in this segment much less effective in eliminating FPR Heuristic proof of these features 1st segment each of 2n arrays used one and only one time n independent experiments, so if probability of surviving one experiment is p, then probability of surviving n experiments is pn 2nd segment distributions of SLR now constrained by fact that gene has survived one comparison (probability of survival q > p) choose additional n – 1 comparisons carefully, so each array is used a 2nd time only linear independence assures probabilistic independence of next n – 1 comparisons, qn - 1 3rd segment Because of linear dependence, every additional comparison further constrains the underlying SLR distributions (ripple effect) Some caveats for using additional 2n – 1 linearly independent comparisons within a strategy of recurrence Statistical algorithms of Affymetrix MSV 5.0 are only approximately linear close enough not to be a problem Implicitly assumed that across-experiment noise (different days) is small compared to within-experiment noise (biologic and technical variability) but we assume this all the time anyway Some caveats for using additional 2n – 1 linearly independent comparisons within a strategy of recurrence Must choose additional n – 1 comparisons wisely, so the complete set of 2n – 1 comparisons is linearly independent Each array should be used only twice One possibility (if comparing Normal N vs. Autoimmune A would be: N1 vs. A2, N2 vs. A3, …., Nn -1 vs. An Use of the remaining (n – 1)2 non-independent comparisons can reduce the FPR yet further Some caveats for using additional 2n – 1 linearly independent comparisons within a strategy of recurrence Best also to perform same-versus-same comparisons corresponding to additional independent comparisons for example, if comparing normal N3 from 3rd experiment against A1 from 1st then perform N1 vs. N3 and A1 vs. A3 do not use N3 vs. A1, if these same-vs.-same comparisons have disproportionately increased SD or SLRMEAN 0 Strategy of recurrence versus strategy of means: Final verdict For an equal number n of experimental replicates, as judged by ROC curves, the two strategies seem to perform fairly equivalently strategy of recurrence achieves this equivalence at a lower threshold l NOTE: we’ve shown this only for the case where the distribution of SLR is normal The advantage of a strategy of recurrence is that one obtains an additional n – 1 independent comparisons 9. The biological problem Murine Models of Lupus MRL/+, MRL/lpr NZW, NZB, NZB/W F1 BXSB LG -/gld, -/lpr IMMUNOLOGIC ABNORMALITIES IN AUTOIMMUNE DISEASE ? Manifestation of Disease ? Predisposing Factor SUMMARY OF IL -1 DATA Under expression of IL - 1 apparent by: 1. BIOSSAY: Secreted IL - 1 Intracellular IL - 1 Membrane-associated IL - 1 2. WESTERN: Total cellular IL - 1 Total cellular IL - 1 3. NORTHERN: IL - 1 mRNA IL - 1 mRNA NORMAL STRAINS A/J AKR/J A. Thy B. 10 B. 10A B. 10BR BALB/c C3HeB/FeJ C3H/HeN C57BL/6 CBA/J DBA/2J SWR Serum-induced changed in m cytokine mRNA Down regulated Up regulated Unchanged IL-1 IL-1 IL-6 IL-12, p35 IL-12, p40 GM-CSF MIP-1 RANTES TNF- IL-10 M-CSF MIF TGF-1 TGF-2 TGF-3 10. Application of arrays to biology The question… Are there other genes in autoimmune mice that show a similar dysregulation to that of IL-1 and other cytokines? Normally expressed in absence of serum (FBS) Abnormally expressed in presence of serum (FBS) Experimental system Primary cultures of peritoneal macrophages from mice of 3 strains BALB/c (normal) MRL/+ (autoimmune lupus) MRL/lpr (autoimmune lupus) Each array represents mRNA pooled from distinct sets of ~ 6 mice harvested on separate days Macrophages were stimulated with bacterial endotoxin (lipopolysaccharide, LPS) for 8 or 24 hours Table 1. Microarray comparisons. Strains comparisons* MRL/lpr vs. BALB/c +/– FBS Duration LPS stimulation Number + + – – 8h 24 h 8h 24 h 2 2 2 2 MRL/+ vs. BALB/c + + 8h 24 h 2 2 BALB/c vs. BALB/c + + 8h 24 h 2 2 MRL/+ vs. MRL/+ + + 8h 24 h 1 1 MRL/lpr vs. MRL/lpr + + 8h 24 h 1 1 of Detection of differentially expressed genes Threshold SLR of 0.8 (1.74-fold induction) within strategy of recurrence (n = 2) Specificity ~99.9% (FPR ~ 0.5-1.0 gene per 1000) Sensitivity depends on true SLR of gene, and ranges from 0.33 (SLR of 1.0, 2-fold induction) to >0.96 (SLR of 2.0, 4-fold induction) within Expt.: #1 across #2 Expt.: #1 #2 MRL/+ MRL/+ MRL/+ MRL/+ BALB/c BALB/c BALB/c BALB/c 4684 “present” genes “present” genes 3833 4777 intersection 4052 intersection 3456 “present” genes 3671 469 differentially expressed genes (n=2 comparisons) 661 intersection 217 217 differentially expressed genes (n=3 linearly independent comparisons) Table 2. Summary of the results of microarray-based comparison of gene expression. "Within" comparisons* Comparison Present genes † MRL/+ vs. BALB/c R.10, 8 h MRL/+ vs. BALB/c R.10, 24 h MRL/lpr vs. BALB/c R.10, 8 h MRL/lpr vs. BALB/c R.10, 24 h MRL/lpr vs. BALB/c R.0, 8 h MRL/lpr vs. BALB/c R.0, 24 h BALB/c vs. BALB/c R.10, 8 h BALB/c vs. BALB/c R.10, 24 h MRL vs. MRL, R.10, 8 h MRl vs. MRL, R.10, 24 h 4684 3833 5217 4984 5063 5413 5246 5709 5257 5523 5289 5830 4022 5133 5140 5461 4512 5224 5104 5364 "Across" comparisons* Intersection‡ Differential expression § 3456 469 4444 244 4831 208 5064 225 5006 193 5138 162 3813 62 4686 49 4087 136 4626 51 Present genes † 4777 4052 4820 5637 5199 5424 5640 5428 5416 5409 5702 5534 4118 5141 5040 5628 4782 4991 5298 5332 Intersection of "within" & "across" differential expression ¶ Intersection‡ Differential expression § 3671 661 217 4563 366 109 4936 189 131 5161 199 151 5031 202 149 5231 210 121 3935 259 11 4757 293 9 4226 494 36 4778 310 8 Union of 8 & 24 h differential expression 270 223 201 18 42 * "Within" comparisons refer to comparisons between RNA samples obtained from m cultured on the same day, whereas "across" comparisons refer to comparisons between RNA samples obtained from m cultured on different days as parts of replicate experiments. † As determined by Affymetrix MSV 5.0. ‡ Refers to those genes which were present in both replicate experiments. § Refers to those genes whose SLR • 0.8 on both comparisons or whose SLR Š 0.8 on both comparisons. ¶ Refers to those genes that fulfilled the criteria for differential expression in both the "within" and the "across" comparisons. Refers to those genes that were differentially expressed at 8 h and/or 24 h. Table 3. Mean SLR magnitude for differentially expressed genes. Differentially expressed gene set Comparison MRL/+ vs. BALB/c R.10, 8 h MRL/+ vs. BALB/c R.10, 24 h MRL/lpr vs. BALB/c R.10, 8 h MRL/lpr vs. BALB/c R.10, 24 h Mean SLR magnitude for each differentially expressed gene set in same-vs-same comparisons BALB/c vs. BALB/c, BALB/c vs. BALB/c, MRL vs. MRL MRL vs. MRL 8h 24 h 8h 24 h No. genes Mean SLR magnitude 217 2.1± 1.2 0.3 ± 0.3 0.4 ± 0.3 0.5 ± 0.6 0.3 ± 0.2 109 2.0 ± 1.3 0.4 ± 0.4 0.4 ± 0.4 0.6 ± 0.6 0.3 ± 0.2 131 1.9 ± 1.3 0.4 ± 0.3 0.4 ± 0.4 0.6 ± 0.6 0.3 ± 0.3 151 1.7 ± 1.1 0.5 ± 0.4 0.5 ± 0.4 0.7 ± 0.7 0.3 ± 0.3 Expected FPR We expect FPR of 0.5-1.0 per 1000 genes ~ 5000 genes were expressed or ‘present’ therefore, 2.5-5 genes per comparison are FPR (out of 100-200 genes) Same-vs.-same comparisons gave FPR about twice expected Summary of differentially expressed genes Differentially expressed only in presence of FBS 280 genes Differentially expressed only in absence of FBS 80 genes Differentially expressed both in presence and in absence of FBS 112 genes 3 FBS only SLR (FBS) / SLR (FBS-free) FBS & FBS-free FBS-free only 2 1 0 8h 24 h Duration of LPS Stimulation Global expression patterns of genes differentially expressed only in presence of FBS 5 M RL/lpr vs. BALB/c + FBS 4 3 SLR, 24 h 2 1 0 -1 -2 -3 -4 -5 mean distance = 1.6 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 5 5 M RL/lpr vs. BALB/c FBS-fre e 4 3 SLR, 24 h 2 1 0 -1 -2 -3 -4 -5 mean distance = 0.8 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 5 5 BALB/c vs. BALB/c + FBS 4 3 SLR, 24 h 2 1 0 -1 -2 -3 -4 -5 mean distance = 0.7 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 5 5 M RL/lpr vs. M RL/lpr + FBS 4 3 SLR, 24 h 2 1 0 -1 -2 -3 -4 -5 mean distance = 0.8 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 5 5 5 MRL/lpr vs. BALB/c + FBS 3 3 2 2 1 1 0 -1 0 -1 -2 -2 -3 -3 -4 -4 -5 MRL/lpr vs. BALB/c FBS-free 4 SLR, 24 h SLR, 24 h 4 mean distance = 1.6 -5 -4 -3 -2 -1 0 1 2 3 4 -5 5 mean distance = 0.8 -5 -4 -3 -2 SLR, 8 h 1 2 3 4 5 5 BALB/c vs. BALB/c + FBS 4 3 3 2 2 1 1 0 -1 0 -1 -2 -2 -3 -3 -4 -4 mean distance = 0.7 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 MRL/lpr vs. MRL/lpr + FBS 4 SLR, 24 h SLR, 24 h 0 SLR, 8 h 5 -5 -1 5 -5 mean distance = 0.8 -5 -4 -3 -2 -1 0 SLR, 8 h 1 2 3 4 5 Global expression pattern of genes differentially expressed in both presence and absence of FBS 8 10 MRL/lpr vs. BALB/c + FBS 8 MRL/lpr vs. BALB/c FBS-free 6 6 4 SLR, 24 h SLR, 24 h 4 2 0 -2 -4 2 0 -2 -4 -6 -6 -8 -10 -10 mean distance = 2.6 -8 -6 -4 -2 0 SLR, 8 h 2 4 6 8 -8 10 mean distance = 2.5 -8 -6 -4 -2 0 SLR, 8 h 2 4 6 8 Acknowledgments Angelika Longacre L. Ridgway Scott My lab personnel and collaborators Hanli Fan Joyce Rauch Jason Koh University of Chicago Bioinformatics and Computational Biology Core Facility Richard Quigg Terry Clark University of Chicago Functional Genomics Facility Xinmin Li Jaejung Kim Jamie Zhou Chris Dyanov Miglena Petkova