Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Probability Distributions Used in Reliability Engineering Probability Distributions Used in Reliability Engineering Andrew N. OโConnor Mohammad Modarres Ali Mosleh Center for Risk and Reliability 0151 Glenn L Martin Hall University of Maryland College Park, Maryland Published by the Center for Risk and Reliability International Standard Book Number (ISBN): 978-0-9966468-1-9 Copyright © 2016 by the Center for Reliability Engineering University of Maryland, College Park, Maryland, USA All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from The Center for Reliability Engineering, Reliability Engineering Program. The Center for Risk and Reliability University of Maryland College Park, Maryland 20742-7531 In memory of Willie Mae Webb This book is dedicated to the memory of Miss Willie Webb who passed away on April 10 2007 while working at the Center for Risk and Reliability at the University of Maryland (UMD). She initiated the concept of this book, as an aid for students conducting studies in Reliability Engineering at the University of Maryland. Upon passing, Willie bequeathed her belongings to fund a scholarship providing financial support to Reliability Engineering students at UMD. Preface Reliability Engineers are required to combine a practical understanding of science and engineering with statistics. The reliability engineerโs understanding of statistics is focused on the practical application of a wide variety of accepted statistical methods. Most reliability texts provide only a basic introduction to probability distributions or only provide a detailed reference to few distributions. Most texts in statistics provide theoretical detail which is outside the scope of likely reliability engineering tasks. As such the objective of this book is to provide a single reference text of closed form probability formulas and approximations used in reliability engineering. This book provides details on 22 probability distributions. Each distribution section provides a graphical visualization and formulas for distribution parameters, along with distribution formulas. Common statistics such as moments and percentile formulas are followed by likelihood functions and in many cases the derivation of maximum likelihood estimates. Bayesian non-informative and conjugate priors are provided followed by a discussion on the distribution characteristics and applications in reliability engineering. Each section is concluded with online and hardcopy references which can provide further information followed by the relationship to other distributions. The book is divided into six parts. Part 1 provides a brief coverage of the fundamentals of probability distributions within a reliability engineering context. Part 1 is limited to concise explanations aimed to familiarize readers. For further understanding the reader is referred to the references. Part 2 to Part 6 cover Common Life Distributions, Univariate Continuous Distributions, Univariate Discrete Distributions and Multivariate Distributions respectively. The authors would like to thank the many students in the Reliability Engineering Program particularly Reuel Smith for proof reading. Contents i Contents PREFACE ...................................................................................................................................................... V CONTENTS .................................................................................................................................................... I 1. FUNDAMENTALS OF PROBABILITY DISTRIBUTIONS ...................................................... 1 1.1. Probability Theory .......................................................................................................... 2 1.1.1. Theory of Probability ................................................................................................ 2 1.1.2. Interpretations of Probability Theory ....................................................................... 2 1.1.3. Laws of Probability.................................................................................................... 3 1.1.4. Law of Total Probability ............................................................................................ 4 1.1.5. Bayesโ Law ................................................................................................................ 4 1.1.6. Likelihood Functions ................................................................................................. 5 1.1.7. Fisher Information Matrix ......................................................................................... 6 1.2. Distribution Functions .................................................................................................... 9 1.2.1. Random Variables ..................................................................................................... 9 1.2.2. Statistical Distribution Parameters ........................................................................... 9 1.2.3. Probability Density Function ..................................................................................... 9 1.2.4. Cumulative Distribution Function ........................................................................... 11 1.2.5. Reliability Function ................................................................................................. 12 1.2.6. Conditional Reliability Function .............................................................................. 13 1.2.7. 100ฮฑ% Percentile Function ..................................................................................... 13 1.2.8. Mean Residual Life.................................................................................................. 13 1.2.9. Hazard Rate ............................................................................................................ 13 1.2.10. Cumulative Hazard Rate ......................................................................................... 14 1.2.11. Characteristic Function ........................................................................................... 15 1.2.12. Joint Distributions ................................................................................................... 16 1.2.13. Marginal Distribution .............................................................................................. 17 1.2.14. Conditional Distribution.......................................................................................... 17 1.2.15. Bathtub Distributions.............................................................................................. 17 1.2.16. Truncated Distributions .......................................................................................... 18 1.2.17. Summary ................................................................................................................. 19 1.3. Distribution Properties ................................................................................................. 20 1.3.1. Median / Mode ....................................................................................................... 20 1.3.2. Moments of Distribution ........................................................................................ 20 1.3.3. Covariance .............................................................................................................. 21 1.4. Parameter Estimation ................................................................................................... 22 1.4.1. Probability Plotting Paper ....................................................................................... 22 1.4.2. Total Time on Test Plots ......................................................................................... 23 1.4.3. Least Mean Square Regression ............................................................................... 24 1.4.4. Method of Moments .............................................................................................. 25 1.4.5. Maximum Likelihood Estimates .............................................................................. 26 1.4.6. Bayesian Estimation ................................................................................................ 27 ii Probability Distributions Used in Reliability Engineering 1.4.7. 1.5. Confidence Intervals ............................................................................................... 30 Related Distributions .................................................................................................... 33 1.6. Supporting Functions .................................................................................................... 34 1.6.1. Beta Function ๐๐๐๐, ๐๐ ................................................................................................ 34 1.6.2. Incomplete Beta Function ๐ฉ๐ฉ๐ฉ๐ฉ(๐๐; ๐๐, ๐๐) .................................................................... 34 1.6.3. Regularized Incomplete Beta Function ๐ฐ๐ฐ๐ฐ๐ฐ(๐๐; ๐๐, ๐๐) .................................................. 34 1.6.4. Complete Gamma Function ๐ช๐ช(๐๐) ........................................................................... 34 1.6.5. Upper Incomplete Gamma Function ๐ช๐ช(๐๐, ๐๐) .......................................................... 35 1.6.6. Lower Incomplete Gamma Function ๐๐(๐๐, ๐๐)........................................................... 35 1.6.7. Digamma Function ๐๐๐๐ ........................................................................................... 36 1.6.8. Trigamma Function ๐๐โฒ๐๐.......................................................................................... 36 1.7. Referred Distributions .................................................................................................. 37 1.7.1. Inverse Gamma Distribution ๐ฐ๐ฐ๐ฐ๐ฐ(๐ถ๐ถ, ๐ท๐ท).................................................................... 37 1.7.2. Student T Distribution ๐ป๐ป(๐ถ๐ถ, ๐๐, ๐๐๐๐) ......................................................................... 37 1.7.3. F Distribution ๐ญ๐ญ(๐๐๐๐, ๐๐๐๐) ........................................................................................ 37 1.7.4. Chi-Square Distribution ๐๐๐๐(๐๐) ............................................................................... 37 1.7.5. Hypergeometric Distribution ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ๐ฏ(๐๐; ๐๐, ๐๐, ๐ต๐ต) ....................................... 38 1.7.6. Wishart Distribution ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ(๐๐; ๐บ๐บ, ๐๐) .............................................................. 38 1.8. 2. Nomenclature and Notation ......................................................................................... 39 COMMON LIFE DISTRIBUTIONS ............................................................................................40 2.1. Exponential Continuous Distribution ............................................................................ 41 2.2. Lognormal Continuous Distribution .............................................................................. 49 2.3. Weibull Continuous Distribution .................................................................................. 59 3. BATHTUB LIFE DISTRIBUTIONS ...........................................................................................68 3.1. 2-Fold Mixed Weibull Distribution ................................................................................ 69 3.2. Exponentiated Weibull Distribution ............................................................................. 76 3.3. Modified Weibull Distribution ...................................................................................... 81 4. UNIVARIATE CONTINUOUS DISTRIBUTIONS ...................................................................85 4.1. Beta Continuous Distribution ....................................................................................... 86 4.2. Birnbaum Saunders Continuous Distribution ............................................................... 93 4.3. Contents iii Gamma Continuous Distribution .................................................................................. 99 4.4. Logistic Continuous Distribution ................................................................................. 108 4.5. Normal (Gaussian) Continuous Distribution ............................................................... 115 4.6. Pareto Continuous Distribution .................................................................................. 125 4.7. Triangle Continuous Distribution ................................................................................ 131 4.8. Truncated Normal Continuous Distribution................................................................ 135 4.9. Uniform Continuous Distribution ............................................................................... 145 5. UNIVARIATE DISCRETE DISTRIBUTIONS ....................................................................... 151 5.1. Bernoulli Discrete Distribution ................................................................................... 152 5.2. Binomial Discrete Distribution .................................................................................... 157 5.3. Poisson Discrete Distribution ...................................................................................... 165 6. BIVARIATE AND MULTIVARIATE DISTRIBUTIONS ..................................................... 172 6.1. Bivariate Normal Continuous Distribution .................................................................. 173 6.2. Dirichlet Continuous Distribution ............................................................................... 181 6.3. Multivariate Normal Continuous Distribution ............................................................ 187 6.4. Multinomial Discrete Distribution .............................................................................. 193 7. REFERENCES .............................................................................................................................. 200 iv Probability Distributions Used in Reliability Engineering Introduction 1 Prob Theory 1. Fundamentals of Probability Distributions Prob Theory 2 Probability Distributions Used in Reliability Engineering 1.1. Probability Theory 1.1.1. Theory of Probability The theory of probability formalizes the representation of probabilistic concepts through a set of rules. The most common reference to formalizing the rules of probability is through a set of axioms proposed by Kolmogorov in 1933. Where ๐ธ๐ธ๐๐ is an event in the event space ฮฉ =โช๐๐๐๐=1 ๐ธ๐ธ๐๐ with ๐๐ different events. 0 โค ๐๐(๐ธ๐ธ๐๐ ) โค 1 ๐๐(ฮฉ) = 1 and ๐๐(๐๐) = 0 ๐๐(๐ธ๐ธ1 โช ๐ธ๐ธ2 ) = ๐๐(๐ธ๐ธ1 ) + ๐๐(๐ธ๐ธ2 ) When ๐ธ๐ธ1 and ๐ธ๐ธ2 are mutually exclusive. Other representations of uncertainty exist such as fuzzy logic and theory of evidence (Dempster-Shafer model) which do not follow the theory of probability but almost all reliability concepts are defined based on probability as the metric of uncertainty. For a justification of probability theory see (Singpurwalla 2006). 1.1.2. Interpretations of Probability The two most common interpretations of probability are: โข Frequency Interpretation. In the frequentist interpretation of probability, the probability of an event (failure) is defined as: ๐๐(๐พ๐พ) = lim ๐๐โโ ๐๐๐๐ ๐๐ Also known as the classical approach, this interpretation assumes there exists a real probability of an event, ๐๐. The analyst uses the observed frequency of the event to estimate the value of ๐๐. The more historic events that have occurred, the more confident the analyst is of the estimation of ๐๐. This approach does have limitations, for instance when data from events are not available (e.g. no failures occur in a test) ๐๐ cannot be estimated and this method cannot incorporate โsoft evidenceโ such as expert opinion. โข Subjective Interpretation. The subjective interpretation of probability is also known as the Bayesian school of thought. This method defines the probability of an event as degree of belief the analyst has on the occurrence of event. This means probability is a product of the analystโs state of knowledge. Any evidence which would change the analystโs degree of belief must be considered when calculating the probability (including soft evidence). The assumption is made that the probability assessment is made by a coherent person where any coherent person having the same state of knowledge would make the same assessment. Introduction 3 1.1.3. Laws of Probability The following rules of logic form the basis for many mathematical operations within the theory of probability. Let ๐๐ = ๐ธ๐ธ๐๐ and ๐๐ = ๐ธ๐ธ๐๐ be two events within the sample space ฮฉ where ๐๐ โ ๐๐. Boolean Laws of probability are (Modarres et al. 1999, p.25): ๐๐ โช ๐๐ = ๐๐ โช ๐๐ ๐๐ โฉ ๐๐ = ๐๐ โฉ ๐๐ ๐๐ โช (๐๐ โช ๐๐) = (๐๐ โช ๐๐) โช ๐๐ ๐๐ โฉ (๐๐ โฉ ๐๐) = (๐๐ โฉ ๐๐) โฉ ๐๐ ๐๐ โฉ (๐๐ โช ๐๐) = (๐๐ โฉ ๐๐) โช (๐๐ โฉ ๐๐) ๐๐ โช ๐๐ = ๐๐ ๐๐ โฉ ๐๐ = ๐๐ ๏ฟฝ=ฮฉ XโชX Xโฉ๏ฟฝ X=Ø ๏ฟฝ X=X ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ (๐๐ โช ๐๐) = ๐๐๏ฟฝ โฉ ๐๐๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ (๐๐ โฉ ๐๐) = ๐๐๏ฟฝ โช ๐๐๏ฟฝ ๐๐ โช (๐๐๏ฟฝ โฉ ๐๐) = ๐๐ โช ๐๐ Two events are mutually exclusive if: X โฉ Y = Ø, Commutative Law Associative Law Distributive Law Idempotent Law Complementation Law De Morganโs Theorem ๐๐(๐๐ โฉ ๐๐) = 0 Two events are independent if one event ๐๐ occurring does not affect the probability of the second event ๐๐ occurring: ๐๐(๐๐|๐๐) = ๐๐(๐๐) The rules for evaluating the probability of compound events are: Addition Rule: Multiplication Rule: ๐๐(๐๐ โช ๐๐) = ๐๐(๐๐) + ๐๐(๐๐) โ ๐๐(๐๐ โฉ ๐๐) = ๐๐(๐๐) + ๐๐(๐๐) โ ๐๐(๐๐) ๐๐(๐๐|๐๐) ๐๐(๐๐ โฉ ๐๐) = ๐๐(๐๐) ๐๐(๐๐|๐๐) = ๐๐(๐๐) ๐๐(๐๐|๐๐) When ๐๐ and ๐๐ are independent: ๐๐(๐๐ โช ๐๐) = ๐๐(๐๐) + ๐๐(๐๐) โ ๐๐(๐๐) ๐๐(๐๐) ๐๐(๐๐ โฉ ๐๐) = ๐๐(๐๐) ๐๐(๐๐) Prob Theory The subjective interpretation has the flexibility of including many types of evidence to assist in estimating the probability of an event. This is important in many reliability applications where the events of interest (e. g, system failure) are rare. Prob Theory 4 Probability Distributions Used in Reliability Engineering Generalizations of these equations: ๐๐(๐ธ๐ธ1 โช ๐ธ๐ธ2 โช โฆ โช ๐ธ๐ธ๐๐ ) = [๐๐(๐ธ๐ธ1 ) + ๐๐(๐ธ๐ธ2 ) + โฏ + ๐๐(๐ธ๐ธ๐๐ )] โ [๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ2 ) + ๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ3 ) + โฏ + ๐๐(๐ธ๐ธ๐๐โ1 โฉ ๐ธ๐ธ๐๐ )] + [๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ2 โฉ ๐ธ๐ธ3 ) + ๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ2 โฉ ๐ธ๐ธ4 ) + โฏ ] โ โฏ (โ1)๐๐+1 [๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ2 โฉ โฆ โฉ ๐ธ๐ธ๐๐ )] ๐๐(๐ธ๐ธ1 โฉ ๐ธ๐ธ2 โฉ โฆ โฉ ๐ธ๐ธ๐๐ ) = ๐๐(๐ธ๐ธ1 ) . ๐๐(๐ธ๐ธ2 |๐ธ๐ธ1 ). ๐๐(๐ธ๐ธ3 |๐ธ๐ธ1 โฉ ๐ธ๐ธ2 ) โฆ ๐๐(๐ธ๐ธ๐๐ |๐ธ๐ธ1 โฉ ๐ธ๐ธ2 โฉ โฆ โฉ ๐ธ๐ธ๐๐โ1 ) 1.1.4. Law of Total Probability The probability of ๐๐ can be obtained by the following summation; ๐๐๐ด๐ด ๐๐(๐๐) = ๏ฟฝ ๐๐(๐ด๐ด๐๐ )๐๐(๐๐|๐ด๐ด๐๐ ) ๐๐=1 where ๐ด๐ด = {๐ด๐ด1 , ๐ด๐ด2 , โฆ , ๐ด๐ด๐๐๐ด๐ด } is a partition of the sample space, ฮฉ, and all the elements of A are mutually exclusive, Ai โฉ Aj = Ø, and the union of all A elements cover the complete ๐๐๐ด๐ด ๐ด๐ด๐๐ = ฮฉ. sample space, โช๐๐=1 For example: 1.1.5. Bayesโ Law ๐๐(๐๐) = ๐๐(๐๐ โฉ ๐๐) + ๐๐(๐๐ โฉ ๐๐๏ฟฝ) = ๐๐(๐๐)๐๐(๐๐|๐๐) + ๐๐(๐๐)๐๐(๐๐|๐๐๏ฟฝ) Bayesโ law, can be derived from the multiplication rule and the law of total probability as follows: ๐๐(๐๐) ๐๐(๐ธ๐ธ|๐๐) = ๐๐(๐ธ๐ธ) ๐๐(๐๐|๐ธ๐ธ) ๐๐(๐๐|๐ธ๐ธ) = ๐๐(๐๐|๐ธ๐ธ) = ๐๐(๐๐) ๐๐(๐ธ๐ธ|๐๐) ๐๐(๐ธ๐ธ) ๐๐(๐๐) ๐๐(๐ธ๐ธ|๐๐) โ๐๐ ๐๐(๐ธ๐ธ|๐๐๐๐ )๐๐(๐๐๐๐ ) ๐๐ the unknown of interest (UOI). ๐๐(๐๐) the prior state of knowledge about ๐๐ without the evidence. Also denoted as ๐๐๐๐ (๐๐). ๐ธ๐ธ ๐๐(๐ธ๐ธ|๐๐) the observed random variable, evidence. the likelihood of observing the evidence given the UOI. Also denoted as ๐ฟ๐ฟ(๐ธ๐ธ|๐๐). Introduction the posterior state of knowledge about ๐๐ given the evidence. Also denoted as ๐๐(๐๐|๐ธ๐ธ). โ๐๐ ๐๐(๐ธ๐ธ|๐๐๐๐ )๐๐(๐๐) is the normalizing constant. Thus Bayes formula enables us to use a piece of evidence , ๐ธ๐ธ, to make inference about the unobserved ๐๐ . The continuous form of Bayesโ Law can be written as: ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐(๐๐|๐ธ๐ธ) = โซ ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ In Bayesian statistics the state of knowledge (uncertainty) of an unknown of interest is quantified by assigning a probability distribution to its possible values. Bayesโ law provides a mathematical means by which this uncertainty can be updated given new evidence. 1.1.6. Likelihood Functions The likelihood function is the probability of observing the evidence (e.g., sample data), ๐ธ๐ธ, given the distribution parameters, ๐๐. The probability of observing events is the product of each event likelihood: ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = ๐๐ ๏ฟฝ ๐ฟ๐ฟ(๐๐|๐ก๐ก๐๐ ) ๐๐ ๐๐ is a combinatorial constant which quantifies the number of combination which the observed evidence could have occurred. Methods which use the likelihood function in parameter estimation do not depend on the constant and so it is omitted. The following table summarizes the likelihood functions for different types of observations: Table 1: Summary of Likelihood Functions (Klein & Moeschberger 2003, p.74) Type of Observation Exact Lifetimes Right Censored Left Censored Interval Censored Left Truncated Right Truncated Interval Truncated Likelihood Function Example Description ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐๐(๐ก๐ก๐๐ |๐๐) Failure time is known ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐น๐น(๐ก๐ก๐๐ |๐๐) Component failed before time ๐ก๐ก๐๐ ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐ ๐ (๐ก๐ก๐๐ |๐๐) ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐น๐น(๐ก๐ก๐๐๐ ๐ ๐ ๐ |๐๐) โ ๐น๐น(๐ก๐ก๐๐๐ฟ๐ฟ๐ฟ๐ฟ |๐๐) ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐ฟ๐ฟ๐๐ (๐๐|๐ก๐ก๐๐ ) = ๐๐(๐ก๐ก๐๐ |๐๐) ๐ ๐ (๐ก๐ก๐ฟ๐ฟ |๐๐) ๐๐(๐ก๐ก๐๐ |๐๐) ๐น๐น(๐ก๐ก๐๐ |๐๐) ๐๐(๐ก๐ก๐๐ |๐๐) ๐น๐น(๐ก๐ก๐๐ |๐๐) โ ๐น๐น(๐ก๐ก๐ฟ๐ฟ |๐๐) Component survived to time ๐ก๐ก๐๐ Component failed between ๐ก๐ก๐๐๐ฟ๐ฟ๐ฟ๐ฟ and ๐ก๐ก๐๐๐ ๐ ๐ ๐ Component failed at time ๐ก๐ก๐๐ where observations are truncated before ๐ก๐ก๐ฟ๐ฟ . Component failed at time ๐ก๐ก๐๐ where observations are truncated after ๐ก๐ก๐๐ . Component failed at time ๐ก๐ก๐๐ where observations are truncated before ๐ก๐ก๐ฟ๐ฟ and after ๐ก๐ก๐๐ . Prob Theory ๐๐(๐๐|๐ธ๐ธ) 5 Prob Theory 6 Probability Distributions Used in Reliability Engineering The Likelihood function is used in Bayesian inference and maximum likelihood parameter estimation techniques. In both instances any constant in front of the likelihood function becomes irrelevant. Such constants are therefore not included in the likelihood functions given in this book (nor in most references). For example, consider the case where a test is conducted on ๐๐ components with an exponential time to failure distribution. The test is terminated at ๐ก๐ก๐ ๐ during which ๐๐ components failed at times ๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ and ๐ ๐ = ๐๐ โ ๐๐ components survived. Using the exponential distribution to construct the likelihood function we obtain: ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = = ๐๐๐๐ ๐๐๐น๐น ๏ฟฝ ๐๐๏ฟฝ๐๐๏ฟฝ๐ก๐ก๐๐๐น๐น ๏ฟฝ ๏ฟฝ ๐ ๐ ๏ฟฝ๐๐๏ฟฝ๐ก๐ก๐๐๐๐ ๏ฟฝ ๐๐=1 ๐๐=1 ๐๐๐น๐น ๐น๐น ๏ฟฝ ๐๐๐๐ โ๐๐๐ก๐ก๐๐ ๐๐=1 ๐๐๐น๐น ๐๐๐๐ ๐๐ ๏ฟฝ ๐๐ โ๐๐๐ก๐ก๐๐ ๐๐=1 ๐๐๐๐ ๐น๐น ๐๐ = ๐๐๐๐๐น๐น ๐๐ โ๐๐ โ๐๐=1 ๐ก๐ก๐๐ ๐๐ โฮป โ๐๐=1 ๐ก๐ก๐๐ ๐๐๐น๐น ๐๐๐๐ ๐น๐น ๐๐ = ๐๐๐๐๐น๐น ๐๐ โ๐๐๏ฟฝโ๐๐=1 ๐ก๐ก๐๐ +โ๐๐=1 ๐ก๐ก๐๐ ๏ฟฝ Alternatively, because the test described is a homogeneous Poisson process 1 the likelihood function could also have been constructed using a Poisson distribution. The data can be stated as seeing r failure in time ๐ก๐ก ๐๐ where ๐ก๐ก ๐๐ is the total time on test ๐ก๐ก ๐๐ = ๐๐๐๐ ๐๐ ๐น๐น โ๐๐๐๐=1 ๐ก๐ก๐๐๐น๐น + โ๐๐=1 ๐ก๐ก๐๐ . Therefore the likelihood function would be: ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = ๐๐(๐๐|๐๐๐น๐น , ๐ก๐ก ๐๐ ) = (๐๐๐ก๐ก ๐๐ )๐๐๐น๐น โ๐๐๐ก๐ก ๐๐ ๐๐ ๐๐๐น๐น ! = ๐๐๐๐๐๐๐น๐น ๐๐ โ๐๐๐ก๐ก๐๐ ๐๐๐น๐น ๐น๐น ๐๐๐๐ ๐๐ = ๐๐๐๐๐น๐น ๐๐ โ๐๐๏ฟฝโ๐๐=1 ๐ก๐ก๐๐ +โ๐๐=1 ๐ก๐ก๐๐ ๏ฟฝ As mentioned earlier, in estimation procedures within this book, the constant ๐๐ can be ignored. As such, the two likelihood functions are equal. For more information see (Meeker & Escobar 1998, p.36) or (Rinne 2008, p.403). 1.1.7. Fisher Information Matrix The Fisher Information Matrix has many uses but in reliability applications it is most often used to create Jefferyโs non-informative priors. There are two types of Fisher information matrices, the Expected Fisher Information Matrix ๐ผ๐ผ(๐๐), and the Observed Fisher Information Matrix ๐ฝ๐ฝ(๐๐). 1 Homogeneous in time, where it does not matter if you have ๐๐ components on test at once (exponential test), or you have a single component on test which is replaced after failure ๐๐ times (Poisson process), the evidence produced will be the same. Introduction 7 For a single parameter distribution: ๐ผ๐ผ(๐๐) = โ๐ธ๐ธ ๏ฟฝ 2 ๐๐ฮ(๐๐|๐ฅ๐ฅ) ๐๐ 2 ฮ(๐๐|๐ฅ๐ฅ) ๏ฟฝ = ๏ฟฝ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐๐๐ 2 ๐๐๐๐ where ฮ is the log-likelihood function and ๐ธ๐ธ[๐๐] = โซ ๐๐๐๐(๐ฅ๐ฅ)๐๐๐๐. For a distribution with ๐๐ parameters the Expected Fisher Information Matrix is: ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) โกโ๐ธ๐ธ ๏ฟฝ ๏ฟฝ ๐๐๐๐12 โข โข ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) ๏ฟฝ โ๐ธ๐ธ ๏ฟฝ ๐ผ๐ผ(๐ฝ๐ฝ) = โข ๐๐๐๐2 ๐๐๐๐1 โข โฎ โข 2 โขโ๐ธ๐ธ ๏ฟฝ๐๐ ฮ(๐ฝ๐ฝ|๐๐)๏ฟฝ ๐๐๐๐๐๐ ๐๐๐๐1 โฃ ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) ๏ฟฝ ๐๐๐๐1 ๐๐๐๐2 ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) โ๐ธ๐ธ ๏ฟฝ ๏ฟฝ ๐๐๐๐22 โฎ ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) โ๐ธ๐ธ ๏ฟฝ ๏ฟฝ ๐๐๐๐๐๐ ๐๐๐๐2 โ๐ธ๐ธ ๏ฟฝ ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) ๏ฟฝโค ๐๐๐๐1 ๐๐๐๐๐๐ โฅ ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐) โฅ ๏ฟฝ โฏ โ๐ธ๐ธ ๏ฟฝ ๐๐๐๐2 ๐๐๐๐๐๐ โฅ โฅ โฑ โฎ 2 ฮ(๐ฝ๐ฝ|๐๐) โฅ ๐๐ ๏ฟฝโฅ โฆ โ๐ธ๐ธ ๏ฟฝ ๐๐๐๐๐๐2 โฆ โฏ โ๐ธ๐ธ ๏ฟฝ The Observed Fisher Information Matrix is obtained from a likelihood function constructed from ๐๐ observed samples from the distribution. The expectation term is dropped. For a single parameter distribution: ๐๐ ๐ฝ๐ฝ๐๐ (๐๐) = โ ๏ฟฝ ๐๐=1 ๐๐ 2 ฮ(๐๐|๐ฅ๐ฅ๐๐ ) ๐๐๐๐ 2 For a distribution with ๐๐ parameters the Observed Fisher Information Matrix is: ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐๐๐ ) ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐๐๐ ) ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐๐๐ ) โกโ โค โฏ โ โ ๐๐๐๐1 ๐๐๐๐๐๐ โฅ ๐๐๐๐1 ๐๐๐๐2 ๐๐๐๐12 โข ๐๐ โข ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐ ) ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐๐๐ ) ๐๐ 2 ฮ(๐ฝ๐ฝ|๐๐๐๐ )โฅ ๐๐ โ โ โฏ โ โข 2 ๐ฝ๐ฝ๐๐ (๐ฝ๐ฝ) = ๏ฟฝ ๐๐๐๐2 ๐๐๐๐1 ๐๐๐๐2 ๐๐๐๐๐๐ โฅ ๐๐๐๐2 โข โฅ ๐๐=1 โฑ โฎ โฎ โฎ โข 2 ฮ(๐ฝ๐ฝ|๐๐ ) โฅ 2 ฮ(๐ฝ๐ฝ|๐๐ ) 2 ) ๐๐ ๐๐ ฮ(๐ฝ๐ฝ|๐๐๐๐ ๐๐ โฅ ๐๐ โขโ ๐๐ โฆ โ โ ๐๐๐๐๐๐ ๐๐๐๐1 ๐๐๐๐๐๐ ๐๐๐๐2 ๐๐๐๐๐๐2 โฆ โฃ It can be seen that as ๐๐ becomes large, the average value of the random variable approaches its expected value and so the following asymptotic relationship exists between the observed and expected Fisher information matrices: plim ๐๐โโ 1 ๐ฝ๐ฝ (๐ฝ๐ฝ) = ๐ผ๐ผ(๐ฝ๐ฝ) ๐๐ ๐๐ For large n the following approximation can be used: ๐ฝ๐ฝ๐๐ โ ๐๐๐๐(๐ฝ๐ฝ) Prob Theory The Expected Fisher Information Matrix is obtained from a log-likelihood function from a single random variable. The random variable is replaced by its expected value. Prob Theory 8 Probability Distributions Used in Reliability Engineering ๏ฟฝ the observed Fisher information matrix estimates the varianceWhen evaluated at ๐ฝ๐ฝ = ๐ฝ๐ฝ covariance matrix: ๏ฟฝ )๏ฟฝ ๐ฝ๐ฝ = ๏ฟฝ๐ฝ๐ฝ๐๐ (๐ฝ๐ฝ = ๐ฝ๐ฝ โ1 ๐๐๐๐๐๐(๐๐๏ฟฝ1 ) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐๏ฟฝ1 , ๐๐๏ฟฝ2 ) โฏ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐๏ฟฝ1 , ๐๐๏ฟฝ๐๐ ) โก โค ๏ฟฝ ๏ฟฝ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐ โฏ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐๏ฟฝ2 , ๐๐๏ฟฝ๐๐ )โฅ , ๐๐ ) ๐๐๐๐๐๐(๐๐๏ฟฝ2 ) 1 2 =โข โฎ โฎ โฑ โฎ โข โฅ โฆ ๐๐๐๐๐๐(๐๐๏ฟฝ๐๐ ) โฆ โฃ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐๏ฟฝ1 , ๐๐๏ฟฝ๐๐ ) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐๏ฟฝ2 , ๐๐๏ฟฝ๐๐ ) 9 1.2. Distribution Functions 1.2.1. Random Variables Probability distributions are used to model random events for which the outcome is uncertain such as the time of failure for a component. Before placing a demand on that component, the time it will fail is unknown. The distribution of the probability of failure at different times is modeled by a probability distribution. In this book random variables will be denoted as capital letter such as ๐๐ for time. When the random variable assumes a value we denote this by small caps such as ๐ก๐ก for time. For example, if we wish to find the probability that the component fails before time ๐ก๐ก1 we would find ๐๐(๐๐ โค ๐ก๐ก1 ). Random variables are classified as either discrete or continuous. In a discrete distribution, the random variable can take on a distinct or countable number of possible values such as number of demands to failure. In a continuous distribution the random variable is not constrained to distinct possible values such as time-to-failure distribution. This book will denote continuous random variables as ๐๐ or ๐๐, and discrete random variables as ๐พ๐พ. 1.2.2. Statistical Distribution Parameters The parameters of a distribution are the variables which need to be specified in order to completely specify the distribution. Often parameters are classified by the effect they have on the distributions. Shape parameters define the shape of the distribution, scale parameters stretch the distribution along the random variable axis, and location parameters shift the distribution along the random variable axis. The reader is cautioned that the parameters for a distribution may change depending on the text. Therefore, before using formulas from other sources the parameterization need to be confirmed. Understanding the effect of changing a distributionโs parameter value can be a difficult task. At the beginning of each section a graph of the distribution is shown with varied parameters. 1.2.3. Probability Density Function A probability density function (pdf), denoted as ๐๐(๐ก๐ก) is any function which is always positive and has a unit area: โ ๏ฟฝ ๐๐(๐ก๐ก) ๐๐๐๐ = 1, โโ ๏ฟฝ ๐๐(๐๐) = 1 ๐๐ The probability of an event occurring between limits a and b is the area under the pdf: ๐๐ ๐๐(๐๐ โค ๐๐ โค ๐๐) = ๏ฟฝ ๐๐(๐ก๐ก) ๐๐๐๐ = ๐น๐น(๐๐) โ ๐น๐น(๐๐) ๐๐ Dist Functions Introduction Dist Functions 10 Probability Distributions Used in Reliability Engineering ๐๐ ๐๐(๐๐ โค ๐พ๐พ โค ๐๐) = ๏ฟฝ ๐๐(๐๐) = ๐น๐น(๐๐) โ ๐น๐น(๐๐ โ 1) ๐๐=๐๐ The instantaneous value of a discrete pdf at ๐๐๐๐ can be obtained by minimizing the limits to [๐๐๐๐โ1 , ๐๐๐๐ ]: ๐๐(๐พ๐พ = ๐๐๐๐ ) = ๐๐(๐๐๐๐ < ๐พ๐พ โค ๐๐๐๐ ) = ๐๐(๐๐) The instantaneous value of a continuous pdf is infinitesimal. This result can be seen when minimizing the limits to [๐ก๐ก, ๐ก๐ก + ฮ๐ก๐ก]: ๐๐(๐๐ = ๐ก๐ก) = lim ๐๐(๐ก๐ก < ๐๐ โค ๐ก๐ก + ฮ๐ก๐ก) = lim ๐๐(๐ก๐ก). ฮ๐ก๐ก ฮtโ0 ฮtโ0 Therefore the reader must remember that in order to calculate the probability of an event, an interval for the random variable must be used. Furthermore, a common misunderstanding is that a pdf cannot have a value above one because the probability of an event occurring cannot be greater than one. As can be seen above this is true for discrete distributions, only because ฮ๐๐ = 1. However for continuous the case the pdf is multiplied by a small interval ฮ๐ก๐ก, which ensures that the probability an event occurring within the interval ฮ๐ก๐ก is less than one. ๐๐(๐ก๐ก) ๐๐(๐๐) ๐๐๐๐๐๐ ๐๐๐๐๐๐ ๐๐(๐๐1 < ๐พ๐พ โค ๐๐2 ) ๐๐(๐ก๐ก1 < ๐๐ โค ๐ก๐ก2 ) ๐ก๐ก1 ๐ก๐ก2 ๐ก๐ก ๐๐ ๐๐1 ๐๐2 Figure 1: Left: continuous pdf, right: discrete pdf To derive the continuous pdf relationship to the cumulative density function (cdf), ๐น๐น(๐ก๐ก): lim ๐๐(๐ก๐ก). ฮ๐ก๐ก = lim ๐๐(๐ก๐ก < ๐๐ โค ๐ก๐ก + ฮ๐ก๐ก) = lim ๐น๐น(๐ก๐ก + ฮ๐ก๐ก) โ ๐น๐น(๐ก๐ก) = lim ฮ๐น๐น(๐ก๐ก) ฮtโ0 ฮtโ0 ฮtโ0 ๐๐(๐ก๐ก) = lim ฮtโ0 ฮ๐น๐น(๐ก๐ก) ๐๐๐๐(๐ก๐ก) = ฮ๐ก๐ก ๐๐๐๐ ฮtโ0 The shape of the pdf can be obtained by plotting a normalized histogram of an infinite number of samples from a distribution. It should be noted when plotting a discrete pdf the points from each discrete value should not be joined. For ease of explanation using the area under the graph argument the step 11 plot is intuitive but implies a non-integer random variable. Instead stem plots or column plots are often used. ๐๐(๐๐) ๐๐(๐๐) ๐๐ Figure 2: Discrete data plotting. Left stem plot. Right column plot. ๐๐ 1.2.4. Cumulative Distribution Function The cumulative density function (cdf), denoted by ๐น๐น(๐ก๐ก) is the probability of the random event occurring before ๐ก๐ก, ๐๐(๐๐ โค ๐ก๐ก). For a discrete cdf the height of each step is the pdf value ๐๐(๐๐๐๐ ). ๐ก๐ก ๐น๐น(๐ก๐ก) = ๐๐(๐๐ โค ๐ก๐ก) = ๏ฟฝ ๐๐(๐ฅ๐ฅ) ๐๐๐๐ , โโ ๐น๐น(๐๐) = ๐๐(๐พ๐พ โค ๐๐) = ๏ฟฝ ๐๐(๐๐๐๐ ) The limits of the cdf for โโ < ๐ก๐ก < โ and 0 โค ๐๐ โค โ are given as: lim ๐น๐น(๐ก๐ก) = 0 , ๐ก๐กโโโ lim ๐น๐น(๐ก๐ก) = 1 , ๐ก๐กโโ ๐๐๐๐ โค๐๐ ๐น๐น(โ1) = 0 lim ๐น๐น(๐๐) = 1 ๐๐โโ The cdf can be used to find the probability of the random even occurring between two limits: ๐๐ ๐๐(๐๐ โค ๐๐ โค ๐๐) = ๏ฟฝ ๐๐(๐ก๐ก) ๐๐๐๐ = ๐น๐น(๐๐) โ ๐น๐น(๐๐) ๐๐ ๐๐ ๐๐(๐๐ โค ๐พ๐พ โค ๐๐) = ๏ฟฝ ๐๐(๐๐) = ๐น๐น(๐๐) โ ๐น๐น(๐๐ โ 1) ๐๐=๐๐ Dist Functions Introduction Dist Functions 12 Probability Distributions Used in Reliability Engineering 1 ๐น๐น(๐ก๐ก) ๐๐(๐๐ โค ๐ก๐ก1 ) ๐น๐น(๐ก๐ก1 ) ๐๐(๐ก๐ก) ๐น๐น(๐๐) 1 ๐๐๐๐๐๐ ๐๐(๐พ๐พ โค ๐๐1 ) ๐น๐น(๐๐1 ) ๐ก๐ก ๐ก๐ก1 ๐๐๐๐๐๐ ๐๐ ๐๐1 ๐๐(๐๐) ๐๐๐๐๐๐ ๐๐(๐พ๐พ โค ๐๐1 ) ๐๐(๐๐ โค ๐ก๐ก1 ) ๐๐๐๐๐๐ ๐ก๐ก ๐ก๐ก1 ๐๐1 ๐๐ Figure 3: Left: continuous cdf/pdf, right: discrete cdf/pdf 1.2.5. Reliability Function The reliability function, also known as the survival function, is denoted as ๐ ๐ (๐ก๐ก). It is the probability that the random event (time of failure) occurs after ๐ก๐ก. ๐ ๐ (๐ก๐ก) = ๐๐(๐๐ > ๐ก๐ก) = 1 โ ๐น๐น(๐ก๐ก), โ ๐ ๐ (๐ก๐ก) = ๏ฟฝ ๐๐(๐ก๐ก) ๐๐๐๐ , ๐ก๐ก ๐ ๐ (๐๐) = ๐๐(๐๐ > ๐๐) = 1 โ ๐น๐น(๐๐) โ ๐ ๐ (๐๐) = ๏ฟฝ ๐๐(๐๐๐๐ ) ๐๐=๐๐+1 It should be noted that in most publications the discrete reliability function is defined as โ ๐ ๐ โ (๐๐) = ๐๐(๐๐ โฅ ๐๐) = โโ ๐๐=๐๐ ๐๐(๐๐). This definition results in ๐ ๐ (๐๐) โ 1 โ ๐น๐น(๐๐). Despite this problem it is the most common definition and is included in all the references in this book except (Xie, Gaudoin, et al. 2002) 1 ๐น๐น(๐ก๐ก) 1 ๐ ๐ (๐ก๐ก) ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐๐๐ 0 ๐ก๐ก 13 0 ๐ก๐ก Figure 4: Left continuous cdf, right continuous survival function 1.2.6. Conditional Reliability Function The conditional reliability function, denoted as ๐๐(๐ฅ๐ฅ) is the probability of the component surviving given that it has survived to time t. ๐ ๐ (๐ก๐ก + x) ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = ๐ ๐ (๐ก๐ก) Where: ๐ก๐ก is the given time for which we know the component survived. ๐ฅ๐ฅ is new random variable defined as the time after ๐ก๐ก. ๐ฅ๐ฅ = 0 at ๐ก๐ก. 1.2.7. 100ฮฑ% Percentile Function The 100ฮฑ% percentile function is the interval [0, ๐ก๐ก๐ผ๐ผ ] for which the area under the pdf is ๐ผ๐ผ. ๐ก๐ก๐ผ๐ผ = ๐น๐น โ1 (๐ผ๐ผ) 1.2.8. Mean Residual Life The mean residual life (MRL), denoted as ๐ข๐ข(๐ก๐ก), is the expected life given the component has survived to time, ๐ก๐ก. โ 1.2.9. Hazard Rate ๐ข๐ข(๐ก๐ก) = ๏ฟฝ ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) ๐๐๐๐ = 0 โ 1 ๏ฟฝ ๐ ๐ (๐ฅ๐ฅ) ๐๐๐๐ ๐ ๐ (๐ก๐ก) ๐ก๐ก The hazard function, denoted as h(t), is the conditional probability that a component fails in a small time interval, given that it has survived from time zero until the beginning of the time interval. For the continuous case the probability that an item will fail in a time interval given the item was functioning at time ๐ก๐ก is: Dist Functions Introduction Dist Functions 14 Probability Distributions Used in Reliability Engineering ๐๐(๐ก๐ก < ๐๐ < ๐ก๐ก + ฮ๐ก๐ก|๐๐ > ๐ก๐ก) = ๐๐(๐ก๐ก < ๐๐ < ๐ก๐ก + ฮ๐ก๐ก) ๐น๐น(๐ก๐ก + ฮ๐ก๐ก) โ ๐น๐น(๐ก๐ก) ฮ๐น๐น(๐ก๐ก) = = ๐๐(๐๐ > ๐ก๐ก) ๐ ๐ (๐ก๐ก) ๐ ๐ (๐ก๐ก) By dividing the probability by ฮ๐ก๐ก and finding the limit as ฮ๐ก๐ก โ 0 gives the hazard rate: โ(๐ก๐ก) = lim ฮ๐ก๐กโ0 ๐๐(๐ก๐ก) ๐๐(๐ก๐ก < ๐๐ < ๐ก๐ก + ฮ๐ก๐ก|๐๐ > ๐ก๐ก) ฮ๐น๐น(๐ก๐ก) = lim = ฮ๐ก๐กโ0 ฮ๐ก๐ก๐ก๐ก(๐ก๐ก) ฮ๐ก๐ก ๐ ๐ (๐ก๐ก) The discrete hazard rate is defined as: (Xie, Gaudoin, et al. 2002) โ(๐๐) = ๐๐(๐พ๐พ = ๐๐) ๐๐(๐๐) = ๐๐(๐พ๐พ โฅ ๐๐) ๐ ๐ (๐๐ โ 1) This unintuitive result is due to a popular definition of ๐ ๐ โ (๐๐) = โโ ๐๐=๐๐ ๐๐(๐๐) in which case โ(๐๐) = ๐๐(๐๐)/๐ ๐ โ (๐๐). This definition has been avoided because it violates the formula ๐ ๐ (๐๐) = 1 โ ๐น๐น(๐๐). The discrete hazard rate cannot be used in the same way as a continuous hazard rate with the following differences (Xie, Gaudoin, et al. 2002): โข โข โข โข โ(๐๐) is defined as a probability and so is bounded by [0,1]. โ(๐๐) is not additive for series systems. For the cumulative hazard rate ๐ป๐ป(๐๐) = โ ln[๐ ๐ (๐๐)] โ โ๐๐๐๐=0 โ(๐๐) When a set of data is analyzed using a discrete counterpart of the continuous distribution the values of the hazard rate do not converge. A function called the second failure rate has been proposed (Gupta et al. 1997): ๐๐(๐๐) = ln ๐ ๐ (๐๐ โ 1) = โ ln[1 โ โ(๐๐)] ๐ ๐ (๐๐) This function overcomes the previously mentioned limitations of the discrete hazard rate function and maintains the monotonicity property. For more information, the reader is referred to (Xie, Gaudoin, et al. 2002) Care should be taken not to confuse the hazard rate with the Rate of Occurrence of Failures (ROCOF). ROCOF is the probability that a failure (not necessarily the first) occurs in a small time interval. Unlike the hazard rate, the ROCOF is the absolute rate at which system failures occur and is not conditional on survival to time t. ROCOF is using in measuring the change in the rate of failures for repairable systems. 1.2.10. Cumulative Hazard Rate The cumulative hazard rate, denoted as ๐ป๐ป(๐ก๐ก) an in the continuous case is the area under the hazard rate function. This function is useful to calculate average failure rates. ๐ก๐ก ๐ป๐ป(๐ก๐ก) = ๏ฟฝ โ(๐ข๐ข)๐๐๐๐ = โln[๐ ๐ (๐ก๐ก)] โ ๐ป๐ป(๐๐) = โ ln[๐ ๐ (๐๐)] For a discussion on the discrete cumulative hazard rate see hazard rate. 1.2.11. 15 Characteristic Function The characteristic function of a random variable completely defines its probability distribution. It can be used to derive properties of the distribution from transformations of the random variable. (Billingsley 1995) The characteristic function is defined as the expected value of the function exp(๐๐๐๐๐๐) where ๐ฅ๐ฅ is the random variable of the distribution with a cdf ๐น๐น(๐ฅ๐ฅ), ๐๐ is a parameter that can have any real value and ๐๐ = โโ1: ๐๐๐๐ (๐๐) = ๐ธ๐ธ๏ฟฝ๐๐ ๐๐๐๐๐๐ ๏ฟฝ โ = ๏ฟฝ ๐๐ ๐๐๐๐๐๐ ๐น๐น(๐ฅ๐ฅ) ๐๐๐๐ โโ A useful property of the characteristic function is the sum of independent random variables is the product of the random variables characteristic function. It is often easier to use the natural log of the characteristic function when conducting this operation. ๐๐๐๐+๐๐ (๐๐) = ๐๐๐๐ (๐๐)๐๐๐๐ (๐๐) ln[๐๐๐๐+๐๐ (๐๐)] = ln[๐๐๐๐ (๐๐)] ln[๐๐๐๐ (๐๐)] For example, the addition of two exponentially distributed random variables with the same ๐๐ gives the gamma distribution with ๐๐ = 2: ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐), ๐๐๐๐ , ๐๐๐๐ (๐๐) = ๐๐ + ๐๐๐๐ ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐ (๐๐) = ๐๐๐๐ ๐๐ + ๐๐๐๐ ๐๐๐๐+๐๐ (๐๐) = ๐๐๐๐ (๐๐)๐๐๐๐ (๐๐) โ๐๐2 = (๐๐ + ๐๐๐๐)2 ๐๐ + ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐ = 1, ๐๐) This is the characteristic function of the gamma distribution with ๐๐ = 2. The moment generating function can be calculated from the characteristic function: ๐๐๐๐ (โ๐๐๐๐) = ๐๐๐๐ (๐๐) The ๐๐๐ก๐กโ raw moment can be calculated by differentiating the characteristic function ๐๐ times. For more information on moments see section 1.3.2. (๐๐) ๐ธ๐ธ[๐๐ ๐๐ ] = ๐๐ โ๐๐ ๐๐๐๐ (0) ๐๐๐๐ = ๐๐ โ๐๐ ๏ฟฝ ๐๐ ๐๐๐๐ (๐๐)๏ฟฝ ๐๐๐๐ Dist Functions Introduction Dist Functions 16 Probability Distributions Used in Reliability Engineering 1.2.12. Joint Distributions Joint distributions are multivariate distributions with, ๐๐ random variables (๐๐ > 1). An example of a bivariate distribution (๐๐ = 2) may be the distribution of failure for a vehicle tire which with random variables time, ๐๐, and distance travelled, ๐๐. The dependence between these two variables can be quantified in terms of correlation and covariance. See section 1.3.3 for more discussion. For more on properties of multivariate distributions see (Rencher 1997). The continuous and discrete random variables will be denoted as: ๐ฅ๐ฅ1 ๐ฅ๐ฅ2 ๐๐ = ๏ฟฝ โฎ ๏ฟฝ , ๐ฅ๐ฅ๐๐ ๐๐1 ๐๐2 ๐๐ = ๏ฟฝ ๏ฟฝ โฎ ๐๐๐๐ Joint distributions can be derived from the conditional distributions. For the bivariate case with random variables ๐ฅ๐ฅ and ๐ฆ๐ฆ: ๐๐(๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๐๐(๐ฆ๐ฆ|๐ฅ๐ฅ)๐๐(๐ฅ๐ฅ) = ๐๐(๐ฅ๐ฅ|๐ฆ๐ฆ)๐๐(๐ฆ๐ฆ) For the more general case: ๐๐(๐๐) = ๐๐(๐ฅ๐ฅ1 |๐ฅ๐ฅ2 , โฆ , ๐ฅ๐ฅ๐๐ )๐๐(๐ฅ๐ฅ2 , โฆ , ๐ฅ๐ฅ๐๐ ) = ๐๐(๐ฅ๐ฅ1 ) ๐๐(๐ฅ๐ฅ2 |๐ฅ๐ฅ1 ) โฆ ๐๐(๐ฅ๐ฅ๐๐โ1 |๐ฅ๐ฅ1 , โฆ , ๐ฅ๐ฅ๐๐โ2 ) ๐๐(๐ฅ๐ฅ๐๐ |๐ฅ๐ฅ1 , โฆ , ๐ฅ๐ฅ๐๐ ) If the random variables are independent, their joint distribution is simply the product of the marginal distributions: ๐๐ ๐๐(๐๐) = ๏ฟฝ ๐๐(๐ฅ๐ฅ๐๐ ) ๐ค๐คโ๐๐๐๐๐๐ ๐ฅ๐ฅ๐๐ โฅ ๐ฅ๐ฅ๐๐ ๐๐๐๐๐๐ ๐๐ โ ๐๐ ๐๐=1 ๐๐ ๐๐(๐๐) = ๏ฟฝ ๐๐(๐๐๐๐ ) ๐ค๐คโ๐๐๐๐๐๐ ๐๐๐๐ โฅ ๐๐๐๐ ๐๐๐๐๐๐ ๐๐ โ ๐๐ ๐๐=1 A general multivariate cumulative probability function with n random variables (๐๐1 , ๐๐2 , โฆ , ๐๐๐๐ ) is defined as: ๐น๐น(๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) = ๐๐(๐๐1 โค ๐ก๐ก1 , ๐๐2 โค ๐ก๐ก2 , โฆ , ๐๐๐๐ โค ๐ก๐ก๐๐ ) The survivor function is given as: ๐ ๐ (๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) = ๐๐(๐๐1 > ๐ก๐ก1 , ๐๐2 > ๐ก๐ก2 , โฆ , ๐๐๐๐ > ๐ก๐ก๐๐ ) Different from univariate distributions is the relationship between the CDF and the survivor function (Georges et al. 2001): ๐น๐น(๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) + ๐ ๐ (๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) โค 1 If ๐น๐น(๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) is differentiable then the probability density function is given as: ๐๐(๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) = 17 ๐๐ ๐๐ ๐น๐น(๐ก๐ก1 , ๐ก๐ก2 , โฆ , ๐ก๐ก๐๐ ) ๐๐๐ก๐ก1 ๐๐๐ก๐ก2 โฆ ๐๐๐ก๐ก๐๐ For a discussion on the multivariate hazard rate functions and the construction of joint distributions from marginal distributions see (Singpurwalla 2006). 1.2.13. Marginal Distribution The marginal distribution of a single random variable in a joint distribution can be obtained: ๐๐(๐ฅ๐ฅ1 ) = ๏ฟฝ โฆ ๏ฟฝ ๏ฟฝ ๐๐(๐๐) ๐๐๐ฅ๐ฅ2 ๐๐๐ฅ๐ฅ3 โฆ ๐๐๐ฅ๐ฅ๐๐ ๐ฅ๐ฅ๐๐ ๐ฅ๐ฅ3 ๐ฅ๐ฅ2 ๐๐(๐๐1 ) = ๏ฟฝ ๏ฟฝ โฆ ๏ฟฝ ๐๐(๐๐) ๐๐2 1.2.14. ๐๐3 ๐๐๐๐ Conditional Distribution If the value is known for some random variables the conditional distribution of the remaining random variables is: ๐๐(๐ฅ๐ฅ1 |๐ฅ๐ฅ2 , โฆ , ๐ฅ๐ฅ๐๐ ) = 1.2.15. ๐๐(๐ฅ๐ฅ2 , โฆ , ๐ฅ๐ฅ๐๐ ) ๐๐(๐๐1 |๐๐2 , โฆ , ๐๐๐๐ ) = Bathtub Distributions ๐๐(๐ฅ๐ฅ) = ๐๐(๐ฅ๐ฅ) โซ๐ฅ๐ฅ ๐๐(๐ฅ๐ฅ) ๐๐๐๐1 1 ๐๐(๐๐) ๐๐(๐๐) = ๐๐(๐๐2 , โฆ , ๐๐๐๐ ) โ๐๐1 ๐๐(๐ฅ๐ฅ) Elementary texts on reliability introduce the hazard rate of a system as a bathtub curve. The bathtub curve has three regions, infant mortality (decreasing failure rate), useful life (constant failure rate) and wear out (increasing failure rate). Bathtub distributions have not been a popular choice for modeling life distributions when compared to exponential, Weibull and lognormal distributions. This is because bathtub distributions are generally more complex without closed form moments and more difficult to estimate parameters. Sometimes more complex shapes are required than simple bathtub curves, as such generalizations and modifications to the bathtub curves has been studied. These include an increase in the failure rate followed by a bathtub curve and rollercoaster curves (decreasing followed by unimodal hazard rate). For further reading including applications see (Lai & Xie 2006). Dist Functions Introduction Dist Functions 18 Probability Distributions Used in Reliability Engineering 1.2.16. Truncated Distributions Truncation arises when the existence of a potential observation would be unknown if it were to occur in a certain range. An example of truncation is when the existence of a defect is unknown due to the defectโs amplitude being less than the inspection threshold. The number of flaws below the inspection threshold is unknown. This is not to be confused with censoring which occurs when there is a bound for observing events. An example of right censoring is when a test is time terminated and the failures of the surviving components are not observed, however we know how many components were censored. (Meeker & Escobar 1998, p.266) A truncated distribution is the conditional distribution that results from restricting the domain of another probability distribution. The following general formulas apply to truncated distribution functions, where ๐๐0 (๐ฅ๐ฅ) and ๐น๐น0 (๐ฅ๐ฅ) are the pdf and cdf of the nontruncated distribution. For further reading specific to common distributions see (Cohen 1991) Probability Distribution Function: ๐๐๐๐ (๐ฅ๐ฅ) ๐๐(๐ฅ๐ฅ) = ๏ฟฝ๐น๐น0 (๐๐) โ ๐น๐น0 (๐๐) ๐๐๐๐๐๐ ๐ฅ๐ฅ โ (๐๐, ๐๐] 0 ๐๐๐๐โ๐๐๐๐๐๐๐๐๐๐๐๐ Cumulative Distribution Function: ๐๐๐๐๐๐ ๐ฅ๐ฅ โค ๐๐ โง ๐ฅ๐ฅ 0 โช โซ ๐๐0 (๐ก๐ก) ๐๐๐๐ ๐๐ ๐น๐น(๐ฅ๐ฅ) = ๐๐๐๐๐๐ ๐ฅ๐ฅ โ (๐๐, ๐๐] โจ๐น๐น0 (๐๐) โ ๐น๐น0 (๐๐) โช 1 ๐๐๐๐๐๐ ๐ฅ๐ฅ > ๐๐ โฉ 1.2.17. 19 Summary Table 2: Summary of important reliability function relationships ๐๐(๐๐) ๐ญ๐ญ(๐๐) ๐น๐น(๐๐) ๐๐(๐๐) ๐๐(๐๐) = --- ๐น๐น โฒ (๐ก๐ก) โ๐ ๐ โฒ (๐ก๐ก) โ(๐ก๐ก) exp ๏ฟฝโ ๏ฟฝ โ(๐ฅ๐ฅ)๐๐๐๐ ๏ฟฝ ๐ญ๐ญ(๐๐) = ๏ฟฝ ๐๐(๐ฅ๐ฅ)๐๐๐๐ ---- 1 โ ๐ ๐ (๐ก๐ก) 1 โ exp ๏ฟฝโ ๏ฟฝ โ(๐ฅ๐ฅ)๐๐๐๐ ๏ฟฝ ๐น๐น(๐๐) = 1 โ ๏ฟฝ ๐๐(๐ฅ๐ฅ)๐๐๐๐ 1 โ ๐น๐น(๐ก๐ก) ---- exp ๏ฟฝโ ๏ฟฝ โ(๐ฅ๐ฅ)๐๐๐๐ ๏ฟฝ ๐๐(๐๐) = ๐๐(๐ก๐ก) ๐น๐น โฒ (๐ก๐ก) 1 โ ๐น๐น(๐ก๐ก) ๐ ๐ โฒ (๐ก๐ก) ๐ ๐ (๐ก๐ก) ๐ก๐ก 0 ๐ก๐ก 0 ๐ก๐ก 1 โ โซ0 ๐๐(๐ฅ๐ฅ)๐๐๐๐ โ 1 ๏ฟฝ 1 โ ๐น๐น(๐ฅ๐ฅ) โ๐๐๐๐ ๏ฟฝ ๐๐(๐ฅ๐ฅ)๐๐๐๐ ๐๐(๐๐) = โซ0 ๐ฅ๐ฅ๐ฅ๐ฅ(๐ก๐ก + ๐ฅ๐ฅ)๐๐๐๐ โซโ[1 โ ๐น๐น(๐ฅ๐ฅ)]๐๐๐๐ โซโ ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ ๐ก๐ก ๐ก๐ก โ 1 โ ๐น๐น(๐ก๐ก) ๐ ๐ (๐ก๐ก) โซ๐ก๐ก ๐๐(๐ฅ๐ฅ)๐๐๐๐ ๐ก๐ก โ ๐ก๐ก ๐๐ ๐ก๐ก ๐๐ โ ๐ก๐ก exp{โ๐ป๐ป(๐ก๐ก)} ๐ป๐ป โฒ (๐ก๐ก) ---- ๐ก๐ก โln{๐ ๐ (๐ฅ๐ฅ)} ๐๐{exp[โ๐ป๐ป(๐ก๐ก)]} ๐๐๐๐ 1 โ exp{โ๐ป๐ป(๐ก๐ก)} ๐๐ ๐ฏ๐ฏ(๐๐) = ln ๏ฟฝ ๐ฏ๐ฏ(๐๐) --- ๏ฟฝ โ(๐ฅ๐ฅ)๐๐๐๐ 0 โ ๐ก๐ก โซ๐ก๐ก exp๏ฟฝโ โซ๐๐ โ(๐ฅ๐ฅ)๐๐๐๐๏ฟฝ ๐๐๐๐ ๐ก๐ก exp๏ฟฝโ โซ๐๐ โ(๐ฅ๐ฅ)๐๐๐ฅ๐ฅ๏ฟฝ โ โซ๐ก๐ก exp{โ๐ป๐ป(๐ฅ๐ฅ)} ๐๐๐๐ exp{โ๐ป๐ป(๐ฅ๐ฅ)} Dist Functions Introduction Dist Properties 20 Probability Distributions Used in Reliability Engineering 1.3. Distribution Properties 1.3.1. Median / Mode The median of a distribution, denoted as ๐ก๐ก0.5 is when the cdf and reliability function are equal to 0.5. ๐ก๐ก0.5 = ๐น๐น โ1 (0.5) = ๐ ๐ โ1 (0.5) The mode is the highest point of the pdf, ๐ก๐ก๐๐ . This is the point where a failure has the highest probability. Samples from this distribution would occur most often around the mode. 1.3.2. Moments of Distribution The moments of a distribution are given by: โ ๐๐๐๐ = ๏ฟฝ (๐ฅ๐ฅ โ ๐๐)๐๐ ๐๐(๐ฅ๐ฅ) ๐๐๐๐ , โโ ๐๐ ๐๐๐๐ = ๏ฟฝ๏ฟฝ๐๐๐๐ โ ๐๐๏ฟฝ ๐๐(๐๐) ๐๐ When ๐๐ = 0 the moments, ๐๐๐๐โฒ , are called the raw moments, described as moments about the origin. In respect to probability distributions the first two raw moments are important. ๐๐0โฒ always equals one, and ๐๐1โฒ is the distributions mean which is the expected value of the random variable for the distribution: โ mean = ๐ธ๐ธ[๐๐] = ๐๐: ๐๐โฒ0 = ๏ฟฝ ๐๐(๐ฅ๐ฅ) ๐๐๐๐ = 1, โโ โ ๐๐โฒ1 = ๏ฟฝ ๐ฅ๐ฅ๐ฅ๐ฅ(๐ฅ๐ฅ) ๐๐๐๐ , โโ ๐๐โฒ0 = ๏ฟฝ ๐๐(๐๐๐๐ ) = 1 ๐๐ ๐๐โฒ1 = ๏ฟฝ ๐๐๐๐ ๐๐(๐๐๐๐ ) ๐๐ Some important properties of the expected value ๐ธ๐ธ[๐๐] when transformations of the random variable occur are: ๐ธ๐ธ[๐๐ + ๐๐] = ๐๐๐๐ + ๐๐ ๐ธ๐ธ[๐๐ + ๐๐] = ๐๐๐๐ + ๐๐๐๐ ๐ธ๐ธ[๐๐๐๐] = ๐๐๐๐๐๐ ๐ธ๐ธ[๐๐๐๐] = ๐๐๐๐ ๐๐๐๐ + ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) When ๐๐ = ๐๐ the moments, ๐๐๐๐ , are called the central moments, described as moments about the mean. In this book, the first five central moments are important. ๐๐0 is equal to ๐๐0โฒ = 1. ๐๐1 is the variance which quantifies the amount the random variable deviates from the mean. ๐๐2 and ๐๐3 are used to calculate the skewness and kurtosis. โ ๐๐0 = ๏ฟฝ ๐๐(๐ฅ๐ฅ) ๐๐๐๐ = 1, โ โโ ๐๐1 = ๏ฟฝ (๐ฅ๐ฅ โ ๐๐)๐๐(๐ฅ๐ฅ) ๐๐๐๐ = 0, โโ ๐๐0 = ๏ฟฝ ๐๐(๐๐๐๐ ) = 1 ๐๐ ๐๐1 = ๏ฟฝ(๐๐๐๐ โ ๐๐)๐๐(๐๐๐๐ ) = 0 ๐๐ 21 variance = ๐ธ๐ธ[(๐๐ โ ๐ธ๐ธ[๐๐])2 ] = ๐ธ๐ธ[๐๐ 2 ] โ {๐ธ๐ธ[๐๐]}2 = ๐๐ 2: โ ๐๐2 = ๏ฟฝ (๐ฅ๐ฅ โ ๐๐)2 ๐๐(๐ฅ๐ฅ) ๐๐๐๐ , ๐๐2 = ๏ฟฝ(๐๐๐๐ โ ๐๐)2 ๐๐(๐๐๐๐ ) โโ ๐๐ Some important properties of the variance exist when transformations of the random variable occur are: ๐๐๐๐๐๐[๐๐ + ๐๐] = ๐๐๐๐๐๐[๐๐] ๐๐๐๐๐๐[๐๐ + ๐๐] = ๐๐๐๐2 + ๐๐๐๐2 ± 2๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) ๐๐๐๐๐๐[๐๐๐๐] = ๐๐2 ๐๐๐๐2 ๐๐๐๐๐๐[๐๐๐๐] = (๐๐๐๐)2 ๏ฟฝ๏ฟฝ ๐๐๐๐ 2 ๐๐๐๐ 2 ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) ๏ฟฝ +๏ฟฝ ๏ฟฝ +2 ๏ฟฝ ๐๐ ๐๐ ๐๐๐๐ The skewness is a measure of the asymmetry of the distribution. ๐พ๐พ1 = ๐๐3 โ2 ๐๐23 The kurtosis is a measure of the whether the data is peaked or flat. 1.3.3. Covariance ๐พ๐พ2 = ๐๐4 ๐๐22 Covariance is a measure of the dependence between random variables. ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) = ๐ธ๐ธ[(๐๐ โ ๐๐๐๐ )(๐๐ โ ๐๐๐๐ )] = ๐ธ๐ธ[๐๐๐๐] โ ๐๐๐๐ ๐๐๐๐ A normalized measure of covariance is correlation, ๐๐. The correlation has the limits โ1 โค ๐๐ โค 1. When ๐๐ = 1 the random variables have a linear dependency (i.e, an increase in X will result in the same increase in Y). When ๐๐ = โ1 the random variables have a negative linear dependency (i.e, an increase in X will result in the same decrease in Y). The relationship between covariance and correlation is: ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) ๐๐๐๐,๐๐ = ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐, ๐๐) = ๐๐๐๐ ๐๐๐๐ If the two random variables are independent than the correlation is equal to zero, however the reverse is not always true. If the correlation is zero the random variables does not need to be independent. For derivations and more information the reader is referred to (Dekking et al. 2007, p.138). Dist Properties Introduction Parameter Est 22 Probability Distributions Used in Reliability Engineering 1.4. Parameter Estimation 1.4.1. Probability Plotting Paper Most plotting methods transform the data available into a straight line for a specific distribution. From a line of best fit the parameters of the distribution can be estimated. Most plotting paper plots the random variable (time or demands) against the pdf, cdf or hazard rate and transform the data points to a linear relationship by adjusting the scale of each axis. Probability plotting is done using the following steps (Nelson 1982, p.108): 1. Order the data such that ๐ฅ๐ฅ1 โค ๐ฅ๐ฅ2 โค โฏ โค ๐ฅ๐ฅ๐๐ โค โฏ โค ๐ฅ๐ฅ๐๐ . 2. Assign a rank to each failure. For complete data this is simply the value i. Censored data is discussed after step 7. 3. Calculate the plotting position. The cdf may simply be calculated as ๐๐/๐๐ however this produces a biased result, instead the following non-parametric Blom estimates, are recommended as suitable for many cases by (Kimball 1960): 1 โ๏ฟฝ(๐ก๐ก๐๐ ) = (๐๐ โ ๐๐ + 0.625)(๐ก๐ก๐๐+1 โ ๐ก๐ก๐๐ ) ๐น๐น๏ฟฝ (๐ก๐ก๐๐ ) = ๐๐ โ 0.375 ๐๐ + 0.25 ๐๐ฬ(๐ก๐ก๐๐ ) = 1 (๐๐ + 0.25)(๐ก๐ก๐๐+1 โ ๐ก๐ก๐๐ ) ๐ ๐ ๏ฟฝ(๐ก๐ก๐๐ ) = Other proposed estimators are: ๐๐ โ ๐๐ + 0.625 (๐๐ + 0.25) ๐๐ ๐๐ ๐๐ โ 0.3 Median (approximate): ๐น๐น๏ฟฝ(๐ก๐ก๐๐ ) = ๐๐ + 0.4 Naive: ๐น๐น๏ฟฝ(๐ก๐ก๐๐ ) = Midpoint: ๐น๐น๏ฟฝ(๐ก๐ก๐๐ ) = Mean โถ ๐น๐น๏ฟฝ (๐ก๐ก๐๐ ) = Mode: ๐น๐น๏ฟฝ(๐ก๐ก๐๐ ) = ๐๐ โ 0.5 ๐๐ ๐๐ ๐๐ + 1 ๐๐ โ 1 ๐๐ โ 1 4. Plot points on probability paper. The choice of distribution should be from experience, or multiple distributions should be used to assess the best fit. Probability paper is available from http://www.weibull.com/GPaper/. 23 5. Assess the data and chosen distributions. If the data plots in straight line then the distribution may be a reasonable fit. 6. Draw a line of best fit. This is a subjective assessment which minimizes the deviation of the points from the chosen line. 7. Obtained the desired information. This may be the distribution parameters or estimates of reliability or hazard rate trends. When multiple failure modes are observed only one failure mode should be plotted with the other failures being treated as censored. Two popular methods to treat censored data two methods are: Rank Adjustment Method. (Manzini et al. 2009, p.140) Here the adjusted rank, ๐๐๐ก๐ก๐๐ is calculated only for non-censored units (with ๐๐๐ก๐ก๐๐ still being the rank for all ordered times). This adjusted rank is used for step 2 with the remaining steps unchanged: (๐๐ + 1) โ ๐๐๐ก๐ก๐๐โ1 ๐๐๐ก๐ก๐๐ = ๐๐๐ก๐ก๐๐โ1 + 2 + ๐๐ โ ๐๐๐ก๐ก๐๐ Kaplan Meier Estimator. Here the estimate for reliability is: ๐๐ ๐ ๐ ๏ฟฝ(๐ก๐ก๐๐ ) = ๏ฟฝ ๏ฟฝ1 โ ๏ฟฝ ๐๐ โ ๐๐ + 1 ๐ก๐ก๐๐ <๐ก๐ก๐๐ Where ๐๐ is the number of failures in rank j (for non-grouped data ๐๐ = 1). From this estimate a cdf can be given as ๐น๐น๏ฟฝ (๐ก๐ก๐๐ ) = 1 โ ๐ ๐ ๏ฟฝ (๐ก๐ก๐๐ ). For a detailed derivation and properties of this estimator see (Andersen et al. 1996, p.255) Probability plots are fast and not dependent on complex numerical methods and can be used without a detailed knowledge of statistics. It provides a visual representation of the data for which qualitative statements can be made. It can be useful in estimating initial values for numerical methods. Limitation of this technique is that it is not objective and two different people making the same plot will obtain different answers. It also does not provide confidence intervals. For more detail of probability plotting the reader is referred to (Nelson 1982, p.104) and (Meeker & Escobar 1998, p.122) 1.4.2. Total Time on Test Plots Total time on Test (TTT) plots is a graph which provides a visual representation of the hazard rate trend, i.e increasing, constant or decreasing. This assists in identifying the distribution from which the data may come from. To plot TTT (Rinne 2008, p.334): 1. Order the data such that ๐ฅ๐ฅ1 โค ๐ฅ๐ฅ2 โค โฏ โค ๐ฅ๐ฅ๐๐ โค โฏ โค ๐ฅ๐ฅ๐๐ . 2. Calculate the TTT positions: ๐๐ ๐๐๐๐๐๐๐๐ = ๏ฟฝ(๐๐ โ ๐๐ + 1)๏ฟฝ๐ฅ๐ฅ๐๐ โ ๐ฅ๐ฅ๐๐โ1 ๏ฟฝ ; ๐๐ = 1,2, โฆ , ๐๐ ๐๐=1 3. Calculate the normalized TTT positions: Parameter Est Introduction Parameter Est 24 Probability Distributions Used in Reliability Engineering ๐๐ 4. Plot the points ๏ฟฝ , ๐๐๐๐๐๐๐๐โ ๏ฟฝ. 5. Analyze graph: ๐๐ ๐๐๐๐๐๐๐๐โ = ๐๐๐๐๐๐๐๐ ; ๐๐ = 1,2, โฆ , ๐๐ ๐๐๐๐๐๐๐๐ 1 ๐๐๐๐๐๐๐๐ 0 0 ๐๐๏ฟฝ ๐๐ 1 Figure 5: Time on test plot interpretation Compared to probability plotting, TTT plots are simple, scale invariant and can represent any data set even those from different distributions on the same plot. However it only provides an indication of failure rate properties and cannot be used directly to estimate parameters. For more information about TTT plots the reader is referred to (Rinne 2008, p.334). 1.4.3. Least Mean Square Regression When the relationship between two variables, ๐ฅ๐ฅ and ๐ฆ๐ฆ is assumed linear (๐ฆ๐ฆ = ๐๐๐๐ + ๐๐), an estimate of the lineโs parameters can be obtained from ๐๐ sample data points, (๐ฅ๐ฅ๐๐ , ๐ฆ๐ฆ๐๐ ) using least mean square (LMS) regression. The least square method minimizes the square of the residual. ๐๐ ๐๐ = ๏ฟฝ ๐๐๐๐2 ๐๐=1 25 The residual can be defined in many ways. Minimize y residuals ๐๐๐๐ = ๐ฆ๐ฆ๐๐ โ ๐๐(๐ฅ๐ฅ๐๐ ; ๐๐, ๐๐) ๐๐ ๏ฟฝ= ๐ฆ๐ฆ๐๐ ๐๐(๐ฅ๐ฅ๐๐ ; ๐๐, ๐๐) ๐๐โ๐ฅ๐ฅ๐๐ ๐ฆ๐ฆ๐๐ โ (โ๐ฅ๐ฅ๐๐ )(โ๐ฆ๐ฆ๐๐ ) ๐๐โ๐ฅ๐ฅ๐๐2 ๐๐ฬ = ๐ฆ๐ฆ ๐๐๐๐ Minimize x residuals ๐๐๐๐ = ๐ฅ๐ฅ๐๐ โ ๐๐(๐ฆ๐ฆ๐๐ ; ๐๐, ๐๐) โ ๐๐ ๏ฟฝ= 2 ๏ฟฝโ๐ฅ๐ฅ๐๐2 ๏ฟฝ โ๐ฆ๐ฆ๐๐ โ๐ฅ๐ฅ๐๐ โ ๐๐ ๏ฟฝ ๐๐ ๐๐ ๐๐ฬ = ๐ฆ๐ฆ (๐ฅ๐ฅ๐๐ , ๐ฆ๐ฆ๐๐ ) 2 ๐๐โ๐ฆ๐ฆ๐๐2 โ ๏ฟฝโ๐ฆ๐ฆ๐๐2 ๏ฟฝ ๐๐โ๐ฅ๐ฅ๐๐ ๐ฆ๐ฆ๐๐ โ (โ๐ฅ๐ฅ๐๐ )(โ๐ฆ๐ฆ๐๐ ) โ๐ฆ๐ฆ๐๐ โ๐ฅ๐ฅ๐๐ โ ๐๐ ๏ฟฝ ๐๐ ๐๐ Regression line ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ (๐ฅ๐ฅ๐๐ , ๐ฆ๐ฆ๐๐ ) Regression line ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ 0 ๐ฅ๐ฅ 0 ๐๐(๐ฆ๐ฆ๐๐ ; ๐๐, ๐๐) ๐๐๐๐ ๐ฆ๐ฆ๐๐ ๐ฅ๐ฅ Figure 6: Left minimize y residual, right minimize x residual The LMS method can be used to estimate the line of best fit when using plotting parameter estimation methods. When plotting on a regular scale in software such as Microsoft Excel, it is often easy to conduct linear least mean square (LMS) regression using in built functions. Where available this book provides the formulas to plot the sample data in a straight line in a regular scale plot. It also provides the transformation from the linear LMS regression estimates of ๐๐ ๏ฟฝ and ๐๐ฬ to the distribution parameter estimates. For more on least square methods in a reliability engineering context see (Nelson 1990, p.167). MS regression can also be conducted on multivariate distributions, see (Rao & Toutenburg 1999) and can also be conducted on non-linear data directly, see (Bjõrck 1996). 1.4.4. Method of Moments To estimate the distribution parameters using the method of moments the sample moments are equated to the parameter moments and solved for the unknown parameters. The following sample moments can be used: The sample mean is given as: ๐ฅ๐ฅฬ = ๐๐ 1 ๏ฟฝ ๐ฅ๐ฅ๐๐ ๐๐ ๐๐=1 Parameter Est Introduction Parameter Est 26 Probability Distributions Used in Reliability Engineering The unbiased sample variance is given as: ๐๐ 1 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐ฅ๐ฅฬ )2 ๐๐ 2 = ๐๐ โ 1 ๐๐=1 Method of moments is not as accurate as Bayesian or maximum likelihood estimates but is easy and fast to calculate. The method of moment estimates are often used as a starting point for numerical methods to optimize maximum likelihood and least square estimators. 1.4.5. Maximum Likelihood Estimates Maximum likelihood estimates (MLE) are based on a frequentist approach to parameter estimation usually obtained by maximizing the natural log of the likelihood function. ฮ(ฮธ|E) = ln{๐ฟ๐ฟ(๐๐|๐ธ๐ธ)} Algebraically this is done by solving the first order partial derivatives of the log-likelihood function. This calculation has been included in this book for distributions where the result is in closed form. Otherwise the log-likelihood function can be maximized directly using numerical methods. MLE for ๐๐๏ฟฝ is obtained by solving for ๐๐: ๐๐ฮ =0 ๐๐๐๐ Denote the true parameters of the distribution as ๐ฝ๐ฝ๐๐ , MLEs have the following properties (Rinne 2008, p.406): โข Consistency. As the number of samples increases the difference between the estimated and actual parameter decreases: ๏ฟฝ = ๐ฝ๐ฝ plim ๐ฝ๐ฝ โข Asymptotic normality. ๐๐โโ lim ๐๐๏ฟฝ ~๐๐๐๐๐๐๐๐(๐๐0 , [๐ผ๐ผ๐๐ (๐๐0 )]โ1 ) ๐๐โโ โข โข where ๐ผ๐ผ๐๐ (๐๐) = ๐๐๐๐(๐๐) is the Fisher information matrix. Therefore ๐๐๏ฟฝ is asymptotically unbiased: lim ๐ธ๐ธ[๐๐๏ฟฝ] = ๐๐0 Asymptotic efficiency. ๐๐โโ lim ๐๐๐๐๐๐[๐๐๏ฟฝ] = [๐ผ๐ผ๐๐ (๐๐0 )]โ1 ๐๐โโ Invariance. The MLE of ๐๐(๐๐0 ) is ๐๐(๐๐๏ฟฝ ) if ๐๐(. ) is a continuous and continuously differentiable function. The advantages of MLE are that it is a very common technique that has been widely published and is implemented in many software packages. The MLE method can easily handle censored data. The disadvantage to MLE is the bias introduced for small sample sizes and unbounded estimates may result when no failures have been observed. The 27 numerical optimization of the log-likelihood function may be non-trivial with high sensitivity to starting values and the presence of local maximums. For more information in a reliability context see (Nelson 1990, p.284). 1.4.6. Bayesian Estimation Bayesian estimation uses a subjective interpretation of the theory of probability and for parameter point estimation and confidence intervals uses Bayesโ rule to update our state of knowledge of the unknown of interest (UIO). Recall from section 1.1.5 Bayes rule, ๐๐(๐๐|๐ธ๐ธ) = ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) โซ ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ , ๐๐(๐๐|๐ธ๐ธ) = ๐๐(๐๐) ๐๐(๐ธ๐ธ|๐๐) โ๐๐ ๐๐(๐ธ๐ธ|๐๐๐๐ )๐๐(๐๐) respectively for continuous and discrete forms of variable of ๐๐. The Prior Distribution ๐๐๐๐ (๐๐) The prior distribution is probability distribution of the UOI, ๐๐, which captures our state of knowledge of ๐๐ prior to the evidence being observed. It is common for this distribution to represent soft evidence or intervals about the possible values of ๐๐. If the distribution is dispersed it represents little being known about the parameter. If the distribution is concentrated in an area then it reflects a good knowledge about the likely values of ๐๐. Prior distributions should be a proper probability distribution of ๐๐. A distribution is proper when it integrates to one and improper otherwise. The prior should also not be selected based on the form of the likelihood function. When the prior has a constant which does not affect the posterior distribution (such as improper priors) it will be omitted from the formulas within this book. Non-informative Priors. Occasions arise when it is not possible to express a subjective prior distribution due to lack of information, time or cost. Alternatively a subjective prior distribution may introduce unwanted bias through model convenience (conjugates) or due to elicitation methods. In such cases a non-informative prior may be desirable. The following methods exist for creating a non-informative prior (Yang and Berger 1998): โข Principle of Indifference - Improper Uniform Priors. An equal probability is assigned across all the possible values of the parameter. This is done using an improper uniform distribution with a constant, usually 1, over the range of the possible values for ๐๐. When placed in Bayes formula the constant cancels out, however the denominator is integrated over all possible values of ๐๐. In most cases this prior distribution will result in a proper posterior, but not always. Improper Uniform Priors may be chosen to enable the use of conjugate priors. For example using exponential likelihood model, with an improper uniform prior, 1, over the limits [0, โ) with evidence of ๐๐๐น๐น failures in total time, ๐ก๐ก ๐๐ : Prior: ๐๐0 (๐๐) = 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0) Likelihood: ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) = ๐๐nF ๐๐ โ๐๐๐ก๐ก๐๐ Parameter Est Introduction Parameter Est 28 Probability Distributions Used in Reliability Engineering Posterior: ๐๐(๐๐|๐ธ๐ธ) = 1. ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) โ 1. โซ0 ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ Using conjugate relationship (see Conjugate Priors for calculations): โข ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + nF , tT ) Principle of Indifference - Proper Uniform Priors. An equal probability is assigned across the values of the parameter within a range defined by the uniform distribution. The uniform distribution is obtained by estimating the far 1 left and right bounds (๐๐ and ๐๐) of the parameter ๐๐ giving ๐๐๐๐ (๐๐) = = ๐๐, where ๐๐โ๐๐ c is a constant. When placed in Bayes formula the constant cancels out, however the denominator is integrated over the bound [๐๐, ๐๐]. Care needs to be taken in choosing ๐๐ and ๐๐ because no matter how much evidence suggests otherwise the posterior distribution will always be zero outside these bounds. Using an exponential likelihood model, with a proper uniform prior, ๐๐, over the limits [๐๐, ๐๐] with evidence of ๐๐๐น๐น failures in total time, ๐ก๐ก ๐๐ : Prior: ๐๐0 (๐๐) = 1 ๐๐โ๐๐ Likelihood: Posterior: ๐๐(๐๐|๐ธ๐ธ) = = ๐๐ โ ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) = ๐๐nF ๐๐ โ๐๐๐ก๐ก๐๐ ๐๐. ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐ ๐๐. โซ๐๐ ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ for a โค ฮป โค b Using conjugate relationship this results in a truncated Gamma distribution: ๐๐(๐๐) = ๏ฟฝ ๐๐. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + nF , tT ) for a โค ฮป โค b 0 otherwise โข Jeffreyโs Prior. Proposed by Jeffery in 1961, this prior is defined as ๐๐0 (๐๐) = ๏ฟฝ๐๐๐๐๐๐ (๐ฐ๐ฐ๐๐ ) where ๐ฐ๐ฐ๐๐ is the Fisher information matrix. This derivation is motivated by the fact that it is not dependent upon the set of parameter variables that is chosen to describe parameter space. Jeffery himself suggested the need to make ad hoc modifications to the prior to avoid problems in multidimensional distributions. Jefforyโs prior is normally improper. (Bernardo et al. 1992) โข Reference Prior. Proposed by Bernardo in 1979, this prior maximizes the expected posterior information from the data, therefore reducing the effect of the prior. When there is no nuisance parameters and certain regularity conditions are satisfied the reference prior is identical to the Jeffreyโs prior. Due to the need to order or group the importance of parameters, it may occur that different posteriors will result from the same data depending on the importance the user places on each parameter. This prior overcomes the problems which arise when using Jefferyโs prior in multivariate applications. โข 29 Maximal Data Information Prior (MDIP). Developed by Zelluer in 1971 maximizes the likelihood function with relation to the prior. (Berry et al. 1995, p.182) For further detail on the differences between each type of non-informative prior see (Berry et al. 1995, p.179) Conjugate Priors. Calculating posterior distributions can be extremely complex and in most cases requires expensive computations. A special case exists however by which the posterior distribution is of the same form as the prior distribution. The Bayesian updating mathematics can be reduced to simple calculations to update the model parameters. As an example the gamma function is a conjugate prior to a Poisson likelihood function: Prior: ๐๐๐๐ (๐๐) = ๐ฝ๐ฝ๐ผ๐ผ ๐๐๐ผ๐ผโ1 โฮฒฮป ๐๐ ฮ(๐ผ๐ผ) Likelihood: ๐ฟ๐ฟ๐๐ (๐ก๐ก๐๐ |๐๐) = ๐๐๐น๐น ๐๐๐๐๐๐ ๐ก๐ก๐๐๐๐ โ๐๐๐ก๐ก ๐๐ ๐๐ ๐๐๐๐ ! Likelihood: ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) = ๏ฟฝ ๐ฟ๐ฟ๐๐ (๐ก๐ก๐๐ |๐๐) = ๐๐=1 Posterior: ๐๐(๐๐|๐ธ๐ธ) = = ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) โ โซ0 ๐๐๐๐ (๐๐) ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ ๏ฟฝ โ 0 โ = ๐๐โ๐๐ โ๐ก๐ก๐๐๐๐ โ๐๐โ๐ก๐ก ๐๐ ๐๐ โ๐๐๐๐ ! ๐ฝ๐ฝ๐ผ๐ผ ๐๐๐ผ๐ผโ1 ๐๐โ๐๐ โ๐ก๐ก๐๐๐๐ โฮฒฮป โ๐๐โ๐ก๐ก๐๐ ๐๐ ๐๐ ฮ(๐ผ๐ผ)โ๐๐๐๐ ! ๐ฝ๐ฝ๐ผ๐ผ ๐๐๐ผ๐ผโ1 ๐๐โ๐๐ โ๐ก๐ก๐๐๐๐ โฮฒฮป โ๐๐โ๐ก๐ก ๐๐ ๐๐๐๐ ๐๐ ๐๐ ฮ(๐ผ๐ผ)โ๐๐๐๐ ! ๐๐๐ผ๐ผโ1+โ๐๐ ๐๐ โฮป(ฮฒ+โ๐ก๐ก๐๐) โ โซ0 ๐๐๐ผ๐ผโ1+โ๐๐ ๐๐ โฮป(ฮฒ+โ๐ก๐ก๐๐) ๐๐๐๐ Using the identity ฮ(๐ง๐ง) = โซ๐๐ ๐ฅ๐ฅ ๐ง๐งโ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐ we can calculate the denominator using the change of variable ๐ข๐ข = ๐๐(๐ฝ๐ฝ + โ๐ก๐ก๐๐ ). This results in ๐๐ = ๐ข๐ข ๐ฝ๐ฝ+โ๐ก๐ก๐๐ , and ๐๐๐๐ = ๐๐๐๐ ๐ฝ๐ฝ+โ๐ก๐ก๐๐ of ๐ข๐ข the same as ๐๐ . Substituting back into the posterior equation gives: ๐๐(๐๐|๐ธ๐ธ) = ๐๐๐ผ๐ผโ1+โ๐๐ ๐๐ โฮป(ฮฒ+โ๐ก๐ก๐๐ ) โ ๐ผ๐ผโ1+โ๐๐ ๐ข๐ข 1 ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐ โu ๐๐๐๐ ๐ฝ๐ฝ + โ๐ก๐ก๐๐ ๐ฝ๐ฝ + โ๐ก๐ก๐๐ 0 Let ๐ง๐ง = ๐ผ๐ผ + โ๐๐ = ๐๐๐ผ๐ผโ1+โ๐๐ ๐๐ โฮป(ฮฒ+โ๐ก๐ก๐๐ ) โ 1 โซ ๐ข๐ข๐ผ๐ผโ1+โ๐๐ ๐๐ โu ๐๐๐๐ (๐ฝ๐ฝ + โ๐ก๐ก๐๐ )๐ผ๐ผ+โ๐๐ 0 with the limits Parameter Est Introduction Parameter Est 30 Probability Distributions Used in Reliability Engineering ๐๐(๐๐|๐ธ๐ธ) = โ Using ฮ(๐ง๐ง) = โซ๐๐ ๐ฅ๐ฅ ๐ง๐งโ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐: ๐๐(๐๐|๐ธ๐ธ) = Let ๐ผ๐ผ โฒ = ๐ผ๐ผ + โ๐๐, ๐ฝ๐ฝโฒ = ๐ฝ๐ฝ + โ๐ก๐ก๐๐ : ๐๐๐ผ๐ผโ1+โ๐๐ ๐๐ โฮป(ฮฒ+โ๐ก๐ก๐๐ ) โ 1 โซ ๐ข๐ข ๐ง๐งโ1 ๐๐ โu ๐๐๐๐ (๐ฝ๐ฝ + โ๐ก๐ก๐๐ )๐ผ๐ผ+โ๐๐ 0 ๐๐๐ผ๐ผโ1+โ๐๐ (๐ฝ๐ฝ + โ๐ก๐ก๐๐ )๐ผ๐ผ+โ๐๐ โฮป(ฮฒ+โ๐ก๐ก ) ๐๐ ๐๐ ฮ(๐ผ๐ผ + โ๐๐) โฒ ๐ผ๐ผ โฒ ๐๐๐ผ๐ผ โ1 ๐ฝ๐ฝโฒ ๐๐(๐๐|๐ธ๐ธ) = ฮ(๐ผ๐ผ โฒ ) โฒ ๐๐ โฮฒ ฮป As can be seen the posterior is a gamma distribution with the parameters ๐ผ๐ผ โฒ = ๐ผ๐ผ + โ๐๐, ๐ฝ๐ฝโฒ = ๐ฝ๐ฝ + โ๐ก๐ก๐๐ . Therefore the prior and posterior are of the same form, and Bayesโ rule does not need to be re-calculated for each update. Instead the user can simply update the parameters with the new evidence. The Likelihood Function ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) The reader is referred to section 1.1.6 for a discussion on the construction of the likelihood function. The Posterior Distribution ๐๐(๐๐|๐ธ๐ธ) The posterior distribution is a probability distribution of the UOI, ๐๐, which captures our state of knowledge of ๐๐ including all prior information and the evidence. Point Estimate. From the posterior distribution we may want to give a point estimate of ฮธ. The Bayesian estimator when using a quadratic loss function is the posterior mean (Christensen & Huffman 1985): ๐๐๏ฟฝ = ๐ธ๐ธ[๐๐(๐๐|๐ธ๐ธ)] = โซ ๐๐๐๐(๐๐|๐ธ๐ธ) ๐๐๐๐ = ๐๐๐๐ For more information on utility, loss functions and estimators in a Bayesian context see (Berger 1993). 1.4.7. Confidence Intervals Assuming a random variable is distributed by a given distribution, there exists the true ๏ฟฝ , may or distribution parameters, ๐ฝ๐ฝ๐๐ , which is unknown. The parameter point estimates, ๐ฝ๐ฝ may not be close to the true parameter values. Confidence intervals provide the range over which the true parameter values may exist with a certain level of confidence. Confidence intervals only quantify uncertainty due to sampling error arising from a limited number of samples. Uncertainty due to incorrect model selection or incorrect assumptions is not included. (Meeker & Escobar 1998, p.49) Increasing the desired confidence ๐พ๐พ results in an increased confidence interval. Increasing the sample size generally decreases the confidence interval. There are many methods to calculate confidence intervals. Some popular methods are: 31 โข Exact Confidence Intervals. It may be mathematically shown that the parameter of a distribution itself follows a distribution. In such cases exact confidence intervals can be derived. This is only the case in very few distributions. โข Fisher Information Matrix (Nelson 1990, p.292). For a large number of samples, the asymptotic normal property can be used to estimate confidence intervals: lim ๐๐๏ฟฝ ~๐๐๐๐๐๐๐๐(๐๐0 , [๐๐๐๐(๐๐0 )]โ1 ) ๐๐โโ Combining this with the asymptotic property ๐๐๏ฟฝ โ ๐๐0 as ๐๐ โ โ gives the following estimate for the distribution of ๐๐๏ฟฝ: โ1 lim ๐๐๏ฟฝ ~๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๏ฟฝ, ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ ๐๐โโ 100๐พ๐พ% approximate confidence intervals are calculated using percentiles of the normal distribution. If the range of ๐๐ is unbounded (โโ, โ) the approximate two sided confidence intervals are: 1 + ๐พ๐พ โ1 ๐๐๐พ๐พ = ๐๐๏ฟฝ โ ฮฆโ1 ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝ 2 1 + ๐พ๐พ โ1 ๏ฟฝ๏ฟฝ๏ฟฝ๐พ๐พ = ๐๐๏ฟฝ + ฮฆโ1 ๏ฟฝ ๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝ 2 If the range of ๐๐ is (0, โ) the approximate two sided confidence intervals are: โก๐ท๐ท โ1 ๏ฟฝ1 + ๐พ๐พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝโ1 โค 2 โฅ ๐๐๐พ๐พ = ๐๐๏ฟฝ. exp โข โ๐๐๏ฟฝ โข โฅ โฃ โฆ โก๐ท๐ท โ1 ๏ฟฝ1 + ๐พ๐พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝโ1 โค 2 ๏ฟฝ๏ฟฝ๏ฟฝ๐พ๐พ = ๐๐๏ฟฝ. exp โข โฅ ๐๐ ๐๐๏ฟฝ โข โฅ โฃ โฆ If the range of ๐๐ is (0,1) the approximate two sided confidence intervals are: โ1 โก๐ท๐ท โ1 ๏ฟฝ1 + ๐พ๐พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝโ1 โคโซ โง 2 โฅ ๐๐๐พ๐พ = ๐๐๏ฟฝ. ๐๐๏ฟฝ + (1 โ ๐๐๏ฟฝ ) exp โข ๐๐๏ฟฝ(1 โ ๐๐๏ฟฝ ) โจ โข โฅโฌ โฉ โฃ โฆโญ โ1 โ1 1 + ๐พ๐พ โ1 โก๐ท๐ท ๏ฟฝ โง ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๏ฟฝ๏ฟฝ โคโซ 2 ๏ฟฝ๏ฟฝ๏ฟฝ๐พ๐พ = ๐๐๏ฟฝ. ๐๐๏ฟฝ + (1 โ ๐๐๏ฟฝ ) exp โข โฅ ๐๐ โ๐๐๏ฟฝ(1 โ ๐๐๏ฟฝ ) โจ โข โฅโฌ โฉ โฃ โฆโญ The advantage of this method is it can be calculated for all distributions and is easy to calculate. The disadvantage is that the assumption of a normal distribution is asymptotic and so sufficient data is required for the confidence interval estimate to be accurate. The number of samples needed for an accurate estimate changes from distribution to distribution. It also produces symmetrical confidence intervals which may be very inaccurate. For more information see (Nelson 1990, p.292). Parameter Est Introduction Parameter Est 32 Probability Distributions Used in Reliability Engineering โข Likelihood Ratio Intervals (Nelson 1990, p.292). The test statistic for the likelihood ratio is: ๐ท๐ท = 2[ฮ๏ฟฝ๐๐๏ฟฝ๏ฟฝ โ ฮ(๐๐)] ๐ท๐ท is approximately Chi-Square distributed with one degree of freedom. ๐ท๐ท = 2๏ฟฝฮ๏ฟฝ๐๐๏ฟฝ๏ฟฝ โ ฮ(๐๐)๏ฟฝ โค ๐๐ 2 (๐พ๐พ; 1) Where ๐พ๐พ is the 100๐พ๐พ% confidence interval for ๐๐. The two sided confidence limits ๐๐๐พ๐พ and ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐๐พ๐พ are calculated by solving: ฮ(๐๐) = ฮ๏ฟฝ๐๐๏ฟฝ๏ฟฝ โ ๐๐ 2 (๐พ๐พ; 1) 2 The limits are normally solved numerically. The likelihood ratio intervals are always within the limits of the parameter and gives asymmetrical confidence limits. It is much more accurate than the Fisher information matrix method particularly for one sided limits although it is more complicated to calculate. This method must be solved numerically and so will not be discussed further in this book. โข Bayesian Confidence Intervals. In Bayesian statistics the uncertainty of a parameter, ๐๐, is quantified as a distribution ๐๐(๐๐). Therefore the two sided 100๐พ๐พ% confidence intervals are found by solving: ๐๐๐พ๐พ 1 โ ๐พ๐พ = ๏ฟฝ ๐๐(๐๐) ๐๐๐๐ , 2 โโ โ 1 + ๐พ๐พ = ๏ฟฝ ๐๐(๐๐) ๐๐๐๐ 2 ๐๐๐พ๐พ Other methods exist to calculate approximate confidence intervals. A summary of some techniques used in reliability engineering is included in (Lawless 2002). Introduction ๐๐๐ต๐ต = ๐๐๐๐ = ln(๐๐๐ฟ๐ฟ ) 1 XC = XN(k) โ µ 2 ๏ฟฝ๏ฟฝ ๏ฟฝ ฯ ๐๐๐บ๐บ1 ๐๐๐บ๐บ1 + ๐๐๐บ๐บ2 ๐ผ๐ผ = ๐๐1 , ๐ฝ๐ฝ = ๐๐2 ๐๐ = 1 ๐๐๐ต๐ต = Binomial ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐, ๐๐) lim ๐๐ โ โ ๐๐๐๐ = ๐๐ V Chi-square ๐๐ 2(๐ฃ๐ฃ) ๐๐๐น๐น = Poisson ๐๐๐๐๐๐๐๐(๐๐) Gamma ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) Student t ๐ก๐ก(๐ฃ๐ฃ) ๐ฃ๐ฃ = 2, ๐๐ ๐๐๐ธ๐ธ = ln(๐๐๐๐ โ๐๐) Pareto ๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ 1๏ฟฝ ๐ฝ๐ฝ ๐ฝ๐ฝ = 2 ๐๐ = โ2/๐ผ๐ผ Exponential ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐ฝ๐ฝ = 1 Rayleigh ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐๐) ๐๐๐ ๐ = ๏ฟฝ๐๐๐ธ๐ธ ๐๐๐๐ = ๐๐๐ธ๐ธ ๐ฃ๐ฃ = 2, ๐ผ๐ผ = โ2, ๐ฝ๐ฝ = 2 ๐๐๐น๐น = ๐๐๐๐2 ๐๐1 = 1, ๐๐2 = ๐ฃ๐ฃ ๐ผ๐ผ = 1, ๐ฝ๐ฝ = 1 ๐๐ = 1 ๐๐๐บ๐บ = ๐๐๐ธ๐ธ1 + โฏ + ๐๐๐ธ๐ธ๐ธ๐ธ ๐๐๐ถ๐ถ1 ๐๐2 ๐๐๐ถ๐ถ2 ๐๐1 Xฯ2 = ๐๐๐๐2 ๐๐๐๐ (๐๐; ๐๐๐๐) = ๐น๐น๐บ๐บ (๐ก๐ก; ๐๐ + 1, ๐๐) โ๐น๐น๐บ๐บ (๐ก๐ก; ๐๐, ๐๐) F ๐น๐น(๐๐1 , ๐๐2 ) Chi ๐๐(๐ฃ๐ฃ) Beta ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐๐๐(1) โค โฏ โค ๐๐๐๐(๐๐) , ๐๐๐๐(๐๐) ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๐๐๐บ๐บ = ๏ฟฝ ๐๐๐ต๐ต๐ต๐ต ๐๐ (๐๐๐๐๐๐) 1 ๐ผ๐ผ = 2๐ฃ๐ฃ1 , ๐ฝ๐ฝ = 2๐ฃ๐ฃ2 Bernoulli ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐) Normal ๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 ) XC = |๐ฟ๐ฟ๐ต๐ต | ๐๐๐ถ๐ถ1 , ๐๐๐ถ๐ถ1 + ๐๐๐ถ๐ถ2 Special Case Weibull ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐๐๐ = exp(โ๐๐๐๐๐ธ๐ธ ) Standard Uniform ๐๐๐๐๐๐๐๐(0,1) ๐๐ = 0 ๐๐ = 1 Uniform ๐๐๐๐๐๐๐๐(๐๐, ๐๐) Figure 7: Relationships between common distributions (Leemis & McQueston 2008). Many relations are not included such as central limit convergence to the normal distribution and many transforms which would have made the figure unreadable. For further details refer to individual sections and (Leemis & McQueston 2008). Related Dist Transformation 1.5. Related Distributions Lognormal ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐๐๐๐ , ๐๐๐๐2 ) 33 Supporting Func 34 Probability Distributions Used in Reliability Engineering 1.6. Supporting Functions 1.6.1. Beta Function ๐๐(๐๐, ๐๐) B(๐ฅ๐ฅ, ๐ฆ๐ฆ) is the Beta function and is the Euler integral of the first kind. Where ๐ฅ๐ฅ > 0 and ๐ฆ๐ฆ > 0. 1 ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๏ฟฝ ๐ข๐ข ๐ฅ๐ฅโ1 (1 โ ๐ข๐ข)๐ฆ๐ฆโ1 ๐๐๐๐ 0 Relationships: ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๐ต๐ต(๐ฆ๐ฆ, ๐ฅ๐ฅ) ฮ(๐ฅ๐ฅ)ฮ(๐ฆ๐ฆ) ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) = ฮ(๐ฅ๐ฅ + ๐ฆ๐ฆ) โ ๐๐ โ ๐ฆ๐ฆ ๏ฟฝ ๏ฟฝ ๐๐ ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๏ฟฝ ๐ฅ๐ฅ + ๐๐ ๐๐=0 More formulas, definitions and special values can be found in the Digital Library of Mathematical Functions on the National Institute of Standards and Technology (NIST) website, http://dlmf.nist.gov. 1.6.2. Incomplete Beta Function ๐ฉ๐ฉ๐๐ (๐๐; ๐๐, ๐๐) ๐ต๐ต๐ก๐ก (๐ก๐ก; ๐ฅ๐ฅ, ๐ฆ๐ฆ) is the incomplete Beta function expressed by: ๐ก๐ก ๐ต๐ต๐ก๐ก (๐ก๐ก; ๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๏ฟฝ ๐ข๐ข ๐ฅ๐ฅโ1 (1 โ ๐ข๐ข)๐ฆ๐ฆโ1 ๐๐๐๐ 0 1.6.3. Regularized Incomplete Beta Function ๐ฐ๐ฐ๐๐ (๐๐; ๐๐, ๐๐) ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ฅ๐ฅ, ๐ฆ๐ฆ) is the regularized incomplete Beta function: ๐ต๐ต๐ก๐ก (t ; ๐ฅ๐ฅ, ๐ฆ๐ฆ) ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ฅ๐ฅ, ๐ฆ๐ฆ) = ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) ๐ฅ๐ฅ+๐ฆ๐ฆโ1 Properties: = ๏ฟฝ ๐๐=๐ฅ๐ฅ (๐ฅ๐ฅ + ๐ฆ๐ฆ โ 1)! . ๐ก๐ก ๐๐ (1 โ ๐ก๐ก)๐ฅ๐ฅ+๐ฆ๐ฆโ1โ๐๐ ๐๐! (๐ฅ๐ฅ + ๐ฆ๐ฆ โ 1 โ ๐๐)! ๐ผ๐ผ0 (0; ๐ฅ๐ฅ, ๐ฆ๐ฆ) = 0 ๐ผ๐ผ1 (1; ๐ฅ๐ฅ, ๐ฆ๐ฆ) = 1 ๐ผ๐ผ๐ก๐ก (t; ๐ฅ๐ฅ, ๐ฆ๐ฆ) = 1 โ ๐ผ๐ผ(1 โ t; ๐ฆ๐ฆ, ๐ฅ๐ฅ) 1.6.4. Complete Gamma Function ๐ช๐ช(๐๐) ฮ(๐๐) is a generalization of the factorial function ๐๐! to include non-integer values. For ๐๐ > 0 35 โ ฮ(๐๐) = ๏ฟฝ ๐ก๐ก ๐๐โ1 ๐๐ โ๐ก๐ก ๐๐๐๐ 0 โ โ = ๏ฟฝ ๐ก๐ก ๐๐โ1 ๐๐ โ๐ก๐ก ๏ฟฝ + (๐๐ โ 1) ๏ฟฝ ๐ก๐ก ๐๐โ2 ๐๐ โ๐ก๐ก ๐๐๐๐ 0 โ = (๐๐ โ 1) ๏ฟฝ ๐ก๐ก ๐๐โ2 ๐๐ โ๐ก๐ก ๐๐๐๐ 0 0 When ๐๐ is an integer: Special values: = (๐๐ โ 1)ฮ(๐๐ โ 1) ฮ(๐๐) = (๐๐ โ 1)! ฮ(1) = 1 ฮ(2) = 1 1 ฮ ๏ฟฝ ๏ฟฝ = โ๐๐ 2 Relation to the incomplete gamma functions: ฮ(๐๐) = ฮ(๐๐, ๐ก๐ก) + ๐พ๐พ(๐๐, ๐ก๐ก) More formulas, definitions and special values can be found in the Digital Library of Mathematical Functions on the National Institute of Standards and Technology (NIST) website, http://dlmf.nist.gov. 1.6.5. Upper Incomplete Gamma Function ๐ช๐ช(๐๐, ๐๐) For ๐๐ > 0 When ๐๐ is an integer: โ ฮ(๐๐, ๐ก๐ก) = ๏ฟฝ ๐ฅ๐ฅ ๐๐โ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐ ๐ก๐ก ฮ(๐๐, ๐ก๐ก) = (๐๐ โ 1)! ๐๐ โ๐ก๐ก ๐๐โ1 ๐๐ ๐ก๐ก ๏ฟฝ ๐๐=0 ๐๐! More formulas, definitions and special values can be found on the NIST website, http://dlmf.nist.gov. 1.6.6. Lower Incomplete Gamma Function ๐๐(๐๐, ๐๐) For ๐๐ > 0 When ๐๐ is an integer: ๐ก๐ก ฮณ(๐๐, ๐ก๐ก) = ๏ฟฝ ๐ฅ๐ฅ ๐๐โ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐ 0 ๐๐โ1 ๐๐ ๐ก๐ก ฮณ(๐๐, ๐ก๐ก) = (๐๐ โ 1)! ๏ฟฝ1 โ ๐๐ โ๐ก๐ก ๏ฟฝ ๐๐=0 ๐๐! ๏ฟฝ Supporting Func Introduction Supporting Func 36 Probability Distributions Used in Reliability Engineering More formulas, definitions and special values can be found on the NIST website, http://dlmf.nist.gov. 1.6.7. Digamma Function ๐๐(๐๐) ๐๐(๐ฅ๐ฅ) is the digamma function defined as: ฮ โฒ (๐ฅ๐ฅ) ๐๐ ๐๐(๐ฅ๐ฅ) = ln[ฮ(๐ฅ๐ฅ)] = ๐๐๐๐๐๐ ๐ฅ๐ฅ > 0 ฮ(๐ฅ๐ฅ) ๐๐๐๐ 1.6.8. Trigamma Function ๐๐โฒ(๐๐) ๐๐ โฒ (๐ฅ๐ฅ) is the trigamma function defined as: โ ๐๐2 ๐๐ โฒ (๐ฅ๐ฅ) = 2 ๐๐๐๐ฮ(๐ฅ๐ฅ) = ๏ฟฝ(๐ฅ๐ฅ + ๐๐)โ2 ๐๐๐ฅ๐ฅ ๐๐=0 1.7. Referred Distributions 1.7.1. Inverse Gamma Distribution ๐ฐ๐ฐ๐ฐ๐ฐ(๐ถ๐ถ, ๐ท๐ท) The pdf to the inverse gamma distribution is: โ๐ฝ๐ฝ ๐ฝ๐ฝ๐ผ๐ผ . ๐๐ ๐ฅ๐ฅ . ๐ผ๐ผ๐ฅ๐ฅ (0, โ) ๐๐(๐ฅ๐ฅ; ๐ผ๐ผ, ๐ฝ๐ฝ) = ฮ(๐ผ๐ผ)๐ฅ๐ฅ ๐ผ๐ผ+1 With mean: ๐ฝ๐ฝ ๐๐ = for ๐ผ๐ผ > 1 ๐ผ๐ผ โ 1 1.7.2. Student T Distribution ๐ป๐ป(๐ถ๐ถ, ๐๐, ๐๐๐๐ ) The pdf to the standard student t distribution with ๐๐ = 0, ๐๐ 2 = 1 is: ๐ผ๐ผ+1 โ 2 ๐ฅ๐ฅ 2 ฮ[(๐ผ๐ผ + 1)โ2] . ๏ฟฝ1 + ๏ฟฝ ๐๐(๐ฅ๐ฅ; ๐ผ๐ผ) = ๐ผ๐ผ โ๐ผ๐ผ๐ผ๐ผฮ(๐ผ๐ผโ2) The generalized student t distribution is: With mean ๐ผ๐ผ+1 โ 2 (๐ฅ๐ฅ โ ๐๐)2 ฮ[(๐ผ๐ผ + 1)โ2] ๐๐(๐ฅ๐ฅ; ๐ผ๐ผ, ๐๐, ๐๐ 2 ) = . ๏ฟฝ1 + ๏ฟฝ ๐ผ๐ผ๐๐ 2 ๐๐โ๐ผ๐ผ๐ผ๐ผฮ(๐ผ๐ผโ2) 1.7.3. F Distribution ๐ญ๐ญ(๐๐๐๐ , ๐๐๐๐ ) ๐๐ = ๐๐ Also known as the Variance Ratio or Fisher-Snedecor distribution the pdf is: With cdf: ๐๐(๐ฅ๐ฅ; ๐ผ๐ผ) = ๐ผ๐ผ๐ก๐ก ๏ฟฝ (๐๐1 ๐ฅ๐ฅ)๐๐1 . ๐๐2๐๐2 1 ๏ฟฝ . {๐๐ +๐๐ } ๐๐ ๐๐ ๐ฅ๐ฅ๐ฅ๐ฅ ๏ฟฝ 1 , 2 ๏ฟฝ (๐๐1 ๐ฅ๐ฅ + ๐๐2 ) 1 2 2 2 ๐๐1 ๐๐2 , ๏ฟฝ, 2 2 ๐ค๐คโ๐๐๐๐๐๐ ๐ก๐ก = 1.7.4. Chi-Square Distribution ๐๐๐๐ (๐๐) ๐๐1 ๐ฅ๐ฅ ๐๐1 ๐ฅ๐ฅ + ๐๐2 The pdf to the chi-square distribution is: With mean: ๐ฅ๐ฅ ๐ฅ๐ฅ (๐ฃ๐ฃโ2)โ2 ๐๐๐๐๐๐ ๏ฟฝโ ๏ฟฝ 2 ๐๐(๐ฅ๐ฅ; ๐ฃ๐ฃ) = ๐ฃ๐ฃ 2๐ฃ๐ฃโ2 ๐ค๐ค ๏ฟฝ ๏ฟฝ 2 ๐๐ = ๐ฃ๐ฃ 37 Referred Dist Introduction Referred Dist 38 Probability Distributions Used in Reliability Engineering 1.7.5. Hypergeometric Distribution ๐ฏ๐ฏ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐๐; ๐๐, ๐๐, ๐ต๐ต) The hypergeometric distribution models probability of ๐๐ successes in ๐๐ Bernoulli trials from population ๐๐ containing ๐๐ success without replacement. ๐๐ = ๐๐/๐๐. The pdf to the hypergeometric distribution is: With mean: ๐๐(๐๐; ๐๐, ๐๐, ๐๐) = ๐๐ = ๏ฟฝm ๏ฟฝ๏ฟฝNโm ๏ฟฝ k nโk ๐๐๐๐ ๐๐ ๏ฟฝN ๏ฟฝ n 1.7.6. Wishart Distribution ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐พ๐๐๐ ๐ (๐๐; ๐บ๐บ, ๐๐) The Wishart distribution is the multivariate generalization of the gamma distribution. The pdf is given as: With mean: ๐๐๐๐ (๐๐; ๐บ๐บ, ๐๐) = 1 |๐ฑ๐ฑ|2(nโdโ1) 2๐๐๐๐ โ2 |๐บ๐บ|nโ2 ฮ๐๐ 1 โ๐๐ ๐๐ exp ๏ฟฝโ 2 ๐ก๐ก๐ก๐ก๏ฟฝ๐๐ ๐บ๐บ๏ฟฝ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐๐ = ๐๐๐บ๐บ 39 1.8. Nomenclature and Notation Functions are presented in the following form: ๐๐(๐๐๐๐๐๐๐๐๐๐๐๐ ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ ; ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ |๐๐๐๐๐๐๐๐๐๐ ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ๐ฃ) ๐๐ In continuous distributions the number of items under test = ๐๐๐๐ + ๐๐๐ ๐ + ๐๐๐ผ๐ผ . In discrete distributions the total number of trials. ๐๐๐น๐น The number of items which failed before the conclusion of the test. ๐๐๐ผ๐ผ The number of items which have interval data ๐ก๐ก๐๐๐๐ The time at which a component survived to. The item may have been removed from the test for a reason other than failure. ๐๐๐๐ The number of items which survived to the end of the test. ๐ก๐ก๐๐๐น๐น , ๐ก๐ก๐๐ The time at which a component fails. ๐ก๐ก๐๐๐๐๐๐ The upper limit of a censored interval in which an item failed ๐ก๐ก๐ฟ๐ฟ The lower truncated limit of sample. ๐ก๐ก ๐๐ Time on test = โ๐ก๐ก๐๐ + โ๐ก๐ก๐ ๐ ๐พ๐พ Discrete random variable ๐๐ A discrete random variable with a known value ๐๐ A bold symbol denotes a vector or matrix ๐๐ Upper confidence interval for UOI ๐๐~๐๐๐๐๐๐๐๐๐๐ The random variable ๐๐ is distributed as a ๐๐-variate normal distribution. ๐ก๐ก๐๐๐ฟ๐ฟ๐ฟ๐ฟ The lower limit of a censored interval in which an item failed ๐ก๐ก๐๐ The upper truncated limit of sample. ๐๐ or ๐๐ Continuous random variable (T is normally a random time) ๐ฅ๐ฅ or ๐ก๐ก A continuous random variable with a known value ๐ฅ๐ฅ๏ฟฝ The hat denotes an estimated value ๐๐ Generalized unknown of interest (UOI) ๐๐ Lower confidence interval for UOI Nomenclature Introduction Life Distribution 40 Probability Distributions Used in Reliability Engineering 2. Common Life Distributions Exponential Continuous Distribution 41 2.1. Exponential Continuous Distribution Exponential Probability Density Function - f(t) ฮป=0.5 0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00 ฮป=1 ฮป=2 ฮป=3 0 0.5 1 1.5 2 2.5 3 3.5 4 Cumulative Density Function - F(t) 1.00 0.80 ฮป=0.5 0.60 ฮป=1 0.40 ฮป=2 0.20 ฮป=3 0.00 0 0.5 1 1.5 2 2.5 3 3.5 4 1 1.5 2 2.5 3 3.5 4 Hazard Rate - h(t) 3.00 2.50 2.00 1.50 1.00 0.50 0.00 0 0.5 42 Common Life Distributions Parameters & Description Parameters ๐๐ Exponential Limits Function Time Domain Scale Parameter: Equal to the hazard rate. ๐ก๐ก โฅ 0 ๐๐(๐ก๐ก) = ๐๐eโฮปt PDF ๐น๐น(๐ก๐ก) = 1 โ eโฮปt CDF R(t) = eโฮปt Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life ๐๐ > 0 Residual ๐๐(๐ฅ๐ฅ) = eโฮปx ๐๐(๐ ๐ ) = ๐๐ , ๐๐ + ๐ ๐ ๐น๐น(๐ ๐ ) = ๐ ๐ > โ๐๐ ๐๐ ๐ ๐ (๐๐ + ๐ ๐ ) ๐ ๐ (๐ ๐ ) = ๐๐(๐ ๐ ) = 1 ๐๐ + ๐ ๐ 1 ๐๐ + ๐ ๐ Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. 1 ๐๐ ๐ข๐ข(๐ ๐ ) = 1 ๐๐๐๐ ๐ป๐ป(๐ก๐ก) = ๐๐๐๐ ๐ป๐ป(๐ ๐ ) = ๐๐ ๐ ๐ 2 ๐ข๐ข(๐ก๐ก) = โ(๐ก๐ก) = ๐๐ Hazard Rate Cumulative Hazard Rate Properties and Moments โ(๐ ๐ ) = ๐๐ ๐ ๐ ๐๐๐๐(2) ๐๐ Median Mode 0 Mean - 1st Raw Moment Variance - 2nd Central Moment Skewness - 3rd Central Moment Excess kurtosis - Laplace Domain 4th Central Moment Characteristic Function 100ฮฑ% Percentile Function 1 ๐๐ 1 ๐๐2 2 6 ๐๐๐๐ ๐ก๐ก + ๐๐๐๐ 1 ๐ก๐ก๐ผ๐ผ = โ ln(1 โ ๐ผ๐ผ) ๐๐ Exponential Continuous Distribution 43 Parameter Estimation Plotting Method X-Axis Y-Axis ๐ก๐ก๐๐ Likelihood Function nF Likelihood Functions nS F failures survivors when there is no interval data this reduces to: ๐ค๐คโ๐๐๐๐๐๐ Point Estimates Fisher Information 100๐พ๐พ% Confidence Interval (excluding interval data) UI LI nF interval failures ๐ก๐ก๐๐ = ๏ฟฝ t Fi + ๏ฟฝ t Si = ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐๐๐๐ ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก nS nI LI RI ฮ(๐ธ๐ธ|๐๐) = r. ln(ฮป) โ ๏ฟฝ ฮปt Fi โ ๏ฟฝ ฮปt s + ๏ฟฝ ln ๏ฟฝeโฮปti โ eโฮปti ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 i=1 failures survivors when there is no interval data this reduces to: ฮ(๐ธ๐ธ|๐๐) = nF . ln(๐๐) โ ๐๐๐ก๐ก ๐๐ โฮ =0 โฮป nI S ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) = ฮปnF ๏ฟฝ eโฮป.ti . ๏ฟฝ eโti . ๏ฟฝ ๏ฟฝeโฮปti โ eโฮปti ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) = ๐๐nF ๐๐ โ๐๐๐ก๐ก๐๐ Log-Likelihood Functions ๐๐ฬ = โ๐๐ ๐๐๐๐[1 โ ๐น๐น(๐ก๐ก๐๐ )] solve for ๐๐ to get ๐๐ฬ: nS nF interval failures ๐ก๐ก๐๐ = ๏ฟฝ t Fi + ๏ฟฝ t Si where nI LI RI ฮปti ฮปti t LI โ t RI nF i e i e โ ๏ฟฝ t Fi โ ๏ฟฝ t Si โ ๏ฟฝ ๏ฟฝ ๏ฟฝ=0 ๐ฟ๐ฟ๐ฟ๐ฟ ๐ ๐ ๐ ๐ ๐๐๐ก๐ก ๐๐๐ก๐ก ฮป ๐๐ ๐๐ โ ๐๐ ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 i=1 failures survivors interval failures When there is only complete and right-censored data the point estimate is: nF ๐๐ฬ = ๐ค๐คโ๐๐๐๐๐๐ ๐ก๐ก๐๐ = ๏ฟฝ t Fi + ๏ฟฝ t Si = ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐๐๐๐ ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐ก๐ก ๐๐ ฮปlower 2-Sided ๐ผ๐ผ(๐๐) = 1 ๐๐ ฮปupper 2-Sided ฮปupper 1-Sided Type I (Time Terminated) ๐๐ 21โฮณ (2nF ) ๏ฟฝ ๐๐ 21+ฮณ (2nF + 2) 2 (2nF + 2) ๐๐(ฮณ) Type II (Failure Terminated) ๐๐ 21โฮณ (2nF ) ๐๐ 21+ฮณ (2nF ) 2 (2nF ) ๐๐(ฮณ) ๏ฟฝ ๏ฟฝ 2 2๐ก๐ก ๐๐ 2 ๏ฟฝ 2๐ก๐ก ๐๐ ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ 2 2๐ก๐ก ๐๐ ๏ฟฝ 2๐ก๐ก ๐๐ 2๐ก๐ก ๐๐ 2๐ก๐ก๐๐ Exponential Least Mean Square - ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ 44 Common Life Distributions Exponential 2 ๐๐(๐ผ๐ผ) is the ๐ผ๐ผ percentile of the Chi-squared distribution. (Modarres et al. 1999, pp.151-152) Note: These confidence intervals are only valid for complete and right-censored data or when approximations of interval data are used (such as the median). They are exact confidence bounds and therefore approximate methods such as use of the Fisher information matrix need not be used. Bayesian Non-informative Priors ๐ ๐ (๐๐) (Yang and Berger 1998, p.6) Type Prior Posterior 1 ๐๐ โ ๐๐ Uniform Proper Prior with limits ๐๐ โ [๐๐, ๐๐] Uniform Improper Prior with limits ๐๐ โ [0, โ) Jeffreyโs Prior Truncated Gamma Distribution For a โค ฮป โค b ๐๐. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + nF , t T ) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0) 1 โ๐๐ Otherwise ๐๐(๐๐) = 0 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(2, 0) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0) ๐๐ Novick and Hall where ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + nF , t T ) ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 12 + nF , t T ) when ๐๐ โ [0, โ) ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; nF , t T ) when ๐๐ โ [0, โ) ๐ก๐ก ๐๐ = โ t Fi + โ t Si = total time in test Conjugate Priors UOI ๐๐ from ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ฮป) Example Likelihood Model Exponential Evidence ๐๐๐น๐น failures in ๐ก๐ก ๐๐ unit of time Dist. of UOI Prior Para Posterior Parameters Gamma ๐๐0 , ฮ0 ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ฮ = ฮ๐๐ + ๐ก๐ก ๐๐ Description , Limitations and Uses Three vehicle tires were run on a test area for 1000km have punctures at the following distances: Tire 1: No punctures Tire 2: 400km, 900km Tire 3: 200km Punctures are a random failure with constant failure rate therefore an exponential distribution would be appropriate. Due to an exponential distribution being homogeneous in time, the renewal process of the second tire failing twice with a repair can be considered as two separate tires on test with single failures. See example in section 1.1.6. Total distance on test is 3 × 1000 = 3000km. Total number of Exponential Continuous Distribution 45 failures is 3. Therefore using MLE the estimate of ๐๐: nF 3 = = 1E-3 ๐ก๐ก ๐๐ 3000 With 90% confidence interval (distance terminated test): 2 2 (6) (8) ๐๐(0.95) ๐๐(0.05) = 0.272๐ธ๐ธ-3, = 2.584๐ธ๐ธ-3๏ฟฝ ๏ฟฝ 6000 6000 A Bayesian point estimate using the Jeffery non-informative improper prior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(12, 0), with posterior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 3.5, 3000) has a point estimate: 3.5 ๐๐ฬ = E[๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 3.5, 3000)] = = 1.16ฬ E โ 3 3000 Characteristics With 90% confidence interval using inverse Gamma cdf: [๐น๐น๐บ๐บโ1 (0.05) = 0.361๐ธ๐ธ-3, ๐น๐น๐บ๐บโ1 (0.95) = 2.344๐ธ๐ธ-3] Constant Failure Rate. The exponential distribution is defined by a constant failure rate, ๐๐. This means the component is not subject to wear or accumulation of damage as time increases. ๐๐(๐๐) = ๐๐. As can be seen, ๐๐ is the initial value of the distribution. Increases in ๐๐ increase the probability density at ๐๐(0). HPP. The exponential distribution is the time to failure distribution of a single event in the Homogeneous Poisson Process (HPP). Scaling property Minimum property ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) ๐๐ ๐๐๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐ก๐ก; ๏ฟฝ ๐๐ ๐๐ min{๐๐1 , ๐๐2 , โฆ , ๐๐๐๐ }~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐ก๐ก; ๏ฟฝ ๐๐๐๐ ๏ฟฝ ๐๐=1 Variate Generation property ln(1 โ ๐ข๐ข) ๐น๐น โ1 (๐ข๐ข) = , 0 < ๐ข๐ข < 1 โ๐๐ Memoryless property. Pr(๐๐ > ๐ก๐ก + ๐ฅ๐ฅ|๐๐ > ๐ก๐ก) = Pr(๐๐ > ๐ฅ๐ฅ) Properties from (Leemis & McQueston 2008). Applications No Wearout. The exponential distribution is used to model occasions when there is no wearout or cumulative damage. It can be used to approximate the failure rate in a componentโs useful life period (after burn in and before wear out). Exponential ๐๐ฬ = 46 Common Life Distributions Homogeneous Poisson Process (HPP). The exponential distribution is used to model the inter arrival times in a repairable system or the arrival times in queuing models. See Poisson and Gamma distribution for more detail. Exponential Electronic Components. Some electronic components such as capacitors or integrated circuits have been found to follow an exponential distribution. Early efforts at collecting reliability data assumed a constant failure rate and therefore many reliability handbooks only provide a failure rate estimates for components. Random Shocks. It is common for the exponential distribution to model the occurrence of random shocks An example is the failure of a vehicle tire due to puncture from a nail (random shock). The probability of failure in the next mile is independent of how many miles the tire has travelled (memoryless). The probability of failure when the tire is new is the same as when the tire is old (constant failure rate). In general component life distributions do not have a constant failure rate, for example due to wear or early failures. Therefore the exponential distribution is often inappropriate to model most life distributions, particularly mechanical components. Resources Online: http://www.weibull.com/LifeDataWeb/the_exponential_distribution.h tm http://mathworld.wolfram.com/ExponentialDistribution.html http://en.wikipedia.org/wiki/Exponential_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Balakrishnan, N. & Basu, A.P., 1996. Exponential Distribution: Theory, Methods and Applications 1st ed., CRC. Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience. Relationship to Other Distributions 2-Para Exponential Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐, ๐ฝ๐ฝ) Special Case: Gamma Distribution Then ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; ๐๐, ๐๐) Let 1 ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; µ = 0, ฮฒ = ) ๐๐ ๐๐1 โฆ ๐๐๐๐ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐๐๐ ๐๐๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐๐ ๐๐๐ก๐ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) The gamma distribution is the probability density function of the sum of k exponentially distributed time random variables sharing the same constant rate of occurrence, ๐๐. This is a Homogeneous Poisson Process. Exponential Continuous Distribution Special Case: ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; ๐๐ = 1, ๐๐) Let ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐พ๐พ + ๐๐๐พ๐พ+1 โฆ Then ๐พ๐พ~๐๐๐๐๐๐๐๐(k; µ = ๐๐๐๐) Poisson Distribution The Poisson distribution is the probability of observing exactly k occurrences within a time interval [0, t] where the inter-arrival times of each occurrence is exponentially distributed. This is a Homogeneous Poisson Process. Special Cases: ๐๐๐๐๐๐๐๐(k = 1; µ = ๐๐๐๐) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) Let Weibull Distribution ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Special Case: Let Geometric Distribution ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; ๐๐) Rayleigh Distribution ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐ก๐ก; ๐ผ๐ผ) Chi-square ๐๐ 2 (๐ฅ๐ฅ; ๐ฃ๐ฃ) Pareto Distribution ๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐ผ๐ผ) ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) Then Then ๐๐๐๐๐๐ โ1 ๐๐ = X1/ฮฒ ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ผ๐ผ = ๐๐ ๐ฝ๐ฝ , ๐ฝ๐ฝ) 1 ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐ก๐ก; ๐ผ๐ผ = , ๐ฝ๐ฝ = 1๏ฟฝ ๐๐ ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐๐๐ ๐๐ = [๐๐], ๐๐ ๐๐๐๐ ๐ก๐กโ๐๐ ๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐ ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ผ๐ผ, ๐ฝ๐ฝ) The geometric distribution is the discrete equivalent of the continuous exponential distribution. The geometric distribution is also memoryless. Let ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) Then Special Case: Let Then ๐๐๐๐๐๐ ๐๐~๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐ผ๐ผ = 1 ๐๐ = โX โ๐๐ ) 1 ๐๐ 2 (๐ฅ๐ฅ; ๐ฃ๐ฃ = 2) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐ฅ๐ฅ; ฮป = ๏ฟฝ 2 ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) ๐๐๐๐๐๐ ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐ = ๐ผ๐ผ) ๐๐ = ln(๐๐โ๐๐) Exponential ๐๐1 , ๐๐2 โฆ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) Given ๐๐๐๐๐๐๐๐(๐๐; ๐๐) 47 48 Common Life Distributions Exponential Let Logistic Distribution ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(µ , ๐ ๐ ) Then ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐ = 1) ๐๐๐๐๐๐ ๐๐ = ln ๏ฟฝ ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1) (Hastings et al. 2000, p.127): ๐๐ โ๐๐ ๏ฟฝ 1 + ๐๐ โ๐๐ Lognormal Continuous Distribution 49 2.2. Lognormal Continuous Distribution ฮผ'=0 ฯ'=1 ฮผ'=0.5 ฯ'=1 ฮผ'=1 ฯ'=1 ฮผ'=1.5 ฯ'=1 0.90 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00 ฮผ'=1 ฯ'=1 ฮผ'=1 ฯ'=1 ฮผ'=1 ฯ'=1.5 ฮผ'=1 ฯ'=2 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 0 2 4 6 Cumulative Density Function - F(t) 0 2 4 0 2 4 0 2 4 0.90 1.00 0.80 0.80 0.70 0.60 0.60 0.50 0.40 0.40 0.30 0.20 0.20 0.00 0.00 0.10 0 2 4 6 Hazard Rate - h(t) 0.70 1.80 1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00 0.60 0.50 0.40 0.30 0.20 0.10 0.00 0 2 4 6 Lognormal Probability Density Function - f(t) 50 Common Life Distributions Lognormal Parameters & Description ๐๐๐๐ Scale parameter: The mean of the normally distributed ln(๐ฅ๐ฅ). This parameter only determines the scale and not the location as in a normal distribution. ๐๐2 ๐๐๐๐ = ln ๏ฟฝ ๏ฟฝ 2 ๏ฟฝ๐๐ + ๐๐2 โโ < ๐๐๐๐ < โ Parameters ฯ2N Shape parameter: The standard deviation of the normally distributed ln(x). This parameter only determines the shape and not the scale as in a normal distribution. ๐๐ 2 + ๐๐2 ฯ2N = ln ๏ฟฝ ๏ฟฝ ๐๐2 ฯ2N > 0 Limits t>0 Distribution Formulas ๐๐(๐ก๐ก) = PDF 1 ln(๐ก๐ก) โ ๐๐๐๐ 2 exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐๐๐๐ ๐๐๐๐ ๐ก๐กโ2๐๐ 1 = 1 ๐๐๐๐(๐ก๐ก) โ ๐๐๐๐ ๐๐ ๏ฟฝ ๏ฟฝ ๐๐๐๐ . ๐ก๐ก ๐๐๐๐ where ๐๐ is the standard normal pdf. ๐ก๐ก 2 1 1 ln(๐ก๐ก โ ) โ ๐๐๐๐ exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐๐ก๐ก โ โ 2 ๐๐๐๐ ๐๐๐๐ โ2๐๐ 0 ๐ก๐ก where ๐ก๐ก โ is the time variable over which the pdf is integrated. ๐น๐น(๐ก๐ก) = 1 ๏ฟฝ = CDF ๐๐๐๐(๐ก๐ก) โ ๐๐๐๐ 1 1 + erf ๏ฟฝ ๏ฟฝ 2 2 ๐๐๐๐ โ2 ln(๐ก๐ก) โ ๐๐๐๐ ๏ฟฝ = ฮฆ๏ฟฝ ๐๐๐๐ where ฮฆ is the standard normal cdf. ln(๐ก๐ก) โ ๐๐๐๐ R(t) = 1 โ ฮฆ ๏ฟฝ ๏ฟฝ ๐๐๐๐ Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Where ln(๐ฅ๐ฅ + t) โ ๐๐๐๐ ๏ฟฝ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) 1 โ ฮฆ ๏ฟฝ ๐๐๐๐ ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = = ln(t) โ ๐๐๐๐ ๐ ๐ (๐ก๐ก) 1 โ ฮฆ๏ฟฝ ๏ฟฝ ๐๐๐๐ Lognormal Continuous Distribution 51 ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. ln(๐ก๐ก) โ ๐๐๐๐ ๏ฟฝ ๐๐๐๐ โ(๐ก๐ก) = ln(๐ก๐ก) โ ๐๐๐๐ ๐ก๐ก. ๐๐๐๐ ๏ฟฝ1 โ ฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐๐๐ ๐๐ ๏ฟฝ Hazard Rate Cumulative Hazard Rate ๐ป๐ป(๐ก๐ก) = โln[๐ ๐ (๐ก๐ก)] Properties and Moments ๐๐ (๐๐๐๐) Median 2 ๐๐ (๐๐๐๐โ๐๐๐๐ ) Mode Mean - 1st Raw Moment Variance - 2nd 2 ๏ฟฝ๐๐ ๐๐๐๐ Central Moment Skewness - 3rd Central Moment 2 2 ๐๐๐๐ ๏ฟฝ 2 2 โ 1๏ฟฝ. ๐๐ 2๐๐๐๐ +๐๐๐๐ ๏ฟฝ๐๐ ๐๐ + 2๏ฟฝ. ๏ฟฝ๐๐ ๐๐ โ 1 2 2 2 2 ๐๐ 4๐๐๐๐ + 2๐๐ 3๐๐๐๐ + 3๐๐ 2๐๐๐๐ โ 3 Excess kurtosis - 4th Central Moment Characteristic Function ๐๐ ๏ฟฝ๐๐๐๐ + Deriving a unique characteristic equation is not trivial and complex series solutions have been proposed. (Leipnik 1991) 100ฮฑ% Percentile Function ๐ก๐ก๐ผ๐ผ = e(๐๐๐๐+๐ง๐ง๐ผ๐ผ.๐๐๐๐) where ๐ง๐ง๐ผ๐ผ is the 100pthof the standard normal distribution Parameter Estimation ๐ก๐กฮฑ = e(๐๐๐๐ +๐๐๐๐ฮฆ โ1 (๐ผ๐ผ)) Plotting Method Least Mean Square ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ X-Axis Y-Axis ln(๐ก๐ก๐๐ ) ๐๐๐๐๐๐๐๐๐๐๐๐๐๐[๐น๐น(๐ก๐ก๐๐ )] ๐๐ ๐๐๏ฟฝ ๐๐ = โ ๐๐ 1 ๐๐๏ฟฝ ๐๐ = ๐๐ Lognormal Mean Residual Life โ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ ๐ ๐ (๐ก๐ก) ๐๐๐๐2 ๐ก๐ก lim ๐ข๐ข(๐ก๐ก) โ [1 + ๐๐(1)] ๐ก๐กโโ ln(๐ก๐ก) โ ๐๐๐๐ Where ๐๐(1) is Landau's notation. (Kleiber & Kotz 2003, p.114) ๐ข๐ข(๐ก๐ก) = 52 Common Life Distributions Maximum Likelihood Function Lognormal Likelihood Functions nF nS nI 1 ๐น๐น ๏ฟฝ ๏ฟฝ1 โ ฮฆ๏ฟฝ๐ง๐ง๐๐๐๐ ๏ฟฝ๏ฟฝ . ๏ฟฝ ๏ฟฝฮฆ๏ฟฝ๐ง๐ง๐๐๐ ๐ ๐ ๐ ๏ฟฝ โ ฮฆ๏ฟฝ๐ง๐ง๐๐๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ F ๐๐(๐ง๐ง๐๐ ) . ๏ฟฝ ๐๐ . t ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 ๐๐ i i=1 i=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ where survivors failures ln(t xi ) โ ๐๐๐๐ ๐ง๐ง๐๐๐ฅ๐ฅ = ๏ฟฝ ๏ฟฝ ๐๐๐๐ nF Log-Likelihood Function nI failures survivors interval failures ln(t xi ) โ ๐๐๐๐ ๐ง๐ง๐๐๐ฅ๐ฅ = ๏ฟฝ ๏ฟฝ ๐๐๐๐ solve for ๐๐๐๐ to get MLE ๐๐๏ฟฝ: ๐๐ nF nS ฯ๏ฟฝziS ๏ฟฝ โฮ โµN . NF 1 1 = + ๏ฟฝ ln(t Fi ) + ๏ฟฝ โµN ฯN ฯN ฯN 1 โ ฮฆ๏ฟฝziS ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 nI where failures 1 ฯ๏ฟฝziRI ๏ฟฝ โ ฯ๏ฟฝziLI ๏ฟฝ โ๏ฟฝ ๏ฟฝ ๏ฟฝ=0 ฯN ฮฆ๏ฟฝziRI ๏ฟฝ โ ฮฆ๏ฟฝziLI ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 survivors interval failures solve for ๐๐๐๐ to get ๐๐๏ฟฝ: ๐๐ ln(t xi ) โ ๐๐๐๐ ๐ง๐ง๐๐๐ฅ๐ฅ = ๏ฟฝ ๏ฟฝ ๐๐๐๐ nF nS ziS . ฯ๏ฟฝziS ๏ฟฝ โฮ โnF 1 1 2 = + 3 ๏ฟฝ๏ฟฝln(t Fi ) โ µN ๏ฟฝ + ๏ฟฝ โฯN ฯN ฯN ฯN 1 โ ฮฆ๏ฟฝziS ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 failures 1 ziRI . ฯ๏ฟฝziRI ๏ฟฝ โ ziLI ฯ๏ฟฝziLI ๏ฟฝ โ๏ฟฝ ๏ฟฝ ๏ฟฝ ฯN ฮฆ๏ฟฝziRI ๏ฟฝ โ ฮฆ๏ฟฝziLI ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 nI where MLE Point Estimates i=1 + ๏ฟฝ ln๏ฟฝฮฆ๏ฟฝ๐ง๐ง๐๐๐ ๐ ๐ ๐ ๏ฟฝ โ ฮฆ๏ฟฝ๐ง๐ง๐๐๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 where โฮ =0 โฯN nS 1 ๐น๐น ๐๐ ฮ(µN , ฯN |E) = ๏ฟฝ ln ๏ฟฝ ๐น๐น ๐๐๏ฟฝ๐ง๐ง๐๐ ๏ฟฝ๏ฟฝ + ๏ฟฝ ln๏ฟฝ1 โ ฮฆ๏ฟฝ๐ง๐ง๐๐ ๏ฟฝ๏ฟฝ ๐๐ . ๐ก๐ก ๐๐ ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 โฮ =0 โµN interval failures interval failures ln(t xi ) โ ๐๐๐๐ ๐ง๐ง๐๐๐ฅ๐ฅ = ๏ฟฝ ๏ฟฝ ๐๐๐๐ survivors =0 When there is only complete failure data the point estimates can be given as: 2 ๐น๐น โ ln(๐ก๐ก๐๐๐น๐น ) ๏ฟฝ2 = โ๏ฟฝln๏ฟฝ๐ก๐ก๐๐ ๏ฟฝ โ ๐๐๏ฟฝ๐ก๐ก ๏ฟฝ ๐๐๏ฟฝ = ฯ ๐๐ N nF nF Lognormal Continuous Distribution 53 Note: In almost all cases the MLE methods for a normal distribution can be used by taking the ๐๐๐๐(๐๐). However Normal distribution estimation methods cannot be used with interval data. (Johnson et al. 1994, p.220) 1 โก 2 ๐๐ ๐ผ๐ผ(๐๐๐๐ , ๐๐๐๐2 ) = โข ๐๐ โข โฃ0 (Kleiber & Kotz 2003, p.119). Fisher Information 100๐พ๐พ% Confidence Intervals 1-Sided Lower ๐๐๐ต๐ต ๐๐๐๐๐ต๐ต (for complete data) ๐๐๏ฟฝ๐๐ โ 0 2-Sided Lower ๐๐ ๏ฟฝ ๐๐ ๐ก๐กฮณ (nF โ 1) โ๐๐๐น๐น (๐๐๐น๐น โ 1) 2 ๐๐๏ฟฝ ๐๐ 2 ๐๐๐พ๐พ (๐๐๐น๐น โ 1) โค โฅ 1 โฅ โ 4โฆ 2๐๐ ๐๐๏ฟฝ๐๐ โ 2 ๐๐๏ฟฝ ๐๐ Lognormal In most cases the unbiased estimators are used: 2 ๐น๐น โ ln(๐ก๐ก๐๐๐น๐น ) ๏ฟฝ2 = โ๏ฟฝln๏ฟฝ๐ก๐ก๐๐ ๏ฟฝ โ ๐๐๏ฟฝ๐ก๐ก ๏ฟฝ ๐๐๏ฟฝ = ฯ ๐๐ N nF nF โ 1 ๐๐ ๏ฟฝ ๐๐ ๐ก๐ก 1โฮณ (nF โ 1) โ๐๐๐น๐น ๏ฟฝ 2 ๏ฟฝ (๐๐๐น๐น โ 1) 2 ๐๐ 1+ฮณ (๐๐๐น๐น โ 1) ๏ฟฝ ๏ฟฝ 2 2-Sided Upper ๐๐๏ฟฝ๐๐ + 2 ๐๐๏ฟฝ ๐๐ ๐๐ ๏ฟฝ ๐๐ ๐ก๐ก 1โฮณ (nF โ 1) โ๐๐๐น๐น ๏ฟฝ 2 ๏ฟฝ (๐๐๐น๐น โ 1) 2 ๐๐ 1โฮณ (๐๐๐น๐น โ 1) ๏ฟฝ ๏ฟฝ 2 Where ๐ก๐กฮณ (nF โ 1) is the 100๐พ๐พth percentile of the t-distribution with ๐๐๐น๐น โ 1 degrees of freedom and ๐๐๐พ๐พ2 (๐๐๐น๐น โ 1) is the 100๐พ๐พth percentile of the ๐๐ 2 distribution with ๐๐๐น๐น โ 1 degrees of freedom. (Nelson 1982, pp.218-219) 1 Sided - Lower ๐๐ exp ๏ฟฝ๐๐๏ฟฝ๐๐ + 2 Sided 2 2 4 ๐๐๏ฟฝ ๐๐๏ฟฝ ๐๐๏ฟฝ ๐๐ ๐๐ ๐๐ โ ๐๐1โ๐ผ๐ผ ๏ฟฝ + ๏ฟฝ 2 ๐๐๐น๐น 2(๐๐๐น๐น โ 1) exp ๏ฟฝ๐๐๏ฟฝ๐๐ + 2 2 4 ๐๐๏ฟฝ ๐๐๏ฟฝ ๐๐๏ฟฝ ๐๐ ๐๐ ๐๐ ± ๐๐1โ๐ผ๐ผ๏ฟฝ2๏ฟฝ + ๏ฟฝ 2 ๐๐๐น๐น 2(๐๐๐น๐น โ 1) These formulas are the Cox approximation for the confidence intervals of the lognormal distribution mean where ๐๐๐๐ = ๐ท๐ทโ1 (๐๐), the inverse of the standard normal cdf. (Zhou & Gao 1997) Zhou & Gao recommend using the parametric bootstrap method for small sample sizes. (Angus 1994) Bayesian Type Non-informative Priors when ๐๐๐๐๐ต๐ต is known, ๐ ๐ ๐๐ (๐๐๐ต๐ต ) (Yang and Berger 1998, p.22) Uniform Proper Prior with limits ๐๐๐๐ โ [๐๐, ๐๐] Prior Posterior 1 ๐๐ โ ๐๐ Truncated Normal Distribution For a โค µN โค b โ๐๐๐น๐น ln ๐ก๐ก๐๐๐น๐น ฯ2N ๐๐. ๐๐๐๐๐๐๐๐ ๏ฟฝµN ; ๐๐=1 , ๏ฟฝ ๐๐๐น๐น nF Otherwise ๐๐(๐๐๐๐ ) = 0 54 Common Life Distributions Lognormal ๐๐๐๐๐๐๐๐ ๏ฟฝµN ; Non-informative Priors when ๐๐๐ต๐ต is known, ๐ ๐ ๐๐ (๐๐๐๐๐๐ ) (Yang and Berger 1998, p.23) Type Prior Posterior Uniform Proper Prior with limits ๐๐๐๐2 โ [๐๐, ๐๐] 1 ๐๐ โ ๐๐ Uniform Improper Prior with limits ๐๐๐๐2 โ (0, โ) 1 Truncated Inverse Gamma Distribution For a โค ฯ2N โค b 2 (๐๐๐น๐น โ 2) SN ๐๐. ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; , ๏ฟฝ 2 2 Otherwise ๐๐(๐๐๐๐2 ) = 0 2 (๐๐๐น๐น โ 2) SN , ๏ฟฝ 2 2 See section 1.7.1 ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; 1 ๐๐๐๐2 Jefferyโs, Reference, MDIP Prior Type ๐น๐น โ๐๐๐๐=1 ln ๐ก๐ก๐๐๐น๐น ฯ2N , ๏ฟฝ ๐๐๐น๐น nF when ๐๐๐๐ โ (โ, โ) 1 All 2 ๐๐๐น๐น SN , ๏ฟฝ 2 2 with limits ๐๐๐๐2 โ (0, โ) See section 1.7.1 ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; Non-informative Priors when ๐๐๐ต๐ต and ๐๐๐๐๐ต๐ต are unknown, ๐ ๐ ๐๐ (๐๐๐ต๐ต , ฯ2N ) (Yang and Berger 1998, p.23) Prior Posterior Improper Uniform with limits: ๐๐๐๐ โ (โ, โ) ๐๐๐๐2 โ (0, โ) 1 Jefferyโs Prior 1 ๐๐๐๐4 Reference Prior ordering {๐๐, ๐๐} โ ๐๐๐๐ (๐๐, ๐๐๐๐2 ) 1 ๐๐๐๐ ๏ฟฝ2 + ๐๐ 2 where ๐๐ = ๐๐๐๐ /๐๐๐๐ 2 SN ๏ฟฝ ๐๐๐น๐น (๐๐๐น๐น โ 3) See section 1.7.2 2 (๐๐๐น๐น โ 3) SN ๐๐(ฯ2N |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; , ๏ฟฝ 2 2 See section 1.7.1 ๐๐(๐๐๐๐ |๐ธ๐ธ)~๐๐ ๏ฟฝµN ; ๐๐๐น๐น โ 3, ๏ฟฝ๏ฟฝ๏ฟฝ ๐ก๐ก๐๐ , S2 ๏ฟฝ ๐๐๐น๐น (๐๐๐น๐น + 1) when ๐๐๐๐ โ (โ, โ) See section 1.7.2 2 (๐๐๐น๐น + 1) SN 2 , ๏ฟฝ ๐๐(ฯN |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; 2 2 when ๐๐๐๐2 โ (0, โ) See section 1.7.1 ๐๐(๐๐๐๐ |๐ธ๐ธ)~๐๐ ๏ฟฝµN ; ๐๐ ๐น๐น + 1, ๏ฟฝ๏ฟฝ๏ฟฝ ๐ก๐ก๐๐ , No closed form Lognormal Continuous Distribution 1 ๐๐๐๐ Reference where ๐๐ and ๐๐ 2 are separate groups. 2 SN ๏ฟฝ nF (nF โ 1) when ๐๐๐๐ โ (โ, โ) See section 1.7.2 (๐๐๐น๐น โ 1) ๐๐๐๐2 ๐๐(ฯ2N |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2N ; , ๏ฟฝ 2 2 when ๐๐๐๐2 โ (0, โ) See section 1.7.1 ๐๐(๐๐๐๐ |๐ธ๐ธ)~๐๐ ๏ฟฝµN ; ๐๐ ๐น๐น โ 1, ๏ฟฝ๏ฟฝ๏ฟฝ ๐ก๐ก๐๐ , MDIP Prior ๐๐๐๐2 UOI ๐๐๐น๐น = ๏ฟฝ(ln ๐ก๐ก๐๐ โ ๐๐=1 Likelihood Model ๐๐๐๐2 from Lognormal with known ๐๐๐๐ ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐๐๐ , ๐๐๐๐2 ) ๐๐๐๐ from Lognormal with known ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐๐๐ , ๐๐๐๐2 ) ๐๐๐๐2 ๏ฟฝ๏ฟฝ๏ฟฝ) ๐ก๐ก๐๐ 2 and ๐๐๐น๐น 1 ๏ฟฝ๏ฟฝ๏ฟฝ ๐ก๐ก๐๐ = ๏ฟฝ ln ๐ก๐ก๐๐ nF ๐๐=1 Conjugate Priors Evidence ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Dist. of UOI Gamma ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Normal Prior Para Posterior Parameters ๐๐ = ๐๐๐๐ + ๐๐๐น๐น /2 ๐๐๐น๐น ๐๐0 , ๐๐0 1 ๐๐ = ๐๐๐๐ + ๏ฟฝ(ln ๐ก๐ก๐๐ โ ๐๐๐๐ )2 2 ๐๐=1 ๐๐๐๐ , ๐๐๐๐2 ฯ2 = Description , Limitations and Uses Example ๐๐ ๐น๐น ln(๐ก๐ก๐๐ ) µ0 โ๐๐=1 2+ ฯ0 ๐๐๐๐2 ๐๐ = 1 ๐๐๐น๐น + ๐๐02 ๐๐๐๐2 1 1 nF + ฯ20 ๐๐๐๐2 5 components are put on a test with the following failure times: 98, 116, 2485, 2526, , 2920 hours Taking the natural log of these failure times allows us to use a normal distribution to approximate the parameters. ln(๐ก๐ก๐๐ ): 4.590, 4.752, 7.979, 7.818, 7.834 ๐๐๐๐(hours) MLE Estimates are: โ ln๏ฟฝ๐ก๐ก๐๐๐น๐น ๏ฟฝ 32.974 = = 6.595 nF 5 2 โ๏ฟฝln๏ฟฝ๐ก๐ก๐๐๐น๐น ๏ฟฝ โ ๐๐๏ฟฝ๐ก๐ก ๏ฟฝ 2 ๐๐๏ฟฝ = 3.091 ๐๐ = nF โ 1 ๐๐๏ฟฝ ๐๐ = 90% confidence interval for ๐๐๐๐ : ๐๐ ๏ฟฝ๐๐ ๏ฟฝ๐๐๏ฟฝ๐๐ โ ๐ก๐ก{0.95} (4), โ4 ๐๐ ๏ฟฝ๐๐ + ๐๐ ๏ฟฝ๐๐ โ4 ๐ก๐ก{0.95} (4)๏ฟฝ Lognormal where 55 56 Common Life Distributions [4.721, 8.469] Lognormal 90% confidence interval for ๐๐๐๐2 : 4 4 2 2 , ๐๐๏ฟฝ ๏ฟฝ ๏ฟฝ๐๐๏ฟฝ ๐๐ 2 ๐๐ 2 ๐๐{0.95} (4) ๐๐{0.05} (4) [1.303, 17.396] A Bayesian point estimate using the Jeffery non-informative improper prior 1โ๐๐๐๐4 with posterior for ๐๐๐๐ ~๐๐(6, 6.595, 0.412 ) and ๐๐๐๐2 ~๐ผ๐ผ๐ผ๐ผ(3, 6.182) has a point estimates: ๐๐๏ฟฝ๐๐ = E[๐๐(6,6.595,0.412 )] = µ = 6.595 6.182 2 ๐๐๏ฟฝ = 3.091 ๐๐ = E[๐ผ๐ผ๐ผ๐ผ(3,6.182)] = 2 Characteristics With 90% confidence intervals: ๐๐๐๐ [๐น๐น๐๐โ1 (0.05) = 5.348, 2 ๐๐๐๐ [1/๐น๐น๐บ๐บโ1 (0.95) = 0.982, ๐น๐น๐๐โ1 (0.95) = 7.842] 1/๐น๐น๐บ๐บโ1 (0.05) = 7.560] ๐๐๐ต๐ต Characteristics. ๐๐๐๐ determines the scale and not the location as in a normal distribution. The distribution if fixed at f(0)=0 and an increase in the scale parameter stretches the distribution across the x-axis. This has the effect of increasing the mode, mean and median of the distribution. ๐๐๐ต๐ต Characteristics. ๐๐๐๐ determines the shape and not the scale as in a normal distribution. For values of ฯN > 1 the distribution rises very sharply at the beginning and decreases with a shape similar to an Exponential or Weibull with 0 < ๐ฝ๐ฝ < 1. As ฯN โ 0 the mode, mean and median converge to ๐๐ ๐๐๐๐ . The distribution becomes narrower and approaches a Dirac delta function at ๐ก๐ก = ๐๐ ๐๐๐๐ . Hazard Rate. (Kleiber & Kotz 2003, p.115)The hazard rate is unimodal with โ(0) = 0 and all dirivitives of โโ(๐ก๐ก) = 0 and a slow decrease to zero as ๐ก๐ก โ 0. The mode of the hazard rate: ๐ก๐ก๐๐ = exp(๐๐ + ๐ง๐ง๐๐ ๐๐) where ๐ง๐ง๐๐ is given by: ๐๐(๐ง๐ง๐๐ ) (๐ง๐ง๐๐ + ๐๐๐๐ ) = 1 โ ฮฆ(๐ง๐ง๐๐ ) therefore โ๐๐๐๐ < ๐ง๐ง๐๐ < โ๐๐๐๐ + ๐๐ โ1 and therefore: 2 2 ๐๐ ๐๐๐๐โ๐๐๐๐ < ๐ก๐ก๐๐ < ๐๐ ๐๐๐๐โ๐๐๐๐ +1 2 ๐๐ โ๐๐ ๐๐ ๐๐ and so for large ๐๐ : As ๐๐๐๐ โ โ, ๐ก๐ก๐๐ โ ๐๐ ๐๐ exp๏ฟฝ๐๐๐๐ โ 12๐๐๐๐2 ๏ฟฝ max โ(๐ก๐ก) โ ๐๐๐๐ โ2๐๐ Lognormal Continuous Distribution 57 2 As ๐๐๐๐ โ 0, ๐ก๐ก๐๐ โ ๐๐ ๐๐๐๐โ๐๐๐๐ +1 and so for large ๐๐๐๐ : 1 max โ(๐ก๐ก) โ 2 ๐๐ โ๐๐2 +1 ๐๐ ๐๐ ๐๐๐๐ ๐๐ Scale/Product Property: Let: ๐๐๐๐ ๐๐๐๐ ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๏ฟฝ๐๐๐๐๐๐ , ฯ2Nj ๏ฟฝ If ๐๐๐๐ and ๐๐๐๐+1 are independent: 2 ๏ฟฝ ๐๐๐๐ ๐๐๐๐ ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ๏ฟฝ๐๐๐๐๐๐ + ln๏ฟฝ๐๐๐๐ ๏ฟฝ๏ฟฝ, ๏ฟฝ ๐๐๐๐๐๐ ๏ฟฝ Lognormal versus Weibull. In analyzing life data to these distributions it is often the case that both may be a good fit, especially in the middle of the distribution. The Weibull distribution has an earlier lower tail and produces a more pessimistic estimate of the component life. (Nelson 1990, p.65) Applications General Life Distributions. The lognormal distribution has been found to accurately model many life distributions and is a popular choice for life distributions. The increasing hazard rate in early life models the weaker subpopulation (burn in) and the remaining decreasing hazard rate describes the main population. In particular this has been applied to some electronic devices and fatigue-fracture data. (Meeker & Escobar 1998, p.262) Failure Modes from Multiplicative Errors. The lognormal distribution is very suitable for failure processes that are a result of multiplicative errors. Specific applications include failure of components due to fatigue cracks. (Provan 1987) Repair Times. The lognormal distribution has commonly been used to model repair times. It is natural for a repair time probability to increase quickly to a mode value. For example very few repairs have an immediate or quick fix. However, once the time of repair passes the mean it is likely that there are serious problems, and the repair will take a substantial amount of time. Parameter Variability. The lognormal distribution can be used to model parameter variability. This was done when estimating the uncertainty in the parameter ๐๐ in a Nuclear Reactor Safety Study (NUREG-75/014). Theory of Breakage. The distribution models particle sizes observed in breakage processes (Crow & Shimizu 1988) Resources Online: Lognormal Mean / Median / Mode: ๐๐๐๐๐๐๐๐(๐๐) < ๐๐๐๐๐๐๐๐๐๐๐๐(๐๐) < ๐ธ๐ธ[๐๐] 58 Common Life Distributions Lognormal http://www.weibull.com/LifeDataWeb/the_lognormal_distribution.ht m http://mathworld.wolfram.com/LogNormalDistribution.html http://en.wikipedia.org/wiki/Log-normal_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Crow, E.L. & Shimizu, K., 1988. Lognormal distributions, CRC Press. Aitchison, J.J. & Brown, J., 1957. The Lognormal Distribution, New York: Cambridge University Press. Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience. Relationship to Other Distributions Normal Distribution ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐ 2 ) Let; Then: Where: ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐๐๐๐ , ฯ2N ) ๐๐ = ln(๐๐) ๐๐~๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 ) ๐๐2 ๐๐๐๐ = ln ๏ฟฝ ๏ฟฝ, ๏ฟฝ๐๐ 2 + ๐๐2 ๐๐ 2 + ๐๐2 ๐๐๐๐ = ln ๏ฟฝ ๏ฟฝ ๐๐2 Weibull Continuous Distribution 59 2.3. Weibull Continuous Distribution Probability Density Function - f(t) 0.90 ฮฑ=1 ฮฒ=2 ฮฑ=1.5 ฮฒ=2 ฮฑ=2 ฮฒ=2 ฮฑ=3 ฮฒ=2 0.80 0.70 0.60 0.50 0.40 0.30 0.20 0.10 0.00 0 1 0 2 2 4 Cumulative Density Function - F(t) 1.00 1.00 0.80 0.80 0.60 0.60 0.40 ฮฑ=1 ฮฒ=0.5 ฮฑ=1 ฮฒ=1 ฮฑ=1 ฮฒ=2 ฮฑ=1 ฮฒ=5 0.20 0.00 0 1 6 ฮฑ=1 ฮฒ=2 ฮฑ=1.5 ฮฒ=2 ฮฑ=2 ฮฒ=2 ฮฑ=3 ฮฒ=2 0.20 0.00 2 Hazard Rate - h(t) 0 10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 5 4 3 2 1 0 0 0.40 1 2 2 4 ฮฑ=1 ฮฒ=2 ฮฑ=1.5 ฮฒ=2 ฮฑ=2 ฮฒ=2 ฮฑ=3 ฮฒ=2 0 2 4 Weibull ฮฑ=1 ฮฒ=0.5 ฮฑ=1 ฮฒ=1 ฮฑ=1 ฮฒ=2 ฮฑ=1 ฮฒ=5 1.80 1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00 60 Common Life Distributions Weibull Parameters & Description Parameters ๐ผ๐ผ ๐ฝ๐ฝ Scale Parameter: The value of ๐ผ๐ผ equals the 63.2th percentile and has a unit equal to ๐ก๐ก. Note that this is not equal to the mean. ๐ผ๐ผ > 0 Shape Parameter: Also known as the slope (referring to a linear CDF plot) ๐ฝ๐ฝ determines the shape of the distribution. ๐ฝ๐ฝ > 0 ๐ก๐ก โฅ 0 Limits Distribution PDF CDF Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Formulas ๐๐(๐ก๐ก) = ๐ฝ๐ฝ๐ก๐ก ๐ฝ๐ฝโ1 โ๏ฟฝ ๐ก๐ก ๏ฟฝ๐ฝ๐ฝ ๐๐ ๐ผ๐ผ ๐ผ๐ผ ๐ฝ๐ฝ ๐น๐น(๐ก๐ก) = 1 โ ๐๐ R(t) = ๐๐ ๐ก๐ก ๐ฝ๐ฝ โ๏ฟฝ ๏ฟฝ ๐ผ๐ผ ๏ฟฝ ๐ ๐ (๐ก๐ก + x) ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = = ๐๐ ๐ ๐ (๐ก๐ก) ๐ก๐ก ๐ฝ๐ฝ โ(๐ก๐ก+x)๐ฝ๐ฝ ๏ฟฝ ๐ผ๐ผ ๐ฝ๐ฝ Where ๐ก๐ก is the given time we know the component has survived to ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. (Kleiber & Kotz 2003, p.176) Mean Residual Life ๐ก๐ก ๐ฝ๐ฝ โ๏ฟฝ ๏ฟฝ ๐ผ๐ผ ๐ข๐ข(๐ก๐ก) = ๐๐ ๐ก๐ก ๐ฝ๐ฝ ๏ฟฝ ๏ฟฝ ๐ผ๐ผ โ ๏ฟฝ ๐๐ t ๐ฅ๐ฅ ๐ฝ๐ฝ โ๏ฟฝ ๏ฟฝ ๐ผ๐ผ which has the asymptotic property of: lim ๐ข๐ข(๐ก๐ก) = ๐ก๐ก 1โ๐ฝ๐ฝ ๐๐๐๐ ๐ก๐กโโ Hazard Rate Cumulative Hazard Rate ฮฒ ๐ก๐ก ๐ฝ๐ฝโ1 โ(๐ก๐ก) = . ๏ฟฝ ๏ฟฝ ฮฑ ๐ผ๐ผ ๐ก๐ก ๐ฝ๐ฝ ๐ป๐ป(๐ก๐ก) = ๏ฟฝ ๏ฟฝ ๐ผ๐ผ Properties and Moments Median Mode 1 ๐ผ๐ผ๏ฟฝ๐๐๐๐(2)๏ฟฝ๐ฝ๐ฝ 1 ๐ฝ๐ฝ โ 1 ๐ฝ๐ฝ ๐ผ๐ผ ๏ฟฝ ๏ฟฝ if ๐ฝ๐ฝ โฅ 1 ๐ฝ๐ฝ otherwise no mode exists Weibull Continuous Distribution 61 1 ๐ผ๐ผฮ ๏ฟฝ1 + ๏ฟฝ ๐ฝ๐ฝ Mean - 1st Raw Moment 3 ฮ ๏ฟฝ1 + ๏ฟฝ ฮฑ3 โ 3µฯ2 โ µ3 ฮฒ ฯ3 Skewness - 3rd Central Moment โ6ฮ14 + 12ฮ12 ฮ2 โ 3ฮ22 โ 4ฮ1 ฮ3 + ฮ4 (ฮ2 โ ฮ12 )2 where: ๐๐ ฮ๐๐ = ฮ ๏ฟฝ1 + ๏ฟฝ ๐ฝ๐ฝ Excess kurtosis - 4th Central Moment โ Characteristic Function ๏ฟฝ ๐๐=0 100p% Percentile Function (๐๐๐๐)๐๐ ๐ผ๐ผ ๐๐ ๐๐ ฮ ๏ฟฝ1 + ๏ฟฝ ๐๐! ๐ฝ๐ฝ 1 ๐ก๐ก๐๐ = ๐ผ๐ผ[โ ln(1 โ ๐๐)]๐ฝ๐ฝ Parameter Estimation Plotting Method Least Mean Square ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ Likelihood Functions X-Axis ๐ผ๐ผ๏ฟฝ = ๐๐ โ๐๐ ๐ฝ๐ฝฬ = ๐๐ 1 ln ๏ฟฝ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ 1 โ ๐น๐น ln(๐ก๐ก๐๐ ) Maximum Likelihood Function tF ๐ฝ๐ฝโ1 ๐ฝ๐ฝ tS ๐ฝ๐ฝ nS โ๏ฟฝ i ๏ฟฝ โ๏ฟฝ i ๏ฟฝ ๐ฝ๐ฝ๏ฟฝ๐ก๐ก๐๐๐น๐น ๏ฟฝ L(ฮฑ, ฮฒ|E) = ๏ฟฝ ๐๐ ๐ผ๐ผ . ๏ฟฝ ๐๐ ๐ผ๐ผ ๐ฝ๐ฝ ๐ผ๐ผ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 nF nI Log-Likelihood Function ๐๐ Y-Axis failures ๐ฝ๐ฝ tLI โ๏ฟฝ i ๏ฟฝ ๐ผ๐ผ โ๏ฟฝ tRI i ๏ฟฝ ๐ผ๐ผ ๐ฝ๐ฝ survivors ๏ฟฝ ๏ฟฝ๐๐ โ ๐๐ ๏ฟฝ i=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ interval failures nF ๐ฝ๐ฝ ๐ก๐ก๐๐๐น๐น ฮ(ฮฑ, ฮฒ|E) = nF ln(ฮฒ) โ ฮฒnF ln(๐ผ๐ผ) + ๏ฟฝ ๏ฟฝ(๐ฝ๐ฝ โ 1) ln๏ฟฝ๐ก๐ก๐๐๐น๐น ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐ผ๐ผ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 nS ฮฒ t Si nI failures tLI i ๏ฟฝ ฮฑ โ๏ฟฝ ฮฒ tRI i ๏ฟฝ ฮฑ โ๏ฟฝ ฮฒ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ + ๏ฟฝ ln ๏ฟฝe โe ๏ฟฝ ฮฑ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1๏ฟฝ๏ฟฝ๏ฟฝ i=1 survivors interval failures Weibull 2 1 ๐ผ๐ผ 2 ๏ฟฝฮ ๏ฟฝ1 + ๏ฟฝ โ ฮ 2 ๏ฟฝ1 + ๏ฟฝ๏ฟฝ ๐ฝ๐ฝ ๐ฝ๐ฝ Variance - 2nd Central Moment 62 Common Life Distributions โฮ =0 โฮฑ solve for ๐ผ๐ผ to get ๐ผ๐ผ๏ฟฝ: nS nF โฮ โฮฒnF ฮฒ ฮฒ ฮฒ ฮฒ = + ฮฒ+1 ๏ฟฝ๏ฟฝt Fi ๏ฟฝ + ฮฒ+1 ๏ฟฝ๏ฟฝt Si ๏ฟฝ โฮฑ ฮฑ ฮฑ ฮฑ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 ๏ฟฝ๏ฟฝ๏ฟฝ failures ฮฒ tRI ๏ฟฝi ๏ฟฝ t LI ฮฑ i interval failures solve for ๐ฝ๐ฝ to get ๐ฝ๐ฝฬ : nF nS ฮฒ ฮฒ t Fi t Fi t Fi t Si t Si โฮ nF = + ๏ฟฝ ๏ฟฝln ๏ฟฝ ๏ฟฝ โ ๏ฟฝ ๏ฟฝ . ln ๏ฟฝ ๏ฟฝ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ln ๏ฟฝ ๏ฟฝ ฮฒ ฮฑ ฮฑ ๐ผ๐ผ ฮฑ ๐ผ๐ผ โฮฒ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 i=1 t RI i MLE Point Estimates survivors ฮฒ ฮฒ tLI ๏ฟฝi ๏ฟฝ t RI ฮฑ i nI ๏ฟฝ ๏ฟฝ e โ๏ฟฝ ๏ฟฝ e โ ฮฑ ฮฒโ ฮฑ โ=0 +๏ฟฝ โ ฮฒ ฮฒ RI LI โ ฮฑโ t t ๏ฟฝi ๏ฟฝ ๏ฟฝi ๏ฟฝ i=1 e ฮฑ โe ฮฑ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ โ โ Weibull โฮ =0 โฮฒ ฮฒ failures ฮฒ t RI i tLI i ๏ฟฝ ฮฑ ๏ฟฝ ฮฒ survivors ฮฒ ฮฒ tRI ๏ฟฝi ๏ฟฝ t LI ฮฑ i t LI i nI โ ln ๏ฟฝ ๏ฟฝ . ๏ฟฝ ๏ฟฝ . e โln ๏ฟฝ ๐ผ๐ผ ๏ฟฝ . ๏ฟฝ ฮฑ ๏ฟฝ . e โ ๐ผ๐ผ ฮฑ โ=0 +๏ฟฝโ ฮฒ ฮฒ RI LI โ โ t t ๏ฟฝi ๏ฟฝ ๏ฟฝi ๏ฟฝ i=1 e ฮฑ โe ฮฑ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ โ โ interval failures When there is only complete failure and/or right censored data the point estimates can be solved using (Rinne 2008, p.439): 1 ๏ฟฝ ฮฒ ๏ฟฝ ฮฒ ๏ฟฝ ฮฒ ฮฒ๏ฟฝ = ๏ฟฝ ๏ฟฝ ฮฒ โ๏ฟฝt Fi ๏ฟฝ + โ๏ฟฝt Si ๏ฟฝ ๏ฟฝ=๏ฟฝ ฮฑ ๏ฟฝ nF ๏ฟฝ ฮฒ โ๏ฟฝt Fi ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ + โ๏ฟฝt Si ๏ฟฝ ln๏ฟฝt Si ๏ฟฝ ๏ฟฝ ฮฒ โ๏ฟฝt Fi ๏ฟฝ + โ๏ฟฝt Si ๏ฟฝ ๏ฟฝ ฮฒ 1 โ ๏ฟฝ ln(t Fi )๏ฟฝ ๐๐๐น๐น โ1 Note: Numerical methods are needed to solve ฮฒ๏ฟฝ then substitute to find ๏ฟฝ. Numerical methods to find Weibull MLE estimates for complete and ฮฑ censored data for 2 parameter and 3 parameter Weibull distribution are detailed in (Rinne 2008). Fisher Information Matrix (Rinne p.412) 2008, ๐ฝ๐ฝ2 ๐ฝ๐ฝ2 ฮ โฒ (2) โก โค โก ๐ผ๐ผ 2 ๐ผ๐ผ 2 โ๐ผ๐ผ โฅ โข ๐ผ๐ผ(๐ผ๐ผ, ๐ฝ๐ฝ) = โข โฒ โฒโฒ (2) = โข (2) ฮ 1 + ฮ โข โฅ โข1 โ ๐พ๐พ โฃ โ๐ผ๐ผ ๐ฝ๐ฝ2 โฆ โฃ ๐ผ๐ผ 1 โ ๐พ๐พ ๐ผ๐ผ โค โฅ ๐๐ 2 2 + (1 โ ๐พ๐พ ) โฅ 6 โฅ โฆ ๐ฝ๐ฝ2 ๐ฝ๐ฝ2 0.422784 โก โค 2 ๐ผ๐ผ โ๐ผ๐ผ โฅ โ โข โข0.422784 1.823680 โฅ โฃ โ๐ผ๐ผ ๐ฝ๐ฝ2 โฆ Weibull Continuous Distribution 100๐พ๐พ% Confidence Interval (complete data) 63 Bayesian Bayesian analysis is applied to either one of two re-parameterizations of the Weibull Distribution: (Rinne 2008, p.517) ๐๐(๐ก๐ก; ๐๐, ๐ฝ๐ฝ) = ๐๐๐๐๐ก๐ก ๐ฝ๐ฝโ1 exp๏ฟฝโ๐๐๐ก๐ก ๐ฝ๐ฝ ๏ฟฝ or ๐๐(๐ก๐ก; ๐๐, ๐ฝ๐ฝ) = Type ๐ฝ๐ฝ ๐ฝ๐ฝโ1 ๐ก๐ก ๐ฝ๐ฝ ๐ก๐ก exp ๏ฟฝโ ๏ฟฝ ๐๐ ๐๐ where ๐๐ = ๐ผ๐ผ โ๐ฝ๐ฝ where ๐๐ = 1 = ๐ผ๐ผ ๐ฝ๐ฝ ๐๐ Non-informative Priors ๐ ๐ ๐๐ (๐๐) (Rinne 2008, p.517) Prior 1 ๐๐ โ ๐๐ Uniform Proper Prior with known ๐ฝ๐ฝ and limits ๐๐ โ [๐๐, ๐๐] Jeffreyโs Prior when ๐ฝ๐ฝ is known. Jeffreyโs Prior for unknown ๐๐ and ๐ฝ๐ฝ. where Posterior Truncated Gamma Distribution For a โค ฮป โค b ๐๐. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + nF , t T,ฮฒ ) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0) ๐๐ ฮฒ 1 ๐๐๐๐ Otherwise ๐๐(๐๐) = 0 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; nF , t T,ฮฒ ) when ๐๐ โ [0, โ) No closed form (Rinne 2008, p.527) ฮฒ ๐ก๐ก ๐๐,๐ฝ๐ฝ = โ๏ฟฝt Fi ๏ฟฝ + โ๏ฟฝt Si ๏ฟฝ = adjusted total time in test Conjugate Priors It was found by Soland that no joint continuous prior distribution exists for the Weibull distribution. Soland did however propose a procedure which used a continuous distribution for ฮฑ and a discrete distribution for ฮฒ which will not be included here. (Martz & Waller 1982) UOI ๐๐ where ๐๐ = ๐ผ๐ผ โ๐ฝ๐ฝ from ๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Likelihood Model Evidence Weibull with known ๐ฝ๐ฝ ๐๐๐น๐น failures at times ๐ก๐ก๐๐๐น๐น Dist of UOI Gamma Prior Para Posterior Parameters ๐๐0 , ฮ0 ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ฮ = ฮ0 + ๐ก๐ก ๐๐,๐ฝ๐ฝ (Rinne 2008, p.520) Weibull The asymptotic variance-covariance matrix of (๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ) is: (Rinne 2008, pp.412-417) ๐ผ๐ผ๏ฟฝ 2 1 1.1087 0.2570๐ผ๐ผ๏ฟฝ โ1 ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ๏ฟฝ = ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ๏ฟฝ๏ฟฝ = ๏ฟฝ ๏ฟฝ ๐ฝ๐ฝฬ2 ๐๐๐น๐น 0.2570๐ผ๐ผ๏ฟฝ 0.6079๐ฝ๐ฝฬ2 Weibull 64 Common Life Distributions ๐๐ where ๐๐ = ๐ผ๐ผ ๐ฝ๐ฝ from ๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Weibull with known ๐ฝ๐ฝ Example ๐๐๐น๐น failures at times ๐ก๐ก๐๐๐น๐น Inverted Gamma ๐ผ๐ผ = ๐ผ๐ผ๐๐ + ๐๐๐น๐น ฮฒ = ฮฒ0 + ๐ก๐ก ๐๐,๐ฝ๐ฝ (Rinne 2008, p.524) ๐ผ๐ผ0 , ฮฒ0 Description , Limitations and Uses 5 components are put on a test with the following failure times: 535, 613, 976, 1031, 1875 hours ฮฒ๏ฟฝ is found by numerically solving: ฮฒ๏ฟฝ = ๏ฟฝ ๏ฟฝ is found by solving: ฮฑ Covariance Matrix is: ๏ฟฝ ฮฒ โ๏ฟฝt Fi ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ โ๏ฟฝt Fi ๏ฟฝ ๏ฟฝ ฮฒ โ1 โ 6.8118๏ฟฝ ฮฒ๏ฟฝ = 2.275 ๏ฟฝ=๏ฟฝ ฮฑ 1 ๏ฟฝ ฮฒ ๏ฟฝ ฮฒ F โ๏ฟฝt i ๏ฟฝ ๐ผ๐ผ๏ฟฝ 2 1 1.1087 ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ๏ฟฝ = ๏ฟฝ ๐ฝ๐ฝฬ2 5 0.2570๐ผ๐ผ๏ฟฝ nF ๏ฟฝ = 1140 0.2570๐ผ๐ผ๏ฟฝ 0.6079๐ฝ๐ฝฬ2 ๏ฟฝ=๏ฟฝ 55679 58.596 ๏ฟฝ 58.596 0.6293 90% confidence interval for ๐ผ๐ผ๏ฟฝ: ๐ท๐ทโ1 (0.95)โ55679 ๏ฟฝ๐ผ๐ผ๏ฟฝ. exp ๏ฟฝ ๏ฟฝ, โ๐ผ๐ผ๏ฟฝ [811, 1602] 90% confidence interval for ๐ฝ๐ฝ: ๐ท๐ท โ1 (0.95)โ0.6293 ๏ฟฝ๐ฝ๐ฝฬ. exp ๏ฟฝ ๏ฟฝ, โ๐ฝ๐ฝฬ [1.282, ๐ท๐ท โ1 (0.95)โ0.6293 ๐ฝ๐ฝฬ . exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ฝ๐ฝฬ 4.037] ๐ผ๐ผ๏ฟฝ. exp ๏ฟฝ ๐ท๐ท โ1 (0.95)โ55679 ๏ฟฝ๏ฟฝ ๐ผ๐ผ๏ฟฝ Note that with only 5 samples the assumption that the parameter distribution is approximately normal is probably inaccurate and therefore the confidence intervals need to be used with caution. Characteristics The Weibull distribution is also known as a โType III asymptotic distribution for minimum valuesโ. ฮฒ Characteristics: ฮฒ < 1. The hazard rate decreases with time. Weibull Continuous Distribution 65 Note that for ๐ฝ๐ฝ = 0.999, ๐๐(0) = โ, but for ๐ฝ๐ฝ = 1.001, ๐๐(0) = 0. This rapid change creates complications when maximizing likelihood functions. (Weibull.com) As ๐ฝ๐ฝ โ โ, the ๐๐๐๐๐๐๐๐ โ ๐ผ๐ผ. ฮฑ Characteristics. Increasing ๐ผ๐ผ stretches the distribution over the time scale. With the ๐๐(0) point fixed this also has the effect of increasing the mode, mean and median. The value for ๐ผ๐ผ is at the 63% Percentile. ๐น๐น(๐ผ๐ผ) = 0.632.. ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ผ๐ผ, ๐ฝ๐ฝ) Scaling property: (Leemis & McQueston 2008) ๐๐๐๐~๐๐๐๐๐๐๐๐๐๐๐๐๐๐๏ฟฝ๐ผ๐ผ๐๐๐ฝ๐ฝ , ๐ฝ๐ฝ๏ฟฝ Minimum property (Rinne 2008, p.107) min{๐๐, ๐๐2 , โฆ , ๐๐๐๐ }~๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ผ๐ผ๐๐ When ๐ฝ๐ฝ is fixed. Variate Generation property 1 โ ๐ฝ๐ฝ , ๐ฝ๐ฝ) 1 ๐น๐น โ1 (๐ข๐ข) = ๐ผ๐ผ[โ ln(1 โ ๐ข๐ข)]๐ฝ๐ฝ , 0 < ๐ข๐ข < 1 Lognormal versus Weibull. In analyzing life data to these distributions it is often the case that both may be a good fit, especially in the middle of the distribution. The Weibull distribution has an earlier lower tail and produces a more pessimistic estimate of the component life. (Nelson 1990, p.65) Applications The Weibull distribution is by far the most popular life distribution used in reliability engineering. This is due to its variety of shapes and generalization or approximation of many other distributions. Analysis assuming a Weibull distribution already includes the exponential life Weibull ฮฒ = 1. The hazard rate is constant (exp distribution) ฮฒ > 1. The hazard rate increases with time. 1 < ฮฒ < 2. The hazard rate increases less as time increases. ฮฒ = 2. The hazard rate increases with a linear relationship to time. ฮฒ > 2. The hazard rate increases more as time increases. ฮฒ < 3.447798. The distribution is positively skewed. (Tail to right). ฮฒ โ 3.447798. The distribution is approximately symmetrical. ฮฒ > 3.447798. The distribution is negatively skewed (Tail to left). 3 < ฮฒ < 4. The distribution approximates a normal distribution. ฮฒ > 10. The distribution approximates a Smallest Extreme Value Distribution. 66 Common Life Distributions distribution as a special case. Weibull There are many physical interpretations of the Weibull Distribution. Due to its minimum property a physical interpretation is the weakest link, where a system such as a chain will fail when the weakest link fails. It can also be shown that the Weibull Distribution can be derived from a cumulative wear model (Rinne 2008, p.15) The following is a non-exhaustive list of applications where the Weibull distribution has been used in: โข Acceptance sampling โข Warranty analysis โข Maintenance and renewal โข Strength of material modeling โข Wear modeling โข Electronic failure modeling โข Corrosion modeling A detailed list with references to practical examples is contained in (Rinne 2008, p.275) Resources Online: http://www.weibull.com/LifeDataWeb/the_weibull_distribution.htm http://mathworld.wolfram.com/WeibullDistribution.html http://en.wikipedia.org/wiki/Weibull_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web calculator) http://www.qualitydigest.com/jan99/html/weibull.html (how to use conduct Weibull analysis in Excel, William W. Dorner) Books: Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC. Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience. Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience. Relationship to Other Distributions Three Parameter Weibull Distribution ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ, ๐พ๐พ) Exponential The three parameter model adds a locator parameter to the two parameter Weibull distribution allowing a shift along the x-axis. This creates a period of guaranteed zero failures to the beginning of the product life and is therefore only used in special cases. Special Case: ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ, ๐พ๐พ = 0) Let Weibull Continuous Distribution Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐ก๐ก; ๐ผ๐ผ) ฯ Distribution ๐๐(๐ก๐ก|๐ฃ๐ฃ) Special Case: Special Case: ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐๐๐๐๐ ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(ฮป = ๐ผ๐ผ โ๐ฝ๐ฝ ) ๐๐ = X ฮฒ 1 ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐ก๐ก; ๐ผ๐ผ = , ๐ฝ๐ฝ = 1๏ฟฝ ๐๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐ก๐ก; ๐ผ๐ผ) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ = 2) Special Case: ๐๐(๐ก๐ก|๐ฃ๐ฃ = 2) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๏ฟฝ๐ก๐ก|๐ผ๐ผ = โ2, ๐ฝ๐ฝ = 2๏ฟฝ Weibull Rayleigh Distribution Then 67 68 Probability Distributions Used in Reliability Engineering 3. Bathtub Life Distributions 2-Fold Mixed Weibull Distribution 69 3.1. 2-Fold Mixed Weibull Distribution All shapes shown are variations from ๐๐ = 0.5 ๐ผ๐ผ1 = 2 ๐ฝ๐ฝ1 = 0.5 ๐ผ๐ผ2 = 10 ๐ฝ๐ฝ2 = 20 Probability Density Function - f(t) ฮฑ1=2 ฮฒ1=0.1 ฮฑ1=2 ฮฒ1=0.5 ฮฑ1=2 ฮฒ1=1 ฮฑ1=2 ฮฒ1=5 ฮฑ2=10 ฮฒ2=20 ฮฑ2=10 ฮฒ2=10 ฮฑ2=10 ฮฒ2=1 ฮฑ2=5 ฮฒ2=10 p=0 p=0.3 p=0.7 p=1 Mixed Wbl 0 5 10 0 5 10 0 5 10 5 10 0 10 10 0 10 Cumulative Density Function - F(t) 0 5 10 0 10 0 Hazard Rate - h(t) 0 5 70 Bathtub Life Distributions Parameters & Description Parameters Limits ๐ผ๐ผ๐๐ ๐ผ๐ผ๐๐ > 0 ๐๐ 0 โค ๐๐ โค 1 ๐ฝ๐ฝ๐๐ Scale Parameter: This is the scale for each Weibull Distribution. Shape Parameters: The shape of each Weibull Distribution ๐ฝ๐ฝ๐๐ > 0 Mixing Parameter. This determines the weight each Weibull Distribution has on the overall density function. ๐ก๐ก โฅ 0 Mixed Wbl Distribution Formulas ๐๐(๐ก๐ก) = ๐๐๐๐1 (๐ก๐ก) + (1 โ ๐๐)๐๐2 (๐ก๐ก) PDF where ๐๐๐๐ (๐ก๐ก) = ๐ฝ๐ฝ๐๐ ๐ฝ๐ฝ๐๐ ๐ก๐ก ๐ฝ๐ฝ๐๐โ1 โ๏ฟฝ๐ผ๐ผ๐ก๐ก ๏ฟฝ ๐๐ ๐๐ ๐ผ๐ผ๐๐ ๐ฝ๐ฝ๐๐ and ๐๐ โ {1,2} ๐น๐น(๐ก๐ก) = ๐๐๐น๐น1 (๐ก๐ก) + (1 โ ๐๐)๐น๐น2 (๐ก๐ก) CDF where ๐น๐น๐๐ (๐ก๐ก) = 1 โ ๐๐ โ๏ฟฝ ๐ก๐ก ๐ฝ๐ฝ๐๐ ๏ฟฝ ๐ผ๐ผ๐๐ and ๐๐ โ {1,2} ๐ ๐ (๐ก๐ก) = ๐๐๐ ๐ 1 (๐ก๐ก) + (1 โ ๐๐)๐ ๐ 2 (๐ก๐ก) Reliability where ๐ ๐ ๐๐ (๐ก๐ก) = ๐๐ Hazard Rate where โ๏ฟฝ ๐ก๐ก ๐ฝ๐ฝ๐๐ ๏ฟฝ ๐ผ๐ผ๐๐ and ๐๐ โ {1,2} โ(๐ก๐ก) = ๐ค๐ค1 (๐ก๐ก)โ1 (๐ก๐ก) + ๐ค๐ค2 (๐ก๐ก)โ2 (๐ก๐ก) ๐ค๐ค๐๐ (๐ก๐ก) = ๐๐๐๐ ๐ ๐ ๐๐ (๐ก๐ก) โ๐๐๐๐=1 ๐๐๐๐ ๐ ๐ ๐๐ (๐ก๐ก) and ๐๐ โ {1,2} Properties and Moments Median Solved numerically Mode Mean - Solved numerically 1st Raw Moment ๐๐๐ผ๐ผ1 ฮ ๏ฟฝ1 + Variance - 2nd Central Moment 1 1 ๏ฟฝ + (1 โ p)๐ผ๐ผ2 ฮ ๏ฟฝ1 + ๏ฟฝ ๐ฝ๐ฝ1 ๐ฝ๐ฝ2 ๐๐. ๐๐๐๐๐๐[๐๐1 ] + (1 โ ๐๐)๐๐๐๐๐๐[๐๐2 ] +๐๐(๐ธ๐ธ[๐๐1 ] โ ๐ธ๐ธ[๐๐])2 +(1 โ ๐๐)(๐ธ๐ธ[๐๐2 ] โ ๐ธ๐ธ[๐๐])2 ๐๐. ๐ผ๐ผ 2 ๏ฟฝฮ ๏ฟฝ1 + 2 ๐ฝ๐ฝ1 ๏ฟฝ โ ฮ 2 ๏ฟฝ1 + 1 ๐ฝ๐ฝ1 ๏ฟฝ๏ฟฝ 2-Fold Mixed Weibull Distribution +(1 โ ๐๐)๐ผ๐ผ 2 ๏ฟฝฮ ๏ฟฝ1 + +๐๐ ๏ฟฝ๐ผ๐ผ1 ฮ ๏ฟฝ1 + 1 ๐ฝ๐ฝ1 ๏ฟฝ โ ฮ 2 ๏ฟฝ1 + ๏ฟฝ โ ๐ธ๐ธ[๐๐]๏ฟฝ +(1 โ ๐๐) ๏ฟฝ๐ผ๐ผ2 ฮ ๏ฟฝ1 + 100p% Percentile Function 2 ๐ฝ๐ฝ2 1 ๐ฝ๐ฝ2 71 2 ๏ฟฝ โ ๐ธ๐ธ[๐๐]๏ฟฝ 1 ๐ฝ๐ฝ2 ๏ฟฝ๏ฟฝ 2 Solved numerically Parameter Estimation Plotting Method (Jiang & Murthy 1995) X-Axis Y-Axis ๐ฅ๐ฅ = ๐๐๐๐(๐ก๐ก) 1 ๐ฆ๐ฆ = ๐๐๐๐ ๏ฟฝ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ 1 โ ๐น๐น Using the Weibull Probability Plot the parameters can be estimated. Jiang & Murthy, 1995, provide a comprehensive coverage of this procedure and detail error in previous methods. A typical WPP for a 2-fold Mixed Weibull Distribution is: 2 1.5 1 0.5 0 -0.5 -1 -1.5 -2 -1 0 1 2 3 4 5 6 WPP for 2-Fold Weibull Mixture Model ๐๐ = 12, ๐ผ๐ผ1 = 5, ๐ฝ๐ฝ1 = 0.5, ๐ผ๐ผ2 = 10, ๐ฝ๐ฝ2 = 5 Sub Populations: The dotted lines in the WPP is the lines representing the subpopulations: ๐ฟ๐ฟ1 = ๐ฝ๐ฝ1 [๐ฅ๐ฅ โ ln(๐ผ๐ผ1 )] ๐ฟ๐ฟ2 = ๐ฝ๐ฝ2 [๐ฅ๐ฅ โ ln(๐ผ๐ผ2 )] Mixed Wbl Plot Points on a Weibull Probability Plot 72 Bathtub Life Distributions Asymptotes (Jiang & Murthy 1995): As ๐ฅ๐ฅ โ โโ (๐ก๐ก โ 0) there exists an asymptote approximated by: ๐ฆ๐ฆ โ ๐ฝ๐ฝ1 [๐ฅ๐ฅ โ ln(๐ผ๐ผ1 )] + ln(๐๐) where ๐๐ ๐ค๐คโ๐๐๐๐ ๐ฝ๐ฝ1 โ ๐ฝ๐ฝ2 ๐ผ๐ผ1 ๐ฝ๐ฝ1 ๐๐ = ๏ฟฝ ๐๐ + (1 โ ๐๐). ๏ฟฝ ๏ฟฝ ๐ค๐คโ๐๐๐๐ ๐ฝ๐ฝ1 = ๐ฝ๐ฝ2 ๐ผ๐ผ2 Mixed Wbl As ๐ฅ๐ฅ โ โ (๐ก๐ก โ โ) the asymptote straight line can be approximated by: ๐ฆ๐ฆ โ ๐ฝ๐ฝ1 [๐ฅ๐ฅ โ ln(๐ผ๐ผ1 )] Parameter Estimation Jiang and Murthy divide the parameter estimation procedure into three cases: Well Mixed Case ๐ท๐ท๐๐ โ ๐ท๐ท๐๐ ๐๐๐๐๐๐ ๐ถ๐ถ๐๐ โ ๐ถ๐ถ๐๐ - Estimate the parameters of ๐ผ๐ผ1 and ๐ฝ๐ฝ1 from the ๐ฟ๐ฟ1 line (right asymptote). - Estimate the parameter ๐๐ from the separation distance between the left and right asymptotes. - Find the point where the curve crosses ๐ฟ๐ฟ1 (point I). The slope at point I is: ๐ฝ๐ฝฬ = ๐๐๐ฝ๐ฝ1 + (1 โ ๐๐)๐ฝ๐ฝ2 - Determine slope at point I and use to estimate ๐ฝ๐ฝ2 - Draw a line through the intersection point I with slope ๐ฝ๐ฝ2 and use the intersection point to estimate ๐ผ๐ผ2 . Well Separated Case ๐ท๐ท๐๐ โ ๐ท๐ท๐๐ ๐๐๐๐๐๐ ๐ถ๐ถ๐๐ โซ ๐ถ๐ถ๐๐ ๐๐๐๐ ๐ถ๐ถ๐๐ โช ๐ถ๐ถ๐๐ - Determine visually if data is scattered along the bottom (or top) to determine if ๐ผ๐ผ1 โช ๐ผ๐ผ2 (or ๐ผ๐ผ1 โซ ๐ผ๐ผ2 ). - If ๐ผ๐ผ1 โช ๐ผ๐ผ2 (๐ผ๐ผ1 โซ ๐ผ๐ผ2 ) locate the inflection, ๐ฆ๐ฆ๐๐ , to the left (right) of the point I. This point ๐ฆ๐ฆ๐๐ โ ln[โ ln(1 โ ๐๐)] { or ๐ฆ๐ฆ๐๐ โ ln[โ ln(๐๐)] }. Using this formula estimate p. - Estimate ๐ผ๐ผ1 and ๐ผ๐ผ2 : ๐๐ 1โ๐๐ โข If ๐ผ๐ผ1 โช ๐ผ๐ผ2 calculate point ๐ฆ๐ฆ1 = ln ๏ฟฝ๐๐๐๐ ๏ฟฝ1 โ ๐๐ + ๏ฟฝ๏ฟฝ and ๐ฆ๐ฆ2 = ln ๏ฟฝ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ. ๐๐๐๐๐๐(1) โข ๐๐๐๐๐๐(1) Find the coordinates where ๐ฆ๐ฆ1 and ๐ฆ๐ฆ2 intersect the WPP curve. At these points estimate ๐ผ๐ผ1 = ๐๐ ๐ฅ๐ฅ1 and ๐ผ๐ผ2 = ๐๐ ๐ฅ๐ฅ2 . ๐๐ 1โ๐๐ If ๐ผ๐ผ1 โซ ๐ผ๐ผ2 calculate point ๐ฆ๐ฆ1 = ln ๏ฟฝโ ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ and ๐ฆ๐ฆ2 = ln ๏ฟฝโ๐๐๐๐ ๏ฟฝ๐๐ + ๏ฟฝ๏ฟฝ. ๐๐๐๐๐๐(1) ๐๐๐๐๐๐(1) Find the coordinates where ๐ฆ๐ฆ1 and ๐ฆ๐ฆ2 intersect the WPP curve. At these points estimate ๐ผ๐ผ1 = ๐๐ ๐ฅ๐ฅ1 and ๐ผ๐ผ2 = ๐๐ ๐ฅ๐ฅ2 . - Estimate ๐ฝ๐ฝ1 : โข If ๐ผ๐ผ1 โช ๐ผ๐ผ2 draw and approximate ๐ฟ๐ฟ2 ensuring it intersects ๐ผ๐ผ2 . Estimate ๐ฝ๐ฝ2 from the slope of ๐ฟ๐ฟ2 . โข If ๐ผ๐ผ1 โซ ๐ผ๐ผ2 draw and approximate ๐ฟ๐ฟ1 ensuring it intersects ๐ผ๐ผ1 . Estimate ๐ฝ๐ฝ1 from the slope of ๐ฟ๐ฟ1 . - Find the point where the curve crosses ๐ฟ๐ฟ1 (point I). The slope at point I is: ๐ฝ๐ฝฬ = ๐๐๐ฝ๐ฝ1 + (1 โ ๐๐)๐ฝ๐ฝ2 - Determine slope at point I and use to estimate ๐ฝ๐ฝ2 2-Fold Mixed Weibull Distribution 73 Common Shape Parameter ๐ท๐ท๐๐ = ๐ท๐ท๐๐ ๐ผ๐ผ If ๏ฟฝ 2๏ฟฝ ๐ผ๐ผ1 ๐ฝ๐ฝ1 โ1 then: - Estimate the parameters of ๐ผ๐ผ1 and ๐ฝ๐ฝ1 from the ๐ฟ๐ฟ1 line (right asymptote). - Estimate the parameter ๐๐ from the separation distance between the left and right asymptotes. - Draw a vertical line through ๐ฅ๐ฅ = ln(๐ผ๐ผ1 ). The intersection with the WPP can yield an estimate of ๐ผ๐ผ2 using: ๐๐ + exp(1) ๐ฆ๐ฆ1 = ๏ฟฝ ๐ผ๐ผ1 ๐ฝ๐ฝ1 โช1 then: - Find inflection point and estimate the y coordinate ๐ฆ๐ฆ๐๐ . Estimate p using: ๐ฆ๐ฆ๐๐ โ ln[โ ln(๐๐)] ๐๐ 1โ๐๐ - If ๐ผ๐ผ1 โช ๐ผ๐ผ2 calculate point ๐ฆ๐ฆ1 = ln ๏ฟฝ๐๐๐๐ ๏ฟฝ1 โ ๐๐ + ๏ฟฝ๏ฟฝ and ๐ฆ๐ฆ2 = ln ๏ฟฝ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ. Find the ๐๐๐๐๐๐(1) ๐๐๐๐๐๐(1) coordinates where ๐ฆ๐ฆ1 and ๐ฆ๐ฆ2 intersect the WPP curve. At these points estimate ๐ผ๐ผ1 = ๐๐ ๐ฅ๐ฅ1 and ๐ผ๐ผ2 = ๐๐ ๐ฅ๐ฅ2 . - Using the left or right asymptote estimate ๐ฝ๐ฝ1 = ๐ฝ๐ฝ2 from the slope. Maximum Likelihood Bayesian MLE and Bayesian techniques can be used using numerical methods however estimates obtained from the graphical methods are useful for initial guesses. A literature review of MLE and Bayesian methods is covered in (Murthy et al. 2003). Description , Limitations and Uses Characteristics Hazard Rate Shape. The hazard rate can be approximated at its limits by (Jiang & Murthy 1995): ๐๐๐๐๐๐๐๐๐๐ ๐ก๐ก: โ(๐ก๐ก) โ ๐๐โ1 (๐ก๐ก) ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ ๐ก๐ก: โ(๐ก๐ก) โ โ1 This result proves that the hazard rate (increasing or decreasing) of โ1 will dominate the limits of the mixed Weibull distribution. Therefore the hazard rate cannot be a bathtub curve shape. Instead the possible shapes of the hazard rate is: โข Decreasing โข Unimodal โข Decreasing followed by unimodal (rollercoaster) โข Bi-modal The reason this distribution has been included as a bathtub distribution is because on many occasions the hazard rate of a complex product may follow the โrollercoasterโ shape instead which is given as decreasing followed by unimodal shape. The shape of the hazard rate is only determined by the two shape parameters ๐ฝ๐ฝ1 and ๐ฝ๐ฝ2 . A complete study on the characterization of Mixed Wbl ๐ผ๐ผ If ๏ฟฝ 2๏ฟฝ 1 โ ๐๐ ๏ฟฝ ๐ผ๐ผ ๐ฝ๐ฝ1 exp ๏ฟฝ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ ๐ผ๐ผ1 74 Bathtub Life Distributions the 2-Fold Mixed Weibull Distribution is contained in Jiang and Murthy 1998. p Values The mixture ratio, ๐๐๐๐ , for each Weibull Distribution may be used to estimate the percentage of each subpopulation. However this is not a reliable measure and it known to be misleading (Berger & Sellke 1987) N-Fold Distribution (Murthy et al. 2003) A generalization to the 2-fold mixed Weibull distribution is the n-fold case. This distribution is defined as: ๐๐ Mixed Wbl ๐๐(๐ก๐ก) = ๏ฟฝ ๐๐๐๐ ๐๐๐๐ (๐ก๐ก) ๐๐=1 ๐ฝ๐ฝ ๐๐ ๐ฝ๐ฝ๐๐ ๐ก๐ก ๐ฝ๐ฝ๐๐ โ1 โ๏ฟฝ๐ผ๐ผ๐ก๐ก ๏ฟฝ ๐๐ where ๐๐๐๐ (๐ก๐ก) = ๐๐ ๐๐ ๐๐๐๐๐๐ ๏ฟฝ ๐๐๐๐ = 1 ๐ผ๐ผ๐๐ ๐ฝ๐ฝ๐๐ and the hazard rate is given as: ๐๐ ๐๐=1 โ(๐ก๐ก) = ๏ฟฝ ๐ค๐ค๐๐ (๐ก๐ก)โ๐๐ (๐ก๐ก) ๐ค๐คโ๐๐๐๐๐๐ ๐๐=1 ๐ค๐ค๐๐ (๐ก๐ก) = ๐๐๐๐ ๐ ๐ ๐๐ (๐ก๐ก) โ๐๐๐๐=1 ๐๐๐๐ ๐ ๐ ๐๐ (๐ก๐ก) It has been found that in many instances a higher number of folds will not significantly increase the accuracy of the model but does impose a significant overhead in the number of parameters to estimate. The 3-Fold Weibull Mixture Distribution has been studied by Jiang and Murthy 1996. 2-Fold Weibull 3-Parameter Distribution A common variation to the model presented here is to have the second Weibull distribution modeled with three parameters. Resources Books / Journals: Jiang, R. & Murthy, D., 1995. Modeling Failure-Data by Mixture of 2 Weibull Distributions : A Graphical Approach. IEEE Transactions on Reliability, 44, 477-488. Murthy, D., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience. Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC. Jiang, R. & Murthy, D., 1996. A mixture model involving three Weibull distributions. In Proceedings of the Second Australiaโ Japan Workshop on Stochastic Models in Engineering, Technology and Management. Gold Coast, Australia, pp. 260-270. 2-Fold Mixed Weibull Distribution 75 Jiang, R. & Murthy, D., 1998. Mixture of Weibull distributions parametric characterization of failure rate function. Applied Stochastic Models and Data Analysis, (14), 47-65. Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20: Advances in Reliability 1st ed., Elsevier Science & Technology. Relationship to Other Distributions Weibull Distribution Mixed Wbl ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Special Case: ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) = 2๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น(๐ก๐ก; ๐ผ๐ผ = ๐ผ๐ผ1 , ๐ฝ๐ฝ = ๐ฝ๐ฝ1 , ๐๐ = 1) ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) = 2๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น(๐ก๐ก; ๐ผ๐ผ = ๐ผ๐ผ2 , ๐ฝ๐ฝ = ๐ฝ๐ฝ2 , ๐๐ = 0) 76 Bathtub Life Distributions 3.2. Exponentiated Weibull Distribution Probability Density Function - f(t) 0.30 ฮฑ=5 ฮฒ=1.5 v=0.2 ฮฑ=5 ฮฒ=1.5 v=0.5 ฮฑ=5 ฮฒ=1.5 v=0.8 ฮฑ=5 ฮฒ=1.5 v=3 0.25 Exp-Weibull 0.20 0.30 0.25 0.20 0.15 0.15 0.10 0.10 0.05 0.05 0.00 ฮฑ=5 ฮฒ=1 v=0.5 ฮฑ=5 ฮฒ=1.5 v=0.5 ฮฑ=5 ฮฒ=2 v=0.5 ฮฑ=5 ฮฒ=3 v=0.5 0.00 0 5 10 15 0 5 0 5 0 5 Cumulative Density Function - F(t) 1.00 1.00 0.80 0.80 0.60 0.60 0.40 0.40 0.20 0.20 0.00 0.00 0 5 10 15 Hazard Rate - h(t) 0.60 0.60 0.50 0.50 0.40 0.40 0.30 0.30 0.20 0.20 0.10 0.10 0.00 0.00 0 5 10 15 Exponentiated Weibull Continuous Distribution 77 Parameters & Description ๐ผ๐ผ > 0 Scale Parameter. Parameters ๐ผ๐ผ ๐ฃ๐ฃ > 0 Shape Parameter. Limits ๐ฃ๐ฃ ๐ฝ๐ฝ ๐ฝ๐ฝ > 0 Shape Parameter. ๐ก๐ก โฅ 0 Distribution Formulas PDF ๐ฝ๐ฝ๐ฝ๐ฝ๐ก๐ก ๐ฝ๐ฝโ1 ๐ก๐ก ๐ฝ๐ฝ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ฝ๐ฝ ๐ผ๐ผ ๐ผ๐ผ ๐ฃ๐ฃโ1 = ๐ฃ๐ฃ{๐น๐น๐๐ (๐ก๐ก)}๐ฃ๐ฃโ1 ๐๐๐๐ (๐ก๐ก) ๐ก๐ก ๐ฝ๐ฝ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐ผ๐ผ Where ๐น๐น๐๐ (๐ก๐ก) and ๐๐๐๐ (๐ก๐ก) are the cdf and pdf of the two parameter Weibull distribution respectively. t ฮฒ ๐น๐น(๐ก๐ก) = ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ฮฑ = [๐น๐น๐๐ (๐ก๐ก)]๐ฃ๐ฃ CDF t ฮฒ R(t) = 1 โ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ฮฑ = 1 โ [๐น๐น๐๐ (๐ก๐ก)]๐ฃ๐ฃ Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life v ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = 1 โ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๐ฃ๐ฃ ๐ก๐ก + ๐ฅ๐ฅ ๐ฝ๐ฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ผ๐ผ ๐ฃ๐ฃ ๐ก๐ก ๐ฝ๐ฝ 1 โ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ผ๐ผ Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. Residual Hazard Rate ๐ ๐ (๐ก๐ก + x) = ๐ ๐ (๐ก๐ก) v ๐ข๐ข(๐ก๐ก) = โ(๐ก๐ก) = ๐ฝ๐ฝ๐ฝ๐ฝ๏ฟฝ๐ก๐ก๏ฟฝ๐ผ๐ผ๏ฟฝ ๐ฃ๐ฃ โ ๐ก๐ก ๐ฝ๐ฝ โซt ๏ฟฝ1 โ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ๐ผ๐ผ๏ฟฝ ๏ฟฝ๏ฟฝ ๏ฟฝ ๐๐๐๐ ๐ฃ๐ฃ ๐ฝ๐ฝโ1 ๐ก๐ก ๐ฝ๐ฝ 1 โ ๏ฟฝ1 โ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ผ๐ผ ๐ฝ๐ฝ ๐ฃ๐ฃโ1 ๏ฟฝ1 โ exp ๏ฟฝโ๏ฟฝ๐ก๐ก๏ฟฝ๐ผ๐ผ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ฝ๐ฝ ๐ฝ๐ฝ exp ๏ฟฝโ๏ฟฝ๐ก๐ก๏ฟฝ๐ผ๐ผ ๏ฟฝ ๏ฟฝ ๐ฃ๐ฃ 1 โ ๏ฟฝ1 โ exp ๏ฟฝโ๏ฟฝ๐ก๐ก๏ฟฝ๐ผ๐ผ๏ฟฝ ๏ฟฝ๏ฟฝ Exp-Weibull ๐๐(๐ก๐ก) = 78 Bathtub Life Distributions For small t: (Murthy et al. 2003, p.130) ๐ฝ๐ฝ๐ฝ๐ฝ ๐ก๐ก ๐ฝ๐ฝ๐ฝ๐ฝโ1 โ(๐ก๐ก) โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐ผ๐ผ ๐ผ๐ผ For large t: (Murthy et al. 2003, p.130) ๐ฝ๐ฝ ๐ก๐ก ๐ฝ๐ฝโ1 โ(๐ก๐ก) โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐ผ๐ผ ๐ผ๐ผ Properties and Moments Median ๐ผ๐ผ ๏ฟฝโ ln ๏ฟฝ1 โ 2 For ๐ฝ๐ฝ๐ฝ๐ฝ > 1 the mode can be approximated (Murthy et al. 2003, p.130): ๐ฃ๐ฃ 1 ๏ฟฝ๐ฝ๐ฝ(๐ฝ๐ฝ โ 8๐ฃ๐ฃ + 2๐ฝ๐ฝ๐ฝ๐ฝ + 9๐ฝ๐ฝ๐ฃ๐ฃ 2 ) 1 ๐ผ๐ผ ๏ฟฝ ๏ฟฝ โ 1 โ ๏ฟฝ๏ฟฝ 2 ๐ฝ๐ฝ๐ฝ๐ฝ ๐ฃ๐ฃ Mode Exp-Weibull 1๏ฟฝ โ1๏ฟฝ ๐ฃ๐ฃ ๏ฟฝ๏ฟฝ ๐ฝ๐ฝ Mean - 1st Raw Moment Solved numerically see Murthy et al. 2003, p.128 Variance - 2nd Central Moment 100๐๐% Percentile Function ๐ก๐ก๐๐ = ๐ผ๐ผ ๏ฟฝโ ln ๏ฟฝ1 โ ๐๐ Parameter Estimation 1๏ฟฝ 1๏ฟฝ ๐ฝ๐ฝ ๐ฃ๐ฃ ๏ฟฝ๏ฟฝ Plotting Method (Jiang & Murthy 1999) Plot Points on a Weibull Probability Plot X-Axis Y-Axis ๐ฅ๐ฅ = ๐๐๐๐(๐ก๐ก) ๐ฆ๐ฆ = ๐๐๐๐ ๏ฟฝ๐๐๐๐ ๏ฟฝ 1 ๏ฟฝ๏ฟฝ 1 โ ๐น๐น Using the Weibull Probability Plot the parameters can be estimated. (Jiang & Murthy 1999), provide a comprehensive coverage of this. A typical WPP for an exponentiated Weibull distribution is: 3 2.5 2 1.5 1 0.5 0 -0.5 -1 -1.5 -2 -1 -0.5 0 0.5 1 1.5 2 2.5 3 WPP for exponentiated Weibull distribution ๐ผ๐ผ = 5, ๐ฝ๐ฝ = 2, ๐ฃ๐ฃ = 0.4 Asymptotes (Jiang & Murthy 1999): As ๐ฅ๐ฅ โ โโ (๐ก๐ก โ 0) there exists an asymptote approximated by: Exponentiated Weibull Continuous Distribution 79 ๐ฆ๐ฆ โ ๐ฝ๐ฝ๐ฝ๐ฝ[๐ฅ๐ฅ โ ln(๐ผ๐ผ)] As ๐ฅ๐ฅ โ โ (๐ก๐ก โ โ) the asymptote straight line can be approximated by: ๐ฆ๐ฆ โ ๐ฝ๐ฝ[๐ฅ๐ฅ โ ln(๐ผ๐ผ)] Both asymptotes intersect the x-axis at ๐๐๐๐(๐ผ๐ผ) however both have different slopes unless ๐ฃ๐ฃ = 1 and the WPP is the same as a two parameter Weibull distribution. Parameter Estimation Plot estimates of the asymptotes ensuring they cross the x-axis at the same point. Use the right asymptote to estimate ๐ผ๐ผ and ๐ฝ๐ฝ. Use the left asymptote to estimate ๐ฃ๐ฃ. Bayesian MLE and Bayesian techniques can be used in the standard way however estimates obtained from the graphical methods are useful for initial guesses when using numerical methods to solve equations. A literature review of MLE and Bayesian methods is covered in (Murthy et al. 2003). Description , Limitations and Uses Characteristics PDF Shape: (Murthy et al. 2003, p.129) ๐ท๐ท๐ท๐ท <= 1. The pdf is monotonically decreasing, ๐๐(0) = โ. ๐ท๐ท๐ท๐ท = ๐๐. The pdf is monotonically decreasing, ๐๐(0) = 1/๐ผ๐ผ. ๐ท๐ท๐ท๐ท > 1. The pdf is unimodal. ๐๐(0) = 0. The pdf shape is determined by ๐ฝ๐ฝ๐ฝ๐ฝ in a similar way to the ๐ฝ๐ฝ for a two parameter Weibull distribution. Hazard Rate Shape: (Murthy et al. 2003, p.129) ๐ท๐ท โค ๐๐ and ๐ท๐ท๐ท๐ท โค ๐๐. The hazard rate is monotonically decreasing. ๐ท๐ท โฅ ๐๐ and ๐ท๐ท๐ท๐ท โฅ ๐๐. The hazard rate is monotonically increasing. ๐ท๐ท < 1 and ๐ท๐ท๐ท๐ท > 1. The hazard rate is unimodal. ๐ท๐ท > 1 and ๐ท๐ท๐ท๐ท < 1. The hazard rate is a bathtub curve. Weibull Distribution. The Weibull distribution is a special case of the expatiated distribution when ๐ฃ๐ฃ = 1. When ๐ฃ๐ฃ is an integer greater than 1, then the cdf represents a multiplicative Weibull model. Standard Exponentiated Weibull. (Xie et al. 2004) When ๐ผ๐ผ = 1 the distribution is the standard exponentiated Weibull distribution with cdf: ๐ฃ๐ฃ ๐น๐น(๐ก๐ก) = ๏ฟฝ1 โ exp๏ฟฝโ๐ก๐ก ๐ฝ๐ฝ ๏ฟฝ๏ฟฝ Minimum Failure Rate. (Xie et al. 2004) When the hazard rate is a bathtub curve (๐ฝ๐ฝ > 1 and ๐ฝ๐ฝ๐ฝ๐ฝ < 1) then the minimum failure rate point is: ๐ก๐ก โฒ = ๐ผ๐ผ[โ ln(1 โ ๐ฆ๐ฆ1 )] 1๏ฟฝ ๐ฝ๐ฝ Exp-Weibull Maximum Likelihood 80 Bathtub Life Distributions where ๐ฆ๐ฆ1 is the solution to: (๐ฝ๐ฝ โ 1)๐ฆ๐ฆ(1 โ ๐ฆ๐ฆ ๐ฃ๐ฃ ) + ๐ฝ๐ฝ ln(1 โ ๐ฆ๐ฆ) [1 + ๐ฃ๐ฃ๐ฃ๐ฃ โ ๐ฃ๐ฃ โ ๐ฆ๐ฆ ๐ฃ๐ฃ ] = 0 Maximum Mean Residual Life. (Xie et al. 2004) By solving the derivative of the MRL function to zero, the maximum MRL is found by solving to t: 1๏ฟฝ ๐ฝ๐ฝ ๐ก๐ก โ = ๐ผ๐ผ[โ ln(1 โ ๐ฆ๐ฆ2 )] Exp-Weibull where ๐ฆ๐ฆ2 is the solution to: ×๏ฟฝ โ 1 [โ ln(1โ๐ฆ๐ฆ)] ๏ฟฝ๐ฝ๐ฝ Resources โ1๏ฟฝ๐ฝ๐ฝ ๐ฝ๐ฝ๐ฝ๐ฝ(1 โ ๐ฆ๐ฆ)๐ฆ๐ฆ ๐ฃ๐ฃโ1 [โ ln(1 โ ๐ฆ๐ฆ)] ๐ฝ๐ฝ ๐ฃ๐ฃ [1 โ ๏ฟฝ1 โ ๐๐ โ๐ฅ๐ฅ ๏ฟฝ ๐๐๐๐ โ (1 โ ๐ฆ๐ฆ ๐ฃ๐ฃ )2 = 0 Books / Journals: Mudholkar, G. & Srivastava, D., 1993. Exponentiated Weibull family for analyzing bathtub failure-rate data. Reliability, IEEE Transactions on, 42(2), 299-302. Jiang, R. & Murthy, D., 1999. The exponentiated Weibull family: a graphical approach. Reliability, IEEE Transactions on, 48(1), 6872. Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean residual life and failure rate function for some generalized Weibull distributions. Reliability Engineering and System Safety, 84(3), 293โ299. Murthy, D., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience. Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC. Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20: Advances in Reliability 1st ed., Elsevier Science & Technology. Relationship to Other Distributions Weibull Distribution ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Special Case: ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐ผ๐ผ = ๐ผ๐ผ, ๐ฝ๐ฝ = ๐ฝ๐ฝ, ๐ฃ๐ฃ = 1) Modified Weibull Distribution 81 3.3. Modified Weibull Distribution Probability Density Function - f(t) b=0.5 b=0.8 b=1 b=2 0.5 1 0 0.5 ฮป=1 ฮป=5 ฮป=10 0 0.2 Cumulative Density Function - F(t) b=0.5 b=0.8 b=1 b=2 0 0.5 1 a=5 a=10 a=20 0 0.5 ฮป=1 ฮป=5 ฮป=10 0 0.2 Hazard Rate - h(t) 0 5 0 5 0 Note: The hazard rate plots are on a different scale to the PDF and CDF 5 Modified Weibull 0 a=5 a=10 a=20 82 Bathtub Life Distributions Parameters & Description Parameters Limits ๐๐ ๐๐ > 0 ๐๐ ๐๐ โฅ 0 ๐๐ ๐๐ โฅ 0 Modified Weibull Distribution Scale Parameter. Shape Parameter: The shape of the distribution is completely determined by b. When 0 < ๐๐ < 1 the distribution has a bathtub shaped hazard rate. Scale Parameter. ๐ก๐ก โฅ 0 Formulas ๐๐(๐ก๐ก) = a(b + ฮปt) t bโ1 exp(ฮปt) exp๏ฟฝโat b exp(ฮปt)๏ฟฝ PDF ๐น๐น(๐ก๐ก) = 1 โ exp[โ๐๐๐ก๐ก ๐๐ ๐๐๐๐๐๐(๐๐๐๐)] CDF R(t) = exp[โ๐๐๐ก๐ก ๐๐ ๐๐๐๐๐๐(๐๐๐๐)] Reliability Mean Residual Life โ ๐ข๐ข(๐ก๐ก) = exp๏ฟฝ๐๐๐๐ ๐๐ ๐๐ ๐๐๐๐ ๏ฟฝ ๏ฟฝ exp๏ฟฝ๐๐๐ฅ๐ฅ ๐๐ ๐๐ ๐๐๐๐ ๏ฟฝ ๐๐๐๐ ๐ก๐ก โ(๐ก๐ก) = a(b + ฮปt)t bโ1 eฮปt Hazard Rate Properties and Moments Median Solved numerically (see 100p%) Mode Mean - Solved numerically 1st Variance - Raw Moment Solved numerically 2nd Solved numerically Central Moment Solve for ๐ก๐ก๐๐ numerically: ln(1 โ ๐๐) ๐ก๐ก๐๐b exp(ฮปt p ) = โ a 100p% Percentile Function Parameter Estimation Plotting Method (Lai et al. 2003) Plot Points on a Weibull Probability Plot X-Axis Y-Axis ln(๐ก๐ก๐๐ ) 1 ๐๐๐๐ ๏ฟฝ๐๐๐๐ ๏ฟฝ ๏ฟฝ๏ฟฝ 1 โ ๐น๐น Using the Weibull Probability Plot the parameters can be estimated. (Lai et al. 2003). Asymptotes (Lai et al. 2003): Modified Weibull Distribution 83 As ๐ฅ๐ฅ โ โโ (๐ก๐ก โ 0) the asymptote straight line can be approximated as: ๐ฆ๐ฆ โ ๐๐๐๐ + ln(๐๐) As ๐ฅ๐ฅ โ โ (๐ก๐ก โ โ) the asymptote straight line can be approximated as (not used for parameter estimate but more for model validity): ๐ฆ๐ฆ โ ๐๐ exp(๐ฅ๐ฅ) = ๐๐๐๐ Intersections (Lai et al. 2003): Y-Axis Intersection (0, ๐ฅ๐ฅ0 ) ln(๐๐) + ๐๐ = ๐ฆ๐ฆ0 Solving these gives an approximate value for each parameter which can be used as an initial guess for numerical methods solving MLE or Bayesian methods. A typical WPP for an Modified Weibull Distribution is: 10 9 8 7 6 5 4 3 2 1 0 -1 -9 -8 -7 -6 -5 -4 -3 -2 -1 0 1 2 WPP for modified Weibull distribution ๐๐ = 5, ๐๐ = 0.2, ๐๐ = 10 Description , Limitations and Uses Characteristics Parameter Characteristics:(Lai et al. 2003) ๐๐ < ๐๐ < 1 ๐๐๐๐๐ ๐ ๐๐ > 0. The hazard rate has a bathtub curve shape. โ(๐ก๐ก) โ โ ๐๐๐๐ ๐ก๐ก โ 0. โ(๐ก๐ก) โ โ ๐๐๐๐ ๐ก๐ก โ โ. ๐๐ โฅ ๐๐ ๐๐๐๐๐๐ ๐๐ > 0. Has an increasing hazard rate function. โ(0) = 0. โ(๐ก๐ก) โ โ ๐๐๐๐ ๐ก๐ก โ โ . ๐๐ = ๐๐. The function has the same form as a Weibull Distribution. โ(0) = ๐๐๐๐. โ(๐ก๐ก) โ โ ๐๐๐๐ ๐ก๐ก โ โ Modified Weibull ln(๐๐) + ๐๐๐ฅ๐ฅ0 + ๐๐๐๐ ๐ฅ๐ฅ0 = 0 X-Axis Intersection (๐ฆ๐ฆ0 , 0) 84 Bathtub Life Distributions Minimum Failure Rate. (Xie et al. 2004) When the hazard rate is a bathtub curve (0 < ๐๐ < 1 ๐๐๐๐๐๐ ๐๐ > 0) then the minimum failure rate point is given as: โ๐๐ โ ๐๐ ๐ก๐ก โ = ๐๐ Modified Weibull Maximum Mean Residual Life. (Xie et al. 2004) By solving the derivative of the MRL function to zero, the maximum MRL is found by solving to t: โ ๐๐(๐๐ + ๐๐๐๐)๐ก๐ก ๐๐โ1 ๐๐ ๐๐๐๐ ๏ฟฝ exp(โ๐๐๐ฅ๐ฅ ๐๐ ๐๐ (๐๐๐๐)๐๐๐๐ โ exp๏ฟฝ๐๐๐ก๐ก ๐๐ ๐๐ ๐๐๐๐ ๏ฟฝ = 0 ๐ก๐ก Shape. The shape of the hazard rate cannot have a flat โusage periodโ and a strong โwear outโ gradient. Resources Books / Journals: Lai, C., Xie, M. & Murthy, D., 2003. A modified Weibull distribution. IEEE Transactions on Reliability, 52(1), 33-37. Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience. Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean residual life and failure rate function for some generalized Weibull distributions. Reliability Engineering and System Safety, 84(3), 293โ299. Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC. Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20: Advances in Reliability 1st ed., Elsevier Science & Technology. Relationship to Other Distributions Weibull Distribution ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Special Case: ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; a = ๐ผ๐ผ, ๐๐ = ๐ฝ๐ฝ, ๐๐ = 0) Univariate Continuous Distributions 85 4. Univariate Continuous Distributions Univariate Cont 86 Univariate Continuous Distributions 4.1. Beta Continuous Distribution Probability Density Function - f(t) ฮฑ=5 ฮฒ=2 ฮฑ=2 ฮฒ=2 ฮฑ=1 ฮฒ=2 ฮฑ=0.5 ฮฒ=2 3.00 2.50 2.50 2.00 2.00 1.50 1.50 1.00 1.00 0.50 0.50 0.00 0.00 0 ฮฑ=1 ฮฒ=1 ฮฑ=5 ฮฒ=5 ฮฑ=0.5 ฮฒ=0.5 0.5 1 0 0.2 0.4 0.6 0.8 1 Beta Cumulative Density Function - F(t) 1.00 1.00 0.80 0.80 0.60 0.60 0.40 0.40 0.20 0.20 0.00 0.00 0 0.5 Hazard Rate - h(t) 10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 1 ฮฑ=5 ฮฒ=2 ฮฑ=2 ฮฒ=2 ฮฑ=1 ฮฒ=2 ฮฑ=0.5 ฮฒ=2 ฮฑ=1 ฮฒ=1 ฮฑ=5 ฮฒ=5 ฮฑ=0.5 ฮฒ=0.5 0 0.2 0.4 0.6 0.8 1 4.00 3.50 3.00 2.50 2.00 1.50 1.00 ฮฑ=1 ฮฒ=1 ฮฑ=5 ฮฒ=5 ฮฑ=0.5 ฮฒ=0.5 0.50 0.00 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Beta Continuous Distribution 87 Parameters & Description ๐ผ๐ผ Parameters ๐ผ๐ผ > 0 Shape Parameter. ๐ฝ๐ฝ ๐ฝ๐ฝ > 0 ๐๐๐ฟ๐ฟ โโ < ๐๐๐ฟ๐ฟ < ๐๐๐๐ ๐๐๐๐ Shape Parameter. Lower Bound: ๐๐๐ฟ๐ฟ is the lower bound but has also been called a location parameter. In the standard Beta distribution ๐๐๐ฟ๐ฟ = 0. Upper Bound: ๐๐๐๐ is the upper bound. In the standard Beta distribution ๐๐๐๐ = 1. The scale parameter may also be defined as ๐๐๐๐ โ ๐๐๐ฟ๐ฟ . ๐๐๐ฟ๐ฟ < ๐๐๐๐ < โ ๐๐๐ฟ๐ฟ < ๐ก๐ก โค ๐๐๐๐ Limits Distribution Formulas ๐ต๐ต(๐ฅ๐ฅ, ๐ฆ๐ฆ) is the Beta function, ๐ต๐ต๐ก๐ก (๐ก๐ก|๐ฅ๐ฅ, ๐ฆ๐ฆ) is the incomplete Beta function, ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ฅ๐ฅ, ๐ฆ๐ฆ) is the regularized Beta function, ฮ(๐๐) is the complete gamma which is discussed in section 1.6. General Form: PDF When ๐๐๐ฟ๐ฟ = 0, ๐๐๐๐ = 1: ฮ(๐ผ๐ผ + ๐ฝ๐ฝ) (๐ก๐ก โ ๐๐๐ฟ๐ฟ )๐ผ๐ผโ1 (๐๐๐๐ โ ๐ก๐ก)๐ฝ๐ฝโ1 . (๐๐๐๐ โ ๐๐๐ฟ๐ฟ )๐ผ๐ผ+๐ฝ๐ฝโ1 ฮ(๐ผ๐ผ)ฮ(๐ฝ๐ฝ) ฮ(๐ผ๐ผ + ๐ฝ๐ฝ) ๐ผ๐ผโ1 (1 โ ๐ก๐ก)๐ฝ๐ฝโ1 . ๐ก๐ก ฮ(๐ผ๐ผ)ฮ(๐ฝ๐ฝ) 1 = . ๐ก๐ก ๐ผ๐ผโ1 (1 โ ๐ก๐ก)๐ฝ๐ฝโ1 ๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐(๐ก๐ก|๐ผ๐ผ, ๐ฝ๐ฝ) = CDF Reliability Conditional Survivor Function Mean Residual Life ฮ(๐ผ๐ผ + ๐ฝ๐ฝ) ๐ก๐ก ๐ผ๐ผโ1 (1 โ ๐ข๐ข)๐ฝ๐ฝโ1 ๐๐๐๐ ๏ฟฝ ๐ข๐ข ฮ(๐ผ๐ผ)ฮ(๐ฝ๐ฝ) 0 ๐ต๐ต๐ก๐ก (๐ก๐ก|๐ผ๐ผ, ๐ฝ๐ฝ) = ๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) = ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ผ๐ผ, ๐ฝ๐ฝ) F(t) = R(t) = 1 โ ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = ๐ ๐ (๐ก๐ก + x) 1 โ ๐ผ๐ผ๐ก๐ก (๐ก๐ก + ๐ฅ๐ฅ|๐ผ๐ผ, ๐ฝ๐ฝ) = ๐ ๐ (๐ก๐ก) 1 โ ๐ผ๐ผ๐ก๐ก (๐ก๐ก|๐ผ๐ผ, ๐ฝ๐ฝ) Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. โ โซt {๐ต๐ต(ฮฑ, ฮฒ) โ ๐ต๐ตx (x|ฮฑ, ฮฒ)}dx ๐ต๐ต(ฮฑ, ฮฒ) โ ๐ต๐ตt (t|ฮฑ, ฮฒ) (Gupta and Nadarajah 2004, p.44) ๐ข๐ข(๐ก๐ก) = Beta ๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ, ๐๐๐ฟ๐ฟ , ๐๐๐๐ ) = 88 Univariate Continuous Distributions Hazard Rate t ฮฑโ1 (1 โ t) ๐ต๐ต(ฮฑ, ฮฒ) โ ๐ต๐ตt (t|ฮฑ, ฮฒ) (Gupta and Nadarajah 2004, p.44) โ(๐ก๐ก) = Properties and Moments Numerically solve for t: ๐ก๐ก0.5 = ๐น๐น โ1 (๐ผ๐ผ, ๐ฝ๐ฝ) Median ๐ผ๐ผโ1 Mode ๐ผ๐ผ+๐ฝ๐ฝโ2 Mean - 1st Raw Moment for ๐ผ๐ผ > 1 ๐๐๐๐๐๐ ๐ฝ๐ฝ > 1 ฮฑ ฮฑ+ฮฒ ฮฑฮฒ (ฮฑ + ฮฒ)2 (ฮฑ + ฮฒ + 1) Variance - 2nd Central Moment 2(ฮฒ โ ฮฑ)๏ฟฝฮฑ + ฮฒ + 1 Skewness - 3rd Central Moment 3 6[๐ผ๐ผ + ๐ผ๐ผ Beta Excess kurtosis - 4th Central Moment (ฮฑ + ฮฒ + 2)๏ฟฝฮฑฮฒ 2 (1 โ 2๐ฝ๐ฝ) + ๐ฝ๐ฝ2 (1 + ๐ฝ๐ฝ) โ 2๐ผ๐ผ๐ผ๐ผ(2 + ๐ฝ๐ฝ)] ๐ผ๐ผ๐ผ๐ผ(๐ผ๐ผ + ๐ฝ๐ฝ + 2)(๐ผ๐ผ + ๐ฝ๐ฝ + 3) 1F1 (ฮฑ; ฮฑ + ฮฒ; it) Where 1F1 is the confluent hypergeometric function defined as: โ (ฮฑ)k ๐ฅ๐ฅ ๐๐ (ฮฑ; ฮฒ; x) = ๏ฟฝ . F 1 1 (ฮฒ)k ๐๐! Characteristic Function k=0 (Gupta and Nadarajah 2004, p.44) Numerically solve for t: ๐ก๐ก๐๐ = ๐น๐น โ1 (๐ผ๐ผ, ๐ฝ๐ฝ) 100p% Percentile Function Parameter Estimation Maximum Likelihood Function Likelihood Functions Log-Likelihood Functions โฮ =0 โฮฑ L(ฮฑ, ฮฒ|E) = nF ฮ(๐ผ๐ผ + ๐ฝ๐ฝ)nF ๐ฝ๐ฝโ1 ๏ฟฝ ๐ก๐ก๐๐๐น๐น ๐ผ๐ผโ1 ๏ฟฝ1 โ ๐ก๐ก๐๐๐น๐น ๏ฟฝ . ฮ(๐ผ๐ผ)ฮ(๐ฝ๐ฝ) ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 failures ฮ(ฮฑ, ฮฒ|E) = nF {ln[๐ค๐ค(๐ผ๐ผ + ๐ฝ๐ฝ) โ ๐๐๐๐[๐ค๐ค(๐ผ๐ผ)] โ ๐๐๐๐[๐ค๐ค(๐ฝ๐ฝ)]} +(ฮฑ โ where ๐๐(๐ฅ๐ฅ) = ๐๐ ๐๐๐๐ nF 1) ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ i=1 nF + (ฮฒ โ 1) ๏ฟฝ ln(1 โ t Fi ) ๐๐(ฮฑ) โ ๐๐(ฮฑ + ฮฒ) = nF i=1 1 ๏ฟฝ ln(t Fi ) nF i=1 ln[ฮ(๐ฅ๐ฅ)] is the digamma function see section 1.6.7. Beta Continuous Distribution 89 (Johnson et al. 1995, p.223) โฮ =0 โฮฒ Point Estimates Fisher Information Matrix nF 1 ๐๐(ฮฒ) โ ๐๐(ฮฑ + ฮฒ) = ๏ฟฝ ln(1 โ t i ) nF i=1 (Johnson et al. 1995, p.223) Point estimates are obtained by using numerical methods to solve the simultaneous equations above. ๐๐ โฒ (๐ผ๐ผ) โ ๐๐ โฒ (๐ผ๐ผ + ๐ฝ๐ฝ) ๐ผ๐ผ(๐ผ๐ผ, ๐ฝ๐ฝ) = ๏ฟฝ โ๐๐ โฒ (๐ผ๐ผ + ๐ฝ๐ฝ) ๐๐ 2 โ๐๐ โฒ (๐ผ๐ผ + ๐ฝ๐ฝ) ๏ฟฝ โ ๐๐ โฒ (๐ผ๐ผ + ๐ฝ๐ฝ) ๐๐ โฒ (๐ฝ๐ฝ) โ2 where ๐๐ โฒ (๐ฅ๐ฅ) = 2 ๐๐๐๐ฮ(๐ฅ๐ฅ) = โโ is the Trigamma function. ๐๐=0(๐ฅ๐ฅ + ๐๐) ๐๐๐ฅ๐ฅ See section 1.6.8. (Yang and Berger 1998, p.5) Confidence Intervals For a large number of samples the Fisher information matrix can be used to estimate confidence intervals. See section 1.4.7. Bayesian Non-informative Priors โdet๏ฟฝ๐ผ๐ผ(๐ผ๐ผ, ๐ฝ๐ฝ)๏ฟฝ where ๐ผ๐ผ(๐ผ๐ผ, ๐ฝ๐ฝ) is given above. Beta Jefferyโs Prior Conjugate Priors UOI ๐๐ from ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐) ๐๐ from ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) Example Likelihood Model Bernoulli Binomial Evidence ๐๐ failures in 1 trail ๐๐ failures in ๐๐ trials Dist. of UOI Prior Para Posterior Parameters Beta ๐ผ๐ผ0 , ๐ฝ๐ฝ0 ๐ผ๐ผ = ๐ผ๐ผ๐๐ + ๐๐ ๐ฝ๐ฝ = ๐ฝ๐ฝ๐๐ + 1 โ ๐๐ Beta ๐ผ๐ผ๐๐ , ๐ฝ๐ฝ๐๐ Description , Limitations and Uses ๐ผ๐ผ = ๐ผ๐ผ๐๐ + ๐๐ ฮฒ = ฮฒo + n โ k For examples on the use of the beta distribution as a conjugate prior see the binomial distribution. A non-homogeneous (operate in different environments) population of 5 switches have the following probabilities of failure on demand. 0.1176, 0.1488, 0.3684, 0.8123, 0.9783 Estimate the population variability function: nF 1 ๏ฟฝ ln(t Fi ) = โ1.0549 nF i=1 90 Univariate Continuous Distributions nF 1 ๏ฟฝ ln(1 โ t i ) = โ1.25 nF i=1 Numerically Solving: ๐๐(ฮฑ) + 1.0549 = ๐๐(ฮฒ) + 1.25 Gives: ๐ผ๐ผ๏ฟฝ = 0.7369 ๐๐๏ฟฝ = 0.6678 1.5924 โ1.0207 ๐ผ๐ผ(๐ผ๐ผ, ๐ฝ๐ฝ) = ๏ฟฝ ๏ฟฝ โ1.0207 2.0347 โ1 โ1 0.1851 0.0929 ๏ฟฝ ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ๏ฟฝ๏ฟฝ = ๏ฟฝ๐๐๐น๐น ๐ผ๐ผ๏ฟฝ๐ผ๐ผ๏ฟฝ, ๐ฝ๐ฝฬ ๏ฟฝ๏ฟฝ = ๏ฟฝ 0.0929 0.1449 Beta 90% confidence interval for ๐ผ๐ผ: ๐ท๐ทโ1 (0.95)โ0.1851 ๐ท๐ทโ1 (0.95)โ0.1851 ๏ฟฝ๐ผ๐ผ๏ฟฝ. exp ๏ฟฝ ๏ฟฝ, ๐ผ๐ผ๏ฟฝ. exp ๏ฟฝ ๏ฟฝ๏ฟฝ โ๐ผ๐ผ๏ฟฝ ๐ผ๐ผ๏ฟฝ [0.282, 1.92] Characteristics 90% confidence interval for ๐ฝ๐ฝ: ๐ท๐ทโ1 (0.95)โ0.1449 ๐ท๐ท โ1 (0.95)โ0.1449 ๏ฟฝ๐ฝ๐ฝฬ. exp ๏ฟฝ ๏ฟฝ, ๐ฝ๐ฝฬ . exp ๏ฟฝ ๏ฟฝ๏ฟฝ ฬ โ๐ฝ๐ฝ ๐ฝ๐ฝฬ [0.262, 1.71] The Beta distribution was originally known as a Pearson Type I distribution (and Type II distribution which is a special case of a Type I). ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) is the mirror distribution of ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ฝ๐ฝ, ๐ผ๐ผ). If ๐๐~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) and let ๐๐ = 1 โ ๐๐ then ๐๐~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ฝ๐ฝ, ๐ผ๐ผ). Location / Scale Parameters (NIST Section 1.3.6.6.17) ๐๐๐ฟ๐ฟ and ๐๐๐๐ can be transformed into a location and scale parameter: ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ = ๐๐๐ฟ๐ฟ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ = ๐๐๐๐ โ ๐๐๐ฟ๐ฟ Shapes(Gupta and Nadarajah 2004, p.41): ๐๐ < ๐ผ๐ผ < 1. As ๐ฅ๐ฅ โ 0, ๐๐(๐ฅ๐ฅ) โ โ. ๐๐ < ๐ฝ๐ฝ < 1. As ๐ฅ๐ฅ โ 1, ๐๐(๐ฅ๐ฅ) โ โ. ๐ถ๐ถ > 1, ๐ท๐ท > 1. As ๐ฅ๐ฅ โ 0, ๐๐(๐ฅ๐ฅ) โ 0. There is a single mode at ๐ผ๐ผโ1 . ๐ผ๐ผ+๐ฝ๐ฝโ2 ๐ถ๐ถ < 1, ๐ท๐ท < 1. The distribution is a U shape. There is a ๐ผ๐ผโ1 single anti-mode at . ๐ผ๐ผ+๐ฝ๐ฝโ2 ๐ถ๐ถ > 0, ๐ท๐ท > 0. There exists inflection points at: (๐ผ๐ผ โ 1)(๐ฝ๐ฝ โ 1) ๐ผ๐ผ โ 1 1 ± .๏ฟฝ ๐ผ๐ผ + ๐ฝ๐ฝ โ 2 ๐ผ๐ผ + ๐ฝ๐ฝ โ 2 ๐ผ๐ผ + ๐ฝ๐ฝ โ 3 Beta Continuous Distribution 91 ๐ถ๐ถ = ๐ท๐ท. The distribution is symmetrical about ๐ฅ๐ฅ = 0.5. As ๐ผ๐ผ = ๐ฝ๐ฝ becomes large, the beta distribution approaches the normal distribution. The Standard Uniform Distribution arises when ๐ผ๐ผ = ๐ฝ๐ฝ = 1. ๐ถ๐ถ = ๐๐, ๐ท๐ท = ๐๐ or ๐ถ๐ถ = ๐๐, ๐ท๐ท = ๐๐. Straight line. (๐ถ๐ถโ ๐๐)(๐ท๐ทโ ๐๐) < 0. J Shaped. Hazard Rate and MRL (Gupta and Nadarajah 2004, p.45): ๐ถ๐ถ โฅ ๐๐, ๐ท๐ท โฅ ๐๐. โ(๐ก๐ก) is increasing. ๐ข๐ข(๐ก๐ก) is decreasing. ๐ถ๐ถ โค ๐๐, ๐ท๐ท โค ๐๐. โ(๐ก๐ก) is decreasing. ๐ข๐ข(๐ก๐ก) is increasing. ๐ถ๐ถ > 1, 0 < ๐ฝ๐ฝ < 1. โ(๐ก๐ก) is bathtub shaped and ๐ข๐ข(๐ก๐ก) is an upside down bathtub shape. ๐๐ < ๐ผ๐ผ < 1, ๐ท๐ท > 1. โ(๐ก๐ก) is upside down bathtub shaped and ๐ข๐ข(๐ก๐ก) is bathtub shape. Parameter Model. The Beta distribution is often used to model parameters which are constrained to take place between an interval. In particular the distribution of a probability parameter 0 โค ๐๐ โค 1 is popular with the Beta distribution. Proportions. Used to model proportions. An example of this is the likelihood ratios for estimating uncertainty. Online: http://mathworld.wolfram.com/BetaDistribution.html http://en.wikipedia.org/wiki/Beta_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web calculator) http://www.itl.nist.gov/div898/handbook/eda/section3/eda366h.htm Resources Books: Gupta, A.K. & Nadarajah, S., 2004. Handbook of beta distribution and its applications, CRC Press. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience. Relationship to Other Distributions Beta Applications Bayesian Analysis. The Beta distribution is often used as a conjugate prior in Bayesian analysis for the Bernoulli, Binomial and Geometric Distributions to produce closed form posteriors. The Beta(0,0) distribution is an improper prior sometimes used to represent ignorance of parameter values. The Beta(1,1) is a standard uniform distribution which may be used as a noninformative prior. When used as a conjugate prior to a Bernoulli or Binomial process the parameter ๐ผ๐ผ may represent the number of successes and ๐ฝ๐ฝ the total number of failures with the total number of trials being ๐๐ = ๐ผ๐ผ + ๐ฝ๐ฝ. 92 Univariate Continuous Distributions Let Chi-square Distribution Xi ~๐๐ 2 (๐ฃ๐ฃ๐๐ ) Then 2 ๐๐ (๐ก๐ก; ๐ฃ๐ฃ) ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐) Beta Normal Distribution ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐) Gamma Distribution ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; k, ฮป) Dirichlet Distribution ๐ท๐ท๐ท๐ท๐๐๐๐ (๐ฅ๐ฅ; ๐ถ๐ถ) 1 Y= 1 X1 X1 + X 2 Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐ผ๐ผ = 2๐ฃ๐ฃ1 , ๐ฝ๐ฝ = 2๐ฃ๐ฃ2 ๏ฟฝ Let Uniform Distribution ๐๐๐๐๐๐ Then Xi ~๐๐๐๐๐๐๐๐(0,1) ๐๐๐๐๐๐ X1 โค X2 โค โฏ โค Xn Xr ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐, ๐๐ โ ๐๐ + 1) Where ๐๐ and ๐๐ are integers. Special Case: ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; 1,1, ๐๐, ๐๐) = ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐) For large ๐ผ๐ผ and ๐ฝ๐ฝ with fixed ๐ผ๐ผ/๐ฝ๐ฝ: ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) โ ๐๐๐๐๐๐๐๐ ๏ฟฝµ = ฮฑ ฮฑฮฒ , ๐๐ = ๏ฟฝ ๏ฟฝ (ฮฑ + ฮฒ)2 (ฮฑ + ฮฒ + 1) ฮฑ+ฮฒ As ๐ผ๐ผ and ๐ฝ๐ฝ increase the mean remains constant and the variance is reduced. Let Then ๐๐1 , ๐๐2 ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(k i , ฮปi ) ๐๐๐๐๐๐ Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ = ๐๐1 , ๐ฝ๐ฝ = ๐๐2 ) Y= X1 X1 + X 2 Special Case: ๐ท๐ท๐ท๐ท๐๐๐๐=1 (x; [ฮฑ1 , ฮฑ0 ]) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐ = ๐ฅ๐ฅ; ๐ผ๐ผ = ๐ผ๐ผ1 , ๐ฝ๐ฝ = ๐ผ๐ผ0 ) Birnbaum Sanders Continuous Distribution 93 4.2. Birnbaum Saunders Continuous Distribution Probability Density Function - f(t) ฮฑ=0.2 ฮฒ=1 ฮฑ=0.5 ฮฒ=1 ฮฑ=1 ฮฒ=1 ฮฑ=2.5 ฮฒ=1 2.0 1.5 ฮฑ=1 ฮฒ=0.5 ฮฑ=1 ฮฒ=1 ฮฑ=1 ฮฒ=2 ฮฑ=1 ฮฒ=4 1.4 1.2 1.0 0.8 1.0 0.6 0.4 0.5 0.2 0.0 0.0 0 1 2 3 0 1 2 3 0 1 2 3 0 1 2 3 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.8 0.6 0.4 ฮฑ=0.2 ฮฒ=1 ฮฑ=0.5 ฮฒ=1 ฮฑ=1 ฮฒ=1 ฮฑ=2.5 ฮฒ=1 0.2 0.0 0 1 2 Birnbaum Sanders Cumulative Density Function - F(t) 3 Hazard Rate - h(t) 2.5 1.6 2.0 1.4 1.2 1.5 1.0 0.8 1.0 0.6 0.5 0.4 0.0 0.0 0.2 0 1 2 3 94 Univariate Continuous Distributions Parameters & Description Parameters Limits ฮฒ Scale parameter. ฮฒ is the scale parameter equal to the median. ฮฒ>0 ๐ผ๐ผ ฮฑ>0 Shape parameter. 0<t< โ Distribution Formulas ๐๐(๐ก๐ก) = = Birnbaum Sanders PDF ๏ฟฝtโฮฒ + ๏ฟฝฮฒโt 2๐ผ๐ผ๐ผ๐ผโ2๐๐ 2 1 ๏ฟฝtโฮฒ โ ๏ฟฝฮฒโt exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐ผ๐ผ ๏ฟฝtโฮฒ + ๏ฟฝฮฒโt ฯ(z) 2๐ผ๐ผ๐ผ๐ผ where ๐๐(๐ง๐ง) is the standard normal pdf and: ๏ฟฝtโฮฒ โ ๏ฟฝฮฒโt ๐ง๐ง๐ต๐ต๐ต๐ต = ๐ผ๐ผ ๏ฟฝtโฮฒ โ ๏ฟฝฮฒโt ๏ฟฝ ๐ผ๐ผ = ฮฆ(๐ง๐ง๐ต๐ต๐ต๐ต ) CDF ๐น๐น(๐ก๐ก) = ฮฆ ๏ฟฝ Reliability R(t) = ฮฆ ๏ฟฝ ๏ฟฝฮฒโt โ ๏ฟฝtโฮฒ ๏ฟฝ ๐ผ๐ผ = ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต ) Where Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Residual Life Hazard Rate Cumulative Hazard Rate ๐ง๐ง๐ต๐ต๐ต๐ต = ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = ๏ฟฝtโฮฒ โ ๏ฟฝฮฒโt , ๐ผ๐ผ โฒ ) ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต = ๐ ๐ (๐ก๐ก) ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต ) โฒ ๐ง๐ง๐ต๐ต๐ต๐ต = ๏ฟฝ(t + x)โฮฒ โ ๏ฟฝฮฒโ(t + x) ๐ผ๐ผ ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. โ โซ ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต )๐๐๐๐ ๐ข๐ข(๐ก๐ก) = ๐ก๐ก ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต ) โ(๐ก๐ก) = ๏ฟฝtโฮฒ + ๏ฟฝฮฒโt ฯ(zBS ) ๏ฟฝ ๏ฟฝ 2๐ผ๐ผ๐ผ๐ผ ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต ) ๐ป๐ป(๐ก๐ก) = โ ln[ฮฆ(โ๐ง๐ง๐ต๐ต๐ต๐ต )] Birnbaum Sanders Continuous Distribution 95 Properties and Moments ฮฒ Median Numerically solve for ๐ก๐ก: ๐ก๐ก 3 + ๐ฝ๐ฝ(1 + ๐ผ๐ผ 2 )๐ก๐ก 2 + ๐ฝ๐ฝ2 (3๐ผ๐ผ 2 โ 1)๐ก๐ก โ ๐ฝ๐ฝ3 = 0 Mode Mean - 1st Raw Moment ฮฒ ๏ฟฝ1 + Variance - 2nd Central Moment ฮฑ2 ๏ฟฝ 2 ฮฑ2 ฮฒ2 ๏ฟฝ1 + 5ฮฑ2 ๏ฟฝ 4 4ฮฑ(11ฮฑ2 + 6) Skewness - 3rd Central Moment 3 (5ฮฑ2 + 4)2 (Lemonte et al. 2007) Excess kurtosis - 4th Central Moment 6ฮฑ2 (93ฮฑ2 + 40) (5ฮฑ2 + 4)2 (Lemonte et al. 2007) 2 ฮฒ ๐ก๐ก๐พ๐พ = ๏ฟฝฮฑฮฆโ1 (ฮณ) + ๏ฟฝ4 + [ฮฑฮฆโ1 (ฮณ)]2 ๏ฟฝ 4 100๐พ๐พ % Percentile Function Parameter Estimation Maximum Likelihood Function Likelihood Function Log-Likelihood Function โฮ =0 โฮฑ โฮ =0 โฮฒ MLE Point For complete data: nF 2 1 ๏ฟฝt i โฮฒ โ ๏ฟฝฮฒโt i ๏ฟฝt i โฮฒ + ๏ฟฝฮฒโt i ๐ฟ๐ฟ(๐๐, ๐ผ๐ผ|๐ธ๐ธ) = ๏ฟฝ exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐ผ๐ผ 2๐ผ๐ผ๐ก๐ก๐๐ โ2๐๐ i=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures 1 2 ๐๐๐น๐น 3 ๐๐๐น๐น ๐ฝ๐ฝ ๐ฝ๐ฝ 2 1 ๐ก๐ก๐๐ ๐ฝ๐ฝ ฮ(๐ผ๐ผ, ๐ฝ๐ฝ|๐ธ๐ธ) = โnF ln(ฮฑฮฒ) + ๏ฟฝ ln ๏ฟฝ๏ฟฝ ๏ฟฝ + ๏ฟฝ ๏ฟฝ ๏ฟฝ โ 2 ๏ฟฝ ๏ฟฝ + โ 2๏ฟฝ ๐ก๐ก๐๐ ๐ก๐ก๐๐ 2๐ผ๐ผ ๐ฝ๐ฝ ๐ก๐ก๐๐ ๐๐=1 ๐๐=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures ๐๐๐น๐น ๐๐๐น๐น โฮ nF 2 1 ๐ฝ๐ฝ 1 = โ ๏ฟฝ1 + 2 ๏ฟฝ + 3 ๏ฟฝ ๐ก๐ก๐๐ + 3 ๏ฟฝ = 0 โฮฑ ฮฑ ฮฑ ฮฑ ฮฒ ๐ผ๐ผ ๐ก๐ก๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 ๐๐=1 ๐๐๐น๐น failures ๐๐๐น๐น ๐๐๐น๐น โฮ nF 1 1 1 1 =โ +๏ฟฝ + ๏ฟฝ ๐ก๐ก๐๐ โ 2 ๏ฟฝ = 0 โฮฒ 2ฮฒ ๐ก๐ก๐๐ + ๐ฝ๐ฝ 2๐ผ๐ผ 2 ๐ฝ๐ฝ2 2๐ผ๐ผ ๐ก๐ก๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 ๐๐=1 ๐๐=1 ๐ฝ๐ฝฬ is found by solving: failures Birnbaum Sanders 3+ 96 Univariate Continuous Distributions Estimates where ๐๐๐น๐น ๐ฝ๐ฝ2 โ ๐ฝ๐ฝ[2๐ ๐ + ๐๐(๐ฝ๐ฝ)] + ๐ ๐ [๐๐ + ๐๐(๐ฝ๐ฝ)] = 0 1 1 ๐๐(๐ฝ๐ฝ) = ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐ ๐ฝ๐ฝ + ๐ก๐ก๐๐ โ1 ๐๐=1 Point estimates for ๐ผ๐ผ๏ฟฝ is: (Lemonte et al. 2007) where Birnbaum Sanders , ๐๐=1 ๐๐๐น๐น โ1 1 1 ๐ ๐ = ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐๐น๐น ๐ก๐ก๐๐ ๐๐=1 ๐๐ ๐ฝ๐ฝฬ ๐ผ๐ผ๏ฟฝ = ๏ฟฝ + โ 2 ๐ฝ๐ฝฬ ๐ ๐ 2 โก 2 ๐ผ๐ผ ๐ผ๐ผ(๐๐, ๐ผ๐ผ) = โข โข0 โฃ Fisher Information 100๐พ๐พ% Confidence Intervals ๐๐๐น๐น 1 ๐๐ = ๏ฟฝ ๐ก๐ก๐๐ , ๐๐๐น๐น โค โฅ ๐ผ๐ผ(2๐๐)โ1โ2 ๐๐(๐ผ๐ผ) + 1โฅ โฆ ๐ผ๐ผ 2 ๐ฝ๐ฝ2 0 ๐๐ 2 2 ๐๐(๐ผ๐ผ) = ๐ผ๐ผ๏ฟฝ โ ๐๐ exp ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ1 โ ฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ 2 ๐ผ๐ผ ๐ผ๐ผ (Lemonte et al. 2007) Calculated from the Fisher information matrix. See section 1.4.7. For a literature review of proposed confidence intervals see (Lemonte et al. 2007). Description , Limitations and Uses Example 5 components are put on a test with the following failure times: 98, 116, 2485, 2526, , 2920 hours ๐๐๐น๐น 1 ๐๐ = ๏ฟฝ ๐ก๐ก๐๐ = 1629 ๐๐๐น๐น ๐๐=1 โ1 ๐๐๐น๐น Solving: 1 1 ๐ ๐ = ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐๐น๐น ๐ก๐ก๐๐ ๐๐๐น๐น ๐๐=1 = 250.432 โ1 ๐๐๐น๐น โ1 1 1 1 1 ๏ฟฝ ๏ฟฝ + ๐ ๐ ๏ฟฝ๐๐ + ๏ฟฝ ๏ฟฝ ๏ฟฝ ๏ฟฝ=0 ๐ฝ๐ฝ โ ๐ฝ๐ฝ ๏ฟฝ2๐ ๐ + ๏ฟฝ ๏ฟฝ ๐๐ ๐ฝ๐ฝ + ๐ก๐ก๐๐ ๐๐ ๐ฝ๐ฝ + ๐ก๐ก๐๐ 2 ๐๐=1 ฮฒ๏ฟฝ = 601.949 ๐๐=1 ๐๐ ๐ฝ๐ฝฬ ๐ผ๐ผ๏ฟฝ = ๏ฟฝ + โ 2 = 1.763 ๐ฝ๐ฝฬ ๐ ๐ Birnbaum Sanders Continuous Distribution 90% confidence interval for ๐ผ๐ผ: 2 โก โง๐ท๐ทโ1 (0.95)๏ฟฝ ๐ผ๐ผ โซ 2๐๐ ๐น๐น โข๐ผ๐ผ๏ฟฝ. exp , โ๐ผ๐ผ๏ฟฝ โข โจ โฌ โฃ โฉ โญ [1.048, 97 2 โง๐ท๐ทโ1 (0.95)๏ฟฝ ๐ผ๐ผ โซโค 2๐๐๐น๐น โฅ ๐ผ๐ผ๏ฟฝ. exp ๐ผ๐ผ๏ฟฝ โจ โฌโฅ โฉ โญโฆ 2.966] 90% confidence interval for ๐ฝ๐ฝ: 2 2 ๐๐ ๐๐(๐ผ๐ผ๏ฟฝ) = ๐ผ๐ผ๏ฟฝ๏ฟฝ โ ๐๐ exp ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ1 โ ฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ = 1.442 ๐ผ๐ผ๏ฟฝ ๐ผ๐ผ๏ฟฝ 2 ๐ผ๐ผ๏ฟฝ(2๐๐)โ1โ2 ๐๐(๐ผ๐ผ๏ฟฝ) + 1 ๐ผ๐ผ๐ฝ๐ฝ๐ฝ๐ฝ = = 10.335๐ธ๐ธ-6 ๐ผ๐ผ๏ฟฝ 2 ๐ฝ๐ฝฬ2 Note that this confidence interval uses the assumption of the parameters being normally distributed which is only true for large sample sizes. Therefore these confidence intervals may be inaccurate. Bayesian methods must be done numerically. Characteristics The Birnbaum-Saunders distribution is a stochastic model of the Minerโs rule. Characteristic of ๐ถ๐ถ. As ๐ผ๐ผ decreases the distribution becomes more symmetrical around the value of ๐ฝ๐ฝ. Hazard Rate. The hazard rate is always unimodal. The hazard rate has the following asymptotes: (Meeker & Escobar 1998, p.107) โ(0) = 0 1 lim โ(๐ก๐ก) = ๐ก๐กโโ 2๐ฝ๐ฝ๐ผ๐ผ 2 The change point of the unimodal hazard rate for ๐ผ๐ผ < 0.6 must be solved numerically, however for ๐ผ๐ผ > 0.6 can be approximated using: (Kundu et al. 2008) ๐ฝ๐ฝ ๐ก๐ก๐๐ = (โ0.4604 + 1.8417๐ผ๐ผ)2 Lognormal and Inverse Gaussian Distribution. The shape and behavior of the Birnbaum-Saunders distribution is similar to that of the lognormal and inverse Gaussian distribution. This similarity is seen primarily in the center of the distributions. (Meeker & Escobar 1998, p.107) Let: ๐๐~๐ต๐ต๐ต๐ต(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Birnbaum Sanders 96762 96762 โค โก โง๐ท๐ทโ1 (0.95)๏ฟฝ โซ โง๐ท๐ทโ1 (0.95)๏ฟฝ โซ ๐๐๐น๐น ๐๐๐น๐น โข๐ฝ๐ฝฬ . exp โฅ , ๐ฝ๐ฝฬ . exp โ๐ฝ๐ฝฬ โ๐ฝ๐ฝฬ โข โจ โฌ โจ โฌโฅ โฃ โฉ โญ โฉ โญโฆ [100.4, 624.5] 98 Univariate Continuous Distributions Scaling property (Meeker & Escobar 1998, p.107) ๐๐๐๐~๐ต๐ต๐ต๐ต(๐ก๐ก; ๐ผ๐ผ, ๐๐๐๐) where ๐๐ > 0 Birnbaum Sanders Inverse property (Meeker & Escobar 1998, p.107) 1 1 ~๐ต๐ต๐ต๐ต ๏ฟฝ๐ก๐ก; ๐ผ๐ผ, ๏ฟฝ ๐๐ ๐ฝ๐ฝ Applications Fatigue-Fracture. The distribution has been designed to model crack growth to critical crack size. The model uses the Minerโs rule which allows for non-constant fatigue cycles through accumulated damage. The assumption is that the crack growth during any one cycle is independent of the growth during any other cycle. The growth for each cycle has the same distribution from cycle to cycle. This is different from the proportional degradation model used to derive the log normal distribution model, with the rate of degradation being dependent on accumulated damage. (http://www.itl.nist.gov/div898/handbook/apr/section1/apr166.htm) Resources Online: http://www.itl.nist.gov/div898/handbook/eda/section3/eda366a.htm http://www.itl.nist.gov/div898/handbook/apr/section1/apr166.htm http://en.wikipedia.org/wiki/Birnbaum%E2%80%93Saunders_distrib ution Books: Birnbaum, Z.W. & Saunders, S.C., 1969. A New Family of Life Distributions. Journal of Applied Probability, 6(2), 319-327. Lemonte, A.J., Cribari-Neto, F. & Vasconcellos, K.L., 2007. Improved statistical inference for the two-parameter Birnbaum-Saunders distribution. Computational Statistics & Data Analysis, 51(9), 46564681. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions, Vol. 2, 2nd ed., Wiley-Interscience. Rausand, M. & Høyland, A., 2004. System reliability theory, WileyIEEE. Gamma Continuous Distribution 99 4.3. Gamma Continuous Distribution Probability Density Function - f(t) 0.30 k=0.5 ฮป=1 k=3 ฮป=1 k=6 ฮป=1 k=12 ฮป=1 0.25 0.20 0.15 0.10 0.05 0.00 0 10 0.50 0.45 0.40 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 20 k=3 ฮป=1.8 k=3 ฮป=1.2 k=3 ฮป=0.8 k=3 ฮป=0.4 0 2 4 6 8 0 2 4 6 8 0 2 4 6 8 Cumulative Density Function - F(t) 1.00 0.80 0.80 0.60 0.60 0.40 Gamma 1.00 0.40 k=0.5 ฮป=1 k=3 ฮป=1 k=6 ฮป=1 k=12 ฮป=1 0.20 0.00 0 10 0.20 0.00 20 Hazard Rate - h(t) 1.60 1.60 1.40 1.40 1.20 1.20 1.00 1.00 0.80 0.80 0.60 0.60 0.40 0.40 0.20 0.20 0.00 0.00 0 10 20 100 Univariate Continuous Distributions Parameters & Description ๐๐ ๐๐ > 0 Parameters ๐๐ ๐๐ > 0 Limits Distribution When k is an integer (Erlang distribution) Scale Parameter: Equal to the rate (frequency) of events/shocks. Sometimes defined as 1/๐๐ where ๐๐ is the average time between events/shocks. Shape Parameter: As an integer ๐๐ can be interpreted as the number of events/shocks until failure. When not restricted to an integer, ๐๐ and be interpreted as a measure of the ability to resist shocks. ๐ก๐ก โฅ 0 When k is continuous Gamma ฮ(๐๐) is the complete gamma function. ฮ(๐๐, ๐ก๐ก) and ๐พ๐พ(๐๐, ๐ก๐ก) are the incomplete gamma functions see section 1.6. ๐๐(๐ก๐ก) = PDF ๐๐๐๐ ๐ก๐ก ๐๐โ1 โฮปt e (k โ 1)! ๐๐โ1 (๐๐๐๐)๐๐ ๐น๐น(๐ก๐ก) = 1 โ ๐๐ โ๐๐๐๐ ๏ฟฝ ๐๐! CDF ๐๐=0 ๐๐โ1 (๐๐๐๐)๐๐ ๐ ๐ (๐ก๐ก) = ๐๐ โ๐๐๐๐ ๏ฟฝ ๐๐! Reliability ๐๐=0 Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life Residual ๐๐ โ๐๐๐๐ [๐๐(๐ก๐ก + ๐ฅ๐ฅ)]๐๐ ๐๐! (๐๐๐๐)๐๐ ๐๐โ1 โ๐๐=0 ๐๐! โ๐๐โ1 ๐๐=0 ๐๐(๐ก๐ก) = ๐๐๐๐ ๐ก๐ก ๐๐โ1 โฮปt e ฮ(๐๐) with Laplace transformation: ๐๐ k ๐๐(๐ ๐ ) = ๏ฟฝ ๏ฟฝ ๐๐ + ๐ ๐ ๐น๐น(๐ก๐ก) = ๐พ๐พ(๐๐, ๐๐๐๐) ๐ค๐ค(๐๐) ๐ ๐ (๐ก๐ก) = ๐ค๐ค(๐๐, ๐๐๐๐) ๐ค๐ค(๐๐) = ๐๐๐๐ 1 ๏ฟฝ ๐ฅ๐ฅ ๐๐โ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐ ๐ค๐ค(๐๐) 0 = โ 1 ๏ฟฝ ๐ฅ๐ฅ ๐๐โ1 ๐๐ โ๐ฅ๐ฅ ๐๐๐๐ ๐ค๐ค(๐๐) ๐๐๐๐ ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ๐ค๐ค(๐๐, ๐๐๐๐ + ๐๐๐๐)) = ๐ ๐ (๐ก๐ก) ๐ค๐ค(๐๐, ๐๐๐๐) Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. ๐ข๐ข(๐ก๐ก) = โ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ ๐ ๐ (๐ก๐ก) ๐ข๐ข(๐ก๐ก) = โ โซ๐ก๐ก ๐ค๐ค(๐๐, ๐๐๐๐)๐๐๐๐ ๐ค๐ค(๐๐, ๐๐๐๐) The mean residual life does not have a closed form but has the expansion: Gamma Continuous Distribution 101 ๐๐ โ 1 (๐๐ โ 1)(๐๐ โ 2) + + ๐๐(๐ก๐ก โ3 ) ๐ก๐ก ๐ก๐ก 2 Where ๐๐(๐ก๐ก โ3 ) is Landau's notation. (Kleiber & Kotz 2003, p.161) ๐ข๐ข(๐ก๐ก) = 1 + โ(๐ก๐ก) = ๐๐๐๐ ๐ก๐ก ๐๐โ1 (๐๐๐๐)๐๐ ๐ค๐ค(๐๐) โ๐๐โ1 ๐๐=0 ๐๐! โ(๐ก๐ก) = ๐๐๐๐ ๐ก๐ก ๐๐โ1 โ๐๐๐๐ ๐๐ ๐ค๐ค(๐๐, ๐๐๐๐) Series expansion of the hazard rate is: (Kleiber & Kotz 2003, p.161) โ1 (๐๐ โ 1)(๐๐ โ 2) โ(๐ก๐ก) = ๏ฟฝ + ๐๐(๐ก๐ก โ3 )๏ฟฝ 2 ๐ก๐ก Limits of โ(๐ก๐ก) (Rausand & Høyland 2004) Hazard Rate lim โ(๐ก๐ก) = โ ๐๐๐๐๐๐ lim โ(๐ก๐ก) = ๐๐ ๐ค๐คโ๐๐๐๐ 0 < ๐๐ < 1 ๐ก๐กโ0 ๐ก๐กโโ lim โ(๐ก๐ก) = 0 ๐๐๐๐๐๐ lim โ(๐ก๐ก) = ๐๐ ๐ค๐คโ๐๐๐๐ ๐๐ โฅ 1 ๐ก๐กโ0 ๐ก๐กโโ ๐๐โ1 ๐ป๐ป(๐ก๐ก) = ๐๐๐๐ โ ๐๐๐๐ ๏ฟฝ๏ฟฝ ๐๐=0 (๐๐๐๐)๐๐ ๏ฟฝ ๐๐! ๐ป๐ป(๐ก๐ก) = โ ๐๐๐๐ ๏ฟฝ Properties and Moments ๐ค๐ค(๐๐, ๐๐๐๐) ๏ฟฝ ๐ค๐ค(๐๐) Numerically solve for t when: ๐ก๐ก0.5 = ๐น๐น โ1 (0.5; ๐๐, ๐๐) or ฮณ(k, ฮปt) = ฮ(k, ฮปt) Median where ฮณ(k, ฮปt) is the lower incomplete gamma function, see section 1.6.6. ๐๐ โ 1 ๐๐๐๐๐๐ ๐๐ โฅ 1 ๐๐ Mode Mean - 1st Raw Moment Variance - 2nd Central Moment Skewness - 3rd Central Moment Excess kurtosis - 4th Central Moment Characteristic Function 100ฮฑ% Percentile Function ๐๐๐๐ ๐๐๐๐๐๐๐๐ ๐๐๐๐๐๐ 0 < ๐๐ < 1 k ๐๐ k ๐๐2 2/โ๐๐ 6/๐๐ it โk ๏ฟฝ1 โ ๏ฟฝ ฮป Numerically solve for t: ๐ก๐ก๐ผ๐ผ = ๐น๐น โ1 (๐ผ๐ผ; ๐๐, ๐๐) Gamma Cumulative Hazard Rate 102 Univariate Continuous Distributions Parameter Estimation Maximum Likelihood Function ๐ฟ๐ฟ(๐๐, ๐๐|๐ธ๐ธ) = Likelihood Functions Log-Likelihood Functions โฮ =0 โk Gamma โฮ =0 โฮป Point Estimates Fisher Information Matrix nF ๐๐๐๐๐๐๐น๐น ๏ฟฝ ๐ก๐ก๐๐ ๐๐โ1 eโฮปti ฮ(๐๐)๐๐๐น๐น ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝi=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures nF nF i=1 i=1 ฮ(๐๐, ๐๐|๐ธ๐ธ) = ๐๐๐๐๐น๐น ln(๐๐) โ ๐๐๐น๐น ln๏ฟฝ๐ค๐ค(๐๐)๏ฟฝ + (k โ 1) ๏ฟฝ ln(t i ) โ ฮป ๏ฟฝ t i nF where ๐๐(๐ฅ๐ฅ) = ๐๐ ๐๐๐๐ 0 = ๐๐๐น๐น ln(ฮป) โ ๐๐๐น๐น ๐๐(๐๐) + ๏ฟฝ{ln(t i )} i=1 ln[ฮ(๐ฅ๐ฅ)] is the digamma function see section 1.6.7. nF ๐๐๐๐๐น๐น 0= โ ๏ฟฝ ti ฮป i=1 Point estimates for ๐๐๏ฟฝ and ๐๐ฬ are obtained by using numerical methods to solve the simultaneous equations above. (Kleiber & Kotz 2003, p.165) ๐ผ๐ผ(๐๐, ๐๐) = ๏ฟฝ ๐๐ 2 ๐๐ โฒ (๐๐) ๐๐ ๐๐ ๏ฟฝ ๐๐๐๐2 โ2 where ๐๐ โฒ (๐ฅ๐ฅ) = 2 ๐๐๐๐ฮ(๐ฅ๐ฅ) = โโ is the Trigamma function. ๐๐=0(๐ฅ๐ฅ + ๐๐) ๐๐๐ฅ๐ฅ (Yang and Berger 1998, p.10) Confidence Intervals For a large number of samples the Fisher information matrix can be used to estimate confidence intervals. Bayesian Type Uniform Improper Prior with limits: ๐๐ โ (0, โ) ๐๐ โ (0, โ) Jeffreyโs Prior Reference Order: {๐๐, ๐๐} Reference Order: {๐๐, ๐๐} Prior Non-informative Priors, ๐ ๐ (๐๐, ๐๐) (Yang and Berger 1998, p.6) Posterior 1 No Closed Form ๐๐๏ฟฝ๐๐. ๐๐ โฒ (๐๐) โ 1 No Closed Form ๐๐๏ฟฝ๐๐ โฒ (๐๐) No Closed Form ๐๐๏ฟฝ๐๐. ๐๐ โฒ (๐๐) โ 1 ๐ผ๐ผ No Closed Form Gamma Continuous Distribution where ๐๐ โฒ (๐ฅ๐ฅ) = UOI ฮ from ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ฮ) ฮ from ๐๐๐๐๐๐๐๐(๐๐; ฮ๐ก๐ก) ๐๐ where ๐๐ = ๐ผ๐ผ โ๐ฝ๐ฝ from ๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) ๐๐ 2 ๐๐๐ฅ๐ฅ 2 Likelihood Model Exponential Poisson Weibull with known ๐ฝ๐ฝ Gamma with ๐๐ known ๐๐ = from ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ฅ๐ฅ; ๐๐, ๐๐) ๐๐๐ธ๐ธ ๐ผ๐ผ from ๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐ผ๐ผ) Pareto with known ๐๐ where: โ2 is the Trigamma function ๐๐๐๐ฮ(๐ฅ๐ฅ) = โโ ๐๐=0(๐ฅ๐ฅ + ๐๐) Conjugate Priors Evidence ๐๐๐น๐น failures in ๐ก๐ก ๐๐ ๐๐๐น๐น failures in ๐ก๐ก ๐๐ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ ๐๐๐น๐น failures in ๐ก๐ก ๐๐ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ ๐ก๐ก ๐๐ = โ t Fi + Dist. of UOI Prior Para Posterior Parameters Gamma ๐๐0 , ๐๐0 ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ๐๐ = ๐๐๐๐ + ๐ก๐ก ๐๐ Gamma ๐๐0 , ๐๐0 Gamma Gamma Gamma Gamma โ t Si ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ๐๐ = ๐๐๐๐ + ๐ก๐ก ๐๐ ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ๐๐๐น๐น ๐๐0 , ๐๐0 ๐๐0 , ๐๐0 ๐ฝ๐ฝ ๐๐ = ๐๐๐๐ + ๏ฟฝ ๐ก๐ก๐๐ ๐๐=1 (Rinne 2008, p.520) ๐๐ = ๐๐๐๐ + ๐๐๐น๐น /2 ๐๐ 1 ๐๐ = ๐๐๐๐ + ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 2 ๐๐0 , ฮ0 k 0 , ฮป0 = ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐๐๐๐ ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก ๐๐=1 ๐๐ = ๐๐0 + ๐๐๐น๐น ๐๐๐ธ๐ธ ฮ = ฮ๐๐ + ๐ก๐ก ๐๐ k = k ๐๐ + ๐๐๐น๐น ๐๐๐น๐น ๐ฅ๐ฅ๐๐ ฮป = ฮป๐๐ + ๏ฟฝ ln ๏ฟฝ ๏ฟฝ ๐๐ ๐๐=1 Description , Limitations and Uses Example 1 For an example using the gamma distribution as a conjugate prior see the Poisson or Exponential distributions. A renewal process has an exponential time between failure with parameter ๐๐ = 0.01 under the homogeneous Poisson process conditions. What is the probability the forth failure will occur before 200 hours. Example 2 ๐น๐น(200; 4,0.01) = 0.1429 5 components are put on a test with the following failure times: 38, 42, 44, 46, 55 hours Solving: 5๐๐ 0= โ 225 ฮป 0 = 5ln(ฮป) โ 5๐๐(๐๐) + 18.9954 Gamma ๐๐ 2 Normal with from known ๐๐ 2 ๐๐๐๐๐๐๐๐(๐ฅ๐ฅ; ๐๐, ๐๐ ) 103 104 Univariate Continuous Distributions Gives: k๏ฟฝ = 21.377 ๐๐ฬ = 0.4749 90% confidence interval for ๐๐: ๏ฟฝ๐ฝ๐ฝ๐๐ ๏ฟฝ๐๐๏ฟฝ, ๐๐ฬ๏ฟฝ๏ฟฝ โ1 0.0479 0.4749 ๐ผ๐ผ(๐๐, ๐๐) = ๏ฟฝ ๏ฟฝ 0.4749 4.8205 = ๏ฟฝ๐๐๐น๐น ๐ผ๐ผ๏ฟฝ๐๐๏ฟฝ, ๐๐ฬ๏ฟฝ๏ฟฝ โ1 ๐ท๐ทโ1 (0.95)โ179.979 ๏ฟฝ, ๏ฟฝ๐๐๏ฟฝ. exp ๏ฟฝ โ๐๐๏ฟฝ [7.6142, ๐ท๐ท โ1 (0.95)โ179.979 ๐๐๏ฟฝ . exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐๏ฟฝ 60.0143] 90% confidence interval for ๐๐: Gamma ๏ฟฝ๐๐ฬ. exp ๏ฟฝ 179.979 โ17.730 =๏ฟฝ ๏ฟฝ โ17.730 1.7881 ๐ท๐ทโ1 (0.95)โ1.7881 ๏ฟฝ, โ๐๐ฬ [0.0046, ๐ท๐ทโ1 (0.95)โ1.7881 ๐๐ฬ. exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐ฬ 48.766] Note that this confidence interval uses the assumption of the parameters being normally distributed which is only true for large sample sizes. Therefore these confidence intervals may be inaccurate. Bayesian methods must be done numerically. Characteristics The gamma distribution was originally known as a Pearson Type III distribution. This distribution includes a location parameter ๐พ๐พ which shifts the distribution along the x-axis. ๐๐(๐ก๐ก; ๐๐, ๐๐, ๐พ๐พ) = ๐๐๐๐ (๐ก๐ก โ ๐พ๐พ)๐๐โ1 โฮป(tโฮณ) e ฮ(๐๐) When k is an integer, the Gamma distribution is called an Erlang distribution. ๐๐ Characteristics: ๐๐ < 1. ๐๐(0) = โ. There is no mode. ๐๐ = ๐๐. ๐๐(0) = ๐๐. The gamma distribution reduces to an exponential distribution with failure rate ฮป. Mode at ๐ก๐ก = 0. ๐๐ > 1. ๐๐(0) = 0 Large ๐๐. The gamma distribution approaches a normal ๐๐ ๐๐ distribution with ๐๐ = , ๐๐ = ๏ฟฝ 2 . ๐๐ ๐๐ Homogeneous Poisson Process (HPP). Components with an exponential time to failure which undergo instantaneous renewal with an identical item undergo a HPP. The Gamma distribution is Gamma Continuous Distribution 105 probability distribution of the kth failed item and is derived from the convolution of ๐๐ exponentially distributed random variables, ๐๐๐๐ . (See related distributions, exponential distribution). Scaling property: ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) ๐๐ ๐๐๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐๐, ๏ฟฝ ๐๐ Convolution property: ๐๐1 + ๐๐2 + โฆ + ๐๐๐๐ ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(โ๐๐๐๐ , ๐๐) Where ๐๐ is fixed. Properties from (Leemis & McQueston 2008) Renewal Theory, Homogenous Poisson Process. Used to model a renewal process where the component time to failure is exponentially distributed and the component is replaced instantaneously with a new identical component. The HPP can also be used to model ruin theory (used in risk assessments) and queuing theory. System Failure. Can be used to model system failure with ๐๐ backup systems. Life Distribution. The gamma distribution is flexible in shape and can give good approximations to life data. Bayesian Analysis. The gamma distribution is often used as a prior in Bayesian analysis to produce closed form posteriors. Online: http://mathworld.wolfram.com/GammaDistribution.html http://en.wikipedia.org/wiki/Gamma_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web calculator) http://www.itl.nist.gov/div898/handbook/eda/section3/eda366b.htm Resources Books: Artin, E., 1964. The Gamma Function, New York: Holt, Rinehart & Winston. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous Univariate Distributions, Vol. 1 2nd ed., Wiley-Interscience. Bowman, K.O. & Shenton, L.R., 1988. Properties of estimators for the gamma distribution, CRC Press. Relationship to Other Distributions Generalized Gamma Distribution Gamma Applications 106 Univariate Continuous Distributions ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; ๐๐, ๐๐, ฮณ, ๐๐) ๐๐(๐ก๐ก; ๐๐, ๐๐, ๐พ๐พ, ๐๐) = ๐๐๐๐๐๐๐๐ (๐ก๐ก โ ๐พ๐พ)๐๐๐๐โ1 exp{โ[๐๐(๐ก๐ก โ ๐พ๐พ)]๐๐ } ฮ(๐๐) ๐๐ - Scale Parameter ๐๐ - Shape Parameter ๐พ๐พ - Location parameter ๐๐ - Second shape parameter Gamma The generalized gamma distribution has been derived because it is a generalization of a large amount of probability distributions. Such as: ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; 1, ๐๐, 0,1) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; 1, , ฮฒ, 1๏ฟฝ = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐, ๐ฝ๐ฝ) ๐๐ 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; 1, , 0, ๐ฝ๐ฝ๏ฟฝ = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) ๐ผ๐ผ 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; 1, , ฮณ, ๐ฝ๐ฝ๏ฟฝ = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ, ๐พ๐พ) ๐ผ๐ผ ๐๐ 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; , , 0,1๏ฟฝ = ฯ2 (t; n) 2 2 ๐๐ 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; , , 0,2๏ฟฝ = ฯ(t; n) 2 โ2 1 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐ก๐ก; 1, , 0,2๏ฟฝ = Rayleigh(t; ฯ) ๐๐ Let Then Exponential Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ฮป) Special Case: Then ๐๐๐๐๐๐๐๐(๐๐; ๐๐๐๐) ๐๐๐๐๐๐ ๐๐๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐๐ ๐๐๐ก๐ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) This is gives the Gamma distribution its convolution property. Let Poisson Distribution ๐๐1 โฆ ๐๐๐๐ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; ๐๐ = 1, ๐๐) ๐๐1 โฆ ๐๐๐๐ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐๐๐ ๐๐๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐๐ ๐๐๐ก๐ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) The Poisson distribution is the probability that exactly ๐๐ failures have been observed in time ๐ก๐ก. This is the probability that ๐ก๐ก is between ๐๐๐๐ and ๐๐๐๐+1 . ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐๐๐) = ๏ฟฝ ๐๐+1 ๐๐ where ๐๐ is an integer. ๐๐๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐ฅ๐ฅ, ๐๐)๐๐๐๐ = ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐๐ + 1, ๐๐) โ ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐๐, ๐๐) Gamma Continuous Distribution Normal Distribution ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐) Chi-square Distribution ๐๐ 2 (๐ก๐ก; ๐ฃ๐ฃ) Inverse Gamma Distribution ๐ผ๐ผ๐ผ๐ผ(๐ก๐ก; ๐ผ๐ผ, ฮฒ) Special Case for large k: lim ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) = ๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐ = ๐๐โโ Special Case: ๐๐ 2 (๐ก๐ก; ๐ฃ๐ฃ) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ก๐ก; ๐๐ = where ๐ฃ๐ฃ is an integer Let Then Let Beta Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; ฮฑ, ฮฒ) ๐ท๐ท๐ท๐ท๐๐๐๐ (๐๐; ๐๐) Then Let: Then: Let: Then: ๐๐ ๐๐ , ๐๐ = ๏ฟฝ 2 ๏ฟฝ ๐๐ ๐๐ ๐ฃ๐ฃ 1 , ๐๐ = ) 2 2 X~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) ๐๐๐๐๐๐ Y= 1 X ๐๐1 , ๐๐2 ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(k i , ฮปi ) ๐๐๐๐๐๐ Y= X1 X1 + X 2 Y~I๐บ๐บ(๐ผ๐ผ = ๐๐, ฮฒ = ฮป) Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ = ๐๐1 , ๐ฝ๐ฝ = ๐๐2 ) ๐๐๐๐ ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐๐๐ ) ๐๐. ๐๐. ๐๐ ๐๐๐๐๐๐ ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, โ๐๐๐๐ ) ๐๐ ๐๐ = ๏ฟฝ ๐๐๐๐ ๐๐=1 ๐๐1 ๐๐2 ๐๐๐๐ ๐๐ = ๏ฟฝ , , โฆ , ๏ฟฝ ๐๐ ๐๐ ๐๐ ๐๐~๐ท๐ท๐ท๐ท๐๐๐๐ (๐ผ๐ผ1 , โฆ , ๐ผ๐ผ๐๐ ) *i.i.d: independent and identically distributed Wishart Distribution ๐๐๐๐๐๐โ๐๐๐๐๐ก๐ก๐๐ (๐๐; ๐บ๐บ) The Wishart Distribution is the multivariate generalization of the gamma distribution. Gamma Dirichlet Distribution 107 108 Univariate Continuous Distributions 4.4. Logistic Continuous Distribution Probability Density Function - f(t) 0.3 ฮผ=0 ฮผ=2 ฮผ=4 0.2 0.2 s=0.7 s=1 s=2 0.3 0.2 0.1 0.1 0.1 0.0 0.0 -6 -1 4 9 -7 -2 3 Logistic Cumulative Density Function - F(t) 1.0 1.0 0.8 0.8 0.6 0.6 0.4 ฮผ=0 s=1 ฮผ=2 s=1 ฮผ=4 s=1 0.2 0.0 0.4 s=0.7 s=1 s=2 0.2 0.0 -6 -1 4 9 -7 -2 3 Hazard Rate - h(t) 1.0 ฮผ=0 ฮผ=2 ฮผ=4 0.8 0.6 ฮผ=0 s=0.7 ฮผ=0 s=1 ฮผ=0 s=2 1.4 1.2 1.0 0.8 0.6 0.4 0.4 0.2 0.2 0.0 0.0 -6 -1 4 9 -7 -2 3 Logistic Continuous Distribution 109 Parameters & Description Parameters µ ๐ ๐ Limits Location parameter. ๐๐ is the mean, median and mode of the distribution. โโ < ๐๐ < โ Scale parameter. Proportional to the standard deviation of the distribution. s>0 โโ < t < โ Distribution Formulas ๐๐(๐ก๐ก) = PDF where = ez eโz = z 2 ) s(1 + e s(1 + eโz )2 ๐ก๐ก โ ๐๐ 1 sech2 ๏ฟฝ ๏ฟฝ 2๐ ๐ 4๐ ๐ CDF = 1 ez = 1 + eโz 1 + ez 1 1 ๐ก๐ก โ ๐๐ + tanh ๏ฟฝ ๏ฟฝ 2 2 2๐ ๐ R(t) = Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life ๐ก๐ก โ ๐๐ ๐ ๐ Residual Hazard Rate Cumulative Hazard Rate ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = Logistic ๐น๐น(๐ก๐ก) = ๐ง๐ง = 1 1 + ez ๐ก๐ก โ ๐๐ 1 + exp ๏ฟฝ ๏ฟฝ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ๐ ๐ = ๐ก๐ก + ๐ฅ๐ฅ โ ๐๐ ๐ ๐ (๐ก๐ก) 1 + exp ๏ฟฝ ๏ฟฝ ๐ ๐ Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. t ๐ข๐ข(๐ก๐ก) = (1 + ๐๐ z ) ๏ฟฝs. ln ๏ฟฝ๐๐ ๏ฟฝs + ๐๐ ๐๐๏ฟฝ ๐ ๐ ๏ฟฝ 1 ๐น๐น(๐ก๐ก) = s(1 + eโz ) ๐ ๐ 1 = ๐๐ โ ๐ก๐ก s + s exp ๏ฟฝ ๏ฟฝ ๐ ๐ โ(๐ก๐ก) = ๐ก๐ก โ ๐๐ ๐ป๐ป(๐ก๐ก) = ln ๏ฟฝ1 + exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ ๐ โ t๏ฟฝ 110 Univariate Continuous Distributions Properties and Moments µ Median µ Mode µ Mean - 1st Raw Moment ฯ2 2 s 3 Variance - 2nd Central Moment 0 Skewness - 3rd Central Moment Excess kurtosis - 4th 6 5 Central Moment ๐๐ ๐๐๐๐๐๐ ๐ต๐ต(1 โ ๐๐๐๐๐๐, 1 + ๐๐๐๐๐๐) for |๐ ๐ ๐ ๐ | < 1 Characteristic Function 100๐พ๐พ % Percentile Function Parameter Estimation ๐พ๐พ ๐ก๐ก๐พ๐พ = ๐๐ + ๐ ๐ ln ๏ฟฝ ๏ฟฝ 1 โ ๐พ๐พ Logistic Plotting Method Least Mean Square ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ X-Axis Y-Axis Likelihood Function For complete data: ti ln[F] โ ln[1 โ ๐น๐น] Maximum Likelihood Function โฮ =0 โµ โฮ =0 โs 1 m ๐๐ฬ = โ๐๐๐ ๐ ฬ ๐ก๐ก๐๐ โ ๐๐ ๏ฟฝ โ๐ ๐ ๐ก๐ก๐๐ โ ๐๐ 2 i=1 s ๏ฟฝ1 + exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ โ๐ ๐ nF ๐ฟ๐ฟ(๐๐, ๐ ๐ |๐ธ๐ธ) = ๏ฟฝ Log-Likelihood Function ๐ ๐ ฬ = ๐๐๐น๐น exp ๏ฟฝ failures ๐๐๐น๐น ๐ก๐ก๐๐ โ ๐๐ ๐ก๐ก๐๐ โ ๐๐ ฮ(๐๐, ๐ ๐ |๐ธ๐ธ) = โnF ln s + ๏ฟฝ ๏ฟฝ ๏ฟฝ โ 2 ๏ฟฝ ln ๏ฟฝ1 + exp ๏ฟฝ ๏ฟฝ๏ฟฝ โ๐ ๐ โ๐ ๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 ๐๐=1 ๐๐๐น๐น failures โฮ nF 2 1 = โ ๏ฟฝ ๐ก๐ก๐๐ โ ๐๐ = 0 โµ s ๐ ๐ ๐๐=1 ๏ฟฝ1 + exp ๏ฟฝ ๐ ๐ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures ๐๐๐น๐น ๐ก๐ก๐๐ โ ๐๐ โฮ nF 1 ๐ก๐ก๐๐ โ ๐๐ 1 โ exp ๏ฟฝ ๐ ๐ ๏ฟฝ = โ โ ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ ๐ก๐ก โ ๐๐ ๏ฟฝ = 0 โs s ๐ ๐ ๐ ๐ 1 + exp ๏ฟฝ ๐๐ ๏ฟฝ ๐๐=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐ ๐ failures Logistic Continuous Distribution MLE Point Estimates 111 The MLE estimates for ๐๐ฬ and ๐ ๐ ฬ are found by solving the following equations: ๐๐๐น๐น ๐ก๐ก๐๐ โ ๐๐ โ1 1 1 โ ๏ฟฝ ๏ฟฝ1 + exp ๏ฟฝ ๏ฟฝ๏ฟฝ = 0 ๐ ๐ 2 nF ๐๐=1 ๐๐๐น๐น ๐ก๐ก๐๐ โ ๐๐ ๐ก๐ก๐๐ โ ๐๐ 1 โ exp ๏ฟฝ ๐ ๐ ๏ฟฝ 1 ๏ฟฝ 1 + ๏ฟฝ๏ฟฝ ๐ก๐ก โ ๐๐ = 0 ๐ ๐ nF 1 + exp ๏ฟฝ ๐๐ ๏ฟฝ ๐๐=1 ๐ ๐ These estimates are biased. (Balakrishnan 1991) provides tables derived from Monte Carlo simulation to correct the bias. Fisher Information (Antle et al. 1970) 100๐พ๐พ% Confidence Intervals 1 3๐ ๐ 2 ๐ผ๐ผ(๐๐, ๐ ๐ ) = ๏ฟฝ 0 ๐๐ 2 0 +3 9๐ ๐ 2 ๏ฟฝ Confidence intervals are most often obtained from tables derived from Monte Carlo simulation. Corrections from using the Fisher Information matrix method are given in (Antle et al. 1970). Type Non-informative Priors ๐ ๐ ๐๐ (๐๐, ๐๐) Prior 1 ๐ ๐ Jeffery Prior Description , Limitations and Uses Example The accuracy of a cutting machine used in manufacturing is desired to be measured. 5 cuts at the required length are made and measured as: 7.436, 10.270, 10.466, 11.039, 11.854 ๐๐๐๐ Numerically solving MLE equations gives: ๐๐ฬ = 10.446 ๐ ๐ ฬ = 0.815 This gives a mean of 10.446 and a variance of 2.183. Compared to the same data used in the Normal distribution section it can be seen that this estimate is very similar to a normal distribution. 90% confidence interval for ๐๐: 3๐ ๐ ฬ 2 ๏ฟฝ๐๐ฬ โ ฮฆโ1 (0.95)๏ฟฝ , ๐๐๐น๐น 3๐ ๐ ฬ 2 ๐๐ฬ + ฮฆโ1 (0.95)๏ฟฝ ๏ฟฝ ๐๐๐น๐น [9.408, 11.4844] Logistic Bayesian 112 Univariate Continuous Distributions 90% confidence interval for ๐ ๐ : โก โง โ1 โซ 9๐ ๐ ฬ 2 (0.95)๏ฟฝ โข โช๐ท๐ท ๐๐๐น๐น (3 + ๐๐ 2 )โช โข๐ ๐ ฬ . exp , โ๐ ๐ ฬ โจ โฌ โข โข โช โช โฃ โฉ โญ โง โ1 โซโค 9๐ ๐ ฬ 2 (0.95)๏ฟฝ โช๐ท๐ท ๐๐๐น๐น (3 + ๐๐ 2 )โชโฅ โฅ ๐ ๐ ฬ . exp ๐ ๐ ฬ โจ โฌโฅ โช โชโฅ โฉ โญโฆ [0.441, 1.501] Note that this confidence interval uses the assumption of the parameters being normally distributed which is only true for large sample sizes. Therefore these confidence intervals may be inaccurate. Bayesian methods must be calculated using numerical methods. Logistic Characteristics The logistic distribution is most often used to model growth rates (and has been used extensively in biology and chemical applications). In reliability engineering it is most often used as a life distribution. Shape. There is no shape parameter and so the logistic distribution is always a bell shaped curve. Increasing ๐๐ shifts the curve to the right, increasing ๐ ๐ increases the spread of the curve. Normal Distribution. The shape of the logistic distribution is very similar to that of a normal distribution with the logistic distribution having slightly โlonger tailsโ. It would take a large number of samples to distinguish between the distributions. The main difference is that the hazard rate approaches 1/๐ ๐ for large ๐ก๐ก. The logistic function has historically been preferred over the normal distribution because of its simplified form. (Meeker & Escobar 1998, p.89) Alternative Parameterization. It is equally as popular to present the logistic distribution using the true standard deviation ๐๐ = ๐๐๐๐โโ3. This form is used in reference book, Balakrishnan 1991, and gives the following cdf: ๐น๐น(๐ก๐ก) = 1 โ๐๐ ๐ก๐ก โ ๐๐ 1 + exp ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐ โ3 Standard Logistic Distribution. The standard logistic distribution has ๐๐ = 0, ๐ ๐ = 1. The standard logistic distribution random variable, ๐๐, is related to the logistic distribution: ๐๐ = ๐๐ โ ๐๐ ๐ ๐ Logistic Continuous Distribution 113 Let: ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐, ๐ ๐ ) Scaling property (Leemis & McQueston 2008) ๐๐๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐, ๐๐๐๐) Rate Relationships. The distribution has the following rate relationships which make it suitable for modeling growth (Hastings et al. 2000, p.127): โ(๐ก๐ก) = where ๐ง๐ง = ln ๏ฟฝ ๐น๐น(๐ก๐ก) ๏ฟฝ = ln[๐น๐น(๐ก๐ก)] โ ln[1 โ ๐น๐น(๐ก๐ก)] ๐ ๐ (๐ก๐ก) when ๐๐ = 0 and ๐ ๐ = 1: ๐ง๐ง = ๐ก๐ก โ ๐๐ ๐ ๐ ๐๐๐๐(๐ก๐ก) = ๐น๐น(๐ก๐ก)๐ ๐ (๐ก๐ก) ๐๐๐๐ Growth Model. The logistic distribution most common use is a growth model. Probability of Detection. The cdf of logistic distribution is commonly used to represent the probability of detection damaged materials sensors and detection instruments. For example probability of detection of embedded flaws in metals using ultrasonic signals. Life Distribution. In reliability applications it is used as a life distribution. It is similar in shape to a normal distribution and so is often used instead of a normal distribution due to its simplified form. (Meeker & Escobar 1998, p.89) Logistic Regression. Logistic regression is a generalized linear regression model used predict binary outcomes. (Agresti 2002) Resources Online: http://mathworld.wolfram.com/LogisticDistribution.html http://en.wikipedia.org/wiki/Logistic_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) http://www.weibull.com/LifeDataWeb/the_logistic_distribution.htm Books: Balakrishnan, 1991. Handbook of the Logistic Distribution 1st ed., CRC. Logistic ๐๐(๐ก๐ก) = Applications f(t) ๐น๐น(๐ก๐ก) = R(t) ๐ ๐ 114 Univariate Continuous Distributions Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience. Relationship to Other Distributions Let Exponential Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) Then ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐ = 1) ๐๐๐๐๐๐ ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1) (Hastings et al. 2000, p.127) Let Pareto Distribution ๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) ๐๐ โ๐๐ ๐๐ = ln ๏ฟฝ ๏ฟฝ 1 + ๐๐ โ๐๐ Then ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) X ฮฑ ๐๐ = โ ln ๏ฟฝ๏ฟฝ ๏ฟฝ โ 1๏ฟฝ ฮธ ๐๐๐๐๐๐ ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1) (Hastings et al. 2000, p.127) Logistic Let Gumbel Distribution ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ผ๐ผ , ๐ฝ๐ฝ) Then ๐๐๐๐ ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ผ๐ผ , ๐ฝ๐ฝ) ๐๐๐๐๐๐ ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐๐๐๐๐๐๐๐๐๐๐๐(0, ๐ฝ๐ฝ) (Hastings et al. 2000, p.127) ๐๐ = X1 โ X2 Normal Continuous Distribution 115 4.5. Normal (Gaussian) Continuous Distribution Probability Density Function - f(t) ฮผ=-3 ฯ=1 ฮผ=0 ฯ=1 ฮผ=3 ฯ=1 0.5 0.4 ฮผ=0 ฯ=0.6 ฮผ=0 ฯ=1 ฮผ=0 ฯ=2 0.6 0.3 0.4 0.2 0.2 0.1 0.0 -6 -4 0.0 0 -2 2 4 6 -6 -4 -2 0 2 4 6 0 2 4 6 Cumulative Density Function - F(t) -4 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 -2 Hazard Rate - h(t) ฮผ=-3 ฯ=1 6 ฮผ=0 ฯ=1 5 ฮผ=3 ฯ=1 -6 0 -4 -2 2 4 6 -6 -4 -2 4 6 ฮผ=-3 ฯ=0.6 ฮผ=0 ฯ=1 5 ฮผ=0 ฯ=2 4 3 3 2 2 1 1 0 0 0 2 4 6 -6 -4 Normal -6 1.0 -2 0 2 4 6 116 Univariate Continuous Distributions Parameters & Description Parameters ๐๐ ๐๐ 2 Limits Location parameter: The mean of the distribution. โโ < µ < โ ฯ2 > 0 Scale parameter: The standard deviation of the distribution. โโ < t < โ Distribution Formulas 1 tโµ 2 exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ฯ ฯโ2๐๐ 1 ๐ก๐ก โ µ ๏ฟฝ = ๐๐ ๏ฟฝ ฯ ฯ ๐๐(๐ก๐ก) = PDF 1 where ๐๐ is the standard normal pdf with ๐๐ = 0 and ๐๐ 2 = 1. Normal ๐น๐น(๐ก๐ก) = ๐ก๐ก 1 ฮธโµ 2 ๏ฟฝ exp ๏ฟฝ โ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๐๐๐๐ 2 ฯ ฯโ2๐๐ โโ 1 = CDF 1 1 ๐ก๐ก โ µ + erf ๏ฟฝ ๏ฟฝ 2 2 ฯโ2 tโµ ๏ฟฝ = ฮฆ๏ฟฝ ฯ where ฮฆ is the standard normal cdf with ๐๐ = 0 and ๐๐ 2 = 1. tโµ R(t) = 1 โ ฮฆ ๏ฟฝ ๏ฟฝ ฯ µโt = ฮฆ๏ฟฝ ๏ฟฝ ฯ Reliability Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life Residual Hazard Rate Cumulative Hazard Rate µโxโt ๏ฟฝ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ฮฆ ๏ฟฝ ฯ ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = = µโt ๐ ๐ (๐ก๐ก) ฮฆ๏ฟฝ ๏ฟฝ ฯ Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. ๐ข๐ข(๐ก๐ก) = โ โ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ = ๐ ๐ (๐ก๐ก) ๐ ๐ (๐ก๐ก) tโµ ๐๐ ๏ฟฝ ๏ฟฝ ฯ โ(๐ก๐ก) = µโt ๐๐ ๏ฟฝฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ ฯ ๐๐ โ ๐ก๐ก ๐ป๐ป(๐ก๐ก) = โ ln ๏ฟฝ๐ท๐ท ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐ Normal Continuous Distribution 117 Properties and Moments µ Median µ Mode Mean - 1st Variance - µ Raw Moment 2nd ฯ2 Central Moment 0 Skewness - 3rd Central Moment 0 Excess kurtosis - 4th Central Moment 1 exp ๏ฟฝ๐๐๐๐๐๐ โ 2๐๐ 2 ๐ก๐ก 2 ๏ฟฝ Characteristic Function ๐ก๐ก๐ผ๐ผ = µ + ฯฮฆโ1 (ฮฑ) = µ + ฯโ2 erf โ1 (2๐ผ๐ผ โ 1) 100๐ผ๐ผ% Percentile Function Parameter Estimation Plotting Method X-Axis Y-Axis ๐ก๐ก๐๐ ๐๐๐๐๐๐๐๐๐๐๐๐๐๐[๐น๐น(๐ก๐ก๐๐ )] ๏ฟฝ= ฯ Maximum Likelihood Function Likelihood Function For complete data: ๐ฟ๐ฟ(๐๐, ๐๐|๐ธ๐ธ) = = Log-Likelihood Function µ๏ฟฝ = โ 1 ๏ฟฝ2 1 , ฯ = 2 ๐๐ ๐๐ nF 1 ๐ก๐ก๐๐ โ ๐๐ 2 ๏ฟฝ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ nF 2 ๐๐ i=1 ๏ฟฝฯโ2ฯ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ 1 failures ๐๐๐น๐น 1 ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 ๏ฟฝ nF exp ๏ฟฝโ 2๐๐ 2 ๏ฟฝฯโ2ฯ๏ฟฝ ๐๐=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ 1 failures ๐๐๐น๐น 1 ฮ(๐๐, ๐๐|๐ธ๐ธ) = โnF ln๏ฟฝฯโ2ฯ๏ฟฝ โ 2 ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 2๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 โฮ =0 โµ solve for ๐๐ to get MLE ๐๐ฬ : โฮ =0 โฯ solve for ๐๐ to get ๐๐๏ฟฝ: failures ๐๐๐น๐น โฮ ๐๐๐๐๐น๐น 1 = 2 โ 2 ๏ฟฝ ๐ก๐ก๐๐ = 0 โµ ๐๐ ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures ๐๐๐น๐น โฮ nF 1 = โ + 3 ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 = 0 โฯ ฯ ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures ๐๐ ๐๐ Normal Least Mean Square ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ 118 Univariate Continuous Distributions MLE Point Estimates When there is only complete failure data the point estimates can be given as: ๐๐๐น๐น ๐๐๐น๐น 1 ๐๐ฬ = ๏ฟฝ ๐ก๐ก๐๐ nF ๏ฟฝ2 = 1 ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 ๐๐ nF ๐๐=1 ๐๐=1 In most cases the unbiased estimators are used: ๐๐๐น๐น 1 ๐๐ฬ = ๏ฟฝ ๐ก๐ก๐๐ nF ๐๐=1 Fisher Information 100๐พ๐พ% Confidence Intervals Normal (for complete data) ๏ฟฝ2 = ๐๐ ๐ผ๐ผ(๐๐, ๐๐ 2 ) = ๏ฟฝ 1 Sided - Lower ๐๐ ๐๐๐๐ ๐๐ฬ โ ๏ฟฝ2 ๐๐ ๐๐๏ฟฝ โ๐๐ ๐ก๐กฮณ (n โ 1) (๐๐ โ 1) ๐๐๐ผ๐ผ2 (๐๐ โ 1) ๐๐๐น๐น 1 ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 nF โ 1 ๐๐=1 1/๐๐ 2 0 0 ๏ฟฝ โ1/2๐๐ 4 2 Sided - Lower ๐๐ฬ โ ๏ฟฝ2 ๐๐ ๐๐๏ฟฝ โ๐๐ ๐ก๐ก๏ฟฝ1+ฮณ๏ฟฝ (n โ 1) 2 (๐๐ โ 1) 2 Sided - Upper ๐๐ฬ + ๏ฟฝ2 ๐๐ ๐๐ 21+ฮณ (๐๐ โ 1) ๏ฟฝ 2 ๏ฟฝ ๐๐๏ฟฝ โ๐๐ ๐ก๐ก๏ฟฝ1+ฮณ๏ฟฝ (n โ 1) 2 (๐๐ โ 1) ๐๐ 21โฮณ (๐๐ โ 1) ๏ฟฝ 2 ๏ฟฝ (Nelson 1982, pp.218-220) Where ๐ก๐กฮณ (n โ 1) is the 100๐พ๐พth percentile of the t-distribution with ๐๐ โ 1 degrees of freedom and ๐๐๐พ๐พ2 (๐๐ โ 1) is the 100๐พ๐พth percentile of the ๐๐ 2 -distribution with ๐๐ โ 1 degrees of freedom. Bayesian Type Non-informative Priors when ๐๐๐๐ is known, ๐ ๐ ๐๐ (๐๐) (Yang and Berger 1998, p.22) Prior 1 ๐๐ โ ๐๐ Uniform Proper Prior with limits ๐๐ โ [๐๐, ๐๐] 1 All Type Uniform Proper Prior with limits ๐๐ 2 โ [๐๐, ๐๐] Posterior Truncated Normal Distribution For a โค µ โค b โ๐๐๐น๐น ๐ก๐ก๐๐๐น๐น ฯ2 ๐๐. ๐๐๐๐๐๐๐๐ ๏ฟฝµ; ๐๐=1 , ๏ฟฝ ๐๐๐น๐น nF Otherwise ๐๐(๐๐) = 0 ๐๐๐๐๐๐๐๐ ๏ฟฝµ; when ๐๐ โ (โ, โ) ๐น๐น โ๐๐๐๐=1 ๐ก๐ก๐๐๐น๐น ฯ2 , ๏ฟฝ ๐๐๐น๐น nF Non-informative Priors when ๐๐ is known, ๐ ๐ ๐๐ (๐๐๐๐ ) (Yang and Berger 1998, p.23) Prior Posterior 1 ๐๐ โ ๐๐ Truncated Inverse Gamma Distribution For a โค ฯ2 โค b Normal Continuous Distribution 119 (๐๐๐น๐น โ 2) S2 , ๏ฟฝ 2 2 Otherwise ๐๐(๐๐ 2 ) = 0 ๐๐. ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; 1 Uniform Improper Prior with limits ๐๐ 2 โ (0, โ) 1 ๐๐ 2 Jefferyโs, Reference, MDIP Prior Type ๐๐๐น๐น S2 , ๏ฟฝ 2 2 2 with limits ๐๐ โ (0, โ) See section 1.7.1 ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; Non-informative Priors when ๐๐ and ๐๐๐๐ are unknown, ๐ ๐ ๐๐ (๐๐, ๐๐๐๐ ) (Yang and Berger 1998, p.23) Prior Posterior 1 Jefferyโs Prior 1 ๐๐ 4 Reference Prior ordering {๐๐, ๐๐} Reference where ๐๐ and ๐๐ 2 are separate groups. ๐๐๐๐ (๐๐, ๐๐ 2 ) 1 โ ๐๐๏ฟฝ2 + ๐๐ 2 where ๐๐ = ๐๐/๐๐ 1 ๐๐ 2 S2 ๏ฟฝ nF (nF โ 3) See section 1.7.2 (๐๐๐น๐น โ 3) S2 2 ๐๐(๐๐ |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; , ๏ฟฝ 2 2 See section 1.7.1 ๐๐(๐๐|๐ธ๐ธ)~๐๐ ๏ฟฝµ; nF โ 3, ๐ก๐กฬ , S2 ๏ฟฝ nF (nF + 1) when ๐๐ โ (โ, โ) See section 1.7.2 (๐๐๐น๐น + 1) S2 ๐๐(๐๐ 2 |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; , ๏ฟฝ 2 2 2 when ๐๐ โ (0, โ) See section 1.7.1 ๐๐(๐๐|๐ธ๐ธ)~๐๐ ๏ฟฝµ; nF + 1, ๐ก๐กฬ , No Closed Form S2 ๏ฟฝ nF (nF โ 1) when ๐๐ โ (โ, โ) See section 1.7.2 (๐๐๐น๐น โ 1) S2 ๐๐(๐๐ 2 |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; , ๏ฟฝ 2 2 2 when ๐๐ โ (0, โ) See section 1.7.1 ๐๐(๐๐|๐ธ๐ธ)~๐๐ ๏ฟฝµ; nF โ 1, ๐ก๐กฬ , Normal Improper Uniform with limits: ๐๐ โ (โ, โ) ๐๐ 2 โ (0, โ) MDIP Prior (๐๐๐น๐น โ 2) S2 , ๏ฟฝ 2 2 See section 1.7.1 ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯ2 ; 120 Univariate Continuous Distributions where ๐๐๐น๐น ๐๐ = ๏ฟฝ(๐ก๐ก๐๐ โ ๐ก๐กฬ )2 2 ๐๐=1 and ๐ก๐กฬ = ๐๐๐น๐น 1 ๏ฟฝ ๐ก๐ก๐๐ nF ๐๐=1 Conjugate Priors UOI Likelihood Model Evidence Dist. of UOI Prior Para Posterior Parameters ๐๐ ๐๐ from Normal ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐ 2 ) ๐๐ 2 from 2 ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐ ) ๐๐๐๐ from ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐๐๐ , ๐๐๐๐2 ) Normal with known ๐๐ 2 ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Normal with known ๐๐ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Lognormal with known ๐๐๐๐2 ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Normal Gamma u๐๐ , ๐ฃ๐ฃ0 ๐๐0 , ๐๐0 ๐ฃ๐ฃ = 1 1 nF + ๐ฃ๐ฃ0 ๐๐ 2 ๐๐ = ๐๐๐๐ + ๐๐๐น๐น /2 ๐๐๐น๐น 1 ๐๐ = ๐๐๐๐ + ๏ฟฝ(๐ก๐ก๐๐ โ ๐๐)2 2 ๐๐=1 ๐๐ Normal ๐ข๐ข๐๐ , ๐ฃ๐ฃ0 ๐น๐น ln(๐ก๐ก๐๐ ) u0 โ๐๐=1 2+ ฯ0 ๐๐๐๐2 ๐ข๐ข = 1 ๐๐๐น๐น + ๐ฃ๐ฃ 2 ๐๐๐๐2 ๐ฃ๐ฃ = Description , Limitations and Uses Example ๐น๐น ๐น๐น ๐ก๐ก๐๐ ๐ข๐ข0 โ๐๐=1 + ๐ฃ๐ฃ0 ๐๐ 2 ๐ข๐ข = 1 ๐๐๐น๐น + ๐ฃ๐ฃ0 ๐๐ 2 1 1 ๐๐๐น๐น + ๐ฃ๐ฃ 2 ๐๐๐๐2 The accuracy of a cutting machine used in manufacturing is desired to be measured. 5 cuts at the required length are made and measured as: 7.436, 10.270, 10.466, 11.039, 11.854 ๐๐๐๐ MLE Estimates are: โ ๐ก๐ก๐๐๐น๐น = 10.213 nF 2 โ๏ฟฝ๐ก๐ก๐๐๐น๐น โ ๐๐๏ฟฝ๐ก๐ก ๏ฟฝ ๏ฟฝ2 = ๐๐ = 2.789 nF โ 1 ๐๐ฬ = 90% confidence interval for ๐๐: ๐๐๏ฟฝ ๐๐๏ฟฝ ๏ฟฝ๐๐ฬ โ ๐ก๐ก{0.95} (4), ๐๐ฬ + ๐ก๐ก{0.95} (4)๏ฟฝ โ5 โ5 [10.163, 10.262] Normal Continuous Distribution 121 90% confidence interval for ๐๐ 2 : 4 4 ๏ฟฝ2 ๏ฟฝ2 , ๐๐ ๏ฟฝ ๏ฟฝ๐๐ ๐๐{20.95} (4) ๐๐{20.05} (4) [1.176, 15.697] A Bayesian point estimate using the Jeffery non-informative improper prior 1โ๐๐ 4 with posterior for ๐๐~๐๐(6, 10.213, 0.558 ) and ๐๐ 2 ~๐ผ๐ผ๐ผ๐ผ(3, 5.578) has a point estimates: ๐๐ฬ = E[๐๐(6,6.595,0.412 )] = µ = 10.213 ๏ฟฝ2 = E[๐ผ๐ผ๐บ๐บ(3,5.578)] = 5.578 = 2.789 ๐๐ 2 ๐น๐น๐๐โ1 (0.95) = 11.665] 1/๐น๐น๐บ๐บโ1 (0.05) = 6.822] Also known as a Gaussian distribution or bell curve. Unit Normal Distribution. Also known as the standard normal distribution is when ๐๐ = 0 and ๐๐ = 1 with pdf ๐๐(๐ง๐ง) and cdf ฮฆ(z). If X is normally distributed with mean ๐๐ and standard deviation ๐๐ then the following transformation is used: ๐ฅ๐ฅ โ ๐๐ ๐ง๐ง = ๐๐ Central Limit Theorem. Let ๐๐1 , ๐๐2 , โฆ , ๐๐๐๐ be a sequence of ๐๐ independent and identically distributed (i.i.d) random variables each having a mean of ๐๐ and a variance of ๐๐ 2 . As the sample size increases, the distribution of the sample average of these random variables approaches the normal distribution with mean ๐๐ and variance ๐๐ 2 /๐๐ irrespective of the shape of the original distribution. Formally: ๐๐๐๐ = ๐๐1 + โฏ + ๐๐๐๐ If we define a new random variables: ๐๐๐๐ โ ๐๐๐๐ ๐๐๐๐ = , ๐๐๐๐๐๐ ๐๐โ๐๐ ๐๐ = ๐๐๐๐ ๐๐ The distribution of ๐๐๐๐ converges to the standard normal distribution. The distribution of ๐๐๐๐ converges to a normal distribution with mean ๐๐ and standard deviation of ๐๐/โ๐๐. Sigma Intervals. Often intervals of the normal distribution are expressed in terms of distance away from the mean in units of sigma. Normal Characteristics With 90% confidence intervals: ๐๐ [๐น๐น๐๐โ1 (0.05) = 8.761, 2 ๐๐ [1/๐น๐น๐บ๐บโ1 (0.95) = 0.886, 122 Univariate Continuous Distributions The following is approximate values for each sigma: Interval ๐๐ ± ๐๐ ๐๐ ± 2๐๐ ๐๐ ± 3๐๐ ๐๐ ± 4๐๐ ๐๐ ± 5๐๐ ๐๐ ± 6๐๐ ๐ฝ๐ฝ(๐๐ + ๐๐๐๐) โ ๐ฝ๐ฝ(๐๐ โ ๐๐๐๐) 68.2689492137% 95.4499736104% 99.7300203937% 99.9936657516% 99.9999426697% 99.9999998027% Truncated Normal. Often in reliability engineering a truncated normal distribution may be used due to the limitation that ๐ก๐ก โฅ 0. See Truncated Normal Continuous Distribution. Inflection Points: Inflection points occur one standard deviation away from the mean (๐๐ ± ๐๐). Mean / Median / Mode: The mean, median and mode are always equal to ๐๐. Normal Hazard Rate. The hazard rate is increasing for all ๐ก๐ก. The Standard Normal Distributionโs hazard rate approaches โ(๐ก๐ก) = ๐ก๐ก as ๐ก๐ก becomes large. Let: Convolution Property ๐๐ Scaling Property X~๐๐๐๐๐๐๐๐(µ, ฯ2 ) ๏ฟฝ ๐๐๐๐ ~๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ ๐๐๐๐ , โ๐๐๐๐2 ๏ฟฝ ๐๐=1 ๐๐๐๐ + ๐๐~๐๐๐๐๐๐๐๐(๐๐๐๐ + ๐๐, ๐๐ 2 ๐๐ 2 ) Linear Combination Property: ๐๐ ๏ฟฝ ๐๐๐๐ ๐๐๐๐ + ๐๐๐๐ ~๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ{๐๐๐๐ ๐๐๐๐ + ๐๐๐๐ } , โ๏ฟฝ๐๐๐๐2 ๐๐๐๐2 ๏ฟฝ๏ฟฝ ๐๐=1 Applications Approximations to Other Distributions. The origin of the Normal Distribution was from an approximation of the Binomial distribution. Due to the Central Limit Theory the Normal distribution can be used to approximate many distributions as detailed under โRelated Distributionsโ. Strength Stress Interference. When the strength of a component follows a distribution and the stress that component is subjected to follows a distribution there exists a probability that the stress will be greater than the strength. When both distributions are a normal distribution, there is a closed for solution to the interference Normal Continuous Distribution 123 probability. Life Distribution. When used as a life distribution a truncated Normal Distribution may be used due to the constraint ๐ก๐ก โฅ 0. However it is often found that the difference in results is negligible. (Rausand & Høyland 2004) Time Distributions. The normal distribution may be used to model simple repair or inspection tasks that have a typical duration with variation which is symmetrical about the mean. This is typical for inspection and preventative maintenance times. Analysis of Variance (ANOVA). A test used to analyze variance and dependence of variables. A popular model used to conduct ANOVA assumes the data comes from a normal population. Online: http://www.weibull.com/LifeDataWeb/the_normal_distribution.htm http://mathworld.wolfram.com/NormalDistribution.html http://en.wikipedia.org/wiki/Normal_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution 2nd ed., CRC. Simon, M.K., 2006. Probability Distributions Involving Gaussian Random Variables: A Handbook for Engineers and Scientists, Springer. Relationship to Other Distributions Truncated Normal Distribution Let: ๐๐๐๐๐๐๐๐๐๐(๐ฅ๐ฅ; ๐๐, ๐๐, ๐๐๐ฟ๐ฟ , ๐๐๐๐ ) Then: Lognormal Distribution Let: ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐ก๐ก; ๐๐๐๐ , ๐๐๐๐2 ) Then: Where: ๐๐~๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 ) ๐๐ โ (โ, โ) ๐๐~T๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 , ๐๐๐ฟ๐ฟ , ๐๐๐๐ ) ๐๐ โ [๐๐๐ฟ๐ฟ , ๐๐๐๐ ] ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐๐๐๐ , ฯ2N ) ๐๐ = ln(๐๐) ๐๐~๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 ) Normal Resources Six Sigma Quality Management. Six sigma is a business management strategy which aims to reduce costs in manufacturing processes by removing variance in quality (defects). Current manufacturing standards aim for an expected 3.4 defects out of one million parts: 2ฮฆ(โ6). (Six Sigma Academy 2009) 124 Univariate Continuous Distributions Rayleigh Distribution ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐ก๐ก; ๐๐) ๐๐2 ๐๐๐๐ = ln ๏ฟฝ ๏ฟฝ, 2 ๏ฟฝ๐๐ + ๐๐2 Let Then Normal Binomial Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) Poisson Distribution ๐๐๐๐๐๐๐๐(k; µ) Beta Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; ๐ผ๐ผ, ๐ฝ๐ฝ) Gamma Distribution ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) ๐๐๐๐๐๐ Y~๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ ๐ โ(๐๐) Let Chi-square Distribution ๐๐ 2 (๐ก๐ก; ๐ฃ๐ฃ) ๐๐1 , ๐๐2 ~๐๐๐๐๐๐๐๐(0, ฯ) ๐๐ 2 + ๐๐2 ๐๐๐๐ = ๏ฟฝln ๏ฟฝ ๏ฟฝ ๐๐2 Then ๐๐๐๐ ~๐๐๐๐๐๐๐๐(µ, ฯ2 ) ๐๐๐๐๐๐ Y~๐๐ 2 (๐ก๐ก; ๐ฃ๐ฃ) Y = ๏ฟฝX12 + X22 v Xk โ µ 2 Y = ๏ฟฝ๏ฟฝ ๏ฟฝ ฯ k=1 Limiting Case for constant ๐๐: lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) = ๐๐๐๐๐๐๐๐๏ฟฝk; µ = n๐๐, ๐๐ 2 = ๐๐๐๐(1 โ ๐๐)๏ฟฝ ๐๐โโ ๐๐=๐๐ The Normal distribution can be used as an approximation of the Binomial distribution when ๐๐๐๐ โฅ 10 and ๐๐๐๐(1 โ ๐๐) โฅ 10. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) โ ๐๐๐๐๐๐๐๐๏ฟฝ๐ก๐ก = ๐๐ + 0.5; ๐๐ = ๐๐๐๐, ๐๐ 2 = ๐๐๐๐(1 โ ๐๐)๏ฟฝ lim ๐น๐น๐๐๐๐๐๐๐๐ (๐๐; ๐๐) = ๐น๐น๐๐๐๐๐๐๐๐ (๐๐; ๐๐โฒ = ๐๐, ๐๐ = ๏ฟฝ๐๐) ๐๐โโ This is a good approximation when ๐๐ > 1000. When ๐๐ > 10 the same approximation can be made with a correction: lim ๐น๐น๐๐๐๐๐๐๐๐ (๐๐; ๐๐) = ๐น๐น๐๐๐๐๐๐๐๐ (๐๐; ๐๐โฒ = ๐๐ โ 0.5, ๐๐ = ๏ฟฝ๐๐) ๐๐โโ For large ๐ผ๐ผ and ๐ฝ๐ฝ with fixed ๐ผ๐ผ/๐ฝ๐ฝ: ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ, ๐ฝ๐ฝ) โ ๐๐๐๐๐๐๐๐ ๏ฟฝµ = ฮฑ ฮฑฮฒ , ๐๐ = ๏ฟฝ ๏ฟฝ 2 (ฮฑ (ฮฑ + ฮฒ) + ฮฒ + 1) ฮฑ+ฮฒ As ๐ผ๐ผ and ๐ฝ๐ฝ increase the mean remains constant and the variance is reduced. Special Case for large k: lim ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) = ๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐ = ๐๐โโ ๐๐ ๐๐ , ๐๐ = ๏ฟฝ 2 ๏ฟฝ ๐๐ ๐๐ Pareto Continuous Distribution 125 4.6. Pareto Continuous Distribution Probability Density Function - f(t) 1.0 ฮธ=1 ฯ=1 ฮธ=2 ฯ=1 ฮธ=3 ฯ=1 0.8 0.6 0.4 0.2 0.0 0 2 4 6 8 Cumulative Density Function - F(t) 2 4 6 8 Hazard Rate - h(t) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 2 4 6 2 4 6 8 8 ฮธ=1 ฯ=1 ฮธ=1 ฯ=2 ฮธ=1 ฯ=4 0 2 4 6 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 ฮธ=1 ฯ=1 ฮธ=2 ฯ=1 ฮธ=3 ฯ=1 0 0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 ฮธ=1 ฯ=1 ฮธ=2 ฯ=1 ฮธ=3 ฯ=1 0 ฮธ=1 ฯ=1 ฮธ=1 ฯ=2 ฮธ=1 ฯ=4 Pareto 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 8 ฮธ=1 ฯ=1 ฮธ=1 ฯ=2 ฮธ=1 ฯ=4 0 2 4 6 8 126 Univariate Continuous Distributions Parameters & Description Parameters Limits ฮธ ฮธ>0 ๐ผ๐ผ ฮฑ>0 Distribution ฮฑฮธฮฑ t ฮฑ+1 ฮธ ฮฑ ๐น๐น(๐ก๐ก) = 1 โ ๏ฟฝ ๏ฟฝ t ฮธ ฮฑ R(t) = ๏ฟฝ ๏ฟฝ t Reliability Pareto ฮธโคt< โ ๐๐(๐ก๐ก) = CDF Mean Residual Life Shape parameter. Sometimes called the Pareto index. Formulas PDF Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Location parameter. ฮธ is the lower limit of t. Sometimes refered to as tminimum. ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = (t)ฮฑ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) = ๐ ๐ (๐ก๐ก) (t + x)ฮฑ Where ๐ก๐ก is the given time we know the component has survived to time ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. ๐ข๐ข(๐ก๐ก) = โ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ ๐ ๐ (๐ก๐ก) โ(๐ก๐ก) = Hazard Rate Cumulative Hazard Rate ฮฑ t ๐ก๐ก ๐ป๐ป(๐ก๐ก) = ๐ผ๐ผ ln ๏ฟฝ ๏ฟฝ ๐๐ Properties and Moments Median Mode Mean - 1st Raw Moment Variance - 2nd Central Moment Skewness - 3rd Central Moment ฮธ21โฮฑ ฮธ ฮฑฮธ , for ฮฑ > 1 ฮฑโ1 ฮฑฮธ2 , for ฮฑ > 2 (ฮฑ โ 1)2 (ฮฑ โ 2) 2(1 + ฮฑ) ฮฑ โ 2 ๏ฟฝ , for ฮฑ > 3 (ฮฑ โ 3) ฮฑ Pareto Continuous Distribution 127 6(ฮฑ3 + ฮฑ2 โ 6ฮฑ โ 2) , for ฮฑ > 4 ฮฑ(ฮฑ โ 3)(ฮฑ โ 4) Excess kurtosis - 4th Central Moment ๐ผ๐ผ(โ๐๐๐๐๐๐)๐ผ๐ผ ฮ(โ๐ผ๐ผ, โ๐๐๐๐๐๐) Characteristic Function ๐ก๐ก๐พ๐พ = ฮธ(1 โ ฮณ)โ1โฮฑ 100๐พ๐พ % Percentile Function Parameter Estimation Plotting Method Least Mean Square ๐ฆ๐ฆ = ๐๐๐๐ + ๐๐ X-Axis Likelihood Function For complete data: ln(๐ก๐ก๐๐ ) ln[1 โ ๐น๐น] Maximum Likelihood Function nF 1 ๐ฟ๐ฟ(๐๐, ๐ผ๐ผ|๐ธ๐ธ) = ฮฑnF ฮธฮฑnF ๏ฟฝ ฮฑ+1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 t i failures Log-Likelihood Function ๐ผ๐ผ๏ฟฝ = โ๐๐ ๐๐ ๏ฟฝ ๐๐ = exp ๏ฟฝ ๏ฟฝ ๐ผ๐ผ๏ฟฝ ๐๐๐น๐น ฮ(๐๐, ๐ผ๐ผ|๐ธ๐ธ) = nF ln(ฮฑ) + nF ฮฑln(ฮธ) โ (๐ผ๐ผ + 1) ๏ฟฝ ln ๐ก๐ก๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures solve for ๐ผ๐ผ to get ๐ผ๐ผ๏ฟฝ: ๐๐๐น๐น โฮ nF = โ + nF ln ฮธ โ ๏ฟฝ ln ๐ก๐ก๐๐ = 0 โฮฑ ฮฑ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures MLE Point Estimates The likelihood function increases as ๐๐ increases. Therefore the MLE point estimate is the largest ๐๐ which satisfies ฮธ โค t i < โ: ๐๐๏ฟฝ = min๏ฟฝt1 , โฆ , t nF ๏ฟฝ Substituting ๐๐๏ฟฝ gives the MLE for ๐ผ๐ผ๏ฟฝ: ๐ผ๐ผ๏ฟฝ = Fisher Information 100๐พ๐พ% Confidence Intervals ๐๐๐น๐น ๐น๐น โ๐๐๐๐=1 ๏ฟฝln ๐ก๐ก๐๐ โ ln๐๐๏ฟฝ๏ฟฝ ๐ผ๐ผ(๐๐, ๐ผ๐ผ) = ๏ฟฝ 1-Sided Lower ๐ผ๐ผ๏ฟฝ if ๐๐ is unknown ๐ผ๐ผ๏ฟฝ 2 ๐๐ (2n โ 2) 2nF {1โฮณ} โ1/๐ผ๐ผ 2 0 0 ๏ฟฝ 1/๐๐ 2 2-Sided Lower ๐ผ๐ผ๏ฟฝ 2 ๐๐ 1โฮณ (2n โ 2) 2nF ๏ฟฝ 2 ๏ฟฝ 2-Sided Upper ๐ผ๐ผ๏ฟฝ 2 ๐๐ 1+ฮณ (2n โ 2) 2nF ๏ฟฝ 2 ๏ฟฝ Pareto โฮ =0 โฮฑ Y-Axis 128 Univariate Continuous Distributions (for complete data) ๐ผ๐ผ๏ฟฝ 2 ๐๐ (2n) 2nF {1โฮณ} ๐ผ๐ผ๏ฟฝ if ๐๐ is known ๐ผ๐ผ๏ฟฝ 2 ๐๐ 1โฮณ (2n) 2nF ๏ฟฝ 2 ๏ฟฝ ๐ผ๐ผ๏ฟฝ 2 ๐๐ 1+ฮณ (2n โ 2) 2nF ๏ฟฝ 2 ๏ฟฝ (Johnson et al. 1994, p.583) Where ๐๐๐พ๐พ2 (๐๐) is the 100๐พ๐พth percentile of the ๐๐ 2 -distribution with ๐๐ degrees of freedom. Bayesian Non-informative Priors when ๐ฝ๐ฝ is known, ๐ ๐ ๐๐ (๐ถ๐ถ) (Yang and Berger 1998, p.22) Type Prior Jeffery Reference and Pareto Conjugate Priors UOI Likelihood Model Evidence ๐๐ from ๐๐๐๐๐๐๐๐(๐ก๐ก; a, ๐๐) Uniform with known a ๐๐๐น๐น failures at times ๐ก๐ก๐๐ ๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ฮธ, ๐ผ๐ผ) Pareto with known ๐ผ๐ผ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ Pareto with known ๐๐ ๐๐๐น๐น failures at times ๐ก๐ก๐๐ ๐๐ from ๐ผ๐ผ from ๐๐๐๐๐๐๐๐๐๐๐๐(๐ก๐ก; ฮธ, ๐ผ๐ผ) 1 ๐ผ๐ผ Dist. of UOI Prior Para Pareto ๐๐๐๐ , ๐ผ๐ผ0 Pareto Gamma a 0 , ฮ0 k 0 , ฮป0 Description , Limitations and Uses Example Posterior Parameters ๐๐ = max{๐ก๐ก1 , โฆ , ๐ก๐ก๐๐๐น๐น } ๐ผ๐ผ = ๐ผ๐ผ0 + ๐๐๐น๐น a = a๐๐ โ ๐ผ๐ผ๐๐๐น๐น where a0 > ๐ผ๐ผ๐๐๐น๐น ฮ = ฮ0 k = k ๐๐ + ๐๐๐น๐น ๐๐๐น๐น ๐ฅ๐ฅ๐๐ ฮป = ฮป๐๐ + ๏ฟฝ ln ๏ฟฝ ๏ฟฝ ๐๐ ๐๐=1 5 components are put on a test with the following failure times: 108, 125, 458, 893, 13437 hours MLE Estimates are: ๐๐๏ฟฝ = 108 Substituting ๐๐๏ฟฝ gives the MLE for ๐ผ๐ผ๏ฟฝ: ๐ผ๐ผ๏ฟฝ = 5 = 0.8029 ๐น๐น โ๐๐๐๐=1 (ln ๐ก๐ก๐๐ โ ln(108)) 90% confidence interval for ๐ผ๐ผ๏ฟฝ: Pareto Continuous Distribution ๏ฟฝ ๐ผ๐ผ ๏ฟฝ 10 Characteristics ๐๐2 {0.05} (8), ๐ผ๐ผ ๏ฟฝ 10 129 ๐๐2 {0.95} (8)๏ฟฝ [0.2194, 1.2451] 80/20 Rule. Most commonly described as the basis for the โ80/20 ruleโ (In a quality context, for example, 80% of manufacturing defects will be a result from 20% of the causes). Conditional Distribution. The conditional probability distribution given that the event is greater than or equal to a value ๐๐1 exceeding ๐๐ is a Pareto distribution with the same index ๐ผ๐ผ but with a minimum ๐๐1 instead of ๐๐. Types. This distribution is known as a Pareto distribution of the first kind. The Pareto distribution of the second kind (not detailed here) is also known as the Lomax distribution. Pareto also proposed a third distribution now known as a Pareto distribution of the third kind. Pareto and the Lognormal Distribution. The Lognormal distribution models similar physical phenomena as the Pareto distribution. The two distributions have different weights at the extremities. Minimum property Applications Resources For constant ๐๐. Xi ~๐๐๐๐๐๐๐๐๐๐๐๐(ฮธ, ฮฑi ) Pareto Let: ๐๐ min{๐๐, ๐๐2 , โฆ , ๐๐๐๐ }~๐๐๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐, ๏ฟฝ ๐ผ๐ผ๐๐ ๏ฟฝ ๐๐=1 Rare Events. The survival function โslowlyโ decreases compared to most life distributions which makes it suitable for modeling rare events which have large outcomes. Examples include natural events such as the distribution of the daily rain fall, or the size of manufacturing defects. Online: http://mathworld.wolfram.com/ParetoDistribution.html http://en.wikipedia.org/wiki/Pareto_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Arnold, B., 1983. Pareto distributions, Fairland, MD: International Cooperative Pub. House. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous Univariate Distributions, Vol. 1 2nd ed., Wiley-Interscience. Relationship to Other Distributions 130 Univariate Continuous Distributions Exponential Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) Chi-Squared Distribution ฯ2 (๐ฅ๐ฅ; ๐ฃ๐ฃ) Pareto Logistic Distribution ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(µ, ๐ ๐ ) Let Then Let Then ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐ = ๐ผ๐ผ) ๐๐๐๐๐๐ ๐๐~๐๐ 2 (๐ฃ๐ฃ = 2) (Johnson et al. 1994, p.526) Let Then ๐๐~๐๐๐๐๐๐๐๐๐๐๐๐(๐๐, ๐ผ๐ผ) ๐๐ = ln(๐๐โ๐๐) ๐๐๐๐๐๐ ๐๐๐๐๐๐ ๐๐ = 2ฮฑ ln(๐๐โ๐๐) X ฮฑ ๐๐ = โ ln ๏ฟฝ๏ฟฝ ๏ฟฝ โ 1๏ฟฝ ฮธ ๐๐~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1) (Hastings et al. 2000, p.127) Triangle Continuous Distribution 131 4.7. Triangle Continuous Distribution Probability Density Function - f(t) a=0 b=1 c=0.5 a=0 b=1.5 c=0.5 a=0 b=2 c=0.5 2.0 1.5 1.0 0.5 0.0 0 0.5 1 1.5 2.0 1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 2 a=0 b=1 c=0.25 a=0 b=1 c=0.5 a=0 b=1 c=0.75 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Cumulative Density Function - F(t) 1.0 0.8 0.8 0.6 0.6 0.4 Triangle 1.0 0.4 a=0 b=1 c=0.5 a=0 b=1.5 c=0.5 a=0 b=2 c=0.5 0.2 0.0 0.2 0.0 0 0.5 1 1.5 2 Hazard Rate - h(t) 10 9 8 7 6 5 4 3 2 1 0 0 0.5 10 9 8 7 6 5 4 3 2 1 0 1 1.5 2 a=0 b=1 c=0.25 a=0 b=1 c=0.5 a=0 b=1 c=0.75 0 0.5 1 132 Univariate Continuous Distributions Parameters & Description Parameters Random Variable ๐๐ โโ โค ๐๐ < ๐๐ ๐๐ ๐๐ โค ๐๐ โค ๐๐ ๐๐ ๐๐ < ๐๐ < โ Distribution Minimum Value. ๐๐ is the lower bound Maximum Value. ๐๐ is the upper bound. Mode Value. ๐๐ is the mode of the distribution (top of the triangle). ๐๐ โค ๐ก๐ก โค ๐๐ Formulas 2(t โ a) โง for a โค t โค c โช (b โ a)(c โ a) ๐๐(๐ก๐ก) = โจ 2(b โ t) โช(b โ a)(b โ c) for c โค t โค b โฉ PDF (t โ a)2 โง for a โค t โค c โช (b โ a)(c โ a) ๐น๐น(๐ก๐ก) = 2 (b โ t) โจ โช1 โ (b โ a)(b โ c) for c โค t โค b โฉ Triangle CDF (t โ a)2 โง1 โ for a โค t โค c โช (b โ a)(c โ a) ๐ ๐ (๐ก๐ก) = 2 โจ (b โ t) for c โค t โค b โช(b โ a)(b โ c) โฉ Reliability Properties and Moments Median 1 ๐๐ + ๏ฟฝ2(๐๐ โ ๐๐)(๐๐ โ ๐๐) ๐๐๐๐๐๐ ๐๐ โฅ 1 ๐๐ โ ๏ฟฝ2(๐๐ โ ๐๐)(๐๐ โ ๐๐) ๐๐๐๐๐๐ ๐๐ < Mode Mean - 1st Raw Moment Variance - 2nd Central Moment Skewness - 3rd Central Moment Excess kurtosis - 4th Central Moment ๐๐ ๐๐ โ ๐๐ 2 ๐๐ โ ๐๐ 2 ๐๐ + ๐๐ + ๐๐ 3 ๐๐2 + ๐๐ 2 + ๐๐ 2 โ ๐๐๐๐ โ ๐๐๐๐ โ ๐๐๐๐ 18 โ2(๐๐ + ๐๐ โ 2๐๐)(2๐๐ โ ๐๐ โ ๐๐)(๐๐ โ 2๐๐ + ๐๐) 5(๐๐2 + ๐๐ 2 + ๐๐ 2 โ ๐๐๐๐ โ ๐๐๐๐ โ ๐๐๐๐)3/2 โ3 5 Triangle Continuous Distribution Characteristic Function โ2 100ฮณ % Percentile Function 133 (b โ c)eita โ (b โ a)eitc + (c โ a)eitb (b โ a)(c โ a)(b โ c)t 2 ๐ก๐ก๐พ๐พ = a + ๏ฟฝฮณ(b โ a)(c โ a) ๐๐๐๐๐๐ ๐พ๐พ < ๐น๐น(๐๐) ๐ก๐ก๐พ๐พ = b โ ๏ฟฝ(1 โ ฮณ)(b โ a)(b โ c) ๐๐๐๐๐๐ ๐พ๐พ โฅ ๐น๐น(๐๐) Parameter Estimation Maximum Likelihood Function r nF 2(t i โ a) 2(b โ t i ) ๐ฟ๐ฟ(๐๐, ๐๐, ๐๐|๐ธ๐ธ) = ๏ฟฝ .๏ฟฝ (b๏ฟฝ๏ฟฝ๏ฟฝ โ a)(c โ๏ฟฝ๏ฟฝ a) ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝi=1 ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=r+1 (b โ a)(b โ c) Likelihood Functions failers to the left of c r 2 ๐๐๐น๐น ti =๏ฟฝ ๏ฟฝ bโa Where failure times are ordered: ๐๐1 โค ๐๐2 โค โฏ โค ๐๐๐๐ โค โฏ โค ๐๐๐๐๐น๐น and ๐๐ is the number of failure times less than ๐๐ and ๐ ๐ is the number of failure times greater than ๐๐. Therefore ๐๐๐น๐น = ๐๐ + ๐ ๐ . The MLE estimates a๏ฟฝ , b๏ฟฝ , and c๏ฟฝ are obtained by numerically calculating the likelihood function for different r and selecting the maximum where c๏ฟฝ = Xr๏ฟฝ . where max ๐๐ โค ๐๐ โค ๐๐ ๐ฟ๐ฟ(๐๐, ๐๐, ๐๐|๐ธ๐ธ) = ๏ฟฝ 2 ๐๐๐น๐น ๏ฟฝ {๐๐(๐๐, ๐๐, ๐๐ฬ (๐๐, ๐๐)} bโa nF ti โ a b โ ti ๏ฟฝ i=r+1 (b โ t r ) i=1 (t r โ a) rโ1 ๐๐(๐๐, ๐๐, ๐๐) = ๏ฟฝ ๐๐(๐๐, ๐๐) = arg max ๐๐(๐๐, ๐๐, ๐๐) ๐๐โ{1,โฆ,๐๐๐น๐น } Note that the MLE estimates for a and ๐๐ are not the same as the uniform distribution: a๏ฟฝ โ min(t1F , t F2 โฆ ) ๏ฟฝb โ max(t1F , t F2 โฆ ) (Kotz & Dorp 2004) Description , Limitations and Uses Example When eliciting an opinion from an expert on the possible value of a quantity, ๐ฅ๐ฅ, the expert may give : - Lowest possible value = 0 - Highest possible value = 1 - Estimate of most likely value (mode) = 0.7 The corresponding distribution for ๐ฅ๐ฅ may be a triangle distribution with parameters: ๐๐ = 0, ๐๐ = 1, ๐๐ = 0.7 Triangle Point Estimates failures to the right of c nF โa b โ ti ๏ฟฝ ๏ฟฝ (c (b โ a) โ c) i=r+1 i=1 134 Univariate Continuous Distributions Characteristics Standard Triangle Distribution. The standard triangle distribution has ๐๐ = 0, ๐๐ = 1. This distribution has a mean at ๏ฟฝ๐๐ โ2 and median at 1 โ ๏ฟฝ(1 โ ๐๐)โ2. Symmetrical Triangle Distribution. The symmetrical triangle distribution occurs when ๐๐ = (๐๐ โ ๐๐)/2. The symmetrical triangle distribution is formed from the average of two uniform random variables (see related distributions). Applications Subjective Representation. The triangle distribution is often used to model subjective evidence where ๐๐ and ๐๐ are the bounds of the estimation and ๐๐ is an estimation of the mode. Triangle Substitution to the Beta Distribution. Due to the triangle distribution having bounded support it may be used in place of the beta distribution. Monte Carlo Simulation. Used to approximate distributions of variables when the underlying distribution is unknown. A distribution of interest is obtained by conducting Monte Carlo simulation of a model using the triangle distributions as inputs. Resources Online: http://mathworld.wolfram.com/TriangularDistribution.html http://en.wikipedia.org/wiki/Triangular_distribution Books: Kotz, S. & Dorp, J.R.V., 2004. Beyond Beta: Other Continuous Families Of Distributions With Bounded Support And Applications, World Scientific Publishing Company. Relationship to Other Distributions Let Uniform Distribution ๐๐๐๐๐๐๐๐(๐ก๐ก; a, b) Beta Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; ฮฑ, ๐ฝ๐ฝ ) Then Xi ~๐๐๐๐๐๐๐๐(๐๐, ๐๐) ๐๐๐๐๐๐ Y~๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ ๏ฟฝa, Special Cases: Y= bโa , b๏ฟฝ 2 X1 + X 2 2 ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1,2 ) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐(0,0,1) ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(2,1 ) = ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐(0,1,1) Truncated Normal Continuous Distribution 135 4.8. Truncated Normal Continuous Distribution Probability Density Function - f(t) ฮผ=-1 ฯ=1 a=0 b=2 ฮผ=0 ฯ=1 a=0 b=2 ฮผ=1 ฯ=1 a=0 b=2 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 -1 0 1 2 ฮผ=0.5 ฯ=0.5 ฮผ=0.5 ฯ=1 ฮผ=0.5 ฯ=3 1.8 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 3 -1 0 1 2 3 Cumulative Density Function - F(t) 1.0 0.8 0.8 0.6 0.6 0.4 0.4 ฮผ=-1 a=0 b=2 ฮผ=0 a=0 b=2 ฮผ=1 a=0 b=2 0.2 0.0 -1 0 1 2 Trunc Normal 1.0 ฮผ=0.5 ฯ=0.5 a=0 ฮผ=0.5 ฯ=1 b=2 ฮผ=0.5 ฯ=3 0.2 0.0 3 -1 0 1 Hazard Rate - h(t) ฮผ=-1 ฯ=1 a=0 b=2 ฮผ=0 ฯ=1 a=0 b=2 ฮผ=1 ฯ=1 a=0 b=2 1.4 1.2 1.0 2 3 ฮผ=0.5 ฯ=0.5 a=0 ฮผ=0.5 ฯ=1 b=2 ฮผ=0.5 ฯ=3 1.2 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 -1 0 1 2 3 -1 0 1 2 3 136 Univariate Continuous Distributions Parameters & Description ๐๐ โโ < µ < โ ๐๐๐ฟ๐ฟ โโ < ๐๐๐ฟ๐ฟ < ๐๐๐๐ ๐๐๐๐ ๐๐๐ฟ๐ฟ < ๐๐๐๐ < โ ๐๐ 2 Parameters ฯ2 > 0 Limits Trunc Normal Distribution Left Truncated Normal ๐ฅ๐ฅ โ [0, โ) for 0 โค ๐ฅ๐ฅ โค โ PDF ๐๐(zx ) ๐๐(๐ฅ๐ฅ) = ฯฮฆ(โz0 ) otherwise ๐๐(๐ฅ๐ฅ) = 0 Location parameter: The mean of the distribution. Scale parameter: The standard deviation of the distribution. Lower Bound: ๐๐๐ฟ๐ฟ is the lower bound. The standard normal transform of ๐๐๐ฟ๐ฟ ๐๐ โ๐๐ is ๐ง๐ง๐๐ = ๐ฟ๐ฟ . ๐๐ Upper Bound: ๐๐๐๐ is the upper bound. The standard normal transform of ๐๐๐๐ ๐๐ โ๐๐ is ๐ง๐ง๐๐ = ๐๐ . ๐๐ ๐๐๐ฟ๐ฟ < ๐ฅ๐ฅ โค ๐๐๐๐ General Truncated Normal ๐ฅ๐ฅ โ [๐๐๐ฟ๐ฟ , ๐๐๐๐ ] for ๐๐๐ฟ๐ฟ โค ๐ฅ๐ฅ โค ๐๐๐๐ 1 ๐๐(zx ) ฯ ๐๐(๐ฅ๐ฅ) = ฮฆ(zb ) โ ฮฆ(za ) otherwise ๐๐(๐ฅ๐ฅ) = 0 where ๐๐ is the standard normal pdf with ๐๐ = 0 and ๐๐ 2 = 1 ฮฆ is the standard normal cdf with ๐๐ = 0 and ๐๐ 2 = 1 ๐๐โ๐๐ ๐ง๐ง๐๐ = ๏ฟฝ ๏ฟฝ ๐๐ for ๐ฅ๐ฅ < 0 CDF ๐น๐น(๐ฅ๐ฅ) = 0 for 0 โค ๐ฅ๐ฅ < โ ๐น๐น(๐ฅ๐ฅ) = for ๐ฅ๐ฅ < 0 Reliability ฮฆ(zx ) โ ฮฆ(z0 ) ฮฆ(โz0 ) ๐ ๐ (๐ฅ๐ฅ) = 1 for 0 โค ๐ฅ๐ฅ < โ ๐ ๐ (๐ฅ๐ฅ) = ฮฆ(z0 ) โ ฮฆ(zx ) ฮฆ(โz0 ) for ๐ฅ๐ฅ < aL ๐น๐น(๐ฅ๐ฅ) = 0 for ๐๐๐ฟ๐ฟ โค ๐ฅ๐ฅ โค ๐๐๐๐ ฮฆ(zx ) โ ฮฆ(za ) ๐น๐น(๐ฅ๐ฅ) = ฮฆ(zb ) โ ฮฆ(za ) for ๐ฅ๐ฅ > bU for ๐ฅ๐ฅ < aL ๐น๐น(๐ฅ๐ฅ) = 1 ๐ ๐ (๐ฅ๐ฅ) = 1 for ๐๐๐ฟ๐ฟ โค ๐ฅ๐ฅ โค ๐๐๐๐ ฮฆ(zb ) โ ฮฆ(zx ) ๐ ๐ (๐ฅ๐ฅ) = ฮฆ(zb ) โ ฮฆ(za ) for ๐ฅ๐ฅ > bU ๐ ๐ (๐ฅ๐ฅ) = 0 Truncated Normal Continuous Distribution for ๐ก๐ก < 0 ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) for ๐ก๐ก < aL ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) for 0 โค ๐ก๐ก < โ Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) 137 for ๐๐๐ฟ๐ฟ โค ๐ก๐ก โค ๐๐๐๐ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ๐ ๐ (๐ก๐ก) 1 โ ฮฆ(zt+x ) = 1 โ ฮฆ(zt ) µโxโt ฮฆ๏ฟฝ ๏ฟฝ ฯ = µโt ฮฆ๏ฟฝ ๏ฟฝ ฯ ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) ๐ ๐ (๐ก๐ก) ฮฆ(zb ) โ ฮฆ(zt+x ) = ฮฆ(zb ) โ ฮฆ(zt ) ๐๐(๐ฅ๐ฅ) = ๐ ๐ (๐ฅ๐ฅ|๐ก๐ก) = for ๐ก๐ก > bU ๐๐(๐ฅ๐ฅ) = 0 ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. This operation is the equivalent of t replacing the lower bound. โ โ(๐ฅ๐ฅ) = 0 for 0 โค ๐ฅ๐ฅ < โ 1 ๐๐(zx )[1 โ ฮฆ(zx )] โ(๐ฅ๐ฅ) = ฯ [1 โ ฮฆ(z0 )]2 Cumulative Hazard Rate Properties and Moments Median ๐ป๐ป(๐ก๐ก) = โ ln[๐ ๐ (๐ก๐ก)] โ(๐ฅ๐ฅ) = 0 for ๐๐๐ฟ๐ฟ โค ๐ฅ๐ฅ โค ๐๐๐๐ 1 โ(๐ฅ๐ฅ) = ฯ for ๐ฅ๐ฅ > bU ๐๐(zx )[ฮฆ(zb ) โ ฮฆ(zx )] [ฮฆ(zb ) โ ฮฆ(za )]2 โ(๐ฅ๐ฅ) = 0 ๐ป๐ป(๐ก๐ก) = โ ln[๐ ๐ (๐ก๐ก)] Left Truncated Normal ๐ฅ๐ฅ โ [0, โ) General Truncated Normal ๐ฅ๐ฅ โ [๐๐๐ฟ๐ฟ , ๐๐๐๐ ] ๐๐ ๐ค๐คโ๐๐๐๐๐๐ ๐๐ โฅ 0 0 ๐ค๐คโ๐๐๐๐๐๐ ๐๐ < 0 ๐๐ ๐ค๐คโ๐๐๐๐๐๐ ๐๐ โ [๐๐๐ฟ๐ฟ , ๐๐๐๐ ] ๐๐๐ฟ๐ฟ ๐ค๐คโ๐๐๐๐๐๐ ๐๐ < ๐๐๐ฟ๐ฟ ๐๐๐๐ ๐ค๐คโ๐๐๐๐๐๐ ๐๐ > ๐๐๐๐ No closed form Mode Mean 1st Raw Moment where Variance 2nd Central Moment for ๐ฅ๐ฅ < aL µ+ ๐๐๐๐(z0 ) ฮฆ(โz0 ) โµ ๐ง๐ง0 = ฯ ฯ2 [1 โ {โฮ0 }2 โ ฮ1 ] where No closed form where µ + ๐๐ ๐ง๐ง๐๐ = where ๐๐(za ) โ ๐๐(zb ) ฮฆ(zb ) โ ฮฆ(za ) ๐๐๐ฟ๐ฟ โ µ , ฯ ๐ง๐ง๐๐ = ๐๐๐๐ โ µ ฯ ฯ2 [1 โ {โฮ0 }2 โ ฮ1 ] Trunc Normal for ๐ฅ๐ฅ < 0 Hazard Rate โ โซ ๐ ๐ (๐ฅ๐ฅ)๐๐๐ฅ๐ฅ โซ๐ก๐ก ๐ ๐ (๐ฅ๐ฅ)๐๐๐๐ ๐ข๐ข(๐ก๐ก) = ๐ก๐ก = ๐ ๐ (๐ก๐ก) ๐ ๐ (๐ก๐ก) Mean Residual Life 138 Univariate Continuous Distributions ฮ๐๐ = ๐ง๐ง0๐๐ ๐๐(๐ง๐ง0 ) ฮฆ(๐ง๐ง0 ) โ 1 โ1 Skewness 3rd Central Moment 3 ๐๐ 2 where ฮ๐๐ = ๐ง๐ง๐๐๐๐ ๐๐(๐ง๐ง๐๐ ) โ ๐ง๐ง๐๐๐๐ ๐๐(๐ง๐ง๐๐ ) ฮฆ(๐ง๐ง๐๐ ) โ ฮฆ(๐ง๐ง๐๐ ) [2ฮ30 + (3ฮ1 โ 1)ฮ0 + ฮ2 ] ๐๐ = 1 โ ฮ1 โ ฮ20 1 [โ3ฮ40 โ 6ฮ1 ฮ20 โ 2ฮ20 โ 4ฮ2 ฮ0 โ 3ฮ1 โ ฮ3 + 3] ๐๐ 2 Excess kurtosis 4th Central Moment Characteristic Function See (Abadir & Magdalinos 2002, pp.1276-1287) 100๐ผ๐ผ% Percentile Function ๐ก๐ก๐ผ๐ผ = µ + ฯฮฆโ1 {ฮฑ + ฮฆ(z0 )[1 โ ฮฑ]} ๐ก๐ก๐ผ๐ผ = µ + ฯฮฆโ1 {ฮฑฮฆ(zb ) + ฮฆ(za )[1 โ ฮฑ]} Trunc Normal Parameter Estimation Maximum Likelihood Function Likelihood Function For limits [๐๐๐ฟ๐ฟ , ๐๐๐๐ ]: ๐ฟ๐ฟ(๐๐, ๐๐, ๐๐๐ฟ๐ฟ , ๐๐๐๐ ) = 1 nF 1 ๐ฅ๐ฅ๐๐ โ ๐๐ 2 ๏ฟฝ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐๐ i=1 ๏ฟฝฯโ2ฯ{ฮฆ(zb ) โ ฮฆ(za )}๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ nF failures For limits [0, โ) ๐๐๐น๐น 1 2 = nF exp ๏ฟฝโ 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐) ๏ฟฝ 2๐๐ ๏ฟฝฯโ2ฯ{ฮฆ(zb ) โ ฮฆ(za )}๏ฟฝ ๐๐=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ 1 failures ๐ฟ๐ฟ(๐๐, ๐๐) = nF 1 ๐ฅ๐ฅ๐๐ โ ๐๐ 2 ๏ฟฝ exp ๏ฟฝโ ๏ฟฝ ๏ฟฝ ๏ฟฝ 2 ๐๐ i=1 ๏ฟฝฮฆ{โz 0 }ฯโ2ฯ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ 1 1 nF failures ๐๐๐น๐น 1 2 = nF exp ๏ฟฝโ 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐) ๏ฟฝ 2๐๐ ๏ฟฝฮฆ{โz0 }ฯโ2ฯ๏ฟฝ ๐๐=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures Log-Likelihood Function For limits [๐๐๐ฟ๐ฟ , ๐๐๐๐ ]: ฮ(๐๐, ๐๐, ๐๐๐ฟ๐ฟ , ๐๐๐๐ |๐ธ๐ธ) ๐๐๐น๐น 1 = โnF ln[ฮฆ(zb ) โ ฮฆ(za )] โ nF ln๏ฟฝฯโ2ฯ๏ฟฝ โ 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐)2 2๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 For limits [0, โ) failures Truncated Normal Continuous Distribution 139 ๐๐๐น๐น 1 ฮ(๐๐, ๐๐|๐ธ๐ธ) = โ๐๐๐น๐น ln(๐ท๐ท{โ๐ง๐ง0 }) โ ๐๐๐น๐น ln๏ฟฝ๐๐โ2๐๐๏ฟฝ โ 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐)2 2๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures ๐๐๐น๐น โฮ =0 โµ โฮ โnF ฯ(za ) โ ฯ(zb ) 1 = ๏ฟฝ ๏ฟฝ + 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐) = 0 ) ) โµ ฯ ฮฆ(zb โ ฮฆ(za ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 failures ๐๐๐น๐น โฮ =0 โฯ โฮ โnF za ฯ(za ) โ zb ฯ(zb ) nF 1 = 2 ๏ฟฝ ๏ฟฝ โ + 3 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐)2 = 0 โฯ ฯ ฮฆ(zb ) โ ฮฆ(za ) ฯ ๐๐ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๐๐=1 MLE Point Estimates First Estimate the values for ๐ง๐ง๐๐ and ๐ง๐ง๐๐ by solving the simultaneous equations numerically (Cohen 1991, p.33): failures Where: ๐ป๐ป2 (๐ง๐ง๐๐ , ๐ง๐ง๐๐ ) = ๐๐๐๐ = ๐๐๐๐ โ ๐๐๐๐ โ ๐ง๐ง๐๐ ๐ฅ๐ฅฬ โ ๐๐๐ฟ๐ฟ = ๐ง๐ง๐๐ โ ๐ง๐ง๐๐ ๐๐๐๐ โ ๐๐๐ฟ๐ฟ 1 + ๐ง๐ง๐๐ ๐๐๐๐ โ ๐ง๐ง๐๐ ๐๐๐๐ โ (๐๐๐๐ โ ๐๐๐๐ )2 ๐ ๐ 2 = 2 (๐ง๐ง๐๐ โ ๐ง๐ง๐๐ ) (๐๐๐๐ โ ๐๐๐ฟ๐ฟ )2 ๐๐(๐ง๐ง๐๐ ) ๐๐(๐ง๐ง๐๐ ) , ๐๐๐๐ = ฮฆ(๐ง๐ง๐๐ ) โ ฮฆ(๐ง๐ง๐๐ ) ฮฆ(๐ง๐ง๐๐ ) โ ฮฆ(๐ง๐ง๐๐ ) ๐ง๐ง๐๐ = ๐๐๐น๐น ๐๐๐ฟ๐ฟ โ ๐๐ , ๐๐ ๐ง๐ง๐๐ = ๐๐๐๐ โ ๐๐ ๐๐ ๐๐๐น๐น 1 1 ๐ฅ๐ฅฬ = ๐น๐น ๏ฟฝ ๐ฅ๐ฅ๐๐ , ๐ ๐ 2 = ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐ฅ๐ฅฬ )2 ๐๐ ๐๐๐น๐น โ 1 0 0 The distribution parameters can then be estimated using: ๐๐๐๐ โ ๐๐๐ฟ๐ฟ ๐๐๏ฟฝ = , ๐๐ฬ = ๐๐๐ฟ๐ฟ โ ๐๐๏ฟฝ๐ง๐ง๏ฟฝ ๐๐ ๐ง๐ง๐๐ โ ๐ง๐ง๏ฟฝ ๏ฟฝ ๐๐ (Cohen 1991, p.44) provides a graphical procedure to estimate parameters to use as the starting point for numerical solvers. For the case where the limits are [0, โ) first numerically solve for ๐ง๐ง0 : 1 โ Q0 (Q0 โ ๐ง๐ง0 ) ๐ ๐ 2 = (Q0 โ ๐ง๐ง0 )2 ๐ฅ๐ฅฬ where ๐๐(๐ง๐ง0 ) ๐๐0 = 1 โ ฮฆ(๐ง๐ง0 ) The distribution parameters can be estimated using: Trunc Normal ๐ป๐ป1 (๐ง๐ง๐๐ , ๐ง๐ง๐๐ ) = 140 Univariate Continuous Distributions ๐๐๏ฟฝ = ๐ฅ๐ฅฬ , ๐๐0 โ ๐ง๐ง๏ฟฝ0 ๐๐ฬ = โ๐๐๏ฟฝ๐ง๐ง๏ฟฝ0 When the limits ๐๐๐ฟ๐ฟ and ๐๐๐๐ are unknown, the likelihood function is maximized when the difference, ฮฆ(zb ) โ ฮฆ(za ), is at its minimum. This occurs when the difference between ๐๐๐๐ โ ๐๐๐ฟ๐ฟ is at its minimum. Therefore the MLE estimates for ๐๐๐ฟ๐ฟ and ๐๐๐๐ are: Fisher Information (Cohen 1991, p.40) a๏ฟฝL = min(t1F , t F2 โฆ ) b๏ฟฝU = max(t1F , t F2 โฆ ) ๐ผ๐ผ(๐๐, ๐๐ Trunc Normal Where 100๐พ๐พ% Confidence Intervals 2) 1 1 2(๐ฅ๐ฅฬ โ ๐๐) โก โค [1 โ ๐๐๐๐โฒ + ๐๐๐๐โฒ ] โ ๐๐๐๐ + ๐๐๐๐ ๏ฟฝ ๏ฟฝ ๐๐ 2 ๐๐ โข ๐๐ 2 โฅ =โข 2 2 โฅ 1 2(๐ฅ๐ฅฬ โ ๐๐) 1 3[๐ ๐ + (๐ฅ๐ฅฬ โ ๐๐) ] โข ๏ฟฝ โ ๐๐๐๐ + ๐๐๐๐ ๏ฟฝ โ 1 โ ๐๐๐๐ + ๐๐๐๐ ๏ฟฝโฅ ๏ฟฝ ๐๐ ๐๐ 2 ๐๐ 2 โฃ๐๐ 2 โฆ ๐๐๐๐โฒ = ๐๐๐๐ (๐๐๐๐ โ ๐ง๐ง๐๐ ), ๐๐๐๐ = ๐๐๐ฟ๐ฟ ๐๐โฒ๐๐ + ๐๐๐๐ , ๐๐๐๐ = ๐๐๐ฟ๐ฟ (๐๐๐๐ + ๐๐๐๐ ), ๐๐๐๐โฒ = โ๐๐๐๐ (๐๐๐๐ + ๐ง๐ง๐๐ ) ๐๐๐๐ = ๐๐๐๐ ๐๐โฒ๐๐ + ๐๐๐๐ ๐๐๐๐ = ๐๐๐๐ (๐๐๐๐ + ๐๐๐๐ ) Calculated from the Fisher information matrix. See section 1.4.7. For further detail and examples see (Cohen 1991, p.41) Bayesian No closed form solutions to priors exist. Description , Limitations and Uses Example 1 The size of washers delivered from a manufacturer is desired to be modeled. The manufacture has already removed all washers below 15.95mm and washers above 16.05mm. The washers received have the following diameters: 15.976, 15.970, 15.955, 16.007, 15.966, 15.952, 15.955 ๐๐๐๐ From data: ๐ฅ๐ฅฬ = 15.973, ๐ ๐ 2 = 4.3950๐ธ๐ธ-4 Using numerical solver MLE Estimates for ๐ง๐ง๐๐ and ๐ง๐ง๐๐ are: Therefore ๐ง๐ง๏ฟฝ ๐๐ = 0, ๐๐๏ฟฝ = ๏ฟฝ ๐ง๐ง๐๐ = 3.3351 ๐๐๐๐ โ ๐๐๐ฟ๐ฟ = 0.029984 ๐ง๐ง๐๐ โ ๐ง๐ง๏ฟฝ ๏ฟฝ ๐๐ ๐๐ฬ = ๐๐๐ฟ๐ฟ โ ๐๐๏ฟฝ๐ง๐ง๏ฟฝ ๐๐ = 15.95 To calculate confidence intervals, first calculate: Truncated Normal Continuous Distribution ๐๐๐๐โฒ = 0.63771, ๐๐๐๐ = 10.970, ๐๐๐๐ = 187.71, 90% confidence intervals: 141 ๐๐๐๐โฒ = โ0.010246 ๐๐๐๐ = โ0.16138 ๐๐๐๐ = โ2.54087 391.57 โ10699 ๐ผ๐ผ(๐๐, ๐๐) = ๏ฟฝ ๏ฟฝ โ10699 โ209183 [๐ฝ๐ฝ๐๐ (๐๐ฬ , ๐๐๏ฟฝ)]โ1 = [๐๐๐น๐น ๐ผ๐ผ(๐๐ฬ , ๐๐๏ฟฝ)]โ1 = ๏ฟฝ 1.1835๐ธ๐ธ-4 โ6.0535๐ธ๐ธ-6๏ฟฝ โ6.0535๐ธ๐ธ-6 โ2.2154๐ธ๐ธ-7 90% confidence interval for ๐๐: ๏ฟฝ๐๐ฬ โ ฮฆโ1 (0.95)โ1.1835๐ธ๐ธ-4, ๐๐ฬ + ฮฆโ1 (0.95)โ1.1835๐ธ๐ธ-4๏ฟฝ [15.932, 15.968] ๐ท๐ท โ1 (0.95)โ2.2154๐ธ๐ธ-7 ๐๐๏ฟฝ. exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐๏ฟฝ 3.0769๐ธ๐ธ-2] An estimate can be made on how many washers the manufacturer discards: The distribution of washer sizes is a Normal Distribution with estimated parameters ๐๐ฬ = 15.95, ๐๐๏ฟฝ = 0.029984. The percentage of washers wish pass quality control is: ๐น๐น(16.05) โ ๐น๐น(15.95) = 49.96% It is likely that there is too much variance in the manufacturing process for this system to be efficient. Example 2 The following example adjusts the calculations used in the Normal Distribution to account for the fact that the limit on distance is [0, โ). The accuracy of a cutting machine used in manufacturing is desired to be measured. 5 cuts at the required length are made and measured as: 7.436, 10.270, 10.466, 11.039, 11.854 ๐๐๐๐ From data: ๐ฅ๐ฅฬ = 10.213, ๐ ๐ 2 = 2.789 Using numerical solver MLE Estimates for ๐ง๐ง0 is: Therefore ๐๐๏ฟฝ = ๐ง๐ง๏ฟฝ0 = โ4.5062 ๐ฅ๐ฅฬ = 2.26643 ๐๐0 โ ๐ง๐ง๏ฟฝ0 Trunc Normal 90% confidence interval for ๐๐: ๐ท๐ทโ1 (0.95)โ2.2154๐ธ๐ธ-7 ๏ฟฝ๐๐๏ฟฝ. exp ๏ฟฝ ๏ฟฝ, โ๐๐๏ฟฝ [2.922๐ธ๐ธ-2, 142 Univariate Continuous Distributions ๐๐ฬ = โ๐๐๏ฟฝ๐ง๐ง๏ฟฝ ๐๐ = 10.213 To calculate confidence intervals, first calculate: ๐๐0 = 1.5543๐ธ๐ธ-5, ๐๐๐๐ = โ0.16138 ๐๐0โฒ = 7.0042๐ธ๐ธ-5, 90% confidence intervals: 0.19466 โ2.9453๐ธ๐ธ-6 ๐ผ๐ผ(๐๐, ๐๐) = ๏ฟฝ ๏ฟฝ โ2.9453๐ธ๐ธ-6 0.12237 2.4728๐ธ๐ธ-5 [๐ฝ๐ฝ๐๐ (๐๐ฬ , ๐๐๏ฟฝ)]โ1 = [๐๐๐น๐น ๐ผ๐ผ(๐๐ฬ , ๐๐๏ฟฝ)]โ1 = ๏ฟฝ 1.0274 ๏ฟฝ 2.4728๐ธ๐ธ-5 1.6343 90% confidence interval for ๐๐: ๐๐ฬ + ฮฆโ1 (0.95)โ1.0274๏ฟฝ ๏ฟฝ๐๐ฬ โ ฮฆโ1 (0.95)โ1.0274, [8.546, 11.88] Trunc Normal 90% confidence interval for ๐๐: ๐ท๐ทโ1 (0.95)โ1.6343 ๏ฟฝ๐๐๏ฟฝ. exp ๏ฟฝ ๏ฟฝ, โ๐๐๏ฟฝ [0.8962, ๐ท๐ทโ1 (0.95)โ1.6343 ๐๐๏ฟฝ. exp ๏ฟฝ ๏ฟฝ๏ฟฝ ๐๐๏ฟฝ 5.732] To compare these results to a non-truncated normal distribution: 90% Lower CI Point Est 90% Upper CI Norm - ๐๐ 10.163 10.213 10.262 Classical 1.176 2.789 15.697 Norm - ๐๐ 2 Classical Norm - ๐๐ 8.761 10.213 11.665 Bayesian 0.886 2.789 6.822 Norm - ๐๐ 2 Bayesian TNorm - ๐๐ 8.546 10.213 11.88 0.80317 5.1367 32.856 TNorm - ๐๐ 2 *Note: The TNorm ๐๐ estimate and interval are squared. The truncated normal produced results which had a wider confidence in the parameter estimates, however the point estimates were within each others confidence intervals. In this case the truncation correction might be ignored for ease of calculation. Characteristics For large ๐๐/๐๐ truncation may have negligible affect. In this case the use the Normal Continuous Distribution as an approximation. Let: X~๐๐๐๐๐๐๐๐๐๐(µ, ฯ2 ) where ๐๐ โ [๐๐, ๐๐] Convolution Property. The sum of truncated normal distribution random variables is not a truncated normal distribution. When truncation is symmetrical about the mean the sum of truncated normal distribution random variables is well approximated using: Truncated Normal Continuous Distribution ๐๐ ๐๐ = ๏ฟฝ ๐๐๐๐ where ๐๐=1 ๐๐ โ ๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ ๐๐๐๐ , ๏ฟฝ ๐๐๐๐๐๐(๐๐๐๐ )๏ฟฝ 143 bi โ ai = µi 2 where ๐๐ โ [ โ๐๐๐๐ , โ๐๐๐๐ ] Linear Transformation Property (Cozman & Krotkov 1997) Applications ๐๐ = ๐๐๐๐ + ๐๐ ๐๐~๐๐๐๐๐๐๐๐๐๐(๐๐๐๐ + ๐๐, ๐๐2 ๐๐ 2 ) where ๐๐ โ [๐๐๐๐ + ๐๐, ๐๐๐๐ + ๐๐] Life Distribution. When used as a life distribution a truncated Normal Distribution may be used due to the constraint tโฅ0. However it is often found that the difference in results is negligible. (Rausand & Høyland 2004) Failures After Pre-test Screening. When a customer receives a product from a vendor, the product may have already been subject to burn-in testing. The customer will not know the number of failures which occurred during the burn-in, but may know the duration. As such the failure distribution is left truncated. (Meeker & Escobar 1998, p.269) Flaws under the inspection threshold. When a flaw is not detected due to the flawโs amplitude being less than the inspection threshold the distribution is left truncated. (Meeker & Escobar 1998, p.266) Worst Case Measurements. Sometimes only the worst performers from a population are monitored and have data collected. Therefore the threshold which determined that the item be monitored is the truncation limit. (Meeker & Escobar 1998, p.267) Screening Out Units With Large Defects. In quality control processes it may be common to remove defects which exceed a limit. The remaining population of defects delivered to the customer has a right truncated distribution. (Meeker & Escobar 1998, p.270) Resources Online: http://en.wikipedia.org/wiki/Truncated_normal_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) http://www.ntrand.com/truncated-normal-distribution/ Books: Cohen, 1991. Truncated and Censored Samples 1st ed., CRC Trunc Normal Repair Time Distributions. The truncated normal distribution may be used to model simple repair or inspection tasks that have a typical duration with little variation using the limits [0, โ) 144 Univariate Continuous Distributions Press. Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution 2nd ed., CRC. Schneider, H., 1986. Truncated and censored samples from normal populations, M. Dekker. Relationship to Other Distributions Normal Distribution Let: ๐๐๐๐๐๐๐๐(๐ฅ๐ฅ; ๐๐, ๐๐ 2 ) Then: ๐๐~๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 ) ๐๐ โ (โ, โ) ๐๐~T๐๐๐๐๐๐๐๐(๐๐, ๐๐ 2 , ๐๐๐ฟ๐ฟ , ๐๐๐๐ ) ๐๐ โ [๐๐๐ฟ๐ฟ , ๐๐๐๐ ] Trunc Normal For further relationships see Normal Continuous Distribution Uniform Continuous Distribution 145 4.9. Uniform Continuous Distribution Probability Density Function - f(t) a=0 b=1 a=0.75 b=1.25 a=0.25 b=1.5 2.00 1.50 1.00 0.50 0.00 0 0.5 1 1.5 1 1.5 1 1.5 Uniform Cont Cumulative Density Function - F(t) a=0 b=1 a=0.75 b=1.25 a=0.25 b=1.5 1.00 0.80 0.60 0.40 0.20 0.00 0 0.5 Hazard Rate - h(t) 10.00 9.00 8.00 7.00 6.00 5.00 4.00 3.00 2.00 1.00 0.00 a=0 b=1 a=0.75 b=1.25 a=0.25 b=1.5 0 0.5 146 Univariate Continuous Distributions Parameters & Description Parameters Random Variable ๐๐ ๐๐ Distribution Minimum Value. ๐๐ is the lower bound of the uniform distribution. 0 โค ๐๐ < ๐๐ Maximum Value. ๐๐ is the upper bound of the uniform distribution. ๐๐ < ๐๐ < โ Time Domain ๐๐ โค ๐ก๐ก โค ๐๐ 1 ๐๐(๐ก๐ก) = ๏ฟฝb โ a for a โค t โค b 0 otherwise 1 = {๐ข๐ข(๐ก๐ก โ ๐๐) โ ๐ข๐ข(๐ก๐ก โ ๐๐)} bโa PDF Uniform Cont Where ๐ข๐ข(๐ก๐ก โ ๐๐) is the Heaviside step function. 0 for t < ๐๐ tโa ๐น๐น(๐ก๐ก) = ๏ฟฝ for a โค t โค b bโa 1 for t > ๐๐ tโa {๐ข๐ข(๐ก๐ก โ ๐๐) โ ๐ข๐ข(๐ก๐ก โ ๐๐)} = bโa + ๐ข๐ข(๐ก๐ก โ ๐๐) CDF 1 for t < ๐๐ bโt ๐ ๐ (๐ก๐ก) = ๏ฟฝ for a โค t โค b bโa 0 for t > ๐๐ Reliability Laplace ๐๐(๐ ๐ ) = ๐๐ โ๐๐๐๐ โ ๐๐ โ๐๐๐๐ ๐ ๐ (๐๐ โ ๐๐) ๐น๐น(๐ ๐ ) = ๐๐ โ๐๐๐๐ โ ๐๐ โ๐๐๐๐ ๐ ๐ 2 (๐๐ โ ๐๐) ๐ ๐ (๐ ๐ ) = For ๐ก๐ก < ๐๐: Conditional Survivor Function ๐๐(๐๐ > ๐ฅ๐ฅ + ๐ก๐ก|๐๐ > ๐ก๐ก) Mean Life Residual ๐๐ โ๐๐๐๐ โ ๐๐ โ๐๐๐๐ 1 + ๐ ๐ 2 (๐๐ โ ๐๐) ๐ ๐ 1 for t + x < ๐๐ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) b โ (t + x) ๐๐(๐ฅ๐ฅ) = =๏ฟฝ for a โค t + x โค b ๐ ๐ (๐ก๐ก) bโa 0 for t > ๐๐ For ๐๐ โค ๐ก๐ก โค ๐๐: 1 for t + x < ๐๐ ๐ ๐ (๐ก๐ก + ๐ฅ๐ฅ) b โ (t + x) ๐๐(๐ฅ๐ฅ) = =๏ฟฝ for a โค t + x โค b ๐ ๐ (๐ก๐ก) bโt 0 for t + x > ๐๐ For ๐ก๐ก > ๐๐: ๐๐(๐ฅ๐ฅ) = 0 Where ๐ก๐ก is the given time we know the component has survived to. ๐ฅ๐ฅ is a random variable defined as the time after ๐ก๐ก. Note: ๐ฅ๐ฅ = 0 at ๐ก๐ก. For ๐ก๐ก < ๐๐: For ๐๐ โค ๐ก๐ก โค ๐๐: 1 ๐ข๐ข(๐ก๐ก) = 2(๐๐ + ๐๐) โ ๐ก๐ก ๐ข๐ข(๐ก๐ก) = ๐๐ โ ๐ก๐ก โ (๐๐ โ ๐๐)2 2(๐ก๐ก โ ๐๐) Uniform Continuous Distribution For ๐ก๐ก > ๐๐: 147 ๐ข๐ข(๐ก๐ก) = 0 1 โ(๐ก๐ก) = ๏ฟฝb โ t for a โค t โค b 0 otherwise Hazard Rate 0 for t < ๐๐ bโt ๐ป๐ป(๐ก๐ก) = ๏ฟฝ โ ln ๏ฟฝ ๏ฟฝ for a โค t โค b bโa โ for t > ๐๐ Cumulative Hazard Rate Properties and Moments Median Mean - 1st Raw Moment 1 (๐๐ 2 + ๐๐) 1 (๐๐ 12 Skewness - 3rd Central Moment โ ๐๐)2 0 6 โ5 Central Moment eitb โ eita it(b โ a) Characteristic Function 100ฮฑ% Percentile Function Parameter Estimation ๐ก๐ก๐ผ๐ผ = ฮฑ(b โ a) + a Maximum Likelihood Function Likelihood Functions LI nS nI 1 nF b โ t Si t RI i โ ti ๐ฟ๐ฟ(๐๐, ๐๐|๐ธ๐ธ) = ๏ฟฝ ๏ฟฝ .๏ฟฝ ๏ฟฝ ๏ฟฝ . ๏ฟฝ ๏ฟฝ1 + ๏ฟฝ ๏ฟฝ๏ฟฝ b๏ฟฝ๏ฟฝ๏ฟฝ โ a๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝi=1 b๏ฟฝโ๏ฟฝ๏ฟฝ๏ฟฝ a ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ bโa i=1 ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ failures survivors interval failures This assumes that all times are within the bound a, b. Point Estimates When there is only complete failure data: 1 nF ๐ฟ๐ฟ(๐๐, ๐๐|๐ธ๐ธ) = ๏ฟฝ ๏ฟฝ bโa where ๐๐ โค ๐ก๐ก๐๐ โค ๐๐ The likelihood function is maximized when a is large, b is small with the restriction that all times are between a and b. Thus: a๏ฟฝ = min(t1F , t F2 โฆ ) b๏ฟฝ = max(t1F , t F2 โฆ ) Uniform Cont Variance - 2nd Central Moment Excess kurtosis - + ๐๐) Any value between ๐๐ and ๐๐ Mode 4th 1 (๐๐ 2 148 Univariate Continuous Distributions When ๐๐ = 0 and ๐๐ is estimated with complete data the following estimates may be used where ๐ก๐ก๐๐๐๐๐๐ = max(t1F , t F2 โฆ t Fn ). (Johnson et al. 1995, p.286) 1. MLE. b๏ฟฝ = ๐ก๐ก๐๐๐๐๐ฅ๐ฅ n+2 2. Min Mean Square Error. b๏ฟฝ = ๐ก๐ก n+1 ๐๐๐๐๐๐ n+1 3. Unbiased Estimator. b๏ฟฝ = ๐ก๐ก๐๐๐๐๐๐ 4. n 1 b๏ฟฝ = 2 ๏ฟฝn ๐ก๐ก๐๐๐๐๐๐ Closest Estimator. Procedures for parameter estimating when there is censored data is detailed in (Johnson et al. 1995, p.286) โ1 โก (๐๐ โ ๐๐)2 ๐ผ๐ผ(๐๐, ๐๐) = โข 1 โข (๐๐ โฃ โ ๐๐)2 Fisher Information Uniform Cont Bayesian 1 โค (๐๐ โ ๐๐)2 โฅ โ1 โฅ (๐๐ โ ๐๐)2 โฆ The Uniform distribution is widely used in Bayesian methods as a non-informative prior or to model evidence which only suggests bounds on the parameter. Non-informative Prior. The Uniform distribution can be used as a non-informative prior. As can be seen below, the only affect the uniform prior has on Bayes equation is to limit the range of the parameter for which the denominator integrates over. ๐๐(๐๐|๐ธ๐ธ) = 1 ๏ฟฝ ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐ โ ๐๐ = ๐๐ ๐๐ 1 โซ๐๐ ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๏ฟฝ๐๐ โ ๐๐ ๏ฟฝ ๐๐๐๐ โซ๐๐ ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๐๐๐๐ ๐ฟ๐ฟ(๐ธ๐ธ|๐๐) ๏ฟฝ Parameter Bounds. This type of distribution allows an easy method to mathematically model soft data where only the parameter bounds can be estimated. An example is where uniform distribution can model a personโs opinion on the value ๐๐ where they know that it could not be lower than ๐๐ or greater than ๐๐, but is unsure of any particular value ๐๐ could take. Non-informative Priors Jeffreyโs Prior 1 ๐๐ โ ๐๐ Description , Limitations and Uses Example For an example of the uniform distribution being used in Bayesian updating as a prior, ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1,1) see the binomial distribution. Given the following data calculate the MLE parameter estimates: 240, 585, 223, 751, 255 a๏ฟฝ = 223 Uniform Continuous Distribution Characteristics 149 b๏ฟฝ = 751 The Uniform distribution is a special case of the Beta distribution when ๐ผ๐ผ = ๐ฝ๐ฝ = 1. The uniform distribution has an increasing failure rate with lim โ(๐ก๐ก) = ๐ก๐กโ๐๐ โ. The Standard Uniform Distribution has parameters ๐๐ = 0 and ๐๐ = 1. This results in ๐๐(๐ก๐ก) = 1 for ๐๐ โค ๐ก๐ก โค ๐๐ and 0 otherwise. Uniformity Property If ๐ก๐ก > ๐๐ and ๐ก๐ก + ๐ฅ๐ฅ < ๐๐ then: ๐๐~๐๐๐๐๐๐๐๐(๐๐, ๐๐) ๐ก๐ก+โ Variate Generation Property ๐น๐น โ1 (๐ข๐ข) = ๐ข๐ข(๐๐ โ ๐๐) + ๐๐ Applications Residual Property If k is a real constant where ๐๐ < ๐๐ < ๐๐ then: Pr(๐๐|๐๐ > ๐๐)~๐๐๐๐๐๐๐๐(๐๐ = ๐๐, ๐๐) Random Number Generator. The uniform distribution is widely used as the basis for the generation of random numbers for other statistical distributions. The random uniform values are mapped to the desired distribution by solving the inverse cdf. Bayesian Inference. The uniform distribution can be used ss a noninformative prior and to model soft evidence. Resources Special Case of Beta Distribution. In applications like Bayesian statistics the uniform distribution is used as an uninformative prior by using a beta distribution of ๐ผ๐ผ = ๐ฝ๐ฝ = 1. Online: http://mathworld.wolfram.com/UniformDistribution.html http://en.wikipedia.org/wiki/Uniform_distribution_(continuous) http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience. Relationship to Other Distributions Beta Distribution Let Uniform Cont โ 1 ๐๐๐๐ = ๐๐ โ ๐๐ ๐๐ โ ๐๐ ๐ก๐ก The probability that a random variable falls within any interval of fixed length is independent of the location, ๐ก๐ก, and is only dependent on the interval size, ๐ฅ๐ฅ. ๐๐(๐ก๐ก โ ๐ก๐ก + โ) = ๏ฟฝ 150 Univariate Continuous Distributions ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; ฮฑ, ฮฒ, a, b) Exponential Distribution Uniform Cont ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) Then Xi ~๐๐๐๐๐๐๐๐(0,1) ๐๐๐๐๐๐ X1 โค X2 โค โฏ โค Xn Xr ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐, ๐๐ โ ๐๐ + 1) Where ๐๐ and ๐๐ are integers. Special Case: ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ก๐ก; ๐๐, ๐๐|๐ผ๐ผ = 1, ฮฒ = 1 ) = ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐) Let Then ๐๐~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐๐๐ Y = exp(โ๐๐๐๐) ๐๐~๐๐๐๐๐๐๐๐(0,1) Discrete Distributions 151 5. Univariate Discrete Distributions Univar Discrete 152 Discrete Distributions 5.1. Bernoulli Discrete Distribution Probability Density Function - f(k) 0.70 0.70 0.60 0.60 p=0.3 0.50 0.50 0.40 0.40 0.30 0.30 0.20 0.20 0.10 0.10 0.00 0.00 0 1 2 p=0.55 0 1 2 Cumulative Density Function - F(k) 1.00 1.00 0.80 0.80 Bernoulli 0.60 p=0.3 0.60 0.40 0.40 0.20 0.20 0.00 0.00 0 1 2 p=0.55 0 1 2 Hazard Rate - h(k) 1.00 1.00 0.80 0.80 0.60 p=0.3 0.60 0.40 0.40 0.20 0.20 0.00 p=0.55 0.00 0 1 2 0 1 2 Bernoulli Discrete Distribution 153 Parameters & Description ๐๐ Parameters Random Variable Question Distribution 0 โค ๐๐ โค 1 Bernoulli probability Probability of success. ๐๐ โ {0, 1} The probability of getting exactly ๐๐ (0 or 1) successes in 1 trial with probability p. Formulas ๐๐(๐๐) = pk (1 โ p)1โk 1 โ p for k = 0 =๏ฟฝ p for k = 1 PDF ๐น๐น(๐๐) = (1 โ p)1โk 1 โ p for k = 0 =๏ฟฝ 1 for k = 1 CDF ๐ ๐ (๐๐) = 1 โ (1 โ p)1โk p for k = 0 =๏ฟฝ 0 for k = 1 Reliability โ(๐๐) = ๏ฟฝ Hazard Rate 1โp 1 Properties and Moments Mean - 1st Raw Moment Central Moment qโp Skewness - 3rd Central Moment ๏ฟฝpq Excess kurtosis - 4th Central Moment Characteristic Function Parameter Estimation ๐๐ ๐๐(1 โ ๐๐) ๐ค๐คโ๐๐๐๐๐๐ ๐๐ = (1 โ ๐๐) 6p2 โ 6p + 1 p(1 โ p) (1 โ p) + peit Maximum Likelihood Function Likelihood Function ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = ๐๐โ๐๐๐๐ (1 โ ๐๐)๐๐โโ๐๐๐๐ where ๐๐ is the number of Bernoulli trials ๐๐๐๐ โ {0,1}, and โ๐๐๐๐ = โ๐๐๐๐=1 ๐๐๐๐ Bernoulli 2nd for k = 0 for k = 1 ๐๐0.5 = โ๐๐โ ๐ค๐คโ๐๐๐๐ ๐๐ โ 0.5 ๐๐0.5 = {0,1} ๐ค๐คโ๐๐๐๐ ๐๐ = 0.5 Mode Variance - parameter. 154 Discrete Distributions ๐๐L =0 ๐๐p solve for ๐๐ ๐๐L = โk. p โ(ki)โ1 (1 โ p)nโโki โ (n โ โk)pโki (1 โ p)nโ1โโki = 0 ๐๐p โk. pโ(ki)โ1 (1 โ p)nโโki = (n โ โk i )pโki (1 โ p)nโ1โโki โk i . pโ1 = (n โ โk i )(1 โ p)โ1 (1 โ p) n โ โk i = โk i p p= Fisher Information MLE Point Estimates โk i n ๐ผ๐ผ(๐๐) = The MLE point estimate for p: p๏ฟฝ = Fisher Information Confidence Intervals 1 ๐๐(1 โ ๐๐) ๐ผ๐ผ(๐๐) = โk n 1 ๐๐(1 โ ๐๐) See discussion in binomial distribution. Bernoulli Bayesian Type Non-informative Priors for p, ๐ ๐ (๐๐) (Yang and Berger 1998, p.6) Prior Posterior Uniform Proper Prior with limits ๐๐ โ [๐๐, ๐๐] 1 ๐๐ โ ๐๐ Uniform Improper Proir with limits ๐๐ โ [0,1] 1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 1,1) Jeffreyโs Prior Reference Prior MDIP Novick and Hall 1 1 1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐๐; , ๏ฟฝ 2 2 ๏ฟฝ๐๐(1 โ ๐๐) 1.6186๐๐๐๐ (1 โ ๐๐)1โ๐๐ ๐๐โ1 (1 โ ๐๐)โ1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(0,0) Truncated Beta Distribution For a โค ๐๐ โค b ๐๐. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 1 + ๐๐, 2 โ ๐๐) Otherwise ๐๐(๐๐) = 0 ๐ต๐ต๐๐๐ก๐ก๐ก๐ก(๐๐; 1 + ๐๐, 2 โ ๐๐) 1 ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐๐; + ๐๐, 1.5 โ ๐๐๏ฟฝ 2 when ๐๐ โ [0,1] Proper - No Closed Form ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, 1 โ ๐๐) when ๐๐ โ [0,1] Bernoulli Discrete Distribution 155 Conjugate Priors UOI ๐๐ from ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐) Example Likelihood Model Bernoulli Evidence ๐๐ failures in 1 trail Dist of UOI Prior Para Posterior Parameters Beta ๐ผ๐ผ0 , ๐ฝ๐ฝ0 ๐ผ๐ผ = ๐ผ๐ผ๐๐ + ๐๐ ๐ฝ๐ฝ = ๐ฝ๐ฝ๐๐ + 1 โ ๐๐ Description , Limitations and Uses When a demand is placed on a machine it undergoes a Bernoulli trial with success defined as a successful start. It is known the probability of a successful start, ๐๐, equals 0.8. Therefore the probability the machine does not start. ๐๐(0) = 0.2. For an example with multiple Bernoulli trials see the binomial distribution. Characteristics A Bernoulli process is a probabilistic experiment that can have one of two outcomes, success (๐๐ = 1) with the probability of success is ๐๐, and failure (๐๐ = 0) with the probability of failure is ๐๐ โก 1 โ ๐๐. Single Trial. Itโs important to emphasis that the Bernoulli distribution is for a single trial or event. The case of multiple Bernoulli trials with replacement is the binomial distribution. The case of multiple Bernoulli trials without replacement is the hypergeometric distribution. n ๏ฟฝ K i ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(ฮ ๐๐; ๐๐ = ฮ ๐๐๐๐ ) i=1 Applications Used to model a single event which have only two outcomes. In reliability engineering it is most often used to model demands or shocks to a component where the component will fail with probability p. In practice it is rare for only a single event to be considered and so a binomial distribution is most often used (with the assumption of replacement). The conditions and assumptions of a Bernoulli trial however are used as the basis for each trial in a binomial distribution. See โRelated Distributionsโ and binomial distribution for more details. Resources Online: Bernoulli ๐พ๐พ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐) Maximum Property max{๐พ๐พ1 , ๐พ๐พ2 , โฆ , ๐พ๐พ๐๐ }~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐ = 1 โ ฮ {1 โ ๐๐๐๐ }) Minimum property min{๐พ๐พ1 , ๐พ๐พ2 , โฆ , ๐พ๐พ๐๐ }~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐ = ฮ ๐๐๐๐ ) Product Property 156 Discrete Distributions http://mathworld.wolfram.com/BernoulliDistribution.html http://en.wikipedia.org/wiki/Bernoulli_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Books: Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for scientists and engineers, Birkhäuser. Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete Distributions 3rd ed., Wiley-Interscience. Relationship to Other Distributions The Binomial distribution counts the number of successes in n independent observations of a Bernoulli process. Let Binomial Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐โฒ|n, p) Then ๐พ๐พ๐๐ ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k i ; p) ๐๐ = ๏ฟฝ ๐พ๐พ๐๐ ๐๐=1 Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k โฒ = โk i |n, p) where ๐๐โฒ โ {1, 2, โฆ , ๐๐} Special Case: Bernoulli ๐๐๐๐๐๐ ๐๐ ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k; p) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k; p|n = 1) Binomial Discrete Distribution 157 5.2. Binomial Discrete Distribution Probability Density Function - f(k) 0.60 0.50 n=1 p=0.4 0.30 n=5 p=0.4 0.25 n=15 p=0.4 0.40 0.20 0.30 0.15 0.20 0.10 0.10 0.05 0.00 0.00 0 5 n=20 p=0.2 n=20 p=0.5 n=20 p=0.8 10 0 5 10 15 20 Cumulative Density Function - F(k) 1.00 1.00 0.80 0.80 0.60 0.60 0.40 0.40 n=1 p=0.4 n=5 p=0.4 n=15 p=0.4 0.20 0.00 0 5 10 0 5 10 15 20 Hazard Rate - h(k) 1.0 1.0 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2 0.0 0.0 0 5 10 0 10 20 Binomial 0.00 0.20 158 Discrete Distributions Parameters & Description Parameters Random Variable Question Distribution PDF ๐๐ ๐๐ ๐๐ โ {1, 2 โฆ , โ} 0 โค ๐๐ โค 1 Number of Trials. Bernoulli probability parameter. Probability of success in a single trial. ๐๐ โ {0, 1, 2 โฆ , ๐๐} The probability of getting exactly ๐๐ successes in ๐๐ trials. Formulas n ๐๐(๐๐) = ๏ฟฝ ๏ฟฝ pk (1 โ p)nโk k where k combinations from n: ๐๐! ๐๐ ๐๐ ๏ฟฝ ๏ฟฝ = ๐๐๐ถ๐ถ๐๐ = ๐ถ๐ถ๐๐๐๐ = = ๐ถ๐ถ ๐๐โ1 ๐๐ ๐๐! (๐๐ โ ๐๐)! ๐๐ ๐๐โ1 k ๐น๐น(๐๐) = ๏ฟฝ j=0 ๐๐! pj (1 โ p)nโj ๐๐! (๐๐ โ ๐๐)! = ๐ผ๐ผ1โ๐๐ (๐๐ โ ๐๐, ๐๐ + 1) where ๐ผ๐ผ๐๐ (๐๐, ๐๐) is the Regularized Incomplete Beta function. See section 1.6.3. CDF When ๐๐ โฅ 20 and ๐๐ โค 0.05, or if ๐๐ โฅ 100 and ๐๐๐๐ โค 10, this can be approximated by a Poisson distribution with ๐๐ = ๐๐๐๐: k Binomial ๐น๐น(๐๐) โ eโµ ๏ฟฝ j=0 µj ฮ(๐๐ + 1, ๐๐) = j! ๐๐! โ ๐น๐น๐๐2 (2๐๐, 2๐๐ + 2) When ๐๐๐๐ โฅ 10 and ๐๐๐๐(1 โ ๐๐) โฅ 10 then the cdf can be approximated using a normal distribution: ๐๐ + 0.5 โ ๐๐๐๐ ๐น๐น(๐๐) โ ฮฆ ๏ฟฝ ๏ฟฝ ๏ฟฝ๐๐๐๐(1 โ ๐๐) k ๐ ๐ (๐๐) = 1 โ ๏ฟฝ Reliability n = ๏ฟฝ j=0 j=k+1 ๐๐! pj (1 โ p)nโj ๐๐! (๐๐ โ ๐๐)! ๐๐! pj (1 โ p)nโj ๐๐! (๐๐ โ ๐๐)! = ๐ผ๐ผ๐๐ (๐๐ + 1, ๐๐ โ ๐๐) where ๐ผ๐ผ๐๐ (๐๐, ๐๐) is the Regularized Incomplete Beta function. See section 1.6.3. Binomial Discrete Distribution Hazard Rate 159 โ1 n (1 + ฮธ)n โ โkj=0 ๏ฟฝ ๏ฟฝ ฮธj k ๏ฟฝ โ(๐๐) = ๏ฟฝ1 + n ๏ฟฝ ๏ฟฝ ฮธk k where ๐๐ = (Gupta et al. 1997) ๐๐ 1 โ ๐๐ Properties and Moments ๐๐0.5 is either {โ๐๐๐๐โ, โ๐๐๐๐โ} Median โ(๐๐ + 1)๐๐โ Mode ๐๐๐๐ Mean - 1st Raw Moment Variance - 2nd Skewness - ๐๐๐๐(1 โ ๐๐) Central Moment 3rd 1 โ 2p Central Moment ๏ฟฝnp(1 โ p) 6p2 โ 6p + 1 np(1 โ p) Excess kurtosis - 4th Central Moment ๏ฟฝ1 โ p + peit ๏ฟฝ Characteristic Function 100ฮฑ% Percentile Function n Numerically solve for ๐๐ (which is not arduous for ๐๐ โค 10): ๐๐๐ผ๐ผ = ๐น๐น โ1 (๐๐, ๐๐) k ฮฑ โ ๏ฟฝฮฆโ1 (๐ผ๐ผ)๏ฟฝ๐๐๐๐(1 โ ๐๐) + ๐๐๐๐ โ 0.5๏ฟฝ Parameter Estimation Maximum Likelihood Function Likelihood Function For complete data only: ๐๐๐ต๐ต ni ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = ๏ฟฝ ๏ฟฝk ๏ฟฝ pki (1 โ p)niโki i=1 i = pโki (1 โ p)โniโโki ๐๐ ๐ต๐ต ๐๐๐๐ , โ๐๐๐๐ = Where ๐๐๐ต๐ต is the number of Binomial processes, โ๐๐๐๐ = โ๐๐=1 ๐๐๐ต๐ต โ๐๐=1 ๐๐๐๐ and the combinatory term is ignored (see section 1.1.6 for discussion). Binomial For ๐๐๐๐ โฅ 10 and ๐๐๐๐(1 โ ๐๐) โฅ 10 the normal approximation may be used: 160 Discrete Distributions ๐๐L =0 ๐๐p solve for ๐๐ ๐๐L = โk i . pโ(ki)โ1 (1 โ p)โniโโki โ (โni โ โk i )pโki (1 โ p)โniโ1โโki ๐๐p โk i . pโ(ki)โ1 (1 โ p)โniโโki = (โni โ โk i )pโki (1 โ p)โ1+โniโโki โk i . pโ1 = (โni โ โk i )(1 โ p)โ1 (1 โ p) โni โ โk i = โk i p p= MLE Point Estimates โk i โni The MLE point estimate for p: p๏ฟฝ = Fisher Information Binomial Confidence Intervals ๐ผ๐ผ(๐๐) = โk i โni 1 ๐๐(1 โ ๐๐) The confidence intervals for the binomial distribution parameter p is a controversial subject which is still debated. The Wilson interval is recommended for small and large ๐๐. (Brown et al. 2001) ๐๐ = where ๐๐ = ๐๐๐๐ฬ + ๐ ๐ 2 โ2 ๐ ๐ ๏ฟฝ๐ ๐ 2 + 4๐๐๐๐ฬ (1 โ ๐๐ฬ ) + 2(๐๐ + ๐ ๐ 2 ) ๐๐ + ๐ ๐ 2 ๐๐๐๐ฬ + ๐ ๐ 2 โ2 ๐ ๐ ๏ฟฝ๐ ๐ 2 + 4๐๐๐๐ฬ (1 โ ๐๐ฬ ) โ 2(๐๐ + ๐ ๐ 2 ) ๐๐ + ๐ ๐ 2 ๐พ๐พ + 1 ๐ ๐ = ฮฆโ1 ๏ฟฝ ๏ฟฝ 2 It should be noted that most textbooks use the Wald interval (normal approximation) given below, however many articles have shown these estimates to be erratic and cannot be trusted. (Brown et al. 2001) ๐๐ = ๐๐ฬ + ๐ ๐ ๏ฟฝ ๐๐ = ๐๐ฬ โ ๐ ๐ ๏ฟฝ ๐๐ฬ (1 โ ๐๐ฬ ) ๐๐ ๐๐ฬ (1 โ ๐๐ฬ ) ๐๐ For a comparison of binomial confidence interval estimates the reader is referred to (Brown et al. 2001). The following webpage has links to online calculators which use many different methods. http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval Binomial Discrete Distribution 161 Bayesian Non-informative Priors for p given n, ๐ ๐ (๐๐|๐๐) (Yang and Berger 1998, p.6) Type Prior Posterior Uniform Proper Prior with limits ๐๐ โ [๐๐, ๐๐] 1 ๐๐ โ ๐๐ Uniform Improper Proir with limits ๐๐ โ [0,1] 1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 1,1) Jeffreyโs Prior Reference Prior MDIP Novick and Hall UOI Example Otherwise ๐๐(๐๐) = 0 ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 1 + ๐๐, 1 + ๐๐ โ ๐๐) 1 1 1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐๐; , ๏ฟฝ 2 2 ๏ฟฝ๐๐(1 โ ๐๐) 1 1 ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐๐; + ๐๐, + ๐๐ โ ๐๐๏ฟฝ 2 2 when ๐๐ โ [0,1] 1.6186๐๐๐๐ (1 โ ๐๐)1โ๐๐ ๐๐โ1 (1 Likelihood Model Binomial โ ๐๐)โ1 Proper - No Closed Form ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐ โ ๐๐) when ๐๐ โ [0,1] = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(0,0) Conjugate Priors Evidence ๐๐ failures in ๐๐ trial Dist of UOI Prior Para Beta ๐ผ๐ผ๐๐ , ๐ฝ๐ฝ๐๐ Posterior Parameters ๐ผ๐ผ = ๐ผ๐ผ๐๐ + ๐๐ ฮฒ = ฮฒo + n โ k Description , Limitations and Uses Five machines are measured for performance on demand. The machines can either fail or succeed in their application. The machines are tested for 10 demands with the following data for each machine: Machine/Trail 1 2 3 4 5 ๐๐๐๐ 1 2 3 F=3 F=2 F=2 F=3 F=2 ๐๐๐๐ฬ 4 5 6 7 8 S=7 S=8 S=8 S=7 S=8 ๐๐(1 โ ๐๐ฬ ) 9 10 Assuming machines are homogeneous estimate the parameter ๐๐: Using MLE: p๏ฟฝ = โk i 12 = = 0.24 โni 50 Binomial ๐๐ from ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) Truncated Beta Distribution For a โค ๐๐ โค b ๐๐. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 1 + ๐๐, 1 + ๐๐ โ ๐๐) 162 Discrete Distributions 90% confidence intervals for ๐๐: ๐ ๐ = ฮฆโ1 (0.95) = 1.64485 ๐๐๐๐๐๐๐๐๐๐๐๐ = ๐๐๐๐ฬ + ๐ ๐ 2 โ2 ๐ ๐ ๏ฟฝ๐ ๐ 2 + 4๐๐๐๐ฬ (1 โ ๐๐ฬ ) โ = 0.1557 2(๐๐ + ๐ ๐ 2 ) ๐๐ + ๐ ๐ 2 ๐๐๐ข๐ข๐ข๐ข๐ข๐ข๐ข๐ข๐ข๐ข = ๐๐๐๐ฬ + ๐ ๐ 2 โ2 ๐ ๐ ๏ฟฝ๐ ๐ 2 + 4๐๐๐๐ฬ (1 โ ๐๐ฬ ) + = 0.351 2(๐๐ + ๐ ๐ 2 ) ๐๐ + ๐ ๐ 2 A Bayesian point estimate using a uniform prior distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1, 1), with posterior ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 13, 39) has a point estimate: ๐๐ฬ = E[๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; 13, 39)] = 13 = 0.25 52 With 90% confidence interval using inverse Beta cdf: โ1 [๐น๐น๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต (0.05) = 0.1579, โ1 ๐น๐น๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต (0.95) = 0.3532] Binomial The probability of observing no failures in the next 10 trials with replacement is: ๐๐(0; 10,0.25) = 0.0563 Characteristics The probability of observing less than 5 failures in the next 10 trials with replacement is: ๐๐(0; 10,0.25) = 0.9803 CDF Approximations. The Binomial distribution is one of the most widely used distributions throughout history. Although simple, the CDF function was tedious to calculate prior to the use of computers. As a result approximations using the Poisson and Normal distribution have been used. For details see โRelated Distributionsโ. With Replacement. The Binomial distribution models probability of ๐๐ successes in ๐๐ Bernoulli trials. However, the ๐๐ successes can occur anywhere among the ๐๐ trials with ๐๐๐ถ๐ถ๐๐ different combinations. Therefore the Binomial distribution assumes replacement. The equivalent distribution which assumes without replacement is the hypergeometric distribution. Symmetrical. The distribution is symmetrical when ๐๐ = 0.5. Compliment. ๐๐(๐๐; ๐๐, ๐๐) = ๐๐(๐๐ โ ๐๐; ๐๐, 1 โ ๐๐). Tables usually only provide values up to ๐๐/2 allowing the reader to calculate to ๐๐ using the compliment formula. Assumptions. The binomial distribution describes the behavior of a count variable K if the following conditions apply: Binomial Discrete Distribution 1. 2. 3. 4. The number of observations n is fixed. Each observation is independent. Each observation represents one of two outcomes ("success" or "failure"). The probability of "success" is the same for each outcome. Convolution Property Applications Resources 163 When ๐๐ is fixed. ๐พ๐พ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐, ๐๐) ๏ฟฝ K i ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(โ๐๐๐๐ , ๐๐) Used to model independent repeated trials which have two outcomes. Examples used in Reliability Engineering are: โข Number of independent components which fail, ๐๐, from a population, ๐๐ after receiving a shock. โข Number of failures to start, ๐๐, from ๐๐ demands on a component. โข Number of independent items defective, ๐๐, from a population of ๐๐ items. Online: http://mathworld.wolfram.com/BinomialDistribution.html http://en.wikipedia.org/wiki/Binomial_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc) Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete Distributions 3rd ed., Wiley-Interscience. Relationship to Other Distributions The Binomial distribution counts the number of successes ๐๐ in ๐๐ independent observations of a Bernoulli process. Let Bernoulli Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k โฒ ; p) Then ๐พ๐พ๐๐ ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(kโฒi ; ๐๐) ๐๐๐๐๐๐ ๐๐ ๐๐ = ๏ฟฝ ๐พ๐พ๐๐ ๐๐=1 Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(โk โฒ i ; ๐๐, ๐๐) where ๐๐ โ {1, 2, โฆ , ๐๐} Special Case: ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐|๐๐ = 1) Binomial Books: Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for scientists and engineers, Birkhäuser. 164 Discrete Distributions Hypergeometric Distribution ๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป (๐๐; ๐๐, ๐๐, ๐๐) Normal Distribution ๐๐๐๐๐๐๐๐(๐ก๐ก; ๐๐, ๐๐ 2 ) The hypergeometric distribution models probability of ๐๐ successes in ๐๐ Bernoulli trials from a population ๐๐, with ๐๐ successors without replacement. ๐๐(๐๐; ๐๐, ๐๐, ๐๐) Limiting Case for ๐๐ โซ ๐๐ and ๐๐ not near 0 or 1: ๐๐ lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐ = ) = ๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป(๐๐; ๐๐, ๐๐, ๐๐) ๐๐โโ ๐๐ Limiting Case for constant ๐๐: lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) = ๐๐๐๐๐๐๐๐๏ฟฝk๏ฟฝµ = n๐๐, ๐๐ 2 = ๐๐๐๐(1 โ ๐๐)๏ฟฝ ๐๐โโ ๐๐=๐๐ The Normal distribution can be used as an approximation of the Binomial distribution when ๐๐๐๐ โฅ 10 and ๐๐๐๐(1 โ ๐๐) โฅ 10. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) โ ๐๐๐๐๐๐๐๐๏ฟฝ๐๐ + 0.5๏ฟฝ๐๐ = ๐๐๐๐, ๐๐ 2 = ๐๐๐๐(1 โ ๐๐)๏ฟฝ Limiting Case for constant ๐๐๐๐: lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) = ๐๐๐๐๐๐๐๐(k; µ = n๐๐) ๐๐โโ n๐๐=๐๐ Binomial The Poisson distribution is the limiting case of the Binomial distribution when ๐๐ is large but the ratio of ๐๐๐๐ remains constant. Hence the Poisson distribution models rare events. Poisson Distribution ๐๐๐๐๐๐๐๐(๐๐; ๐๐) Multinomial Distribution ๐๐๐๐๐๐๐๐๐๐ (๐ค๐ค|n, ๐ฉ๐ฉ) The Poisson distribution can be used as an approximation to the Binomial distribution when ๐๐ โฅ 20 and ๐๐ โค 0.05, or if ๐๐ โฅ 100 and ๐๐๐๐ โค 10. The Binomial is expressed in terms of the total number of a probability of success, ๐๐, and trials, ๐๐. Where a Poisson distribution is expressed in terms of a success rate and does not need to know the total number of trials. The derivation of the Poisson distribution from the binomial can be found at http://mathworld.wolfram.com/PoissonDistribution.html. This interpretation can also be used to understand the conditional distribution of a Poisson random variable: Let ๐พ๐พ1 , ๐พ๐พ2 ~๐๐๐๐๐๐๐ ๐ (µ) Given ๐๐ = ๐พ๐พ1 + ๐พ๐พ2 = ๐๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐๐๐๐๐๐๐๐๐๐๐ Then µ1 ๐พ๐พ1 |n~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝk; n, p = ๏ฟฝ µ1 + µ2 Special Case: ๐๐๐๐๐๐๐๐๐๐=2 (๐ค๐ค|n, ๐ฉ๐ฉ) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) Poisson Discrete Distribution 165 5.3. Poisson Discrete Distribution Probability Density Function - f(k) 0.20 0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00 ฮผ=4 ฮผ=10 ฮผ=20 ฮผ=30 0 5 10 15 20 25 30 35 40 45 Cumulative Density Function - F(k) 1.00 0.80 0.60 ฮผ=4 ฮผ=10 ฮผ=20 ฮผ=30 0.40 0.20 0 5 10 15 20 25 30 35 40 45 10 15 20 25 30 35 40 45 Hazard Rate - h(k) 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0 5 Poisson 0.00 166 Discrete Distributions Parameters & Description ๐๐ Parameters Shape Parameter: The value of ๐๐ is the expected number of events per time period or other physical dimensions. If the Poisson distribution is modeling failure events, then ๐๐ = ๐๐๐๐ is the average number of failures that would occur in the space ๐ก๐ก. In this case ๐ก๐ก is fixed and ๐๐ becomes the distribution parameter. Some texts use the symbol ๐๐. ๐๐ > 0 ๐๐ is an integer, Random Variable Distribution Formulas ๐๐(๐๐) = PDF ๐๐๐๐ โµ (๐๐๐๐)๐๐ โฮปt ๐๐ = ๐๐ k! k! k ๐น๐น(๐๐) = eโµ ๏ฟฝ j=0 Poisson CDF Reliability ๐๐ โฅ 0 µj ฮ(๐๐ + 1, ๐๐) = j! ๐๐! = ๐น๐น๐๐2 (2๐๐, 2๐๐ + 2) Where ๐น๐น๐๐2 (๐ฅ๐ฅ|๐ฃ๐ฃ) is the Chi-square CDF. When ๐๐ > 10 the ๐น๐น(๐๐) can be approximated by a normal distribution: ๐๐ + 0.5 โ ๐๐ ๐น๐น(๐๐) โ ฮฆ ๏ฟฝ ๏ฟฝ โ๐๐ R(k) = 1 โ F(k) k Hazard Rate k! µj โ(๐๐) = ๏ฟฝ1 + ๏ฟฝeµ โ 1 โ ๏ฟฝ ๏ฟฝ๏ฟฝ µ j! โ1 j=1 (Gupta et al. 1997) Properties and Moments Median Mode Mean - 1st Raw Moment See 100ฮฑ% Percentile Function when ๐ผ๐ผ = 0.5. โ๐๐โ where โ๐๐โ is the floor function 2 Variance - 2nd Central Moment 2 โ๐๐โ = is the floor function (largest integer not greater than ๐๐) ๐๐ ๐๐ Poisson Discrete Distribution Skewness - 3rd Central Moment Excess kurtosis - 4th 167 1/๏ฟฝµ 1/µ Central Moment exp{µ๏ฟฝeik โ 1๏ฟฝ} Characteristic Function 100ฮฑ% Percentile Function Numerically solve for ๐๐ (which is not arduous for ๐๐ โค 10): ๐๐๐ผ๐ผ = ๐น๐น โ1 (๐ผ๐ผ) For ๐๐ > 10 the normal approximation may be used: k ฮฑ โ ๏ฟฝ๏ฟฝµฮฆโ1 (๐ผ๐ผ) + ๐๐ โ 0.5๏ฟฝ Parameter Estimation Maximum Likelihood Estimates Likelihood Functions For complete data: ๐น๐น n ๐๐ ๐๐๐๐ โµ ๐ฟ๐ฟ(๐๐|๐ธ๐ธ) = ๏ฟฝ F ๐๐ i=1 k i ! ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ known k Log-Likelihood Function MLE Point Estimates n ฮ = โnµ + ๏ฟฝ{๐๐๐๐ ln(๐๐) โ ln(๐๐๐๐ !)} ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ i=1 known k n โฮ 1 = โn + ๏ฟฝ ๐๐๐๐ = 0 โµ µ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ i=1 For complete data solving µ๏ฟฝ = โฮ โµ n known k = 0 gives: n 1 1 . ๏ฟฝ ๐๐๐๐ ๐๐๐๐ ฮป๏ฟฝ = . ๏ฟฝ ๐๐๐๐ ๐๐ ๐ก๐ก๐ก๐ก i=1 i=1 Note that in this context: t = the unit of time for which the rate, ๐๐ is being measured. ๐๐ = the number of Poisson processes for which the exact number of failures, k, was known. ๐๐๐๐ = the number of failures that occurred within the ith Poisson process. When there is only one Poisson process this reduces to: ๐๐ µ๏ฟฝ = ๐๐ ๐๐๐๐ ฮป๏ฟฝ = ๐ก๐ก For censored data numerical methods are needed to maximize the loglikelihood function. Poisson โฮ =0 โµ where ๐๐ is the number of poisson processes. 168 Discrete Distributions Fisher Information ๐ผ๐ผ(๐๐) = 100๐พ๐พ% Confidence Interval (complete data only) ๐๐lower 2 Sided 1 ๐๐ ๐๐upper 2 Sided Conservative two sided confidence intervals. ๐๐ 21โฮณ (2โ๐๐๐๐ ) ๐๐ 21+ฮณ (2โ๐๐๐๐ + 2) When ๐๐ is large (๐๐ > 10) two sided intervals 1 + ๐พ๐พ ๐๐ฬ ฮป๏ฟฝ โ ฮฆโ1 ๏ฟฝ ๏ฟฝ๏ฟฝ 2 ๐ก๐ก๐๐ 1 + ๐พ๐พ ๐๐ฬ ฮป๏ฟฝ + ฮฆโ1 ๏ฟฝ ๏ฟฝ๏ฟฝ 2 ๐ก๐ก๐ก๐ก ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ 2๐ก๐ก๐ก๐ก 2 ๏ฟฝ 2๐ก๐ก๐ก๐ก (Nelson 1982, p.201) Note: The first confidence intervals are conservative in that at least 100๐พ๐พ%. Exact confidence intervals cannot be easily achieved for discrete distributions. Bayesian Non-informative Priors ๐ ๐ (๐๐) in known time interval ๐ก๐ก Poisson Type Prior Posterior Uniform Proper Prior with limits ๐๐ โ [๐๐, ๐๐] 1 ๐๐ โ ๐๐ Uniform Improper Prior with limits ๐๐ โ [0, โ) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0) 1 Jeffreyโs Prior โ๐๐ Truncated Gamma Distribution For a โค ฮป โค b ๐๐. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + k, t) Otherwise ๐๐(๐๐) = 0 ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 1 + k, t) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(2, 0) 1 โ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0) ๐๐ Novick and Hall ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 12 + k, t) when ๐๐ โ [0, โ) ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; k, t) when ๐๐ โ [0, โ) Conjugate Priors UOI ๐๐ from ๐๐๐๐๐๐๐๐(๐๐; µ) Example Likelihood Model Exponential Evidence ๐๐๐น๐น failures in ๐ก๐ก ๐๐ unit of time Dist of UOI Prior Para Gamma ๐๐0 , ฮ0 Description , Limitations and Uses Posterior Parameters ๐๐ = ๐๐๐๐ + ๐๐๐น๐น ฮ = ฮ๐๐ + ๐ก๐ก ๐๐ Three vehicle tires were run on a test area for 1000km have punctures at the following distances: Tire 1: No punctures Tire 2: 400km, 900km Tire 3: 200km Poisson Discrete Distribution 169 Punctures can be modeled as a renewal process with perfect repair and an inter-arrival time modeled by an exponential distribution. Due to the Poisson distribution being homogeneous in time, the test from multiple tires can be combined and considered a test of one tire with multiple renewals. See example in section 1.1.6. Total time on test is 3 × 1000 = 3000km. Total number of failures is 3. Therefore using MLE the estimate of ๐๐: ๐๐ฬ = k 3 = = 1E-3 ๐ก๐ก ๐๐ 3000 With 90% confidence interval (conservative): 2 2 (6) (8) ๐๐(0.95) ๐๐(0.05) = 0.272๐ธ๐ธ-3, = 2.584๐ธ๐ธ-3๏ฟฝ ๏ฟฝ 6000 6000 A Bayesian point estimate using the Jeffery non-informative improper prior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(12, 0), with posterior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 3.5, 3000) has a point estimate: 3.5 = 1.16ฬ E โ 3 ๐๐ฬ = E[๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐; 3.5, 3000)] = 3000 Characteristics With 90% confidence interval using inverse Gamma cdf: [๐น๐น๐บ๐บโ1 (0.05) = 0.361๐ธ๐ธ-3, ๐น๐น๐บ๐บโ1 (0.95) = 2.344๐ธ๐ธ-3] The Poisson distribution is also known as the Rare Event distribution. ฮผ characteristics: โข ๐๐ is the expected number of events for the unit of time being measured. โข When the unit of time varies ฮผ can be transformed into a rate and time measure, ๐๐๐๐. โข For ๐๐ โฒ 10 the distribution is skewed to the right. โข For ๐๐ โณ 10 the distribution approaches a normal distribution with a ๐๐ = ๐๐ and ๐๐ = โ๐๐. ๐พ๐พ~๐๐๐๐๐๐๐๐(๐๐) Convolution property ๐พ๐พ1 + ๐พ๐พ2 + โฆ + ๐พ๐พ๐๐ ~๐๐๐๐๐๐๐๐(๐๐; โ๐๐๐๐ ) Poisson If the following assumptions are met than the process follows a Poisson distribution: โข The chance of two simultaneous events is negligible or impossible (such as renewal of a single component); โข The expected value of the random number of events in a region is proportional to the size of the region. โข The random number of events in non-overlapping regions are independent. 170 Discrete Distributions Applications Homogeneous Poisson Process (HPP). The Poisson distribution gives the distribution of exactly k failures occurring in a HPP. See relation to exponential and gamma distributions. Renewal Theory. Used in renewal theory as the counting function and may model non-homogeneous (aging) components by using a time dependent failure rate, ๐๐ (๐ก๐ก). Binomial Approximation. Used to model the Binomial distribution when the number of trials is large and ฮผ remains moderate. This can greatly simplify Binomial distribution calculations. Rare Event. Used to model rare events when the number of trials is large compared to the rate at which events occur. Resources Online: http://mathworld.wolfram.com/PoissonDistribution.html http://en.wikipedia.org/wiki/Poisson_distribution http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web calculator) Books: Haight, F.A., 1967. Handbook of the Poisson distribution [by] Frank A. Haight, New York,: Wiley. Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience. Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete Distributions 3rd ed., Wiley-Interscience. Poisson Relationship to Other Distributions Let ๐พ๐พ~๐๐๐๐๐๐๐๐(k; µ = ๐๐๐๐) Given Exponential Distribution ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐|๐๐) ๐๐1 , T2 โฆ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(t; ๐๐) The time between each arrival of T is exponentially distributed. Special Cases: Let Gamma Distribution ๐ก๐ก๐ก๐ก๐ก๐ก๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐พ๐พ + ๐๐๐พ๐พ+1 โฆ Then Then ๐๐๐๐๐๐๐๐(k; ๐๐๐๐|๐๐ = 1) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐ก๐ก; ๐๐) ๐๐1 โฆ ๐๐๐๐ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐๐) ๐๐๐๐๐๐ ๐๐๐ก๐ก = ๐๐1 + ๐๐2 + โฏ + ๐๐๐๐ ๐๐๐ก๐ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐) The Poisson distribution is the probability that exactly ๐๐ failures have been observed in time ๐ก๐ก. This is the probability that ๐ก๐ก is between ๐๐๐๐ and ๐๐๐๐+1 . Poisson Discrete Distribution ๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐๐๐) = ๏ฟฝ ๐๐+1 ๐๐ where ๐๐ is an integer. 171 ๐๐๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐ฅ๐ฅ, ๐๐)๐๐๐๐ = ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐๐ + 1, ๐๐) โ ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐ก๐ก; ๐๐, ๐๐) Limiting Case for constant ๐๐๐๐: lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐; ๐๐, ๐๐) = ๐๐๐๐๐๐๐๐(k|µ = n๐๐) ๐๐โโ n๐๐=๐๐ The Poisson distribution is the limiting case of the Binomial distribution when ๐๐ is large but the ratio of ๐๐๐๐ remains constant. Hence the Poisson distribution models rare events. Binomial Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) The Poisson distribution can be used as an approximation to the Binomial distribution when ๐๐ โฅ 20 and ๐๐ โค 0.05, or if ๐๐ โฅ 100 and ๐๐๐๐ โค 10. The Binomial is expressed in terms of the total number of a probability of success, ๐๐, and trials, ๐๐. Where a Poisson distribution is expressed in terms of a success rate and does not need to know the total number of trials. The derivation of the Poisson distribution from the binomial can be found at http://mathworld.wolfram.com/PoissonDistribution.html. Normal Distribution ๐๐๐๐๐๐๐๐(๐๐|๐๐โฒ, ๐๐) Chi-square Distribution ๐๐ 2 (๐ก๐ก|๐ฃ๐ฃ) lim ๐น๐น๐๐๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐) = ๐น๐น๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐โฒ = ๐๐, ๐๐ 2 = ๐๐) ๐๐โโ This is a good approximation when ๐๐ > 1000. When ๐๐ > 10 the same approximation can be made with a correction: lim ๐น๐น๐๐๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐) = ๐น๐น๐๐๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐โฒ = ๐๐ โ 0.5, ๐๐ 2 = ๐๐) ๐๐โโ ๐๐๐๐๐๐๐๐(๐๐|๐๐) = ๐๐ 2 (๐ฅ๐ฅ = 2๐๐, ๐ฃ๐ฃ = 2๐๐ + 2) Poisson This interpretation can also be used to understand the conditional distribution of a Poisson random variable: Let ๐พ๐พ1 , ๐พ๐พ2 ~๐๐๐๐๐๐๐๐(µ) Given ๐๐ = ๐พ๐พ1 + ๐พ๐พ2 = ๐๐๐๐๐๐๐๐๐๐๐๐ ๐๐๐๐ ๐๐๐๐๐๐๐๐๐๐๐๐ Then µ1 ๐พ๐พ1 |n~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝk; n๏ฟฝp = ๏ฟฝ µ1 + µ2 172 Discrete Distributions Poisson 6. Bivariate and Multivariate Distributions Bivariate Normal Distribution 173 6.1. Bivariate Normal Continuous Distribution Probability Density Function - f(x,y) y x x y x x y ๐๐ = 45๐๐ ๐๐ = 0๐๐ ๐๐ < 0 x 135๐๐ < ๐๐ < 180๐๐ Adapted from (Kotz et al. 2000, p.256) x ๐๐ = 45๐๐ x 90๐๐ < ๐๐ < 135๐๐ Bi-var Normal x 45๐๐ < ๐๐ < 90๐๐ y y y x ๐๐ = 45๐๐ ๐๐ = 0๐๐ y ๐๐๐ฅ๐ฅ < ๐๐๐ฆ๐ฆ x 0๐๐ < ๐๐ < 45๐๐ ๐๐ = 0 ๐๐๐ฅ๐ฅ = ๐๐๐ฆ๐ฆ y ๐๐ > 0 0 y y ๐๐๐ฅ๐ฅ > ๐๐๐ฆ๐ฆ 174 Bivariate and Multivariate Distributions Parameters & Description Location parameter: The mean of each random variable. โโ < µj < โ ๐๐ โ {๐ฅ๐ฅ, ๐ฆ๐ฆ} ๐๐๐ฅ๐ฅ , ๐๐๐ฆ๐ฆ ๐๐๐ฅ๐ฅ , ๐๐๐ฆ๐ฆ ๐๐๐๐ > 0 ๐๐ โ {๐ฅ๐ฅ, ๐ฆ๐ฆ} ๐๐ โ1 โค ๐๐ โค 1 Parameters โโ < x < โ ๐๐๐๐๐๐ Limits Distribution Scale parameter: The standard deviation of each random variable. Correlation Coefficient: The correlation between the two random variables. ๐๐๐๐๐๐[๐๐๐๐] ๐๐ = ๐๐๐๐๐๐๐๐(๐๐, ๐๐) = ๐๐๐ฅ๐ฅ ๐๐๐ฆ๐ฆ ๐ธ๐ธ[(๐๐ โ ๐๐๐ฅ๐ฅ )(๐๐ โ ๐๐๐ฆ๐ฆ )] = ๐๐๐ฅ๐ฅ ๐๐๐ฆ๐ฆ โโ<y< โ Formulas 1 ๐๐(x, y) = exp ๏ฟฝ zx 2 + zy 2 โ 2๐๐zx zy ๏ฟฝ โ2(1 โ ๐๐2 ) 2๐๐ฯx ฯy ๏ฟฝ1 โ ๐๐2 = ๐๐(๐ฅ๐ฅ)๐๐(๐ฆ๐ฆ|๐ฅ๐ฅ) ๐ฆ๐ฆ โ ๐๐๐๐ ๐ฅ๐ฅ โ ๐๐๐๐ = ๐๐(๐ฅ๐ฅ)๐๐ ๏ฟฝ ๏ฟฝ = ๐๐(๐ฆ๐ฆ)๐๐ ๏ฟฝ ๏ฟฝ 2 ๏ฟฝ1 โ ๐๐ ๏ฟฝ1 โ ๐๐2 PDF Where ๐๐ is the standard normal distribution and: x โ µj zj = ๐๐ โ {๐ฅ๐ฅ, ๐ฆ๐ฆ} ฯj Bi-var Normal โ Marginal PDF Conditional PDF CDF โ ๐๐(๐ฅ๐ฅ) = ๏ฟฝ ๐๐(๐ฅ๐ฅ, ๐ฆ๐ฆ) ๐๐๐๐ ๐๐(๐ฆ๐ฆ) = ๏ฟฝ ๐๐(๐ฅ๐ฅ, ๐ฆ๐ฆ) ๐๐๐๐ โโ 1 1 = exp ๏ฟฝ โ (zx )2 ๏ฟฝ 2 ๐๐๐ฅ๐ฅ โ2๐๐ = ๐๐๐๐๐๐๐๐(๐๐๐ฅ๐ฅ , ๐๐๐ฅ๐ฅ ) โโ 1 2 exp ๏ฟฝ โ ๏ฟฝzy ๏ฟฝ ๏ฟฝ 2 ๐๐๐ฆ๐ฆ โ2๐๐ = ๐๐๐๐๐๐๐๐๏ฟฝ๐๐๐ฆ๐ฆ , ๐๐๐ฆ๐ฆ ๏ฟฝ = 1 ๐๐๐ฅ๐ฅ 2 ๐๐(๐ฅ๐ฅ|๐ฆ๐ฆ) = ๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐ฅ๐ฅ|๐ฆ๐ฆ = ๐๐๐ฅ๐ฅ + ๐๐ ๏ฟฝ ๏ฟฝ ๏ฟฝ๐ฆ๐ฆ โ ๐๐๐ฆ๐ฆ ๏ฟฝ, ๐๐๐ฅ๐ฅ|๐ฆ๐ฆ = ๐๐๐ฅ๐ฅ2 (1 โ ๐๐2 )๏ฟฝ ๐๐๐ฆ๐ฆ ๐๐๐ฆ๐ฆ 2 ๐๐(๐ฆ๐ฆ|๐ฅ๐ฅ) = ๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐ฆ๐ฆ|๐ฅ๐ฅ = ๐๐๐ฆ๐ฆ + ๐๐ ๏ฟฝ ๏ฟฝ (๐ฆ๐ฆ โ ๐๐๐ฅ๐ฅ ), ๐๐๐ฆ๐ฆ|๐ฅ๐ฅ = ๐๐๐ฆ๐ฆ2 (1 โ ๐๐2 )๏ฟฝ ๐๐๐ฅ๐ฅ ๐น๐น(x, y) = 1 2๐๐ฯx ฯy ๏ฟฝ1 โ ฯ2 x y ๏ฟฝ ๏ฟฝ exp ๏ฟฝ โโ โโ zu2 + zv2 โ 2ฯzu zv ๏ฟฝ du dv โ2(1 โ ฯ2 ) Bivariate Normal Distribution where zj = Reliability ๐ ๐ (x, y) = where 1 2๐๐ฯx ฯy ๏ฟฝ1 โ ฯ2 โ x โ µj ฯj โ ๏ฟฝ ๏ฟฝ exp ๏ฟฝ x zj = y x โ µj ฯj 175 zu2 + zv2 โ 2ฯzu zv ๏ฟฝ du dv 2(1 โ ฯ2 ) Properties and Moments Median Mode Mean - 1st Raw Moment ๐๐๐ฅ๐ฅ ๏ฟฝ๐๐ ๏ฟฝ ๐ฆ๐ฆ ๐๐๐ฅ๐ฅ ๏ฟฝ๐๐ ๏ฟฝ ๐ฆ๐ฆ ๐๐๐ฅ๐ฅ ๐๐ ๐ธ๐ธ ๏ฟฝ ๏ฟฝ = ๏ฟฝ๐๐ ๏ฟฝ ๐ฆ๐ฆ ๐๐ The mean of the marginal distributions is: ๐ธ๐ธ[๐๐] = ๐๐๐ฅ๐ฅ ๐ธ๐ธ[๐๐] = ๐๐๐ฆ๐ฆ Variance - 2nd Central Moment The mean of the conditional distributions gives the following lines (also called the regression lines): ฯx E(X|Y = y) = µx + ฯ. (y โ µy ) ฯy ฯy E(Y|X = x) = µy + ฯ. (y โ µx ) ฯx ๐๐ 2 ๐๐ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ ๏ฟฝ ๏ฟฝ = ๏ฟฝ 1 ๐๐ ๐๐๐๐1 ๐๐2 ๐๐๐๐1 ๐๐2 ๏ฟฝ ๐๐22 100ฮฑ% Percentile Function Variance of conditional distributions: ๐๐๐๐๐๐(X|Y = y) = ฯ2x (1 โ ฯ2 ) ๐๐๐๐๐๐(๐๐|๐๐ = ๐ฅ๐ฅ) = ฯ2y (1 โ ฯ2 ) An ellipse containing 100ฮฑ % of the distribution is (Kotz et al. 2000, p.254): (zx 2 + zy 2 โ 2ฯzx zy ) = ln(1 โ ๐ผ๐ผ) โ2(1 โ ๐๐2 ) Bi-var Normal Variance of marginal distributions: ๐๐๐๐๐๐(X) = ฯ2x ๐๐๐๐๐๐(Y) = ฯ2y 176 Bivariate and Multivariate Distributions where zj = x โ µj ฯj ๐๐ โ {๐ฅ๐ฅ, ๐ฆ๐ฆ} For the standard bivariate normal: x 2 + y 2 โ 2ฯxy = ln(1 โ ๐ผ๐ผ) โ2(1 โ ฯ2 ) Parameter Estimation Maximum Likelihood Function MLE Point Estimates When there is only complete failure data the MLE estimates can be given as (Kotz et al. 2000, p.294): ๏ฟฝ๐ฅ๐ฅ = ๐๐ ๐๐๐น๐น ๐๐๐น๐น 1 ๏ฟฝ ๐ฅ๐ฅ๐๐ nF 2 ๏ฟฝ2x = 1 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐๏ฟฝ) ฯ ๐ฅ๐ฅ nF ๐๐=1 ๐๐๐น๐น 1 ๐๐๏ฟฝ๐ฆ๐ฆ = ๏ฟฝ ๐ฆ๐ฆ๐๐ nF ๐๐=1 ๐๐=1 ๐๐๐น๐น ๐๐๐น๐น 2 ๏ฟฝ2y = 1 ๏ฟฝ๏ฟฝ๐ฆ๐ฆ๐๐ โ ๐๐๏ฟฝ๏ฟฝ ฯ ๐ฆ๐ฆ nF ๐๐=1 1 ๐๐๏ฟฝ = ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐๐ฅ๐ฅ )๏ฟฝ๐ฆ๐ฆ๐๐ โ ๐๐๐ฆ๐ฆ ๏ฟฝ ๐๐ ๏ฟฝ๐๐ ๏ฟฝn ๐ฅ๐ฅ ๐ฆ๐ฆ F ๐๐=1 If one or more of the variables are known, different estimators are given in (Kotz et al. 2000, pp.294-305). ๏ฟฝ2 to give the unbiased A correction factor of -1 can be introduced to the ๐๐ estimators: Bi-var Normal ๏ฟฝ2x = ฯ ๐๐๐น๐น 1 2 ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐ ๏ฟฝ) ๐ฅ๐ฅ nF โ 1 ๐๐=1 ๏ฟฝ2y = ฯ ๐๐๐น๐น 1 2 ๏ฟฝ๏ฟฝ๐ฆ๐ฆ๐๐ โ ๐๐๏ฟฝ๏ฟฝ ๐ฆ๐ฆ nF โ 1 ๐๐=1 Bayesian Non-informative Priors: A complete coverage of numerous reference prior distributions with different parameter ordering is contained in (Berger & Sun 2008). For a summary of the general Bayesian priors and conjugates see the multivariate normal distribution. Description , Limitations and Uses Example The accuracy of a cutting machine used in manufacturing is desired to be measured. 5 cuts at the required length are made. The lengths and room temperature were measured as: 7.436, 10.270, 10.466, 11.039, 11.854 ๐๐๐๐ 19.51, 21.23, 21.41, 22.78, 26.78 oC Bivariate Normal Distribution MLE estimates are: 177 โ xi = 10.213 n โ ti = 22.342 ๐๐๏ฟฝ๐๐ = n ๐๐๏ฟฝ๐ฅ๐ฅ = 2 โ(xi โ ๐๐ ๏ฟฝ) ๐ฟ๐ฟ = 2.7885 nโ1 2 โ(t i โ ๐๐๏ฟฝ) ๐๐ 2 = 7.5033 ๐๐๏ฟฝ ๐๐ = nโ1 2 ๐๐๏ฟฝ ๐ฅ๐ฅ = ๐๐๐น๐น 1 ๐๐๏ฟฝ = ๏ฟฝ(๐ฅ๐ฅ๐๐ โ ๐๐๐ฅ๐ฅ )(๐ก๐ก๐๐ โ ๐๐ ๐๐ ) = 0.1454 ๐๐ ๏ฟฝ๐๐ ๏ฟฝn ๐ฅ๐ฅ ๐๐ F ๐๐=1 If you know the temperature is 24 oC what is the likely cutting distance distribution? Characteristic ๐๐๐ฅ๐ฅ 2 = ๐๐๐ฅ๐ฅ2 (1 โ ๐๐2 )๏ฟฝ ๐๐(๐ฅ๐ฅ|๐ก๐ก = 24) = ๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐ฅ๐ฅ|๐ก๐ก = ๐๐๐ฅ๐ฅ + ๐๐ ๏ฟฝ ๏ฟฝ (๐ก๐ก โ ๐๐ ๐๐ ), ๐๐๐ฅ๐ฅ|๐ก๐ก ๐๐๐ก๐ก ๐๐(๐ฅ๐ฅ|๐ก๐ก = 24) = ๐๐๐๐๐๐๐๐(10.303, 2.730) Also known as Binormal Distribution. Let U, V and W be three independent normally distributed random variables. Then let: ๐๐ = ๐๐ + ๐๐ ๐๐ = ๐๐ + ๐๐ Then (๐๐, ๐๐) has a bivariate normal distribution. (Balakrishnan & Lai 2009, p.483) Correlation Coefficient ๐๐. (Yang et al. 2004, p.49) ๐๐ > 0. When X increases then Y also tends to increase. When ๐๐ = 1 X and Y have a perfect positive linear relationship such that ๐๐ = ๐๐ + ๐๐๐๐ where ๐๐ is positive. ๐๐ < 0. When X increases then Y also tends to decrease. When ๐๐ = โ1 X and Y have a perfect negative linear relationship such that ๐๐ = ๐๐ + ๐๐๐๐ where ๐๐ is negative. ๐๐ = ๐๐. Increases or decreases in X have no affect on Y. X and Bi-var Normal Independence. If ๐๐ and ๐๐ are jointly normal random variables, then they are independent when ๐๐ = 0. This gives a contour plot of ๐๐(๐ฅ๐ฅ, ๐ฆ๐ฆ) with concentric circles around the origin. When given a value on the ๐ฆ๐ฆ axis it does not assist in estimating the value on the ๐ฅ๐ฅ axis and therefore are independent. When ๐๐ and ๐๐ are independent, the pdf reduces to: zx 2 + zy 2 1 ๐๐(x, y) = exp ๏ฟฝโ ๏ฟฝ 2๐๐ฯx ฯy 2 178 Bivariate and Multivariate Distributions Y are independent. Ellipse Axis. (Kotz et al. 2000, p.254) The slope of the main axis from the x-axis is given as: 2๐๐๐๐๐ฅ๐ฅ ๐๐๐ฆ๐ฆ 1 ๏ฟฝ ๐๐ = ๐ก๐ก๐ก๐ก๐๐โ1 ๏ฟฝ 2 2 ๐๐๐ฅ๐ฅ โ ๐๐๐ฆ๐ฆ2 If ๐๐๐ฅ๐ฅ = ๐๐๐ฆ๐ฆ for positive ๐๐ the main axis of the ellipse is 45o from the xaxis. For negative ๐๐ the main axis of the ellipse is -45o from the x-axis. Circular Normal Density Function. (Kotz et al. 2000, p.255) When ๐๐๐ฅ๐ฅ = ๐๐๐ฆ๐ฆ and ๐๐ = 0 the bivariate distribution is known as a circular normal density function. Elliptical Normal Distribution (Kotz et al. 2000, p.255). If ๐๐ = 0 and ๐๐๐ฅ๐ฅ โ ๐๐๐ฆ๐ฆ then the distribution may be known as an elliptical normal distribution. Standard Bivariate Normal Distribution. Occurs when ๐๐ = 0 and ๐๐ = 1. For positive ฯ the main axis of the ellipse is 45o from the x-axis. For negative ฯ the main axis of the ellipse is -45o from the x-axis. ๐๐(x, y) = 1 2๐๐๏ฟฝ1 โ ฯ2 exp ๏ฟฝโ x 2 + y 2 โ 2ฯxy ๏ฟฝ 2(1 โ ฯ2 ) Bi-var Normal Mean / Median / Mode: As per the univariate distributions the mean, median and mode are equal. Matrix Form. The bivariate distribution may be written in matrix form as: ๐๐1 ๐๐ 2 ๐๐๐๐1 ๐๐2 ๐๐ ๐ฟ๐ฟ = ๏ฟฝ 1 ๏ฟฝ ๐๐ = ๏ฟฝ๐๐ ๏ฟฝ ๐บ๐บ = ๏ฟฝ 1 ๏ฟฝ ๐๐2 2 ๐๐๐๐1 ๐๐2 ๐๐22 when ๐ฟ๐ฟ โผ ๐๐๐๐๐๐๐๐2 (๐๐, ๐บ๐บ) ๐๐(๐ฑ๐ฑ) = 1 exp ๏ฟฝโ (๐ฑ๐ฑ โ ๐๐)T ๐บ๐บ โ1 (๐ฑ๐ฑ โ ๐๐)๏ฟฝ 2 2๐๐๏ฟฝ|๐บ๐บ| 1 Where |๐บ๐บ| is the determinant of ๐บ๐บ. This is the form used in multivariate normal distribution. The following properties are given in matrix form: Convolution Property ๐๐ โผ ๐๐๐๐๐๐๐๐๏ฟฝ๐๐๐๐ , ๐บ๐บ๐ฒ๐ฒ ๏ฟฝ Let ๐ฟ๐ฟ โผ ๐๐๐๐๐๐๐๐(๐๐๐๐ , ๐บ๐บ๐ฑ๐ฑ ) Where ๐ฟ๐ฟ โฅ ๐๐ (independent) Then ๐ฟ๐ฟ + ๐๐~๐๐๐๐๐๐๐๐๏ฟฝ๐๐๐๐ + ๐๐๐๐ , ๐บ๐บ๐๐ + ๐บ๐บ๐๐ ๏ฟฝ Bivariate Normal Distribution 179 Note if X and Y are dependent then ๐ฟ๐ฟ + ๐๐ may not be even be normally distributed.(Novosyolov 2006) Scaling Property Let 1 matrix Then 2 matrix ๐๐ = ๐จ๐จ๐จ๐จ + ๐๐ ๐๐~๐๐๐๐๐๐๐๐(๐จ๐จ๐จ๐จ + ๐๐, ๐จ๐จ๐บ๐บ๐จ๐จ๐ป๐ป ) Y is a p x b is a p x 1 matrix A is a p x Marginalize Property: ๐๐1 ๐๐ 2 ๐๐ Let ๏ฟฝ 1 ๏ฟฝ ~๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ 1 ๐๐2 2 ๐๐๐๐1 ๐๐2 ๐๐๐๐1 ๐๐2 ๏ฟฝ๏ฟฝ ๐๐22 Conditional Property: ๐๐1 ๐๐ 2 ๐๐ Let ๏ฟฝ 1 ๏ฟฝ ~๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ 1 ๐๐2 2 ๐๐๐๐1 ๐๐2 ๐๐๐๐1 ๐๐2 ๏ฟฝ๏ฟฝ ๐๐22 Then Then Where ๐๐1 โผ ๐๐๐๐๐๐๐๐(๐๐1 , ๐๐1 ) ๐๐(๐ฅ๐ฅ1 |๐ฅ๐ฅ2 ) = ๐๐๐๐๐๐๐๐๏ฟฝ๐๐1|2 , ๐๐1|2 ๏ฟฝ ๐๐ ๐๐1|2 = ๐๐1 + ๐๐ ๏ฟฝ 1๏ฟฝ (๐ฅ๐ฅ2 โ ๐๐2 ) ๐๐2 ๐๐1|2 = ๐๐1 ๏ฟฝ1 โ ๐๐2 It should be noted that the standard deviation of the marginal distribution does not depend on the given value. Applications The bivariate distribution is used in many more applications which are common to the multivariate normal distribution. Please refer to multivariate normal distribution for a more complete coverage. Resources Online: http://mathworld.wolfram.com/BivariateNormalDistribution.html http://en.wikipedia.org/wiki/Multivariate_normal_distribution http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_binormal_ distri.htm (interactive visual representation) Books: Balakrishnan, N. & Lai, C., 2009. Continuous Bivariate Distributions 2nd ed., Springer. Bi-var Normal Graphical Representation of Multivariate Normal. As with all bivariate distributions having only two dependent variables allows it to be easily graphed (in a three dimensional graph) and visualized. As such the bivariate normal is popular in introducing higher dimensional cases. 180 Bivariate and Multivariate Distributions Yang, K. et al., 2004. Multivariate Statistical Methods in Quality Management 1st ed., McGraw-Hill Professional. Patel, J.K, Read, C.B, 1996. Handbook of the Normal Distribution, 2nd Edition, CRC Bi-var Normal Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer. Dirichlet Distribution 181 6.2. Dirichlet Continuous Distribution Probability Density Function - f(x) ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; [2,2,2]๐๐ ) ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; [10,10,10]๐๐ ) ๐๐ ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; [1,1,1]๐๐ ) ๐๐ ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; [2,1,2]๐๐ ) ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; ๏ฟฝ12, 12, 12๏ฟฝ ) Dirichlet ๐ท๐ท๐ท๐ท๐๐2 ([๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 ]๐๐ ; ๏ฟฝ12, 1,2๏ฟฝ ) 182 Bivariate and Multivariate Distributions Parameters & Description ๐๐ = [๐ผ๐ผ1 , ๐ผ๐ผ2 , โฆ , ๐ผ๐ผ๐๐ , ๐ผ๐ผ0 ]๐๐ Shape Matrix. Note that the matrix ๐ถ๐ถ is ๐๐ + 1 in length. ฮฑi > 0 Dimensions. The number of random variables being modeled. ๐๐ โฅ 1 (integer) ๐๐ 0 โค xi โค 1 ๐๐ Limits ๏ฟฝ ๐ฅ๐ฅ๐๐ โค 1 ๐๐=1 Distribution Formulas ๐ผ๐ผ0 โ1 ๐๐ 1 ๐๐(๐ฑ๐ฑ) = ๏ฟฝ1 โ ๏ฟฝ ๐ฅ๐ฅ๐๐ ๏ฟฝ B(๐๐) ฮฑ โ1 xi i i=1 ๐๐=1 PDF d ๏ฟฝ where B(๐ถ๐ถ) is the multinomial beta function: โdi=0 ฮ(ฮฑi ) B(๐๐) = ฮ๏ฟฝโdi=0 ฮฑi ๏ฟฝ Dirichlet The special case of the Dirichlet distribution is the beta distribution when ๐๐ = 1. Let Where Let Marginal PDF ๐ผ๐ผ โผ ๐ท๐ท๐ท๐ท๐๐๐ ๐ (๐ถ๐ถ๐๐ ) ๐๐(๐ฎ๐ฎ) = ฮ๏ฟฝ๐ผ๐ผฮฃ โ ๐ผ๐ผ ๐ฟ๐ฟ = ๏ฟฝ ๏ฟฝ ~๐ท๐ท๐ท๐ท๐๐๐๐ ( ๐ถ๐ถ) ๐ฝ๐ฝ ๐ฟ๐ฟ = [๐๐1 , โฆ , ๐๐๐ ๐ , ๐๐๐ ๐ +1 , โฆ , ๐๐๐๐ ]๐๐ ๐ผ๐ผ = [๐๐1 , โฆ , ๐๐๐ ๐ ]๐๐ ๐ฝ๐ฝ = [๐๐๐ ๐ +1 , โฆ , ๐๐๐๐ ]๐๐ ๐ผ๐ผฮฃ = โ๐๐๐๐=0 ๐ผ๐ผ๐๐ = sum of ๐ถ๐ถ matrix elements. where ๐ถ๐ถ๐๐ = ๏ฟฝ๐ผ๐ผ1 , ๐ผ๐ผ2 , โฆ , ๐ผ๐ผ๐ ๐ , ๐ผ๐ผฮฃ โ โ๐ ๐ ๐๐=1 ๐ผ๐ผ๐๐ ๏ฟฝ ฮ(ฮฑฮฃ ) โ๐ ๐ ๐๐=1 ๐ผ๐ผ๐๐ ๏ฟฝ โsi=1 ฮ(ฮฑi ) ๐ ๐ ๏ฟฝ1 โ ๏ฟฝ ๐ฅ๐ฅ๐๐ ๏ฟฝ ๐๐=1 ๐ผ๐ผฮฃโ1โโ๐ ๐ ๐๐=1 ๐ผ๐ผ๐๐ s ๏ฟฝ ๐๐ ฮฑ โ1 xi i i=1 Dirichlet Distribution 183 When marginalized to one variable: ๐๐๐๐ ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ผ๐ผ๐๐ , ๐ผ๐ผฮฃ โ ๐ผ๐ผ๐๐ ) ๐๐(๐ฅ๐ฅ๐๐ ) = ฮ(ฮฑฮฃ ) ฮ(๐ผ๐ผฮฃ โ ๐ผ๐ผ๐๐ )ฮ(ฮฑi ) (1 โ ๐ฅ๐ฅ๐๐ )๐ผ๐ผฮฃ โ๐ผ๐ผ๐๐ โ1 xiฮฑiโ1 ๐ผ๐ผ|๐ฝ๐ฝ = ๐๐ โผ ๐ท๐ท๐ท๐ท๐๐๐๐โ๐ ๐ ๏ฟฝ๐ถ๐ถ๐ข๐ข|๐๐ ๏ฟฝ where ๐ถ๐ถ๐๐|๐๐ = [๐ผ๐ผ๐๐+1 , ๐ผ๐ผ๐ ๐ +2 , โฆ , ๐ผ๐ผ๐๐ , ๐ผ๐ผ0 ]๐๐ (Kotz et al. 2000, p.488) Conditional PDF ฮ๏ฟฝโ๐๐=0 ๐ผ๐ผ๐๐ ๏ฟฝ ๐ ๐ ๐๐(๐ฎ๐ฎ|๐ฏ๐ฏ) = ๏ฟฝ1 โsi=0 ฮ(ฮฑi ) ๐ ๐ โ ๏ฟฝ ๐ฅ๐ฅ๐๐ ๏ฟฝ CDF ๐ฅ๐ฅ2 = ๏ฟฝ ๏ฟฝ โฆ๏ฟฝ 0 0 ๐ฅ๐ฅ๐๐ 0 ๐ผ๐ผ0 โ1 ๐๐ ๏ฟฝ1 โ ๏ฟฝ ๐ฅ๐ฅ๐๐ ๏ฟฝ ๐๐=1 ๐ ๐ ๏ฟฝ ๐ผ๐ผ โ1 ๐ฅ๐ฅ๐๐ ๐๐ ๐๐=1 ๐๐=1 ๐น๐น(๐ฑ๐ฑ) = P(X1 โค x1 , X2 โค x2 , โฆ , Xd โค xd ) ๐ฅ๐ฅ1 ๐ผ๐ผ0 โ1 d ๏ฟฝ ฮฑ โ1 xi i i=1 d๐๐, โฆ , ๐๐๐ฅ๐ฅ2 , ๐๐๐ฅ๐ฅ1 Numerical methods have been explored to evaluate this integral, see (Kotz et al. 2000, pp.497-500) Reliability ๐ ๐ (๐ฑ๐ฑ) = P(X1 > x1 , X2 > x2 , โฆ , Xd > xd ) โ โ ๐ผ๐ผ0 โ1 ๐๐ โ = ๏ฟฝ ๏ฟฝ โฆ ๏ฟฝ ๏ฟฝ1 โ ๏ฟฝ ๐ฅ๐ฅ๐๐ ๏ฟฝ ๐ฅ๐ฅ1 ๐ฅ๐ฅ2 ๐ฅ๐ฅ๐๐ ๐๐=1 d ๏ฟฝ ฮฑ โ1 xi i i=1 d๐๐, โฆ , ๐๐๐ฅ๐ฅ2 , ๐๐๐ฅ๐ฅ1 Properties and Moments Median Mode Mean - 1st Raw Moment Solve numerically using ๐น๐น(๐๐) = 0.5 ๐ฅ๐ฅ๐๐ = ๐ผ๐ผ๐๐ โ1 ๐ผ๐ผฮฃ โ๐๐ for ๐ผ๐ผ๐๐ > 0 otherwise no mode Let ๐ผ๐ผฮฃ = โ๐๐๐๐=0 ๐ผ๐ผ๐๐ : ๐ธ๐ธ[๐ฟ๐ฟ] = ๐๐ = ๐ถ๐ถ ๐ผ๐ผฮฃ ๐ธ๐ธ[๐๐๐๐ ] = ๐๐๐๐ = ๐ผ๐ผ๐๐ ๐๐ฮฃ where ๐ ๐ ๐ถ๐ถ๐๐ = ๏ฟฝ๐ผ๐ผ1 , ๐ผ๐ผ2 , โฆ , ๐ผ๐ผ๐ ๐ , ๐ผ๐ผฮฃ โ ๏ฟฝ ๐ผ๐ผ๐๐ ๏ฟฝ ๐๐=1 Mean of the conditional distribution: ๐ถ๐ถ๐๐|๐๐ ๐ธ๐ธ[๐ผ๐ผ|๐ฝ๐ฝ = ๐๐] = ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐ผ๐ผฮฃ Dirichlet Mean of the marginal distribution: ๐ถ๐ถ๐ข๐ข ๐ธ๐ธ[๐ผ๐ผ] = ๐๐๐๐ = ๐ผ๐ผฮฃ ๐๐ 184 Bivariate and Multivariate Distributions where ๐ถ๐ถ๐๐|๐๐ = [๐ผ๐ผ๐๐+1 , ๐ผ๐ผ๐ ๐ +2 , โฆ , ๐ผ๐ผ๐๐ , ๐ผ๐ผ0 ]๐๐ Let ๐ผ๐ผฮฃ = โ๐๐๐๐=0 ๐ผ๐ผ๐๐ : Variance - 2nd Central Moment ๐๐๐๐๐๐[๐๐๐๐ ] = ๐ผ๐ผ๐๐ (๐ผ๐ผฮฃ โ ๐ผ๐ผ๐๐ ) ๐ผ๐ผฮฃ2 (๐ผ๐ผฮฃ + 1) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐๐๐๐ , ๐๐๐๐ ๏ฟฝ = Parameter Estimation โ๐ผ๐ผ๐๐ ๐ผ๐ผ๐๐ ๐ผ๐ผฮฃ2 (๐ผ๐ผฮฃ + 1) Maximum Likelihood Function MLE Point Estimates The MLE estimates of ๐ผ๐ผ๏ฟฝ๐ค๐ค can be obtained from n observations of ๐๐๐๐ by numerically maximizing the log-likelihood function: (Kotz et al. 2000, p.505) ๐๐ ๐๐ ๐๐ ๐๐=0 ๐๐=0 ๐๐=1 1 ฮ(๐๐|E) = ๐๐ ๏ฟฝ๐๐๐๐ฮ(๐ผ๐ผฮฃ ) โ ๏ฟฝ ln ๐ค๐ค๏ฟฝ๐ผ๐ผ๐๐ ๏ฟฝ๏ฟฝ + ๐๐ ๏ฟฝ ๏ฟฝ ๏ฟฝ๐ผ๐ผ๐๐ โ 1๏ฟฝ ๏ฟฝ ln๏ฟฝ๐ฅ๐ฅ๐๐๐๐ ๏ฟฝ๏ฟฝ ๐๐ The method of moments are used to provide initial guesses of ๐ผ๐ผ๐๐ for the numerical methods. Fisher Information Matrix ๐๐ 2 ๐ผ๐ผ๐๐๐๐ = โ๐๐๐๐ โฒ (๐ผ๐ผฮฃ ), ๐๐ โ ๐๐ ๐ผ๐ผ๐๐๐๐ = ๐๐๐๐ โฒ (๐ผ๐ผ๐๐ ) โ ๐๐๐๐ โฒ (๐ผ๐ผฮฃ ) Where ๐๐ โฒ (๐ฅ๐ฅ) = 2 ๐๐๐๐ฮ(๐ฅ๐ฅ) is the trigamma function. See section 1.6.8. ๐๐๐ฅ๐ฅ (Kotz et al. 2000, p.506) 100๐พ๐พ% Confidence Intervals The confidence intervals can be obtained from the fisher information matrix. Dirichlet Bayesian Non-informative Priors Jefferyโs Prior ๏ฟฝdet๏ฟฝ๐ผ๐ผ(๐ถ๐ถ)๏ฟฝ where ๐ผ๐ผ(๐ถ๐ถ) is given above. Conjugate Priors UOI ๐๐ from ๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐๐ก๐ก , ๐๐) Likelihood Model Multinomiald Evidence ๐๐๐๐,๐๐ failures in ๐๐ trials with ๐๐ possible states. Dist of UOI Prior Para Posterior Parameters Dirichletd+1 ๐ถ๐ถ๐๐ ๐ถ๐ถ = ๐ถ๐ถ๐๐ + ๐๐ Dirichlet Distribution 185 Description , Limitations and Uses Example Five machines are measured for performance on demand. The machines can either fail, partially fail or success in their application. The machines are tested for 10 demands with the following data for each machine: Machine/Trail 1 2 3 4 5 ๐๐๐๐ 1 2 3 4 5 F=3 P=2 F=2 P=2 F=2 P=3 F=3 P=3 F=2 P=3 ๐๐๐๐ ๏ฟฝ๐น๐น ๐๐๐๐ ๏ฟฝ๐๐ 6 7 8 9 S=5 S=6 S=5 S=4 S=5 ๐๐๐๐ ๏ฟฝ๐น๐น 10 Estimate the multinomial distribution parameter ๐๐ = [๐๐๐น๐น , ๐๐๐๐ , ๐๐๐๐ ]: Using a non-informative improper prior ๐ท๐ท๐ท๐ท๐๐3 (0,0,0) after updating: ๐๐๐น๐น ๐๐ = ๏ฟฝ๐๐๐๐ ๏ฟฝ ๐๐๐๐ Characteristic 12 ๐ถ๐ถ = ๏ฟฝ13๏ฟฝ 25 ๐๐ ๏ฟฝ๐น๐น = 12 50 ๏ฟฝ๐๐ = 13 ๐ธ๐ธ[๐๐] = ๏ฟฝ๐๐ ๏ฟฝ 50 ๐๐ ๏ฟฝ๐๐ = 25 50 7.15๐ธ๐ธโ5 ๐๐๐๐๐๐[๐๐] = ๏ฟฝ 7.54๐ธ๐ธโ5 ๏ฟฝ 9.80๐ธ๐ธโ5 Confidence intervals for the parameters ๐๐ = [๐๐๐น๐น , ๐๐๐๐ , ๐๐๐๐ ] can also be calculated using the cdf of the marginal distribution ๐น๐น(๐ฅ๐ฅ๐๐ ). Beta Generalization. The Dirichlet distribution is a generalization of the beta distribution. The beta distribution is seen when ๐๐ = 1. ๐ถ๐ถ Interpretation. The higher ๐ผ๐ผ๐๐ the sharper and more certain the distribution is. This follows from its use in Bayesian statistics to model the multinomial distribution parameter ๐๐. As more evidence is used, the ๐ผ๐ผ๐๐ values get higher which reduces uncertainty. The values of ๐ผ๐ผ๐๐ can also be interpreted as a count for each state of the multinomial distribution. This formulation is popular because it is a more simple presentation where the matrix of ๐ถ๐ถ and ๐๐ are the same size. However it should be noted that last term of the vector ๐๐ is dependent on {๐ฅ๐ฅ1 โฆ ๐ฅ๐ฅ๐๐โ1 } through the relationship ๐ฅ๐ฅ๐๐ = 1 โ โ๐๐โ1 ๐๐=1 ๐ฅ๐ฅ๐๐ . Neutrality. (Kotz et al. 2000, p.500) If ๐๐1 and ๐๐2 are non negative Dirichlet Alternative Formulation. The most common formulation of the Dirichlet distribution is as follows: ๐๐ = [๐ผ๐ผ1 , ๐ผ๐ผ2 , โฆ , ๐ผ๐ผ๐๐ ]๐๐ where ฮฑi > 0 ๐ฑ๐ฑ = [๐ฅ๐ฅ1 , ๐ฅ๐ฅ2 , โฆ , ๐ฅ๐ฅ๐๐ ]๐๐ where 0 โค xi โค 1, โ๐๐ ๐๐=1 ๐ฅ๐ฅ๐๐ = 1 m 1 ฮฑi โ1 ๐๐(๐ฑ๐ฑ) = ๏ฟฝ xi B(๐๐) i=1 186 Bivariate and Multivariate Distributions random variables such that ๐๐1 + ๐๐2 โค 1 then ๐๐๐๐ is called neutral if the following are independent: ๐๐๐๐ (๐๐ โ ๐๐) ๐๐๐๐ โฅ 1 โ ๐๐๐๐ If ๐ฟ๐ฟ โผ ๐ท๐ท๐ท๐ท๐๐๐๐ (๐ถ๐ถ) then ๐ฟ๐ฟ is a neutral vector with each Xi being neutral under all permutations of the above definition. This property is unique to the Dirichlet distribution. Applications Bayesian Statistics. The Dirichlet distribution is often used as a conjugate prior to the multinomial likelihood function. Resources Online: http://en.wikipedia.org/wiki/Dirichlet_distribution http://www.cis.hut.fi/ahonkela/dippa/node95.html Books: Kotz, S., Balakrishnan, N. & Johnson, N.L., 2000. Continuous Multivariate Distributions, Volume 1, Models and Applications, 2nd Edition 2nd ed., Wiley-Interscience. Congdon, P., 2007. Bayesian Statistical Modelling 2nd ed., Wiley. MacKay, D.J. & Petoy, L.C., 1995. A hierarchical Dirichlet language model. Natural language engineering. Relationship to Other Distributions Beta Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐ฅ๐ฅ; ๐ผ๐ผ, ๐ฝ๐ฝ) Dirichlet Gamma Distribution ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐ฅ๐ฅ; ๐๐, ๐๐) Special Case: ๐ท๐ท๐ท๐ท๐๐๐๐=1 (x; [ฮฑ1 , ฮฑ0 ]) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐ = ๐ฅ๐ฅ; ๐ผ๐ผ = ๐ผ๐ผ1 , ๐ฝ๐ฝ = ๐ผ๐ผ0 ) Let: Then: Let: Then: ๐๐๐๐ ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, ๐๐๐๐ ) ๐๐. ๐๐. ๐๐ ๐๐๐๐๐๐ ๐๐~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐๐, โ๐๐๐๐ ) ๐๐1 ๐๐2 ๐๐๐๐ ๐๐ = ๏ฟฝ , , โฆ , ๏ฟฝ ๐๐ ๐๐ ๐๐ ๐๐~๐ท๐ท๐ท๐ท๐๐๐๐ (๐ผ๐ผ1 , โฆ , ๐ผ๐ผ๐๐ ) *i.i.d: independent and identically distributed ๐๐ ๐๐ = ๏ฟฝ ๐๐๐๐ ๐๐=1 Multivariate Normal Distribution 187 6.3. Multivariate Normal Continuous Distribution *Note for a graphical representation see bivariate normal distribution Parameters & Description ๐๐ = [๐๐1 , ๐๐2 , โฆ , ๐๐๐๐ ]๐๐ Parameters โฏ โฑ โฏ ๐๐11 โ=๏ฟฝ โฎ ๐๐๐๐1 โโ < µi < โ ๐๐1๐๐ โฎ ๏ฟฝ ๐๐๐๐๐๐ ๐๐๐๐๐๐ > 0 ๐๐๐๐๐๐ โฅ 0 ๐๐ โฅ 2 (integer) ๐๐ Limits Location Vector: A ddimensional vector giving the mean of each random variable. Covariance Matrix: A ๐๐ × ๐๐ matrix which quantifies the random variable variance and dependence. This matrix determines the shape of the distribution. ฮฃ is symmetric positive definite matrix. Dimensions. The number of dependent variables. โโ < xi < โ Distribution Formulas ๐๐(๐ฑ๐ฑ) = PDF 1 (2๐๐)๐๐/2 ๏ฟฝ|๐บ๐บ| 1 exp ๏ฟฝโ (๐ฑ๐ฑ โ ๐๐)T ๐บ๐บ โ1 (๐ฑ๐ฑ โ ๐๐)๏ฟฝ 2 Where |๐บ๐บ| is the determinant of ๐บ๐บ. Where ๐ผ๐ผ = ๏ฟฝ๐๐1 , โฆ , ๐๐๐๐ ๏ฟฝ ๐ฝ๐ฝ = ๏ฟฝ๐๐๐๐+1 , โฆ , ๐๐๐๐ ๏ฟฝ Marginal PDF ๐๐(๐๐) = ๏ฟฝ ๐๐(๐๐) ๐๐๐๐ = โโ ๐๐ ๐ผ๐ผ โผ ๐๐๐๐๐๐๐๐๐๐ (๐๐๐๐ , ๐บ๐บ๐๐๐๐ ) โ Conditional PDF ๐๐ ๐บ๐บ๐ข๐ข๐ข๐ข ๏ฟฝ๏ฟฝ ๐บ๐บ๐๐๐๐ 1 (2๐๐)๐๐/2 ๏ฟฝ|๐บ๐บ๐๐๐๐ | 1 โ1 (๐ฎ๐ฎ โ ๐๐ )๏ฟฝ exp ๏ฟฝโ (๐ฎ๐ฎ โ ๐๐๐๐ )T ๐บ๐บuu ๐๐ 2 ๐ผ๐ผ|๐ฝ๐ฝ = ๐๐ โผ ๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐ข๐ข|๐ฃ๐ฃ , ๐บ๐บ๐ข๐ข|๐ฃ๐ฃ ๏ฟฝ Multivar Normal ๐๐๐ข๐ข ๐บ๐บ๐ข๐ข๐ข๐ข ๐ผ๐ผ ๐ฟ๐ฟ = ๏ฟฝ ๏ฟฝ ~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ ๐๐ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฝ๐ฝ ๐๐ ๐๐ ๐ฟ๐ฟ = ๏ฟฝ๐๐1 , โฆ , ๐๐๐๐ , ๐๐๐๐+1 , โฆ , ๐๐๐๐ ๏ฟฝ Let 188 Bivariate and Multivariate Distributions Where CDF ๐น๐น(๐ฑ๐ฑ) = Reliability ๐ ๐ (๐ฑ๐ฑ) = ๐๐ ๐บ๐บ โ1 (๐๐ โ ๐๐ ) ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐๐ข๐ข + ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฃ๐ฃ๐ฃ๐ฃ ๐ฃ๐ฃ ๐๐ ๐บ๐บ โ1 ๐บ๐บ ๐บ๐บ๐ข๐ข|๐ฃ๐ฃ = ๐บ๐บ๐ข๐ข๐ข๐ข โ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฃ๐ฃ๐ฃ๐ฃ ๐ข๐ข๐ข๐ข ๐ฑ๐ฑ 1 ๏ฟฝ exp ๏ฟฝโ (๐ฑ๐ฑ โ ๐๐)T ๐บ๐บโ1 (๐ฑ๐ฑ โ ๐๐)๏ฟฝ d๐๐ ๐๐/2 2 (2๐๐) ๏ฟฝ|๐บ๐บ| โโ 1 โ 1 ๏ฟฝ exp ๏ฟฝโ (๐ฑ๐ฑ โ ๐๐)T ๐บ๐บ โ1 (๐ฑ๐ฑ โ ๐๐)๏ฟฝ d๐๐ 2 (2๐๐)๐๐/2 ๏ฟฝ|๐บ๐บ| ๐ฑ๐ฑ 1 Properties and Moments ๐๐ Median ๐๐ Mode Mean - 1st ๐ธ๐ธ[๐ฟ๐ฟ] = ๐๐ Raw Moment Mean of the marginal distribution: ๐ธ๐ธ[๐ผ๐ผ] = ๐๐๐ข๐ข ๐ธ๐ธ[๐ฝ๐ฝ] = ๐๐๐ฃ๐ฃ Variance - 2nd Central Moment Mean of the conditional distribution: ๐๐ ๐บ๐บ โ1 (๐๐ โ ๐๐ ) ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐๐ข๐ข + ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฃ๐ฃ๐ฃ๐ฃ ๐ฃ๐ฃ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ[๐ฟ๐ฟ] = ๐บ๐บ Covariance of marginal distributions: ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐) = ๐บ๐บ๐ฎ๐ฎ๐ฎ๐ฎ Covariance of conditional distributions: ๐๐ ๐บ๐บ โ1 ๐บ๐บ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐๐|๐๐) = ๐บ๐บ๐ข๐ข๐ข๐ข โ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฃ๐ฃ๐ฃ๐ฃ ๐ข๐ข๐ข๐ข Multivar Normal Parameter Estimation Maximum Likelihood Function MLE Point Estimates When given complete data of ๐๐๐น๐น samples: ๐๐๐๐ = ๏ฟฝ๐๐1,๐ก๐ก , ๐๐2,๐ก๐ก , โฆ , ๐๐๐๐,๐ก๐ก ๏ฟฝ ๐ป๐ป where ๐ก๐ก = (1,2, โฆ , ๐๐๐น๐น ) The following MLE estimates are given: (Kotz et al. 2000, p.161) ๐๐๐น๐น 1 ๐๐ ๏ฟฝ = ๏ฟฝ ๐๐๐ก๐ก nF ๐๐๐น๐น ๐ก๐ก=1 1 ฮฃ๏ฟฝ๐๐๐๐ = ๏ฟฝ๏ฟฝ๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ค๐ค ๏ฟฝ(๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ฅ๐ฅ ) nF ๐ก๐ก=1 A review of different estimators is given in (Kotz et al. 2000). When estimates are from a low number of samples (๐๐๐น๐น < 30) a correction Multivariate Normal Distribution 189 factor of -1 can be introduced to give the unbiased estimators (Tong 1990, p.53): ๐๐๐น๐น 1 ฮฃ๏ฟฝ๐๐๐๐ = ๏ฟฝ๏ฟฝ๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ค๐ค ๏ฟฝ(๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ฅ๐ฅ ) nF โ 1 ๐ก๐ก=1 Fisher Information Matrix ๐ผ๐ผ๐๐,๐๐ = Bayesian Type Non-informative Priors when ๐บ๐บ is known, ๐๐0 (๐๐) (Yang and Berger 1998, p.22) Prior (๐๐๐๐ ๐บ๐บ โ1 ๐๐๐๐ )โ(๐๐โ2) Shrinkage Uniform Improper Prior with limits ๐บ๐บ โ (0, โ) Jefferyโs Prior Posterior 1 Uniform Improper, Jeffrey, Reference Prior Type ๐๐๐๐๐๐ โ1 ๐๐๐๐ ฮฃ ๐๐๐๐๐๐ ๐๐๐๐๐๐ MDIP ๐ก๐ก=1 No Closed Form Prior Posterior 1 1 ๐๐๏ฟฝ๐บ๐บ โ๐๐ ๏ฟฝ๐ฌ๐ฌ๏ฟฝ~ ๐๐๐๐๐๐โ๐๐๐๐๐ก๐ก๐๐ ๏ฟฝ๐บ๐บ โ๐๐ ; ๐๐๐น๐น โ ๐๐ โ 1, ๐๐๏ฟฝ๐บ๐บ โ๐๐ ๏ฟฝ๐ฌ๐ฌ๏ฟฝ~ ๐บ๐บโ1 ๏ฟฝ ๐๐๐น๐น ๐บ๐บโ1 ๏ฟฝ ๐๐๐น๐น with limits ๐บ๐บ โ (0, โ) ๐๐๐๐๐๐โ๐๐๐๐๐ก๐ก๐๐ ๏ฟฝ๐บ๐บโ๐๐ ; ๐๐๐น๐น , 1 |๐บ๐บ| โi<๐๐(ฮปi โ ฮปj ) Proper - No Closed Form 1 |๐บ๐บ| No Closed Form 1 |๐บ๐บ|(logฮป1 โ logฮปd )dโ2 โi<๐๐(ฮปi โ ฮปj ) Proper - No Closed Form Non-informative Priors when ๐๐ and ๐บ๐บ are unknown for bivariate normal, ๐๐๐๐ (๐๐, ๐บ๐บ). A complete coverage of numerous reference prior distributions with different parameter ordering is contained in (Berger & Sun 2008) Multivar Normal Reference Prior Ordered {๐๐1 , ๐๐๐๐ , ๐๐๐๐ , . . , ๐๐๐๐โ1 } when ๐๐ โ (โ, โ) Non-informative Priors when ๐๐ is known, ๐๐๐๐ (๐บ๐บ) (Yang & Berger 1994) d+1 |๐บ๐บ| 2 Reference Prior Ordered {๐๐๐๐ , ๐๐๐๐ , . . , ๐๐๐๐ } ๐๐๐น๐น 1 ๐บ๐บ ๐๐(๐๐|๐ฌ๐ฌ)~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝµ; ๏ฟฝ ๐๐๐ก๐ก , ๏ฟฝ nF nF 190 Bivariate and Multivariate Distributions Type Prior Posterior 1 Uniform Improper Prior 1 No Closed Form 1 |๐บ๐บ| โi<๐๐(ฮปi โ ฮปj ) No Closed Form 1 |๐บ๐บ| No Closed Form Jefferyโs Prior Reference Prior Ordered {๐๐๐๐ , ๐๐๐๐ , . . , ๐๐๐๐ } Reference Prior Ordered {๐๐1 , ๐๐๐๐ , ๐๐๐๐ , . . , ๐๐๐๐โ1 } MDIP No Closed Form d+1 |๐บ๐บ| 2 1 |๐บ๐บ|(logฮป1 โ logฮปd )dโ2 โi<๐๐(ฮปi โ ฮปj ) No Closed Form where ๐๐๐๐ is the ๐๐๐ก๐กโ eigenvalue of ฮฃ, and ๐ ๐ ๏ฟฝ and ๐ ๐ are population and sample multiple correlation coefficients where: ๐๐๐น๐น 1 ๏ฟฝ๏ฟฝ๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ค๐ค ๏ฟฝ(๐ฅ๐ฅ๐๐,๐ก๐ก โ ๐๐๏ฟฝ๐ฅ๐ฅ ) S๐๐๐๐ = nF โ 1 ๐ก๐ก=1 and ๐๐๐น๐น 1 ๐ฑ๐ฑ๏ฟฝ = ๏ฟฝ ๐๐๐ก๐ก nF ๐ก๐ก=1 Conjugate Priors UOI Multivar Normal ๐๐ from ๐๐๐๐๐๐๐๐๐๐ (๐๐, ๐บ๐บ) Likelihood Model Multi-variate Normal with known ๐บ๐บ Evidenc e ๐๐๐น๐น events at ๐๐ points Dist of UOI Multivariate Normal Prior Para ๐ผ๐ผ๐๐ , ๐๐๐๐ Description , Limitations and Uses Posterior Parameters ๐ผ๐ผ = ๐๐0โ1 ๐๐๐จ๐จ + nF ๐๐ โ1 ๐ฑ๐ฑ๏ฟฝ ๐๐0โ1 + nF ๐บ๐บโ1 ๐๐ = ๐๐0โ1 1 + nF ๐บ๐บ โ๐๐ Example See bivariate normal distribution. Characteristic Standard Spherical Normal Distribution. When ๐๐ = 0, ๐บ๐บ = ๐ผ๐ผ we obtain the standard spherical normal distribution: 1 1 ๐๐(๐ฑ๐ฑ) = exp ๏ฟฝโ ๐ฑ๐ฑ T ๐ฑ๐ฑ๏ฟฝ (2๐๐)๐๐/2 2 Covariance Matrix. (Yang et al. 2004, p.49) Diagonal Elements. The diagonal elements of ฮฃ is the variance of each random variable. ๐๐๐๐๐๐ = ๐๐๐๐๐๐(๐๐๐๐ ) Non Diagonal Elements. Non diagonal elements give the covariance ๐๐๐๐๐๐ = ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐๐๐๐ , ๐๐๐๐ ๏ฟฝ = ๐๐๐๐๐๐ . Hence the matrix is symmetric. Independent Variables. If ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐๐๐๐ , ๐๐๐๐ ๏ฟฝ = ๐๐๐๐๐๐ = 0 then ๐๐๐๐ and Multivariate Normal Distribution - 191 ๐๐๐๐ and independent. ๐๐๐๐๐๐ > 0. When ๐๐๐๐ increases then ๐๐๐๐ and tends to increase. ๐๐๐๐๐๐ < 0. When ๐๐๐๐ increases then ๐๐๐๐ and tends to decrease. Ellipsoid Axis. The ellipsoids has axes pointing in the direction of the eigenvectors of ๐บ๐บ. The magnitude of these axes are given by the corresponding eigenvalues. Mean / Median / Mode: As per the univariate distributions the mean, median and mode are equal. Convolution Property ๐๐ โผ ๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐๐ , ๐บ๐บ๐ฒ๐ฒ ๏ฟฝ Let ๐ฟ๐ฟ โผ ๐๐๐๐๐๐๐๐๐๐ (๐๐๐๐ , ๐บ๐บ๐ฑ๐ฑ ) Where ๐ฟ๐ฟ โฅ ๐๐ (independent) Then ๐ฟ๐ฟ + ๐๐~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐๐ + ๐๐๐๐ , ๐บ๐บ๐๐ + ๐บ๐บ๐๐ ๏ฟฝ Note if X and Y are dependent then X + Y may not be normally distributed. (Novosyolov 2006) Scaling Property Let Then ๐๐ = ๐จ๐จ๐จ๐จ + ๐๐ Y is a p x 1 matrix b is a p x 1 matrix ๐๐~๐๐๐๐๐๐๐๐๐๐ (๐จ๐จ๐จ๐จ + ๐๐, ๐จ๐จ๐บ๐บ๐จ๐จ๐ป๐ป ) A is a p x d matrix Marginalize Property: Let Then ๐๐๐ข๐ข ๐บ๐บ๐ข๐ข๐ข๐ข ๐ผ๐ผ ๐ฟ๐ฟ = ๏ฟฝ ๏ฟฝ ~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ ๐๐ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฝ๐ฝ ๐๐ ๐ผ๐ผ โผ ๐๐๐๐๐๐๐๐๐๐ (๐๐๐๐ , ๐บ๐บ๐๐๐๐ ) Conditional Property: Let Where ๐ผ๐ผ|๐ฝ๐ฝ = ๐๐ โผ ๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐๐ข๐ข|๐ฃ๐ฃ , ๐บ๐บ๐ข๐ข|๐ฃ๐ฃ ๏ฟฝ ๐ผ๐ผ is a p x 1 matrix ๐บ๐บ๐ข๐ข๐ข๐ข ๏ฟฝ๏ฟฝ ๐บ๐บ๐๐๐๐ ๐๐ โ1 ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐๐ข๐ข + ๐บ๐บ๐ข๐ข๐ข๐ข ๐บ๐บ๐ฃ๐ฃ๐ฃ๐ฃ (๐ฝ๐ฝ โ ๐๐๐ฃ๐ฃ ) ๐๐ โ1 ๐บ๐บ๐ข๐ข|๐ฃ๐ฃ = ๐บ๐บ๐ข๐ข๐ข๐ข โ ๐บ๐บ๐ข๐ข๐ข๐ข ๐บ๐บ๐ฃ๐ฃ๐ฃ๐ฃ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ผ๐ผ is a p x 1 matrix It should be noted that the standard deviation of the marginal distribution does not depend on the given values in V. Applications Convenient Properties. (Balakrishnan & Lai 2009, p.477) Popularity of the multivariate normal distribution over other multivariate distributions is due to the convenience of the conditional and marginal distribution properties which both produce univariate normal distributions. Multivar Normal Then ๐๐๐ข๐ข ๐บ๐บ๐ข๐ข๐ข๐ข ๐ผ๐ผ ๐ฟ๐ฟ = ๏ฟฝ ๏ฟฝ ~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๏ฟฝ ๐๐ ๏ฟฝ , ๏ฟฝ ๐๐ ๐บ๐บ๐ข๐ข๐ข๐ข ๐ฝ๐ฝ ๐ฃ๐ฃ ๐บ๐บ๐ข๐ข๐ข๐ข ๏ฟฝ๏ฟฝ ๐บ๐บ๐๐๐๐ 192 Bivariate and Multivariate Distributions Kalman Filter. The Kalman filter estimates the current state of a system in the presence of noisy measurements. This process uses multivariate normal distributions to model the noise. Multivariate Analysis of Variance (MANOVA). A test used to analyze variance and dependence of variables. A popular model used to conduct MANOVA assumes the data comes from a multivariate normal population. Gaussian Regression Process. This is a statistical model for observations or events that occur in a continuous domain of time or space, where every point is associated with a normally distributed random variable and every finite collection of these random variables has a multivariate normal distribution. Multi-Linear Regression. Multi-linear regression attempts to model the relationship between parameters and variables by fitting a linear equation. One model to do such a task (MLE) fits a distribution to the observed variance where a multivariate normal distribution is often assumed. Gaussian Bayesian Belief Networks (BBN). BBNs graphical represent the dependence between variables in a probability distribution. When using continuous random variables BBNs quickly become tremendously complicated. However due to the multivariate normal distributionโs conditional and marginal properties this task is simplified and popular. Multivar Normal Resources Online: http://mathworld.wolfram.com/BivariateNormalDistribution.html http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_binormal _distri.htm (interactive visual representation) Books: Patel, J.K, Read, C.B, 1996. Handbook of the Normal Distribution, 2nd Edition, CRC Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer. Yang, K. et al., 2004. Multivariate Statistical Methods in Quality Management 1st ed., McGraw-Hill Professional. Bertsekas, D.P. & Tsitsiklis, J.N., 2008. Introduction to Probability, 2nd Edition, Athena Scientific. Multinomial Distribution 193 6.4. Multinomial Discrete Distribution Probability Density Function - f(k) 1 1 5 ๐๐ Trinomial Distribution, ๐๐([๐๐1 , ๐๐2 , ๐๐3 ]๐๐ ) where ๐๐ = 8, ๐ฉ๐ฉ = ๏ฟฝ , , ๏ฟฝ . Note ๐๐3 is not shown 3 4 12 because it is determined using ๐๐3 = ๐๐ โ ๐๐1 โ ๐๐2 Multinomial 1 1 1 ๐๐ Trinomial Distribution, ๐๐([๐๐1 , ๐๐2 , ๐๐3 ]๐๐ ) where ๐๐ = 20, ๐ฉ๐ฉ = ๏ฟฝ , , ๏ฟฝ . Note ๐๐3 is not shown 3 2 6 because it is determined as ๐๐3 = ๐๐ โ ๐๐1 โ ๐๐2 194 Bivariate and Multivariate Distributions Parameters & Description ๐๐ Parameters ๐ฉ๐ฉ = [๐๐1 , ๐๐2 , โฆ , ๐๐๐๐ ]๐๐ ๐๐ n>0 (integer) Number of Trials. This is sometimes called the index. (Johnson et al. 1997, p.31) 0 โค pi โค 1 Event Probability Matrix: The probability of event i occurring. ๐๐๐๐ is often called cell probabilities. (Johnson et al. 1997, p.31) ๐๐ ๏ฟฝ ๐๐๐๐ = 1 ๐๐=1 ๐๐ โฅ 2 (integer) Dimensions. The number of mutually exclusive states of the system. k i โ {0, โฆ , ๐๐} ๐๐ Limits ๏ฟฝ ๐๐๐๐ = ๐๐ ๐๐=1 Distribution Formulas where PDF d n k ๐๐(๐ค๐ค) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ pi i 1 2 d i=1 n! n! ฮ(n + 1) n = = ๏ฟฝk , k , . . , k ๏ฟฝ = 1 2 n k1 ! k 2 ! โฆ k d ! โdi=1 k i ! โdi=1 ฮ(k i + 1) Note that in p there is only d-1 โfreeโ variables as the last ๐๐๐๐ = 1 โ โ๐๐โ1 ๐๐=1 ๐๐๐๐ giving the distribution: s dโ1 n k pi i . ๏ฟฝ1 โ ๏ฟฝ pi ๏ฟฝ ๐๐(๐ค๐ค) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ 1 2 n i=1 Multinomial i=1 nโโdโ1 i=1 ki Now the special case of binomial distribution when ๐๐ = 2 can be seen. Let Where Marginal PDF where ๐๐๐๐ ๐ผ๐ผ ๐ฒ๐ฒ = ๏ฟฝ ๏ฟฝ ~๐๐๐๐๐๐๐๐๐๐ ๏ฟฝ๐๐, ๏ฟฝ๐๐ ๏ฟฝ๏ฟฝ ๐ฝ๐ฝ ๐๐ ๐ฒ๐ฒ = [๐พ๐พ1 , โฆ , ๐พ๐พ๐ ๐ , ๐พ๐พ๐ ๐ +1 , โฆ , ๐พ๐พ๐๐ ]๐๐ ๐ผ๐ผ = [๐พ๐พ1 , โฆ , ๐พ๐พ๐ ๐ ]๐๐ ๐ฝ๐ฝ = [๐พ๐พ๐ ๐ +1 , โฆ , ๐พ๐พ๐๐ ]๐๐ ๐ผ๐ผ โผ ๐๐๐๐๐๐๐๐๐ ๐ (๐๐, ๐๐๐๐ ) ๐๐ ๐๐๐๐ = ๏ฟฝ๐๐1 , ๐๐2 , โฆ , ๐๐๐ ๐ โ1 , ๏ฟฝ1 โ โ๐ ๐ โ1 ๐๐=1 ๐๐๐๐ ๏ฟฝ๏ฟฝ Multinomial Distribution 195 s n k ๐๐(๐๐) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ pi i 1 2 s i=1 When only two states ๐๐ = [๐๐, (1 โ ๐๐)]๐๐ : n k ๐๐(๐๐๐๐ ) = ๏ฟฝk ๏ฟฝ pi i (1 โ ๐๐๐๐ )๐๐โ๐๐๐๐ i where Conditional PDF ๐ผ๐ผ|๐ฝ๐ฝ = ๐๐ โผ ๐๐๐๐๐๐๐๐๐ ๐ ๏ฟฝ๐๐๐ข๐ข|๐ฃ๐ฃ , ๐๐๐ข๐ข|๐๐ ๏ฟฝ ๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐ โ ๐๐๐ฃ๐ฃ = ๐๐ โ ๏ฟฝ ๐๐๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = 1 โ๐ ๐ ๐๐=1 ๐๐๐๐ ๐๐=๐ ๐ +1 [๐๐1 , ๐๐2 , โฆ , ๐๐๐ ๐ ]๐๐ ๐น๐น(๐ค๐ค) = P(K1 โค k1 , K 2 โค k 2 , โฆ , K d โค k d ) k1 k2 kd d n j = ๏ฟฝ ๏ฟฝ โฆ ๏ฟฝ ๏ฟฝj , j , . . , j ๏ฟฝ ๏ฟฝ pii 1 2 d i=1 CDF j1 =0 j2 =0 jd=0 ๐ ๐ (๐ค๐ค) = P(K1 > k1 , K 2 > k 2 , โฆ , K d > k d ) n n n d n j = ๏ฟฝ ๏ฟฝ โฆ ๏ฟฝ ๏ฟฝj , j , . . , j ๏ฟฝ ๏ฟฝ pii 1 2 d i=1 Reliability j1 =k1 +1 j2 =k2 +1 jd=kd +1 Properties and Moments ๐๐๐๐๐๐๐๐๐๐๐๐(๐๐๐๐ ) is either {โ๐๐๐๐๐๐ โ, โ๐๐๐๐๐๐ โ} Median3 Mode Mean - 1st Raw Moment ๐๐๐๐๐๐๐๐(๐๐๐๐ ) = โ(๐๐ + 1)๐๐๐๐ โ ๐ธ๐ธ[๐ฒ๐ฒ] = ๐๐ = ๐๐๐๐ Mean of the conditional distribution: ๐ธ๐ธ[๐ผ๐ผ|๐ฝ๐ฝ = ๐๐] = ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐๐ข๐ข|๐ฃ๐ฃ ๐๐๐ข๐ข|๐๐ where ๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐ โ ๐๐๐ฃ๐ฃ = ๐๐ โ ๏ฟฝ ๐๐๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = 3 1 โ๐ ๐ ๐๐=1 ๐๐๐๐ โ๐ฅ๐ฅโ = is the floor function (largest integer not greater than ๐ฅ๐ฅ) โ๐ฅ๐ฅโ = is the ceiling function (smallest integer not less than ๐ฅ๐ฅ) ๐๐=๐ ๐ +1 [๐๐1 , ๐๐2 , โฆ , ๐๐๐ ๐ ]๐๐ Multinomial Mean of the marginal distribution: ๐ธ๐ธ[๐ผ๐ผ] = ๐๐๐๐ = ๐๐๐๐๐ข๐ข ๐ธ๐ธ[๐พ๐พ๐๐ ] = ๐๐๐๐๐๐ = ๐๐๐๐๐๐ 196 Bivariate and Multivariate Distributions ๐๐๐๐๐๐[๐พ๐พ๐๐ ] = ๐๐๐๐๐๐ (1 โ ๐๐๐๐ ) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐พ๐พ๐๐ , ๐พ๐พ๐๐ ๏ฟฝ = โ๐๐๐๐๐๐ ๐๐๐๐ Variance - 2nd Central Moment Covariance of marginal distributions: ๐๐๐๐๐๐[๐พ๐พ๐๐ ] = ๐๐๐๐๐๐ (1 โ ๐๐๐๐ ) Covariance of conditional distributions: ๐๐๐๐๐๐๏ฟฝ๐พ๐พ๐๐|๐๐,๐๐ ๏ฟฝ = ๐๐๐ข๐ข|๐ฃ๐ฃ ๐๐๐ข๐ข|๐ฃ๐ฃ,๐๐ (1 โ ๐๐๐ข๐ข|๐ฃ๐ฃ,๐๐ ) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐พ๐พ๐๐|๐๐,๐๐ , ๐พ๐พ๐๐|๐๐,๐๐ ๏ฟฝ = โ๐๐๐ข๐ข|๐ฃ๐ฃ ๐๐๐ข๐ข|๐ฃ๐ฃ,๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ,๐๐ where ๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = ๐๐ โ ๐๐๐ฃ๐ฃ = ๐๐ โ ๏ฟฝ ๐๐๐๐ ๐๐๐ข๐ข|๐ฃ๐ฃ = Parameter Estimation 1 โ๐ ๐ ๐๐=1 ๐๐๐๐ ๐๐=๐ ๐ +1 [๐๐1 , ๐๐2 , โฆ , ๐๐๐ ๐ ]๐๐ Maximum Likelihood Function MLE Point Estimates As with the binomial distribution the MLE estimates, given the vector k(and therefore n), is:(Johnson et al. 1997, p.51) ๏ฟฝ= ๐ฉ๐ฉ ๐ค๐ค n Where there are ๐๐ observations of ๐๐๐ก๐ก each containing ๐๐๐ก๐ก trails: Multinomial ๏ฟฝ= ๐ฉ๐ฉ 1 โnt=1 nt ๐๐ ๏ฟฝ ๐๐๐ก๐ก ๐ก๐ก=1 100๐พ๐พ% Confidence Intervals An approximation of the joint interval confidence limits for 100๐พ๐พ% given by Goodman in 1965 is:(Johnson et al. 1997, p.51) (Complete Data) 1 4 ๏ฟฝ๐ด๐ด + 2๐๐๐๐ โ ๐ด๐ด๏ฟฝ๐ด๐ด + ๐๐๐๐ (๐๐ โ ๐๐๐๐ )๏ฟฝ 2(๐๐ + ๐ด๐ด) ๐๐ ๐๐๐๐ lower confidence limit: ๐๐๐๐ upper confidence limit: 4 1 ๏ฟฝ๐ด๐ด + 2๐๐๐๐ + ๐ด๐ด๏ฟฝ๐ด๐ด + ๐๐๐๐ (๐๐ โ ๐๐๐๐ )๏ฟฝ ๐๐ 2(๐๐ + ๐ด๐ด) where ฮฆ is the standard normal CDF and: dโ1+ฮณ ๐ด๐ด = ๐๐๐๐โ1+๐พ๐พ = ฮฆโ1 ๏ฟฝ ๏ฟฝ d ๐๐ Multinomial Distribution 197 A complete coverage of estimation techniques and confidence intervals is contained in (Johnson et al. 1997, pp.51-65). A more accurate method which requires numerical methods is given in (Sison & Glaz 1995) Bayesian Type Uniform Prior Jeffreys Prior One Group Reference Prior Non-informative Priors, ๐ ๐ (๐๐) (Yang and Berger 1998, p.6) Prior Posterior 1 = ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ผ๐ผ๐๐ = 1) ๐ถ๐ถ ๏ฟฝโ๐๐๐๐=1 ๐๐๐๐ ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ฉ๐ฉ|1 + ๐ค๐ค) ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ฉ๐ฉ๏ฟฝ12+๐ค๐ค) 1 = ๐ท๐ท๐ท๐ท๐๐๐๐+1 ๏ฟฝ๐ผ๐ผ๐๐ = 2๏ฟฝ where ๐ถ๐ถ is a constant In terms of the reference prior, this approach considers all parameters are of equal importance.(Berger & Bernardo 1992) d-group Reference Prior ๐ถ๐ถ ๐๐ ๏ฟฝโ๐๐โ1 ๐๐=1 ๏ฟฝ๐๐๐๐ ๏ฟฝ1 โ โ๐๐=1 ๐๐๐๐ ๏ฟฝ๏ฟฝ where ๐ถ๐ถ is a constant Proper. See m-group posterior when ๐๐ = 1. This approach considers each parameter to be of different importance (group length 1) and so the parameters must be ordered by importance. (Berger & Bernardo 1992) m-group Reference Prior ๐๐๐๐ (๐๐) = ๏ฟฝ๏ฟฝ1 ๐ถ๐ถ ๐๐๐๐ โ โ๐๐=1 ๐๐๐๐ ๏ฟฝ โ๐๐โ1 ๐๐=1 ๐๐๐๐ ๐๐๐๐ โ๐๐โ1 ๐๐=1 ๏ฟฝ1 โ โ๐๐=1 ๐๐๐๐ ๏ฟฝ ๐๐๐๐+1 ๐ฉ๐ฉ๐ข๐ข = ๏ฟฝ๐๐๐๐๐๐โ1+1 , โฆ , ๐๐๐๐๐๐ ๏ฟฝ ๐ถ๐ถ is a constant Posterior: ๐๐(๐๐|๐๐) โ ๐๐ ๐๐ ๏ฟฝ1 โ โ๐๐=1 ๐๐๐๐ ๏ฟฝ ๐๐ ๐๐๐๐ โ12 ๐๐๐๐ ๐๐โ1 ๏ฟฝโ๐๐โ1 ๐๐=1 ๐๐๐๐ โ๐๐=1 ๏ฟฝ1 โ โ๐๐=1 ๐๐๐๐ ๏ฟฝ ๐๐๐๐+1 This approach splits the parameters into m different groups of importance. Within the group order is not important, but the groups need to be ordered by importance. It is common to have ๐๐ = 2 and split the parameters into importance and nuisance parameters. (Berger & Bernardo 1992) Multinomial where groups are given by: ๐๐ ๐๐ ๐ฉ๐ฉ๐๐ = ๏ฟฝ๐๐1 , โฆ ๐๐๐๐1 ๏ฟฝ , ๐ฉ๐ฉ๐๐ = ๏ฟฝ๐๐๐๐1+1 , โฆ , ๐๐๐๐1+๐๐2 ๏ฟฝ ๐๐๐๐ = ๐๐1 + โฏ + ๐๐๐๐ for ๐๐ = 1, โฆ , ๐๐ 198 Bivariate and Multivariate Distributions ๐๐ MDIP Novick and Hallโs Prior (improper) UOI ๐๐=1 ๐๐ ๏ฟฝ ๐๐๐๐โ1 = ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ผ๐ผ๐๐ = 0) ๐๐=1 Conjugate Priors (Fink 1997) Multinomiald Evidence ๐๐๐๐,๐๐ failures in ๐๐ trials with ๐๐ possible states. ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ฉ๐ฉ|๐ค๐ค) Dist of UOI Prior Para Posterior Parameters Dirichletd+1 ๐ถ๐ถ๐๐ ๐ถ๐ถ = ๐ถ๐ถ๐๐ + ๐๐ Description , Limitations and Uses A six sided dice being thrown 60 times produces the following multinomial distribution: Face Number 1 2 3 4 5 6 Characteristic ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ฉ๐ฉโฒ|๐๐๐๐ + 1 + k i ) ๐๐ ๐๐๐๐ ๐๐ = ๐ท๐ท๐ท๐ท๐๐๐๐+1 (๐ผ๐ผ๐๐ = ๐๐๐๐ + 1) Likelihood Model ๐๐ from ๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐๐ก๐ก , ๐๐) Example ๏ฟฝ Times Observed 12 7 11 10 8 12 12 โก6โค โข12โฅ ๐๐ = โข โฅ โข10โฅ โข8โฅ โฃ12โฆ 0.2 โก 0.1 โค โข 0.2 โฅ โฅ ๐๐ = โข โข0.16ฬ โฅ โข0.13ฬ โฅ โฃ 0.2 โฆ ๐๐ = 60 Binomial Generalization. The multinomial distribution is a generalization of the binomial distribution where more than two states of the system are allowed. The binomial distribution is a special case where ๐๐ = 2. Covariance. All covarianceโs are negative. This is because the increase in one parameter ๐๐๐๐ must result in the decrease of ๐๐๐๐ to satisfy ฮฃ๐๐๐๐ = 1. Multinomial With Replacement. The multinomial distribution assumes replacement. The equivalent distribution which assumes without replacement is the multivariate hypergeometric distribution. Convolution Property Let Then ๐ฒ๐ฒ๐๐ โผ ๐๐๐๐๐๐๐๐๐๐ (๐๐; ๐๐๐ก๐ก , ๐ฉ๐ฉ) ๏ฟฝ ๐ฒ๐ฒ๐๐ ~๐๐๐๐๐๐๐๐๐๐ (โ๐๐๐ก๐ก ; โ๐๐๐ก๐ก , ๐๐) *This does not hold when the p parameter differs. Applications Partial Failures. When the states of a system under demands cannot Multinomial Distribution 199 be modeled with two states (success or failure) the multinomial distribution may be used. Examples of this include when modeling discrete states of component degradation. Resources Online: http://en.wikipedia.org/wiki/Multinomial_distribution http://mathworld.wolfram.com/MultinomialDistribution.html http://www.math.uah.edu/stat/bernoulli/Multinomial.xhtml Books: Johnson, N.L., Kotz, S. & Balakrishnan, N., 1997. Discrete Multivariate Distributions 1st ed., Wiley-Interscience. Relationship to Other Distributions Binominal Distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) Special Case: ๐๐๐๐๐๐๐๐๐๐=2 (๐ค๐ค|n, ๐ฉ๐ฉ) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐๐|๐๐, ๐๐) Multinomial 200 Probability Distributions Used in Reliability Engineering 7. References Abadir, K. & Magdalinos, T., 2002. The Characteristic Function from a Family of Truncated Normal Distributions. Econometric Theory, 18(5), p.1276-1287. Agresti, A., 2002. Categorical data analysis, John Wiley and Sons. Aitchison, J.J. & Brown, J.A.C., 1957. The Lognormal Distribution, New York: Cambridge University Press. Andersen, P.K. et al., 1996. Statistical Models Based on Counting Processes Corrected., Springer. Angus, J.E., 1994. Bootstrap one-sided confidence intervals for the log-normal mean. Journal of the Royal Statistical Society. Series D (The Statistician), 43(3), p.395โ401. Anon, Six Sigma | Process Management | Strategic Process Management | Welcome to SSA & Company. Antle, C., Klimko, L. & Harkness, W., 1970. Confidence Intervals for the Parameters of the Logistic Distribution. Biometrika, 57(2), p.397-402. Aoshima, M. & Govindarajulu, Z., 2002. Fixed-width confidence interval for a lognormal mean. International Journal of Mathematics and Mathematical Sciences, 29(3), p.143โ153. Arnold, B.C., 1983. Pareto distributions, Fairland, MD: International Co-operative Pub. House. Artin, E., 1964. The Gamma Function, New York: Holt, Rinehart & Winston. Balakrishnan, 1991. Handbook of the Logistic Distribution 1st ed., CRC. References Balakrishnan, N. & Basu, A.P., 1996. Exponential Distribution: Theory, Methods and Applications 1st ed., CRC. Balakrishnan, N. & Lai, C.-D., 2009. Continuous Bivariate Distributions 2nd ed., Springer. Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20: Advances in Reliability 1st ed., Elsevier Science & Technology. Berger, J.O., 1993. Statistical Decision Theory and Bayesian Analysis 2nd ed., Springer. References 201 Berger, J.O. & Bernardo, J.M., 1992. Ordered Group Reference Priors with Application to the Multinomial Problem. Biometrika, 79(1), p.25-37. Berger, J.O. & Sellke, T., 1987. Testing a Point Null Hypothesis: The Irreconcilability of P Values and Evidence. Journal of the American Statistical Association, 82(397), p.112-122. Berger, J.O. & Sun, D., 2008. Objective priors for the bivariate normal model. The Annals of Statistics, 36(2), p.963-982. Bernardo, J.M. et al., 1992. On the development of reference priors. Bayesian statistics, 4, p.35โ60. Berry, D.A., Chaloner, K.M. & Geweke, J.K., 1995. Bayesian Analysis in Statistics and Econometrics: Essays in Honor of Arnold Zellner 1st ed., Wiley-Interscience. Bertsekas, D.P. & Tsitsiklis, J.N., 2008. Introduction to Probability 2nd ed., Athena Scientific. Billingsley, P., 1995. Probability and Measure, 3rd Edition 3rd ed., Wiley-Interscience. Birnbaum, Z.W. & Saunders, S.C., 1969. A New Family of Life Distributions. Journal of Applied Probability, 6(2), p.319-327. Bjõrck, A., 1996. Numerical Methods for Least Squares Problems 1st ed., SIAM: Society for Industrial and Applied Mathematics. Bowman, K.O. & Shenton, L.R., 1988. Properties of estimators for the gamma distribution, CRC Press. Brown, L.D., Cai, T.T. & DasGupta, A., 2001. Interval estimation for a binomial proportion. Statistical Science, p.101โ117. Christensen, R. & Huffman, M.D., 1985. Bayesian Point Estimation Using the Predictive Distribution. The American Statistician, 39(4), p.319-321. Cohen, 1991. Truncated and Censored Samples 1st ed., CRC Press. Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for scientists and engineers, Birkhäuser. Cozman, F. & Krotkov, E., 1997. Truncated Gaussians as Tolerance Sets. Crow, E.L. & Shimizu, K., 1988. Lognormal distributions, CRC Press. References Congdon, P., 2007. Bayesian Statistical Modelling 2nd ed., Wiley. 202 Probability Distributions Used in Reliability Engineering Dekking, F.M. et al., 2007. A Modern Introduction to Probability and Statistics: Understanding Why and How, Springer. Fink, D., 1997. A compendium of conjugate priors. See http://www. people. cornell. edu/pages/df36/CONJINTRnew% 20TEX. pdf, p.46. Georges, P. et al., 2001. Multivariate Survival Modelling: A Unified Approach with Copulas. SSRN eLibrary. Gupta and Nadarajah, 2004. Handbook of beta distribution and its applications, CRC Press. Gupta, P.L., Gupta, R.C. & Tripathi, R.C., 1997. On the monotonic properties of discrete failure rates. Journal of Statistical Planning and Inference, 65(2), p.255-268. Haight, F.A., 1967. Handbook of the Poisson distribution, New York,: Wiley. Hastings, N.A.J., Peacock, B. & Evans, M., 2000. Statistical Distributions, 3rd Edition 3rd ed., John Wiley & Sons Inc. Jiang, R. & Murthy, D.N.P., 1996. A mixture model involving three Weibull distributions. In Proceedings of the Second AustraliaโJapan Workshop on Stochastic Models in Engineering, Technology and Management. Gold Coast, Australia, pp. 260-270. Jiang, R. & Murthy, D.N.P., 1998. Mixture of Weibull distributions - parametric characterization of failure rate function. Applied Stochastic Models and Data Analysis, (14), p.47-65. Jiang, R. & Murthy, D.N.P., 1995. Modeling Failure-Data by Mixture of2 Weibull Distributions : A Graphical Approach. IEEE Transactions on Reliability, 44, p.477-488. Jiang, R. & Murthy, D.N.P., 1999. The exponentiated Weibull family: a graphical approach. Reliability, IEEE Transactions on, 48(1), p.68-72. References Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete Distributions 3rd ed., Wiley-Interscience. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous Univariate Distributions, Vol. 1 2nd ed., Wiley-Interscience. Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience. References 203 Johnson, N.L., Kotz, S. & Balakrishnan, N., 1997. Discrete Multivariate Distributions 1st ed., Wiley-Interscience. Kimball, B.F., 1960. On the Choice of Plotting Positions on Probability Paper. Journal of the American Statistical Association, 55(291), p.546-560. Kleiber, C. & Kotz, S., 2003. Statistical Size Distributions in Economics and Actuarial Sciences 1st ed., Wiley-Interscience. Klein, J.P. & Moeschberger, M.L., 2003. Survival analysis: techniques for censored and truncated data, Springer. Kotz, S., Balakrishnan, N. & Johnson, N.L., 2000. Continuous Multivariate Distributions, Volume 1, Models and Applications, 2nd Edition 2nd ed., WileyInterscience. Kotz, S. & Dorp, J.R. van, 2004. Beyond Beta: Other Continuous Families Of Distributions With Bounded Support And Applications, World Scientific Publishing Company. Kundu, D., Kannan, N. & Balakrishnan, N., 2008. On the hazard function of BirnbaumSaunders distribution and associated inference. Comput. Stat. Data Anal., 52(5), p.2692-2702. Lai, C.D., Xie, M. & Murthy, D.N.P., 2003. A modified Weibull distribution. IEEE Transactions on Reliability, 52(1), p.33-37. Lai, C.-D. & Xie, M., 2006. Stochastic Ageing and Dependence for Reliability 1st ed., Springer. Lawless, J.F., 2002. Statistical Models and Methods for Lifetime Data 2nd ed., WileyInterscience. Leemis, L.M. & McQueston, J.T., 2008. Univariate distribution relationships. The American Statistician, 62(1), p.45โ53. Leipnik, R.B., 1991. On Lognormal Random Variables: I-the Characteristic Function. The ANZIAM Journal, 32(03), p.327-347. Limpert, E., Stahel, W. & Abbt, M., 2001. Log-normal Distributions across the Sciences: Keys and Clues. BioScience, 51(5), p.352, 341. References Lemonte, A.J., Cribari-Neto, F. & Vasconcellos, K.L.P., 2007. Improved statistical inference for the two-parameter Birnbaum-Saunders distribution. Computational Statistics & Data Analysis, 51(9), p.4656-4681. 204 Probability Distributions Used in Reliability Engineering MacKay, D.J.C. & Petoy, L.C.B., 1995. A hierarchical Dirichlet language model. Natural language engineering. Manzini, R. et al., 2009. Maintenance for Industrial Systems 1st ed., Springer. Martz, H.F. & Waller, R., 1982. Bayesian reliability analysis, JOHN WILEY & SONS, INC, 605 THIRD AVE, NEW YORK, NY 10158. Meeker, W.Q. & Escobar, L.A., 1998. Statistical Methods for Reliability Data 1st ed., Wiley-Interscience. Modarres, M., Kaminskiy, M. & Krivtsov, V., 1999. Reliability engineering and risk analysis, CRC Press. Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience. Nelson, W.B., 1990. Accelerated Testing: Statistical Models, Test Plans, and Data Analysis, Wiley-Interscience. Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience. Novosyolov, A., 2006. The sum of dependent normal variables may be not normal. http://risktheory.ru/papers/sumOfDep.pdf. Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution 2nd ed., CRC. Pham, H., 2006. Springer Handbook of Engineering Statistics 1st ed., Springer. Provan, J.W., 1987. Probabilistic approaches to the material-related reliability of fracture-sensitive structures. Probabilistic fracture mechanics and reliability(A 87-35286 15-38). Dordrecht, Martinus Nijhoff Publishers, 1987,, p.1โ45. Rao, C.R. & Toutenburg, H., 1999. Linear Models: Least Squares and Alternatives 2nd ed., Springer. Rausand, M. & Høyland, A., 2004. System reliability theory, Wiley-IEEE. References Rencher, A.C., 1997. Multivariate Statistical Inference and Applications, Volume 2, Methods of Multivariate Analysis Har/Dis., Wiley-Interscience. Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC. Schneider, H., 1986. Truncated and censored samples from normal populations, M. Dekker. References 205 Simon, M.K., 2006. Probability Distributions Involving Gaussian Random Variables: A Handbook for Engineers and Scientists, Springer. Singpurwalla, N.D., 2006. Reliability and Risk: A Bayesian Perspective 1st ed., Wiley. Sison, C.P. & Glaz, J., 1995. Simultaneous Confidence Intervals and Sample Size Determination for Multinomial Proportions. Journal of the American Statistical Association, 90(429). Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer. Xie, M., Gaudoin, O. & Bracquemond, C., 2002. Redefining Failure Rate Function for Discrete Distributions. International Journal of Reliability, Quality & Safety Engineering, 9(3), p.275. Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean residual life and failure rate function for some generalized Weibull distributions. Reliability Engineering and System Safety, 84(3), p.293โ299. Xie, M., Tang, Y. & Goh, T.N., 2002. A modified Weibull extension with bathtub-shaped failure rate function. Reliability Engineering and System Safety, 76(3), p.279โ 285. Yang and Berger, 1998. A Catalog of Noninformative Priors (DRAFT). Yang, K. et al., 2004. Multivariate Statistical Methods in Quality Management 1st ed., McGraw-Hill Professional. Yang, R. & Berger, J.O., 1994. Estimation of a Covariance Matrix Using the Reference Prior. The Annals of Statistics, 22(3), p.1195-1211. Zhou, X.H. & Gao, S., 1997. Confidence intervals for the log-normal mean. Statistics in medicine, 16(7), p.783โ790. References