Download Textbook: Probability Distributions Used in Reliability Engineering

Document related concepts

Probability wikipedia , lookup

Statistics wikipedia , lookup

History of statistics wikipedia , lookup

Transcript
Probability Distributions
Used in
Reliability Engineering
Probability Distributions
Used in
Reliability Engineering
Andrew N. Oโ€™Connor
Mohammad Modarres
Ali Mosleh
Center for Risk and Reliability
0151 Glenn L Martin Hall
University of Maryland
College Park, Maryland
Published by the Center for Risk and Reliability
International Standard Book Number (ISBN): 978-0-9966468-1-9
Copyright © 2016 by the Center for Reliability Engineering
University of Maryland, College Park, Maryland, USA
All rights reserved. No part of this book may be reproduced or transmitted in any form or
by any means, electronic or mechanical, including photocopying, recording, or by any
information storage and retrieval system, without permission in writing from The Center
for Reliability Engineering, Reliability Engineering Program.
The Center for Risk and Reliability
University of Maryland
College Park, Maryland 20742-7531
In memory of Willie Mae Webb
This book is dedicated to the memory of Miss Willie Webb who
passed away on April 10 2007 while working at the Center for Risk
and Reliability at the University of Maryland (UMD). She initiated
the concept of this book, as an aid for students conducting studies
in Reliability Engineering at the University of Maryland. Upon
passing, Willie bequeathed her belongings to fund a scholarship
providing financial support to Reliability Engineering students at
UMD.
Preface
Reliability Engineers are required to combine a practical understanding of science and
engineering with statistics. The reliability engineerโ€™s understanding of statistics is focused
on the practical application of a wide variety of accepted statistical methods. Most
reliability texts provide only a basic introduction to probability distributions or only provide
a detailed reference to few distributions. Most texts in statistics provide theoretical detail
which is outside the scope of likely reliability engineering tasks. As such the objective of
this book is to provide a single reference text of closed form probability formulas and
approximations used in reliability engineering.
This book provides details on 22 probability distributions. Each distribution section
provides a graphical visualization and formulas for distribution parameters, along with
distribution formulas. Common statistics such as moments and percentile formulas are
followed by likelihood functions and in many cases the derivation of maximum likelihood
estimates. Bayesian non-informative and conjugate priors are provided followed by a
discussion on the distribution characteristics and applications in reliability engineering.
Each section is concluded with online and hardcopy references which can provide further
information followed by the relationship to other distributions.
The book is divided into six parts. Part 1 provides a brief coverage of the fundamentals of
probability distributions within a reliability engineering context. Part 1 is limited to concise
explanations aimed to familiarize readers. For further understanding the reader is referred
to the references. Part 2 to Part 6 cover Common Life Distributions, Univariate Continuous
Distributions, Univariate Discrete Distributions and Multivariate Distributions respectively.
The authors would like to thank the many students in the Reliability Engineering Program
particularly Reuel Smith for proof reading.
Contents
i
Contents
PREFACE ...................................................................................................................................................... V
CONTENTS .................................................................................................................................................... I
1.
FUNDAMENTALS OF PROBABILITY DISTRIBUTIONS ...................................................... 1
1.1.
Probability Theory .......................................................................................................... 2
1.1.1.
Theory of Probability ................................................................................................ 2
1.1.2.
Interpretations of Probability Theory ....................................................................... 2
1.1.3.
Laws of Probability.................................................................................................... 3
1.1.4.
Law of Total Probability ............................................................................................ 4
1.1.5.
Bayesโ€™ Law ................................................................................................................ 4
1.1.6.
Likelihood Functions ................................................................................................. 5
1.1.7.
Fisher Information Matrix ......................................................................................... 6
1.2.
Distribution Functions .................................................................................................... 9
1.2.1.
Random Variables ..................................................................................................... 9
1.2.2.
Statistical Distribution Parameters ........................................................................... 9
1.2.3.
Probability Density Function ..................................................................................... 9
1.2.4.
Cumulative Distribution Function ........................................................................... 11
1.2.5.
Reliability Function ................................................................................................. 12
1.2.6.
Conditional Reliability Function .............................................................................. 13
1.2.7.
100ฮฑ% Percentile Function ..................................................................................... 13
1.2.8.
Mean Residual Life.................................................................................................. 13
1.2.9.
Hazard Rate ............................................................................................................ 13
1.2.10. Cumulative Hazard Rate ......................................................................................... 14
1.2.11. Characteristic Function ........................................................................................... 15
1.2.12. Joint Distributions ................................................................................................... 16
1.2.13. Marginal Distribution .............................................................................................. 17
1.2.14. Conditional Distribution.......................................................................................... 17
1.2.15. Bathtub Distributions.............................................................................................. 17
1.2.16. Truncated Distributions .......................................................................................... 18
1.2.17. Summary ................................................................................................................. 19
1.3.
Distribution Properties ................................................................................................. 20
1.3.1.
Median / Mode ....................................................................................................... 20
1.3.2.
Moments of Distribution ........................................................................................ 20
1.3.3.
Covariance .............................................................................................................. 21
1.4.
Parameter Estimation ................................................................................................... 22
1.4.1.
Probability Plotting Paper ....................................................................................... 22
1.4.2.
Total Time on Test Plots ......................................................................................... 23
1.4.3.
Least Mean Square Regression ............................................................................... 24
1.4.4.
Method of Moments .............................................................................................. 25
1.4.5.
Maximum Likelihood Estimates .............................................................................. 26
1.4.6.
Bayesian Estimation ................................................................................................ 27
ii
Probability Distributions Used in Reliability Engineering
1.4.7.
1.5.
Confidence Intervals ............................................................................................... 30
Related Distributions .................................................................................................... 33
1.6.
Supporting Functions .................................................................................................... 34
1.6.1.
Beta Function ๐๐๐’™๐’™, ๐’š๐’š ................................................................................................ 34
1.6.2.
Incomplete Beta Function ๐‘ฉ๐‘ฉ๐‘ฉ๐‘ฉ(๐’•๐’•; ๐’™๐’™, ๐’š๐’š) .................................................................... 34
1.6.3.
Regularized Incomplete Beta Function ๐‘ฐ๐‘ฐ๐‘ฐ๐‘ฐ(๐’•๐’•; ๐’™๐’™, ๐’š๐’š) .................................................. 34
1.6.4.
Complete Gamma Function ๐šช๐šช(๐’Œ๐’Œ) ........................................................................... 34
1.6.5.
Upper Incomplete Gamma Function ๐šช๐šช(๐’Œ๐’Œ, ๐’•๐’•) .......................................................... 35
1.6.6.
Lower Incomplete Gamma Function ๐›„๐›„(๐’Œ๐’Œ, ๐’•๐’•)........................................................... 35
1.6.7.
Digamma Function ๐๐๐๐ ........................................................................................... 36
1.6.8.
Trigamma Function ๐๐โ€ฒ๐’™๐’™.......................................................................................... 36
1.7.
Referred Distributions .................................................................................................. 37
1.7.1.
Inverse Gamma Distribution ๐‘ฐ๐‘ฐ๐‘ฐ๐‘ฐ(๐œถ๐œถ, ๐œท๐œท).................................................................... 37
1.7.2.
Student T Distribution ๐‘ป๐‘ป(๐œถ๐œถ, ๐๐, ๐ˆ๐ˆ๐ˆ๐ˆ) ......................................................................... 37
1.7.3.
F Distribution ๐‘ญ๐‘ญ(๐’๐’๐Ÿ๐Ÿ, ๐’๐’๐Ÿ๐Ÿ) ........................................................................................ 37
1.7.4.
Chi-Square Distribution ๐Œ๐Œ๐Œ๐Œ(๐’—๐’—) ............................................................................... 37
1.7.5.
Hypergeometric Distribution ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ๐‘ฏ(๐’Œ๐’Œ; ๐’๐’, ๐’Ž๐’Ž, ๐‘ต๐‘ต) ....................................... 38
1.7.6.
Wishart Distribution ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ(๐’™๐’™; ๐šบ๐šบ, ๐’๐’) .............................................................. 38
1.8.
2.
Nomenclature and Notation ......................................................................................... 39
COMMON LIFE DISTRIBUTIONS ............................................................................................40
2.1.
Exponential Continuous Distribution ............................................................................ 41
2.2.
Lognormal Continuous Distribution .............................................................................. 49
2.3.
Weibull Continuous Distribution .................................................................................. 59
3.
BATHTUB LIFE DISTRIBUTIONS ...........................................................................................68
3.1.
2-Fold Mixed Weibull Distribution ................................................................................ 69
3.2.
Exponentiated Weibull Distribution ............................................................................. 76
3.3.
Modified Weibull Distribution ...................................................................................... 81
4.
UNIVARIATE CONTINUOUS DISTRIBUTIONS ...................................................................85
4.1.
Beta Continuous Distribution ....................................................................................... 86
4.2.
Birnbaum Saunders Continuous Distribution ............................................................... 93
4.3.
Contents
iii
Gamma Continuous Distribution .................................................................................. 99
4.4.
Logistic Continuous Distribution ................................................................................. 108
4.5.
Normal (Gaussian) Continuous Distribution ............................................................... 115
4.6.
Pareto Continuous Distribution .................................................................................. 125
4.7.
Triangle Continuous Distribution ................................................................................ 131
4.8.
Truncated Normal Continuous Distribution................................................................ 135
4.9.
Uniform Continuous Distribution ............................................................................... 145
5.
UNIVARIATE DISCRETE DISTRIBUTIONS ....................................................................... 151
5.1.
Bernoulli Discrete Distribution ................................................................................... 152
5.2.
Binomial Discrete Distribution .................................................................................... 157
5.3.
Poisson Discrete Distribution ...................................................................................... 165
6.
BIVARIATE AND MULTIVARIATE DISTRIBUTIONS ..................................................... 172
6.1.
Bivariate Normal Continuous Distribution .................................................................. 173
6.2.
Dirichlet Continuous Distribution ............................................................................... 181
6.3.
Multivariate Normal Continuous Distribution ............................................................ 187
6.4.
Multinomial Discrete Distribution .............................................................................. 193
7.
REFERENCES .............................................................................................................................. 200
iv
Probability Distributions Used in Reliability Engineering
Introduction
1
Prob Theory
1. Fundamentals of Probability
Distributions
Prob Theory
2
Probability Distributions Used in Reliability Engineering
1.1. Probability Theory
1.1.1. Theory of Probability
The theory of probability formalizes the representation of probabilistic concepts through a
set of rules. The most common reference to formalizing the rules of probability is through
a set of axioms proposed by Kolmogorov in 1933. Where ๐ธ๐ธ๐‘–๐‘– is an event in the event space
ฮฉ =โˆช๐‘›๐‘›๐‘–๐‘–=1 ๐ธ๐ธ๐‘–๐‘– with ๐‘›๐‘› different events.
0 โ‰ค ๐‘ƒ๐‘ƒ(๐ธ๐ธ๐‘–๐‘– ) โ‰ค 1
๐‘ƒ๐‘ƒ(ฮฉ) = 1 and ๐‘ƒ๐‘ƒ(๐œ™๐œ™) = 0
๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆช ๐ธ๐ธ2 ) = ๐‘ƒ๐‘ƒ(๐ธ๐ธ1 ) + ๐‘ƒ๐‘ƒ(๐ธ๐ธ2 )
When ๐ธ๐ธ1 and ๐ธ๐ธ2 are mutually exclusive.
Other representations of uncertainty exist such as fuzzy logic and theory of evidence
(Dempster-Shafer model) which do not follow the theory of probability but almost all
reliability concepts are defined based on probability as the metric of uncertainty. For a
justification of probability theory see (Singpurwalla 2006).
1.1.2. Interpretations of Probability
The two most common interpretations of probability are:
โ€ข
Frequency Interpretation. In the frequentist interpretation of probability, the
probability of an event (failure) is defined as:
๐‘ƒ๐‘ƒ(๐พ๐พ) = lim
๐‘›๐‘›โ†’โˆž
๐‘›๐‘›๐‘“๐‘“
๐‘›๐‘›
Also known as the classical approach, this interpretation assumes there exists a
real probability of an event, ๐‘๐‘. The analyst uses the observed frequency of the
event to estimate the value of ๐‘๐‘. The more historic events that have occurred,
the more confident the analyst is of the estimation of ๐‘๐‘. This approach does have
limitations, for instance when data from events are not available (e.g. no failures
occur in a test) ๐‘๐‘ cannot be estimated and this method cannot incorporate โ€œsoft
evidenceโ€ such as expert opinion.
โ€ข
Subjective Interpretation. The subjective interpretation of probability is also
known as the Bayesian school of thought. This method defines the probability of
an event as degree of belief the analyst has on the occurrence of event. This
means probability is a product of the analystโ€™s state of knowledge. Any evidence
which would change the analystโ€™s degree of belief must be considered when
calculating the probability (including soft evidence). The assumption is made that
the probability assessment is made by a coherent person where any coherent
person having the same state of knowledge would make the same assessment.
Introduction
3
1.1.3. Laws of Probability
The following rules of logic form the basis for many mathematical operations within the
theory of probability.
Let ๐‘‹๐‘‹ = ๐ธ๐ธ๐‘–๐‘– and ๐‘Œ๐‘Œ = ๐ธ๐ธ๐‘—๐‘— be two events within the sample space ฮฉ where ๐‘–๐‘– โ‰  ๐‘—๐‘—.
Boolean Laws of probability are (Modarres et al. 1999, p.25):
๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ = ๐‘Œ๐‘Œ โˆช ๐‘‹๐‘‹
๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ = ๐‘Œ๐‘Œ โˆฉ ๐‘‹๐‘‹
๐‘‹๐‘‹ โˆช (๐‘Œ๐‘Œ โˆช ๐‘๐‘) = (๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ) โˆช ๐‘๐‘
๐‘‹๐‘‹ โˆฉ (๐‘Œ๐‘Œ โˆฉ ๐‘๐‘) = (๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) โˆฉ ๐‘๐‘
๐‘‹๐‘‹ โˆฉ (๐‘Œ๐‘Œ โˆช ๐‘๐‘) = (๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) โˆช (๐‘‹๐‘‹ โˆฉ ๐‘๐‘)
๐‘‹๐‘‹ โˆช ๐‘‹๐‘‹ = ๐‘‹๐‘‹
๐‘‹๐‘‹ โˆฉ ๐‘‹๐‘‹ = ๐‘‹๐‘‹
๏ฟฝ=ฮฉ
XโˆชX
Xโˆฉ๏ฟฝ
X=Ø
๏ฟฝ
X=X
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
(๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ) = ๐‘‹๐‘‹๏ฟฝ โˆฉ ๐‘Œ๐‘Œ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) = ๐‘‹๐‘‹๏ฟฝ โˆช ๐‘Œ๐‘Œ๏ฟฝ
๐‘‹๐‘‹ โˆช (๐‘‹๐‘‹๏ฟฝ โˆฉ ๐‘Œ๐‘Œ) = ๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ
Two events are mutually exclusive if:
X โˆฉ Y = Ø,
Commutative Law
Associative Law
Distributive Law
Idempotent Law
Complementation Law
De Morganโ€™s Theorem
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) = 0
Two events are independent if one event ๐‘Œ๐‘Œ occurring does not affect the probability of the
second event ๐‘‹๐‘‹ occurring:
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹|๐‘Œ๐‘Œ) = ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹)
The rules for evaluating the probability of compound events are:
Addition Rule:
Multiplication Rule:
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ) = ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) + ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ) โˆ’ ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ)
= ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) + ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ) โˆ’ ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ|๐‘‹๐‘‹)
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) = ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ|๐‘‹๐‘‹) = ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ) ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹|๐‘Œ๐‘Œ)
When ๐‘‹๐‘‹ and ๐‘Œ๐‘Œ are independent:
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆช ๐‘Œ๐‘Œ) = ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) + ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ) โˆ’ ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ)
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) = ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ) ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ)
Prob Theory
The subjective interpretation has the flexibility of including many types of
evidence to assist in estimating the probability of an event. This is important in
many reliability applications where the events of interest (e. g, system failure) are
rare.
Prob Theory
4
Probability Distributions Used in Reliability Engineering
Generalizations of these equations:
๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆช ๐ธ๐ธ2 โˆช โ€ฆ โˆช ๐ธ๐ธ๐‘›๐‘› ) = [๐‘ƒ๐‘ƒ(๐ธ๐ธ1 ) + ๐‘ƒ๐‘ƒ(๐ธ๐ธ2 ) + โ‹ฏ + ๐‘ƒ๐‘ƒ(๐ธ๐ธ๐‘›๐‘› )]
โˆ’ [๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 ) + ๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ3 ) + โ‹ฏ + ๐‘ƒ๐‘ƒ(๐ธ๐ธ๐‘›๐‘›โˆ’1 โˆฉ ๐ธ๐ธ๐‘›๐‘› )]
+ [๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 โˆฉ ๐ธ๐ธ3 ) + ๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 โˆฉ ๐ธ๐ธ4 ) + โ‹ฏ ]
โˆ’ โ‹ฏ (โˆ’1)๐‘›๐‘›+1 [๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 โˆฉ โ€ฆ โˆฉ ๐ธ๐ธ๐‘›๐‘› )]
๐‘ƒ๐‘ƒ(๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 โˆฉ โ€ฆ โˆฉ ๐ธ๐ธ๐‘›๐‘› ) = ๐‘ƒ๐‘ƒ(๐ธ๐ธ1 ) . ๐‘ƒ๐‘ƒ(๐ธ๐ธ2 |๐ธ๐ธ1 ). ๐‘ƒ๐‘ƒ(๐ธ๐ธ3 |๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 )
โ€ฆ ๐‘ƒ๐‘ƒ(๐ธ๐ธ๐‘›๐‘› |๐ธ๐ธ1 โˆฉ ๐ธ๐ธ2 โˆฉ โ€ฆ โˆฉ ๐ธ๐ธ๐‘›๐‘›โˆ’1 )
1.1.4. Law of Total Probability
The probability of ๐‘‹๐‘‹ can be obtained by the following summation;
๐‘›๐‘›๐ด๐ด
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) = ๏ฟฝ ๐‘ƒ๐‘ƒ(๐ด๐ด๐‘–๐‘– )๐‘ƒ๐‘ƒ(๐‘‹๐‘‹|๐ด๐ด๐‘–๐‘– )
๐‘–๐‘–=1
where ๐ด๐ด = {๐ด๐ด1 , ๐ด๐ด2 , โ€ฆ , ๐ด๐ด๐‘›๐‘›๐ด๐ด } is a partition of the sample space, ฮฉ, and all the elements of A
are mutually exclusive, Ai โˆฉ Aj = Ø, and the union of all A elements cover the complete
๐‘›๐‘›๐ด๐ด
๐ด๐ด๐‘–๐‘– = ฮฉ.
sample space, โˆช๐‘–๐‘–=1
For example:
1.1.5. Bayesโ€™ Law
๐‘ƒ๐‘ƒ(๐‘‹๐‘‹) = ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ) + ๐‘ƒ๐‘ƒ(๐‘‹๐‘‹ โˆฉ ๐‘Œ๐‘Œ๏ฟฝ)
= ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ)๐‘ƒ๐‘ƒ(๐‘‹๐‘‹|๐‘Œ๐‘Œ) + ๐‘ƒ๐‘ƒ(๐‘Œ๐‘Œ)๐‘ƒ๐‘ƒ(๐‘‹๐‘‹|๐‘Œ๐‘Œ๏ฟฝ)
Bayesโ€™ law, can be derived from the multiplication rule and the law of total probability as
follows:
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ) ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ) = ๐‘ƒ๐‘ƒ(๐ธ๐ธ) ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ|๐ธ๐ธ)
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ|๐ธ๐ธ) =
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ|๐ธ๐ธ) =
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ) ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ)
๐‘ƒ๐‘ƒ(๐ธ๐ธ)
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ) ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ)
โˆ‘๐‘–๐‘– ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ๐‘–๐‘– )๐‘ƒ๐‘ƒ(๐œƒ๐œƒ๐‘–๐‘– )
๐œƒ๐œƒ
the unknown of interest (UOI).
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ)
the prior state of knowledge about ๐œƒ๐œƒ without the evidence. Also denoted as
๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ).
๐ธ๐ธ
๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ)
the observed random variable, evidence.
the likelihood of observing the evidence given the UOI. Also denoted as
๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ).
Introduction
the posterior state of knowledge about ๐œƒ๐œƒ given the evidence. Also denoted
as ๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ).
โˆ‘๐‘–๐‘– ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ๐‘–๐‘– )๐‘ƒ๐‘ƒ(๐œƒ๐œƒ) is the normalizing constant.
Thus Bayes formula enables us to use a piece of evidence , ๐ธ๐ธ, to make inference about
the unobserved ๐œƒ๐œƒ .
The continuous form of Bayesโ€™ Law can be written as:
๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ) ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ)
๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ) =
โˆซ ๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ) ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ) ๐‘‘๐‘‘๐‘‘๐‘‘
In Bayesian statistics the state of knowledge (uncertainty) of an unknown of interest is
quantified by assigning a probability distribution to its possible values. Bayesโ€™ law provides
a mathematical means by which this uncertainty can be updated given new evidence.
1.1.6. Likelihood Functions
The likelihood function is the probability of observing the evidence (e.g., sample data), ๐ธ๐ธ,
given the distribution parameters, ๐œƒ๐œƒ. The probability of observing events is the product of
each event likelihood:
๐ฟ๐ฟ(๐œƒ๐œƒ|๐ธ๐ธ) = ๐‘๐‘ ๏ฟฝ ๐ฟ๐ฟ(๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– )
๐‘–๐‘–
๐‘๐‘ is a combinatorial constant which quantifies the number of combination which the
observed evidence could have occurred. Methods which use the likelihood function in
parameter estimation do not depend on the constant and so it is omitted.
The following table summarizes the likelihood functions for different types of observations:
Table 1: Summary of Likelihood Functions (Klein & Moeschberger 2003, p.74)
Type of Observation
Exact Lifetimes
Right Censored
Left Censored
Interval Censored
Left Truncated
Right Truncated
Interval Truncated
Likelihood Function
Example Description
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) = ๐‘“๐‘“(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
Failure time is known
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) = ๐น๐น(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
Component failed before time ๐‘ก๐‘ก๐‘–๐‘–
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) = ๐‘…๐‘…(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) = ๐น๐น(๐‘ก๐‘ก๐‘–๐‘–๐‘…๐‘…๐‘…๐‘… |๐œƒ๐œƒ) โˆ’ ๐น๐น(๐‘ก๐‘ก๐‘–๐‘–๐ฟ๐ฟ๐ฟ๐ฟ |๐œƒ๐œƒ)
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) =
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) =
๐ฟ๐ฟ๐‘–๐‘– (๐œƒ๐œƒ|๐‘ก๐‘ก๐‘–๐‘– ) =
๐‘“๐‘“(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
๐‘…๐‘…(๐‘ก๐‘ก๐ฟ๐ฟ |๐œƒ๐œƒ)
๐‘“๐‘“(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
๐น๐น(๐‘ก๐‘ก๐‘ˆ๐‘ˆ |๐œƒ๐œƒ)
๐‘“๐‘“(๐‘ก๐‘ก๐‘–๐‘– |๐œƒ๐œƒ)
๐น๐น(๐‘ก๐‘ก๐‘ˆ๐‘ˆ |๐œƒ๐œƒ) โˆ’ ๐น๐น(๐‘ก๐‘ก๐ฟ๐ฟ |๐œƒ๐œƒ)
Component survived to time ๐‘ก๐‘ก๐‘–๐‘–
Component failed between
๐‘ก๐‘ก๐‘–๐‘–๐ฟ๐ฟ๐ฟ๐ฟ and ๐‘ก๐‘ก๐‘–๐‘–๐‘…๐‘…๐‘…๐‘…
Component failed at time ๐‘ก๐‘ก๐‘–๐‘– where
observations are truncated before ๐‘ก๐‘ก๐ฟ๐ฟ .
Component failed at time ๐‘ก๐‘ก๐‘–๐‘– where
observations are truncated after ๐‘ก๐‘ก๐‘ˆ๐‘ˆ .
Component failed at time ๐‘ก๐‘ก๐‘–๐‘– where
observations are truncated before ๐‘ก๐‘ก๐ฟ๐ฟ
and after ๐‘ก๐‘ก๐‘ˆ๐‘ˆ .
Prob Theory
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ|๐ธ๐ธ)
5
Prob Theory
6
Probability Distributions Used in Reliability Engineering
The Likelihood function is used in Bayesian inference and maximum likelihood parameter
estimation techniques. In both instances any constant in front of the likelihood function
becomes irrelevant. Such constants are therefore not included in the likelihood functions
given in this book (nor in most references).
For example, consider the case where a test is conducted on ๐‘›๐‘› components with an
exponential time to failure distribution. The test is terminated at ๐‘ก๐‘ก๐‘ ๐‘  during which ๐‘Ÿ๐‘Ÿ
components failed at times ๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘Ÿ๐‘Ÿ and ๐‘ ๐‘  = ๐‘›๐‘› โˆ’ ๐‘Ÿ๐‘Ÿ components survived. Using the
exponential distribution to construct the likelihood function we obtain:
๐ฟ๐ฟ(๐œ†๐œ†|๐ธ๐ธ) =
=
๐‘›๐‘›๐‘†๐‘†
๐‘›๐‘›๐น๐น
๏ฟฝ ๐‘“๐‘“๏ฟฝ๐œ†๐œ†๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ ๏ฟฝ ๐‘…๐‘…๏ฟฝ๐œ†๐œ†๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐‘†๐‘† ๏ฟฝ
๐‘–๐‘–=1
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
๐น๐น
๏ฟฝ ๐œ†๐œ†๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐‘†๐‘†
๐‘†๐‘†
๏ฟฝ ๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–=1
๐‘›๐‘›๐‘†๐‘†
๐น๐น
๐‘†๐‘†
= ๐œ†๐œ†๐‘›๐‘›๐น๐น ๐‘’๐‘’ โˆ’๐œ†๐œ† โˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘– ๐‘’๐‘’ โˆ’ฮป โˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘–
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐‘†๐‘†
๐น๐น
๐‘†๐‘†
= ๐œ†๐œ†๐‘›๐‘›๐น๐น ๐‘’๐‘’ โˆ’๐œ†๐œ†๏ฟฝโˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘– +โˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘– ๏ฟฝ
Alternatively, because the test described is a homogeneous Poisson process 1 the
likelihood function could also have been constructed using a Poisson distribution. The
data can be stated as seeing r failure in time ๐‘ก๐‘ก ๐‘‡๐‘‡ where ๐‘ก๐‘ก ๐‘‡๐‘‡ is the total time on test ๐‘ก๐‘ก ๐‘‡๐‘‡ =
๐‘›๐‘›๐‘†๐‘† ๐‘†๐‘†
๐น๐น
โˆ‘๐‘›๐‘›๐‘–๐‘–=1
๐‘ก๐‘ก๐‘–๐‘–๐น๐น + โˆ‘๐‘–๐‘–=1
๐‘ก๐‘ก๐‘–๐‘– . Therefore the likelihood function would be:
๐ฟ๐ฟ(๐œ†๐œ†|๐ธ๐ธ) = ๐‘“๐‘“(๐œ†๐œ†|๐‘›๐‘›๐น๐น , ๐‘ก๐‘ก ๐‘‡๐‘‡ )
=
(๐œ†๐œ†๐‘ก๐‘ก ๐‘‡๐‘‡ )๐‘›๐‘›๐น๐น โˆ’๐œ†๐œ†๐‘ก๐‘ก
๐‘’๐‘’ ๐‘‡๐‘‡
๐‘›๐‘›๐น๐น !
= ๐‘๐‘๐œ†๐œ†๐‘›๐‘›๐น๐น ๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘‡๐‘‡
๐‘›๐‘›๐น๐น
๐น๐น
๐‘›๐‘›๐‘†๐‘†
๐‘†๐‘†
= ๐œ†๐œ†๐‘›๐‘›๐น๐น ๐‘’๐‘’ โˆ’๐œ†๐œ†๏ฟฝโˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘– +โˆ‘๐‘–๐‘–=1 ๐‘ก๐‘ก๐‘–๐‘– ๏ฟฝ
As mentioned earlier, in estimation procedures within this book, the constant ๐‘๐‘ can be
ignored. As such, the two likelihood functions are equal. For more information see (Meeker
& Escobar 1998, p.36) or (Rinne 2008, p.403).
1.1.7. Fisher Information Matrix
The Fisher Information Matrix has many uses but in reliability applications it is most often
used to create Jefferyโ€™s non-informative priors. There are two types of Fisher information
matrices, the Expected Fisher Information Matrix ๐ผ๐ผ(๐œƒ๐œƒ), and the Observed Fisher
Information Matrix ๐ฝ๐ฝ(๐œƒ๐œƒ).
1 Homogeneous in time, where it does not matter if you have ๐‘›๐‘› components on test at
once (exponential test), or you have a single component on test which is replaced after
failure ๐‘›๐‘› times (Poisson process), the evidence produced will be the same.
Introduction
7
For a single parameter distribution:
๐ผ๐ผ(๐œƒ๐œƒ) = โˆ’๐ธ๐ธ ๏ฟฝ
2
๐œ•๐œ•ฮ›(๐œƒ๐œƒ|๐‘ฅ๐‘ฅ)
๐œ•๐œ• 2 ฮ›(๐œƒ๐œƒ|๐‘ฅ๐‘ฅ)
๏ฟฝ = ๏ฟฝ๏ฟฝ
๏ฟฝ ๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ 2
๐œ•๐œ•๐œ•๐œ•
where ฮ› is the log-likelihood function and ๐ธ๐ธ[๐‘ˆ๐‘ˆ] = โˆซ ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘. For a distribution with ๐‘๐‘
parameters the Expected Fisher Information Matrix is:
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
โŽกโˆ’๐ธ๐ธ ๏ฟฝ
๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ12
โŽข
โŽข
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
๏ฟฝ
โˆ’๐ธ๐ธ ๏ฟฝ
๐ผ๐ผ(๐œฝ๐œฝ) = โŽข
๐œ•๐œ•๐œƒ๐œƒ2 ๐œ•๐œ•๐œƒ๐œƒ1
โŽข
โ‹ฎ
โŽข
2
โŽขโˆ’๐ธ๐ธ ๏ฟฝ๐œ•๐œ• ฮ›(๐œฝ๐œฝ|๐’™๐’™)๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ ๐œ•๐œ•๐œƒ๐œƒ1
โŽฃ
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ1 ๐œ•๐œ•๐œƒ๐œƒ2
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
โˆ’๐ธ๐ธ ๏ฟฝ
๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ22
โ‹ฎ
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
โˆ’๐ธ๐ธ ๏ฟฝ
๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ ๐œ•๐œ•๐œƒ๐œƒ2
โˆ’๐ธ๐ธ ๏ฟฝ
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™)
๏ฟฝโŽค
๐œ•๐œ•๐œƒ๐œƒ1 ๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ โŽฅ
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™) โŽฅ
๏ฟฝ
โ‹ฏ โˆ’๐ธ๐ธ ๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ2 ๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ โŽฅ
โŽฅ
โ‹ฑ
โ‹ฎ
2 ฮ›(๐œฝ๐œฝ|๐’™๐’™) โŽฅ
๐œ•๐œ•
๏ฟฝโŽฅ
โ€ฆ โˆ’๐ธ๐ธ ๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘2
โŽฆ
โ‹ฏ โˆ’๐ธ๐ธ ๏ฟฝ
The Observed Fisher Information Matrix is obtained from a likelihood function constructed
from ๐‘›๐‘› observed samples from the distribution. The expectation term is dropped.
For a single parameter distribution:
๐‘›๐‘›
๐ฝ๐ฝ๐‘›๐‘› (๐œƒ๐œƒ) = โˆ’ ๏ฟฝ
๐‘–๐‘–=1
๐œ•๐œ• 2 ฮ›(๐œƒ๐œƒ|๐‘ฅ๐‘ฅ๐‘–๐‘– )
๐œ•๐œ•๐œƒ๐œƒ 2
For a distribution with ๐‘๐‘ parameters the Observed Fisher Information Matrix is:
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š )
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š )
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š )
โŽกโˆ’
โŽค
โ‹ฏ
โˆ’
โˆ’
๐œ•๐œ•๐œƒ๐œƒ1 ๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ โŽฅ
๐œ•๐œ•๐œƒ๐œƒ1 ๐œ•๐œ•๐œƒ๐œƒ2
๐œ•๐œ•๐œƒ๐œƒ12
โŽข
๐‘›๐‘› โŽข ๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™ )
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š )
๐œ•๐œ• 2 ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š )โŽฅ
๐’Š๐’Š
โˆ’
โˆ’
โ‹ฏ
โˆ’
โŽข
2
๐ฝ๐ฝ๐‘›๐‘› (๐œฝ๐œฝ) = ๏ฟฝ
๐œ•๐œ•๐œƒ๐œƒ2 ๐œ•๐œ•๐œƒ๐œƒ1
๐œ•๐œ•๐œƒ๐œƒ2 ๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ โŽฅ
๐œ•๐œ•๐œƒ๐œƒ2
โŽข
โŽฅ
๐‘–๐‘–=1
โ‹ฑ
โ‹ฎ
โ‹ฎ
โ‹ฎ
โŽข
2 ฮ›(๐œฝ๐œฝ|๐’™๐’™ ) โŽฅ
2 ฮ›(๐œฝ๐œฝ|๐’™๐’™ )
2
)
๐œ•๐œ•
๐œ•๐œ• ฮ›(๐œฝ๐œฝ|๐’™๐’™๐’Š๐’Š
๐’Š๐’Š โŽฅ
๐’Š๐’Š
โŽขโˆ’ ๐œ•๐œ•
โ€ฆ โˆ’
โˆ’
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ ๐œ•๐œ•๐œƒ๐œƒ1
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘ ๐œ•๐œ•๐œƒ๐œƒ2
๐œ•๐œ•๐œƒ๐œƒ๐‘๐‘2 โŽฆ
โŽฃ
It can be seen that as ๐‘›๐‘› becomes large, the average value of the random variable
approaches its expected value and so the following asymptotic relationship exists
between the observed and expected Fisher information matrices:
plim
๐‘›๐‘›โ†’โˆž
1
๐ฝ๐ฝ (๐œฝ๐œฝ) = ๐ผ๐ผ(๐œฝ๐œฝ)
๐‘›๐‘› ๐‘›๐‘›
For large n the following approximation can be used:
๐ฝ๐ฝ๐‘›๐‘› โ‰ˆ ๐‘›๐‘›๐‘›๐‘›(๐œฝ๐œฝ)
Prob Theory
The Expected Fisher Information Matrix is obtained from a log-likelihood function from a
single random variable. The random variable is replaced by its expected value.
Prob Theory
8
Probability Distributions Used in Reliability Engineering
๏ฟฝ the observed Fisher information matrix estimates the varianceWhen evaluated at ๐œฝ๐œฝ = ๐œฝ๐œฝ
covariance matrix:
๏ฟฝ )๏ฟฝ
๐‘ฝ๐‘ฝ = ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› (๐œฝ๐œฝ = ๐œฝ๐œฝ
โˆ’1
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐œƒ๐œƒ๏ฟฝ1 )
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ๏ฟฝ1 , ๐œƒ๐œƒ๏ฟฝ2 ) โ‹ฏ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ๏ฟฝ1 , ๐œƒ๐œƒ๏ฟฝ๐‘‘๐‘‘ )
โŽก
โŽค
๏ฟฝ
๏ฟฝ
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ
โ‹ฏ ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ๏ฟฝ2 , ๐œƒ๐œƒ๏ฟฝ๐‘‘๐‘‘ )โŽฅ
,
๐œƒ๐œƒ
)
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐œƒ๐œƒ๏ฟฝ2 )
1 2
=โŽข
โ‹ฎ
โ‹ฎ
โ‹ฑ
โ‹ฎ
โŽข
โŽฅ
โ€ฆ ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐œƒ๐œƒ๏ฟฝ๐‘‘๐‘‘ ) โŽฆ
โŽฃ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ๏ฟฝ1 , ๐œƒ๐œƒ๏ฟฝ๐‘‘๐‘‘ ) ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐œƒ๐œƒ๏ฟฝ2 , ๐œƒ๐œƒ๏ฟฝ๐‘‘๐‘‘ )
9
1.2. Distribution Functions
1.2.1. Random Variables
Probability distributions are used to model random events for which the outcome is
uncertain such as the time of failure for a component. Before placing a demand on that
component, the time it will fail is unknown. The distribution of the probability of failure at
different times is modeled by a probability distribution. In this book random variables will
be denoted as capital letter such as ๐‘‡๐‘‡ for time. When the random variable assumes a
value we denote this by small caps such as ๐‘ก๐‘ก for time. For example, if we wish to find the
probability that the component fails before time ๐‘ก๐‘ก1 we would find ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก1 ).
Random variables are classified as either discrete or continuous. In a discrete distribution,
the random variable can take on a distinct or countable number of possible values such
as number of demands to failure. In a continuous distribution the random variable is not
constrained to distinct possible values such as time-to-failure distribution.
This book will denote continuous random variables as ๐‘‹๐‘‹ or ๐‘‡๐‘‡, and discrete random
variables as ๐พ๐พ.
1.2.2. Statistical Distribution Parameters
The parameters of a distribution are the variables which need to be specified in order to
completely specify the distribution. Often parameters are classified by the effect they have
on the distributions. Shape parameters define the shape of the distribution, scale
parameters stretch the distribution along the random variable axis, and location
parameters shift the distribution along the random variable axis. The reader is cautioned
that the parameters for a distribution may change depending on the text. Therefore, before
using formulas from other sources the parameterization need to be confirmed.
Understanding the effect of changing a distributionโ€™s parameter value can be a difficult
task. At the beginning of each section a graph of the distribution is shown with varied
parameters.
1.2.3. Probability Density Function
A probability density function (pdf), denoted as ๐‘“๐‘“(๐‘ก๐‘ก) is any function which is always
positive and has a unit area:
โˆž
๏ฟฝ ๐‘“๐‘“(๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘ = 1,
โˆ’โˆž
๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜) = 1
๐‘˜๐‘˜
The probability of an event occurring between limits a and b is the area under the pdf:
๐‘๐‘
๐‘ƒ๐‘ƒ(๐‘Ž๐‘Ž โ‰ค ๐‘‡๐‘‡ โ‰ค ๐‘๐‘) = ๏ฟฝ ๐‘“๐‘“(๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘ = ๐น๐น(๐‘๐‘) โˆ’ ๐น๐น(๐‘Ž๐‘Ž)
๐‘Ž๐‘Ž
Dist Functions
Introduction
Dist Functions
10
Probability Distributions Used in Reliability Engineering
๐‘๐‘
๐‘ƒ๐‘ƒ(๐‘Ž๐‘Ž โ‰ค ๐พ๐พ โ‰ค ๐‘๐‘) = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜) = ๐น๐น(๐‘๐‘) โˆ’ ๐น๐น(๐‘Ž๐‘Ž โˆ’ 1)
๐‘–๐‘–=๐‘Ž๐‘Ž
The instantaneous value of a discrete pdf at ๐‘˜๐‘˜๐‘–๐‘– can be obtained by minimizing the limits
to [๐‘˜๐‘˜๐‘–๐‘–โˆ’1 , ๐‘˜๐‘˜๐‘–๐‘– ]:
๐‘ƒ๐‘ƒ(๐พ๐พ = ๐‘˜๐‘˜๐‘–๐‘– ) = ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜๐‘–๐‘– < ๐พ๐พ โ‰ค ๐‘˜๐‘˜๐‘–๐‘– ) = ๐‘“๐‘“(๐‘˜๐‘˜)
The instantaneous value of a continuous pdf is infinitesimal. This result can be seen when
minimizing the limits to [๐‘ก๐‘ก, ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก]:
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ = ๐‘ก๐‘ก) = lim ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก < ๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก) = lim ๐‘“๐‘“(๐‘ก๐‘ก). ฮ”๐‘ก๐‘ก
ฮ”tโ†’0
ฮ”tโ†’0
Therefore the reader must remember that in order to calculate the probability of an event,
an interval for the random variable must be used. Furthermore, a common
misunderstanding is that a pdf cannot have a value above one because the probability of
an event occurring cannot be greater than one. As can be seen above this is true for
discrete distributions, only because ฮ”๐‘˜๐‘˜ = 1. However for continuous the case the pdf is
multiplied by a small interval ฮ”๐‘ก๐‘ก, which ensures that the probability an event occurring
within the interval ฮ”๐‘ก๐‘ก is less than one.
๐‘“๐‘“(๐‘ก๐‘ก)
๐‘“๐‘“(๐‘˜๐‘˜)
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘ƒ๐‘ƒ(๐‘˜๐‘˜1 < ๐พ๐พ โ‰ค ๐‘˜๐‘˜2 )
๐‘ƒ๐‘ƒ(๐‘ก๐‘ก1 < ๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก2 )
๐‘ก๐‘ก1
๐‘ก๐‘ก2
๐‘ก๐‘ก
๐‘˜๐‘˜
๐‘˜๐‘˜1 ๐‘˜๐‘˜2
Figure 1: Left: continuous pdf, right: discrete pdf
To derive the continuous pdf relationship to the cumulative density function (cdf), ๐น๐น(๐‘ก๐‘ก):
lim ๐‘“๐‘“(๐‘ก๐‘ก). ฮ”๐‘ก๐‘ก = lim ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก < ๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก) = lim ๐น๐น(๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก) โˆ’ ๐น๐น(๐‘ก๐‘ก) = lim ฮ”๐น๐น(๐‘ก๐‘ก)
ฮ”tโ†’0
ฮ”tโ†’0
ฮ”tโ†’0
๐‘“๐‘“(๐‘ก๐‘ก) = lim
ฮ”tโ†’0
ฮ”๐น๐น(๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘(๐‘ก๐‘ก)
=
ฮ”๐‘ก๐‘ก
๐‘‘๐‘‘๐‘‘๐‘‘
ฮ”tโ†’0
The shape of the pdf can be obtained by plotting a normalized histogram of an infinite
number of samples from a distribution.
It should be noted when plotting a discrete pdf the points from each discrete value should
not be joined. For ease of explanation using the area under the graph argument the step
11
plot is intuitive but implies a non-integer random variable. Instead stem plots or column
plots are often used.
๐‘“๐‘“(๐‘˜๐‘˜)
๐‘“๐‘“(๐‘˜๐‘˜)
๐‘˜๐‘˜
Figure 2: Discrete data plotting. Left stem plot. Right column plot.
๐‘˜๐‘˜
1.2.4. Cumulative Distribution Function
The cumulative density function (cdf), denoted by ๐น๐น(๐‘ก๐‘ก) is the probability of the random
event occurring before ๐‘ก๐‘ก, ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก). For a discrete cdf the height of each step is the pdf
value ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ).
๐‘ก๐‘ก
๐น๐น(๐‘ก๐‘ก) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก) = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
โˆ’โˆž
๐น๐น(๐‘˜๐‘˜) = ๐‘ƒ๐‘ƒ(๐พ๐พ โ‰ค ๐‘˜๐‘˜) = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– )
The limits of the cdf for โˆ’โˆž < ๐‘ก๐‘ก < โˆž and 0 โ‰ค ๐‘˜๐‘˜ โ‰ค โˆž are given as:
lim ๐น๐น(๐‘ก๐‘ก) = 0 ,
๐‘ก๐‘กโ†’โˆ’โˆž
lim ๐น๐น(๐‘ก๐‘ก) = 1 ,
๐‘ก๐‘กโ†’โˆž
๐‘˜๐‘˜๐‘–๐‘– โ‰ค๐‘˜๐‘˜
๐น๐น(โˆ’1) = 0
lim ๐น๐น(๐‘˜๐‘˜) = 1
๐‘˜๐‘˜โ†’โˆž
The cdf can be used to find the probability of the random even occurring between two
limits:
๐‘๐‘
๐‘ƒ๐‘ƒ(๐‘Ž๐‘Ž โ‰ค ๐‘‡๐‘‡ โ‰ค ๐‘๐‘) = ๏ฟฝ ๐‘“๐‘“(๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘ = ๐น๐น(๐‘๐‘) โˆ’ ๐น๐น(๐‘Ž๐‘Ž)
๐‘Ž๐‘Ž
๐‘๐‘
๐‘ƒ๐‘ƒ(๐‘Ž๐‘Ž โ‰ค ๐พ๐พ โ‰ค ๐‘๐‘) = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜) = ๐น๐น(๐‘๐‘) โˆ’ ๐น๐น(๐‘Ž๐‘Ž โˆ’ 1)
๐‘–๐‘–=๐‘Ž๐‘Ž
Dist Functions
Introduction
Dist Functions
12
Probability Distributions Used in Reliability Engineering
1
๐น๐น(๐‘ก๐‘ก)
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก1 )
๐น๐น(๐‘ก๐‘ก1 )
๐‘“๐‘“(๐‘ก๐‘ก)
๐น๐น(๐‘˜๐‘˜)
1
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘ƒ๐‘ƒ(๐พ๐พ โ‰ค ๐‘˜๐‘˜1 )
๐น๐น(๐‘˜๐‘˜1 )
๐‘ก๐‘ก
๐‘ก๐‘ก1
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘˜๐‘˜
๐‘˜๐‘˜1
๐‘“๐‘“(๐‘˜๐‘˜)
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘ƒ๐‘ƒ(๐พ๐พ โ‰ค ๐‘˜๐‘˜1 )
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ค ๐‘ก๐‘ก1 )
๐‘๐‘๐‘๐‘๐‘๐‘
๐‘ก๐‘ก
๐‘ก๐‘ก1
๐‘˜๐‘˜1
๐‘˜๐‘˜
Figure 3: Left: continuous cdf/pdf, right: discrete cdf/pdf
1.2.5. Reliability Function
The reliability function, also known as the survival function, is denoted as ๐‘…๐‘…(๐‘ก๐‘ก). It is the
probability that the random event (time of failure) occurs after ๐‘ก๐‘ก.
๐‘…๐‘…(๐‘ก๐‘ก) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ก๐‘ก) = 1 โˆ’ ๐น๐น(๐‘ก๐‘ก),
โˆž
๐‘…๐‘…(๐‘ก๐‘ก) = ๏ฟฝ ๐‘“๐‘“(๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
๐‘ก๐‘ก
๐‘…๐‘…(๐‘˜๐‘˜) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘˜๐‘˜) = 1 โˆ’ ๐น๐น(๐‘˜๐‘˜)
โˆž
๐‘…๐‘…(๐‘˜๐‘˜) = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– )
๐‘–๐‘–=๐‘˜๐‘˜+1
It should be noted that in most publications the discrete reliability function is defined as
โˆ—
๐‘…๐‘…โˆ— (๐‘˜๐‘˜) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ โ‰ฅ ๐‘˜๐‘˜) = โˆ‘โˆž
๐‘–๐‘–=๐‘˜๐‘˜ ๐‘“๐‘“(๐‘˜๐‘˜). This definition results in ๐‘…๐‘… (๐‘˜๐‘˜) โ‰  1 โˆ’ ๐น๐น(๐‘˜๐‘˜). Despite this
problem it is the most common definition and is included in all the references in this book
except (Xie, Gaudoin, et al. 2002)
1
๐น๐น(๐‘ก๐‘ก)
1
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘  ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“
๐‘๐‘๐‘๐‘๐‘๐‘
0
๐‘ก๐‘ก
13
0
๐‘ก๐‘ก
Figure 4: Left continuous cdf, right continuous survival function
1.2.6. Conditional Reliability Function
The conditional reliability function, denoted as ๐‘š๐‘š(๐‘ฅ๐‘ฅ) is the probability of the component
surviving given that it has survived to time t.
๐‘…๐‘…(๐‘ก๐‘ก + x)
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
๐‘…๐‘…(๐‘ก๐‘ก)
Where:
๐‘ก๐‘ก is the given time for which we know the component survived.
๐‘ฅ๐‘ฅ is new random variable defined as the time after ๐‘ก๐‘ก. ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
1.2.7. 100ฮฑ% Percentile Function
The 100ฮฑ% percentile function is the interval [0, ๐‘ก๐‘ก๐›ผ๐›ผ ] for which the area under the pdf is ๐›ผ๐›ผ.
๐‘ก๐‘ก๐›ผ๐›ผ = ๐น๐น โˆ’1 (๐›ผ๐›ผ)
1.2.8. Mean Residual Life
The mean residual life (MRL), denoted as ๐‘ข๐‘ข(๐‘ก๐‘ก), is the expected life given the component
has survived to time, ๐‘ก๐‘ก.
โˆž
1.2.9. Hazard Rate
๐‘ข๐‘ข(๐‘ก๐‘ก) = ๏ฟฝ ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘ =
0
โˆž
1
๏ฟฝ ๐‘…๐‘…(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘…๐‘…(๐‘ก๐‘ก) ๐‘ก๐‘ก
The hazard function, denoted as h(t), is the conditional probability that a component fails
in a small time interval, given that it has survived from time zero until the beginning of the
time interval. For the continuous case the probability that an item will fail in a time interval
given the item was functioning at time ๐‘ก๐‘ก is:
Dist Functions
Introduction
Dist Functions
14
Probability Distributions Used in Reliability Engineering
๐‘ƒ๐‘ƒ(๐‘ก๐‘ก < ๐‘‡๐‘‡ < ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก) =
๐‘ƒ๐‘ƒ(๐‘ก๐‘ก < ๐‘‡๐‘‡ < ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก) ๐น๐น(๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก) โˆ’ ๐น๐น(๐‘ก๐‘ก) ฮ”๐น๐น(๐‘ก๐‘ก)
=
=
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
By dividing the probability by ฮ”๐‘ก๐‘ก and finding the limit as ฮ”๐‘ก๐‘ก โ†’ 0 gives the hazard rate:
โ„Ž(๐‘ก๐‘ก) = lim
ฮ”๐‘ก๐‘กโ†’0
๐‘“๐‘“(๐‘ก๐‘ก)
๐‘ƒ๐‘ƒ(๐‘ก๐‘ก < ๐‘‡๐‘‡ < ๐‘ก๐‘ก + ฮ”๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
ฮ”๐น๐น(๐‘ก๐‘ก)
= lim
=
ฮ”๐‘ก๐‘กโ†’0 ฮ”๐‘ก๐‘ก๐‘ก๐‘ก(๐‘ก๐‘ก)
ฮ”๐‘ก๐‘ก
๐‘…๐‘…(๐‘ก๐‘ก)
The discrete hazard rate is defined as: (Xie, Gaudoin, et al. 2002)
โ„Ž(๐‘˜๐‘˜) =
๐‘ƒ๐‘ƒ(๐พ๐พ = ๐‘˜๐‘˜)
๐‘“๐‘“(๐‘˜๐‘˜)
=
๐‘ƒ๐‘ƒ(๐พ๐พ โ‰ฅ ๐‘˜๐‘˜) ๐‘…๐‘…(๐‘˜๐‘˜ โˆ’ 1)
This unintuitive result is due to a popular definition of ๐‘…๐‘…โˆ— (๐‘˜๐‘˜) = โˆ‘โˆž
๐‘–๐‘–=๐‘˜๐‘˜ ๐‘“๐‘“(๐‘˜๐‘˜) in which case
โ„Ž(๐‘˜๐‘˜) = ๐‘“๐‘“(๐‘˜๐‘˜)/๐‘…๐‘… โˆ— (๐‘˜๐‘˜). This definition has been avoided because it violates the formula
๐‘…๐‘…(๐‘˜๐‘˜) = 1 โˆ’ ๐น๐น(๐‘˜๐‘˜). The discrete hazard rate cannot be used in the same way as a
continuous hazard rate with the following differences (Xie, Gaudoin, et al. 2002):
โ€ข
โ€ข
โ€ข
โ€ข
โ„Ž(๐‘˜๐‘˜) is defined as a probability and so is bounded by [0,1].
โ„Ž(๐‘˜๐‘˜) is not additive for series systems.
For the cumulative hazard rate ๐ป๐ป(๐‘˜๐‘˜) = โˆ’ ln[๐‘…๐‘…(๐‘˜๐‘˜)] โ‰  โˆ‘๐‘˜๐‘˜๐‘–๐‘–=0 โ„Ž(๐‘˜๐‘˜)
When a set of data is analyzed using a discrete counterpart of the continuous
distribution the values of the hazard rate do not converge.
A function called the second failure rate has been proposed (Gupta et al. 1997):
๐‘Ÿ๐‘Ÿ(๐‘˜๐‘˜) = ln
๐‘…๐‘…(๐‘˜๐‘˜ โˆ’ 1)
= โˆ’ ln[1 โˆ’ โ„Ž(๐‘˜๐‘˜)]
๐‘…๐‘…(๐‘˜๐‘˜)
This function overcomes the previously mentioned limitations of the discrete hazard rate
function and maintains the monotonicity property. For more information, the reader is
referred to (Xie, Gaudoin, et al. 2002)
Care should be taken not to confuse the hazard rate with the Rate of Occurrence of
Failures (ROCOF). ROCOF is the probability that a failure (not necessarily the first) occurs
in a small time interval. Unlike the hazard rate, the ROCOF is the absolute rate at which
system failures occur and is not conditional on survival to time t. ROCOF is using in
measuring the change in the rate of failures for repairable systems.
1.2.10.
Cumulative Hazard Rate
The cumulative hazard rate, denoted as ๐ป๐ป(๐‘ก๐‘ก) an in the continuous case is the area under
the hazard rate function. This function is useful to calculate average failure rates.
๐‘ก๐‘ก
๐ป๐ป(๐‘ก๐‘ก) = ๏ฟฝ โ„Ž(๐‘ข๐‘ข)๐‘‘๐‘‘๐‘‘๐‘‘ = โˆ’ln[๐‘…๐‘…(๐‘ก๐‘ก)]
โˆž
๐ป๐ป(๐‘˜๐‘˜) = โˆ’ ln[๐‘…๐‘…(๐‘˜๐‘˜)]
For a discussion on the discrete cumulative hazard rate see hazard rate.
1.2.11.
15
Characteristic Function
The characteristic function of a random variable completely defines its probability
distribution. It can be used to derive properties of the distribution from transformations of
the random variable. (Billingsley 1995)
The characteristic function is defined as the expected value of the function exp(๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–) where
๐‘ฅ๐‘ฅ is the random variable of the distribution with a cdf ๐น๐น(๐‘ฅ๐‘ฅ), ๐œ”๐œ” is a parameter that can have
any real value and ๐‘–๐‘– = โˆšโˆ’1:
๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”) = ๐ธ๐ธ๏ฟฝ๐‘’๐‘’ ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘– ๏ฟฝ
โˆž
= ๏ฟฝ ๐‘’๐‘’ ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘– ๐น๐น(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘
โˆ’โˆž
A useful property of the characteristic function is the sum of independent random variables
is the product of the random variables characteristic function. It is often easier to use the
natural log of the characteristic function when conducting this operation.
๐œ‘๐œ‘๐‘‹๐‘‹+๐‘Œ๐‘Œ (๐œ”๐œ”) = ๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”)๐œ‘๐œ‘๐‘Œ๐‘Œ (๐œ”๐œ”)
ln[๐œ‘๐œ‘๐‘‹๐‘‹+๐‘Œ๐‘Œ (๐œ”๐œ”)] = ln[๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”)] ln[๐œ‘๐œ‘๐‘Œ๐‘Œ (๐œ”๐œ”)]
For example, the addition of two exponentially distributed random variables with the same
๐œ†๐œ† gives the gamma distribution with ๐‘˜๐‘˜ = 2:
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†),
๐‘–๐‘–๐‘–๐‘–
,
๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”) =
๐œ”๐œ” + ๐‘–๐‘–๐‘–๐‘–
๐‘Œ๐‘Œ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐œ‘๐œ‘๐‘Œ๐‘Œ (๐œ”๐œ”) =
๐‘–๐‘–๐‘–๐‘–
๐œ”๐œ” + ๐‘–๐‘–๐‘–๐‘–
๐œ‘๐œ‘๐‘‹๐‘‹+๐‘Œ๐‘Œ (๐œ”๐œ”) = ๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”)๐œ‘๐œ‘๐‘Œ๐‘Œ (๐œ”๐œ”)
โˆ’๐œ†๐œ†2
=
(๐œ”๐œ” + ๐‘–๐‘–๐‘–๐‘–)2
๐‘‹๐‘‹ + ๐‘Œ๐‘Œ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜ = 1, ๐œ†๐œ†)
This is the characteristic function of the gamma distribution with ๐‘˜๐‘˜ = 2.
The moment generating function can be calculated from the characteristic function:
๐œ‘๐œ‘๐‘‹๐‘‹ (โˆ’๐‘–๐‘–๐‘–๐‘–) = ๐‘€๐‘€๐‘‹๐‘‹ (๐œ”๐œ”)
The ๐‘›๐‘›๐‘ก๐‘กโ„Ž raw moment can be calculated by differentiating the characteristic function ๐‘›๐‘›
times. For more information on moments see section 1.3.2.
(๐‘›๐‘›)
๐ธ๐ธ[๐‘‹๐‘‹ ๐‘›๐‘› ] = ๐‘–๐‘– โˆ’๐‘›๐‘› ๐œ‘๐œ‘๐‘‹๐‘‹ (0)
๐‘‘๐‘‘๐‘›๐‘›
= ๐‘–๐‘– โˆ’๐‘›๐‘› ๏ฟฝ ๐‘›๐‘› ๐œ‘๐œ‘๐‘‹๐‘‹ (๐œ”๐œ”)๏ฟฝ
๐‘‘๐‘‘๐œ”๐œ”
Dist Functions
Introduction
Dist Functions
16
Probability Distributions Used in Reliability Engineering
1.2.12.
Joint Distributions
Joint distributions are multivariate distributions with, ๐‘‘๐‘‘ random variables (๐‘‘๐‘‘ > 1). An
example of a bivariate distribution (๐‘‘๐‘‘ = 2) may be the distribution of failure for a vehicle
tire which with random variables time, ๐‘‡๐‘‡, and distance travelled, ๐‘‹๐‘‹. The dependence
between these two variables can be quantified in terms of correlation and covariance. See
section 1.3.3 for more discussion. For more on properties of multivariate distributions see
(Rencher 1997). The continuous and discrete random variables will be denoted as:
๐‘ฅ๐‘ฅ1
๐‘ฅ๐‘ฅ2
๐’™๐’™ = ๏ฟฝ โ‹ฎ ๏ฟฝ ,
๐‘ฅ๐‘ฅ๐‘‘๐‘‘
๐‘˜๐‘˜1
๐‘˜๐‘˜2
๐’Œ๐’Œ = ๏ฟฝ ๏ฟฝ
โ‹ฎ
๐‘˜๐‘˜๐‘‘๐‘‘
Joint distributions can be derived from the conditional distributions. For the bivariate case
with random variables ๐‘ฅ๐‘ฅ and ๐‘ฆ๐‘ฆ:
๐‘“๐‘“(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = ๐‘“๐‘“(๐‘ฆ๐‘ฆ|๐‘ฅ๐‘ฅ)๐‘“๐‘“(๐‘ฅ๐‘ฅ) = ๐‘“๐‘“(๐‘ฅ๐‘ฅ|๐‘ฆ๐‘ฆ)๐‘“๐‘“(๐‘ฆ๐‘ฆ)
For the more general case:
๐‘“๐‘“(๐’™๐’™) = ๐‘“๐‘“(๐‘ฅ๐‘ฅ1 |๐‘ฅ๐‘ฅ2 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘‘๐‘‘ )๐‘“๐‘“(๐‘ฅ๐‘ฅ2 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘‘๐‘‘ )
= ๐‘“๐‘“(๐‘ฅ๐‘ฅ1 ) ๐‘“๐‘“(๐‘ฅ๐‘ฅ2 |๐‘ฅ๐‘ฅ1 ) โ€ฆ ๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘›๐‘›โˆ’1 |๐‘ฅ๐‘ฅ1 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘›๐‘›โˆ’2 ) ๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘›๐‘› |๐‘ฅ๐‘ฅ1 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘›๐‘› )
If the random variables are independent, their joint distribution is simply the product of the
marginal distributions:
๐‘‘๐‘‘
๐‘“๐‘“(๐’™๐’™) = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘–๐‘– ) ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐‘ฅ๐‘ฅ๐‘–๐‘– โŠฅ ๐‘ฅ๐‘ฅ๐‘—๐‘— ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘–๐‘– โ‰  ๐‘—๐‘—
๐‘–๐‘–=1
๐‘‘๐‘‘
๐‘“๐‘“(๐’Œ๐’Œ) = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ) ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐‘˜๐‘˜๐‘–๐‘– โŠฅ ๐‘˜๐‘˜๐‘—๐‘— ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘–๐‘– โ‰  ๐‘—๐‘—
๐‘–๐‘–=1
A general multivariate cumulative probability function with n random variables
(๐‘‡๐‘‡1 , ๐‘‡๐‘‡2 , โ€ฆ , ๐‘‡๐‘‡๐‘›๐‘› ) is defined as:
๐น๐น(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡1 โ‰ค ๐‘ก๐‘ก1 , ๐‘‡๐‘‡2 โ‰ค ๐‘ก๐‘ก2 , โ€ฆ , ๐‘‡๐‘‡๐‘›๐‘› โ‰ค ๐‘ก๐‘ก๐‘›๐‘› )
The survivor function is given as:
๐‘…๐‘…(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) = ๐‘ƒ๐‘ƒ(๐‘‡๐‘‡1 > ๐‘ก๐‘ก1 , ๐‘‡๐‘‡2 > ๐‘ก๐‘ก2 , โ€ฆ , ๐‘‡๐‘‡๐‘›๐‘› > ๐‘ก๐‘ก๐‘›๐‘› )
Different from univariate distributions is the relationship between the CDF and the
survivor function (Georges et al. 2001):
๐น๐น(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) + ๐‘…๐‘…(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) โ‰ค 1
If ๐น๐น(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) is differentiable then the probability density function is given as:
๐‘“๐‘“(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› ) =
17
๐œ•๐œ• ๐‘›๐‘› ๐น๐น(๐‘ก๐‘ก1 , ๐‘ก๐‘ก2 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘› )
๐œ•๐œ•๐‘ก๐‘ก1 ๐œ•๐œ•๐‘ก๐‘ก2 โ€ฆ ๐œ•๐œ•๐‘ก๐‘ก๐‘›๐‘›
For a discussion on the multivariate hazard rate functions and the construction of joint
distributions from marginal distributions see (Singpurwalla 2006).
1.2.13.
Marginal Distribution
The marginal distribution of a single random variable in a joint distribution can be obtained:
๐‘“๐‘“(๐‘ฅ๐‘ฅ1 ) = ๏ฟฝ โ€ฆ ๏ฟฝ ๏ฟฝ ๐‘“๐‘“(๐’™๐’™) ๐‘‘๐‘‘๐‘ฅ๐‘ฅ2 ๐‘‘๐‘‘๐‘ฅ๐‘ฅ3 โ€ฆ ๐‘‘๐‘‘๐‘ฅ๐‘ฅ๐‘‘๐‘‘
๐‘ฅ๐‘ฅ๐‘‘๐‘‘
๐‘ฅ๐‘ฅ3 ๐‘ฅ๐‘ฅ2
๐‘“๐‘“(๐‘˜๐‘˜1 ) = ๏ฟฝ ๏ฟฝ โ€ฆ ๏ฟฝ ๐‘“๐‘“(๐’Œ๐’Œ)
๐‘˜๐‘˜2
1.2.14.
๐‘˜๐‘˜3
๐‘˜๐‘˜๐‘›๐‘›
Conditional Distribution
If the value is known for some random variables the conditional distribution of the
remaining random variables is:
๐‘“๐‘“(๐‘ฅ๐‘ฅ1 |๐‘ฅ๐‘ฅ2 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘‘๐‘‘ ) =
1.2.15.
๐‘“๐‘“(๐‘ฅ๐‘ฅ2 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘‘๐‘‘ )
๐‘“๐‘“(๐‘˜๐‘˜1 |๐‘˜๐‘˜2 , โ€ฆ , ๐‘˜๐‘˜๐‘‘๐‘‘ ) =
Bathtub Distributions
๐‘“๐‘“(๐‘ฅ๐‘ฅ)
=
๐‘“๐‘“(๐‘ฅ๐‘ฅ)
โˆซ๐‘ฅ๐‘ฅ ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘1
1
๐‘“๐‘“(๐‘˜๐‘˜)
๐‘“๐‘“(๐‘˜๐‘˜)
=
๐‘“๐‘“(๐‘˜๐‘˜2 , โ€ฆ , ๐‘˜๐‘˜๐‘‘๐‘‘ ) โˆ‘๐‘˜๐‘˜1 ๐‘“๐‘“(๐‘ฅ๐‘ฅ)
Elementary texts on reliability introduce the hazard rate of a system as a bathtub curve.
The bathtub curve has three regions, infant mortality (decreasing failure rate), useful life
(constant failure rate) and wear out (increasing failure rate). Bathtub distributions have
not been a popular choice for modeling life distributions when compared to exponential,
Weibull and lognormal distributions. This is because bathtub distributions are generally
more complex without closed form moments and more difficult to estimate parameters.
Sometimes more complex shapes are required than simple bathtub curves, as such
generalizations and modifications to the bathtub curves has been studied. These include
an increase in the failure rate followed by a bathtub curve and rollercoaster curves
(decreasing followed by unimodal hazard rate). For further reading including
applications see (Lai & Xie 2006).
Dist Functions
Introduction
Dist Functions
18
Probability Distributions Used in Reliability Engineering
1.2.16.
Truncated Distributions
Truncation arises when the existence of a potential observation would be unknown if it
were to occur in a certain range. An example of truncation is when the existence of a
defect is unknown due to the defectโ€™s amplitude being less than the inspection threshold.
The number of flaws below the inspection threshold is unknown. This is not to be
confused with censoring which occurs when there is a bound for observing events. An
example of right censoring is when a test is time terminated and the failures of the
surviving components are not observed, however we know how many components were
censored. (Meeker & Escobar 1998, p.266)
A truncated distribution is the conditional distribution that results from restricting the
domain of another probability distribution. The following general formulas apply to
truncated distribution functions, where ๐‘“๐‘“0 (๐‘ฅ๐‘ฅ) and ๐น๐น0 (๐‘ฅ๐‘ฅ) are the pdf and cdf of the nontruncated distribution. For further reading specific to common distributions see (Cohen
1991)
Probability Distribution Function:
๐‘“๐‘“๐‘œ๐‘œ (๐‘ฅ๐‘ฅ)
๐‘“๐‘“(๐‘ฅ๐‘ฅ) = ๏ฟฝ๐น๐น0 (๐‘๐‘) โˆ’ ๐น๐น0 (๐‘Ž๐‘Ž) ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘ฅ๐‘ฅ โˆˆ (๐‘Ž๐‘Ž, ๐‘๐‘]
0
๐‘œ๐‘œ๐‘œ๐‘œโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’
Cumulative Distribution Function:
๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘ฅ๐‘ฅ โ‰ค ๐‘Ž๐‘Ž
โŽง ๐‘ฅ๐‘ฅ 0
โŽช โˆซ ๐‘“๐‘“0 (๐‘ก๐‘ก) ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘Ž๐‘Ž
๐น๐น(๐‘ฅ๐‘ฅ) =
๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘ฅ๐‘ฅ โˆˆ (๐‘Ž๐‘Ž, ๐‘๐‘]
โŽจ๐น๐น0 (๐‘๐‘) โˆ’ ๐น๐น0 (๐‘Ž๐‘Ž)
โŽช
1
๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘ฅ๐‘ฅ > ๐‘๐‘
โŽฉ
1.2.17.
19
Summary
Table 2: Summary of important reliability function relationships
๐’‡๐’‡(๐’•๐’•)
๐‘ญ๐‘ญ(๐’•๐’•)
๐‘น๐‘น(๐’•๐’•)
๐’‰๐’‰(๐’•๐’•)
๐’‡๐’‡(๐’•๐’•) =
---
๐น๐น โ€ฒ (๐‘ก๐‘ก)
โˆ’๐‘…๐‘… โ€ฒ (๐‘ก๐‘ก)
โ„Ž(๐‘ก๐‘ก) exp ๏ฟฝโˆ’ ๏ฟฝ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘ ๏ฟฝ
๐‘ญ๐‘ญ(๐’•๐’•) =
๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
----
1 โˆ’ ๐‘…๐‘…(๐‘ก๐‘ก)
1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘ ๏ฟฝ
๐‘น๐‘น(๐’•๐’•) =
1 โˆ’ ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
1 โ€“ ๐น๐น(๐‘ก๐‘ก)
----
exp ๏ฟฝโˆ’ ๏ฟฝ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘ ๏ฟฝ
๐’‰๐’‰(๐’•๐’•) =
๐‘“๐‘“(๐‘ก๐‘ก)
๐น๐น โ€ฒ (๐‘ก๐‘ก)
1 โˆ’ ๐น๐น(๐‘ก๐‘ก)
๐‘…๐‘… โ€ฒ (๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘ก๐‘ก
0
๐‘ก๐‘ก
0
๐‘ก๐‘ก
1 โˆ’ โˆซ0 ๐‘“๐‘“(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
โˆž
1
๏ฟฝ
1 โˆ’ ๐น๐น(๐‘ฅ๐‘ฅ)
โˆ’๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐’–๐’–(๐’•๐’•) =
โˆซ0 ๐‘ฅ๐‘ฅ๐‘ฅ๐‘ฅ(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘ โˆซโˆž[1 โˆ’ ๐น๐น(๐‘ฅ๐‘ฅ)]๐‘‘๐‘‘๐‘‘๐‘‘ โˆซโˆž ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘ก
๐‘ก๐‘ก
โˆž
1 โˆ’ ๐น๐น(๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
โˆซ๐‘ก๐‘ก ๐‘“๐‘“(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘ก
โˆž
๐‘ก๐‘ก
๐‘œ๐‘œ
๐‘ก๐‘ก
๐‘œ๐‘œ
โˆ’
๐‘ก๐‘ก
exp{โˆ’๐ป๐ป(๐‘ก๐‘ก)}
๐ป๐ป โ€ฒ (๐‘ก๐‘ก)
----
๐‘ก๐‘ก
โˆ’ln{๐‘…๐‘…(๐‘ฅ๐‘ฅ)}
๐‘‘๐‘‘{exp[โˆ’๐ป๐ป(๐‘ก๐‘ก)]}
๐‘‘๐‘‘๐‘‘๐‘‘
1 โˆ’ exp{โˆ’๐ป๐ป(๐‘ก๐‘ก)}
๐‘œ๐‘œ
๐‘ฏ๐‘ฏ(๐’•๐’•) =
ln ๏ฟฝ
๐‘ฏ๐‘ฏ(๐’•๐’•)
---
๏ฟฝ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
0
โˆž
๐‘ก๐‘ก
โˆซ๐‘ก๐‘ก exp๏ฟฝโˆ’ โˆซ๐‘œ๐‘œ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘ก
exp๏ฟฝโˆ’ โˆซ๐‘œ๐‘œ โ„Ž(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘ฅ๐‘ฅ๏ฟฝ
โˆž
โˆซ๐‘ก๐‘ก exp{โˆ’๐ป๐ป(๐‘ฅ๐‘ฅ)} ๐‘‘๐‘‘๐‘‘๐‘‘
exp{โˆ’๐ป๐ป(๐‘ฅ๐‘ฅ)}
Dist Functions
Introduction
Dist Properties
20
Probability Distributions Used in Reliability Engineering
1.3. Distribution Properties
1.3.1. Median / Mode
The median of a distribution, denoted as ๐‘ก๐‘ก0.5 is when the cdf and reliability function are
equal to 0.5.
๐‘ก๐‘ก0.5 = ๐น๐น โˆ’1 (0.5) = ๐‘…๐‘… โˆ’1 (0.5)
The mode is the highest point of the pdf, ๐‘ก๐‘ก๐‘š๐‘š . This is the point where a failure has the
highest probability. Samples from this distribution would occur most often around the
mode.
1.3.2. Moments of Distribution
The moments of a distribution are given by:
โˆž
๐œ‡๐œ‡๐‘›๐‘› = ๏ฟฝ (๐‘ฅ๐‘ฅ โˆ’ ๐‘๐‘)๐‘›๐‘› ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
โˆ’โˆž
๐‘›๐‘›
๐œ‡๐œ‡๐‘›๐‘› = ๏ฟฝ๏ฟฝ๐‘˜๐‘˜๐‘—๐‘— โˆ’ ๐‘๐‘๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜)
๐‘–๐‘–
When ๐‘๐‘ = 0 the moments, ๐œ‡๐œ‡๐‘›๐‘›โ€ฒ , are called the raw moments, described as moments about
the origin. In respect to probability distributions the first two raw moments are important.
๐œ‡๐œ‡0โ€ฒ always equals one, and ๐œ‡๐œ‡1โ€ฒ is the distributions mean which is the expected value of the
random variable for the distribution:
โˆž
mean = ๐ธ๐ธ[๐‘‹๐‘‹] = ๐œ‡๐œ‡:
๐œ‡๐œ‡โ€ฒ0 = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ = 1,
โˆ’โˆž
โˆž
๐œ‡๐œ‡โ€ฒ1 = ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘ฅ๐‘ฅ(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
โˆ’โˆž
๐œ‡๐œ‡โ€ฒ0 = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ) = 1
๐‘–๐‘–
๐œ‡๐œ‡โ€ฒ1 = ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘– ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– )
๐‘–๐‘–
Some important properties of the expected value ๐ธ๐ธ[๐‘‹๐‘‹] when transformations of the
random variable occur are:
๐ธ๐ธ[๐‘‹๐‘‹ + ๐‘๐‘] = ๐œ‡๐œ‡๐‘‹๐‘‹ + ๐‘๐‘
๐ธ๐ธ[๐‘‹๐‘‹ + ๐‘Œ๐‘Œ] = ๐œ‡๐œ‡๐‘‹๐‘‹ + ๐œ‡๐œ‡๐‘Œ๐‘Œ
๐ธ๐ธ[๐‘Ž๐‘Ž๐‘Ž๐‘Ž] = ๐‘Ž๐‘Ž๐œ‡๐œ‡๐‘‹๐‘‹
๐ธ๐ธ[๐‘‹๐‘‹๐‘‹๐‘‹] = ๐œ‡๐œ‡๐‘‹๐‘‹ ๐œ‡๐œ‡๐‘Œ๐‘Œ + ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ)
When ๐‘๐‘ = ๐œ‡๐œ‡ the moments, ๐œ‡๐œ‡๐‘›๐‘› , are called the central moments, described as moments
about the mean. In this book, the first five central moments are important. ๐œ‡๐œ‡0 is equal to
๐œ‡๐œ‡0โ€ฒ = 1. ๐œ‡๐œ‡1 is the variance which quantifies the amount the random variable deviates from
the mean. ๐œ‡๐œ‡2 and ๐œ‡๐œ‡3 are used to calculate the skewness and kurtosis.
โˆž
๐œ‡๐œ‡0 = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ = 1,
โˆž
โˆ’โˆž
๐œ‡๐œ‡1 = ๏ฟฝ (๐‘ฅ๐‘ฅ โˆ’ ๐œ‡๐œ‡)๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ = 0,
โˆ’โˆž
๐œ‡๐œ‡0 = ๏ฟฝ ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ) = 1
๐‘–๐‘–
๐œ‡๐œ‡1 = ๏ฟฝ(๐‘˜๐‘˜๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ) = 0
๐‘–๐‘–
21
variance = ๐ธ๐ธ[(๐‘‹๐‘‹ โˆ’ ๐ธ๐ธ[๐‘‹๐‘‹])2 ] = ๐ธ๐ธ[๐‘‹๐‘‹ 2 ] โˆ’ {๐ธ๐ธ[๐‘‹๐‘‹]}2 = ๐œŽ๐œŽ 2:
โˆž
๐œ‡๐œ‡2 = ๏ฟฝ (๐‘ฅ๐‘ฅ โˆ’ ๐œ‡๐œ‡)2 ๐‘“๐‘“(๐‘ฅ๐‘ฅ) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
๐œ‡๐œ‡2 = ๏ฟฝ(๐‘˜๐‘˜๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2 ๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– )
โˆ’โˆž
๐‘–๐‘–
Some important properties of the variance exist when transformations of the random
variable occur are:
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‹๐‘‹ + ๐‘๐‘] = ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‹๐‘‹]
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‹๐‘‹ + ๐‘Œ๐‘Œ] = ๐œŽ๐œŽ๐‘‹๐‘‹2 + ๐œŽ๐œŽ๐‘Œ๐‘Œ2 ± 2๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ)
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘Ž๐‘Ž๐‘Ž๐‘Ž] = ๐‘Ž๐‘Ž2 ๐œŽ๐œŽ๐‘‹๐‘‹2
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‹๐‘‹๐‘‹๐‘‹] = (๐‘‹๐‘‹๐‘‹๐‘‹)2 ๏ฟฝ๏ฟฝ
๐œŽ๐œŽ๐‘‹๐‘‹ 2
๐œŽ๐œŽ๐‘Œ๐‘Œ 2
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ)
๏ฟฝ +๏ฟฝ ๏ฟฝ +2
๏ฟฝ
๐‘‹๐‘‹
๐‘Œ๐‘Œ
๐‘‹๐‘‹๐‘‹๐‘‹
The skewness is a measure of the asymmetry of the distribution.
๐›พ๐›พ1 =
๐œ‡๐œ‡3
โ„2
๐œ‡๐œ‡23
The kurtosis is a measure of the whether the data is peaked or flat.
1.3.3. Covariance
๐›พ๐›พ2 =
๐œ‡๐œ‡4
๐œ‡๐œ‡22
Covariance is a measure of the dependence between random variables.
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ) = ๐ธ๐ธ[(๐‘‹๐‘‹ โˆ’ ๐œ‡๐œ‡๐‘‹๐‘‹ )(๐‘Œ๐‘Œ โˆ’ ๐œ‡๐œ‡๐‘Œ๐‘Œ )] = ๐ธ๐ธ[๐‘‹๐‘‹๐‘‹๐‘‹] โˆ’ ๐œ‡๐œ‡๐‘‹๐‘‹ ๐œ‡๐œ‡๐‘Œ๐‘Œ
A normalized measure of covariance is correlation, ๐œŒ๐œŒ. The correlation has the limits โˆ’1 โ‰ค
๐œŒ๐œŒ โ‰ค 1. When ๐œŒ๐œŒ = 1 the random variables have a linear dependency (i.e, an increase in X
will result in the same increase in Y). When ๐œŒ๐œŒ = โˆ’1 the random variables have a negative
linear dependency (i.e, an increase in X will result in the same decrease in Y). The
relationship between covariance and correlation is:
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ)
๐œŒ๐œŒ๐‘‹๐‘‹,๐‘Œ๐‘Œ = ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐‘‹๐‘‹, ๐‘Œ๐‘Œ) =
๐œŽ๐œŽ๐‘‹๐‘‹ ๐œŽ๐œŽ๐‘Œ๐‘Œ
If the two random variables are independent than the correlation is equal to zero, however
the reverse is not always true. If the correlation is zero the random variables does not
need to be independent. For derivations and more information the reader is referred to
(Dekking et al. 2007, p.138).
Dist Properties
Introduction
Parameter Est
22
Probability Distributions Used in Reliability Engineering
1.4. Parameter Estimation
1.4.1. Probability Plotting Paper
Most plotting methods transform the data available into a straight line for a specific
distribution. From a line of best fit the parameters of the distribution can be estimated.
Most plotting paper plots the random variable (time or demands) against the pdf, cdf or
hazard rate and transform the data points to a linear relationship by adjusting the scale of
each axis. Probability plotting is done using the following steps (Nelson 1982, p.108):
1. Order the data such that ๐‘ฅ๐‘ฅ1 โ‰ค ๐‘ฅ๐‘ฅ2 โ‰ค โ‹ฏ โ‰ค ๐‘ฅ๐‘ฅ๐‘–๐‘– โ‰ค โ‹ฏ โ‰ค ๐‘ฅ๐‘ฅ๐‘›๐‘› .
2. Assign a rank to each failure. For complete data this is simply the value i. Censored
data is discussed after step 7.
3. Calculate the plotting position. The cdf may simply be calculated as ๐‘–๐‘–/๐‘›๐‘› however this
produces a biased result, instead the following non-parametric Blom estimates, are
recommended as suitable for many cases by (Kimball 1960):
1
โ„Ž๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
(๐‘›๐‘› โˆ’ ๐‘–๐‘– + 0.625)(๐‘ก๐‘ก๐‘–๐‘–+1 โˆ’ ๐‘ก๐‘ก๐‘–๐‘– )
๐น๐น๏ฟฝ (๐‘ก๐‘ก๐‘–๐‘– ) =
๐‘–๐‘– โˆ’ 0.375
๐‘›๐‘› + 0.25
๐‘“๐‘“ฬ‚(๐‘ก๐‘ก๐‘–๐‘– ) =
1
(๐‘›๐‘› + 0.25)(๐‘ก๐‘ก๐‘–๐‘–+1 โˆ’ ๐‘ก๐‘ก๐‘–๐‘– )
๐‘…๐‘…๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
Other proposed estimators are:
๐‘›๐‘› โˆ’ ๐‘–๐‘– + 0.625
(๐‘›๐‘› + 0.25)
๐‘–๐‘–
๐‘›๐‘›
๐‘–๐‘– โˆ’ 0.3
Median (approximate): ๐น๐น๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
๐‘›๐‘› + 0.4
Naive: ๐น๐น๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
Midpoint: ๐น๐น๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
Mean โˆถ ๐น๐น๏ฟฝ (๐‘ก๐‘ก๐‘–๐‘– ) =
Mode: ๐น๐น๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) =
๐‘–๐‘– โˆ’ 0.5
๐‘›๐‘›
๐‘–๐‘–
๐‘›๐‘› + 1
๐‘–๐‘– โˆ’ 1
๐‘›๐‘› โˆ’ 1
4. Plot points on probability paper. The choice of distribution should be from experience,
or multiple distributions should be used to assess the best fit. Probability paper is available
from http://www.weibull.com/GPaper/.
23
5. Assess the data and chosen distributions. If the data plots in straight line then the
distribution may be a reasonable fit.
6. Draw a line of best fit. This is a subjective assessment which minimizes the deviation
of the points from the chosen line.
7. Obtained the desired information. This may be the distribution parameters or estimates
of reliability or hazard rate trends.
When multiple failure modes are observed only one failure mode should be plotted with
the other failures being treated as censored. Two popular methods to treat censored data
two methods are:
Rank Adjustment Method. (Manzini et al. 2009, p.140) Here the adjusted rank, ๐‘—๐‘—๐‘ก๐‘ก๐‘–๐‘– is
calculated only for non-censored units (with ๐‘–๐‘–๐‘ก๐‘ก๐‘–๐‘– still being the rank for all ordered times).
This adjusted rank is used for step 2 with the remaining steps unchanged:
(๐‘›๐‘› + 1) โˆ’ ๐‘—๐‘—๐‘ก๐‘ก๐‘–๐‘–โˆ’1
๐‘—๐‘—๐‘ก๐‘ก๐‘–๐‘– = ๐‘—๐‘—๐‘ก๐‘ก๐‘–๐‘–โˆ’1 +
2 + ๐‘›๐‘› โˆ’ ๐‘–๐‘–๐‘ก๐‘ก๐‘–๐‘–
Kaplan Meier Estimator. Here the estimate for reliability is:
๐‘‘๐‘‘
๐‘…๐‘…๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– ) = ๏ฟฝ ๏ฟฝ1 โˆ’
๏ฟฝ
๐‘›๐‘› โˆ’ ๐‘–๐‘– + 1
๐‘ก๐‘ก๐‘—๐‘— <๐‘ก๐‘ก๐‘–๐‘–
Where ๐‘‘๐‘‘ is the number of failures in rank j (for non-grouped data ๐‘‘๐‘‘ = 1). From this
estimate a cdf can be given as ๐น๐น๏ฟฝ (๐‘ก๐‘ก๐‘–๐‘– ) = 1 โˆ’ ๐‘…๐‘…๏ฟฝ (๐‘ก๐‘ก๐‘–๐‘– ). For a detailed derivation and properties
of this estimator see (Andersen et al. 1996, p.255)
Probability plots are fast and not dependent on complex numerical methods and can be
used without a detailed knowledge of statistics. It provides a visual representation of the
data for which qualitative statements can be made. It can be useful in estimating initial
values for numerical methods. Limitation of this technique is that it is not objective and
two different people making the same plot will obtain different answers. It also does not
provide confidence intervals. For more detail of probability plotting the reader is referred
to (Nelson 1982, p.104) and (Meeker & Escobar 1998, p.122)
1.4.2. Total Time on Test Plots
Total time on Test (TTT) plots is a graph which provides a visual representation of the
hazard rate trend, i.e increasing, constant or decreasing. This assists in identifying the
distribution from which the data may come from. To plot TTT (Rinne 2008, p.334):
1. Order the data such that ๐‘ฅ๐‘ฅ1 โ‰ค ๐‘ฅ๐‘ฅ2 โ‰ค โ‹ฏ โ‰ค ๐‘ฅ๐‘ฅ๐‘–๐‘– โ‰ค โ‹ฏ โ‰ค ๐‘ฅ๐‘ฅ๐‘›๐‘› .
2. Calculate the TTT positions:
๐‘–๐‘–
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘–๐‘– = ๏ฟฝ(๐‘›๐‘› โˆ’ ๐‘—๐‘— + 1)๏ฟฝ๐‘ฅ๐‘ฅ๐‘—๐‘— โˆ’ ๐‘ฅ๐‘ฅ๐‘—๐‘—โˆ’1 ๏ฟฝ ; ๐‘–๐‘– = 1,2, โ€ฆ , ๐‘›๐‘›
๐‘—๐‘—=1
3. Calculate the normalized TTT positions:
Parameter Est
Introduction
Parameter Est
24
Probability Distributions Used in Reliability Engineering
๐‘–๐‘–
4. Plot the points ๏ฟฝ , ๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘–๐‘–โˆ— ๏ฟฝ.
5. Analyze graph:
๐‘›๐‘›
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘–๐‘–โˆ— =
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘–๐‘–
; ๐‘–๐‘– = 1,2, โ€ฆ , ๐‘›๐‘›
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘›๐‘›
1
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘–๐‘–
0
0
๐‘–๐‘–๏ฟฝ
๐‘›๐‘›
1
Figure 5: Time on test plot interpretation
Compared to probability plotting, TTT plots are simple, scale invariant and can represent
any data set even those from different distributions on the same plot. However it only
provides an indication of failure rate properties and cannot be used directly to estimate
parameters. For more information about TTT plots the reader is referred to (Rinne 2008,
p.334).
1.4.3. Least Mean Square Regression
When the relationship between two variables, ๐‘ฅ๐‘ฅ and ๐‘ฆ๐‘ฆ is assumed linear (๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘), an
estimate of the lineโ€™s parameters can be obtained from ๐‘›๐‘› sample data points, (๐‘ฅ๐‘ฅ๐‘–๐‘– , ๐‘ฆ๐‘ฆ๐‘–๐‘– ) using
least mean square (LMS) regression. The least square method minimizes the square of
the residual.
๐‘›๐‘›
๐‘†๐‘† = ๏ฟฝ ๐‘Ÿ๐‘Ÿ๐‘–๐‘–2
๐‘–๐‘–=1
25
The residual can be defined in many ways.
Minimize y residuals
๐‘Ÿ๐‘Ÿ๐‘–๐‘– = ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ ๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘–๐‘– ; ๐‘š๐‘š, ๐‘๐‘)
๐‘š๐‘š
๏ฟฝ=
๐‘ฆ๐‘ฆ๐‘–๐‘–
๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘–๐‘– ; ๐‘š๐‘š, ๐‘๐‘)
๐‘›๐‘›โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘– ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ (โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘– )(โˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘– )
๐‘›๐‘›โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘–2
๐‘๐‘ฬ‚ =
๐‘ฆ๐‘ฆ
๐‘Ÿ๐‘Ÿ๐‘–๐‘–
Minimize x residuals
๐‘Ÿ๐‘Ÿ๐‘–๐‘– = ๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐‘“๐‘“(๐‘ฆ๐‘ฆ๐‘–๐‘– ; ๐‘š๐‘š, ๐‘๐‘)
โˆ’
๐‘š๐‘š
๏ฟฝ=
2
๏ฟฝโˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘–2 ๏ฟฝ
โˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘–
โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘–
โˆ’ ๐‘š๐‘š
๏ฟฝ
๐‘›๐‘›
๐‘›๐‘›
๐‘๐‘ฬ‚ =
๐‘ฆ๐‘ฆ
(๐‘ฅ๐‘ฅ๐‘–๐‘– , ๐‘ฆ๐‘ฆ๐‘–๐‘– )
2
๐‘›๐‘›โˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘–2 โˆ’ ๏ฟฝโˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘–2 ๏ฟฝ
๐‘›๐‘›โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘– ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ (โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘– )(โˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘– )
โˆ‘๐‘ฆ๐‘ฆ๐‘–๐‘–
โˆ‘๐‘ฅ๐‘ฅ๐‘–๐‘–
โˆ’ ๐‘š๐‘š
๏ฟฝ
๐‘›๐‘›
๐‘›๐‘›
Regression line
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
(๐‘ฅ๐‘ฅ๐‘–๐‘– , ๐‘ฆ๐‘ฆ๐‘–๐‘– )
Regression line
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
0
๐‘ฅ๐‘ฅ
0
๐‘“๐‘“(๐‘ฆ๐‘ฆ๐‘–๐‘– ; ๐‘š๐‘š, ๐‘๐‘)
๐‘Ÿ๐‘Ÿ๐‘–๐‘–
๐‘ฆ๐‘ฆ๐‘–๐‘–
๐‘ฅ๐‘ฅ
Figure 6: Left minimize y residual, right minimize x residual
The LMS method can be used to estimate the line of best fit when using plotting parameter
estimation methods. When plotting on a regular scale in software such as Microsoft Excel,
it is often easy to conduct linear least mean square (LMS) regression using in built
functions. Where available this book provides the formulas to plot the sample data in a
straight line in a regular scale plot. It also provides the transformation from the linear LMS
regression estimates of ๐‘š๐‘š
๏ฟฝ and ๐‘๐‘ฬ‚ to the distribution parameter estimates.
For more on least square methods in a reliability engineering context see (Nelson 1990,
p.167). MS regression can also be conducted on multivariate distributions, see (Rao &
Toutenburg 1999) and can also be conducted on non-linear data directly, see (Bjõrck
1996).
1.4.4. Method of Moments
To estimate the distribution parameters using the method of moments the sample
moments are equated to the parameter moments and solved for the unknown parameters.
The following sample moments can be used:
The sample mean is given as:
๐‘ฅ๐‘ฅฬ… =
๐‘›๐‘›
1
๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘–
๐‘›๐‘›
๐‘–๐‘–=1
Parameter Est
Introduction
Parameter Est
26
Probability Distributions Used in Reliability Engineering
The unbiased sample variance is given as:
๐‘›๐‘›
1
๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐‘ฅ๐‘ฅฬ… )2
๐‘†๐‘† 2 =
๐‘›๐‘› โˆ’ 1
๐‘–๐‘–=1
Method of moments is not as accurate as Bayesian or maximum likelihood estimates but
is easy and fast to calculate. The method of moment estimates are often used as a starting
point for numerical methods to optimize maximum likelihood and least square estimators.
1.4.5. Maximum Likelihood Estimates
Maximum likelihood estimates (MLE) are based on a frequentist approach to parameter
estimation usually obtained by maximizing the natural log of the likelihood function.
ฮ›(ฮธ|E) = ln{๐ฟ๐ฟ(๐œƒ๐œƒ|๐ธ๐ธ)}
Algebraically this is done by solving the first order partial derivatives of the log-likelihood
function. This calculation has been included in this book for distributions where the result
is in closed form. Otherwise the log-likelihood function can be maximized directly using
numerical methods.
MLE for ๐œƒ๐œƒ๏ฟฝ is obtained by solving for ๐œƒ๐œƒ:
๐œ•๐œ•ฮ›
=0
๐œ•๐œ•๐œ•๐œ•
Denote the true parameters of the distribution as ๐œฝ๐œฝ๐ŸŽ๐ŸŽ , MLEs have the following properties
(Rinne 2008, p.406):
โ€ข
Consistency. As the number of samples increases the difference between the
estimated and actual parameter decreases:
๏ฟฝ = ๐œฝ๐œฝ
plim ๐œฝ๐œฝ
โ€ข
Asymptotic normality.
๐‘›๐‘›โ†’โˆž
lim ๐œƒ๐œƒ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œƒ๐œƒ0 , [๐ผ๐ผ๐‘›๐‘› (๐œƒ๐œƒ0 )]โˆ’1 )
๐‘›๐‘›โ†’โˆž
โ€ข
โ€ข
where ๐ผ๐ผ๐‘›๐‘› (๐œƒ๐œƒ) = ๐‘›๐‘›๐‘›๐‘›(๐œƒ๐œƒ) is the Fisher information matrix. Therefore ๐œƒ๐œƒ๏ฟฝ is
asymptotically unbiased:
lim ๐ธ๐ธ[๐œƒ๐œƒ๏ฟฝ] = ๐œƒ๐œƒ0
Asymptotic efficiency.
๐‘›๐‘›โ†’โˆž
lim ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐œƒ๐œƒ๏ฟฝ] = [๐ผ๐ผ๐‘›๐‘› (๐œƒ๐œƒ0 )]โˆ’1
๐‘›๐‘›โ†’โˆž
Invariance. The MLE of ๐‘“๐‘“(๐œƒ๐œƒ0 ) is ๐‘“๐‘“(๐œƒ๐œƒ๏ฟฝ ) if ๐‘“๐‘“(. ) is a continuous and continuously
differentiable function.
The advantages of MLE are that it is a very common technique that has been widely
published and is implemented in many software packages. The MLE method can easily
handle censored data. The disadvantage to MLE is the bias introduced for small sample
sizes and unbounded estimates may result when no failures have been observed. The
27
numerical optimization of the log-likelihood function may be non-trivial with high sensitivity
to starting values and the presence of local maximums.
For more information in a reliability context see (Nelson 1990, p.284).
1.4.6. Bayesian Estimation
Bayesian estimation uses a subjective interpretation of the theory of probability and for
parameter point estimation and confidence intervals uses Bayesโ€™ rule to update our state
of knowledge of the unknown of interest (UIO). Recall from section 1.1.5 Bayes rule,
๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ) =
๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ) ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ)
โˆซ ๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ) ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ) ๐‘‘๐‘‘๐‘‘๐‘‘
,
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ|๐ธ๐ธ) =
๐‘ƒ๐‘ƒ(๐œƒ๐œƒ) ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ)
โˆ‘๐‘–๐‘– ๐‘ƒ๐‘ƒ(๐ธ๐ธ|๐œƒ๐œƒ๐‘–๐‘– )๐‘ƒ๐‘ƒ(๐œƒ๐œƒ)
respectively for continuous and discrete forms of variable of ๐œƒ๐œƒ.
The Prior Distribution ๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ)
The prior distribution is probability distribution of the UOI, ๐œƒ๐œƒ, which captures our state of
knowledge of ๐œƒ๐œƒ prior to the evidence being observed. It is common for this distribution to
represent soft evidence or intervals about the possible values of ๐œƒ๐œƒ. If the distribution is
dispersed it represents little being known about the parameter. If the distribution is
concentrated in an area then it reflects a good knowledge about the likely values of ๐œƒ๐œƒ.
Prior distributions should be a proper probability distribution of ๐œƒ๐œƒ. A distribution is proper
when it integrates to one and improper otherwise. The prior should also not be selected
based on the form of the likelihood function. When the prior has a constant which does
not affect the posterior distribution (such as improper priors) it will be omitted from the
formulas within this book.
Non-informative Priors. Occasions arise when it is not possible to express a subjective
prior distribution due to lack of information, time or cost. Alternatively a subjective prior
distribution may introduce unwanted bias through model convenience (conjugates) or due
to elicitation methods. In such cases a non-informative prior may be desirable. The
following methods exist for creating a non-informative prior (Yang and Berger 1998):
โ€ข
Principle of Indifference - Improper Uniform Priors. An equal probability is
assigned across all the possible values of the parameter. This is done using an
improper uniform distribution with a constant, usually 1, over the range of the
possible values for ๐œƒ๐œƒ. When placed in Bayes formula the constant cancels out,
however the denominator is integrated over all possible values of ๐œƒ๐œƒ. In most
cases this prior distribution will result in a proper posterior, but not always.
Improper Uniform Priors may be chosen to enable the use of conjugate priors.
For example using exponential likelihood model, with an improper uniform prior,
1, over the limits [0, โˆž) with evidence of ๐‘›๐‘›๐น๐น failures in total time, ๐‘ก๐‘ก ๐‘‡๐‘‡ :
Prior:
๐œ‹๐œ‹0 (๐œ†๐œ†) = 1 โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0)
Likelihood:
๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) = ๐œ†๐œ†nF ๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘‡๐‘‡
Parameter Est
Introduction
Parameter Est
28
Probability Distributions Used in Reliability Engineering
Posterior: ๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
1. ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†)
โˆž
1. โˆซ0 ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) ๐‘‘๐‘‘๐‘‘๐‘‘
Using conjugate relationship (see Conjugate Priors for calculations):
โ€ข
๐œ†๐œ†~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + nF , tT )
Principle of Indifference - Proper Uniform Priors. An equal probability is
assigned across the values of the parameter within a range defined by the
uniform distribution. The uniform distribution is obtained by estimating the far
1
left and right bounds (๐‘Ž๐‘Ž and ๐‘๐‘) of the parameter ๐œƒ๐œƒ giving ๐œ‹๐œ‹๐‘œ๐‘œ (๐œƒ๐œƒ) =
= ๐‘๐‘, where
๐‘๐‘โˆ’๐‘Ž๐‘Ž
c is a constant. When placed in Bayes formula the constant cancels out,
however the denominator is integrated over the bound [๐‘Ž๐‘Ž, ๐‘๐‘]. Care needs to be
taken in choosing ๐‘Ž๐‘Ž and ๐‘๐‘ because no matter how much evidence suggests
otherwise the posterior distribution will always be zero outside these bounds.
Using an exponential likelihood model, with a proper uniform prior, ๐‘๐‘, over the
limits [๐‘Ž๐‘Ž, ๐‘๐‘] with evidence of ๐‘›๐‘›๐น๐น failures in total time, ๐‘ก๐‘ก ๐‘‡๐‘‡ :
Prior:
๐œ‹๐œ‹0 (๐œ†๐œ†) =
1
๐‘๐‘โˆ’๐‘Ž๐‘Ž
Likelihood:
Posterior: ๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
= ๐‘๐‘ โˆ ๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0)
๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) = ๐œ†๐œ†nF ๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘‡๐‘‡
๐‘๐‘. ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†)
๐‘๐‘
๐‘๐‘. โˆซ๐‘Ž๐‘Ž ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) ๐‘‘๐‘‘๐‘‘๐‘‘
for a โ‰ค ฮป โ‰ค b
Using conjugate relationship this results in a truncated Gamma distribution:
๐œ‹๐œ‹(๐œ†๐œ†) = ๏ฟฝ
๐‘๐‘. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + nF , tT ) for a โ‰ค ฮป โ‰ค b
0
otherwise
โ€ข
Jeffreyโ€™s Prior. Proposed by Jeffery in 1961, this prior is defined as ๐œ‹๐œ‹0 (๐œƒ๐œƒ) =
๏ฟฝ๐‘‘๐‘‘๐‘‘๐‘‘๐‘‘๐‘‘ (๐‘ฐ๐‘ฐ๐œƒ๐œƒ ) where ๐‘ฐ๐‘ฐ๐œƒ๐œƒ is the Fisher information matrix. This derivation is motivated
by the fact that it is not dependent upon the set of parameter variables that is
chosen to describe parameter space. Jeffery himself suggested the need to
make ad hoc modifications to the prior to avoid problems in multidimensional
distributions. Jefforyโ€™s prior is normally improper. (Bernardo et al. 1992)
โ€ข
Reference Prior. Proposed by Bernardo in 1979, this prior maximizes the
expected posterior information from the data, therefore reducing the effect of the
prior. When there is no nuisance parameters and certain regularity conditions
are satisfied the reference prior is identical to the Jeffreyโ€™s prior. Due to the need
to order or group the importance of parameters, it may occur that different
posteriors will result from the same data depending on the importance the user
places on each parameter. This prior overcomes the problems which arise when
using Jefferyโ€™s prior in multivariate applications.
โ€ข
29
Maximal Data Information Prior (MDIP). Developed by Zelluer in 1971
maximizes the likelihood function with relation to the prior. (Berry et al. 1995,
p.182)
For further detail on the differences between each type of non-informative prior see (Berry
et al. 1995, p.179)
Conjugate Priors. Calculating posterior distributions can be extremely complex and in
most cases requires expensive computations. A special case exists however by which the
posterior distribution is of the same form as the prior distribution. The Bayesian updating
mathematics can be reduced to simple calculations to update the model parameters. As
an example the gamma function is a conjugate prior to a Poisson likelihood function:
Prior: ๐œ‹๐œ‹๐‘œ๐‘œ (๐œ†๐œ†) =
๐›ฝ๐›ฝ๐›ผ๐›ผ ๐œ†๐œ†๐›ผ๐›ผโˆ’1 โˆ’ฮฒฮป
๐‘’๐‘’
ฮ“(๐›ผ๐›ผ)
Likelihood: ๐ฟ๐ฟ๐‘–๐‘– (๐‘ก๐‘ก๐‘–๐‘– |๐œ†๐œ†) =
๐‘›๐‘›๐น๐น
๐œ†๐œ†๐‘˜๐‘˜๐‘–๐‘– ๐‘ก๐‘ก๐‘–๐‘–๐‘˜๐‘˜ โˆ’๐œ†๐œ†๐‘ก๐‘ก
๐‘’๐‘’ ๐‘–๐‘–
๐‘˜๐‘˜๐‘–๐‘– !
Likelihood: ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) = ๏ฟฝ ๐ฟ๐ฟ๐‘–๐‘– (๐‘ก๐‘ก๐‘–๐‘– |๐œ†๐œ†) =
๐‘–๐‘–=1
Posterior: ๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
=
๐œ‹๐œ‹๐‘œ๐‘œ (๐œ†๐œ†) ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†)
โˆž
โˆซ0 ๐œ‹๐œ‹๐‘œ๐‘œ (๐œ†๐œ†) ๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) ๐‘‘๐‘‘๐‘‘๐‘‘
๏ฟฝ
โˆž
0
โˆž
=
๐œ†๐œ†โˆ‘๐‘˜๐‘˜ โˆ๐‘ก๐‘ก๐‘–๐‘–๐‘˜๐‘˜ โˆ’๐œ†๐œ†โˆ‘๐‘ก๐‘ก
๐‘–๐‘–
๐‘’๐‘’
โˆ๐‘˜๐‘˜๐‘–๐‘– !
๐›ฝ๐›ฝ๐›ผ๐›ผ ๐œ†๐œ†๐›ผ๐›ผโˆ’1 ๐œ†๐œ†โˆ‘๐‘˜๐‘˜ โˆ๐‘ก๐‘ก๐‘–๐‘–๐‘˜๐‘˜ โˆ’ฮฒฮป โˆ’๐œ†๐œ†โˆ‘๐‘ก๐‘ก๐‘–๐‘–
๐‘’๐‘’ ๐‘’๐‘’
ฮ“(๐›ผ๐›ผ)โˆ๐‘˜๐‘˜๐‘–๐‘– !
๐›ฝ๐›ฝ๐›ผ๐›ผ ๐œ†๐œ†๐›ผ๐›ผโˆ’1 ๐œ†๐œ†โˆ‘๐‘˜๐‘˜ โˆ๐‘ก๐‘ก๐‘–๐‘–๐‘˜๐‘˜ โˆ’ฮฒฮป โˆ’๐œ†๐œ†โˆ‘๐‘ก๐‘ก
๐‘–๐‘– ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘’๐‘’ ๐‘’๐‘’
ฮ“(๐›ผ๐›ผ)โˆ๐‘˜๐‘˜๐‘–๐‘– !
๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ ๐‘’๐‘’ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก๐‘–๐‘–)
โˆž
โˆซ0 ๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜
๐‘’๐‘’ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก๐‘–๐‘–) ๐‘‘๐‘‘๐‘‘๐‘‘
Using the identity ฮ“(๐‘ง๐‘ง) = โˆซ๐‘œ๐‘œ ๐‘ฅ๐‘ฅ ๐‘ง๐‘งโˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘ we can calculate the denominator using the
change of variable ๐‘ข๐‘ข = ๐œ†๐œ†(๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– ). This results in ๐œ†๐œ† =
๐‘ข๐‘ข
๐›ฝ๐›ฝ+โˆ‘๐‘ก๐‘ก๐‘–๐‘–
, and ๐‘‘๐‘‘๐‘‘๐‘‘ =
๐‘‘๐‘‘๐‘‘๐‘‘
๐›ฝ๐›ฝ+โˆ‘๐‘ก๐‘ก๐‘–๐‘–
of ๐‘ข๐‘ข the same as ๐œ†๐œ† . Substituting back into the posterior equation gives:
๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ ๐‘’๐‘’ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก๐‘–๐‘– )
โˆž
๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜
๐‘ข๐‘ข
1
๏ฟฝ ๏ฟฝ
๏ฟฝ
๐‘’๐‘’ โˆ’u ๐‘‘๐‘‘๐‘‘๐‘‘
๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘–
๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘–
0
Let ๐‘ง๐‘ง = ๐›ผ๐›ผ + โˆ‘๐‘˜๐‘˜
=
๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ ๐‘’๐‘’ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก๐‘–๐‘– )
โˆž
1
โˆซ ๐‘ข๐‘ข๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ ๐‘’๐‘’ โˆ’u ๐‘‘๐‘‘๐‘‘๐‘‘
(๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– )๐›ผ๐›ผ+โˆ‘๐‘˜๐‘˜ 0
with the limits
Parameter Est
Introduction
Parameter Est
30
Probability Distributions Used in Reliability Engineering
๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
โˆž
Using ฮ“(๐‘ง๐‘ง) = โˆซ๐‘œ๐‘œ ๐‘ฅ๐‘ฅ ๐‘ง๐‘งโˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘:
๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
Let ๐›ผ๐›ผ โ€ฒ = ๐›ผ๐›ผ + โˆ‘๐‘˜๐‘˜, ๐›ฝ๐›ฝโ€ฒ = ๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– :
๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ ๐‘’๐‘’ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก๐‘–๐‘– )
โˆž
1
โˆซ ๐‘ข๐‘ข ๐‘ง๐‘งโˆ’1 ๐‘’๐‘’ โˆ’u ๐‘‘๐‘‘๐‘‘๐‘‘
(๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– )๐›ผ๐›ผ+โˆ‘๐‘˜๐‘˜ 0
๐œ†๐œ†๐›ผ๐›ผโˆ’1+โˆ‘๐‘˜๐‘˜ (๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– )๐›ผ๐›ผ+โˆ‘๐‘˜๐‘˜ โˆ’ฮป(ฮฒ+โˆ‘๐‘ก๐‘ก )
๐‘–๐‘–
๐‘’๐‘’
ฮ“(๐›ผ๐›ผ + โˆ‘๐‘˜๐‘˜)
โ€ฒ
๐›ผ๐›ผ โ€ฒ
๐œ†๐œ†๐›ผ๐›ผ โˆ’1 ๐›ฝ๐›ฝโ€ฒ
๐œ‹๐œ‹(๐œ†๐œ†|๐ธ๐ธ) =
ฮ“(๐›ผ๐›ผ โ€ฒ )
โ€ฒ
๐‘’๐‘’ โˆ’ฮฒ ฮป
As can be seen the posterior is a gamma distribution with the parameters ๐›ผ๐›ผ โ€ฒ = ๐›ผ๐›ผ + โˆ‘๐‘˜๐‘˜,
๐›ฝ๐›ฝโ€ฒ = ๐›ฝ๐›ฝ + โˆ‘๐‘ก๐‘ก๐‘–๐‘– . Therefore the prior and posterior are of the same form, and Bayesโ€™ rule does
not need to be re-calculated for each update. Instead the user can simply update the
parameters with the new evidence.
The Likelihood Function ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ)
The reader is referred to section 1.1.6 for a discussion on the construction of the likelihood
function.
The Posterior Distribution ๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ)
The posterior distribution is a probability distribution of the UOI, ๐œƒ๐œƒ, which captures our
state of knowledge of ๐œƒ๐œƒ including all prior information and the evidence.
Point Estimate. From the posterior distribution we may want to give a point estimate of
ฮธ. The Bayesian estimator when using a quadratic loss function is the posterior mean
(Christensen & Huffman 1985):
๐œƒ๐œƒ๏ฟฝ = ๐ธ๐ธ[๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ)] = โˆซ ๐œƒ๐œƒ๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ) ๐‘‘๐‘‘๐‘‘๐‘‘ = ๐œ‡๐œ‡๐œ‹๐œ‹
For more information on utility, loss functions and estimators in a Bayesian context see
(Berger 1993).
1.4.7. Confidence Intervals
Assuming a random variable is distributed by a given distribution, there exists the true
๏ฟฝ , may or
distribution parameters, ๐œฝ๐œฝ๐ŸŽ๐ŸŽ , which is unknown. The parameter point estimates, ๐œฝ๐œฝ
may not be close to the true parameter values. Confidence intervals provide the range
over which the true parameter values may exist with a certain level of confidence.
Confidence intervals only quantify uncertainty due to sampling error arising from a limited
number of samples. Uncertainty due to incorrect model selection or incorrect assumptions
is not included. (Meeker & Escobar 1998, p.49)
Increasing the desired confidence ๐›พ๐›พ results in an increased confidence interval. Increasing
the sample size generally decreases the confidence interval. There are many methods to
calculate confidence intervals. Some popular methods are:
31
โ€ข
Exact Confidence Intervals. It may be mathematically shown that the
parameter of a distribution itself follows a distribution. In such cases exact
confidence intervals can be derived. This is only the case in very few
distributions.
โ€ข
Fisher Information Matrix (Nelson 1990, p.292). For a large number of
samples, the asymptotic normal property can be used to estimate confidence
intervals:
lim ๐œƒ๐œƒ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œƒ๐œƒ0 , [๐‘›๐‘›๐‘›๐‘›(๐œƒ๐œƒ0 )]โˆ’1 )
๐‘›๐‘›โ†’โˆž
Combining this with the asymptotic property ๐œƒ๐œƒ๏ฟฝ โ†’ ๐œƒ๐œƒ0 as ๐‘›๐‘› โ†’ โˆž gives the following
estimate for the distribution of ๐œƒ๐œƒ๏ฟฝ:
โˆ’1
lim ๐œƒ๐œƒ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œƒ๐œƒ๏ฟฝ, ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ
๐‘›๐‘›โ†’โˆž
100๐›พ๐›พ% approximate confidence intervals are calculated using percentiles of the
normal distribution. If the range of ๐œƒ๐œƒ is unbounded (โˆ’โˆž, โˆž) the approximate two
sided confidence intervals are:
1 + ๐›พ๐›พ
โˆ’1
๐œƒ๐œƒ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ โˆ’ ฮฆโˆ’1 ๏ฟฝ
๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝ
2
1 + ๐›พ๐›พ
โˆ’1
๏ฟฝ๏ฟฝ๏ฟฝ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ + ฮฆโˆ’1 ๏ฟฝ
๐œƒ๐œƒ
๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝ
2
If the range of ๐œƒ๐œƒ is (0, โˆž) the approximate two sided confidence intervals are:
โŽก๐›ท๐›ท โˆ’1 ๏ฟฝ1 + ๐›พ๐›พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝโˆ’1 โŽค
2
โŽฅ
๐œƒ๐œƒ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ. exp โŽข
โˆ’๐œƒ๐œƒ๏ฟฝ
โŽข
โŽฅ
โŽฃ
โŽฆ
โŽก๐›ท๐›ท โˆ’1 ๏ฟฝ1 + ๐›พ๐›พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝโˆ’1 โŽค
2
๏ฟฝ๏ฟฝ๏ฟฝ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ. exp โŽข
โŽฅ
๐œƒ๐œƒ
๐œƒ๐œƒ๏ฟฝ
โŽข
โŽฅ
โŽฃ
โŽฆ
If the range of ๐œƒ๐œƒ is (0,1) the approximate two sided confidence intervals are:
โˆ’1
โŽก๐›ท๐›ท โˆ’1 ๏ฟฝ1 + ๐›พ๐›พ๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝโˆ’1 โŽคโŽซ
โŽง
2
โŽฅ
๐œƒ๐œƒ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ. ๐œƒ๐œƒ๏ฟฝ + (1 โˆ’ ๐œƒ๐œƒ๏ฟฝ ) exp โŽข
๐œƒ๐œƒ๏ฟฝ(1 โˆ’ ๐œƒ๐œƒ๏ฟฝ )
โŽจ
โŽข
โŽฅโŽฌ
โŽฉ
โŽฃ
โŽฆโŽญ
โˆ’1
โˆ’1
1
+
๐›พ๐›พ
โˆ’1
โŽก๐›ท๐›ท ๏ฟฝ
โŽง
๏ฟฝ ๏ฟฝ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ๏ฟฝ โŽคโŽซ
2
๏ฟฝ๏ฟฝ๏ฟฝ๐›พ๐›พ = ๐œƒ๐œƒ๏ฟฝ. ๐œƒ๐œƒ๏ฟฝ + (1 โˆ’ ๐œƒ๐œƒ๏ฟฝ ) exp โŽข
โŽฅ
๐œƒ๐œƒ
โˆ’๐œƒ๐œƒ๏ฟฝ(1 โˆ’ ๐œƒ๐œƒ๏ฟฝ )
โŽจ
โŽข
โŽฅโŽฌ
โŽฉ
โŽฃ
โŽฆโŽญ
The advantage of this method is it can be calculated for all distributions and is
easy to calculate. The disadvantage is that the assumption of a normal
distribution is asymptotic and so sufficient data is required for the confidence
interval estimate to be accurate. The number of samples needed for an accurate
estimate changes from distribution to distribution. It also produces symmetrical
confidence intervals which may be very inaccurate. For more information see
(Nelson 1990, p.292).
Parameter Est
Introduction
Parameter Est
32
Probability Distributions Used in Reliability Engineering
โ€ข
Likelihood Ratio Intervals (Nelson 1990, p.292). The test statistic for the
likelihood ratio is:
๐ท๐ท = 2[ฮ›๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ โˆ’ ฮ›(๐œƒ๐œƒ)]
๐ท๐ท is approximately Chi-Square distributed with one degree of freedom.
๐ท๐ท = 2๏ฟฝฮ›๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ โˆ’ ฮ›(๐œƒ๐œƒ)๏ฟฝ โ‰ค ๐œ’๐œ’ 2 (๐›พ๐›พ; 1)
Where ๐›พ๐›พ is the 100๐›พ๐›พ% confidence interval for ๐œƒ๐œƒ. The two sided confidence limits
๐œƒ๐œƒ๐›พ๐›พ and ๏ฟฝ๏ฟฝ๏ฟฝ
๐œƒ๐œƒ๐›พ๐›พ are calculated by solving:
ฮ›(๐œƒ๐œƒ) = ฮ›๏ฟฝ๐œƒ๐œƒ๏ฟฝ๏ฟฝ โˆ’
๐œ’๐œ’ 2 (๐›พ๐›พ; 1)
2
The limits are normally solved numerically. The likelihood ratio intervals are
always within the limits of the parameter and gives asymmetrical confidence
limits. It is much more accurate than the Fisher information matrix method
particularly for one sided limits although it is more complicated to calculate. This
method must be solved numerically and so will not be discussed further in this
book.
โ€ข
Bayesian Confidence Intervals. In Bayesian statistics the uncertainty of a
parameter, ๐œƒ๐œƒ, is quantified as a distribution ๐œ‹๐œ‹(๐œƒ๐œƒ). Therefore the two sided 100๐›พ๐›พ%
confidence intervals are found by solving:
๐œƒ๐œƒ๐›พ๐›พ
1 โˆ’ ๐›พ๐›พ
= ๏ฟฝ ๐œ‹๐œ‹(๐œƒ๐œƒ) ๐‘‘๐‘‘๐‘‘๐‘‘ ,
2
โˆ’โˆž
โˆž
1 + ๐›พ๐›พ
= ๏ฟฝ ๐œ‹๐œ‹(๐œƒ๐œƒ) ๐‘‘๐‘‘๐‘‘๐‘‘
2
๐œƒ๐œƒ๐›พ๐›พ
Other methods exist to calculate approximate confidence intervals. A summary of some
techniques used in reliability engineering is included in (Lawless 2002).
Introduction
๐‘‹๐‘‹๐ต๐ต =
๐‘‹๐‘‹๐‘๐‘ = ln(๐‘‹๐‘‹๐ฟ๐ฟ )
1
XC =
XN(k) โˆ’ µ 2
๏ฟฝ๏ฟฝ
๏ฟฝ
ฯƒ
๐‘‹๐‘‹๐บ๐บ1
๐‘‹๐‘‹๐บ๐บ1 + ๐‘‹๐‘‹๐บ๐บ2
๐›ผ๐›ผ = ๐‘˜๐‘˜1 , ๐›ฝ๐›ฝ = ๐‘˜๐‘˜2
๐‘›๐‘› = 1
๐‘‹๐‘‹๐ต๐ต =
Binomial
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘›๐‘›, ๐‘๐‘)
lim ๐‘›๐‘› โ†’ โˆž
๐‘›๐‘›๐‘›๐‘› = ๐œ‡๐œ‡
V
Chi-square
๐œ’๐œ’ 2(๐‘ฃ๐‘ฃ)
๐‘‹๐‘‹๐น๐น =
Poisson
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œ‡๐œ‡)
Gamma
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
Student t
๐‘ก๐‘ก(๐‘ฃ๐‘ฃ)
๐‘ฃ๐‘ฃ = 2, ๐œŽ๐œŽ
๐‘‹๐‘‹๐ธ๐ธ = ln(๐‘‹๐‘‹๐‘ƒ๐‘ƒ โ„๐œƒ๐œƒ)
Pareto
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ
1๏ฟฝ
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ = 2
๐œŽ๐œŽ = โˆš2/๐›ผ๐›ผ
Exponential
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐›ฝ๐›ฝ = 1
Rayleigh
๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐œŽ๐œŽ)
๐‘‹๐‘‹๐‘…๐‘… = ๏ฟฝ๐‘‹๐‘‹๐ธ๐ธ
๐‘‹๐‘‹๐‘Š๐‘Š = ๐‘‹๐‘‹๐ธ๐ธ
๐‘ฃ๐‘ฃ = 2, ๐›ผ๐›ผ = โˆš2, ๐›ฝ๐›ฝ = 2
๐‘‹๐‘‹๐น๐น = ๐‘‹๐‘‹๐‘‡๐‘‡2
๐‘›๐‘›1 = 1, ๐‘›๐‘›2 = ๐‘ฃ๐‘ฃ
๐›ผ๐›ผ = 1, ๐›ฝ๐›ฝ = 1
๐‘˜๐‘˜ = 1
๐‘‹๐‘‹๐บ๐บ = ๐‘‹๐‘‹๐ธ๐ธ1 + โ‹ฏ + ๐‘‹๐‘‹๐ธ๐ธ๐ธ๐ธ
๐‘‹๐‘‹๐ถ๐ถ1 ๐‘›๐‘›2
๐‘‹๐‘‹๐ถ๐ถ2 ๐‘›๐‘›1
Xฯ‡2 = ๐‘‹๐‘‹๐œ’๐œ’2
๐‘“๐‘“๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ†๐œ†๐œ†๐œ†) =
๐น๐น๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜ + 1, ๐œ†๐œ†)
โˆ’๐น๐น๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†)
F
๐น๐น(๐‘›๐‘›1 , ๐‘›๐‘›2 )
Chi
๐œ’๐œ’(๐‘ฃ๐‘ฃ)
Beta
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘‹๐‘‹๐‘†๐‘†(1) โ‰ค โ‹ฏ โ‰ค ๐‘‹๐‘‹๐‘†๐‘†(๐‘›๐‘›) , ๐‘‹๐‘‹๐‘†๐‘†(๐‘Ÿ๐‘Ÿ) ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต
๐‘‹๐‘‹๐บ๐บ = ๏ฟฝ ๐‘‹๐‘‹๐ต๐ต๐ต๐ต
๐‘›๐‘› (๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–)
1
๐›ผ๐›ผ = 2๐‘ฃ๐‘ฃ1 , ๐›ฝ๐›ฝ = 2๐‘ฃ๐‘ฃ2
Bernoulli
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘)
Normal
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
XC = |๐‘ฟ๐‘ฟ๐‘ต๐‘ต |
๐‘‹๐‘‹๐ถ๐ถ1
,
๐‘‹๐‘‹๐ถ๐ถ1 + ๐‘‹๐‘‹๐ถ๐ถ2
Special Case
Weibull
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘‹๐‘‹๐‘†๐‘† = exp(โˆ’๐œ†๐œ†๐‘‹๐‘‹๐ธ๐ธ )
Standard Uniform
๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(0,1)
๐‘Ž๐‘Ž = 0
๐‘๐‘ = 1
Uniform
๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘Ž๐‘Ž, ๐‘๐‘)
Figure 7: Relationships between common distributions (Leemis & McQueston 2008).
Many relations are not included such as central limit convergence to the normal
distribution and many transforms which would have made the figure unreadable. For
further details refer to individual sections and (Leemis & McQueston 2008).
Related Dist
Transformation
1.5. Related Distributions
Lognormal
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 )
33
Supporting Func
34
Probability Distributions Used in Reliability Engineering
1.6. Supporting Functions
1.6.1. Beta Function ๐๐(๐’™๐’™, ๐’š๐’š)
B(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the Beta function and is the Euler integral of the first kind.
Where ๐‘ฅ๐‘ฅ > 0 and ๐‘ฆ๐‘ฆ > 0.
1
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = ๏ฟฝ ๐‘ข๐‘ข ๐‘ฅ๐‘ฅโˆ’1 (1 โˆ’ ๐‘ข๐‘ข)๐‘ฆ๐‘ฆโˆ’1 ๐‘‘๐‘‘๐‘‘๐‘‘
0
Relationships:
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = ๐ต๐ต(๐‘ฆ๐‘ฆ, ๐‘ฅ๐‘ฅ)
ฮ“(๐‘ฅ๐‘ฅ)ฮ“(๐‘ฆ๐‘ฆ)
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) =
ฮ“(๐‘ฅ๐‘ฅ + ๐‘ฆ๐‘ฆ)
โˆž ๐‘›๐‘› โˆ’ ๐‘ฆ๐‘ฆ
๏ฟฝ
๏ฟฝ
๐‘›๐‘›
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = ๏ฟฝ
๐‘ฅ๐‘ฅ + ๐‘›๐‘›
๐‘›๐‘›=0
More formulas, definitions and special values can be found in the Digital Library of
Mathematical Functions on the National Institute of Standards and Technology (NIST)
website, http://dlmf.nist.gov.
1.6.2. Incomplete Beta Function ๐‘ฉ๐‘ฉ๐’•๐’• (๐’•๐’•; ๐’™๐’™, ๐’š๐’š)
๐ต๐ต๐‘ก๐‘ก (๐‘ก๐‘ก; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the incomplete Beta function expressed by:
๐‘ก๐‘ก
๐ต๐ต๐‘ก๐‘ก (๐‘ก๐‘ก; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = ๏ฟฝ ๐‘ข๐‘ข ๐‘ฅ๐‘ฅโˆ’1 (1 โˆ’ ๐‘ข๐‘ข)๐‘ฆ๐‘ฆโˆ’1 ๐‘‘๐‘‘๐‘‘๐‘‘
0
1.6.3. Regularized Incomplete Beta Function ๐‘ฐ๐‘ฐ๐’•๐’• (๐’•๐’•; ๐’™๐’™, ๐’š๐’š)
๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the regularized incomplete Beta function:
๐ต๐ต๐‘ก๐‘ก (t ; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ)
๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) =
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ)
๐‘ฅ๐‘ฅ+๐‘ฆ๐‘ฆโˆ’1
Properties:
= ๏ฟฝ
๐‘—๐‘—=๐‘ฅ๐‘ฅ
(๐‘ฅ๐‘ฅ + ๐‘ฆ๐‘ฆ โˆ’ 1)!
. ๐‘ก๐‘ก ๐‘—๐‘— (1 โˆ’ ๐‘ก๐‘ก)๐‘ฅ๐‘ฅ+๐‘ฆ๐‘ฆโˆ’1โˆ’๐‘—๐‘—
๐‘—๐‘—! (๐‘ฅ๐‘ฅ + ๐‘ฆ๐‘ฆ โˆ’ 1 โˆ’ ๐‘—๐‘—)!
๐ผ๐ผ0 (0; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = 0
๐ผ๐ผ1 (1; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = 1
๐ผ๐ผ๐‘ก๐‘ก (t; ๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) = 1 โˆ’ ๐ผ๐ผ(1 โˆ’ t; ๐‘ฆ๐‘ฆ, ๐‘ฅ๐‘ฅ)
1.6.4. Complete Gamma Function ๐šช๐šช(๐’Œ๐’Œ)
ฮ“(๐‘˜๐‘˜) is a generalization of the factorial function ๐‘˜๐‘˜! to include non-integer values.
For ๐‘˜๐‘˜ > 0
35
โˆž
ฮ“(๐‘˜๐‘˜) = ๏ฟฝ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ก๐‘ก ๐‘‘๐‘‘๐‘‘๐‘‘
0
โˆž
โˆž
= ๏ฟฝ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ก๐‘ก ๏ฟฝ + (๐‘˜๐‘˜ โˆ’ 1) ๏ฟฝ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’2 ๐‘’๐‘’ โˆ’๐‘ก๐‘ก ๐‘‘๐‘‘๐‘‘๐‘‘
0
โˆž
= (๐‘˜๐‘˜ โˆ’ 1) ๏ฟฝ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’2 ๐‘’๐‘’ โˆ’๐‘ก๐‘ก ๐‘‘๐‘‘๐‘‘๐‘‘
0
0
When ๐‘˜๐‘˜ is an integer:
Special values:
= (๐‘˜๐‘˜ โˆ’ 1)ฮ“(๐‘˜๐‘˜ โˆ’ 1)
ฮ“(๐‘˜๐‘˜) = (๐‘˜๐‘˜ โˆ’ 1)!
ฮ“(1) = 1
ฮ“(2) = 1
1
ฮ“ ๏ฟฝ ๏ฟฝ = โˆš๐œ‹๐œ‹
2
Relation to the incomplete gamma functions:
ฮ“(๐‘˜๐‘˜) = ฮ“(๐‘˜๐‘˜, ๐‘ก๐‘ก) + ๐›พ๐›พ(๐‘˜๐‘˜, ๐‘ก๐‘ก)
More formulas, definitions and special values can be found in the Digital Library of
Mathematical Functions on the National Institute of Standards and Technology (NIST)
website, http://dlmf.nist.gov.
1.6.5. Upper Incomplete Gamma Function ๐šช๐šช(๐’Œ๐’Œ, ๐’•๐’•)
For ๐‘˜๐‘˜ > 0
When ๐‘˜๐‘˜ is an integer:
โˆž
ฮ“(๐‘˜๐‘˜, ๐‘ก๐‘ก) = ๏ฟฝ ๐‘ฅ๐‘ฅ ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘ก
ฮ“(๐‘˜๐‘˜, ๐‘ก๐‘ก) = (๐‘˜๐‘˜
โˆ’ 1)! ๐‘’๐‘’ โˆ’๐‘ก๐‘ก
๐‘˜๐‘˜โˆ’1 ๐‘›๐‘›
๐‘ก๐‘ก
๏ฟฝ
๐‘›๐‘›=0
๐‘›๐‘›!
More formulas, definitions and special values can be found on the NIST website,
http://dlmf.nist.gov.
1.6.6. Lower Incomplete Gamma Function ๐›„๐›„(๐’Œ๐’Œ, ๐’•๐’•)
For ๐‘˜๐‘˜ > 0
When ๐‘˜๐‘˜ is an integer:
๐‘ก๐‘ก
ฮณ(๐‘˜๐‘˜, ๐‘ก๐‘ก) = ๏ฟฝ ๐‘ฅ๐‘ฅ ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘
0
๐‘˜๐‘˜โˆ’1 ๐‘›๐‘›
๐‘ก๐‘ก
ฮณ(๐‘˜๐‘˜, ๐‘ก๐‘ก) = (๐‘˜๐‘˜ โˆ’ 1)! ๏ฟฝ1 โˆ’ ๐‘’๐‘’ โˆ’๐‘ก๐‘ก ๏ฟฝ
๐‘›๐‘›=0
๐‘›๐‘›!
๏ฟฝ
Supporting Func
Introduction
Supporting Func
36
Probability Distributions Used in Reliability Engineering
More formulas, definitions and special values can be found on the NIST website,
http://dlmf.nist.gov.
1.6.7. Digamma Function ๐๐(๐’™๐’™)
๐œ“๐œ“(๐‘ฅ๐‘ฅ) is the digamma function defined as:
ฮ“ โ€ฒ (๐‘ฅ๐‘ฅ)
๐‘‘๐‘‘
๐œ“๐œ“(๐‘ฅ๐‘ฅ) =
ln[ฮ“(๐‘ฅ๐‘ฅ)] =
๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘ฅ๐‘ฅ > 0
ฮ“(๐‘ฅ๐‘ฅ)
๐‘‘๐‘‘๐‘‘๐‘‘
1.6.8. Trigamma Function ๐๐โ€ฒ(๐’™๐’™)
๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) is the trigamma function defined as:
โˆž
๐‘‘๐‘‘2
๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) = 2 ๐‘™๐‘™๐‘™๐‘™ฮ“(๐‘ฅ๐‘ฅ) = ๏ฟฝ(๐‘ฅ๐‘ฅ + ๐‘–๐‘–)โˆ’2
๐‘‘๐‘‘๐‘ฅ๐‘ฅ
๐‘–๐‘–=0
1.7. Referred Distributions
1.7.1. Inverse Gamma Distribution ๐‘ฐ๐‘ฐ๐‘ฐ๐‘ฐ(๐œถ๐œถ, ๐œท๐œท)
The pdf to the inverse gamma distribution is:
โˆ’๐›ฝ๐›ฝ
๐›ฝ๐›ฝ๐›ผ๐›ผ
. ๐‘’๐‘’ ๐‘ฅ๐‘ฅ . ๐ผ๐ผ๐‘ฅ๐‘ฅ (0, โˆž)
๐‘“๐‘“(๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) =
ฮ“(๐›ผ๐›ผ)๐‘ฅ๐‘ฅ ๐›ผ๐›ผ+1
With mean:
๐›ฝ๐›ฝ
๐œ‡๐œ‡ =
for ๐›ผ๐›ผ > 1
๐›ผ๐›ผ โˆ’ 1
1.7.2. Student T Distribution ๐‘ป๐‘ป(๐œถ๐œถ, ๐๐, ๐ˆ๐ˆ๐Ÿ๐Ÿ )
The pdf to the standard student t distribution with ๐œ‡๐œ‡ = 0, ๐œŽ๐œŽ 2 = 1 is:
๐›ผ๐›ผ+1
โˆ’
2
๐‘ฅ๐‘ฅ 2
ฮ“[(๐›ผ๐›ผ + 1)โ„2]
. ๏ฟฝ1 + ๏ฟฝ
๐‘“๐‘“(๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ) =
๐›ผ๐›ผ
โˆš๐›ผ๐›ผ๐›ผ๐›ผฮ“(๐›ผ๐›ผโ„2)
The generalized student t distribution is:
With mean
๐›ผ๐›ผ+1
โˆ’
2
(๐‘ฅ๐‘ฅ โˆ’ ๐œ‡๐œ‡)2
ฮ“[(๐›ผ๐›ผ + 1)โ„2]
๐‘“๐‘“(๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ, ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 ) =
. ๏ฟฝ1 +
๏ฟฝ
๐›ผ๐›ผ๐œŽ๐œŽ 2
๐œŽ๐œŽโˆš๐›ผ๐›ผ๐›ผ๐›ผฮ“(๐›ผ๐›ผโ„2)
1.7.3. F Distribution ๐‘ญ๐‘ญ(๐’๐’๐Ÿ๐Ÿ , ๐’๐’๐Ÿ๐Ÿ )
๐œ‡๐œ‡ = ๐œ‡๐œ‡
Also known as the Variance Ratio or Fisher-Snedecor distribution the pdf is:
With cdf:
๐‘“๐‘“(๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ) =
๐ผ๐ผ๐‘ก๐‘ก ๏ฟฝ
(๐‘›๐‘›1 ๐‘ฅ๐‘ฅ)๐‘›๐‘›1 . ๐‘›๐‘›2๐‘›๐‘›2
1
๏ฟฝ
.
{๐‘›๐‘› +๐‘›๐‘› }
๐‘›๐‘› ๐‘›๐‘›
๐‘ฅ๐‘ฅ๐‘ฅ๐‘ฅ ๏ฟฝ 1 , 2 ๏ฟฝ (๐‘›๐‘›1 ๐‘ฅ๐‘ฅ + ๐‘›๐‘›2 ) 1 2
2 2
๐‘›๐‘›1 ๐‘›๐‘›2
, ๏ฟฝ,
2 2
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐‘ก๐‘ก =
1.7.4. Chi-Square Distribution ๐Œ๐Œ๐Ÿ๐Ÿ (๐’—๐’—)
๐‘›๐‘›1 ๐‘ฅ๐‘ฅ
๐‘›๐‘›1 ๐‘ฅ๐‘ฅ + ๐‘›๐‘›2
The pdf to the chi-square distribution is:
With mean:
๐‘ฅ๐‘ฅ
๐‘ฅ๐‘ฅ (๐‘ฃ๐‘ฃโˆ’2)โ„2 ๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๏ฟฝโˆ’ ๏ฟฝ
2
๐‘“๐‘“(๐‘ฅ๐‘ฅ; ๐‘ฃ๐‘ฃ) =
๐‘ฃ๐‘ฃ
2๐‘ฃ๐‘ฃโ„2 ๐›ค๐›ค ๏ฟฝ ๏ฟฝ
2
๐œ‡๐œ‡ = ๐‘ฃ๐‘ฃ
37
Referred Dist
Introduction
Referred Dist
38
Probability Distributions Used in Reliability Engineering
1.7.5. Hypergeometric Distribution ๐‘ฏ๐‘ฏ๐’š๐’š๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘๐’‘(๐’Œ๐’Œ; ๐’๐’, ๐’Ž๐’Ž, ๐‘ต๐‘ต)
The hypergeometric distribution models probability of ๐‘˜๐‘˜ successes in ๐‘›๐‘› Bernoulli trials
from population ๐‘๐‘ containing ๐‘š๐‘š success without replacement. ๐‘๐‘ = ๐‘š๐‘š/๐‘๐‘. The pdf to the
hypergeometric distribution is:
With mean:
๐‘“๐‘“(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘š๐‘š, ๐‘๐‘) =
๐œ‡๐œ‡ =
๏ฟฝm
๏ฟฝ๏ฟฝNโˆ’m
๏ฟฝ
k
nโˆ’k
๐‘›๐‘›๐‘›๐‘›
๐‘๐‘
๏ฟฝN
๏ฟฝ
n
1.7.6. Wishart Distribution ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐‘พ๐’•๐’•๐’…๐’… (๐’™๐’™; ๐šบ๐šบ, ๐’๐’)
The Wishart distribution is the multivariate generalization of the gamma distribution. The
pdf is given as:
With mean:
๐‘“๐‘“๐‘‘๐‘‘ (๐’™๐’™; ๐šบ๐šบ, ๐‘›๐‘›) =
1
|๐ฑ๐ฑ|2(nโˆ’dโˆ’1)
2๐‘›๐‘›๐‘›๐‘› โ„2 |๐šบ๐šบ|nโ„2 ฮ“๐‘‘๐‘‘
1
โˆ’๐Ÿ๐Ÿ
๐‘›๐‘› exp ๏ฟฝโˆ’ 2 ๐‘ก๐‘ก๐‘ก๐‘ก๏ฟฝ๐’™๐’™ ๐šบ๐šบ๏ฟฝ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐๐ = ๐‘›๐‘›๐šบ๐šบ
39
1.8. Nomenclature and Notation
Functions are presented in the following form:
๐‘“๐‘“(๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ๐‘Ÿ ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ ; ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ |๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘” ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ)
๐‘›๐‘›
In continuous distributions the number of items under test = ๐‘›๐‘›๐‘“๐‘“ + ๐‘›๐‘›๐‘ ๐‘  + ๐‘›๐‘›๐ผ๐ผ .
In discrete distributions the total number of trials.
๐‘›๐‘›๐น๐น
The number of items which failed before the conclusion of the test.
๐‘›๐‘›๐ผ๐ผ
The number of items which have interval data
๐‘ก๐‘ก๐‘–๐‘–๐‘†๐‘†
The time at which a component survived to. The item may have been
removed from the test for a reason other than failure.
๐‘›๐‘›๐‘†๐‘†
The number of items which survived to the end of the test.
๐‘ก๐‘ก๐‘–๐‘–๐น๐น , ๐‘ก๐‘ก๐‘–๐‘–
The time at which a component fails.
๐‘ก๐‘ก๐‘–๐‘–๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ
The upper limit of a censored interval in which an item failed
๐‘ก๐‘ก๐ฟ๐ฟ
The lower truncated limit of sample.
๐‘ก๐‘ก ๐‘‡๐‘‡
Time on test = โˆ‘๐‘ก๐‘ก๐‘–๐‘– + โˆ‘๐‘ก๐‘ก๐‘ ๐‘ 
๐พ๐พ
Discrete random variable
๐‘˜๐‘˜
A discrete random variable with a known value
๐’™๐’™
A bold symbol denotes a vector or matrix
๐œƒ๐œƒ
Upper confidence interval for UOI
๐‘‹๐‘‹~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘
The random variable ๐‘‹๐‘‹ is distributed as a ๐‘‘๐‘‘-variate normal distribution.
๐‘ก๐‘ก๐‘–๐‘–๐ฟ๐ฟ๐ฟ๐ฟ
The lower limit of a censored interval in which an item failed
๐‘ก๐‘ก๐‘ˆ๐‘ˆ
The upper truncated limit of sample.
๐‘‹๐‘‹ or ๐‘‡๐‘‡
Continuous random variable (T is normally a random time)
๐‘ฅ๐‘ฅ or ๐‘ก๐‘ก
A continuous random variable with a known value
๐‘ฅ๐‘ฅ๏ฟฝ
The hat denotes an estimated value
๐œƒ๐œƒ
Generalized unknown of interest (UOI)
๐œƒ๐œƒ
Lower confidence interval for UOI
Nomenclature
Introduction
Life Distribution
40
Probability Distributions Used in Reliability Engineering
2. Common Life Distributions
Exponential Continuous Distribution
41
2.1. Exponential Continuous
Distribution
Exponential
Probability Density Function - f(t)
ฮป=0.5
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
ฮป=1
ฮป=2
ฮป=3
0
0.5
1
1.5
2
2.5
3
3.5
4
Cumulative Density Function - F(t)
1.00
0.80
ฮป=0.5
0.60
ฮป=1
0.40
ฮป=2
0.20
ฮป=3
0.00
0
0.5
1
1.5
2
2.5
3
3.5
4
1
1.5
2
2.5
3
3.5
4
Hazard Rate - h(t)
3.00
2.50
2.00
1.50
1.00
0.50
0.00
0
0.5
42
Common Life Distributions
Parameters & Description
Parameters
๐œ†๐œ†
Exponential
Limits
Function
Time Domain
Scale Parameter: Equal to the hazard
rate.
๐‘ก๐‘ก โ‰ฅ 0
๐‘“๐‘“(๐‘ก๐‘ก) = ๐œ†๐œ†eโˆ’ฮปt
PDF
๐น๐น(๐‘ก๐‘ก) = 1 โˆ’ eโˆ’ฮปt
CDF
R(t) = eโˆ’ฮปt
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
๐œ†๐œ† > 0
Residual
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = eโˆ’ฮปx
๐‘“๐‘“(๐‘ ๐‘ ) =
๐œ†๐œ†
,
๐œ†๐œ† + ๐‘ ๐‘ 
๐น๐น(๐‘ ๐‘ ) =
๐‘ ๐‘  > โˆ’๐œ†๐œ†
๐œ†๐œ†
๐‘ ๐‘ (๐œ†๐œ† + ๐‘ ๐‘ )
๐‘…๐‘…(๐‘ ๐‘ ) =
๐‘š๐‘š(๐‘ ๐‘ ) =
1
๐œ†๐œ† + ๐‘ ๐‘ 
1
๐œ†๐œ† + ๐‘ ๐‘ 
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
1
๐œ†๐œ†
๐‘ข๐‘ข(๐‘ ๐‘ ) =
1
๐œ†๐œ†๐œ†๐œ†
๐ป๐ป(๐‘ก๐‘ก) = ๐œ†๐œ†๐œ†๐œ†
๐ป๐ป(๐‘ ๐‘ ) =
๐œ†๐œ†
๐‘ ๐‘  2
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โ„Ž(๐‘ก๐‘ก) = ๐œ†๐œ†
Hazard Rate
Cumulative
Hazard Rate
Properties and Moments
โ„Ž(๐‘ ๐‘ ) =
๐œ†๐œ†
๐‘ ๐‘ 
๐‘™๐‘™๐‘™๐‘™(2)
๐œ†๐œ†
Median
Mode
0
Mean - 1st Raw Moment
Variance - 2nd Central Moment
Skewness - 3rd Central Moment
Excess kurtosis -
Laplace Domain
4th
Central Moment
Characteristic Function
100ฮฑ% Percentile Function
1
๐œ†๐œ†
1
๐œ†๐œ†2
2
6
๐‘–๐‘–๐‘–๐‘–
๐‘ก๐‘ก + ๐‘–๐‘–๐‘–๐‘–
1
๐‘ก๐‘ก๐›ผ๐›ผ = โˆ’ ln(1 โˆ’ ๐›ผ๐›ผ)
๐œ†๐œ†
Exponential Continuous Distribution
43
Parameter Estimation
Plotting Method
X-Axis
Y-Axis
๐‘ก๐‘ก๐‘–๐‘–
Likelihood Function
nF
Likelihood
Functions
nS
F
failures
survivors
when there is no interval data this reduces to:
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’
Point
Estimates
Fisher
Information
100๐›พ๐›พ%
Confidence
Interval
(excluding
interval data)
UI
LI
nF
interval failures
๐‘ก๐‘ก๐‘‡๐‘‡ = ๏ฟฝ t Fi + ๏ฟฝ t Si = ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘–๐‘–๐‘–๐‘– ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก
nS
nI
LI
RI
ฮ›(๐ธ๐ธ|๐œ†๐œ†) = r. ln(ฮป) โˆ’ ๏ฟฝ ฮปt Fi โˆ’ ๏ฟฝ ฮปt s + ๏ฟฝ ln ๏ฟฝeโˆ’ฮปti โˆ’ eโˆ’ฮปti ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
i=1
failures
survivors
when there is no interval data this reduces to:
ฮ›(๐ธ๐ธ|๐œ†๐œ†) = nF . ln(๐œ†๐œ†) โˆ’ ๐œ†๐œ†๐‘ก๐‘ก ๐‘‡๐‘‡
โˆ‚ฮ›
=0
โˆ‚ฮป
nI
S
๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) = ฮปnF ๏ฟฝ eโˆ’ฮป.ti . ๏ฟฝ eโˆ’ti . ๏ฟฝ ๏ฟฝeโˆ’ฮปti โˆ’ eโˆ’ฮปti ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1 ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
๐ฟ๐ฟ(๐ธ๐ธ|๐œ†๐œ†) = ๐œ†๐œ†nF ๐‘’๐‘’ โˆ’๐œ†๐œ†๐‘ก๐‘ก๐‘‡๐‘‡
Log-Likelihood
Functions
๐œ†๐œ†ฬ‚ = โˆ’๐‘š๐‘š
๐‘™๐‘™๐‘™๐‘™[1 โˆ’ ๐น๐น(๐‘ก๐‘ก๐‘–๐‘– )]
solve for ๐œ†๐œ† to get ๐œ†๐œ†ฬ‚:
nS
nF
interval failures
๐‘ก๐‘ก๐‘‡๐‘‡ = ๏ฟฝ t Fi + ๏ฟฝ t Si
where
nI
LI
RI
ฮปti
ฮปti
t LI
โˆ’ t RI
nF
i e
i e
โˆ’ ๏ฟฝ t Fi โˆ’ ๏ฟฝ t Si โˆ’ ๏ฟฝ ๏ฟฝ
๏ฟฝ=0
๐ฟ๐ฟ๐ฟ๐ฟ
๐‘…๐‘…๐‘…๐‘…
๐œ†๐œ†๐‘ก๐‘ก
๐œ†๐œ†๐‘ก๐‘ก
ฮป
๐‘’๐‘’ ๐‘–๐‘– โˆ’ ๐‘’๐‘’ ๐‘–๐‘–
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
i=1
failures
survivors
interval failures
When there is only complete and right-censored data the point estimate
is:
nF
๐œ†๐œ†ฬ‚ =
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐‘ก๐‘ก๐‘‡๐‘‡ = ๏ฟฝ t Fi + ๏ฟฝ t Si = ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘–๐‘–๐‘–๐‘– ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก
๐‘ก๐‘ก ๐‘‡๐‘‡
ฮปlower 2-Sided
๐ผ๐ผ(๐œ†๐œ†) =
1
๐œ†๐œ†
ฮปupper 2-Sided
ฮปupper 1-Sided
Type I (Time
Terminated)
๐œ’๐œ’ 21โˆ’ฮณ (2nF )
๏ฟฝ
๐œ’๐œ’ 21+ฮณ (2nF + 2)
2
(2nF + 2)
๐œ’๐œ’(ฮณ)
Type II (Failure
Terminated)
๐œ’๐œ’ 21โˆ’ฮณ (2nF )
๐œ’๐œ’ 21+ฮณ (2nF )
2
(2nF )
๐œ’๐œ’(ฮณ)
๏ฟฝ
๏ฟฝ
2
2๐‘ก๐‘ก ๐‘‡๐‘‡
2
๏ฟฝ
2๐‘ก๐‘ก ๐‘‡๐‘‡
๏ฟฝ
2
๏ฟฝ
๏ฟฝ
2
2๐‘ก๐‘ก ๐‘‡๐‘‡
๏ฟฝ
2๐‘ก๐‘ก ๐‘‡๐‘‡
2๐‘ก๐‘ก ๐‘‡๐‘‡
2๐‘ก๐‘ก๐‘‡๐‘‡
Exponential
Least
Mean
Square - ๐‘ฆ๐‘ฆ =
๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
44
Common Life Distributions
Exponential
2
๐œ’๐œ’(๐›ผ๐›ผ)
is the ๐›ผ๐›ผ percentile of the Chi-squared distribution. (Modarres et al.
1999, pp.151-152) Note: These confidence intervals are only valid for
complete and right-censored data or when approximations of interval
data are used (such as the median). They are exact confidence bounds
and therefore approximate methods such as use of the Fisher
information matrix need not be used.
Bayesian
Non-informative Priors ๐…๐…(๐€๐€)
(Yang and Berger 1998, p.6)
Type
Prior
Posterior
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Proper Prior
with limits ๐œ†๐œ† โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
Uniform Improper Prior
with limits ๐œ†๐œ† โˆˆ [0, โˆž)
Jeffreyโ€™s Prior
Truncated Gamma Distribution
For a โ‰ค ฮป โ‰ค b
๐‘๐‘. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + nF , t T )
1 โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0)
1
โˆš๐œ†๐œ†
Otherwise ๐œ‹๐œ‹(๐œ†๐œ†) = 0
1
โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(2, 0)
1
โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0)
๐œ†๐œ†
Novick and Hall
where
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + nF , t T )
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 12 + nF , t T )
when ๐œ†๐œ† โˆˆ [0, โˆž)
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; nF , t T )
when ๐œ†๐œ† โˆˆ [0, โˆž)
๐‘ก๐‘ก ๐‘‡๐‘‡ = โˆ‘ t Fi + โˆ‘ t Si = total time in test
Conjugate Priors
UOI
๐œ†๐œ†
from
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ฮป)
Example
Likelihood
Model
Exponential
Evidence
๐‘›๐‘›๐น๐น failures
in ๐‘ก๐‘ก ๐‘‡๐‘‡ unit of
time
Dist. of
UOI
Prior
Para
Posterior
Parameters
Gamma
๐‘˜๐‘˜0 , ฮ›0
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
ฮ› = ฮ›๐‘œ๐‘œ + ๐‘ก๐‘ก ๐‘‡๐‘‡
Description , Limitations and Uses
Three vehicle tires were run on a test area for 1000km have
punctures at the following distances:
Tire 1: No punctures
Tire 2: 400km, 900km
Tire 3: 200km
Punctures are a random failure with constant failure rate therefore
an exponential distribution would be appropriate. Due to an
exponential distribution being homogeneous in time, the renewal
process of the second tire failing twice with a repair can be
considered as two separate tires on test with single failures. See
example in section 1.1.6.
Total distance on test is 3 × 1000 = 3000km. Total number of
Exponential Continuous Distribution
45
failures is 3. Therefore using MLE the estimate of ๐œ†๐œ†:
nF
3
=
= 1E-3
๐‘ก๐‘ก ๐‘‡๐‘‡ 3000
With 90% confidence interval (distance terminated test):
2
2
(6)
(8)
๐œ’๐œ’(0.95)
๐œ’๐œ’(0.05)
= 0.272๐ธ๐ธ-3,
= 2.584๐ธ๐ธ-3๏ฟฝ
๏ฟฝ
6000
6000
A Bayesian point estimate using the Jeffery non-informative
improper prior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(12, 0), with posterior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 3.5, 3000) has
a point estimate:
3.5
๐œ†๐œ†ฬ‚ = E[๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 3.5, 3000)] =
= 1.16ฬ‡ E โˆ’ 3
3000
Characteristics
With 90% confidence interval using inverse Gamma cdf:
[๐น๐น๐บ๐บโˆ’1 (0.05) = 0.361๐ธ๐ธ-3,
๐น๐น๐บ๐บโˆ’1 (0.95) = 2.344๐ธ๐ธ-3]
Constant Failure Rate. The exponential distribution is defined by a
constant failure rate, ๐œ†๐œ†. This means the component is not subject to
wear or accumulation of damage as time increases.
๐’‡๐’‡(๐ŸŽ๐ŸŽ) = ๐€๐€. As can be seen, ๐œ†๐œ† is the initial value of the distribution.
Increases in ๐œ†๐œ† increase the probability density at ๐‘“๐‘“(0).
HPP. The exponential distribution is the time to failure distribution of
a single event in the Homogeneous Poisson Process (HPP).
Scaling property
Minimum property
๐‘‡๐‘‡~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
๐œ†๐œ†
๐‘Ž๐‘Ž๐‘Ž๐‘Ž~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐‘ก๐‘ก; ๏ฟฝ
๐‘Ž๐‘Ž
๐‘›๐‘›
min{๐‘‡๐‘‡1 , ๐‘‡๐‘‡2 , โ€ฆ , ๐‘‡๐‘‡๐‘›๐‘› }~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐‘ก๐‘ก; ๏ฟฝ ๐œ†๐œ†๐‘–๐‘– ๏ฟฝ
๐‘–๐‘–=1
Variate Generation property
ln(1 โˆ’ ๐‘ข๐‘ข)
๐น๐น โˆ’1 (๐‘ข๐‘ข) =
, 0 < ๐‘ข๐‘ข < 1
โˆ’๐œ†๐œ†
Memoryless property.
Pr(๐‘‡๐‘‡ > ๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ|๐‘‡๐‘‡ > ๐‘ก๐‘ก) = Pr(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ)
Properties from (Leemis & McQueston 2008).
Applications
No Wearout. The exponential distribution is used to model
occasions when there is no wearout or cumulative damage. It can be
used to approximate the failure rate in a componentโ€™s useful life
period (after burn in and before wear out).
Exponential
๐œ†๐œ†ฬ‚ =
46
Common Life Distributions
Homogeneous Poisson Process (HPP). The exponential
distribution is used to model the inter arrival times in a repairable
system or the arrival times in queuing models. See Poisson and
Gamma distribution for more detail.
Exponential
Electronic Components. Some electronic components such as
capacitors or integrated circuits have been found to follow an
exponential distribution. Early efforts at collecting reliability data
assumed a constant failure rate and therefore many reliability
handbooks only provide a failure rate estimates for components.
Random Shocks. It is common for the exponential distribution to
model the occurrence of random shocks An example is the failure of
a vehicle tire due to puncture from a nail (random shock). The
probability of failure in the next mile is independent of how many
miles the tire has travelled (memoryless). The probability of failure
when the tire is new is the same as when the tire is old (constant
failure rate).
In general component life distributions do not have a constant failure
rate, for example due to wear or early failures. Therefore the
exponential distribution is often inappropriate to model most life
distributions, particularly mechanical components.
Resources
Online:
http://www.weibull.com/LifeDataWeb/the_exponential_distribution.h
tm
http://mathworld.wolfram.com/ExponentialDistribution.html
http://en.wikipedia.org/wiki/Exponential_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Balakrishnan, N. & Basu, A.P., 1996. Exponential Distribution:
Theory, Methods and Applications 1st ed., CRC.
Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience.
Relationship to Other Distributions
2-Para
Exponential
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐›ฝ๐›ฝ)
Special Case:
Gamma
Distribution
Then
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†)
Let
1
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; µ = 0, ฮฒ = )
๐œ†๐œ†
๐‘‡๐‘‡1 โ€ฆ ๐‘‡๐‘‡๐‘˜๐‘˜ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‡๐‘‡๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐‘˜๐‘˜
๐‘‡๐‘‡๐‘ก๐‘ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
The gamma distribution is the probability density function of the sum
of k exponentially distributed time random variables sharing the
same constant rate of occurrence, ๐œ†๐œ†. This is a Homogeneous
Poisson Process.
Exponential Continuous Distribution
Special Case:
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; ๐‘˜๐‘˜ = 1, ๐œ†๐œ†)
Let
๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐พ๐พ + ๐‘‡๐‘‡๐พ๐พ+1 โ€ฆ
Then
๐พ๐พ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k; µ = ๐œ†๐œ†๐œ†๐œ†)
Poisson
Distribution
The Poisson distribution is the probability of observing exactly k
occurrences within a time interval [0, t] where the inter-arrival times
of each occurrence is exponentially distributed. This is a
Homogeneous Poisson Process.
Special Cases:
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k = 1; µ = ๐œ†๐œ†๐œ†๐œ†) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
Let
Weibull
Distribution
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Special Case:
Let
Geometric
Distribution
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜; ๐‘๐‘)
Rayleigh
Distribution
๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐‘ก๐‘ก; ๐›ผ๐›ผ)
Chi-square
๐œ’๐œ’ 2 (๐‘ฅ๐‘ฅ; ๐‘ฃ๐‘ฃ)
Pareto
Distribution
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก; ๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
Then
Then
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
โˆ’1
๐‘Œ๐‘Œ = X1/ฮฒ
๐‘Œ๐‘Œ~๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐›ผ๐›ผ = ๐œ†๐œ† ๐›ฝ๐›ฝ , ๐›ฝ๐›ฝ)
1
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†) = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š ๏ฟฝ๐‘ก๐‘ก; ๐›ผ๐›ผ = , ๐›ฝ๐›ฝ = 1๏ฟฝ
๐œ†๐œ†
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ = [๐‘‹๐‘‹],
๐‘Œ๐‘Œ ๐‘–๐‘–๐‘–๐‘– ๐‘ก๐‘กโ„Ž๐‘’๐‘’ ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘– ๐‘œ๐‘œ๐‘œ๐‘œ ๐‘‹๐‘‹
๐‘Œ๐‘Œ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
The geometric distribution is the discrete equivalent of the
continuous exponential distribution. The geometric distribution is
also memoryless.
Let
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
Then
Special Case:
Let
Then
๐‘Ž๐‘Ž๐‘›๐‘›๐‘›๐‘›
๐‘Œ๐‘Œ~๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐›ผ๐›ผ =
1
๐‘Œ๐‘Œ = โˆšX
โˆš๐œ†๐œ†
)
1
๐œ’๐œ’ 2 (๐‘ฅ๐‘ฅ; ๐‘ฃ๐‘ฃ = 2) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ ๏ฟฝ๐‘ฅ๐‘ฅ; ฮป = ๏ฟฝ
2
๐‘Œ๐‘Œ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ† = ๐›ผ๐›ผ)
๐‘‹๐‘‹ = ln(๐‘Œ๐‘Œโ„๐œƒ๐œƒ)
Exponential
๐‘‡๐‘‡1 , ๐‘‡๐‘‡2 โ€ฆ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
Given
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; ๐œ‡๐œ‡)
47
48
Common Life Distributions
Exponential
Let
Logistic
Distribution
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(µ , ๐‘ ๐‘ )
Then
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ† = 1)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ = ln ๏ฟฝ
๐‘Œ๐‘Œ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1)
(Hastings et al. 2000, p.127):
๐‘’๐‘’ โˆ’๐‘‹๐‘‹
๏ฟฝ
1 + ๐‘’๐‘’ โˆ’๐‘‹๐‘‹
Lognormal Continuous Distribution
49
2.2. Lognormal Continuous
Distribution
ฮผ'=0 ฯƒ'=1
ฮผ'=0.5 ฯƒ'=1
ฮผ'=1 ฯƒ'=1
ฮผ'=1.5 ฯƒ'=1
0.90
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
ฮผ'=1 ฯƒ'=1
ฮผ'=1 ฯƒ'=1
ฮผ'=1 ฯƒ'=1.5
ฮผ'=1 ฯƒ'=2
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
0
2
4
6
Cumulative Density Function - F(t)
0
2
4
0
2
4
0
2
4
0.90
1.00
0.80
0.80
0.70
0.60
0.60
0.50
0.40
0.40
0.30
0.20
0.20
0.00
0.00
0.10
0
2
4
6
Hazard Rate - h(t)
0.70
1.80
1.60
1.40
1.20
1.00
0.80
0.60
0.40
0.20
0.00
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
2
4
6
Lognormal
Probability Density Function - f(t)
50
Common Life Distributions
Lognormal
Parameters & Description
๐œ‡๐œ‡๐‘๐‘
Scale parameter: The mean of the
normally distributed ln(๐‘ฅ๐‘ฅ). This
parameter only determines the scale
and not the location as in a normal
distribution.
๐œ‡๐œ‡2
๐œ‡๐œ‡๐‘๐‘ = ln ๏ฟฝ
๏ฟฝ
2
๏ฟฝ๐œŽ๐œŽ + ๐œ‡๐œ‡2
โˆ’โˆž < ๐œ‡๐œ‡๐‘๐‘ < โˆž
Parameters
ฯƒ2N
Shape parameter: The standard
deviation of the normally distributed
ln(x). This parameter only determines
the shape and not the scale as in a
normal distribution.
๐œŽ๐œŽ 2 + ๐œ‡๐œ‡2
ฯƒ2N = ln ๏ฟฝ
๏ฟฝ
๐œ‡๐œ‡2
ฯƒ2N > 0
Limits
t>0
Distribution
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) =
PDF
1 ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘ 2
exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐œŽ๐œŽ๐‘๐‘
๐œŽ๐œŽ๐‘๐‘ ๐‘ก๐‘กโˆš2๐œ‹๐œ‹
1
=
1
๐‘™๐‘™๐‘™๐‘™(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐œ™๐œ™ ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘ . ๐‘ก๐‘ก
๐œŽ๐œŽ๐‘๐‘
where ๐œ™๐œ™ is the standard normal pdf.
๐‘ก๐‘ก
2
1
1 ln(๐‘ก๐‘ก โˆ— ) โˆ’ ๐œ‡๐œ‡๐‘๐‘
exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ ๐‘‘๐‘‘๐‘ก๐‘ก โˆ—
โˆ—
2
๐œŽ๐œŽ๐‘๐‘
๐œŽ๐œŽ๐‘๐‘ โˆš2๐œ‹๐œ‹ 0 ๐‘ก๐‘ก
where ๐‘ก๐‘ก โˆ— is the time variable over which the pdf is integrated.
๐น๐น(๐‘ก๐‘ก) =
1
๏ฟฝ
=
CDF
๐‘™๐‘™๐‘™๐‘™(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
1 1
+ erf ๏ฟฝ
๏ฟฝ
2 2
๐œŽ๐œŽ๐‘๐‘ โˆš2
ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๏ฟฝ
= ฮฆ๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
where ฮฆ is the standard normal cdf.
ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
R(t) = 1 โˆ’ ฮฆ ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Where
ln(๐‘ฅ๐‘ฅ + t) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๏ฟฝ
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ) 1 โˆ’ ฮฆ ๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
=
ln(t) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘…๐‘…(๐‘ก๐‘ก)
1 โˆ’ ฮฆ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
Lognormal Continuous Distribution
51
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
โ„Ž(๐‘ก๐‘ก) =
ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘ก๐‘ก. ๐œŽ๐œŽ๐‘๐‘ ๏ฟฝ1 โˆ’ ฮฆ ๏ฟฝ
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
๐œ™๐œ™ ๏ฟฝ
Hazard Rate
Cumulative
Hazard Rate
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ln[๐‘…๐‘…(๐‘ก๐‘ก)]
Properties and Moments
๐‘’๐‘’ (๐œ‡๐œ‡๐‘๐‘)
Median
2
๐‘’๐‘’ (๐œ‡๐œ‡๐‘๐‘โˆ’๐œŽ๐œŽ๐‘๐‘ )
Mode
Mean - 1st Raw Moment
Variance -
2nd
2
๏ฟฝ๐‘’๐‘’ ๐œŽ๐œŽ๐‘๐‘
Central Moment
Skewness - 3rd Central Moment
2
2
๐œŽ๐œŽ๐‘๐‘
๏ฟฝ
2
2
โˆ’ 1๏ฟฝ. ๐‘’๐‘’ 2๐œ‡๐œ‡๐‘๐‘ +๐œŽ๐œŽ๐‘๐‘
๏ฟฝ๐‘’๐‘’ ๐œŽ๐œŽ + 2๏ฟฝ. ๏ฟฝ๐‘’๐‘’ ๐œŽ๐œŽ โˆ’ 1
2
2
2
2
๐‘’๐‘’ 4๐œŽ๐œŽ๐‘๐‘ + 2๐‘’๐‘’ 3๐œŽ๐œŽ๐‘๐‘ + 3๐‘’๐‘’ 2๐œŽ๐œŽ๐‘๐‘ โˆ’ 3
Excess kurtosis - 4th Central Moment
Characteristic Function
๐‘’๐‘’
๏ฟฝ๐œ‡๐œ‡๐‘๐‘ +
Deriving a unique characteristic equation
is not trivial and complex series solutions
have been proposed. (Leipnik 1991)
100ฮฑ% Percentile Function
๐‘ก๐‘ก๐›ผ๐›ผ = e(๐œ‡๐œ‡๐‘๐‘+๐‘ง๐‘ง๐›ผ๐›ผ.๐œŽ๐œŽ๐‘๐‘)
where ๐‘ง๐‘ง๐›ผ๐›ผ is the 100pthof the standard
normal distribution
Parameter Estimation
๐‘ก๐‘กฮฑ = e(๐œ‡๐œ‡๐‘๐‘ +๐œŽ๐œŽ๐‘๐‘ฮฆ
โˆ’1 (๐›ผ๐›ผ))
Plotting Method
Least
Mean
Square
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
X-Axis
Y-Axis
ln(๐‘ก๐‘ก๐‘–๐‘– )
๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–[๐น๐น(๐‘ก๐‘ก๐‘–๐‘– )]
๐‘๐‘
๐œ‡๐œ‡๏ฟฝ
๐‘๐‘ = โˆ’
๐‘š๐‘š
1
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘ =
๐‘š๐‘š
Lognormal
Mean Residual
Life
โˆž
โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘…๐‘…(๐‘ก๐‘ก)
๐œŽ๐œŽ๐‘๐‘2 ๐‘ก๐‘ก
lim ๐‘ข๐‘ข(๐‘ก๐‘ก) โ‰ˆ
[1 + ๐‘œ๐‘œ(1)]
๐‘ก๐‘กโ†’โˆž
ln(๐‘ก๐‘ก) โˆ’ ๐œ‡๐œ‡๐‘๐‘
Where ๐‘œ๐‘œ(1) is Landau's notation. (Kleiber & Kotz 2003, p.114)
๐‘ข๐‘ข(๐‘ก๐‘ก) =
52
Common Life Distributions
Maximum Likelihood Function
Lognormal
Likelihood
Functions
nF
nS
nI
1
๐น๐น
๏ฟฝ
๏ฟฝ1 โˆ’ ฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘–๐‘†๐‘† ๏ฟฝ๏ฟฝ . ๏ฟฝ ๏ฟฝฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘–๐‘…๐‘…๐‘…๐‘… ๏ฟฝ โˆ’ ฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘–๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ
F ๐œ™๐œ™(๐‘ง๐‘ง๐‘–๐‘– ) . ๏ฟฝ
๐œŽ๐œŽ
.
t
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1 ๐‘๐‘ i
i=1
i=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
where
survivors
failures
ln(t xi ) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘ง๐‘ง๐‘–๐‘–๐‘ฅ๐‘ฅ = ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
nF
Log-Likelihood
Function
nI
failures
survivors
interval failures
ln(t xi ) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘ง๐‘ง๐‘–๐‘–๐‘ฅ๐‘ฅ = ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
solve for ๐œ‡๐œ‡๐‘๐‘ to get MLE ๐œ‡๐œ‡๏ฟฝ:
๐‘๐‘
nF
nS
ฯ•๏ฟฝziS ๏ฟฝ
โˆ‚ฮ›
โˆ’µN . NF
1
1
=
+
๏ฟฝ ln(t Fi ) +
๏ฟฝ
โˆ‚µN
ฯƒN
ฯƒN
ฯƒN
1 โˆ’ ฮฆ๏ฟฝziS ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
nI
where
failures
1 ฯ•๏ฟฝziRI ๏ฟฝ โˆ’ ฯ•๏ฟฝziLI ๏ฟฝ
โˆ’๏ฟฝ ๏ฟฝ
๏ฟฝ=0
ฯƒN ฮฆ๏ฟฝziRI ๏ฟฝ โˆ’ ฮฆ๏ฟฝziLI ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
survivors
interval failures
solve for ๐œŽ๐œŽ๐‘๐‘ to get ๐œŽ๐œŽ๏ฟฝ:
๐‘๐‘
ln(t xi ) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘ง๐‘ง๐‘–๐‘–๐‘ฅ๐‘ฅ = ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
nF
nS
ziS . ฯ•๏ฟฝziS ๏ฟฝ
โˆ‚ฮ›
โˆ’nF
1
1
2
=
+ 3 ๏ฟฝ๏ฟฝln(t Fi ) โˆ’ µN ๏ฟฝ +
๏ฟฝ
โˆ‚ฯƒN
ฯƒN
ฯƒN
ฯƒN
1 โˆ’ ฮฆ๏ฟฝziS ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
failures
1 ziRI . ฯ•๏ฟฝziRI ๏ฟฝ โˆ’ ziLI ฯ•๏ฟฝziLI ๏ฟฝ
โˆ’๏ฟฝ ๏ฟฝ
๏ฟฝ
ฯƒN
ฮฆ๏ฟฝziRI ๏ฟฝ โˆ’ ฮฆ๏ฟฝziLI ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
nI
where
MLE
Point
Estimates
i=1
+ ๏ฟฝ ln๏ฟฝฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘–๐‘…๐‘…๐‘…๐‘… ๏ฟฝ โˆ’ ฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘–๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
where
โˆ‚ฮ›
=0
โˆ‚ฯƒN
nS
1
๐น๐น
๐‘†๐‘†
ฮ›(µN , ฯƒN |E) = ๏ฟฝ ln ๏ฟฝ
๐น๐น ๐œ™๐œ™๏ฟฝ๐‘ง๐‘ง๐‘–๐‘– ๏ฟฝ๏ฟฝ + ๏ฟฝ ln๏ฟฝ1 โˆ’ ฮฆ๏ฟฝ๐‘ง๐‘ง๐‘–๐‘– ๏ฟฝ๏ฟฝ
๐œŽ๐œŽ
.
๐‘ก๐‘ก
๐‘๐‘
๐‘–๐‘–
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
โˆ‚ฮ›
=0
โˆ‚µN
interval failures
interval failures
ln(t xi ) โˆ’ ๐œ‡๐œ‡๐‘๐‘
๐‘ง๐‘ง๐‘–๐‘–๐‘ฅ๐‘ฅ = ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ๐‘๐‘
survivors
=0
When there is only complete failure data the point estimates can be
given as:
2
๐น๐น
โˆ‘ ln(๐‘ก๐‘ก๐‘–๐‘–๐น๐น )
๏ฟฝ2 = โˆ‘๏ฟฝln๏ฟฝ๐‘ก๐‘ก๐‘–๐‘– ๏ฟฝ โˆ’ ๐œ‡๐œ‡๏ฟฝ๐‘ก๐‘ก ๏ฟฝ
๐œ‡๐œ‡๏ฟฝ
=
ฯƒ
๐‘๐‘
N
nF
nF
Lognormal Continuous Distribution
53
Note: In almost all cases the MLE methods for a normal distribution can
be used by taking the ๐‘™๐‘™๐‘™๐‘™(๐‘‹๐‘‹). However Normal distribution estimation
methods cannot be used with interval data. (Johnson et al. 1994, p.220)
1
โŽก 2
๐œŽ๐œŽ
๐ผ๐ผ(๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 ) = โŽข ๐‘๐‘
โŽข
โŽฃ0
(Kleiber & Kotz 2003, p.119).
Fisher
Information
100๐›พ๐›พ%
Confidence
Intervals
1-Sided Lower
๐๐๐‘ต๐‘ต
๐ˆ๐ˆ๐Ÿ๐Ÿ๐‘ต๐‘ต
(for complete
data)
๐œ‡๐œ‡๏ฟฝ๐‘๐‘ โˆ’
0
2-Sided Lower
๐œŽ๐œŽ
๏ฟฝ
๐‘๐‘
๐‘ก๐‘กฮณ (nF โˆ’ 1)
โˆš๐‘›๐‘›๐น๐น
(๐‘›๐‘›๐น๐น โˆ’ 1)
2
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘ 2
๐œ’๐œ’๐›พ๐›พ (๐‘›๐‘›๐น๐น โˆ’ 1)
โŽค
โŽฅ
1 โŽฅ
โˆ’ 4โŽฆ
2๐œŽ๐œŽ
๐œ‡๐œ‡๏ฟฝ๐‘๐‘ โˆ’
2
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘
Lognormal
In most cases the unbiased estimators are used:
2
๐น๐น
โˆ‘ ln(๐‘ก๐‘ก๐‘–๐‘–๐น๐น )
๏ฟฝ2 = โˆ‘๏ฟฝln๏ฟฝ๐‘ก๐‘ก๐‘–๐‘– ๏ฟฝ โˆ’ ๐œ‡๐œ‡๏ฟฝ๐‘ก๐‘ก ๏ฟฝ
๐œ‡๐œ‡๏ฟฝ
=
ฯƒ
๐‘๐‘
N
nF
nF โˆ’ 1
๐œŽ๐œŽ
๏ฟฝ
๐‘๐‘
๐‘ก๐‘ก 1โˆ’ฮณ (nF โˆ’ 1)
โˆš๐‘›๐‘›๐น๐น ๏ฟฝ 2 ๏ฟฝ
(๐‘›๐‘›๐น๐น โˆ’ 1)
2
๐œ’๐œ’ 1+ฮณ (๐‘›๐‘›๐น๐น โˆ’ 1)
๏ฟฝ
๏ฟฝ
2
2-Sided Upper
๐œ‡๐œ‡๏ฟฝ๐‘๐‘ +
2
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘
๐œŽ๐œŽ
๏ฟฝ
๐‘๐‘
๐‘ก๐‘ก 1โˆ’ฮณ (nF โˆ’ 1)
โˆš๐‘›๐‘›๐น๐น ๏ฟฝ 2 ๏ฟฝ
(๐‘›๐‘›๐น๐น โˆ’ 1)
2
๐œ’๐œ’ 1โˆ’ฮณ (๐‘›๐‘›๐น๐น โˆ’ 1)
๏ฟฝ
๏ฟฝ
2
Where ๐‘ก๐‘กฮณ (nF โˆ’ 1) is the 100๐›พ๐›พth percentile of the t-distribution with ๐‘›๐‘›๐น๐น โˆ’ 1
degrees of freedom and ๐œ’๐œ’๐›พ๐›พ2 (๐‘›๐‘›๐น๐น โˆ’ 1) is the 100๐›พ๐›พth percentile of the ๐œ’๐œ’ 2 distribution with ๐‘›๐‘›๐น๐น โˆ’ 1 degrees of freedom. (Nelson 1982, pp.218-219)
1 Sided - Lower
๐๐
exp ๏ฟฝ๐œ‡๐œ‡๏ฟฝ๐‘๐‘ +
2 Sided
2
2
4
๐œŽ๐œŽ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘
๐‘๐‘
๐‘๐‘
โˆ’ ๐‘๐‘1โˆ’๐›ผ๐›ผ ๏ฟฝ +
๏ฟฝ
2
๐‘›๐‘›๐น๐น 2(๐‘›๐‘›๐น๐น โˆ’ 1)
exp ๏ฟฝ๐œ‡๐œ‡๏ฟฝ๐‘๐‘ +
2
2
4
๐œŽ๐œŽ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
๐‘๐‘
๐‘๐‘
๐‘๐‘
± ๐‘๐‘1โˆ’๐›ผ๐›ผ๏ฟฝ2๏ฟฝ +
๏ฟฝ
2
๐‘›๐‘›๐น๐น 2(๐‘›๐‘›๐น๐น โˆ’ 1)
These formulas are the Cox approximation for the confidence intervals
of the lognormal distribution mean where ๐‘๐‘๐‘๐‘ = ๐›ท๐›ทโˆ’1 (๐‘๐‘), the inverse of
the standard normal cdf. (Zhou & Gao 1997)
Zhou & Gao recommend using the parametric bootstrap method for
small sample sizes. (Angus 1994)
Bayesian
Type
Non-informative Priors when ๐ˆ๐ˆ๐Ÿ๐Ÿ๐‘ต๐‘ต is known, ๐…๐…๐ŸŽ๐ŸŽ (๐๐๐‘ต๐‘ต )
(Yang and Berger 1998, p.22)
Uniform Proper
Prior with limits
๐œ‡๐œ‡๐‘๐‘ โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
Prior
Posterior
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Truncated Normal Distribution
For a โ‰ค µN โ‰ค b
โˆ‘๐‘›๐‘›๐น๐น ln ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ฯƒ2N
๐‘๐‘. ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµN ; ๐‘–๐‘–=1
, ๏ฟฝ
๐‘›๐‘›๐น๐น
nF
Otherwise ๐œ‹๐œ‹(๐œ‡๐œ‡๐‘๐‘ ) = 0
54
Common Life Distributions
Lognormal
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµN ;
Non-informative Priors when ๐๐๐‘ต๐‘ต is known, ๐…๐…๐’๐’ (๐›”๐›”๐Ÿ๐Ÿ๐๐ )
(Yang and Berger 1998, p.23)
Type
Prior
Posterior
Uniform Proper
Prior with limits
๐œŽ๐œŽ๐‘๐‘2 โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Improper
Prior with limits
๐œŽ๐œŽ๐‘๐‘2 โˆˆ (0, โˆž)
1
Truncated Inverse Gamma Distribution
For a โ‰ค ฯƒ2N โ‰ค b
2
(๐‘›๐‘›๐น๐น โˆ’ 2) SN
๐‘๐‘. ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
, ๏ฟฝ
2
2
Otherwise ๐œ‹๐œ‹(๐œŽ๐œŽ๐‘๐‘2 ) = 0
2
(๐‘›๐‘›๐น๐น โˆ’ 2) SN
, ๏ฟฝ
2
2
See section 1.7.1
๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
1
๐œŽ๐œŽ๐‘๐‘2
Jefferyโ€™s,
Reference, MDIP
Prior
Type
๐น๐น
โˆ‘๐‘›๐‘›๐‘–๐‘–=1
ln ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ฯƒ2N
, ๏ฟฝ
๐‘›๐‘›๐น๐น
nF
when ๐œ‡๐œ‡๐‘๐‘ โˆˆ (โˆž, โˆž)
1
All
2
๐‘›๐‘›๐น๐น SN
, ๏ฟฝ
2 2
with limits ๐œŽ๐œŽ๐‘๐‘2 โˆˆ (0, โˆž)
See section 1.7.1
๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
Non-informative Priors when ๐๐๐‘ต๐‘ต and ๐ˆ๐ˆ๐Ÿ๐Ÿ๐‘ต๐‘ต are unknown, ๐…๐…๐’๐’ (๐๐๐‘ต๐‘ต , ฯƒ2N )
(Yang and Berger 1998, p.23)
Prior
Posterior
Improper Uniform
with limits:
๐œ‡๐œ‡๐‘๐‘ โˆˆ (โˆž, โˆž)
๐œŽ๐œŽ๐‘๐‘2 โˆˆ (0, โˆž)
1
Jefferyโ€™s Prior
1
๐œŽ๐œŽ๐‘๐‘4
Reference Prior
ordering {๐œ™๐œ™, ๐œŽ๐œŽ}
โˆ
๐œ‹๐œ‹๐‘œ๐‘œ (๐œ™๐œ™, ๐œŽ๐œŽ๐‘๐‘2 )
1
๐œŽ๐œŽ๐‘๐‘ ๏ฟฝ2 + ๐œ™๐œ™ 2
where
๐œ™๐œ™ = ๐œ‡๐œ‡๐‘๐‘ /๐œŽ๐œŽ๐‘๐‘
2
SN
๏ฟฝ
๐‘›๐‘›๐น๐น (๐‘›๐‘›๐น๐น โˆ’ 3)
See section 1.7.2
2
(๐‘›๐‘›๐น๐น โˆ’ 3) SN
๐œ‹๐œ‹(ฯƒ2N |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
, ๏ฟฝ
2
2
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡๐‘๐‘ |๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµN ; ๐‘›๐‘›๐น๐น โˆ’ 3, ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘ก๐‘ก๐‘๐‘ ,
S2
๏ฟฝ
๐‘›๐‘›๐น๐น (๐‘›๐‘›๐น๐น + 1)
when ๐œ‡๐œ‡๐‘๐‘ โˆˆ (โˆž, โˆž)
See section 1.7.2
2
(๐‘›๐‘›๐น๐น + 1) SN
2
, ๏ฟฝ
๐œ‹๐œ‹(ฯƒN |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
2
2
when ๐œŽ๐œŽ๐‘๐‘2 โˆˆ (0, โˆž)
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡๐‘๐‘ |๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµN ; ๐‘๐‘ ๐น๐น + 1, ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘ก๐‘ก๐‘๐‘ ,
No closed form
Lognormal Continuous Distribution
1
๐œŽ๐œŽ๐‘๐‘
Reference where
๐œ‡๐œ‡ and ๐œŽ๐œŽ 2 are
separate groups.
2
SN
๏ฟฝ
nF (nF โˆ’ 1)
when ๐œ‡๐œ‡๐‘๐‘ โˆˆ (โˆž, โˆž)
See section 1.7.2
(๐‘›๐‘›๐น๐น โˆ’ 1) ๐‘†๐‘†๐‘๐‘2
๐œ‹๐œ‹(ฯƒ2N |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2N ;
, ๏ฟฝ
2
2
when ๐œŽ๐œŽ๐‘๐‘2 โˆˆ (0, โˆž)
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡๐‘๐‘ |๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµN ; ๐‘๐‘ ๐น๐น โˆ’ 1, ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘ก๐‘ก๐‘๐‘ ,
MDIP Prior
๐‘†๐‘†๐‘๐‘2
UOI
๐‘›๐‘›๐น๐น
= ๏ฟฝ(ln ๐‘ก๐‘ก๐‘–๐‘– โˆ’
๐‘–๐‘–=1
Likelihood
Model
๐œŽ๐œŽ๐‘๐‘2
from
Lognormal
with known
๐œ‡๐œ‡๐‘๐‘
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 )
๐œ‡๐œ‡๐‘๐‘
from
Lognormal
with known
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 )
๐œŽ๐œŽ๐‘๐‘2
๏ฟฝ๏ฟฝ๏ฟฝ)
๐‘ก๐‘ก๐‘๐‘ 2
and
๐‘›๐‘›๐น๐น
1
๏ฟฝ๏ฟฝ๏ฟฝ
๐‘ก๐‘ก๐‘๐‘ = ๏ฟฝ ln ๐‘ก๐‘ก๐‘–๐‘–
nF
๐‘–๐‘–=1
Conjugate Priors
Evidence
๐‘›๐‘›๐น๐น
failures
at times
๐‘ก๐‘ก๐‘–๐‘–
Dist. of
UOI
Gamma
๐‘›๐‘›๐น๐น
failures
at times
๐‘ก๐‘ก๐‘–๐‘–
Normal
Prior
Para
Posterior Parameters
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น /2
๐‘›๐‘›๐น๐น
๐‘˜๐‘˜0 , ๐œ†๐œ†0
1
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๏ฟฝ(ln ๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๐‘๐‘ )2
2
๐‘–๐‘–=1
๐œ‡๐œ‡๐‘œ๐‘œ , ๐œŽ๐œŽ๐‘œ๐‘œ2
ฯƒ2 =
Description , Limitations and Uses
Example
๐‘›๐‘›
๐น๐น
ln(๐‘ก๐‘ก๐‘–๐‘– )
µ0 โˆ‘๐‘–๐‘–=1
2+
ฯƒ0
๐œŽ๐œŽ๐‘๐‘2
๐œ‡๐œ‡ =
1 ๐‘›๐‘›๐น๐น
+
๐œŽ๐œŽ02 ๐œŽ๐œŽ๐‘๐‘2
1
1 nF
+
ฯƒ20 ๐œŽ๐œŽ๐‘๐‘2
5 components are put on a test with the following failure times:
98, 116, 2485, 2526, , 2920 hours
Taking the natural log of these failure times allows us to use a normal
distribution to approximate the parameters. ln(๐‘ก๐‘ก๐‘–๐‘– ):
4.590, 4.752, 7.979, 7.818, 7.834 ๐‘™๐‘™๐‘™๐‘™(hours)
MLE Estimates are:
โˆ‘ ln๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ 32.974
=
= 6.595
nF
5
2
โˆ‘๏ฟฝln๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ โˆ’ ๐œ‡๐œ‡๏ฟฝ๐‘ก๐‘ก ๏ฟฝ
2
๐œŽ๐œŽ๏ฟฝ
= 3.091
๐‘๐‘ =
nF โˆ’ 1
๐œ‡๐œ‡๏ฟฝ
๐‘๐‘ =
90% confidence interval for ๐œ‡๐œ‡๐‘๐‘ :
๐œŽ๐œŽ
๏ฟฝ๐‘๐‘
๏ฟฝ๐œ‡๐œ‡๏ฟฝ๐‘๐‘ โˆ’
๐‘ก๐‘ก{0.95} (4),
โˆš4
๐œ‡๐œ‡
๏ฟฝ๐‘๐‘ +
๐œŽ๐œŽ
๏ฟฝ๐‘๐‘
โˆš4
๐‘ก๐‘ก{0.95} (4)๏ฟฝ
Lognormal
where
55
56
Common Life Distributions
[4.721, 8.469]
Lognormal
90% confidence interval for ๐œŽ๐œŽ๐‘๐‘2 :
4
4
2
2
,
๐œŽ๐œŽ๏ฟฝ
๏ฟฝ
๏ฟฝ๐œŽ๐œŽ๏ฟฝ
๐‘๐‘ 2
๐‘๐‘ 2
๐œ’๐œ’{0.95} (4)
๐œ’๐œ’{0.05} (4)
[1.303, 17.396]
A Bayesian point estimate using the Jeffery non-informative improper
prior 1โ„๐œŽ๐œŽ๐‘๐‘4 with posterior for ๐œ‡๐œ‡๐‘๐‘ ~๐‘‡๐‘‡(6, 6.595, 0.412 ) and ๐œŽ๐œŽ๐‘๐‘2 ~๐ผ๐ผ๐ผ๐ผ(3,
6.182) has a point estimates:
๐œ‡๐œ‡๏ฟฝ๐‘๐‘ = E[๐‘‡๐‘‡(6,6.595,0.412 )] = µ = 6.595
6.182
2
๐œŽ๐œŽ๏ฟฝ
= 3.091
๐‘๐‘ = E[๐ผ๐ผ๐ผ๐ผ(3,6.182)] =
2
Characteristics
With 90% confidence intervals:
๐œ‡๐œ‡๐‘๐‘
[๐น๐น๐‘‡๐‘‡โˆ’1 (0.05) = 5.348,
2
๐œŽ๐œŽ๐‘๐‘
[1/๐น๐น๐บ๐บโˆ’1 (0.95) = 0.982,
๐น๐น๐‘‡๐‘‡โˆ’1 (0.95) = 7.842]
1/๐น๐น๐บ๐บโˆ’1 (0.05) = 7.560]
๐๐๐‘ต๐‘ต Characteristics. ๐œ‡๐œ‡๐‘๐‘ determines the scale and not the location
as in a normal distribution. The distribution if fixed at f(0)=0 and an
increase in the scale parameter stretches the distribution across the
x-axis. This has the effect of increasing the mode, mean and median
of the distribution.
๐ˆ๐ˆ๐‘ต๐‘ต Characteristics. ๐œŽ๐œŽ๐‘๐‘ determines the shape and not the scale as
in a normal distribution. For values of ฯƒN > 1 the distribution rises very
sharply at the beginning and decreases with a shape similar to an
Exponential or Weibull with 0 < ๐›ฝ๐›ฝ < 1. As ฯƒN โ†’ 0 the mode, mean
and median converge to ๐‘’๐‘’ ๐œ‡๐œ‡๐‘๐‘ . The distribution becomes narrower and
approaches a Dirac delta function at ๐‘ก๐‘ก = ๐‘’๐‘’ ๐œ‡๐œ‡๐‘๐‘ .
Hazard Rate. (Kleiber & Kotz 2003, p.115)The hazard rate is
unimodal with โ„Ž(0) = 0 and all dirivitives of โ„Žโ€™(๐‘ก๐‘ก) = 0 and a slow
decrease to zero as ๐‘ก๐‘ก โ†’ 0. The mode of the hazard rate:
๐‘ก๐‘ก๐‘š๐‘š = exp(๐œ‡๐œ‡ + ๐‘ง๐‘ง๐‘š๐‘š ๐œŽ๐œŽ)
where ๐‘ง๐‘ง๐‘š๐‘š is given by:
๐œ™๐œ™(๐‘ง๐‘ง๐‘š๐‘š )
(๐‘ง๐‘ง๐‘š๐‘š + ๐œŽ๐œŽ๐‘๐‘ ) =
1 โˆ’ ฮฆ(๐‘ง๐‘ง๐‘š๐‘š )
therefore โˆ’๐œŽ๐œŽ๐‘๐‘ < ๐‘ง๐‘ง๐‘š๐‘š < โˆ’๐œŽ๐œŽ๐‘๐‘ + ๐œŽ๐œŽ โˆ’1 and therefore:
2
2
๐‘’๐‘’ ๐œ‡๐œ‡๐‘๐‘โˆ’๐œŽ๐œŽ๐‘๐‘ < ๐‘ก๐‘ก๐‘š๐‘š < ๐‘’๐‘’ ๐œ‡๐œ‡๐‘๐‘โˆ’๐œŽ๐œŽ๐‘๐‘ +1
2
๐œ‡๐œ‡
โˆ’๐œŽ๐œŽ
๐‘๐‘
๐‘๐‘ and so for large ๐œŽ๐œŽ :
As ๐œŽ๐œŽ๐‘๐‘ โ†’ โˆž, ๐‘ก๐‘ก๐‘š๐‘š โ†’ ๐‘’๐‘’
๐‘๐‘
exp๏ฟฝ๐œ‡๐œ‡๐‘๐‘ โˆ’ 12๐œŽ๐œŽ๐‘๐‘2 ๏ฟฝ
max โ„Ž(๐‘ก๐‘ก) โ‰ˆ
๐œŽ๐œŽ๐‘๐‘ โˆš2๐œ‹๐œ‹
Lognormal Continuous Distribution
57
2
As ๐œŽ๐œŽ๐‘๐‘ โ†’ 0, ๐‘ก๐‘ก๐‘š๐‘š โ†’ ๐‘’๐‘’ ๐œ‡๐œ‡๐‘๐‘โˆ’๐œŽ๐œŽ๐‘๐‘ +1 and so for large ๐œŽ๐œŽ๐‘๐‘ :
1
max โ„Ž(๐‘ก๐‘ก) โ‰ˆ 2 ๐œ‡๐œ‡ โˆ’๐œŽ๐œŽ2 +1
๐‘๐‘
๐‘๐‘
๐œŽ๐œŽ๐‘๐‘ ๐‘’๐‘’
Scale/Product Property:
Let:
๐‘Ž๐‘Ž๐‘—๐‘— ๐‘‹๐‘‹๐‘—๐‘— ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๏ฟฝ๐œ‡๐œ‡๐‘๐‘๐‘๐‘ , ฯƒ2Nj ๏ฟฝ
If ๐‘‹๐‘‹๐‘—๐‘— and ๐‘‹๐‘‹๐‘—๐‘—+1 are independent:
2
๏ฟฝ ๐‘Ž๐‘Ž๐‘—๐‘— ๐‘‹๐‘‹๐‘—๐‘— ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ ๏ฟฝ๏ฟฝ๏ฟฝ๐œ‡๐œ‡๐‘๐‘๐‘๐‘ + ln๏ฟฝ๐‘Ž๐‘Ž๐‘—๐‘— ๏ฟฝ๏ฟฝ, ๏ฟฝ ๐œŽ๐œŽ๐‘๐‘๐‘๐‘
๏ฟฝ
Lognormal versus Weibull. In analyzing life data to these
distributions it is often the case that both may be a good fit, especially
in the middle of the distribution. The Weibull distribution has an
earlier lower tail and produces a more pessimistic estimate of the
component life. (Nelson 1990, p.65)
Applications
General Life Distributions. The lognormal distribution has been
found to accurately model many life distributions and is a popular
choice for life distributions. The increasing hazard rate in early life
models the weaker subpopulation (burn in) and the remaining
decreasing hazard rate describes the main population. In particular
this has been applied to some electronic devices and fatigue-fracture
data. (Meeker & Escobar 1998, p.262)
Failure Modes from Multiplicative Errors. The lognormal
distribution is very suitable for failure processes that are a result of
multiplicative errors. Specific applications include failure of
components due to fatigue cracks. (Provan 1987)
Repair Times. The lognormal distribution has commonly been used
to model repair times. It is natural for a repair time probability to
increase quickly to a mode value. For example very few repairs have
an immediate or quick fix. However, once the time of repair passes
the mean it is likely that there are serious problems, and the repair
will take a substantial amount of time.
Parameter Variability. The lognormal distribution can be used to
model parameter variability. This was done when estimating the
uncertainty in the parameter ๐œ†๐œ† in a Nuclear Reactor Safety Study
(NUREG-75/014).
Theory of Breakage. The distribution models particle sizes
observed in breakage processes (Crow & Shimizu 1988)
Resources
Online:
Lognormal
Mean / Median / Mode:
๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š(๐‘‹๐‘‹) < ๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š(๐‘‹๐‘‹) < ๐ธ๐ธ[๐‘‹๐‘‹]
58
Common Life Distributions
Lognormal
http://www.weibull.com/LifeDataWeb/the_lognormal_distribution.ht
m
http://mathworld.wolfram.com/LogNormalDistribution.html
http://en.wikipedia.org/wiki/Log-normal_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Crow, E.L. & Shimizu, K., 1988. Lognormal distributions, CRC
Press.
Aitchison, J.J. & Brown, J., 1957. The Lognormal Distribution, New
York: Cambridge University Press.
Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience.
Relationship to Other Distributions
Normal
Distribution
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
Let;
Then:
Where:
๐‘‹๐‘‹~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐œ‡๐œ‡๐‘๐‘ , ฯƒ2N )
๐‘Œ๐‘Œ = ln(๐‘‹๐‘‹)
๐‘Œ๐‘Œ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
๐œ‡๐œ‡2
๐œ‡๐œ‡๐‘๐‘ = ln ๏ฟฝ
๏ฟฝ,
๏ฟฝ๐œŽ๐œŽ 2 + ๐œ‡๐œ‡2
๐œŽ๐œŽ 2 + ๐œ‡๐œ‡2
๐œŽ๐œŽ๐‘๐‘ = ln ๏ฟฝ
๏ฟฝ
๐œ‡๐œ‡2
Weibull Continuous Distribution
59
2.3. Weibull Continuous Distribution
Probability Density Function - f(t)
0.90
ฮฑ=1 ฮฒ=2
ฮฑ=1.5 ฮฒ=2
ฮฑ=2 ฮฒ=2
ฮฑ=3 ฮฒ=2
0.80
0.70
0.60
0.50
0.40
0.30
0.20
0.10
0.00
0
1
0
2
2
4
Cumulative Density Function - F(t)
1.00
1.00
0.80
0.80
0.60
0.60
0.40
ฮฑ=1 ฮฒ=0.5
ฮฑ=1 ฮฒ=1
ฮฑ=1 ฮฒ=2
ฮฑ=1 ฮฒ=5
0.20
0.00
0
1
6
ฮฑ=1 ฮฒ=2
ฮฑ=1.5 ฮฒ=2
ฮฑ=2 ฮฒ=2
ฮฑ=3 ฮฒ=2
0.20
0.00
2
Hazard Rate - h(t)
0
10.00
9.00
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
5
4
3
2
1
0
0
0.40
1
2
2
4
ฮฑ=1 ฮฒ=2
ฮฑ=1.5 ฮฒ=2
ฮฑ=2 ฮฒ=2
ฮฑ=3 ฮฒ=2
0
2
4
Weibull
ฮฑ=1 ฮฒ=0.5
ฮฑ=1 ฮฒ=1
ฮฑ=1 ฮฒ=2
ฮฑ=1 ฮฒ=5
1.80
1.60
1.40
1.20
1.00
0.80
0.60
0.40
0.20
0.00
60
Common Life Distributions
Weibull
Parameters & Description
Parameters
๐›ผ๐›ผ
๐›ฝ๐›ฝ
Scale Parameter: The value of ๐›ผ๐›ผ
equals the 63.2th percentile and has
a unit equal to ๐‘ก๐‘ก. Note that this is not
equal to the mean.
๐›ผ๐›ผ > 0
Shape Parameter: Also known as the
slope (referring to a linear CDF plot) ๐›ฝ๐›ฝ
determines the shape of the
distribution.
๐›ฝ๐›ฝ > 0
๐‘ก๐‘ก โ‰ฅ 0
Limits
Distribution
PDF
CDF
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) =
๐›ฝ๐›ฝ๐‘ก๐‘ก ๐›ฝ๐›ฝโˆ’1 โˆ’๏ฟฝ ๐‘ก๐‘ก ๏ฟฝ๐›ฝ๐›ฝ
๐‘’๐‘’ ๐›ผ๐›ผ
๐›ผ๐›ผ ๐›ฝ๐›ฝ
๐น๐น(๐‘ก๐‘ก) = 1 โˆ’ ๐‘’๐‘’
R(t) = ๐‘’๐‘’
๐‘ก๐‘ก ๐›ฝ๐›ฝ
โˆ’๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
๏ฟฝ
๐‘…๐‘…(๐‘ก๐‘ก + x)
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
= ๐‘’๐‘’
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘ก๐‘ก ๐›ฝ๐›ฝ โˆ’(๐‘ก๐‘ก+x)๐›ฝ๐›ฝ
๏ฟฝ
๐›ผ๐›ผ ๐›ฝ๐›ฝ
Where
๐‘ก๐‘ก is the given time we know the component has survived to
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
(Kleiber & Kotz 2003, p.176)
Mean Residual
Life
๐‘ก๐‘ก ๐›ฝ๐›ฝ
โˆ’๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
๐‘ข๐‘ข(๐‘ก๐‘ก) = ๐‘’๐‘’
๐‘ก๐‘ก ๐›ฝ๐›ฝ
๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
โˆž
๏ฟฝ ๐‘’๐‘’
t
๐‘ฅ๐‘ฅ ๐›ฝ๐›ฝ
โˆ’๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
which has the asymptotic property of:
lim ๐‘ข๐‘ข(๐‘ก๐‘ก) = ๐‘ก๐‘ก 1โˆ’๐›ฝ๐›ฝ
๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘กโ†’โˆž
Hazard Rate
Cumulative
Hazard Rate
ฮฒ ๐‘ก๐‘ก ๐›ฝ๐›ฝโˆ’1
โ„Ž(๐‘ก๐‘ก) = . ๏ฟฝ ๏ฟฝ
ฮฑ ๐›ผ๐›ผ
๐‘ก๐‘ก ๐›ฝ๐›ฝ
๐ป๐ป(๐‘ก๐‘ก) = ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
Properties and Moments
Median
Mode
1
๐›ผ๐›ผ๏ฟฝ๐‘™๐‘™๐‘™๐‘™(2)๏ฟฝ๐›ฝ๐›ฝ
1
๐›ฝ๐›ฝ โˆ’ 1 ๐›ฝ๐›ฝ
๐›ผ๐›ผ ๏ฟฝ
๏ฟฝ
if ๐›ฝ๐›ฝ โ‰ฅ 1
๐›ฝ๐›ฝ
otherwise no mode exists
Weibull Continuous Distribution
61
1
๐›ผ๐›ผฮ“ ๏ฟฝ1 + ๏ฟฝ
๐›ฝ๐›ฝ
Mean - 1st Raw Moment
3
ฮ“ ๏ฟฝ1 + ๏ฟฝ ฮฑ3 โˆ’ 3µฯƒ2 โˆ’ µ3
ฮฒ
ฯƒ3
Skewness - 3rd Central Moment
โˆ’6ฮ“14 + 12ฮ“12 ฮ“2 โˆ’ 3ฮ“22 โˆ’ 4ฮ“1 ฮ“3 + ฮ“4
(ฮ“2 โˆ’ ฮ“12 )2
where:
๐‘–๐‘–
ฮ“๐‘–๐‘– = ฮ“ ๏ฟฝ1 + ๏ฟฝ
๐›ฝ๐›ฝ
Excess kurtosis - 4th Central Moment
โˆž
Characteristic Function
๏ฟฝ
๐‘›๐‘›=0
100p% Percentile Function
(๐‘–๐‘–๐‘–๐‘–)๐‘›๐‘› ๐›ผ๐›ผ ๐‘›๐‘›
๐‘›๐‘›
ฮ“ ๏ฟฝ1 + ๏ฟฝ
๐‘›๐‘›!
๐›ฝ๐›ฝ
1
๐‘ก๐‘ก๐‘๐‘ = ๐›ผ๐›ผ[โˆ’ ln(1 โˆ’ ๐‘๐‘)]๐›ฝ๐›ฝ
Parameter Estimation
Plotting Method
Least
Mean
Square
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
Likelihood
Functions
X-Axis
๐›ผ๐›ผ๏ฟฝ = ๐‘’๐‘’ โˆ’๐‘š๐‘š
๐›ฝ๐›ฝฬ‚ = ๐‘š๐‘š
1
ln ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ
1 โˆ’ ๐น๐น
ln(๐‘ก๐‘ก๐‘–๐‘– )
Maximum Likelihood Function
tF
๐›ฝ๐›ฝโˆ’1
๐›ฝ๐›ฝ
tS
๐›ฝ๐›ฝ
nS
โˆ’๏ฟฝ i ๏ฟฝ
โˆ’๏ฟฝ i ๏ฟฝ
๐›ฝ๐›ฝ๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ
L(ฮฑ, ฮฒ|E) = ๏ฟฝ
๐‘’๐‘’ ๐›ผ๐›ผ . ๏ฟฝ ๐‘’๐‘’ ๐›ผ๐›ผ
๐›ฝ๐›ฝ
๐›ผ๐›ผ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
nF
nI
Log-Likelihood
Function
๐‘๐‘
Y-Axis
failures
๐›ฝ๐›ฝ
tLI
โˆ’๏ฟฝ i ๏ฟฝ
๐›ผ๐›ผ
โˆ’๏ฟฝ
tRI
i ๏ฟฝ
๐›ผ๐›ผ
๐›ฝ๐›ฝ
survivors
๏ฟฝ ๏ฟฝ๐‘’๐‘’
โˆ’ ๐‘’๐‘’
๏ฟฝ
i=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
interval failures
nF
๐›ฝ๐›ฝ
๐‘ก๐‘ก๐‘–๐‘–๐น๐น
ฮ›(ฮฑ, ฮฒ|E) = nF ln(ฮฒ) โˆ’ ฮฒnF ln(๐›ผ๐›ผ) + ๏ฟฝ ๏ฟฝ(๐›ฝ๐›ฝ โˆ’ 1) ln๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ โˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
nS
ฮฒ
t Si
nI
failures
tLI
i ๏ฟฝ
ฮฑ
โˆ’๏ฟฝ
ฮฒ
tRI
i ๏ฟฝ
ฮฑ
โˆ’๏ฟฝ
ฮฒ
โˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ + ๏ฟฝ ln ๏ฟฝe
โˆ’e
๏ฟฝ
ฮฑ
๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1๏ฟฝ๏ฟฝ๏ฟฝ
i=1
survivors
interval failures
Weibull
2
1
๐›ผ๐›ผ 2 ๏ฟฝฮ“ ๏ฟฝ1 + ๏ฟฝ โˆ’ ฮ“ 2 ๏ฟฝ1 + ๏ฟฝ๏ฟฝ
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ
Variance - 2nd Central Moment
62
Common Life Distributions
โˆ‚ฮ›
=0
โˆ‚ฮฑ
solve for ๐›ผ๐›ผ to get ๐›ผ๐›ผ๏ฟฝ:
nS
nF
โˆ‚ฮ› โˆ’ฮฒnF
ฮฒ
ฮฒ
ฮฒ
ฮฒ
=
+ ฮฒ+1 ๏ฟฝ๏ฟฝt Fi ๏ฟฝ + ฮฒ+1 ๏ฟฝ๏ฟฝt Si ๏ฟฝ
โˆ‚ฮฑ
ฮฑ
ฮฑ
ฮฑ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1 ๏ฟฝ๏ฟฝ๏ฟฝ
failures
ฮฒ
tRI
๏ฟฝi ๏ฟฝ
t LI
ฮฑ
i
interval failures
solve for ๐›ฝ๐›ฝ to get ๐›ฝ๐›ฝฬ‚ :
nF
nS
ฮฒ
ฮฒ
t Fi
t Fi
t Fi
t Si
t Si
โˆ‚ฮ› nF
=
+ ๏ฟฝ ๏ฟฝln ๏ฟฝ ๏ฟฝ โˆ’ ๏ฟฝ ๏ฟฝ . ln ๏ฟฝ ๏ฟฝ๏ฟฝ โˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ ln ๏ฟฝ ๏ฟฝ
ฮฒ
ฮฑ
ฮฑ
๐›ผ๐›ผ
ฮฑ
๐›ผ๐›ผ
โˆ‚ฮฒ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
i=1
t RI
i
MLE Point
Estimates
survivors
ฮฒ
ฮฒ
tLI
๏ฟฝi ๏ฟฝ
t RI
ฮฑ
i
nI
๏ฟฝ ๏ฟฝ e
โˆ’๏ฟฝ ๏ฟฝ e
โŽž
ฮฑ
ฮฒโŽ› ฮฑ
โŽŸ=0
+๏ฟฝ โŽœ
ฮฒ
ฮฒ
RI
LI
โŽŸ
ฮฑโŽœ
t
t
๏ฟฝi ๏ฟฝ
๏ฟฝi ๏ฟฝ
i=1
e ฮฑ โˆ’e ฮฑ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
โŽ
โŽ 
Weibull
โˆ‚ฮ›
=0
โˆ‚ฮฒ
ฮฒ
failures
ฮฒ
t RI
i
tLI
i ๏ฟฝ
ฮฑ
๏ฟฝ
ฮฒ
survivors
ฮฒ
ฮฒ
tRI
๏ฟฝi ๏ฟฝ
t LI
ฮฑ
i
t LI
i
nI
โˆ’ ln ๏ฟฝ ๏ฟฝ . ๏ฟฝ ๏ฟฝ . e
โŽ›ln ๏ฟฝ ๐›ผ๐›ผ ๏ฟฝ . ๏ฟฝ ฮฑ ๏ฟฝ . e
โŽž
๐›ผ๐›ผ
ฮฑ
โŽŸ=0
+๏ฟฝโŽœ
ฮฒ
ฮฒ
RI
LI
โŽœ
โŽŸ
t
t
๏ฟฝi ๏ฟฝ
๏ฟฝi ๏ฟฝ
i=1
e ฮฑ โˆ’e ฮฑ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
โŽ
โŽ 
interval failures
When there is only complete failure and/or right censored data the point
estimates can be solved using (Rinne 2008, p.439):
1
๏ฟฝ ฮฒ
๏ฟฝ
ฮฒ
๏ฟฝ
ฮฒ
ฮฒ๏ฟฝ = ๏ฟฝ
๏ฟฝ
ฮฒ
โˆ‘๏ฟฝt Fi ๏ฟฝ + โˆ‘๏ฟฝt Si ๏ฟฝ
๏ฟฝ=๏ฟฝ
ฮฑ
๏ฟฝ
nF
๏ฟฝ
ฮฒ
โˆ‘๏ฟฝt Fi ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ + โˆ‘๏ฟฝt Si ๏ฟฝ ln๏ฟฝt Si ๏ฟฝ
๏ฟฝ
ฮฒ
โˆ‘๏ฟฝt Fi ๏ฟฝ + โˆ‘๏ฟฝt Si ๏ฟฝ
๏ฟฝ
ฮฒ
1
โˆ’ ๏ฟฝ ln(t Fi )๏ฟฝ
๐‘›๐‘›๐น๐น
โˆ’1
Note: Numerical methods are needed to solve ฮฒ๏ฟฝ then substitute to find
๏ฟฝ. Numerical methods to find Weibull MLE estimates for complete and
ฮฑ
censored data for 2 parameter and 3 parameter Weibull distribution are
detailed in (Rinne 2008).
Fisher
Information
Matrix
(Rinne
p.412)
2008,
๐›ฝ๐›ฝ2
๐›ฝ๐›ฝ2
ฮ“ โ€ฒ (2)
โŽก
โŽค โŽก ๐›ผ๐›ผ 2
๐›ผ๐›ผ 2
โˆ’๐›ผ๐›ผ
โŽฅ โŽข
๐ผ๐ผ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = โŽข โ€ฒ
โ€ฒโ€ฒ (2) = โŽข
(2)
ฮ“
1
+
ฮ“
โŽข
โŽฅ โŽข1 โˆ’ ๐›พ๐›พ
โŽฃ โˆ’๐›ผ๐›ผ
๐›ฝ๐›ฝ2
โŽฆ โŽฃ ๐›ผ๐›ผ
1 โˆ’ ๐›พ๐›พ
๐›ผ๐›ผ
โŽค
โŽฅ
๐œ‹๐œ‹ 2
2
+ (1 โˆ’ ๐›พ๐›พ ) โŽฅ
6
โŽฅ
โŽฆ
๐›ฝ๐›ฝ2
๐›ฝ๐›ฝ2
0.422784
โŽก
โŽค
2
๐›ผ๐›ผ
โˆ’๐›ผ๐›ผ
โŽฅ
โ‰…โŽข
โŽข0.422784 1.823680 โŽฅ
โŽฃ โˆ’๐›ผ๐›ผ
๐›ฝ๐›ฝ2
โŽฆ
Weibull Continuous Distribution
100๐›พ๐›พ%
Confidence
Interval
(complete
data)
63
Bayesian
Bayesian analysis is applied to either one of two re-parameterizations of the Weibull
Distribution: (Rinne 2008, p.517)
๐‘“๐‘“(๐‘ก๐‘ก; ๐œ†๐œ†, ๐›ฝ๐›ฝ) = ๐œ†๐œ†๐œ†๐œ†๐‘ก๐‘ก ๐›ฝ๐›ฝโˆ’1 exp๏ฟฝโˆ’๐œ†๐œ†๐‘ก๐‘ก ๐›ฝ๐›ฝ ๏ฟฝ
or
๐‘“๐‘“(๐‘ก๐‘ก; ๐œƒ๐œƒ, ๐›ฝ๐›ฝ) =
Type
๐›ฝ๐›ฝ ๐›ฝ๐›ฝโˆ’1
๐‘ก๐‘ก ๐›ฝ๐›ฝ
๐‘ก๐‘ก
exp ๏ฟฝโˆ’ ๏ฟฝ
๐œƒ๐œƒ
๐œƒ๐œƒ
where ๐œ†๐œ† = ๐›ผ๐›ผ โˆ’๐›ฝ๐›ฝ
where ๐œƒ๐œƒ =
1
= ๐›ผ๐›ผ ๐›ฝ๐›ฝ
๐œ†๐œ†
Non-informative Priors ๐…๐…๐ŸŽ๐ŸŽ (๐€๐€) (Rinne 2008, p.517)
Prior
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Proper Prior with
known ๐›ฝ๐›ฝ and limits ๐œ†๐œ† โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
Jeffreyโ€™s Prior when ๐›ฝ๐›ฝ is known.
Jeffreyโ€™s Prior for unknown ๐œƒ๐œƒ
and ๐›ฝ๐›ฝ.
where
Posterior
Truncated Gamma Distribution
For a โ‰ค ฮป โ‰ค b
๐‘๐‘. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + nF , t T,ฮฒ )
1
โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0)
๐œ†๐œ†
ฮฒ
1
๐œƒ๐œƒ๐œƒ๐œƒ
Otherwise ๐œ‹๐œ‹(๐œ†๐œ†) = 0
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; nF , t T,ฮฒ )
when ๐œ†๐œ† โˆˆ [0, โˆž)
No closed form
(Rinne 2008, p.527)
ฮฒ
๐‘ก๐‘ก ๐‘‡๐‘‡,๐›ฝ๐›ฝ = โˆ‘๏ฟฝt Fi ๏ฟฝ + โˆ‘๏ฟฝt Si ๏ฟฝ = adjusted total time in test
Conjugate Priors
It was found by Soland that no joint continuous prior distribution exists for the Weibull
distribution. Soland did however propose a procedure which used a continuous
distribution for ฮฑ and a discrete distribution for ฮฒ which will not be included here. (Martz &
Waller 1982)
UOI
๐œ†๐œ†
where
๐œ†๐œ† = ๐›ผ๐›ผ โˆ’๐›ฝ๐›ฝ
from
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Likelihood
Model
Evidence
Weibull with
known ๐›ฝ๐›ฝ
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–๐น๐น
Dist of
UOI
Gamma
Prior
Para
Posterior
Parameters
๐‘˜๐‘˜0 , ฮ›0
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
ฮ› = ฮ›0 + ๐‘ก๐‘ก ๐‘‡๐‘‡,๐›ฝ๐›ฝ
(Rinne 2008,
p.520)
Weibull
The asymptotic variance-covariance matrix of (๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ) is: (Rinne 2008,
pp.412-417)
๐›ผ๐›ผ๏ฟฝ 2
1 1.1087
0.2570๐›ผ๐›ผ๏ฟฝ
โˆ’1
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ๏ฟฝ = ๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ๏ฟฝ๏ฟฝ =
๏ฟฝ
๏ฟฝ
๐›ฝ๐›ฝฬ‚2
๐‘›๐‘›๐น๐น
0.2570๐›ผ๐›ผ๏ฟฝ 0.6079๐›ฝ๐›ฝฬ‚2
Weibull
64
Common Life Distributions
๐œƒ๐œƒ
where
๐œƒ๐œƒ = ๐›ผ๐›ผ ๐›ฝ๐›ฝ
from
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Weibull with
known ๐›ฝ๐›ฝ
Example
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–๐น๐น
Inverted
Gamma
๐›ผ๐›ผ = ๐›ผ๐›ผ๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
ฮฒ = ฮฒ0 + ๐‘ก๐‘ก ๐‘‡๐‘‡,๐›ฝ๐›ฝ
(Rinne 2008,
p.524)
๐›ผ๐›ผ0 , ฮฒ0
Description , Limitations and Uses
5 components are put on a test with the following failure times:
535, 613, 976, 1031, 1875 hours
ฮฒ๏ฟฝ is found by numerically solving:
ฮฒ๏ฟฝ = ๏ฟฝ
๏ฟฝ is found by solving:
ฮฑ
Covariance Matrix is:
๏ฟฝ
ฮฒ
โˆ‘๏ฟฝt Fi ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ
โˆ‘๏ฟฝt Fi ๏ฟฝ
๏ฟฝ
ฮฒ
โˆ’1
โˆ’ 6.8118๏ฟฝ
ฮฒ๏ฟฝ = 2.275
๏ฟฝ=๏ฟฝ
ฮฑ
1
๏ฟฝ ฮฒ
๏ฟฝ
ฮฒ
F
โˆ‘๏ฟฝt i ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ 2
1 1.1087
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ๏ฟฝ = ๏ฟฝ
๐›ฝ๐›ฝฬ‚2
5
0.2570๐›ผ๐›ผ๏ฟฝ
nF
๏ฟฝ = 1140
0.2570๐›ผ๐›ผ๏ฟฝ
0.6079๐›ฝ๐›ฝฬ‚2
๏ฟฝ=๏ฟฝ
55679 58.596
๏ฟฝ
58.596 0.6293
90% confidence interval for ๐›ผ๐›ผ๏ฟฝ:
๐›ท๐›ทโˆ’1 (0.95)โˆš55679
๏ฟฝ๐›ผ๐›ผ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ,
โˆ’๐›ผ๐›ผ๏ฟฝ
[811,
1602]
90% confidence interval for ๐›ฝ๐›ฝ:
๐›ท๐›ท โˆ’1 (0.95)โˆš0.6293
๏ฟฝ๐›ฝ๐›ฝฬ‚. exp ๏ฟฝ
๏ฟฝ,
โˆ’๐›ฝ๐›ฝฬ‚
[1.282,
๐›ท๐›ท โˆ’1 (0.95)โˆš0.6293
๐›ฝ๐›ฝฬ‚ . exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐›ฝ๐›ฝฬ‚
4.037]
๐›ผ๐›ผ๏ฟฝ. exp ๏ฟฝ
๐›ท๐›ท โˆ’1 (0.95)โˆš55679
๏ฟฝ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ
Note that with only 5 samples the assumption that the parameter
distribution is approximately normal is probably inaccurate and
therefore the confidence intervals need to be used with caution.
Characteristics
The Weibull distribution is also known as a โ€œType III asymptotic
distribution for minimum valuesโ€.
ฮฒ Characteristics:
ฮฒ < 1. The hazard rate decreases with time.
Weibull Continuous Distribution
65
Note that for ๐›ฝ๐›ฝ = 0.999, ๐‘“๐‘“(0) = โˆž, but for ๐›ฝ๐›ฝ = 1.001, ๐‘“๐‘“(0) = 0.
This rapid change creates complications when maximizing likelihood
functions. (Weibull.com) As ๐›ฝ๐›ฝ โ†’ โˆž, the ๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š โ†’ ๐›ผ๐›ผ.
ฮฑ Characteristics. Increasing ๐›ผ๐›ผ stretches the distribution over the
time scale. With the ๐‘“๐‘“(0) point fixed this also has the effect of
increasing the mode, mean and median. The value for ๐›ผ๐›ผ is at the
63% Percentile. ๐น๐น(๐›ผ๐›ผ) = 0.632..
๐‘‹๐‘‹~๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Scaling property: (Leemis & McQueston 2008)
๐‘˜๐‘˜๐‘˜๐‘˜~๐‘Š๐‘Š๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๏ฟฝ๐›ผ๐›ผ๐‘˜๐‘˜๐›ฝ๐›ฝ , ๐›ฝ๐›ฝ๏ฟฝ
Minimum property (Rinne 2008, p.107)
min{๐‘‹๐‘‹, ๐‘‹๐‘‹2 , โ€ฆ , ๐‘‹๐‘‹๐‘›๐‘› }~๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐›ผ๐›ผ๐‘›๐‘›
When ๐›ฝ๐›ฝ is fixed.
Variate Generation property
1
โˆ’
๐›ฝ๐›ฝ , ๐›ฝ๐›ฝ)
1
๐น๐น โˆ’1 (๐‘ข๐‘ข) = ๐›ผ๐›ผ[โˆ’ ln(1 โˆ’ ๐‘ข๐‘ข)]๐›ฝ๐›ฝ , 0 < ๐‘ข๐‘ข < 1
Lognormal versus Weibull. In analyzing life data to these
distributions it is often the case that both may be a good fit, especially
in the middle of the distribution. The Weibull distribution has an
earlier lower tail and produces a more pessimistic estimate of the
component life. (Nelson 1990, p.65)
Applications
The Weibull distribution is by far the most popular life distribution
used in reliability engineering. This is due to its variety of shapes and
generalization or approximation of many other distributions. Analysis
assuming a Weibull distribution already includes the exponential life
Weibull
ฮฒ = 1. The hazard rate is constant (exp distribution)
ฮฒ > 1. The hazard rate increases with time.
1 < ฮฒ < 2. The hazard rate increases less as time increases.
ฮฒ = 2. The hazard rate increases with a linear relationship
to time.
ฮฒ > 2. The hazard rate increases more as time increases.
ฮฒ < 3.447798. The distribution is positively skewed. (Tail to
right).
ฮฒ โ‰ˆ 3.447798. The distribution is approximately
symmetrical.
ฮฒ > 3.447798. The distribution is negatively skewed (Tail to
left).
3 < ฮฒ < 4. The distribution approximates a normal
distribution.
ฮฒ > 10. The distribution approximates a Smallest Extreme
Value Distribution.
66
Common Life Distributions
distribution as a special case.
Weibull
There are many physical interpretations of the Weibull Distribution.
Due to its minimum property a physical interpretation is the weakest
link, where a system such as a chain will fail when the weakest link
fails. It can also be shown that the Weibull Distribution can be derived
from a cumulative wear model (Rinne 2008, p.15)
The following is a non-exhaustive list of applications where the
Weibull distribution has been used in:
โ€ข
Acceptance sampling
โ€ข
Warranty analysis
โ€ข
Maintenance and renewal
โ€ข
Strength of material modeling
โ€ข
Wear modeling
โ€ข
Electronic failure modeling
โ€ข
Corrosion modeling
A detailed list with references to practical examples is contained in
(Rinne 2008, p.275)
Resources
Online:
http://www.weibull.com/LifeDataWeb/the_weibull_distribution.htm
http://mathworld.wolfram.com/WeibullDistribution.html
http://en.wikipedia.org/wiki/Weibull_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web
calculator)
http://www.qualitydigest.com/jan99/html/weibull.html (how to use
conduct Weibull analysis in Excel, William W. Dorner)
Books:
Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed.,
Chapman & Hall/CRC.
Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed.,
Wiley-Interscience.
Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience.
Relationship to Other Distributions
Three Parameter
Weibull
Distribution
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ, ๐›พ๐›พ)
Exponential
The three parameter model adds a locator parameter to the two
parameter Weibull distribution allowing a shift along the x-axis. This
creates a period of guaranteed zero failures to the beginning of the
product life and is therefore only used in special cases.
Special Case:
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ, ๐›พ๐›พ = 0)
Let
Weibull Continuous Distribution
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐‘ก๐‘ก; ๐›ผ๐›ผ)
ฯ‡ Distribution
๐œ’๐œ’(๐‘ก๐‘ก|๐‘ฃ๐‘ฃ)
Special Case:
Special Case:
๐‘‹๐‘‹~๐‘Š๐‘Š๐‘’๐‘’๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(ฮป = ๐›ผ๐›ผ โˆ’๐›ฝ๐›ฝ )
๐‘Œ๐‘Œ = X ฮฒ
1
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†) = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š ๏ฟฝ๐‘ก๐‘ก; ๐›ผ๐›ผ = , ๐›ฝ๐›ฝ = 1๏ฟฝ
๐œ†๐œ†
๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐‘ก๐‘ก; ๐›ผ๐›ผ) = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ = 2)
Special Case:
๐œ’๐œ’(๐‘ก๐‘ก|๐‘ฃ๐‘ฃ = 2) = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๏ฟฝ๐‘ก๐‘ก|๐›ผ๐›ผ = โˆš2, ๐›ฝ๐›ฝ = 2๏ฟฝ
Weibull
Rayleigh
Distribution
Then
67
68
Probability Distributions Used in Reliability Engineering
3. Bathtub Life Distributions
2-Fold Mixed Weibull Distribution
69
3.1. 2-Fold Mixed Weibull
Distribution
All shapes shown are variations from ๐‘๐‘ = 0.5 ๐›ผ๐›ผ1 = 2 ๐›ฝ๐›ฝ1 = 0.5 ๐›ผ๐›ผ2 = 10 ๐›ฝ๐›ฝ2 = 20
Probability Density Function - f(t)
ฮฑ1=2 ฮฒ1=0.1
ฮฑ1=2 ฮฒ1=0.5
ฮฑ1=2 ฮฒ1=1
ฮฑ1=2 ฮฒ1=5
ฮฑ2=10 ฮฒ2=20
ฮฑ2=10 ฮฒ2=10
ฮฑ2=10 ฮฒ2=1
ฮฑ2=5 ฮฒ2=10
p=0
p=0.3
p=0.7
p=1
Mixed Wbl
0
5
10
0
5
10
0
5
10
5
10
0
10
10
0
10
Cumulative Density Function - F(t)
0
5
10
0
10
0
Hazard Rate - h(t)
0
5
70
Bathtub Life Distributions
Parameters & Description
Parameters
Limits
๐›ผ๐›ผ๐‘–๐‘–
๐›ผ๐›ผ๐‘–๐‘– > 0
๐‘๐‘
0 โ‰ค ๐‘๐‘ โ‰ค 1
๐›ฝ๐›ฝ๐‘–๐‘–
Scale Parameter: This is the scale for
each Weibull Distribution.
Shape Parameters: The shape of
each Weibull Distribution
๐›ฝ๐›ฝ๐‘–๐‘– > 0
Mixing Parameter. This determines
the weight each Weibull Distribution
has on the overall density function.
๐‘ก๐‘ก โ‰ฅ 0
Mixed Wbl
Distribution
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) = ๐‘๐‘๐‘“๐‘“1 (๐‘ก๐‘ก) + (1 โˆ’ ๐‘๐‘)๐‘“๐‘“2 (๐‘ก๐‘ก)
PDF
where ๐‘“๐‘“๐‘–๐‘– (๐‘ก๐‘ก) =
๐›ฝ๐›ฝ๐‘–๐‘–
๐›ฝ๐›ฝ๐‘–๐‘– ๐‘ก๐‘ก ๐›ฝ๐›ฝ๐‘–๐‘–โˆ’1 โˆ’๏ฟฝ๐›ผ๐›ผ๐‘ก๐‘ก ๏ฟฝ
๐‘’๐‘’ ๐‘–๐‘–
๐›ผ๐›ผ๐‘–๐‘– ๐›ฝ๐›ฝ๐‘–๐‘–
and ๐‘–๐‘– โˆˆ {1,2}
๐น๐น(๐‘ก๐‘ก) = ๐‘๐‘๐น๐น1 (๐‘ก๐‘ก) + (1 โˆ’ ๐‘๐‘)๐น๐น2 (๐‘ก๐‘ก)
CDF
where ๐น๐น๐‘–๐‘– (๐‘ก๐‘ก) = 1 โˆ’ ๐‘’๐‘’
โˆ’๏ฟฝ
๐‘ก๐‘ก ๐›ฝ๐›ฝ๐‘–๐‘–
๏ฟฝ
๐›ผ๐›ผ๐‘–๐‘–
and ๐‘–๐‘– โˆˆ {1,2}
๐‘…๐‘…(๐‘ก๐‘ก) = ๐‘๐‘๐‘…๐‘…1 (๐‘ก๐‘ก) + (1 โˆ’ ๐‘๐‘)๐‘…๐‘…2 (๐‘ก๐‘ก)
Reliability
where ๐‘…๐‘…๐‘–๐‘– (๐‘ก๐‘ก) = ๐‘’๐‘’
Hazard Rate
where
โˆ’๏ฟฝ
๐‘ก๐‘ก ๐›ฝ๐›ฝ๐‘–๐‘–
๏ฟฝ
๐›ผ๐›ผ๐‘–๐‘–
and ๐‘–๐‘– โˆˆ {1,2}
โ„Ž(๐‘ก๐‘ก) = ๐‘ค๐‘ค1 (๐‘ก๐‘ก)โ„Ž1 (๐‘ก๐‘ก) + ๐‘ค๐‘ค2 (๐‘ก๐‘ก)โ„Ž2 (๐‘ก๐‘ก)
๐‘ค๐‘ค๐‘–๐‘– (๐‘ก๐‘ก) =
๐‘๐‘๐‘–๐‘– ๐‘…๐‘…๐‘–๐‘– (๐‘ก๐‘ก)
โˆ‘๐‘›๐‘›๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘– ๐‘…๐‘…๐‘–๐‘– (๐‘ก๐‘ก)
and ๐‘–๐‘– โˆˆ {1,2}
Properties and Moments
Median
Solved numerically
Mode
Mean -
Solved numerically
1st
Raw Moment
๐‘๐‘๐›ผ๐›ผ1 ฮ“ ๏ฟฝ1 +
Variance - 2nd Central Moment
1
1
๏ฟฝ + (1 โˆ’ p)๐›ผ๐›ผ2 ฮ“ ๏ฟฝ1 + ๏ฟฝ
๐›ฝ๐›ฝ1
๐›ฝ๐›ฝ2
๐‘๐‘. ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‡๐‘‡1 ] + (1 โˆ’ ๐‘๐‘)๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‡๐‘‡2 ]
+๐‘๐‘(๐ธ๐ธ[๐‘‹๐‘‹1 ] โˆ’ ๐ธ๐ธ[๐‘‹๐‘‹])2
+(1 โˆ’ ๐‘๐‘)(๐ธ๐ธ[๐‘‹๐‘‹2 ] โˆ’ ๐ธ๐ธ[๐‘‹๐‘‹])2
๐‘๐‘. ๐›ผ๐›ผ 2 ๏ฟฝฮ“ ๏ฟฝ1 +
2
๐›ฝ๐›ฝ1
๏ฟฝ โˆ’ ฮ“ 2 ๏ฟฝ1 +
1
๐›ฝ๐›ฝ1
๏ฟฝ๏ฟฝ
2-Fold Mixed Weibull Distribution
+(1 โˆ’ ๐‘๐‘)๐›ผ๐›ผ 2 ๏ฟฝฮ“ ๏ฟฝ1 +
+๐‘๐‘ ๏ฟฝ๐›ผ๐›ผ1 ฮ“ ๏ฟฝ1 +
1
๐›ฝ๐›ฝ1
๏ฟฝ โˆ’ ฮ“ 2 ๏ฟฝ1 +
๏ฟฝ โˆ’ ๐ธ๐ธ[๐‘‹๐‘‹]๏ฟฝ
+(1 โˆ’ ๐‘๐‘) ๏ฟฝ๐›ผ๐›ผ2 ฮ“ ๏ฟฝ1 +
100p% Percentile Function
2
๐›ฝ๐›ฝ2
1
๐›ฝ๐›ฝ2
71
2
๏ฟฝ โˆ’ ๐ธ๐ธ[๐‘‹๐‘‹]๏ฟฝ
1
๐›ฝ๐›ฝ2
๏ฟฝ๏ฟฝ
2
Solved numerically
Parameter Estimation
Plotting Method (Jiang & Murthy 1995)
X-Axis
Y-Axis
๐‘ฅ๐‘ฅ = ๐‘™๐‘™๐‘™๐‘™(๐‘ก๐‘ก)
1
๐‘ฆ๐‘ฆ = ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ
1 โˆ’ ๐น๐น
Using the Weibull Probability Plot the parameters can be estimated. Jiang & Murthy, 1995,
provide a comprehensive coverage of this procedure and detail error in previous methods.
A typical WPP for a 2-fold Mixed Weibull Distribution is:
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
-1
0
1
2
3
4
5
6
WPP for 2-Fold Weibull Mixture Model ๐‘๐‘ = 12, ๐›ผ๐›ผ1 = 5, ๐›ฝ๐›ฝ1 = 0.5, ๐›ผ๐›ผ2 = 10, ๐›ฝ๐›ฝ2 = 5
Sub Populations:
The dotted lines in the WPP is the lines representing the subpopulations:
๐ฟ๐ฟ1 = ๐›ฝ๐›ฝ1 [๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ1 )]
๐ฟ๐ฟ2 = ๐›ฝ๐›ฝ2 [๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ2 )]
Mixed Wbl
Plot Points on
a Weibull
Probability Plot
72
Bathtub Life Distributions
Asymptotes (Jiang & Murthy 1995):
As ๐‘ฅ๐‘ฅ โ†’ โˆ’โˆž (๐‘ก๐‘ก โ†’ 0) there exists an asymptote approximated by:
๐‘ฆ๐‘ฆ โ‰ˆ ๐›ฝ๐›ฝ1 [๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ1 )] + ln(๐‘๐‘)
where
๐‘๐‘
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ ๐›ฝ๐›ฝ1 โ‰  ๐›ฝ๐›ฝ2
๐›ผ๐›ผ1 ๐›ฝ๐›ฝ1
๐‘๐‘ = ๏ฟฝ
๐‘๐‘ + (1 โˆ’ ๐‘๐‘). ๏ฟฝ ๏ฟฝ
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ ๐›ฝ๐›ฝ1 = ๐›ฝ๐›ฝ2
๐›ผ๐›ผ2
Mixed Wbl
As ๐‘ฅ๐‘ฅ โ†’ โˆž (๐‘ก๐‘ก โ†’ โˆž) the asymptote straight line can be approximated by:
๐‘ฆ๐‘ฆ โ‰ˆ ๐›ฝ๐›ฝ1 [๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ1 )]
Parameter Estimation
Jiang and Murthy divide the parameter estimation procedure into three cases:
Well Mixed Case ๐œท๐œท๐Ÿ๐Ÿ โ‰  ๐œท๐œท๐Ÿ๐Ÿ ๐’‚๐’‚๐’‚๐’‚๐’‚๐’‚ ๐œถ๐œถ๐Ÿ๐Ÿ โ‰ˆ ๐œถ๐œถ๐Ÿ๐Ÿ
- Estimate the parameters of ๐›ผ๐›ผ1 and ๐›ฝ๐›ฝ1 from the ๐ฟ๐ฟ1 line (right asymptote).
- Estimate the parameter ๐‘๐‘ from the separation distance between the left and right
asymptotes.
- Find the point where the curve crosses ๐ฟ๐ฟ1 (point I). The slope at point I is:
๐›ฝ๐›ฝฬ… = ๐‘๐‘๐›ฝ๐›ฝ1 + (1 โˆ’ ๐‘๐‘)๐›ฝ๐›ฝ2
- Determine slope at point I and use to estimate ๐›ฝ๐›ฝ2
- Draw a line through the intersection point I with slope ๐›ฝ๐›ฝ2 and use the intersection point
to estimate ๐›ผ๐›ผ2 .
Well Separated Case ๐œท๐œท๐Ÿ๐Ÿ โ‰  ๐œท๐œท๐Ÿ๐Ÿ ๐’‚๐’‚๐’‚๐’‚๐’‚๐’‚ ๐œถ๐œถ๐Ÿ๐Ÿ โ‰ซ ๐œถ๐œถ๐Ÿ๐Ÿ ๐’๐’๐’๐’ ๐œถ๐œถ๐Ÿ๐Ÿ โ‰ช ๐œถ๐œถ๐Ÿ๐Ÿ
- Determine visually if data is scattered along the bottom (or top) to determine if ๐›ผ๐›ผ1 โ‰ช ๐›ผ๐›ผ2
(or ๐›ผ๐›ผ1 โ‰ซ ๐›ผ๐›ผ2 ).
- If ๐›ผ๐›ผ1 โ‰ช ๐›ผ๐›ผ2 (๐›ผ๐›ผ1 โ‰ซ ๐›ผ๐›ผ2 ) locate the inflection, ๐‘ฆ๐‘ฆ๐‘Ž๐‘Ž , to the left (right) of the point I. This point
๐‘ฆ๐‘ฆ๐‘Ž๐‘Ž โ‰… ln[โˆ’ ln(1 โˆ’ ๐‘๐‘)] { or ๐‘ฆ๐‘ฆ๐‘Ž๐‘Ž โ‰… ln[โˆ’ ln(๐‘๐‘)] }. Using this formula estimate p.
- Estimate ๐›ผ๐›ผ1 and ๐›ผ๐›ผ2 :
๐‘๐‘
1โˆ’๐‘๐‘
โ€ข
If ๐›ผ๐›ผ1 โ‰ช ๐›ผ๐›ผ2 calculate point ๐‘ฆ๐‘ฆ1 = ln ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ1 โˆ’ ๐‘๐‘ +
๏ฟฝ๏ฟฝ and ๐‘ฆ๐‘ฆ2 = ln ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ.
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
โ€ข
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
Find the coordinates where ๐‘ฆ๐‘ฆ1 and ๐‘ฆ๐‘ฆ2 intersect the WPP curve. At these points
estimate ๐›ผ๐›ผ1 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ1 and ๐›ผ๐›ผ2 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ2 .
๐‘๐‘
1โˆ’๐‘๐‘
If ๐›ผ๐›ผ1 โ‰ซ ๐›ผ๐›ผ2 calculate point ๐‘ฆ๐‘ฆ1 = ln ๏ฟฝโˆ’ ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ and ๐‘ฆ๐‘ฆ2 = ln ๏ฟฝโˆ’๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ๐‘๐‘ +
๏ฟฝ๏ฟฝ.
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
Find the coordinates where ๐‘ฆ๐‘ฆ1 and ๐‘ฆ๐‘ฆ2 intersect the WPP curve. At these points
estimate ๐›ผ๐›ผ1 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ1 and ๐›ผ๐›ผ2 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ2 .
- Estimate ๐›ฝ๐›ฝ1 :
โ€ข
If ๐›ผ๐›ผ1 โ‰ช ๐›ผ๐›ผ2 draw and approximate ๐ฟ๐ฟ2 ensuring it intersects ๐›ผ๐›ผ2 . Estimate ๐›ฝ๐›ฝ2 from
the slope of ๐ฟ๐ฟ2 .
โ€ข
If ๐›ผ๐›ผ1 โ‰ซ ๐›ผ๐›ผ2 draw and approximate ๐ฟ๐ฟ1 ensuring it intersects ๐›ผ๐›ผ1 . Estimate ๐›ฝ๐›ฝ1 from
the slope of ๐ฟ๐ฟ1 .
- Find the point where the curve crosses ๐ฟ๐ฟ1 (point I). The slope at point I is:
๐›ฝ๐›ฝฬ… = ๐‘๐‘๐›ฝ๐›ฝ1 + (1 โˆ’ ๐‘๐‘)๐›ฝ๐›ฝ2
- Determine slope at point I and use to estimate ๐›ฝ๐›ฝ2
2-Fold Mixed Weibull Distribution
73
Common Shape Parameter ๐œท๐œท๐Ÿ๐Ÿ = ๐œท๐œท๐Ÿ๐Ÿ
๐›ผ๐›ผ
If ๏ฟฝ 2๏ฟฝ
๐›ผ๐›ผ1
๐›ฝ๐›ฝ1
โ‰ˆ1 then:
- Estimate the parameters of ๐›ผ๐›ผ1 and ๐›ฝ๐›ฝ1 from the ๐ฟ๐ฟ1 line (right asymptote).
- Estimate the parameter ๐‘๐‘ from the separation distance between the left and right
asymptotes.
- Draw a vertical line through ๐‘ฅ๐‘ฅ = ln(๐›ผ๐›ผ1 ). The intersection with the WPP can yield an
estimate of ๐›ผ๐›ผ2 using:
๐‘๐‘
+
exp(1)
๐‘ฆ๐‘ฆ1 = ๏ฟฝ
๐›ผ๐›ผ1
๐›ฝ๐›ฝ1
โ‰ช1 then:
- Find inflection point and estimate the y coordinate ๐‘ฆ๐‘ฆ๐‘Ÿ๐‘Ÿ . Estimate p using:
๐‘ฆ๐‘ฆ๐‘‡๐‘‡ โ‰… ln[โˆ’ ln(๐‘๐‘)]
๐‘๐‘
1โˆ’๐‘๐‘
- If ๐›ผ๐›ผ1 โ‰ช ๐›ผ๐›ผ2 calculate point ๐‘ฆ๐‘ฆ1 = ln ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ1 โˆ’ ๐‘๐‘ +
๏ฟฝ๏ฟฝ and ๐‘ฆ๐‘ฆ2 = ln ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ. Find the
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(1)
coordinates where ๐‘ฆ๐‘ฆ1 and ๐‘ฆ๐‘ฆ2 intersect the WPP curve. At these points estimate ๐›ผ๐›ผ1 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ1
and ๐›ผ๐›ผ2 = ๐‘’๐‘’ ๐‘ฅ๐‘ฅ2 .
- Using the left or right asymptote estimate ๐›ฝ๐›ฝ1 = ๐›ฝ๐›ฝ2 from the slope.
Maximum
Likelihood
Bayesian
MLE and Bayesian techniques can be used using numerical
methods however estimates obtained from the graphical methods
are useful for initial guesses. A literature review of MLE and
Bayesian methods is covered in (Murthy et al. 2003).
Description , Limitations and Uses
Characteristics
Hazard Rate Shape. The hazard rate can be approximated at its
limits by (Jiang & Murthy 1995):
๐‘†๐‘†๐‘†๐‘†๐‘†๐‘†๐‘†๐‘†๐‘†๐‘† ๐‘ก๐‘ก: โ„Ž(๐‘ก๐‘ก) โ‰ˆ ๐‘๐‘โ„Ž1 (๐‘ก๐‘ก)
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ ๐‘ก๐‘ก: โ„Ž(๐‘ก๐‘ก) โ‰ˆ โ„Ž1
This result proves that the hazard rate (increasing or decreasing) of
โ„Ž1 will dominate the limits of the mixed Weibull distribution. Therefore
the hazard rate cannot be a bathtub curve shape. Instead the
possible shapes of the hazard rate is:
โ€ข
Decreasing
โ€ข
Unimodal
โ€ข
Decreasing followed by unimodal (rollercoaster)
โ€ข
Bi-modal
The reason this distribution has been included as a bathtub
distribution is because on many occasions the hazard rate of a
complex product may follow the โ€œrollercoasterโ€ shape instead which
is given as decreasing followed by unimodal shape.
The shape of the hazard rate is only determined by the two shape
parameters ๐›ฝ๐›ฝ1 and ๐›ฝ๐›ฝ2 . A complete study on the characterization of
Mixed Wbl
๐›ผ๐›ผ
If ๏ฟฝ 2๏ฟฝ
1 โˆ’ ๐‘๐‘
๏ฟฝ
๐›ผ๐›ผ ๐›ฝ๐›ฝ1
exp ๏ฟฝ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ1
74
Bathtub Life Distributions
the 2-Fold Mixed Weibull Distribution is contained in Jiang and
Murthy 1998.
p Values
The mixture ratio, ๐‘๐‘๐‘–๐‘– , for each Weibull Distribution may be used to
estimate the percentage of each subpopulation. However this is not
a reliable measure and it known to be misleading (Berger & Sellke
1987)
N-Fold Distribution (Murthy et al. 2003)
A generalization to the 2-fold mixed Weibull distribution is the n-fold
case. This distribution is defined as:
๐‘›๐‘›
Mixed Wbl
๐‘“๐‘“(๐‘ก๐‘ก) = ๏ฟฝ ๐‘๐‘๐‘–๐‘– ๐‘“๐‘“๐‘–๐‘– (๐‘ก๐‘ก)
๐‘–๐‘–=1
๐›ฝ๐›ฝ
๐‘›๐‘›
๐›ฝ๐›ฝ๐‘–๐‘– ๐‘ก๐‘ก ๐›ฝ๐›ฝ๐‘–๐‘– โˆ’1 โˆ’๏ฟฝ๐›ผ๐›ผ๐‘ก๐‘ก ๏ฟฝ ๐‘–๐‘–
where ๐‘“๐‘“๐‘–๐‘– (๐‘ก๐‘ก) =
๐‘’๐‘’ ๐‘–๐‘– ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๏ฟฝ ๐‘๐‘๐‘–๐‘– = 1
๐›ผ๐›ผ๐‘–๐‘– ๐›ฝ๐›ฝ๐‘–๐‘–
and the hazard rate is given as:
๐‘›๐‘›
๐‘–๐‘–=1
โ„Ž(๐‘ก๐‘ก) = ๏ฟฝ ๐‘ค๐‘ค๐‘–๐‘– (๐‘ก๐‘ก)โ„Ž๐‘–๐‘– (๐‘ก๐‘ก)
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’
๐‘–๐‘–=1
๐‘ค๐‘ค๐‘–๐‘– (๐‘ก๐‘ก) =
๐‘๐‘๐‘–๐‘– ๐‘…๐‘…๐‘–๐‘– (๐‘ก๐‘ก)
โˆ‘๐‘›๐‘›๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘– ๐‘…๐‘…๐‘–๐‘– (๐‘ก๐‘ก)
It has been found that in many instances a higher number of folds
will not significantly increase the accuracy of the model but does
impose a significant overhead in the number of parameters to
estimate. The 3-Fold Weibull Mixture Distribution has been studied
by Jiang and Murthy 1996.
2-Fold Weibull 3-Parameter Distribution
A common variation to the model presented here is to have the
second Weibull distribution modeled with three parameters.
Resources
Books / Journals:
Jiang, R. & Murthy, D., 1995. Modeling Failure-Data by Mixture of
2 Weibull Distributions : A Graphical Approach. IEEE Transactions
on Reliability, 44, 477-488.
Murthy, D., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed.,
Wiley-Interscience.
Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed.,
Chapman & Hall/CRC.
Jiang, R. & Murthy, D., 1996. A mixture model involving three
Weibull distributions. In Proceedings of the Second Australiaโ€“
Japan Workshop on Stochastic Models in Engineering, Technology
and Management. Gold Coast, Australia, pp. 260-270.
2-Fold Mixed Weibull Distribution
75
Jiang, R. & Murthy, D., 1998. Mixture of Weibull distributions parametric characterization of failure rate function. Applied
Stochastic Models and Data Analysis, (14), 47-65.
Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20:
Advances in Reliability 1st ed., Elsevier Science & Technology.
Relationship to Other Distributions
Weibull
Distribution
Mixed Wbl
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Special Case:
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = 2๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น(๐‘ก๐‘ก; ๐›ผ๐›ผ = ๐›ผ๐›ผ1 , ๐›ฝ๐›ฝ = ๐›ฝ๐›ฝ1 , ๐‘๐‘ = 1)
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = 2๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น๐น(๐‘ก๐‘ก; ๐›ผ๐›ผ = ๐›ผ๐›ผ2 , ๐›ฝ๐›ฝ = ๐›ฝ๐›ฝ2 , ๐‘๐‘ = 0)
76
Bathtub Life Distributions
3.2. Exponentiated Weibull
Distribution
Probability Density Function - f(t)
0.30
ฮฑ=5 ฮฒ=1.5 v=0.2
ฮฑ=5 ฮฒ=1.5 v=0.5
ฮฑ=5 ฮฒ=1.5 v=0.8
ฮฑ=5 ฮฒ=1.5 v=3
0.25
Exp-Weibull
0.20
0.30
0.25
0.20
0.15
0.15
0.10
0.10
0.05
0.05
0.00
ฮฑ=5 ฮฒ=1 v=0.5
ฮฑ=5 ฮฒ=1.5 v=0.5
ฮฑ=5 ฮฒ=2 v=0.5
ฮฑ=5 ฮฒ=3 v=0.5
0.00
0
5
10
15
0
5
0
5
0
5
Cumulative Density Function - F(t)
1.00
1.00
0.80
0.80
0.60
0.60
0.40
0.40
0.20
0.20
0.00
0.00
0
5
10
15
Hazard Rate - h(t)
0.60
0.60
0.50
0.50
0.40
0.40
0.30
0.30
0.20
0.20
0.10
0.10
0.00
0.00
0
5
10
15
Exponentiated Weibull Continuous Distribution
77
Parameters & Description
๐›ผ๐›ผ > 0
Scale Parameter.
Parameters
๐›ผ๐›ผ
๐‘ฃ๐‘ฃ > 0
Shape Parameter.
Limits
๐‘ฃ๐‘ฃ
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ > 0
Shape Parameter.
๐‘ก๐‘ก โ‰ฅ 0
Distribution
Formulas
PDF
๐›ฝ๐›ฝ๐›ฝ๐›ฝ๐‘ก๐‘ก ๐›ฝ๐›ฝโˆ’1
๐‘ก๐‘ก ๐›ฝ๐›ฝ
๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ
๐›ฝ๐›ฝ
๐›ผ๐›ผ
๐›ผ๐›ผ
๐‘ฃ๐‘ฃโˆ’1
= ๐‘ฃ๐‘ฃ{๐น๐น๐‘Š๐‘Š (๐‘ก๐‘ก)}๐‘ฃ๐‘ฃโˆ’1 ๐‘“๐‘“๐‘Š๐‘Š (๐‘ก๐‘ก)
๐‘ก๐‘ก ๐›ฝ๐›ฝ
exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ
Where ๐น๐น๐‘Š๐‘Š (๐‘ก๐‘ก) and ๐‘“๐‘“๐‘Š๐‘Š (๐‘ก๐‘ก) are the cdf and pdf of the two parameter
Weibull distribution respectively.
t ฮฒ
๐น๐น(๐‘ก๐‘ก) = ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ
ฮฑ
= [๐น๐น๐‘Š๐‘Š (๐‘ก๐‘ก)]๐‘ฃ๐‘ฃ
CDF
t ฮฒ
R(t) = 1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ
ฮฑ
= 1 โˆ’ [๐น๐น๐‘Š๐‘Š (๐‘ก๐‘ก)]๐‘ฃ๐‘ฃ
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
v
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ
๐‘ฃ๐‘ฃ
๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ ๐›ฝ๐›ฝ
๏ฟฝ ๏ฟฝ๏ฟฝ
๐›ผ๐›ผ
๐‘ฃ๐‘ฃ
๐‘ก๐‘ก ๐›ฝ๐›ฝ
1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ
๐›ผ๐›ผ
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
Residual
Hazard Rate
๐‘…๐‘…(๐‘ก๐‘ก + x)
=
๐‘…๐‘…(๐‘ก๐‘ก)
v
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โ„Ž(๐‘ก๐‘ก) =
๐›ฝ๐›ฝ๐›ฝ๐›ฝ๏ฟฝ๐‘ก๐‘ก๏ฟฝ๐›ผ๐›ผ๏ฟฝ
๐‘ฃ๐‘ฃ
โˆž
๐‘ก๐‘ก ๐›ฝ๐›ฝ
โˆซt ๏ฟฝ1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ๐›ผ๐›ผ๏ฟฝ ๏ฟฝ๏ฟฝ ๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ฃ๐‘ฃ
๐›ฝ๐›ฝโˆ’1
๐‘ก๐‘ก ๐›ฝ๐›ฝ
1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’ ๏ฟฝ ๏ฟฝ ๏ฟฝ๏ฟฝ
๐›ผ๐›ผ
๐›ฝ๐›ฝ
๐‘ฃ๐‘ฃโˆ’1
๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’๏ฟฝ๐‘ก๐‘ก๏ฟฝ๐›ผ๐›ผ ๏ฟฝ ๏ฟฝ๏ฟฝ
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ
exp ๏ฟฝโˆ’๏ฟฝ๐‘ก๐‘ก๏ฟฝ๐›ผ๐›ผ ๏ฟฝ ๏ฟฝ
๐‘ฃ๐‘ฃ
1 โˆ’ ๏ฟฝ1 โˆ’ exp ๏ฟฝโˆ’๏ฟฝ๐‘ก๐‘ก๏ฟฝ๐›ผ๐›ผ๏ฟฝ ๏ฟฝ๏ฟฝ
Exp-Weibull
๐‘“๐‘“(๐‘ก๐‘ก) =
78
Bathtub Life Distributions
For small t: (Murthy et al. 2003, p.130)
๐›ฝ๐›ฝ๐›ฝ๐›ฝ ๐‘ก๐‘ก ๐›ฝ๐›ฝ๐›ฝ๐›ฝโˆ’1
โ„Ž(๐‘ก๐‘ก) โ‰ˆ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ ๐›ผ๐›ผ
For large t: (Murthy et al. 2003, p.130)
๐›ฝ๐›ฝ ๐‘ก๐‘ก ๐›ฝ๐›ฝโˆ’1
โ„Ž(๐‘ก๐‘ก) โ‰ˆ ๏ฟฝ ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ ๐›ผ๐›ผ
Properties and Moments
Median
๐›ผ๐›ผ ๏ฟฝโˆ’ ln ๏ฟฝ1 โˆ’ 2
For ๐›ฝ๐›ฝ๐›ฝ๐›ฝ > 1 the mode can be approximated
(Murthy et al. 2003, p.130):
๐‘ฃ๐‘ฃ
1 ๏ฟฝ๐›ฝ๐›ฝ(๐›ฝ๐›ฝ โˆ’ 8๐‘ฃ๐‘ฃ + 2๐›ฝ๐›ฝ๐›ฝ๐›ฝ + 9๐›ฝ๐›ฝ๐‘ฃ๐‘ฃ 2 )
1
๐›ผ๐›ผ ๏ฟฝ ๏ฟฝ
โˆ’ 1 โˆ’ ๏ฟฝ๏ฟฝ
2
๐›ฝ๐›ฝ๐›ฝ๐›ฝ
๐‘ฃ๐‘ฃ
Mode
Exp-Weibull
1๏ฟฝ
โˆ’1๏ฟฝ
๐‘ฃ๐‘ฃ ๏ฟฝ๏ฟฝ ๐›ฝ๐›ฝ
Mean - 1st Raw Moment
Solved numerically see Murthy et al. 2003,
p.128
Variance - 2nd Central Moment
100๐‘๐‘% Percentile Function
๐‘ก๐‘ก๐‘๐‘ = ๐›ผ๐›ผ ๏ฟฝโˆ’ ln ๏ฟฝ1 โˆ’ ๐‘๐‘
Parameter Estimation
1๏ฟฝ
1๏ฟฝ
๐›ฝ๐›ฝ
๐‘ฃ๐‘ฃ ๏ฟฝ๏ฟฝ
Plotting Method (Jiang & Murthy 1999)
Plot Points on
a Weibull
Probability Plot
X-Axis
Y-Axis
๐‘ฅ๐‘ฅ = ๐‘™๐‘™๐‘™๐‘™(๐‘ก๐‘ก)
๐‘ฆ๐‘ฆ = ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
1
๏ฟฝ๏ฟฝ
1 โˆ’ ๐น๐น
Using the Weibull Probability Plot the parameters can be estimated. (Jiang & Murthy
1999), provide a comprehensive coverage of this. A typical WPP for an exponentiated
Weibull distribution is:
3
2.5
2
1.5
1
0.5
0
-0.5
-1
-1.5
-2
-1
-0.5
0
0.5
1
1.5
2
2.5
3
WPP for exponentiated Weibull distribution ๐›ผ๐›ผ = 5, ๐›ฝ๐›ฝ = 2, ๐‘ฃ๐‘ฃ = 0.4
Asymptotes (Jiang & Murthy 1999):
As ๐‘ฅ๐‘ฅ โ†’ โˆ’โˆž (๐‘ก๐‘ก โ†’ 0) there exists an asymptote approximated by:
Exponentiated Weibull Continuous Distribution
79
๐‘ฆ๐‘ฆ โ‰ˆ ๐›ฝ๐›ฝ๐›ฝ๐›ฝ[๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ)]
As ๐‘ฅ๐‘ฅ โ†’ โˆž (๐‘ก๐‘ก โ†’ โˆž) the asymptote straight line can be approximated by:
๐‘ฆ๐‘ฆ โ‰ˆ ๐›ฝ๐›ฝ[๐‘ฅ๐‘ฅ โˆ’ ln(๐›ผ๐›ผ)]
Both asymptotes intersect the x-axis at ๐‘™๐‘™๐‘™๐‘™(๐›ผ๐›ผ) however both have different slopes unless
๐‘ฃ๐‘ฃ = 1 and the WPP is the same as a two parameter Weibull distribution.
Parameter Estimation
Plot estimates of the asymptotes ensuring they cross the x-axis at the same point. Use
the right asymptote to estimate ๐›ผ๐›ผ and ๐›ฝ๐›ฝ. Use the left asymptote to estimate ๐‘ฃ๐‘ฃ.
Bayesian
MLE and Bayesian techniques can be used in the standard way
however estimates obtained from the graphical methods are useful
for initial guesses when using numerical methods to solve equations.
A literature review of MLE and Bayesian methods is covered in
(Murthy et al. 2003).
Description , Limitations and Uses
Characteristics
PDF Shape: (Murthy et al. 2003, p.129)
๐œท๐œท๐œท๐œท <= 1. The pdf is monotonically decreasing, ๐‘“๐‘“(0) = โˆž.
๐œท๐œท๐œท๐œท = ๐Ÿ๐Ÿ. The pdf is monotonically decreasing, ๐‘“๐‘“(0) = 1/๐›ผ๐›ผ.
๐œท๐œท๐œท๐œท > 1. The pdf is unimodal. ๐‘“๐‘“(0) = 0.
The pdf shape is determined by ๐›ฝ๐›ฝ๐›ฝ๐›ฝ in a similar way to the
๐›ฝ๐›ฝ for a two parameter Weibull distribution.
Hazard Rate Shape: (Murthy et al. 2003, p.129)
๐œท๐œท โ‰ค ๐Ÿ๐Ÿ and ๐œท๐œท๐œท๐œท โ‰ค ๐Ÿ๐Ÿ. The hazard rate is monotonically
decreasing.
๐œท๐œท โ‰ฅ ๐Ÿ๐Ÿ and ๐œท๐œท๐œท๐œท โ‰ฅ ๐Ÿ๐Ÿ. The hazard rate is monotonically
increasing.
๐œท๐œท < 1 and ๐œท๐œท๐œท๐œท > 1. The hazard rate is unimodal.
๐œท๐œท > 1 and ๐œท๐œท๐œท๐œท < 1. The hazard rate is a bathtub curve.
Weibull Distribution. The Weibull distribution is a special case of
the expatiated distribution when ๐‘ฃ๐‘ฃ = 1. When ๐‘ฃ๐‘ฃ is an integer greater
than 1, then the cdf represents a multiplicative Weibull model.
Standard Exponentiated Weibull. (Xie et al. 2004) When ๐›ผ๐›ผ = 1 the
distribution is the standard exponentiated Weibull distribution with
cdf:
๐‘ฃ๐‘ฃ
๐น๐น(๐‘ก๐‘ก) = ๏ฟฝ1 โˆ’ exp๏ฟฝโˆ’๐‘ก๐‘ก ๐›ฝ๐›ฝ ๏ฟฝ๏ฟฝ
Minimum Failure Rate. (Xie et al. 2004) When the hazard rate is a
bathtub curve (๐›ฝ๐›ฝ > 1 and ๐›ฝ๐›ฝ๐›ฝ๐›ฝ < 1) then the minimum failure rate point
is:
๐‘ก๐‘ก โ€ฒ = ๐›ผ๐›ผ[โˆ’ ln(1 โˆ’ ๐‘ฆ๐‘ฆ1 )]
1๏ฟฝ
๐›ฝ๐›ฝ
Exp-Weibull
Maximum
Likelihood
80
Bathtub Life Distributions
where ๐‘ฆ๐‘ฆ1 is the solution to:
(๐›ฝ๐›ฝ โˆ’ 1)๐‘ฆ๐‘ฆ(1 โˆ’ ๐‘ฆ๐‘ฆ ๐‘ฃ๐‘ฃ ) + ๐›ฝ๐›ฝ ln(1 โˆ’ ๐‘ฆ๐‘ฆ) [1 + ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ โˆ’ ๐‘ฃ๐‘ฃ โˆ’ ๐‘ฆ๐‘ฆ ๐‘ฃ๐‘ฃ ] = 0
Maximum Mean Residual Life. (Xie et al. 2004) By solving the
derivative of the MRL function to zero, the maximum MRL is found
by solving to t:
1๏ฟฝ
๐›ฝ๐›ฝ
๐‘ก๐‘ก โˆ— = ๐›ผ๐›ผ[โˆ’ ln(1 โˆ’ ๐‘ฆ๐‘ฆ2 )]
Exp-Weibull
where ๐‘ฆ๐‘ฆ2 is the solution to:
×๏ฟฝ
โˆž
1
[โˆ’ ln(1โˆ’๐‘ฆ๐‘ฆ)] ๏ฟฝ๐›ฝ๐›ฝ
Resources
โˆ’1๏ฟฝ๐›ฝ๐›ฝ
๐›ฝ๐›ฝ๐›ฝ๐›ฝ(1 โˆ’ ๐‘ฆ๐‘ฆ)๐‘ฆ๐‘ฆ ๐‘ฃ๐‘ฃโˆ’1 [โˆ’ ln(1 โˆ’ ๐‘ฆ๐‘ฆ)]
๐›ฝ๐›ฝ
๐‘ฃ๐‘ฃ
[1 โˆ’ ๏ฟฝ1 โˆ’ ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘ โˆ’ (1 โˆ’ ๐‘ฆ๐‘ฆ ๐‘ฃ๐‘ฃ )2 = 0
Books / Journals:
Mudholkar, G. & Srivastava, D., 1993. Exponentiated Weibull family
for analyzing bathtub failure-rate data. Reliability, IEEE
Transactions on, 42(2), 299-302.
Jiang, R. & Murthy, D., 1999. The exponentiated Weibull family: a
graphical approach. Reliability, IEEE Transactions on, 48(1), 6872.
Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean
residual life and failure rate function for some generalized Weibull
distributions. Reliability Engineering and System Safety, 84(3),
293โ€“299.
Murthy, D., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed.,
Wiley-Interscience.
Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed.,
Chapman & Hall/CRC.
Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20:
Advances in Reliability 1st ed., Elsevier Science & Technology.
Relationship to Other Distributions
Weibull
Distribution
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Special Case:
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐›ผ๐›ผ = ๐›ผ๐›ผ, ๐›ฝ๐›ฝ = ๐›ฝ๐›ฝ, ๐‘ฃ๐‘ฃ = 1)
Modified Weibull Distribution
81
3.3. Modified Weibull Distribution
Probability Density Function - f(t)
b=0.5
b=0.8
b=1
b=2
0.5
1
0
0.5
ฮป=1
ฮป=5
ฮป=10
0
0.2
Cumulative Density Function - F(t)
b=0.5
b=0.8
b=1
b=2
0
0.5
1
a=5
a=10
a=20
0
0.5
ฮป=1
ฮป=5
ฮป=10
0
0.2
Hazard Rate - h(t)
0
5
0
5
0
Note: The hazard rate plots are on a different scale to the PDF and CDF
5
Modified Weibull
0
a=5
a=10
a=20
82
Bathtub Life Distributions
Parameters & Description
Parameters
Limits
๐‘Ž๐‘Ž
๐‘Ž๐‘Ž > 0
๐‘๐‘
๐‘๐‘ โ‰ฅ 0
๐œ†๐œ†
๐œ†๐œ† โ‰ฅ 0
Modified Weibull
Distribution
Scale Parameter.
Shape Parameter: The shape of the
distribution is completely determined
by b. When 0 < ๐‘๐‘ < 1 the distribution
has a bathtub shaped hazard rate.
Scale Parameter.
๐‘ก๐‘ก โ‰ฅ 0
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) = a(b + ฮปt) t bโˆ’1 exp(ฮปt) exp๏ฟฝโˆ’at b exp(ฮปt)๏ฟฝ
PDF
๐น๐น(๐‘ก๐‘ก) = 1 โˆ’ exp[โˆ’๐‘Ž๐‘Ž๐‘ก๐‘ก ๐‘๐‘ ๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(๐œ†๐œ†๐œ†๐œ†)]
CDF
R(t) = exp[โˆ’๐‘Ž๐‘Ž๐‘ก๐‘ก ๐‘๐‘ ๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(๐œ†๐œ†๐œ†๐œ†)]
Reliability
Mean Residual
Life
โˆž
๐‘ข๐‘ข(๐‘ก๐‘ก) = exp๏ฟฝ๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐‘๐‘ ๐‘’๐‘’ ๐œ†๐œ†๐œ†๐œ† ๏ฟฝ ๏ฟฝ exp๏ฟฝ๐‘Ž๐‘Ž๐‘ฅ๐‘ฅ ๐‘๐‘ ๐‘’๐‘’ ๐œ†๐œ†๐œ†๐œ† ๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ก๐‘ก
โ„Ž(๐‘ก๐‘ก) = a(b + ฮปt)t bโˆ’1 eฮปt
Hazard Rate
Properties and Moments
Median
Solved numerically (see 100p%)
Mode
Mean -
Solved numerically
1st
Variance -
Raw Moment
Solved numerically
2nd
Solved numerically
Central Moment
Solve for ๐‘ก๐‘ก๐‘๐‘ numerically:
ln(1 โˆ’ ๐‘๐‘)
๐‘ก๐‘ก๐‘๐‘b exp(ฮปt p ) = โˆ’
a
100p% Percentile Function
Parameter Estimation
Plotting Method (Lai et al. 2003)
Plot Points on
a Weibull
Probability Plot
X-Axis
Y-Axis
ln(๐‘ก๐‘ก๐‘–๐‘– )
1
๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
๏ฟฝ๏ฟฝ
1 โˆ’ ๐น๐น
Using the Weibull Probability Plot the parameters can be estimated. (Lai et al. 2003).
Asymptotes (Lai et al. 2003):
Modified Weibull Distribution
83
As ๐‘ฅ๐‘ฅ โ†’ โˆ’โˆž (๐‘ก๐‘ก โ†’ 0) the asymptote straight line can be approximated as:
๐‘ฆ๐‘ฆ โ‰ˆ ๐‘๐‘๐‘๐‘ + ln(๐‘Ž๐‘Ž)
As ๐‘ฅ๐‘ฅ โ†’ โˆž (๐‘ก๐‘ก โ†’ โˆž) the asymptote straight line can be approximated as (not used for
parameter estimate but more for model validity):
๐‘ฆ๐‘ฆ โ‰ˆ ๐œ†๐œ† exp(๐‘ฅ๐‘ฅ) = ๐œ†๐œ†๐œ†๐œ†
Intersections (Lai et al. 2003):
Y-Axis Intersection (0, ๐‘ฅ๐‘ฅ0 )
ln(๐‘Ž๐‘Ž) + ๐œ†๐œ† = ๐‘ฆ๐‘ฆ0
Solving these gives an approximate value for each parameter which can be used as an
initial guess for numerical methods solving MLE or Bayesian methods.
A typical WPP for an Modified Weibull Distribution is:
10
9
8
7
6
5
4
3
2
1
0
-1
-9
-8
-7
-6
-5
-4
-3
-2
-1
0
1
2
WPP for modified Weibull distribution ๐‘Ž๐‘Ž = 5, ๐‘๐‘ = 0.2, ๐œ†๐œ† = 10
Description , Limitations and Uses
Characteristics
Parameter Characteristics:(Lai et al. 2003)
๐ŸŽ๐ŸŽ < ๐‘๐‘ < 1 ๐’‚๐’‚๐’๐’๐’…๐’… ๐€๐€ > 0. The hazard rate has a bathtub curve
shape. โ„Ž(๐‘ก๐‘ก) โ†’ โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐‘ก๐‘ก โ†’ 0. โ„Ž(๐‘ก๐‘ก) โ†’ โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐‘ก๐‘ก โ†’ โˆž.
๐’ƒ๐’ƒ โ‰ฅ ๐Ÿ๐Ÿ ๐’‚๐’‚๐’‚๐’‚๐’‚๐’‚ ๐€๐€ > 0. Has an increasing hazard rate function.
โ„Ž(0) = 0. โ„Ž(๐‘ก๐‘ก) โ†’ โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐‘ก๐‘ก โ†’ โˆž .
๐€๐€ = ๐ŸŽ๐ŸŽ. The function has the same form as a Weibull
Distribution. โ„Ž(0) = ๐‘Ž๐‘Ž๐‘Ž๐‘Ž. โ„Ž(๐‘ก๐‘ก) โ†’ โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐‘ก๐‘ก โ†’ โˆž
Modified Weibull
ln(๐‘Ž๐‘Ž) + ๐‘๐‘๐‘ฅ๐‘ฅ0 + ๐œ†๐œ†๐‘’๐‘’ ๐‘ฅ๐‘ฅ0 = 0
X-Axis Intersection (๐‘ฆ๐‘ฆ0 , 0)
84
Bathtub Life Distributions
Minimum Failure Rate. (Xie et al. 2004) When the hazard rate is a
bathtub curve (0 < ๐‘๐‘ < 1 ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐œ†๐œ† > 0) then the minimum failure rate
point is given as:
โˆš๐‘๐‘ โˆ’ ๐‘๐‘
๐‘ก๐‘ก โˆ— =
๐œ†๐œ†
Modified Weibull
Maximum Mean Residual Life. (Xie et al. 2004) By solving the
derivative of the MRL function to zero, the maximum MRL is found
by solving to t:
โˆž
๐‘Ž๐‘Ž(๐‘๐‘ + ๐œ†๐œ†๐œ†๐œ†)๐‘ก๐‘ก ๐‘๐‘โˆ’1 ๐‘’๐‘’ ๐œ†๐œ†๐œ†๐œ† ๏ฟฝ exp(โˆ’๐‘Ž๐‘Ž๐‘ฅ๐‘ฅ ๐‘๐‘ ๐‘’๐‘’ (๐œ†๐œ†๐œ†๐œ†)๐‘‘๐‘‘๐‘‘๐‘‘ โˆ’ exp๏ฟฝ๐‘Ž๐‘Ž๐‘ก๐‘ก ๐‘๐‘ ๐‘’๐‘’ ๐œ†๐œ†๐œ†๐œ† ๏ฟฝ = 0
๐‘ก๐‘ก
Shape. The shape of the hazard rate cannot have a flat โ€œusage
periodโ€ and a strong โ€œwear outโ€ gradient.
Resources
Books / Journals:
Lai, C., Xie, M. & Murthy, D., 2003. A modified Weibull distribution.
IEEE Transactions on Reliability, 52(1), 33-37.
Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed.,
Wiley-Interscience.
Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean
residual life and failure rate function for some generalized Weibull
distributions. Reliability Engineering and System Safety, 84(3),
293โ€“299.
Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed.,
Chapman & Hall/CRC.
Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20:
Advances in Reliability 1st ed., Elsevier Science & Technology.
Relationship to Other Distributions
Weibull
Distribution
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Special Case:
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = ๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€(๐‘ก๐‘ก; a = ๐›ผ๐›ผ, ๐‘๐‘ = ๐›ฝ๐›ฝ, ๐œ†๐œ† = 0)
Univariate Continuous Distributions
85
4. Univariate Continuous
Distributions
Univariate Cont
86
Univariate Continuous Distributions
4.1. Beta Continuous Distribution
Probability Density Function - f(t)
ฮฑ=5 ฮฒ=2
ฮฑ=2 ฮฒ=2
ฮฑ=1 ฮฒ=2
ฮฑ=0.5 ฮฒ=2
3.00
2.50
2.50
2.00
2.00
1.50
1.50
1.00
1.00
0.50
0.50
0.00
0.00
0
ฮฑ=1 ฮฒ=1
ฮฑ=5 ฮฒ=5
ฮฑ=0.5 ฮฒ=0.5
0.5
1
0
0.2
0.4
0.6
0.8
1
Beta
Cumulative Density Function - F(t)
1.00
1.00
0.80
0.80
0.60
0.60
0.40
0.40
0.20
0.20
0.00
0.00
0
0.5
Hazard Rate - h(t)
10.00
9.00
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
1
ฮฑ=5 ฮฒ=2
ฮฑ=2 ฮฒ=2
ฮฑ=1 ฮฒ=2
ฮฑ=0.5 ฮฒ=2
ฮฑ=1 ฮฒ=1
ฮฑ=5 ฮฒ=5
ฮฑ=0.5 ฮฒ=0.5
0
0.2
0.4
0.6
0.8
1
4.00
3.50
3.00
2.50
2.00
1.50
1.00
ฮฑ=1 ฮฒ=1
ฮฑ=5 ฮฒ=5
ฮฑ=0.5 ฮฒ=0.5
0.50
0.00
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
Beta Continuous Distribution
87
Parameters & Description
๐›ผ๐›ผ
Parameters
๐›ผ๐›ผ > 0
Shape Parameter.
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ > 0
๐‘Ž๐‘Ž๐ฟ๐ฟ
โˆ’โˆž < ๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘๐‘๐‘ˆ๐‘ˆ
๐‘๐‘๐‘ˆ๐‘ˆ
Shape Parameter.
Lower Bound: ๐‘Ž๐‘Ž๐ฟ๐ฟ is the lower bound but
has also been called a location
parameter. In the standard Beta
distribution ๐‘Ž๐‘Ž๐ฟ๐ฟ = 0.
Upper Bound: ๐‘๐‘๐‘ˆ๐‘ˆ is the upper bound.
In the standard Beta distribution ๐‘๐‘๐‘ˆ๐‘ˆ = 1.
The scale parameter may also be defined
as ๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ .
๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘๐‘๐‘ˆ๐‘ˆ < โˆž
๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘ก๐‘ก โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
Limits
Distribution
Formulas
๐ต๐ต(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the Beta function, ๐ต๐ต๐‘ก๐‘ก (๐‘ก๐‘ก|๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the incomplete Beta function, ๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) is the
regularized Beta function, ฮ“(๐‘˜๐‘˜) is the complete gamma which is discussed in section 1.6.
General Form:
PDF
When ๐‘Ž๐‘Ž๐ฟ๐ฟ = 0, ๐‘๐‘๐‘ˆ๐‘ˆ = 1:
ฮ“(๐›ผ๐›ผ + ๐›ฝ๐›ฝ) (๐‘ก๐‘ก โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ )๐›ผ๐›ผโˆ’1 (๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘ก๐‘ก)๐›ฝ๐›ฝโˆ’1
.
(๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ )๐›ผ๐›ผ+๐›ฝ๐›ฝโˆ’1
ฮ“(๐›ผ๐›ผ)ฮ“(๐›ฝ๐›ฝ)
ฮ“(๐›ผ๐›ผ + ๐›ฝ๐›ฝ) ๐›ผ๐›ผโˆ’1
(1 โˆ’ ๐‘ก๐‘ก)๐›ฝ๐›ฝโˆ’1
. ๐‘ก๐‘ก
ฮ“(๐›ผ๐›ผ)ฮ“(๐›ฝ๐›ฝ)
1
=
. ๐‘ก๐‘ก ๐›ผ๐›ผโˆ’1 (1 โˆ’ ๐‘ก๐‘ก)๐›ฝ๐›ฝโˆ’1
๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘“๐‘“(๐‘ก๐‘ก|๐›ผ๐›ผ, ๐›ฝ๐›ฝ) =
CDF
Reliability
Conditional
Survivor Function
Mean Residual
Life
ฮ“(๐›ผ๐›ผ + ๐›ฝ๐›ฝ) ๐‘ก๐‘ก ๐›ผ๐›ผโˆ’1
(1 โˆ’ ๐‘ข๐‘ข)๐›ฝ๐›ฝโˆ’1 ๐‘‘๐‘‘๐‘‘๐‘‘
๏ฟฝ ๐‘ข๐‘ข
ฮ“(๐›ผ๐›ผ)ฮ“(๐›ฝ๐›ฝ) 0
๐ต๐ต๐‘ก๐‘ก (๐‘ก๐‘ก|๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
=
๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
= ๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
F(t) =
R(t) = 1 โˆ’ ๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
๐‘…๐‘…(๐‘ก๐‘ก + x) 1 โˆ’ ๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ|๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
=
๐‘…๐‘…(๐‘ก๐‘ก)
1 โˆ’ ๐ผ๐ผ๐‘ก๐‘ก (๐‘ก๐‘ก|๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
โˆž
โˆซt {๐ต๐ต(ฮฑ, ฮฒ) โˆ’ ๐ต๐ตx (x|ฮฑ, ฮฒ)}dx
๐ต๐ต(ฮฑ, ฮฒ) โˆ’ ๐ต๐ตt (t|ฮฑ, ฮฒ)
(Gupta and Nadarajah 2004, p.44)
๐‘ข๐‘ข(๐‘ก๐‘ก) =
Beta
๐‘“๐‘“(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ, ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ) =
88
Univariate Continuous Distributions
Hazard Rate
t ฮฑโˆ’1 (1 โˆ’ t)
๐ต๐ต(ฮฑ, ฮฒ) โˆ’ ๐ต๐ตt (t|ฮฑ, ฮฒ)
(Gupta and Nadarajah 2004, p.44)
โ„Ž(๐‘ก๐‘ก) =
Properties and Moments
Numerically solve for t:
๐‘ก๐‘ก0.5 = ๐น๐น โˆ’1 (๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Median
๐›ผ๐›ผโˆ’1
Mode
๐›ผ๐›ผ+๐›ฝ๐›ฝโˆ’2
Mean - 1st Raw Moment
for ๐›ผ๐›ผ > 1 ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž ๐›ฝ๐›ฝ > 1
ฮฑ
ฮฑ+ฮฒ
ฮฑฮฒ
(ฮฑ + ฮฒ)2 (ฮฑ + ฮฒ + 1)
Variance - 2nd Central Moment
2(ฮฒ โˆ’ ฮฑ)๏ฟฝฮฑ + ฮฒ + 1
Skewness - 3rd Central Moment
3
6[๐›ผ๐›ผ + ๐›ผ๐›ผ
Beta
Excess kurtosis - 4th Central Moment
(ฮฑ + ฮฒ + 2)๏ฟฝฮฑฮฒ
2 (1
โˆ’ 2๐›ฝ๐›ฝ) + ๐›ฝ๐›ฝ2 (1 + ๐›ฝ๐›ฝ) โˆ’ 2๐›ผ๐›ผ๐›ผ๐›ผ(2 + ๐›ฝ๐›ฝ)]
๐›ผ๐›ผ๐›ผ๐›ผ(๐›ผ๐›ผ + ๐›ฝ๐›ฝ + 2)(๐›ผ๐›ผ + ๐›ฝ๐›ฝ + 3)
1F1 (ฮฑ; ฮฑ
+ ฮฒ; it)
Where 1F1 is the confluent hypergeometric
function defined as:
โˆž
(ฮฑ)k ๐‘ฅ๐‘ฅ ๐‘˜๐‘˜
(ฮฑ;
ฮฒ;
x)
=
๏ฟฝ
.
F
1 1
(ฮฒ)k ๐‘˜๐‘˜!
Characteristic Function
k=0
(Gupta and Nadarajah 2004, p.44)
Numerically solve for t:
๐‘ก๐‘ก๐‘๐‘ = ๐น๐น โˆ’1 (๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
100p% Percentile Function
Parameter Estimation
Maximum Likelihood Function
Likelihood
Functions
Log-Likelihood
Functions
โˆ‚ฮ›
=0
โˆ‚ฮฑ
L(ฮฑ, ฮฒ|E) =
nF
ฮ“(๐›ผ๐›ผ + ๐›ฝ๐›ฝ)nF
๐›ฝ๐›ฝโˆ’1
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๐›ผ๐›ผโˆ’1 ๏ฟฝ1 โˆ’ ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ๏ฟฝ
.
ฮ“(๐›ผ๐›ผ)ฮ“(๐›ฝ๐›ฝ)
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
failures
ฮ›(ฮฑ, ฮฒ|E) = nF {ln[๐›ค๐›ค(๐›ผ๐›ผ + ๐›ฝ๐›ฝ) โˆ’ ๐‘™๐‘™๐‘™๐‘™[๐›ค๐›ค(๐›ผ๐›ผ)] โˆ’ ๐‘™๐‘™๐‘™๐‘™[๐›ค๐›ค(๐›ฝ๐›ฝ)]}
+(ฮฑ โˆ’
where ๐œ“๐œ“(๐‘ฅ๐‘ฅ) =
๐‘‘๐‘‘
๐‘‘๐‘‘๐‘‘๐‘‘
nF
1) ๏ฟฝ ln๏ฟฝt Fi ๏ฟฝ
i=1
nF
+ (ฮฒ โˆ’ 1) ๏ฟฝ ln(1 โˆ’ t Fi )
๐œ“๐œ“(ฮฑ) โˆ’ ๐œ“๐œ“(ฮฑ + ฮฒ) =
nF
i=1
1
๏ฟฝ ln(t Fi )
nF
i=1
ln[ฮ“(๐‘ฅ๐‘ฅ)] is the digamma function see section 1.6.7.
Beta Continuous Distribution
89
(Johnson et al. 1995, p.223)
โˆ‚ฮ›
=0
โˆ‚ฮฒ
Point
Estimates
Fisher
Information
Matrix
nF
1
๐œ“๐œ“(ฮฒ) โˆ’ ๐œ“๐œ“(ฮฑ + ฮฒ) =
๏ฟฝ ln(1 โˆ’ t i )
nF
i=1
(Johnson et al. 1995, p.223)
Point estimates are obtained by using numerical methods to solve the
simultaneous equations above.
๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ) โˆ’ ๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ + ๐›ฝ๐›ฝ)
๐ผ๐ผ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = ๏ฟฝ
โˆ’๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ + ๐›ฝ๐›ฝ)
๐‘‘๐‘‘ 2
โˆ’๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ + ๐›ฝ๐›ฝ)
๏ฟฝ
โˆ’ ๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ + ๐›ฝ๐›ฝ)
๐œ“๐œ“ โ€ฒ (๐›ฝ๐›ฝ)
โˆ’2
where ๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) = 2 ๐‘™๐‘™๐‘™๐‘™ฮ“(๐‘ฅ๐‘ฅ) = โˆ‘โˆž
is the Trigamma function.
๐‘–๐‘–=0(๐‘ฅ๐‘ฅ + ๐‘–๐‘–)
๐‘‘๐‘‘๐‘ฅ๐‘ฅ
See section 1.6.8. (Yang and Berger 1998, p.5)
Confidence
Intervals
For a large number of samples the Fisher information matrix can be
used to estimate confidence intervals. See section 1.4.7.
Bayesian
Non-informative Priors
โˆšdet๏ฟฝ๐ผ๐ผ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ)๏ฟฝ
where ๐ผ๐ผ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) is given above.
Beta
Jefferyโ€™s Prior
Conjugate Priors
UOI
๐‘๐‘
from
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘)
๐‘๐‘
from
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘, ๐‘›๐‘›)
Example
Likelihood
Model
Bernoulli
Binomial
Evidence
๐‘˜๐‘˜ failures
in 1 trail
๐‘˜๐‘˜ failures
in ๐‘›๐‘› trials
Dist. of
UOI
Prior
Para
Posterior
Parameters
Beta
๐›ผ๐›ผ0 , ๐›ฝ๐›ฝ0
๐›ผ๐›ผ = ๐›ผ๐›ผ๐‘œ๐‘œ + ๐‘˜๐‘˜
๐›ฝ๐›ฝ = ๐›ฝ๐›ฝ๐‘œ๐‘œ + 1 โˆ’ ๐‘˜๐‘˜
Beta
๐›ผ๐›ผ๐‘œ๐‘œ , ๐›ฝ๐›ฝ๐‘œ๐‘œ
Description , Limitations and Uses
๐›ผ๐›ผ = ๐›ผ๐›ผ๐‘œ๐‘œ + ๐‘˜๐‘˜
ฮฒ = ฮฒo + n โˆ’ k
For examples on the use of the beta distribution as a conjugate prior
see the binomial distribution.
A non-homogeneous (operate in different environments) population
of 5 switches have the following probabilities of failure on demand.
0.1176,
0.1488,
0.3684,
0.8123,
0.9783
Estimate the population variability function:
nF
1
๏ฟฝ ln(t Fi ) = โˆ’1.0549
nF
i=1
90
Univariate Continuous Distributions
nF
1
๏ฟฝ ln(1 โˆ’ t i ) = โˆ’1.25
nF
i=1
Numerically Solving:
๐œ“๐œ“(ฮฑ) + 1.0549 = ๐œ“๐œ“(ฮฒ) + 1.25
Gives:
๐›ผ๐›ผ๏ฟฝ = 0.7369
๐‘๐‘๏ฟฝ = 0.6678
1.5924 โˆ’1.0207
๐ผ๐ผ(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) = ๏ฟฝ
๏ฟฝ
โˆ’1.0207 2.0347
โˆ’1
โˆ’1
0.1851 0.0929
๏ฟฝ
๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ๏ฟฝ๏ฟฝ = ๏ฟฝ๐‘›๐‘›๐น๐น ๐ผ๐ผ๏ฟฝ๐›ผ๐›ผ๏ฟฝ, ๐›ฝ๐›ฝฬ‚ ๏ฟฝ๏ฟฝ = ๏ฟฝ
0.0929 0.1449
Beta
90% confidence interval for ๐›ผ๐›ผ:
๐›ท๐›ทโˆ’1 (0.95)โˆš0.1851
๐›ท๐›ทโˆ’1 (0.95)โˆš0.1851
๏ฟฝ๐›ผ๐›ผ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ,
๐›ผ๐›ผ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ๏ฟฝ
โˆ’๐›ผ๐›ผ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ
[0.282, 1.92]
Characteristics
90% confidence interval for ๐›ฝ๐›ฝ:
๐›ท๐›ทโˆ’1 (0.95)โˆš0.1449
๐›ท๐›ท โˆ’1 (0.95)โˆš0.1449
๏ฟฝ๐›ฝ๐›ฝฬ‚. exp ๏ฟฝ
๏ฟฝ,
๐›ฝ๐›ฝฬ‚ . exp ๏ฟฝ
๏ฟฝ๏ฟฝ
ฬ‚
โˆ’๐›ฝ๐›ฝ
๐›ฝ๐›ฝฬ‚
[0.262, 1.71]
The Beta distribution was originally known as a Pearson Type I
distribution (and Type II distribution which is a special case of a Type
I).
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) is the mirror distribution of ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ฝ๐›ฝ, ๐›ผ๐›ผ). If ๐‘‹๐‘‹~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) and
let ๐‘Œ๐‘Œ = 1 โˆ’ ๐‘‹๐‘‹ then ๐‘Œ๐‘Œ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ฝ๐›ฝ, ๐›ผ๐›ผ).
Location / Scale Parameters (NIST Section 1.3.6.6.17)
๐‘Ž๐‘Ž๐ฟ๐ฟ and ๐‘๐‘๐‘ˆ๐‘ˆ can be transformed into a location and scale parameter:
๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™ = ๐‘Ž๐‘Ž๐ฟ๐ฟ
๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘ ๐‘  = ๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ
Shapes(Gupta and Nadarajah 2004, p.41):
๐ŸŽ๐ŸŽ < ๐›ผ๐›ผ < 1. As ๐‘ฅ๐‘ฅ โ†’ 0, ๐‘“๐‘“(๐‘ฅ๐‘ฅ) โ†’ โˆž.
๐ŸŽ๐ŸŽ < ๐›ฝ๐›ฝ < 1. As ๐‘ฅ๐‘ฅ โ†’ 1, ๐‘“๐‘“(๐‘ฅ๐‘ฅ) โ†’ โˆž.
๐œถ๐œถ > 1, ๐œท๐œท > 1. As ๐‘ฅ๐‘ฅ โ†’ 0, ๐‘“๐‘“(๐‘ฅ๐‘ฅ) โ†’ 0. There is a single mode at
๐›ผ๐›ผโˆ’1
.
๐›ผ๐›ผ+๐›ฝ๐›ฝโˆ’2
๐œถ๐œถ < 1, ๐œท๐œท < 1. The distribution is a U shape. There is a
๐›ผ๐›ผโˆ’1
single anti-mode at
.
๐›ผ๐›ผ+๐›ฝ๐›ฝโˆ’2
๐œถ๐œถ > 0, ๐œท๐œท > 0. There exists inflection points at:
(๐›ผ๐›ผ โˆ’ 1)(๐›ฝ๐›ฝ โˆ’ 1)
๐›ผ๐›ผ โˆ’ 1
1
±
.๏ฟฝ
๐›ผ๐›ผ + ๐›ฝ๐›ฝ โˆ’ 2 ๐›ผ๐›ผ + ๐›ฝ๐›ฝ โˆ’ 2
๐›ผ๐›ผ + ๐›ฝ๐›ฝ โˆ’ 3
Beta Continuous Distribution
91
๐œถ๐œถ = ๐œท๐œท. The distribution is symmetrical about ๐‘ฅ๐‘ฅ = 0.5. As
๐›ผ๐›ผ = ๐›ฝ๐›ฝ becomes large, the beta distribution approaches the
normal distribution. The Standard Uniform Distribution
arises when ๐›ผ๐›ผ = ๐›ฝ๐›ฝ = 1.
๐œถ๐œถ = ๐Ÿ๐Ÿ, ๐œท๐œท = ๐Ÿ๐Ÿ or ๐œถ๐œถ = ๐Ÿ๐Ÿ, ๐œท๐œท = ๐Ÿ๐Ÿ. Straight line.
(๐œถ๐œถโ€“ ๐Ÿ๐Ÿ)(๐œท๐œทโ€“ ๐Ÿ๐Ÿ) < 0. J Shaped.
Hazard Rate and MRL (Gupta and Nadarajah 2004, p.45):
๐œถ๐œถ โ‰ฅ ๐Ÿ๐Ÿ, ๐œท๐œท โ‰ฅ ๐Ÿ๐Ÿ. โ„Ž(๐‘ก๐‘ก) is increasing. ๐‘ข๐‘ข(๐‘ก๐‘ก) is decreasing.
๐œถ๐œถ โ‰ค ๐Ÿ๐Ÿ, ๐œท๐œท โ‰ค ๐Ÿ๐Ÿ. โ„Ž(๐‘ก๐‘ก) is decreasing. ๐‘ข๐‘ข(๐‘ก๐‘ก) is increasing.
๐œถ๐œถ > 1, 0 < ๐›ฝ๐›ฝ < 1. โ„Ž(๐‘ก๐‘ก) is bathtub shaped and ๐‘ข๐‘ข(๐‘ก๐‘ก) is an
upside down bathtub shape.
๐ŸŽ๐ŸŽ < ๐›ผ๐›ผ < 1, ๐œท๐œท > 1. โ„Ž(๐‘ก๐‘ก) is upside down bathtub shaped and
๐‘ข๐‘ข(๐‘ก๐‘ก) is bathtub shape.
Parameter Model. The Beta distribution is often used to model
parameters which are constrained to take place between an interval.
In particular the distribution of a probability parameter 0 โ‰ค ๐‘๐‘ โ‰ค 1 is
popular with the Beta distribution.
Proportions. Used to model proportions. An example of this is the
likelihood ratios for estimating uncertainty.
Online:
http://mathworld.wolfram.com/BetaDistribution.html
http://en.wikipedia.org/wiki/Beta_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web
calculator)
http://www.itl.nist.gov/div898/handbook/eda/section3/eda366h.htm
Resources
Books:
Gupta, A.K. & Nadarajah, S., 2004. Handbook of beta distribution
and its applications, CRC Press.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous
Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience.
Relationship to Other Distributions
Beta
Applications
Bayesian Analysis. The Beta distribution is often used as a
conjugate prior in Bayesian analysis for the Bernoulli, Binomial and
Geometric Distributions to produce closed form posteriors. The
Beta(0,0) distribution is an improper prior sometimes used to
represent ignorance of parameter values. The Beta(1,1) is a
standard uniform distribution which may be used as a noninformative prior. When used as a conjugate prior to a Bernoulli or
Binomial process the parameter ๐›ผ๐›ผ may represent the number of
successes and ๐›ฝ๐›ฝ the total number of failures with the total number of
trials being ๐‘›๐‘› = ๐›ผ๐›ผ + ๐›ฝ๐›ฝ.
92
Univariate Continuous Distributions
Let
Chi-square
Distribution
Xi ~๐œ’๐œ’ 2 (๐‘ฃ๐‘ฃ๐‘–๐‘– )
Then
2
๐œ’๐œ’ (๐‘ก๐‘ก; ๐‘ฃ๐‘ฃ)
๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ก๐‘ก; ๐‘Ž๐‘Ž, ๐‘๐‘)
Beta
Normal
Distribution
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ)
Gamma
Distribution
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; k, ฮป)
Dirichlet
Distribution
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ (๐‘ฅ๐‘ฅ; ๐œถ๐œถ)
1
Y=
1
X1
X1 + X 2
Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐›ผ๐›ผ = 2๐‘ฃ๐‘ฃ1 , ๐›ฝ๐›ฝ = 2๐‘ฃ๐‘ฃ2 ๏ฟฝ
Let
Uniform
Distribution
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Then
Xi ~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(0,1)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
X1 โ‰ค X2 โ‰ค โ‹ฏ โ‰ค Xn
Xr ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘Ÿ๐‘Ÿ, ๐‘›๐‘› โˆ’ ๐‘Ÿ๐‘Ÿ + 1)
Where ๐‘›๐‘› and ๐‘Ÿ๐‘Ÿ are integers.
Special Case:
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; 1,1, ๐‘Ž๐‘Ž, ๐‘๐‘) = ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ก๐‘ก; ๐‘Ž๐‘Ž, ๐‘๐‘)
For large ๐›ผ๐›ผ and ๐›ฝ๐›ฝ with fixed ๐›ผ๐›ผ/๐›ฝ๐›ฝ:
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) โ‰ˆ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµ =
ฮฑ
ฮฑฮฒ
, ๐œŽ๐œŽ = ๏ฟฝ
๏ฟฝ
(ฮฑ + ฮฒ)2 (ฮฑ + ฮฒ + 1)
ฮฑ+ฮฒ
As ๐›ผ๐›ผ and ๐›ฝ๐›ฝ increase the mean remains constant and the variance is
reduced.
Let
Then
๐‘‹๐‘‹1 , ๐‘‹๐‘‹2 ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(k i , ฮปi )
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ = ๐‘˜๐‘˜1 , ๐›ฝ๐›ฝ = ๐‘˜๐‘˜2 )
Y=
X1
X1 + X 2
Special Case:
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘=1 (x; [ฮฑ1 , ฮฑ0 ]) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜ = ๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ = ๐›ผ๐›ผ1 , ๐›ฝ๐›ฝ = ๐›ผ๐›ผ0 )
Birnbaum Sanders Continuous Distribution
93
4.2. Birnbaum Saunders Continuous
Distribution
Probability Density Function - f(t)
ฮฑ=0.2 ฮฒ=1
ฮฑ=0.5 ฮฒ=1
ฮฑ=1 ฮฒ=1
ฮฑ=2.5 ฮฒ=1
2.0
1.5
ฮฑ=1 ฮฒ=0.5
ฮฑ=1 ฮฒ=1
ฮฑ=1 ฮฒ=2
ฮฑ=1 ฮฒ=4
1.4
1.2
1.0
0.8
1.0
0.6
0.4
0.5
0.2
0.0
0.0
0
1
2
3
0
1
2
3
0
1
2
3
0
1
2
3
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0.8
0.6
0.4
ฮฑ=0.2 ฮฒ=1
ฮฑ=0.5 ฮฒ=1
ฮฑ=1 ฮฒ=1
ฮฑ=2.5 ฮฒ=1
0.2
0.0
0
1
2
Birnbaum Sanders
Cumulative Density Function - F(t)
3
Hazard Rate - h(t)
2.5
1.6
2.0
1.4
1.2
1.5
1.0
0.8
1.0
0.6
0.5
0.4
0.0
0.0
0.2
0
1
2
3
94
Univariate Continuous Distributions
Parameters & Description
Parameters
Limits
ฮฒ
Scale parameter. ฮฒ is the scale
parameter equal to the median.
ฮฒ>0
๐›ผ๐›ผ
ฮฑ>0
Shape parameter.
0<t< โˆž
Distribution
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) =
=
Birnbaum Sanders
PDF
๏ฟฝtโ„ฮฒ + ๏ฟฝฮฒโ„t
2๐›ผ๐›ผ๐›ผ๐›ผโˆš2๐œ‹๐œ‹
2
1 ๏ฟฝtโ„ฮฒ โˆ’ ๏ฟฝฮฒโ„t
exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐›ผ๐›ผ
๏ฟฝtโ„ฮฒ + ๏ฟฝฮฒโ„t
ฯ•(z)
2๐›ผ๐›ผ๐›ผ๐›ผ
where ๐œ™๐œ™(๐‘ง๐‘ง) is the standard normal pdf and:
๏ฟฝtโ„ฮฒ โˆ’ ๏ฟฝฮฒโ„t
๐‘ง๐‘ง๐ต๐ต๐ต๐ต =
๐›ผ๐›ผ
๏ฟฝtโ„ฮฒ โˆ’ ๏ฟฝฮฒโ„t
๏ฟฝ
๐›ผ๐›ผ
= ฮฆ(๐‘ง๐‘ง๐ต๐ต๐ต๐ต )
CDF
๐น๐น(๐‘ก๐‘ก) = ฮฆ ๏ฟฝ
Reliability
R(t) = ฮฆ ๏ฟฝ
๏ฟฝฮฒโ„t โˆ’ ๏ฟฝtโ„ฮฒ
๏ฟฝ
๐›ผ๐›ผ
= ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )
Where
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean Residual
Life
Hazard Rate
Cumulative
Hazard Rate
๐‘ง๐‘ง๐ต๐ต๐ต๐ต =
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
๏ฟฝtโ„ฮฒ โˆ’ ๏ฟฝฮฒโ„t
,
๐›ผ๐›ผ
โ€ฒ )
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ) ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต
=
๐‘…๐‘…(๐‘ก๐‘ก)
ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )
โ€ฒ
๐‘ง๐‘ง๐ต๐ต๐ต๐ต
=
๏ฟฝ(t + x)โ„ฮฒ โˆ’ ๏ฟฝฮฒโ„(t + x)
๐›ผ๐›ผ
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
โˆž
โˆซ ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ข๐‘ข(๐‘ก๐‘ก) = ๐‘ก๐‘ก
ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )
โ„Ž(๐‘ก๐‘ก) =
๏ฟฝtโ„ฮฒ + ๏ฟฝฮฒโ„t ฯ•(zBS )
๏ฟฝ
๏ฟฝ
2๐›ผ๐›ผ๐›ผ๐›ผ
ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ ln[ฮฆ(โˆ’๐‘ง๐‘ง๐ต๐ต๐ต๐ต )]
Birnbaum Sanders Continuous Distribution
95
Properties and Moments
ฮฒ
Median
Numerically solve for ๐‘ก๐‘ก:
๐‘ก๐‘ก 3 + ๐›ฝ๐›ฝ(1 + ๐›ผ๐›ผ 2 )๐‘ก๐‘ก 2 + ๐›ฝ๐›ฝ2 (3๐›ผ๐›ผ 2 โˆ’ 1)๐‘ก๐‘ก โˆ’ ๐›ฝ๐›ฝ3 = 0
Mode
Mean - 1st Raw Moment
ฮฒ ๏ฟฝ1 +
Variance - 2nd Central Moment
ฮฑ2
๏ฟฝ
2
ฮฑ2 ฮฒ2 ๏ฟฝ1 +
5ฮฑ2
๏ฟฝ
4
4ฮฑ(11ฮฑ2 + 6)
Skewness - 3rd Central Moment
3
(5ฮฑ2 + 4)2
(Lemonte et al. 2007)
Excess kurtosis - 4th Central Moment
6ฮฑ2 (93ฮฑ2 + 40)
(5ฮฑ2 + 4)2
(Lemonte et al. 2007)
2
ฮฒ
๐‘ก๐‘ก๐›พ๐›พ = ๏ฟฝฮฑฮฆโˆ’1 (ฮณ) + ๏ฟฝ4 + [ฮฑฮฆโˆ’1 (ฮณ)]2 ๏ฟฝ
4
100๐›พ๐›พ % Percentile Function
Parameter Estimation
Maximum Likelihood Function
Likelihood
Function
Log-Likelihood
Function
โˆ‚ฮ›
=0
โˆ‚ฮฑ
โˆ‚ฮ›
=0
โˆ‚ฮฒ
MLE
Point
For complete data:
nF
2
1 ๏ฟฝt i โ„ฮฒ โˆ’ ๏ฟฝฮฒโ„t i
๏ฟฝt i โ„ฮฒ + ๏ฟฝฮฒโ„t i
๐ฟ๐ฟ(๐œƒ๐œƒ, ๐›ผ๐›ผ|๐ธ๐ธ) = ๏ฟฝ
exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐›ผ๐›ผ
2๐›ผ๐›ผ๐‘ก๐‘ก๐‘–๐‘– โˆš2๐œ‹๐œ‹
i=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
1
2
๐‘›๐‘›๐น๐น
3
๐‘›๐‘›๐น๐น
๐›ฝ๐›ฝ
๐›ฝ๐›ฝ 2
1
๐‘ก๐‘ก๐‘–๐‘– ๐›ฝ๐›ฝ
ฮ›(๐›ผ๐›ผ, ๐›ฝ๐›ฝ|๐ธ๐ธ) = โˆ’nF ln(ฮฑฮฒ) + ๏ฟฝ ln ๏ฟฝ๏ฟฝ ๏ฟฝ + ๏ฟฝ ๏ฟฝ ๏ฟฝ โˆ’ 2 ๏ฟฝ ๏ฟฝ + โˆ’ 2๏ฟฝ
๐‘ก๐‘ก๐‘–๐‘–
๐‘ก๐‘ก๐‘–๐‘–
2๐›ผ๐›ผ
๐›ฝ๐›ฝ ๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–=1
๐‘–๐‘–=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
nF
2
1
๐›ฝ๐›ฝ
1
= โˆ’ ๏ฟฝ1 + 2 ๏ฟฝ + 3 ๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– + 3 ๏ฟฝ = 0
โˆ‚ฮฑ
ฮฑ
ฮฑ
ฮฑ ฮฒ
๐›ผ๐›ผ
๐‘ก๐‘ก๐‘–๐‘–
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
failures
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
nF
1
1
1
1
=โˆ’ +๏ฟฝ
+
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– โˆ’ 2 ๏ฟฝ = 0
โˆ‚ฮฒ
2ฮฒ
๐‘ก๐‘ก๐‘–๐‘– + ๐›ฝ๐›ฝ 2๐›ผ๐›ผ 2 ๐›ฝ๐›ฝ2
2๐›ผ๐›ผ
๐‘ก๐‘ก๐‘–๐‘–
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
๐‘–๐‘–=1
๐‘–๐‘–=1
๐›ฝ๐›ฝฬ‚ is found by solving:
failures
Birnbaum Sanders
3+
96
Univariate Continuous Distributions
Estimates
where
๐‘›๐‘›๐น๐น
๐›ฝ๐›ฝ2 โˆ’ ๐›ฝ๐›ฝ[2๐‘…๐‘… + ๐‘”๐‘”(๐›ฝ๐›ฝ)] + ๐‘…๐‘…[๐‘†๐‘† + ๐‘”๐‘”(๐›ฝ๐›ฝ)] = 0
1
1
๐‘”๐‘”(๐›ฝ๐›ฝ) = ๏ฟฝ ๏ฟฝ
๏ฟฝ
๐‘›๐‘›
๐›ฝ๐›ฝ + ๐‘ก๐‘ก๐‘–๐‘–
โˆ’1
๐‘–๐‘–=1
Point estimates for ๐›ผ๐›ผ๏ฟฝ is:
(Lemonte et al. 2007)
where
Birnbaum Sanders
,
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
โˆ’1
1
1
๐‘…๐‘… = ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–=1
๐‘†๐‘† ๐›ฝ๐›ฝฬ‚
๐›ผ๐›ผ๏ฟฝ = ๏ฟฝ + โˆ’ 2
๐›ฝ๐›ฝฬ‚ ๐‘…๐‘…
2
โŽก 2
๐›ผ๐›ผ
๐ผ๐ผ(๐œƒ๐œƒ, ๐›ผ๐›ผ) = โŽข
โŽข0
โŽฃ
Fisher
Information
100๐›พ๐›พ%
Confidence
Intervals
๐‘›๐‘›๐น๐น
1
๐‘†๐‘† =
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– ,
๐‘›๐‘›๐น๐น
โŽค
โŽฅ
๐›ผ๐›ผ(2๐œ‹๐œ‹)โˆ’1โ„2 ๐‘˜๐‘˜(๐›ผ๐›ผ) + 1โŽฅ
โŽฆ
๐›ผ๐›ผ 2 ๐›ฝ๐›ฝ2
0
๐œ‹๐œ‹
2
2
๐‘˜๐‘˜(๐›ผ๐›ผ) = ๐›ผ๐›ผ๏ฟฝ โˆ’ ๐œ‹๐œ‹ exp ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ1 โˆ’ ฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ
2
๐›ผ๐›ผ
๐›ผ๐›ผ
(Lemonte et al. 2007)
Calculated from the Fisher information matrix. See section 1.4.7. For a
literature review of proposed confidence intervals see (Lemonte et al.
2007).
Description , Limitations and Uses
Example
5 components are put on a test with the following failure times:
98, 116, 2485, 2526, , 2920 hours
๐‘›๐‘›๐น๐น
1
๐‘†๐‘† =
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– = 1629
๐‘›๐‘›๐น๐น
๐‘–๐‘–=1
โˆ’1
๐‘›๐‘›๐น๐น
Solving:
1
1
๐‘…๐‘… = ๏ฟฝ ๏ฟฝ ๏ฟฝ
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘–
๐‘›๐‘›๐น๐น
๐‘–๐‘–=1
= 250.432
โˆ’1
๐‘›๐‘›๐น๐น
โˆ’1
1
1
1
1
๏ฟฝ ๏ฟฝ + ๐‘…๐‘… ๏ฟฝ๐‘†๐‘† + ๏ฟฝ ๏ฟฝ
๏ฟฝ ๏ฟฝ=0
๐›ฝ๐›ฝ โˆ’ ๐›ฝ๐›ฝ ๏ฟฝ2๐‘…๐‘… + ๏ฟฝ ๏ฟฝ
๐‘›๐‘›
๐›ฝ๐›ฝ + ๐‘ก๐‘ก๐‘–๐‘–
๐‘›๐‘›
๐›ฝ๐›ฝ + ๐‘ก๐‘ก๐‘–๐‘–
2
๐‘–๐‘–=1
ฮฒ๏ฟฝ = 601.949
๐‘–๐‘–=1
๐‘†๐‘† ๐›ฝ๐›ฝฬ‚
๐›ผ๐›ผ๏ฟฝ = ๏ฟฝ + โˆ’ 2 = 1.763
๐›ฝ๐›ฝฬ‚ ๐‘…๐‘…
Birnbaum Sanders Continuous Distribution
90% confidence interval for ๐›ผ๐›ผ:
2
โŽก
โŽง๐›ท๐›ทโˆ’1 (0.95)๏ฟฝ ๐›ผ๐›ผ โŽซ
2๐‘›๐‘›
๐น๐น
โŽข๐›ผ๐›ผ๏ฟฝ. exp
,
โˆ’๐›ผ๐›ผ๏ฟฝ
โŽข
โŽจ
โŽฌ
โŽฃ
โŽฉ
โŽญ
[1.048,
97
2
โŽง๐›ท๐›ทโˆ’1 (0.95)๏ฟฝ ๐›ผ๐›ผ โŽซโŽค
2๐‘›๐‘›๐น๐น โŽฅ
๐›ผ๐›ผ๏ฟฝ. exp
๐›ผ๐›ผ๏ฟฝ
โŽจ
โŽฌโŽฅ
โŽฉ
โŽญโŽฆ
2.966]
90% confidence interval for ๐›ฝ๐›ฝ:
2
2
๐œ‹๐œ‹
๐‘˜๐‘˜(๐›ผ๐›ผ๏ฟฝ) = ๐›ผ๐›ผ๏ฟฝ๏ฟฝ โˆ’ ๐œ‹๐œ‹ exp ๏ฟฝ 2 ๏ฟฝ ๏ฟฝ1 โˆ’ ฮฆ ๏ฟฝ ๏ฟฝ๏ฟฝ = 1.442
๐›ผ๐›ผ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ
2
๐›ผ๐›ผ๏ฟฝ(2๐œ‹๐œ‹)โˆ’1โ„2 ๐‘˜๐‘˜(๐›ผ๐›ผ๏ฟฝ) + 1
๐ผ๐ผ๐›ฝ๐›ฝ๐›ฝ๐›ฝ =
= 10.335๐ธ๐ธ-6
๐›ผ๐›ผ๏ฟฝ 2 ๐›ฝ๐›ฝฬ‚2
Note that this confidence interval uses the assumption of the
parameters being normally distributed which is only true for large
sample sizes. Therefore these confidence intervals may be
inaccurate. Bayesian methods must be done numerically.
Characteristics
The Birnbaum-Saunders distribution is a stochastic model of the
Minerโ€™s rule.
Characteristic of ๐œถ๐œถ. As ๐›ผ๐›ผ decreases the distribution becomes more
symmetrical around the value of ๐›ฝ๐›ฝ.
Hazard Rate. The hazard rate is always unimodal. The hazard rate
has the following asymptotes: (Meeker & Escobar 1998, p.107)
โ„Ž(0) = 0
1
lim โ„Ž(๐‘ก๐‘ก) =
๐‘ก๐‘กโ†’โˆž
2๐›ฝ๐›ฝ๐›ผ๐›ผ 2
The change point of the unimodal hazard rate for ๐›ผ๐›ผ < 0.6 must be
solved numerically, however for ๐›ผ๐›ผ > 0.6 can be approximated using:
(Kundu et al. 2008)
๐›ฝ๐›ฝ
๐‘ก๐‘ก๐‘๐‘ =
(โˆ’0.4604 + 1.8417๐›ผ๐›ผ)2
Lognormal and Inverse Gaussian Distribution. The shape and
behavior of the Birnbaum-Saunders distribution is similar to that of
the lognormal and inverse Gaussian distribution. This similarity is
seen primarily in the center of the distributions. (Meeker & Escobar
1998, p.107)
Let:
๐‘‡๐‘‡~๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Birnbaum Sanders
96762
96762 โŽค
โŽก
โŽง๐›ท๐›ทโˆ’1 (0.95)๏ฟฝ
โŽซ
โŽง๐›ท๐›ทโˆ’1 (0.95)๏ฟฝ
โŽซ
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
โŽข๐›ฝ๐›ฝฬ‚ . exp
โŽฅ
,
๐›ฝ๐›ฝฬ‚ . exp
โˆ’๐›ฝ๐›ฝฬ‚
โˆ’๐›ฝ๐›ฝฬ‚
โŽข
โŽจ
โŽฌ
โŽจ
โŽฌโŽฅ
โŽฃ
โŽฉ
โŽญ
โŽฉ
โŽญโŽฆ
[100.4, 624.5]
98
Univariate Continuous Distributions
Scaling property (Meeker & Escobar 1998, p.107)
๐‘๐‘๐‘๐‘~๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐‘๐‘๐‘๐‘)
where ๐‘๐‘ > 0
Birnbaum Sanders
Inverse property (Meeker & Escobar 1998, p.107)
1
1
~๐ต๐ต๐ต๐ต ๏ฟฝ๐‘ก๐‘ก; ๐›ผ๐›ผ, ๏ฟฝ
๐‘‡๐‘‡
๐›ฝ๐›ฝ
Applications
Fatigue-Fracture. The distribution has been designed to model
crack growth to critical crack size. The model uses the Minerโ€™s rule
which allows for non-constant fatigue cycles through accumulated
damage. The assumption is that the crack growth during any one
cycle is independent of the growth during any other cycle. The growth
for each cycle has the same distribution from cycle to cycle. This is
different from the proportional degradation model used to derive the
log normal distribution model, with the rate of degradation being
dependent
on
accumulated
damage.
(http://www.itl.nist.gov/div898/handbook/apr/section1/apr166.htm)
Resources
Online:
http://www.itl.nist.gov/div898/handbook/eda/section3/eda366a.htm
http://www.itl.nist.gov/div898/handbook/apr/section1/apr166.htm
http://en.wikipedia.org/wiki/Birnbaum%E2%80%93Saunders_distrib
ution
Books:
Birnbaum, Z.W. & Saunders, S.C., 1969. A New Family of Life
Distributions. Journal of Applied Probability, 6(2), 319-327.
Lemonte, A.J., Cribari-Neto, F. & Vasconcellos, K.L., 2007. Improved
statistical inference for the two-parameter Birnbaum-Saunders
distribution. Computational Statistics & Data Analysis, 51(9), 46564681.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous
Univariate Distributions, Vol. 2, 2nd ed., Wiley-Interscience.
Rausand, M. & Høyland, A., 2004. System reliability theory, WileyIEEE.
Gamma Continuous Distribution
99
4.3. Gamma Continuous Distribution
Probability Density Function - f(t)
0.30
k=0.5 ฮป=1
k=3 ฮป=1
k=6 ฮป=1
k=12 ฮป=1
0.25
0.20
0.15
0.10
0.05
0.00
0
10
0.50
0.45
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.00
20
k=3 ฮป=1.8
k=3 ฮป=1.2
k=3 ฮป=0.8
k=3 ฮป=0.4
0
2
4
6
8
0
2
4
6
8
0
2
4
6
8
Cumulative Density Function - F(t)
1.00
0.80
0.80
0.60
0.60
0.40
Gamma
1.00
0.40
k=0.5 ฮป=1
k=3 ฮป=1
k=6 ฮป=1
k=12 ฮป=1
0.20
0.00
0
10
0.20
0.00
20
Hazard Rate - h(t)
1.60
1.60
1.40
1.40
1.20
1.20
1.00
1.00
0.80
0.80
0.60
0.60
0.40
0.40
0.20
0.20
0.00
0.00
0
10
20
100
Univariate Continuous Distributions
Parameters & Description
๐œ†๐œ†
๐œ†๐œ† > 0
Parameters
๐‘˜๐‘˜
๐‘˜๐‘˜ > 0
Limits
Distribution
When k is an integer
(Erlang distribution)
Scale Parameter: Equal to the rate
(frequency)
of
events/shocks.
Sometimes defined as 1/๐œƒ๐œƒ where ๐œƒ๐œƒ is the
average time between events/shocks.
Shape Parameter: As an integer ๐‘˜๐‘˜ can be
interpreted
as
the
number
of
events/shocks until failure. When not
restricted to an integer, ๐‘˜๐‘˜ and be
interpreted as a measure of the ability to
resist shocks.
๐‘ก๐‘ก โ‰ฅ 0
When k is continuous
Gamma
ฮ“(๐‘˜๐‘˜) is the complete gamma function. ฮ“(๐‘˜๐‘˜, ๐‘ก๐‘ก) and ๐›พ๐›พ(๐‘˜๐‘˜, ๐‘ก๐‘ก) are the incomplete gamma
functions see section 1.6.
๐‘“๐‘“(๐‘ก๐‘ก) =
PDF
๐œ†๐œ†๐‘˜๐‘˜ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1 โˆ’ฮปt
e
(k โˆ’ 1)!
๐‘˜๐‘˜โˆ’1
(๐œ†๐œ†๐œ†๐œ†)๐‘›๐‘›
๐น๐น(๐‘ก๐‘ก) = 1 โˆ’ ๐‘’๐‘’ โˆ’๐œ†๐œ†๐œ†๐œ† ๏ฟฝ
๐‘›๐‘›!
CDF
๐‘›๐‘›=0
๐‘˜๐‘˜โˆ’1
(๐œ†๐œ†๐œ†๐œ†)๐‘›๐‘›
๐‘…๐‘…(๐‘ก๐‘ก) = ๐‘’๐‘’ โˆ’๐œ†๐œ†๐œ†๐œ† ๏ฟฝ
๐‘›๐‘›!
Reliability
๐‘›๐‘›=0
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
Residual
๐‘’๐‘’ โˆ’๐œ†๐œ†๐œ†๐œ†
[๐œ†๐œ†(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)]๐‘›๐‘›
๐‘›๐‘›!
(๐œ†๐œ†๐œ†๐œ†)๐‘›๐‘›
๐‘˜๐‘˜โˆ’1
โˆ‘๐‘›๐‘›=0
๐‘›๐‘›!
โˆ‘๐‘˜๐‘˜โˆ’1
๐‘›๐‘›=0
๐‘“๐‘“(๐‘ก๐‘ก) =
๐œ†๐œ†๐‘˜๐‘˜ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1 โˆ’ฮปt
e
ฮ“(๐‘˜๐‘˜)
with Laplace transformation:
๐œ†๐œ† k
๐‘“๐‘“(๐‘ ๐‘ ) = ๏ฟฝ
๏ฟฝ
๐œ†๐œ† + ๐‘ ๐‘ 
๐น๐น(๐‘ก๐‘ก) =
๐›พ๐›พ(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
๐›ค๐›ค(๐‘˜๐‘˜)
๐‘…๐‘…(๐‘ก๐‘ก) =
๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
๐›ค๐›ค(๐‘˜๐‘˜)
=
๐œ†๐œ†๐œ†๐œ†
1
๏ฟฝ ๐‘ฅ๐‘ฅ ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘
๐›ค๐›ค(๐‘˜๐‘˜) 0
=
โˆž
1
๏ฟฝ ๐‘ฅ๐‘ฅ ๐‘˜๐‘˜โˆ’1 ๐‘’๐‘’ โˆ’๐‘ฅ๐‘ฅ ๐‘‘๐‘‘๐‘‘๐‘‘
๐›ค๐›ค(๐‘˜๐‘˜) ๐œ†๐œ†๐œ†๐œ†
๐‘š๐‘š(๐‘ฅ๐‘ฅ) =
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ) ๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ† + ๐œ†๐œ†๐œ†๐œ†))
=
๐‘…๐‘…(๐‘ก๐‘ก)
๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โˆž
โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โˆž
โˆซ๐‘ก๐‘ก ๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)๐‘‘๐‘‘๐‘‘๐‘‘
๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
The mean residual life does not have a closed form but has the
expansion:
Gamma Continuous Distribution
101
๐‘˜๐‘˜ โˆ’ 1 (๐‘˜๐‘˜ โˆ’ 1)(๐‘˜๐‘˜ โˆ’ 2)
+
+ ๐‘‚๐‘‚(๐‘ก๐‘ก โˆ’3 )
๐‘ก๐‘ก
๐‘ก๐‘ก 2
Where ๐‘‚๐‘‚(๐‘ก๐‘ก โˆ’3 ) is Landau's notation. (Kleiber & Kotz 2003, p.161)
๐‘ข๐‘ข(๐‘ก๐‘ก) = 1 +
โ„Ž(๐‘ก๐‘ก) =
๐œ†๐œ†๐‘˜๐‘˜ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1
(๐œ†๐œ†๐œ†๐œ†)๐‘›๐‘›
๐›ค๐›ค(๐‘˜๐‘˜) โˆ‘๐‘˜๐‘˜โˆ’1
๐‘›๐‘›=0 ๐‘›๐‘›!
โ„Ž(๐‘ก๐‘ก) =
๐œ†๐œ†๐‘˜๐‘˜ ๐‘ก๐‘ก ๐‘˜๐‘˜โˆ’1 โˆ’๐œ†๐œ†๐œ†๐œ†
๐‘’๐‘’
๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
Series expansion of the hazard rate is: (Kleiber & Kotz 2003, p.161)
โˆ’1
(๐‘˜๐‘˜ โˆ’ 1)(๐‘˜๐‘˜ โˆ’ 2)
โ„Ž(๐‘ก๐‘ก) = ๏ฟฝ
+ ๐‘‚๐‘‚(๐‘ก๐‘ก โˆ’3 )๏ฟฝ
2
๐‘ก๐‘ก
Limits of โ„Ž(๐‘ก๐‘ก) (Rausand & Høyland 2004)
Hazard Rate
lim โ„Ž(๐‘ก๐‘ก) = โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž lim โ„Ž(๐‘ก๐‘ก) = ๐œ†๐œ† ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ 0 < ๐‘˜๐‘˜ < 1
๐‘ก๐‘กโ†’0
๐‘ก๐‘กโ†’โˆž
lim โ„Ž(๐‘ก๐‘ก) = 0 ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž lim โ„Ž(๐‘ก๐‘ก) = ๐œ†๐œ† ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ ๐‘˜๐‘˜ โ‰ฅ 1
๐‘ก๐‘กโ†’0
๐‘ก๐‘กโ†’โˆž
๐‘˜๐‘˜โˆ’1
๐ป๐ป(๐‘ก๐‘ก) = ๐œ†๐œ†๐œ†๐œ† โˆ’ ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ๏ฟฝ
๐‘›๐‘›=0
(๐œ†๐œ†๐œ†๐œ†)๐‘›๐‘›
๏ฟฝ
๐‘›๐‘›!
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ ๐‘™๐‘™๐‘™๐‘™ ๏ฟฝ
Properties and Moments
๐›ค๐›ค(๐‘˜๐‘˜, ๐œ†๐œ†๐œ†๐œ†)
๏ฟฝ
๐›ค๐›ค(๐‘˜๐‘˜)
Numerically solve for t when:
๐‘ก๐‘ก0.5 = ๐น๐น โˆ’1 (0.5; ๐‘˜๐‘˜, ๐œ†๐œ†)
or
ฮณ(k, ฮปt) = ฮ“(k, ฮปt)
Median
where ฮณ(k, ฮปt) is the lower incomplete
gamma function, see section 1.6.6.
๐‘˜๐‘˜ โˆ’ 1
๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘˜๐‘˜ โ‰ฅ 1
๐œ†๐œ†
Mode
Mean - 1st Raw Moment
Variance - 2nd Central Moment
Skewness - 3rd Central Moment
Excess kurtosis -
4th
Central Moment
Characteristic Function
100ฮฑ% Percentile Function
๐‘๐‘๐‘๐‘ ๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ 0 < ๐‘˜๐‘˜ < 1
k
๐œ†๐œ†
k
๐œ†๐œ†2
2/โˆš๐‘˜๐‘˜
6/๐‘˜๐‘˜
it โˆ’k
๏ฟฝ1 โˆ’ ๏ฟฝ
ฮป
Numerically solve for t:
๐‘ก๐‘ก๐›ผ๐›ผ = ๐น๐น โˆ’1 (๐›ผ๐›ผ; ๐‘˜๐‘˜, ๐œ†๐œ†)
Gamma
Cumulative
Hazard Rate
102
Univariate Continuous Distributions
Parameter Estimation
Maximum Likelihood Function
๐ฟ๐ฟ(๐‘˜๐‘˜, ๐œ†๐œ†|๐ธ๐ธ) =
Likelihood
Functions
Log-Likelihood
Functions
โˆ‚ฮ›
=0
โˆ‚k
Gamma
โˆ‚ฮ›
=0
โˆ‚ฮป
Point
Estimates
Fisher
Information
Matrix
nF
๐œ†๐œ†๐‘˜๐‘˜๐‘›๐‘›๐น๐น
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– ๐‘˜๐‘˜โˆ’1 eโˆ’ฮปti
ฮ“(๐‘˜๐‘˜)๐‘›๐‘›๐น๐น
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝi=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
nF
nF
i=1
i=1
ฮ›(๐‘˜๐‘˜, ๐œ†๐œ†|๐ธ๐ธ) = ๐‘˜๐‘˜๐‘›๐‘›๐น๐น ln(๐œ†๐œ†) โˆ’ ๐‘›๐‘›๐น๐น ln๏ฟฝ๐›ค๐›ค(๐‘˜๐‘˜)๏ฟฝ + (k โˆ’ 1) ๏ฟฝ ln(t i ) โˆ’ ฮป ๏ฟฝ t i
nF
where ๐œ“๐œ“(๐‘ฅ๐‘ฅ) =
๐‘‘๐‘‘
๐‘‘๐‘‘๐‘‘๐‘‘
0 = ๐‘›๐‘›๐น๐น ln(ฮป) โˆ’ ๐‘›๐‘›๐น๐น ๐œ“๐œ“(๐‘˜๐‘˜) + ๏ฟฝ{ln(t i )}
i=1
ln[ฮ“(๐‘ฅ๐‘ฅ)] is the digamma function see section 1.6.7.
nF
๐‘˜๐‘˜๐‘›๐‘›๐น๐น
0=
โˆ’ ๏ฟฝ ti
ฮป
i=1
Point estimates for ๐‘˜๐‘˜๏ฟฝ and ๐œ†๐œ†ฬ‚ are obtained by using numerical methods to
solve the simultaneous equations above. (Kleiber & Kotz 2003, p.165)
๐ผ๐ผ(๐‘˜๐‘˜, ๐œ†๐œ†) = ๏ฟฝ
๐‘‘๐‘‘ 2
๐œ“๐œ“ โ€ฒ (๐‘˜๐‘˜)
๐œ†๐œ†
๐œ†๐œ†
๏ฟฝ
๐‘˜๐‘˜๐œ†๐œ†2
โˆ’2
where ๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) = 2 ๐‘™๐‘™๐‘™๐‘™ฮ“(๐‘ฅ๐‘ฅ) = โˆ‘โˆž
is the Trigamma function.
๐‘–๐‘–=0(๐‘ฅ๐‘ฅ + ๐‘–๐‘–)
๐‘‘๐‘‘๐‘ฅ๐‘ฅ
(Yang and Berger 1998, p.10)
Confidence
Intervals
For a large number of samples the Fisher information matrix can be
used to estimate confidence intervals.
Bayesian
Type
Uniform Improper
Prior with limits:
๐œ†๐œ† โˆˆ (0, โˆž)
๐‘˜๐‘˜ โˆˆ (0, โˆž)
Jeffreyโ€™s Prior
Reference
Order:
{๐‘˜๐‘˜, ๐œ†๐œ†}
Reference
Order:
{๐œ†๐œ†, ๐‘˜๐‘˜}
Prior
Non-informative Priors, ๐…๐…(๐’Œ๐’Œ, ๐€๐€)
(Yang and Berger 1998, p.6)
Posterior
1
No Closed Form
๐œ†๐œ†๏ฟฝ๐‘˜๐‘˜. ๐œ“๐œ“ โ€ฒ (๐‘˜๐‘˜) โˆ’ 1
No Closed Form
๐œ†๐œ†๏ฟฝ๐œ“๐œ“ โ€ฒ (๐‘˜๐‘˜)
No Closed Form
๐œ†๐œ†๏ฟฝ๐‘˜๐‘˜. ๐œ“๐œ“ โ€ฒ (๐‘˜๐‘˜) โˆ’
1
๐›ผ๐›ผ
No Closed Form
Gamma Continuous Distribution
where ๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) =
UOI
ฮ›
from
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ฮ›)
ฮ›
from
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; ฮ›๐‘ก๐‘ก)
๐œ†๐œ†
where
๐œ†๐œ† = ๐›ผ๐›ผ โˆ’๐›ฝ๐›ฝ
from
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐‘‘๐‘‘ 2
๐‘‘๐‘‘๐‘ฅ๐‘ฅ 2
Likelihood
Model
Exponential
Poisson
Weibull with
known ๐›ฝ๐›ฝ
Gamma with
๐œ†๐œ†
known ๐‘˜๐‘˜ =
from
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ฅ๐‘ฅ; ๐œ†๐œ†, ๐‘˜๐‘˜)
๐‘˜๐‘˜๐ธ๐ธ
๐›ผ๐›ผ
from
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก; ๐œƒ๐œƒ, ๐›ผ๐›ผ)
Pareto with
known ๐œƒ๐œƒ
where:
โˆ’2 is the Trigamma function
๐‘™๐‘™๐‘™๐‘™ฮ“(๐‘ฅ๐‘ฅ) = โˆ‘โˆž
๐‘–๐‘–=0(๐‘ฅ๐‘ฅ + ๐‘–๐‘–)
Conjugate Priors
Evidence
๐‘›๐‘›๐น๐น
failures in
๐‘ก๐‘ก ๐‘‡๐‘‡
๐‘›๐‘›๐น๐น
failures in
๐‘ก๐‘ก ๐‘‡๐‘‡
๐‘›๐‘›๐น๐น
failures
at times
๐‘ก๐‘ก๐‘–๐‘–
๐‘›๐‘›๐น๐น
failures
at times
๐‘ก๐‘ก๐‘–๐‘–
๐‘›๐‘›๐น๐น
failures in
๐‘ก๐‘ก ๐‘‡๐‘‡
๐‘›๐‘›๐น๐น
failures
at times
๐‘ก๐‘ก๐‘–๐‘–
๐‘ก๐‘ก ๐‘‡๐‘‡ =
โˆ‘ t Fi
+
Dist. of
UOI
Prior
Para
Posterior
Parameters
Gamma
๐‘˜๐‘˜0 , ๐œ†๐œ†0
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๐‘ก๐‘ก ๐‘‡๐‘‡
Gamma
๐‘˜๐‘˜0 , ๐œ†๐œ†0
Gamma
Gamma
Gamma
Gamma
โˆ‘ t Si
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๐‘ก๐‘ก ๐‘‡๐‘‡
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
๐‘˜๐‘˜0 , ๐œ†๐œ†0
๐‘˜๐‘˜0 , ๐œ†๐œ†0
๐›ฝ๐›ฝ
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–=1
(Rinne 2008, p.520)
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น /2
๐‘›๐‘›
1
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
2
๐œ‚๐œ‚0 , ฮ›0
k 0 , ฮป0
= ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก ๐‘–๐‘–๐‘–๐‘– ๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก
๐‘–๐‘–=1
๐œ‚๐œ‚ = ๐œ‚๐œ‚0 + ๐‘›๐‘›๐น๐น ๐‘˜๐‘˜๐ธ๐ธ
ฮ› = ฮ›๐‘œ๐‘œ + ๐‘ก๐‘ก ๐‘‡๐‘‡
k = k ๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
๐‘ฅ๐‘ฅ๐‘–๐‘–
ฮป = ฮป๐‘œ๐‘œ + ๏ฟฝ ln ๏ฟฝ ๏ฟฝ
๐œƒ๐œƒ
๐‘–๐‘–=1
Description , Limitations and Uses
Example 1
For an example using the gamma distribution as a conjugate prior
see the Poisson or Exponential distributions.
A renewal process has an exponential time between failure with
parameter ๐œ†๐œ† = 0.01 under the homogeneous Poisson process
conditions. What is the probability the forth failure will occur before
200 hours.
Example 2
๐น๐น(200; 4,0.01) = 0.1429
5 components are put on a test with the following failure times:
38, 42, 44, 46, 55 hours
Solving:
5๐‘˜๐‘˜
0=
โˆ’ 225
ฮป
0 = 5ln(ฮป) โˆ’ 5๐œ“๐œ“(๐‘˜๐‘˜) + 18.9954
Gamma
๐œŽ๐œŽ 2
Normal with
from
known ๐œ‡๐œ‡
2
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ฅ๐‘ฅ; ๐œ‡๐œ‡, ๐œŽ๐œŽ )
103
104
Univariate Continuous Distributions
Gives:
k๏ฟฝ = 21.377
๐œ†๐œ†ฬ‚ = 0.4749
90% confidence interval for ๐‘˜๐‘˜:
๏ฟฝ๐ฝ๐ฝ๐‘›๐‘› ๏ฟฝ๐‘˜๐‘˜๏ฟฝ, ๐œ†๐œ†ฬ‚๏ฟฝ๏ฟฝ
โˆ’1
0.0479 0.4749
๐ผ๐ผ(๐‘˜๐‘˜, ๐œ†๐œ†) = ๏ฟฝ
๏ฟฝ
0.4749 4.8205
= ๏ฟฝ๐‘›๐‘›๐น๐น ๐ผ๐ผ๏ฟฝ๐‘˜๐‘˜๏ฟฝ, ๐œ†๐œ†ฬ‚๏ฟฝ๏ฟฝ
โˆ’1
๐›ท๐›ทโˆ’1 (0.95)โˆš179.979
๏ฟฝ,
๏ฟฝ๐‘˜๐‘˜๏ฟฝ. exp ๏ฟฝ
โˆ’๐‘˜๐‘˜๏ฟฝ
[7.6142,
๐›ท๐›ท โˆ’1 (0.95)โˆš179.979
๐‘˜๐‘˜๏ฟฝ . exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐‘˜๐‘˜๏ฟฝ
60.0143]
90% confidence interval for ๐œ†๐œ†:
Gamma
๏ฟฝ๐œ†๐œ†ฬ‚. exp ๏ฟฝ
179.979 โˆ’17.730
=๏ฟฝ
๏ฟฝ
โˆ’17.730 1.7881
๐›ท๐›ทโˆ’1 (0.95)โˆš1.7881
๏ฟฝ,
โˆ’๐œ†๐œ†ฬ‚
[0.0046,
๐›ท๐›ทโˆ’1 (0.95)โˆš1.7881
๐œ†๐œ†ฬ‚. exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐œ†๐œ†ฬ‚
48.766]
Note that this confidence interval uses the assumption of the
parameters being normally distributed which is only true for large
sample sizes. Therefore these confidence intervals may be
inaccurate. Bayesian methods must be done numerically.
Characteristics
The gamma distribution was originally known as a Pearson Type III
distribution. This distribution includes a location parameter ๐›พ๐›พ which
shifts the distribution along the x-axis.
๐‘“๐‘“(๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†, ๐›พ๐›พ) =
๐œ†๐œ†๐‘˜๐‘˜ (๐‘ก๐‘ก โˆ’ ๐›พ๐›พ)๐‘˜๐‘˜โˆ’1 โˆ’ฮป(tโˆ’ฮณ)
e
ฮ“(๐‘˜๐‘˜)
When k is an integer, the Gamma distribution is called an Erlang
distribution.
๐’Œ๐’Œ Characteristics:
๐’Œ๐’Œ < 1. ๐‘“๐‘“(0) = โˆž. There is no mode.
๐’Œ๐’Œ = ๐Ÿ๐Ÿ. ๐‘“๐‘“(0) = ๐œ†๐œ†. The gamma distribution reduces to an
exponential distribution with failure rate ฮป. Mode at ๐‘ก๐‘ก = 0.
๐’Œ๐’Œ > 1. ๐‘“๐‘“(0) = 0
Large ๐’Œ๐’Œ. The gamma distribution approaches a normal
๐‘˜๐‘˜
๐‘˜๐‘˜
distribution with ๐œ‡๐œ‡ = , ๐œŽ๐œŽ = ๏ฟฝ 2 .
๐œ†๐œ†
๐œ†๐œ†
Homogeneous Poisson Process (HPP). Components with an
exponential time to failure which undergo instantaneous renewal with
an identical item undergo a HPP. The Gamma distribution is
Gamma Continuous Distribution
105
probability distribution of the kth failed item and is derived from the
convolution of ๐‘˜๐‘˜ exponentially distributed random variables, ๐‘‡๐‘‡๐‘–๐‘– . (See
related distributions, exponential distribution).
Scaling property:
๐‘‡๐‘‡~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
๐œ†๐œ†
๐‘Ž๐‘Ž๐‘Ž๐‘Ž~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘˜๐‘˜, ๏ฟฝ
๐‘Ž๐‘Ž
Convolution property:
๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ€ฆ + ๐‘‡๐‘‡๐‘›๐‘› ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(โˆ‘๐‘˜๐‘˜๐‘–๐‘– , ๐œ†๐œ†)
Where ๐œ†๐œ† is fixed.
Properties from (Leemis & McQueston 2008)
Renewal Theory, Homogenous Poisson Process. Used to model
a renewal process where the component time to failure is
exponentially distributed and the component is replaced
instantaneously with a new identical component. The HPP can also
be used to model ruin theory (used in risk assessments) and queuing
theory.
System Failure. Can be used to model system failure with ๐‘˜๐‘˜ backup
systems.
Life Distribution. The gamma distribution is flexible in shape and
can give good approximations to life data.
Bayesian Analysis. The gamma distribution is often used as a prior
in Bayesian analysis to produce closed form posteriors.
Online:
http://mathworld.wolfram.com/GammaDistribution.html
http://en.wikipedia.org/wiki/Gamma_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web
calculator)
http://www.itl.nist.gov/div898/handbook/eda/section3/eda366b.htm
Resources
Books:
Artin, E., 1964. The Gamma Function, New York: Holt, Rinehart &
Winston.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous
Univariate Distributions, Vol. 1 2nd ed., Wiley-Interscience.
Bowman, K.O. & Shenton, L.R., 1988. Properties of estimators for
the gamma distribution, CRC Press.
Relationship to Other Distributions
Generalized
Gamma
Distribution
Gamma
Applications
106
Univariate Continuous Distributions
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†, ฮณ, ๐œ‰๐œ‰)
๐‘“๐‘“(๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†, ๐›พ๐›พ, ๐œ‰๐œ‰) =
๐œ‰๐œ‰๐œ†๐œ†๐œ‰๐œ‰๐œ‰๐œ‰ (๐‘ก๐‘ก โˆ’ ๐›พ๐›พ)๐œ‰๐œ‰๐œ‰๐œ‰โˆ’1
exp{โˆ’[๐œ†๐œ†(๐‘ก๐‘ก โˆ’ ๐›พ๐›พ)]๐‘˜๐‘˜ }
ฮ“(๐‘˜๐‘˜)
๐œ†๐œ† - Scale Parameter
๐‘˜๐‘˜ - Shape Parameter
๐›พ๐›พ - Location parameter
๐œ‰๐œ‰ - Second shape parameter
Gamma
The generalized gamma distribution has been derived because it is
a generalization of a large amount of probability distributions. Such
as:
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; 1, ๐œ†๐œ†, 0,1) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; 1, , ฮฒ, 1๏ฟฝ = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐›ฝ๐›ฝ)
๐œ‡๐œ‡
1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; 1, , 0, ๐›ฝ๐›ฝ๏ฟฝ = ๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
๐›ผ๐›ผ
1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; 1, , ฮณ, ๐›ฝ๐›ฝ๏ฟฝ = ๐‘Š๐‘Š๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ, ๐›พ๐›พ)
๐›ผ๐›ผ
๐‘›๐‘› 1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; , , 0,1๏ฟฝ = ฯ‡2 (t; n)
2 2
๐‘›๐‘› 1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; ,
, 0,2๏ฟฝ = ฯ‡(t; n)
2 โˆš2
1
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ ๏ฟฝ๐‘ก๐‘ก; 1, , 0,2๏ฟฝ = Rayleigh(t; ฯƒ)
๐œŽ๐œŽ
Let
Then
Exponential
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ฮป)
Special Case:
Then
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; ๐œ†๐œ†๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‡๐‘‡๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐‘˜๐‘˜
๐‘‡๐‘‡๐‘ก๐‘ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
This is gives the Gamma distribution its convolution property.
Let
Poisson
Distribution
๐‘‡๐‘‡1 โ€ฆ ๐‘‡๐‘‡๐‘˜๐‘˜ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; ๐‘˜๐‘˜ = 1, ๐œ†๐œ†)
๐‘‡๐‘‡1 โ€ฆ ๐‘‡๐‘‡๐‘˜๐‘˜ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‡๐‘‡๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐‘˜๐‘˜
๐‘‡๐‘‡๐‘ก๐‘ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
The Poisson distribution is the probability that exactly ๐‘˜๐‘˜ failures have
been observed in time ๐‘ก๐‘ก. This is the probability that ๐‘ก๐‘ก is between ๐‘‡๐‘‡๐‘˜๐‘˜
and ๐‘‡๐‘‡๐‘˜๐‘˜+1 .
๐‘“๐‘“๐‘ƒ๐‘ƒ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ (๐‘˜๐‘˜; ๐œ†๐œ†๐œ†๐œ†) = ๏ฟฝ
๐‘˜๐‘˜+1
๐‘˜๐‘˜
where ๐‘˜๐‘˜ is an integer.
๐‘“๐‘“๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘ฅ๐‘ฅ, ๐œ†๐œ†)๐‘‘๐‘‘๐‘‘๐‘‘
= ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜ + 1, ๐œ†๐œ†) โˆ’ ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†)
Gamma Continuous Distribution
Normal
Distribution
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ)
Chi-square
Distribution
๐œ’๐œ’ 2 (๐‘ก๐‘ก; ๐‘ฃ๐‘ฃ)
Inverse Gamma
Distribution
๐ผ๐ผ๐ผ๐ผ(๐‘ก๐‘ก; ๐›ผ๐›ผ, ฮฒ)
Special Case for large k:
lim ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œ‡๐œ‡ =
๐‘˜๐‘˜โ†’โˆž
Special Case:
๐œ’๐œ’ 2 (๐‘ก๐‘ก; ๐‘ฃ๐‘ฃ) = ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ก๐‘ก; ๐‘˜๐‘˜ =
where ๐‘ฃ๐‘ฃ is an integer
Let
Then
Let
Beta Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ฮฑ, ฮฒ)
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ (๐’™๐’™; ๐›‚๐›‚)
Then
Let:
Then:
Let:
Then:
๐‘˜๐‘˜
๐‘˜๐‘˜
, ๐œŽ๐œŽ = ๏ฟฝ 2 ๏ฟฝ
๐œ†๐œ†
๐œ†๐œ†
๐‘ฃ๐‘ฃ
1
, ๐œ†๐œ† = )
2
2
X~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y=
1
X
๐‘‹๐‘‹1 , ๐‘‹๐‘‹2 ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(k i , ฮปi )
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y=
X1
X1 + X 2
Y~I๐บ๐บ(๐›ผ๐›ผ = ๐‘˜๐‘˜, ฮฒ = ฮป)
Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ = ๐‘˜๐‘˜1 , ๐›ฝ๐›ฝ = ๐‘˜๐‘˜2 )
๐‘Œ๐‘Œ๐‘–๐‘– ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†, ๐‘˜๐‘˜๐‘–๐‘– ) ๐‘–๐‘–. ๐‘–๐‘–. ๐‘‘๐‘‘ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‰๐‘‰~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†, โˆ‘๐‘˜๐‘˜๐‘–๐‘– )
๐‘‘๐‘‘
๐‘‰๐‘‰ = ๏ฟฝ ๐‘Œ๐‘Œ๐‘–๐‘–
๐‘–๐‘–=1
๐‘Œ๐‘Œ1 ๐‘Œ๐‘Œ2
๐‘Œ๐‘Œ๐‘‘๐‘‘
๐’๐’ = ๏ฟฝ , , โ€ฆ , ๏ฟฝ
๐‘‰๐‘‰ ๐‘‰๐‘‰
๐‘‰๐‘‰
๐’๐’~๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ (๐›ผ๐›ผ1 , โ€ฆ , ๐›ผ๐›ผ๐‘˜๐‘˜ )
*i.i.d: independent and identically distributed
Wishart
Distribution
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Šโ„Ž๐‘Ž๐‘Ž๐‘Ÿ๐‘Ÿ๐‘ก๐‘ก๐‘‘๐‘‘ (๐‘›๐‘›; ๐šบ๐šบ)
The Wishart Distribution is the multivariate generalization of the
gamma distribution.
Gamma
Dirichlet
Distribution
107
108
Univariate Continuous Distributions
4.4. Logistic Continuous
Distribution
Probability Density Function - f(t)
0.3
ฮผ=0
ฮผ=2
ฮผ=4
0.2
0.2
s=0.7
s=1
s=2
0.3
0.2
0.1
0.1
0.1
0.0
0.0
-6
-1
4
9
-7
-2
3
Logistic
Cumulative Density Function - F(t)
1.0
1.0
0.8
0.8
0.6
0.6
0.4
ฮผ=0 s=1
ฮผ=2 s=1
ฮผ=4 s=1
0.2
0.0
0.4
s=0.7
s=1
s=2
0.2
0.0
-6
-1
4
9
-7
-2
3
Hazard Rate - h(t)
1.0
ฮผ=0
ฮผ=2
ฮผ=4
0.8
0.6
ฮผ=0 s=0.7
ฮผ=0 s=1
ฮผ=0 s=2
1.4
1.2
1.0
0.8
0.6
0.4
0.4
0.2
0.2
0.0
0.0
-6
-1
4
9
-7
-2
3
Logistic Continuous Distribution
109
Parameters & Description
Parameters
µ
๐‘ ๐‘ 
Limits
Location parameter. ๐œ‡๐œ‡ is the mean,
median and mode of the distribution.
โˆ’โˆž < ๐œ‡๐œ‡ < โˆž
Scale parameter. Proportional to the
standard deviation of the distribution.
s>0
โˆ’โˆž < t < โˆž
Distribution
Formulas
๐‘“๐‘“(๐‘ก๐‘ก) =
PDF
where
=
ez
eโˆ’z
=
z
2
)
s(1 + e
s(1 + eโˆ’z )2
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
1
sech2 ๏ฟฝ
๏ฟฝ
2๐‘ ๐‘ 
4๐‘ ๐‘ 
CDF
=
1
ez
=
1 + eโˆ’z 1 + ez
1 1
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
+ tanh ๏ฟฝ
๏ฟฝ
2 2
2๐‘ ๐‘ 
R(t) =
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
๐‘ ๐‘ 
Residual
Hazard Rate
Cumulative
Hazard Rate
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
Logistic
๐น๐น(๐‘ก๐‘ก) =
๐‘ง๐‘ง =
1
1 + ez
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
1 + exp ๏ฟฝ
๏ฟฝ
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
๐‘ ๐‘ 
=
๐‘ก๐‘ก
+
๐‘ฅ๐‘ฅ
โˆ’
๐œ‡๐œ‡
๐‘…๐‘…(๐‘ก๐‘ก)
1 + exp ๏ฟฝ
๏ฟฝ
๐‘ ๐‘ 
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
t
๐‘ข๐‘ข(๐‘ก๐‘ก) = (1 + ๐‘’๐‘’ z ) ๏ฟฝs. ln ๏ฟฝ๐‘’๐‘’ ๏ฟฝs + ๐‘’๐‘’
๐œ‡๐œ‡๏ฟฝ
๐‘ ๐‘  ๏ฟฝ
1
๐น๐น(๐‘ก๐‘ก)
=
s(1 + eโˆ’z )
๐‘ ๐‘ 
1
=
๐œ‡๐œ‡ โˆ’ ๐‘ก๐‘ก
s + s exp ๏ฟฝ
๏ฟฝ
๐‘ ๐‘ 
โ„Ž(๐‘ก๐‘ก) =
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
๐ป๐ป(๐‘ก๐‘ก) = ln ๏ฟฝ1 + exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐‘ ๐‘ 
โˆ’ t๏ฟฝ
110
Univariate Continuous Distributions
Properties and Moments
µ
Median
µ
Mode
µ
Mean - 1st Raw Moment
ฯ€2 2
s
3
Variance - 2nd Central Moment
0
Skewness - 3rd Central Moment
Excess kurtosis -
4th
6
5
Central Moment
๐‘’๐‘’ ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘– ๐ต๐ต(1 โˆ’ ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–, 1 + ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–) for |๐‘ ๐‘ ๐‘ ๐‘ | < 1
Characteristic Function
100๐›พ๐›พ % Percentile Function
Parameter Estimation
๐›พ๐›พ
๐‘ก๐‘ก๐›พ๐›พ = ๐œ‡๐œ‡ + ๐‘ ๐‘  ln ๏ฟฝ
๏ฟฝ
1 โˆ’ ๐›พ๐›พ
Logistic
Plotting Method
Least Mean
Square
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
X-Axis
Y-Axis
Likelihood
Function
For complete data:
ti
ln[F] โˆ’ ln[1 โˆ’ ๐น๐น]
Maximum Likelihood Function
โˆ‚ฮ›
=0
โˆ‚µ
โˆ‚ฮ›
=0
โˆ‚s
1
m
๐œ‡๐œ‡ฬ‚ = โˆ’๐‘๐‘๐‘ ๐‘ ฬ‚
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
๏ฟฝ
โˆ’๐‘ ๐‘ 
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 2
i=1 s ๏ฟฝ1 + exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
โˆ’๐‘ ๐‘ 
nF
๐ฟ๐ฟ(๐œ‡๐œ‡, ๐‘ ๐‘ |๐ธ๐ธ) = ๏ฟฝ
Log-Likelihood
Function
๐‘ ๐‘ ฬ‚ =
๐‘›๐‘›๐น๐น
exp ๏ฟฝ
failures
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
ฮ›(๐œ‡๐œ‡, ๐‘ ๐‘ |๐ธ๐ธ) = โˆ’nF ln s + ๏ฟฝ ๏ฟฝ
๏ฟฝ โˆ’ 2 ๏ฟฝ ln ๏ฟฝ1 + exp ๏ฟฝ
๏ฟฝ๏ฟฝ
โˆ’๐‘ ๐‘ 
โˆ’๐‘ ๐‘ 
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
failures
โˆ‚ฮ› nF 2
1
=
โˆ’ ๏ฟฝ
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ = 0
โˆ‚µ
s
๐‘ ๐‘ 
๐‘–๐‘–=1 ๏ฟฝ1 + exp ๏ฟฝ ๐‘ ๐‘  ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
โˆ‚ฮ›
nF 1
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 1 โˆ’ exp ๏ฟฝ ๐‘ ๐‘  ๏ฟฝ
= โˆ’ โˆ’ ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡ ๏ฟฝ = 0
โˆ‚s
s
๐‘ ๐‘ 
๐‘ ๐‘ 
1 + exp ๏ฟฝ ๐‘–๐‘–
๏ฟฝ
๐‘–๐‘–=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘ ๐‘ 
failures
Logistic Continuous Distribution
MLE Point
Estimates
111
The MLE estimates for ๐œ‡๐œ‡ฬ‚ and ๐‘ ๐‘ ฬ‚ are found by solving the following
equations:
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ โˆ’1
1 1
โˆ’ ๏ฟฝ ๏ฟฝ1 + exp ๏ฟฝ
๏ฟฝ๏ฟฝ = 0
๐‘ ๐‘ 
2 nF
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 1 โˆ’ exp ๏ฟฝ ๐‘ ๐‘  ๏ฟฝ
1
๏ฟฝ
1 + ๏ฟฝ๏ฟฝ
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡ = 0
๐‘ ๐‘ 
nF
1 + exp ๏ฟฝ ๐‘–๐‘–
๏ฟฝ
๐‘–๐‘–=1
๐‘ ๐‘ 
These estimates are biased. (Balakrishnan 1991) provides tables
derived from Monte Carlo simulation to correct the bias.
Fisher
Information
(Antle et al. 1970)
100๐›พ๐›พ%
Confidence
Intervals
1
3๐‘ ๐‘  2
๐ผ๐ผ(๐œ‡๐œ‡, ๐‘ ๐‘ ) = ๏ฟฝ
0
๐œ‹๐œ‹ 2
0
+3
9๐‘ ๐‘  2
๏ฟฝ
Confidence intervals are most often obtained from tables derived from
Monte Carlo simulation. Corrections from using the Fisher Information
matrix method are given in (Antle et al. 1970).
Type
Non-informative Priors ๐…๐…๐ŸŽ๐ŸŽ (๐๐, ๐’”๐’”)
Prior
1
๐‘ ๐‘ 
Jeffery Prior
Description , Limitations and Uses
Example
The accuracy of a cutting machine used in manufacturing is desired
to be measured. 5 cuts at the required length are made and
measured as:
7.436, 10.270, 10.466, 11.039, 11.854 ๐‘š๐‘š๐‘š๐‘š
Numerically solving MLE equations gives:
๐œ‡๐œ‡ฬ‚ = 10.446
๐‘ ๐‘ ฬ‚ = 0.815
This gives a mean of 10.446 and a variance of 2.183. Compared to
the same data used in the Normal distribution section it can be seen
that this estimate is very similar to a normal distribution.
90% confidence interval for ๐œ‡๐œ‡:
3๐‘ ๐‘ ฬ‚ 2
๏ฟฝ๐œ‡๐œ‡ฬ‚ โˆ’ ฮฆโˆ’1 (0.95)๏ฟฝ
,
๐‘›๐‘›๐น๐น
3๐‘ ๐‘ ฬ‚ 2
๐œ‡๐œ‡ฬ‚ + ฮฆโˆ’1 (0.95)๏ฟฝ
๏ฟฝ
๐‘›๐‘›๐น๐น
[9.408, 11.4844]
Logistic
Bayesian
112
Univariate Continuous Distributions
90% confidence interval for ๐‘ ๐‘ :
โŽก
โŽง โˆ’1
โŽซ
9๐‘ ๐‘ ฬ‚ 2
(0.95)๏ฟฝ
โŽข
โŽช๐›ท๐›ท
๐‘›๐‘›๐น๐น (3 + ๐œ‹๐œ‹ 2 )โŽช
โŽข๐‘ ๐‘ ฬ‚ . exp
,
โˆ’๐‘ ๐‘ ฬ‚
โŽจ
โŽฌ
โŽข
โŽข
โŽช
โŽช
โŽฃ
โŽฉ
โŽญ
โŽง โˆ’1
โŽซโŽค
9๐‘ ๐‘ ฬ‚ 2
(0.95)๏ฟฝ
โŽช๐›ท๐›ท
๐‘›๐‘›๐น๐น (3 + ๐œ‹๐œ‹ 2 )โŽชโŽฅ
โŽฅ
๐‘ ๐‘ ฬ‚ . exp
๐‘ ๐‘ ฬ‚
โŽจ
โŽฌโŽฅ
โŽช
โŽชโŽฅ
โŽฉ
โŽญโŽฆ
[0.441, 1.501]
Note that this confidence interval uses the assumption of the
parameters being normally distributed which is only true for large
sample sizes. Therefore these confidence intervals may be
inaccurate.
Bayesian methods must be calculated using numerical methods.
Logistic
Characteristics
The logistic distribution is most often used to model growth rates (and
has been used extensively in biology and chemical applications). In
reliability engineering it is most often used as a life distribution.
Shape. There is no shape parameter and so the logistic distribution
is always a bell shaped curve. Increasing ๐œ‡๐œ‡ shifts the curve to the
right, increasing ๐‘ ๐‘  increases the spread of the curve.
Normal Distribution. The shape of the logistic distribution is very
similar to that of a normal distribution with the logistic distribution
having slightly โ€˜longer tailsโ€™. It would take a large number of samples
to distinguish between the distributions. The main difference is that
the hazard rate approaches 1/๐‘ ๐‘  for large ๐‘ก๐‘ก. The logistic function has
historically been preferred over the normal distribution because of its
simplified form. (Meeker & Escobar 1998, p.89)
Alternative Parameterization. It is equally as popular to present the
logistic distribution using the true standard deviation ๐œŽ๐œŽ = ๐œ‹๐œ‹๐œ‹๐œ‹โ„โˆš3. This
form is used in reference book, Balakrishnan 1991, and gives the
following cdf:
๐น๐น(๐‘ก๐‘ก) =
1
โˆ’๐œ‹๐œ‹ ๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
1 + exp ๏ฟฝ
๏ฟฝ
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ
โˆš3
Standard Logistic Distribution. The standard logistic distribution
has ๐œ‡๐œ‡ = 0, ๐‘ ๐‘  = 1. The standard logistic distribution random variable,
๐‘๐‘, is related to the logistic distribution:
๐‘๐‘ =
๐‘‹๐‘‹ โˆ’ ๐œ‡๐œ‡
๐‘ ๐‘ 
Logistic Continuous Distribution
113
Let:
๐‘‡๐‘‡~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐‘ ๐‘ )
Scaling property (Leemis & McQueston 2008)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐‘Ž๐‘Ž๐‘Ž๐‘Ž)
Rate Relationships. The distribution has the following rate
relationships which make it suitable for modeling growth (Hastings et
al. 2000, p.127):
โ„Ž(๐‘ก๐‘ก) =
where
๐‘ง๐‘ง = ln ๏ฟฝ
๐น๐น(๐‘ก๐‘ก)
๏ฟฝ = ln[๐น๐น(๐‘ก๐‘ก)] โˆ’ ln[1 โˆ’ ๐น๐น(๐‘ก๐‘ก)]
๐‘…๐‘…(๐‘ก๐‘ก)
when ๐œ‡๐œ‡ = 0 and ๐‘ ๐‘  = 1:
๐‘ง๐‘ง =
๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡
๐‘ ๐‘ 
๐‘‘๐‘‘๐‘‘๐‘‘(๐‘ก๐‘ก)
= ๐น๐น(๐‘ก๐‘ก)๐‘…๐‘…(๐‘ก๐‘ก)
๐‘‘๐‘‘๐‘‘๐‘‘
Growth Model. The logistic distribution most common use is a
growth model.
Probability of Detection. The cdf of logistic distribution is commonly
used to represent the probability of detection damaged materials
sensors and detection instruments. For example probability of
detection of embedded flaws in metals using ultrasonic signals.
Life Distribution. In reliability applications it is used as a life
distribution. It is similar in shape to a normal distribution and so is
often used instead of a normal distribution due to its simplified form.
(Meeker & Escobar 1998, p.89)
Logistic Regression. Logistic regression is a generalized linear
regression model used predict binary outcomes. (Agresti 2002)
Resources
Online:
http://mathworld.wolfram.com/LogisticDistribution.html
http://en.wikipedia.org/wiki/Logistic_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
http://www.weibull.com/LifeDataWeb/the_logistic_distribution.htm
Books:
Balakrishnan, 1991. Handbook of the Logistic Distribution 1st ed.,
CRC.
Logistic
๐‘“๐‘“(๐‘ก๐‘ก) =
Applications
f(t) ๐น๐น(๐‘ก๐‘ก)
=
R(t)
๐‘ ๐‘ 
114
Univariate Continuous Distributions
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous
Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience.
Relationship to Other Distributions
Let
Exponential
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
Then
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ† = 1)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1)
(Hastings et al. 2000, p.127)
Let
Pareto
Distribution
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘’๐‘’ โˆ’๐‘‹๐‘‹
๐‘Œ๐‘Œ = ln ๏ฟฝ
๏ฟฝ
1 + ๐‘’๐‘’ โˆ’๐‘‹๐‘‹
Then
๐‘‹๐‘‹~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
X ฮฑ
๐‘Œ๐‘Œ = โˆ’ ln ๏ฟฝ๏ฟฝ ๏ฟฝ โˆ’ 1๏ฟฝ
ฮธ
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1)
(Hastings et al. 2000, p.127)
Logistic
Let
Gumbel
Distribution
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐›ผ๐›ผ , ๐›ฝ๐›ฝ)
Then
๐‘‹๐‘‹๐‘–๐‘– ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐›ผ๐›ผ , ๐›ฝ๐›ฝ)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Œ๐‘Œ~๐ฟ๐ฟ๐ฟ๐ฟ๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”๐‘”(0, ๐›ฝ๐›ฝ)
(Hastings et al. 2000, p.127)
๐‘Œ๐‘Œ = X1 โˆ’ X2
Normal Continuous Distribution
115
4.5. Normal (Gaussian) Continuous
Distribution
Probability Density Function - f(t)
ฮผ=-3 ฯƒ=1
ฮผ=0 ฯƒ=1
ฮผ=3 ฯƒ=1
0.5
0.4
ฮผ=0 ฯƒ=0.6
ฮผ=0 ฯƒ=1
ฮผ=0 ฯƒ=2
0.6
0.3
0.4
0.2
0.2
0.1
0.0
-6
-4
0.0
0
-2
2
4
6
-6
-4
-2
0
2
4
6
0
2
4
6
Cumulative Density Function - F(t)
-4
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
-2
Hazard Rate - h(t)
ฮผ=-3 ฯƒ=1
6
ฮผ=0 ฯƒ=1
5
ฮผ=3 ฯƒ=1
-6
0
-4
-2
2
4
6
-6
-4
-2
4
6
ฮผ=-3 ฯƒ=0.6
ฮผ=0 ฯƒ=1 5
ฮผ=0 ฯƒ=2
4
3
3
2
2
1
1
0
0
0
2
4
6
-6
-4
Normal
-6
1.0
-2
0
2
4
6
116
Univariate Continuous Distributions
Parameters & Description
Parameters
๐œ‡๐œ‡
๐œŽ๐œŽ 2
Limits
Location parameter: The mean of the
distribution.
โˆ’โˆž < µ < โˆž
ฯƒ2 > 0
Scale parameter: The standard
deviation of the distribution.
โˆ’โˆž < t < โˆž
Distribution
Formulas
1 tโˆ’µ 2
exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2 ฯƒ
ฯƒโˆš2๐œ‹๐œ‹
1 ๐‘ก๐‘ก โˆ’ µ
๏ฟฝ
= ๐œ™๐œ™ ๏ฟฝ
ฯƒ
ฯƒ
๐‘“๐‘“(๐‘ก๐‘ก) =
PDF
1
where ๐œ™๐œ™ is the standard normal pdf with ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ 2 = 1.
Normal
๐น๐น(๐‘ก๐‘ก) =
๐‘ก๐‘ก
1 ฮธโˆ’µ 2
๏ฟฝ exp ๏ฟฝ โˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘
2
ฯƒ
ฯƒโˆš2๐œ‹๐œ‹ โˆ’โˆž
1
=
CDF
1 1
๐‘ก๐‘ก โˆ’ µ
+ erf ๏ฟฝ
๏ฟฝ
2 2
ฯƒโˆš2
tโˆ’µ
๏ฟฝ
= ฮฆ๏ฟฝ
ฯƒ
where ฮฆ is the standard normal cdf with ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ 2 = 1.
tโˆ’µ
R(t) = 1 โˆ’ ฮฆ ๏ฟฝ
๏ฟฝ
ฯƒ
µโˆ’t
= ฮฆ๏ฟฝ
๏ฟฝ
ฯƒ
Reliability
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
Residual
Hazard Rate
Cumulative
Hazard Rate
µโˆ’xโˆ’t
๏ฟฝ
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ) ฮฆ ๏ฟฝ
ฯƒ
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
=
µโˆ’t
๐‘…๐‘…(๐‘ก๐‘ก)
ฮฆ๏ฟฝ
๏ฟฝ
ฯƒ
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โˆž
โˆž
โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘ โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
=
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
tโˆ’µ
๐œ™๐œ™ ๏ฟฝ
๏ฟฝ
ฯƒ
โ„Ž(๐‘ก๐‘ก) =
µโˆ’t
๐œŽ๐œŽ ๏ฟฝฮฆ ๏ฟฝ
๏ฟฝ๏ฟฝ
ฯƒ
๐œ‡๐œ‡ โˆ’ ๐‘ก๐‘ก
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ ln ๏ฟฝ๐›ท๐›ท ๏ฟฝ
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ
Normal Continuous Distribution
117
Properties and Moments
µ
Median
µ
Mode
Mean -
1st
Variance -
µ
Raw Moment
2nd
ฯƒ2
Central Moment
0
Skewness - 3rd Central Moment
0
Excess kurtosis - 4th Central Moment
1
exp ๏ฟฝ๐‘–๐‘–๐‘–๐‘–๐‘–๐‘– โˆ’ 2๐œŽ๐œŽ 2 ๐‘ก๐‘ก 2 ๏ฟฝ
Characteristic Function
๐‘ก๐‘ก๐›ผ๐›ผ = µ + ฯƒฮฆโˆ’1 (ฮฑ)
= µ + ฯƒโˆš2 erf โˆ’1 (2๐›ผ๐›ผ โˆ’ 1)
100๐›ผ๐›ผ% Percentile Function
Parameter Estimation
Plotting Method
X-Axis
Y-Axis
๐‘ก๐‘ก๐‘–๐‘–
๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–[๐น๐น(๐‘ก๐‘ก๐‘–๐‘– )]
๏ฟฝ=
ฯƒ
Maximum Likelihood Function
Likelihood
Function
For complete data:
๐ฟ๐ฟ(๐œ‡๐œ‡, ๐œŽ๐œŽ|๐ธ๐ธ) =
=
Log-Likelihood
Function
µ๏ฟฝ = โˆ’
1 ๏ฟฝ2
1
, ฯƒ = 2
๐‘š๐‘š
๐‘š๐‘š
nF
1 ๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 2
๏ฟฝ
exp
๏ฟฝโˆ’
๏ฟฝ
๏ฟฝ ๏ฟฝ
nF
2 ๐œŽ๐œŽ
i=1
๏ฟฝฯƒโˆš2ฯ€๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
1
failures
๐‘›๐‘›๐น๐น
1
๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2 ๏ฟฝ
nF exp ๏ฟฝโˆ’
2๐œŽ๐œŽ 2
๏ฟฝฯƒโˆš2ฯ€๏ฟฝ
๐‘–๐‘–=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
1
failures
๐‘›๐‘›๐น๐น
1
ฮ›(๐œ‡๐œ‡, ๐œŽ๐œŽ|๐ธ๐ธ) = โˆ’nF ln๏ฟฝฯƒโˆš2ฯ€๏ฟฝ โˆ’ 2 ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
2๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
โˆ‚ฮ›
=0
โˆ‚µ
solve for ๐œ‡๐œ‡ to get MLE ๐œ‡๐œ‡ฬ‚ :
โˆ‚ฮ›
=0
โˆ‚ฯƒ
solve for ๐œŽ๐œŽ to get ๐œŽ๐œŽ๏ฟฝ:
failures
๐‘›๐‘›๐น๐น
โˆ‚ฮ› ๐œ‡๐œ‡๐‘›๐‘›๐น๐น
1
= 2 โˆ’ 2 ๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘– = 0
โˆ‚µ
๐œŽ๐œŽ
๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
nF 1
= โˆ’ + 3 ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2 = 0
โˆ‚ฯƒ
ฯƒ ๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
๐‘๐‘
๐‘š๐‘š
Normal
Least
Mean
Square
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
118
Univariate Continuous Distributions
MLE
Point
Estimates
When there is only complete failure data the point estimates can be
given as:
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
1
๐œ‡๐œ‡ฬ‚ = ๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘–
nF
๏ฟฝ2 = 1 ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
๐œŽ๐œŽ
nF
๐‘–๐‘–=1
๐‘–๐‘–=1
In most cases the unbiased estimators are used:
๐‘›๐‘›๐น๐น
1
๐œ‡๐œ‡ฬ‚ = ๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘–
nF
๐‘–๐‘–=1
Fisher
Information
100๐›พ๐›พ%
Confidence
Intervals
Normal
(for complete
data)
๏ฟฝ2 =
๐œŽ๐œŽ
๐ผ๐ผ(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 ) = ๏ฟฝ
1 Sided - Lower
๐๐
๐ˆ๐ˆ๐Ÿ๐Ÿ
๐œ‡๐œ‡ฬ‚ โˆ’
๏ฟฝ2
๐œŽ๐œŽ
๐œŽ๐œŽ๏ฟฝ
โˆš๐‘›๐‘›
๐‘ก๐‘กฮณ (n โˆ’ 1)
(๐‘›๐‘› โˆ’ 1)
๐œ’๐œ’๐›ผ๐›ผ2 (๐‘›๐‘› โˆ’ 1)
๐‘›๐‘›๐น๐น
1
๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
nF โˆ’ 1
๐‘–๐‘–=1
1/๐œŽ๐œŽ 2
0
0
๏ฟฝ
โˆ’1/2๐œŽ๐œŽ 4
2 Sided - Lower
๐œ‡๐œ‡ฬ‚ โˆ’
๏ฟฝ2
๐œŽ๐œŽ
๐œŽ๐œŽ๏ฟฝ
โˆš๐‘›๐‘›
๐‘ก๐‘ก๏ฟฝ1+ฮณ๏ฟฝ (n โˆ’ 1)
2
(๐‘›๐‘› โˆ’ 1)
2 Sided - Upper
๐œ‡๐œ‡ฬ‚ +
๏ฟฝ2
๐œŽ๐œŽ
๐œ’๐œ’ 21+ฮณ (๐‘›๐‘› โˆ’ 1)
๏ฟฝ
2
๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
โˆš๐‘›๐‘›
๐‘ก๐‘ก๏ฟฝ1+ฮณ๏ฟฝ (n โˆ’ 1)
2
(๐‘›๐‘› โˆ’ 1)
๐œ’๐œ’ 21โˆ’ฮณ (๐‘›๐‘› โˆ’ 1)
๏ฟฝ
2
๏ฟฝ
(Nelson 1982, pp.218-220) Where ๐‘ก๐‘กฮณ (n โˆ’ 1) is the 100๐›พ๐›พth percentile of
the t-distribution with ๐‘›๐‘› โˆ’ 1 degrees of freedom and ๐œ’๐œ’๐›พ๐›พ2 (๐‘›๐‘› โˆ’ 1) is the
100๐›พ๐›พth percentile of the ๐œ’๐œ’ 2 -distribution with ๐‘›๐‘› โˆ’ 1 degrees of freedom.
Bayesian
Type
Non-informative Priors when ๐ˆ๐ˆ๐Ÿ๐Ÿ is known, ๐…๐…๐ŸŽ๐ŸŽ (๐๐)
(Yang and Berger 1998, p.22)
Prior
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Proper
Prior with limits
๐œ‡๐œ‡ โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
1
All
Type
Uniform Proper
Prior with limits
๐œŽ๐œŽ 2 โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
Posterior
Truncated Normal Distribution
For a โ‰ค µ โ‰ค b
โˆ‘๐‘›๐‘›๐น๐น ๐‘ก๐‘ก๐‘–๐‘–๐น๐น ฯƒ2
๐‘๐‘. ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµ; ๐‘–๐‘–=1 , ๏ฟฝ
๐‘›๐‘›๐น๐น
nF
Otherwise ๐œ‹๐œ‹(๐œ‡๐œ‡) = 0
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµ;
when ๐œ‡๐œ‡ โˆˆ (โˆž, โˆž)
๐น๐น
โˆ‘๐‘›๐‘›๐‘–๐‘–=1
๐‘ก๐‘ก๐‘–๐‘–๐น๐น ฯƒ2
, ๏ฟฝ
๐‘›๐‘›๐น๐น
nF
Non-informative Priors when ๐๐ is known, ๐…๐…๐’๐’ (๐ˆ๐ˆ๐Ÿ๐Ÿ )
(Yang and Berger 1998, p.23)
Prior
Posterior
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Truncated Inverse Gamma Distribution
For a โ‰ค ฯƒ2 โ‰ค b
Normal Continuous Distribution
119
(๐‘›๐‘›๐น๐น โˆ’ 2) S2
, ๏ฟฝ
2
2
Otherwise ๐œ‹๐œ‹(๐œŽ๐œŽ 2 ) = 0
๐‘๐‘. ๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
1
Uniform Improper
Prior with limits
๐œŽ๐œŽ 2 โˆˆ (0, โˆž)
1
๐œŽ๐œŽ 2
Jefferyโ€™s,
Reference, MDIP
Prior
Type
๐‘›๐‘›๐น๐น S2
, ๏ฟฝ
2 2
2
with limits ๐œŽ๐œŽ โˆˆ (0, โˆž)
See section 1.7.1
๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
Non-informative Priors when ๐๐ and ๐ˆ๐ˆ๐Ÿ๐Ÿ are unknown, ๐…๐…๐’๐’ (๐๐, ๐ˆ๐ˆ๐Ÿ๐Ÿ )
(Yang and Berger 1998, p.23)
Prior
Posterior
1
Jefferyโ€™s Prior
1
๐œŽ๐œŽ 4
Reference Prior
ordering {๐œ™๐œ™, ๐œŽ๐œŽ}
Reference where
๐œ‡๐œ‡ and ๐œŽ๐œŽ 2 are
separate groups.
๐œ‹๐œ‹๐‘œ๐‘œ (๐œ™๐œ™, ๐œŽ๐œŽ 2 )
1
โˆ
๐œŽ๐œŽ๏ฟฝ2 + ๐œ™๐œ™ 2
where
๐œ™๐œ™ = ๐œ‡๐œ‡/๐œŽ๐œŽ
1
๐œŽ๐œŽ 2
S2
๏ฟฝ
nF (nF โˆ’ 3)
See section 1.7.2
(๐‘›๐‘›๐น๐น โˆ’ 3) S2
2
๐œ‹๐œ‹(๐œŽ๐œŽ |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
, ๏ฟฝ
2
2
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡|๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµ; nF โˆ’ 3, ๐‘ก๐‘กฬ…,
S2
๏ฟฝ
nF (nF + 1)
when ๐œ‡๐œ‡ โˆˆ (โˆž, โˆž)
See section 1.7.2
(๐‘›๐‘›๐น๐น + 1) S2
๐œ‹๐œ‹(๐œŽ๐œŽ 2 |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
, ๏ฟฝ
2
2
2
when ๐œŽ๐œŽ โˆˆ (0, โˆž)
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡|๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµ; nF + 1, ๐‘ก๐‘กฬ…,
No Closed Form
S2
๏ฟฝ
nF (nF โˆ’ 1)
when ๐œ‡๐œ‡ โˆˆ (โˆž, โˆž)
See section 1.7.2
(๐‘›๐‘›๐น๐น โˆ’ 1) S2
๐œ‹๐œ‹(๐œŽ๐œŽ 2 |๐ธ๐ธ)~๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
, ๏ฟฝ
2
2
2
when ๐œŽ๐œŽ โˆˆ (0, โˆž)
See section 1.7.1
๐œ‹๐œ‹(๐œ‡๐œ‡|๐ธ๐ธ)~๐‘‡๐‘‡ ๏ฟฝµ; nF โˆ’ 1, ๐‘ก๐‘กฬ…,
Normal
Improper Uniform
with limits:
๐œ‡๐œ‡ โˆˆ (โˆž, โˆž)
๐œŽ๐œŽ 2 โˆˆ (0, โˆž)
MDIP Prior
(๐‘›๐‘›๐น๐น โˆ’ 2) S2
, ๏ฟฝ
2
2
See section 1.7.1
๐ผ๐ผ๐ผ๐ผ ๏ฟฝฯƒ2 ;
120
Univariate Continuous Distributions
where
๐‘›๐‘›๐น๐น
๐‘†๐‘† = ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐‘ก๐‘กฬ…)2
2
๐‘–๐‘–=1
and
๐‘ก๐‘กฬ… =
๐‘›๐‘›๐น๐น
1
๏ฟฝ ๐‘ก๐‘ก๐‘–๐‘–
nF
๐‘–๐‘–=1
Conjugate Priors
UOI
Likelihood
Model
Evidence
Dist. of
UOI
Prior
Para
Posterior Parameters
๐‘›๐‘›
๐œ‡๐œ‡
from
Normal
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
๐œŽ๐œŽ 2
from
2
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ )
๐œ‡๐œ‡๐‘๐‘
from
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 )
Normal
with known
๐œŽ๐œŽ 2
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
Normal
with known
๐œ‡๐œ‡
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
Lognormal
with known
๐œŽ๐œŽ๐‘๐‘2
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
Normal
Gamma
u๐‘œ๐‘œ , ๐‘ฃ๐‘ฃ0
๐‘˜๐‘˜0 , ๐œ†๐œ†0
๐‘ฃ๐‘ฃ =
1
1 nF
+
๐‘ฃ๐‘ฃ0 ๐œŽ๐œŽ 2
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น /2
๐‘›๐‘›๐น๐น
1
๐œ†๐œ† = ๐œ†๐œ†๐‘œ๐‘œ + ๏ฟฝ(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
2
๐‘–๐‘–=1
๐‘›๐‘›
Normal
๐‘ข๐‘ข๐‘œ๐‘œ , ๐‘ฃ๐‘ฃ0
๐น๐น
ln(๐‘ก๐‘ก๐‘–๐‘– )
u0 โˆ‘๐‘–๐‘–=1
2+
ฯƒ0
๐œŽ๐œŽ๐‘๐‘2
๐‘ข๐‘ข =
1 ๐‘›๐‘›๐น๐น
+
๐‘ฃ๐‘ฃ 2 ๐œŽ๐œŽ๐‘๐‘2
๐‘ฃ๐‘ฃ =
Description , Limitations and Uses
Example
๐น๐น
๐น๐น
๐‘ก๐‘ก๐‘–๐‘–
๐‘ข๐‘ข0 โˆ‘๐‘–๐‘–=1
+
๐‘ฃ๐‘ฃ0
๐œŽ๐œŽ 2
๐‘ข๐‘ข =
1 ๐‘›๐‘›๐น๐น
+
๐‘ฃ๐‘ฃ0 ๐œŽ๐œŽ 2
1
1 ๐‘›๐‘›๐น๐น
+
๐‘ฃ๐‘ฃ 2 ๐œŽ๐œŽ๐‘๐‘2
The accuracy of a cutting machine used in manufacturing is desired
to be measured. 5 cuts at the required length are made and
measured as:
7.436, 10.270, 10.466, 11.039, 11.854 ๐‘š๐‘š๐‘š๐‘š
MLE Estimates are:
โˆ‘ ๐‘ก๐‘ก๐‘–๐‘–๐น๐น
= 10.213
nF
2
โˆ‘๏ฟฝ๐‘ก๐‘ก๐‘–๐‘–๐น๐น โˆ’ ๐œ‡๐œ‡๏ฟฝ๐‘ก๐‘ก ๏ฟฝ
๏ฟฝ2 =
๐œŽ๐œŽ
= 2.789
nF โˆ’ 1
๐œ‡๐œ‡ฬ‚ =
90% confidence interval for ๐œ‡๐œ‡:
๐œŽ๐œŽ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
๏ฟฝ๐œ‡๐œ‡ฬ‚ โˆ’
๐‘ก๐‘ก{0.95} (4),
๐œ‡๐œ‡ฬ‚ +
๐‘ก๐‘ก{0.95} (4)๏ฟฝ
โˆš5
โˆš5
[10.163, 10.262]
Normal Continuous Distribution
121
90% confidence interval for ๐œŽ๐œŽ 2 :
4
4
๏ฟฝ2
๏ฟฝ2
,
๐œŽ๐œŽ
๏ฟฝ
๏ฟฝ๐œŽ๐œŽ
๐œ’๐œ’{20.95} (4)
๐œ’๐œ’{20.05} (4)
[1.176, 15.697]
A Bayesian point estimate using the Jeffery non-informative improper
prior 1โ„๐œŽ๐œŽ 4 with posterior for ๐œ‡๐œ‡~๐‘‡๐‘‡(6, 10.213, 0.558 ) and ๐œŽ๐œŽ 2 ~๐ผ๐ผ๐ผ๐ผ(3,
5.578) has a point estimates:
๐œ‡๐œ‡ฬ‚ = E[๐‘‡๐‘‡(6,6.595,0.412 )] = µ = 10.213
๏ฟฝ2 = E[๐ผ๐ผ๐บ๐บ(3,5.578)] = 5.578 = 2.789
๐œŽ๐œŽ
2
๐น๐น๐‘‡๐‘‡โˆ’1 (0.95) = 11.665]
1/๐น๐น๐บ๐บโˆ’1 (0.05) = 6.822]
Also known as a Gaussian distribution or bell curve.
Unit Normal Distribution. Also known as the standard normal
distribution is when ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ = 1 with pdf ๐œ™๐œ™(๐‘ง๐‘ง) and cdf ฮฆ(z). If X
is normally distributed with mean ๐œ‡๐œ‡ and standard deviation ๐œŽ๐œŽ then the
following transformation is used:
๐‘ฅ๐‘ฅ โˆ’ ๐œ‡๐œ‡
๐‘ง๐‘ง =
๐œŽ๐œŽ
Central Limit Theorem. Let ๐‘‹๐‘‹1 , ๐‘‹๐‘‹2 , โ€ฆ , ๐‘‹๐‘‹๐‘›๐‘› be a sequence of ๐‘›๐‘›
independent and identically distributed (i.i.d) random variables each
having a mean of ๐œ‡๐œ‡ and a variance of ๐œŽ๐œŽ 2 . As the sample size
increases, the distribution of the sample average of these random
variables approaches the normal distribution with mean ๐œ‡๐œ‡ and
variance ๐œŽ๐œŽ 2 /๐‘›๐‘› irrespective of the shape of the original distribution.
Formally:
๐‘†๐‘†๐‘›๐‘› = ๐‘‹๐‘‹1 + โ‹ฏ + ๐‘‹๐‘‹๐‘›๐‘›
If we define a new random variables:
๐‘†๐‘†๐‘›๐‘› โˆ’ ๐‘›๐‘›๐‘›๐‘›
๐‘๐‘๐‘›๐‘› =
, ๐‘Ž๐‘Ž๐‘›๐‘›๐‘‘๐‘‘
๐œŽ๐œŽโˆš๐‘›๐‘›
๐‘Œ๐‘Œ =
๐‘†๐‘†๐‘›๐‘›
๐‘›๐‘›
The distribution of ๐‘๐‘๐‘›๐‘› converges to the standard normal distribution.
The distribution of ๐‘†๐‘†๐‘›๐‘› converges to a normal distribution with mean ๐œ‡๐œ‡
and standard deviation of ๐œŽ๐œŽ/โˆš๐‘›๐‘›.
Sigma Intervals. Often intervals of the normal distribution are
expressed in terms of distance away from the mean in units of sigma.
Normal
Characteristics
With 90% confidence intervals:
๐œ‡๐œ‡
[๐น๐น๐‘‡๐‘‡โˆ’1 (0.05) = 8.761,
2
๐œŽ๐œŽ
[1/๐น๐น๐บ๐บโˆ’1 (0.95) = 0.886,
122
Univariate Continuous Distributions
The following is approximate values for each sigma:
Interval
๐œ‡๐œ‡ ± ๐œŽ๐œŽ
๐œ‡๐œ‡ ± 2๐œŽ๐œŽ
๐œ‡๐œ‡ ± 3๐œŽ๐œŽ
๐œ‡๐œ‡ ± 4๐œŽ๐œŽ
๐œ‡๐œ‡ ± 5๐œŽ๐œŽ
๐œ‡๐œ‡ ± 6๐œŽ๐œŽ
๐šฝ๐šฝ(๐๐ + ๐’๐’๐’๐’) โˆ’ ๐šฝ๐šฝ(๐๐ โˆ’ ๐’๐’๐’๐’)
68.2689492137%
95.4499736104%
99.7300203937%
99.9936657516%
99.9999426697%
99.9999998027%
Truncated Normal. Often in reliability engineering a truncated
normal distribution may be used due to the limitation that ๐‘ก๐‘ก โ‰ฅ 0. See
Truncated Normal Continuous Distribution.
Inflection Points:
Inflection points occur one standard deviation away from the mean
(๐œ‡๐œ‡ ± ๐œŽ๐œŽ).
Mean / Median / Mode:
The mean, median and mode are always equal to ๐œ‡๐œ‡.
Normal
Hazard Rate. The hazard rate is increasing for all ๐‘ก๐‘ก. The Standard
Normal Distributionโ€™s hazard rate approaches โ„Ž(๐‘ก๐‘ก) = ๐‘ก๐‘ก as ๐‘ก๐‘ก becomes
large.
Let:
Convolution Property
๐‘›๐‘›
Scaling Property
X~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(µ, ฯƒ2 )
๏ฟฝ ๐‘‹๐‘‹๐‘–๐‘– ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๏ฟฝ ๐œ‡๐œ‡๐‘–๐‘– , โˆ‘๐œŽ๐œŽ๐‘–๐‘–2 ๏ฟฝ
๐‘–๐‘–=1
๐‘Ž๐‘Ž๐‘Ž๐‘Ž + ๐‘๐‘~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘Ž๐‘Ž๐‘Ž๐‘Ž + ๐‘๐‘, ๐‘Ž๐‘Ž 2 ๐œŽ๐œŽ 2 )
Linear Combination Property:
๐‘›๐‘›
๏ฟฝ ๐‘Ž๐‘Ž๐‘–๐‘– ๐‘‹๐‘‹๐‘–๐‘– + ๐‘๐‘๐‘–๐‘– ~๐‘๐‘๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ ๏ฟฝ๏ฟฝ{๐‘Ž๐‘Ž๐‘–๐‘– ๐œ‡๐œ‡๐‘–๐‘– + ๐‘๐‘๐‘–๐‘– } , โˆ‘๏ฟฝ๐‘Ž๐‘Ž๐‘–๐‘–2 ๐œŽ๐œŽ๐‘–๐‘–2 ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
Applications
Approximations to Other Distributions. The origin of the Normal
Distribution was from an approximation of the Binomial distribution.
Due to the Central Limit Theory the Normal distribution can be used
to approximate many distributions as detailed under โ€˜Related
Distributionsโ€™.
Strength Stress Interference. When the strength of a component
follows a distribution and the stress that component is subjected to
follows a distribution there exists a probability that the stress will be
greater than the strength. When both distributions are a normal
distribution, there is a closed for solution to the interference
Normal Continuous Distribution
123
probability.
Life Distribution. When used as a life distribution a truncated
Normal Distribution may be used due to the constraint ๐‘ก๐‘ก โ‰ฅ 0.
However it is often found that the difference in results is negligible.
(Rausand & Høyland 2004)
Time Distributions. The normal distribution may be used to model
simple repair or inspection tasks that have a typical duration with
variation which is symmetrical about the mean. This is typical for
inspection and preventative maintenance times.
Analysis of Variance (ANOVA). A test used to analyze variance
and dependence of variables. A popular model used to conduct
ANOVA assumes the data comes from a normal population.
Online:
http://www.weibull.com/LifeDataWeb/the_normal_distribution.htm
http://mathworld.wolfram.com/NormalDistribution.html
http://en.wikipedia.org/wiki/Normal_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution
2nd ed., CRC.
Simon, M.K., 2006. Probability Distributions Involving Gaussian
Random Variables: A Handbook for Engineers and Scientists,
Springer.
Relationship to Other Distributions
Truncated Normal
Distribution
Let:
๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡(๐‘ฅ๐‘ฅ; ๐œ‡๐œ‡, ๐œŽ๐œŽ, ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ )
Then:
Lognormal
Distribution
Let:
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐‘ก๐‘ก; ๐œ‡๐œ‡๐‘๐‘ , ๐œŽ๐œŽ๐‘๐‘2 )
Then:
Where:
๐‘‹๐‘‹~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
๐‘‹๐‘‹ โˆˆ (โˆž, โˆž)
๐‘Œ๐‘Œ~T๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 , ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ )
๐‘Œ๐‘Œ โˆˆ [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]
๐‘‹๐‘‹~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(๐œ‡๐œ‡๐‘๐‘ , ฯƒ2N )
๐‘Œ๐‘Œ = ln(๐‘‹๐‘‹)
๐‘Œ๐‘Œ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
Normal
Resources
Six Sigma Quality Management. Six sigma is a business
management strategy which aims to reduce costs in manufacturing
processes by removing variance in quality (defects). Current
manufacturing standards aim for an expected 3.4 defects out of one
million parts: 2ฮฆ(โˆ’6). (Six Sigma Academy 2009)
124
Univariate Continuous Distributions
Rayleigh
Distribution
๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐‘ก๐‘ก; ๐œŽ๐œŽ)
๐œ‡๐œ‡2
๐œ‡๐œ‡๐‘๐‘ = ln ๏ฟฝ
๏ฟฝ,
2
๏ฟฝ๐œŽ๐œŽ + ๐œ‡๐œ‡2
Let
Then
Normal
Binomial
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘)
Poisson
Distribution
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k; µ)
Beta Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Gamma
Distribution
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y~๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…๐‘…โ„Ž(๐œŽ๐œŽ)
Let
Chi-square
Distribution
๐œ’๐œ’ 2 (๐‘ก๐‘ก; ๐‘ฃ๐‘ฃ)
๐‘‹๐‘‹1 , ๐‘‹๐‘‹2 ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(0, ฯƒ)
๐œŽ๐œŽ 2 + ๐œ‡๐œ‡2
๐œŽ๐œŽ๐‘๐‘ = ๏ฟฝln ๏ฟฝ
๏ฟฝ
๐œ‡๐œ‡2
Then
๐‘‹๐‘‹๐‘–๐‘–
~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(µ, ฯƒ2 )
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y~๐œ’๐œ’ 2 (๐‘ก๐‘ก; ๐‘ฃ๐‘ฃ)
Y = ๏ฟฝX12 + X22
v
Xk โˆ’ µ 2
Y = ๏ฟฝ๏ฟฝ
๏ฟฝ
ฯƒ
k=1
Limiting Case for constant ๐‘๐‘:
lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝk; µ = n๐‘๐‘, ๐œŽ๐œŽ 2 = ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)๏ฟฝ
๐‘›๐‘›โ†’โˆž
๐‘๐‘=๐‘๐‘
The Normal distribution can be used as an approximation of the
Binomial distribution when ๐‘›๐‘›๐‘›๐‘› โ‰ฅ 10 and ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘) โ‰ฅ 10.
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘, ๐‘›๐‘›) โ‰ˆ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐‘ก๐‘ก = ๐‘˜๐‘˜ + 0.5; ๐œ‡๐œ‡ = ๐‘›๐‘›๐‘›๐‘›, ๐œŽ๐œŽ 2 = ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)๏ฟฝ
lim ๐น๐น๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ‡๐œ‡) = ๐น๐น๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ (๐‘˜๐‘˜; ๐œ‡๐œ‡โ€ฒ = ๐œ‡๐œ‡, ๐œŽ๐œŽ = ๏ฟฝ๐œ‡๐œ‡)
๐œ‡๐œ‡โ†’โˆž
This is a good approximation when ๐œ‡๐œ‡ > 1000. When ๐œ‡๐œ‡ > 10 the same
approximation can be made with a correction:
lim ๐น๐น๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ‡๐œ‡) = ๐น๐น๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ (๐‘˜๐‘˜; ๐œ‡๐œ‡โ€ฒ = ๐œ‡๐œ‡ โˆ’ 0.5, ๐œŽ๐œŽ = ๏ฟฝ๐œ‡๐œ‡)
๐œ‡๐œ‡โ†’โˆž
For large ๐›ผ๐›ผ and ๐›ฝ๐›ฝ with fixed ๐›ผ๐›ผ/๐›ฝ๐›ฝ:
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ, ๐›ฝ๐›ฝ) โ‰ˆ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝµ =
ฮฑ
ฮฑฮฒ
, ๐œŽ๐œŽ = ๏ฟฝ
๏ฟฝ
2
(ฮฑ
(ฮฑ
+ ฮฒ)
+ ฮฒ + 1)
ฮฑ+ฮฒ
As ๐›ผ๐›ผ and ๐›ฝ๐›ฝ increase the mean remains constant and the variance is
reduced.
Special Case for large k:
lim ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œ‡๐œ‡ =
๐‘˜๐‘˜โ†’โˆž
๐‘˜๐‘˜
๐‘˜๐‘˜
, ๐œŽ๐œŽ = ๏ฟฝ 2 ๏ฟฝ
๐œ†๐œ†
๐œ†๐œ†
Pareto Continuous Distribution
125
4.6. Pareto Continuous Distribution
Probability Density Function - f(t)
1.0
ฮธ=1 ฯƒ=1
ฮธ=2 ฯƒ=1
ฮธ=3 ฯƒ=1
0.8
0.6
0.4
0.2
0.0
0
2
4
6
8
Cumulative Density Function - F(t)
2
4
6
8
Hazard Rate - h(t)
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
2
4
6
2
4
6
8
8
ฮธ=1 ฯƒ=1
ฮธ=1 ฯƒ=2
ฮธ=1 ฯƒ=4
0
2
4
6
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
ฮธ=1 ฯƒ=1
ฮธ=2 ฯƒ=1
ฮธ=3 ฯƒ=1
0
0
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
ฮธ=1 ฯƒ=1
ฮธ=2 ฯƒ=1
ฮธ=3 ฯƒ=1
0
ฮธ=1 ฯƒ=1
ฮธ=1 ฯƒ=2
ฮธ=1 ฯƒ=4
Pareto
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
8
ฮธ=1 ฯƒ=1
ฮธ=1 ฯƒ=2
ฮธ=1 ฯƒ=4
0
2
4
6
8
126
Univariate Continuous Distributions
Parameters & Description
Parameters
Limits
ฮธ
ฮธ>0
๐›ผ๐›ผ
ฮฑ>0
Distribution
ฮฑฮธฮฑ
t ฮฑ+1
ฮธ ฮฑ
๐น๐น(๐‘ก๐‘ก) = 1 โˆ’ ๏ฟฝ ๏ฟฝ
t
ฮธ ฮฑ
R(t) = ๏ฟฝ ๏ฟฝ
t
Reliability
Pareto
ฮธโ‰คt< โˆž
๐‘“๐‘“(๐‘ก๐‘ก) =
CDF
Mean Residual
Life
Shape parameter. Sometimes called
the Pareto index.
Formulas
PDF
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Location parameter. ฮธ is the lower
limit of t. Sometimes refered to as tminimum.
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
(t)ฮฑ
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
=
๐‘…๐‘…(๐‘ก๐‘ก)
(t + x)ฮฑ
Where
๐‘ก๐‘ก is the given time we know the component has survived to time
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
๐‘ข๐‘ข(๐‘ก๐‘ก) =
โˆž
โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘…๐‘…(๐‘ก๐‘ก)
โ„Ž(๐‘ก๐‘ก) =
Hazard Rate
Cumulative
Hazard Rate
ฮฑ
t
๐‘ก๐‘ก
๐ป๐ป(๐‘ก๐‘ก) = ๐›ผ๐›ผ ln ๏ฟฝ ๏ฟฝ
๐œƒ๐œƒ
Properties and Moments
Median
Mode
Mean - 1st Raw Moment
Variance - 2nd Central Moment
Skewness - 3rd Central Moment
ฮธ21โ„ฮฑ
ฮธ
ฮฑฮธ
, for ฮฑ > 1
ฮฑโˆ’1
ฮฑฮธ2
, for ฮฑ > 2
(ฮฑ โˆ’ 1)2 (ฮฑ โˆ’ 2)
2(1 + ฮฑ) ฮฑ โˆ’ 2
๏ฟฝ
, for ฮฑ > 3
(ฮฑ โˆ’ 3)
ฮฑ
Pareto Continuous Distribution
127
6(ฮฑ3 + ฮฑ2 โˆ’ 6ฮฑ โˆ’ 2)
, for ฮฑ > 4
ฮฑ(ฮฑ โˆ’ 3)(ฮฑ โˆ’ 4)
Excess kurtosis - 4th Central Moment
๐›ผ๐›ผ(โˆ’๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–)๐›ผ๐›ผ ฮ“(โˆ’๐›ผ๐›ผ, โˆ’๐‘–๐‘–๐‘–๐‘–๐‘–๐‘–)
Characteristic Function
๐‘ก๐‘ก๐›พ๐›พ = ฮธ(1 โˆ’ ฮณ)โˆ’1โ„ฮฑ
100๐›พ๐›พ % Percentile Function
Parameter Estimation
Plotting Method
Least Mean
Square
๐‘ฆ๐‘ฆ = ๐‘š๐‘š๐‘š๐‘š + ๐‘๐‘
X-Axis
Likelihood
Function
For complete data:
ln(๐‘ก๐‘ก๐‘–๐‘– )
ln[1 โˆ’ ๐น๐น]
Maximum Likelihood Function
nF
1
๐ฟ๐ฟ(๐œƒ๐œƒ, ๐›ผ๐›ผ|๐ธ๐ธ) = ฮฑnF ฮธฮฑnF ๏ฟฝ
ฮฑ+1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1 t i
failures
Log-Likelihood
Function
๐›ผ๐›ผ๏ฟฝ = โˆ’๐‘š๐‘š
๐‘๐‘
๏ฟฝ
๐œƒ๐œƒ = exp ๏ฟฝ ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ
๐‘›๐‘›๐น๐น
ฮ›(๐œƒ๐œƒ, ๐›ผ๐›ผ|๐ธ๐ธ) = nF ln(ฮฑ) + nF ฮฑln(ฮธ) โˆ’ (๐›ผ๐›ผ + 1) ๏ฟฝ ln ๐‘ก๐‘ก๐‘–๐‘–
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
solve for ๐›ผ๐›ผ to get ๐›ผ๐›ผ๏ฟฝ:
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
nF
= โˆ’ + nF ln ฮธ โˆ’ ๏ฟฝ ln ๐‘ก๐‘ก๐‘–๐‘– = 0
โˆ‚ฮฑ
ฮฑ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
MLE Point
Estimates
The likelihood function increases as ๐œƒ๐œƒ increases. Therefore the MLE
point estimate is the largest ๐œƒ๐œƒ which satisfies ฮธ โ‰ค t i < โˆž:
๐œƒ๐œƒ๏ฟฝ = min๏ฟฝt1 , โ€ฆ , t nF ๏ฟฝ
Substituting ๐œƒ๐œƒ๏ฟฝ gives the MLE for ๐›ผ๐›ผ๏ฟฝ:
๐›ผ๐›ผ๏ฟฝ =
Fisher
Information
100๐›พ๐›พ%
Confidence
Intervals
๐‘›๐‘›๐น๐น
๐น๐น
โˆ‘๐‘›๐‘›๐‘–๐‘–=1
๏ฟฝln ๐‘ก๐‘ก๐‘–๐‘– โˆ’ ln๐œƒ๐œƒ๏ฟฝ๏ฟฝ
๐ผ๐ผ(๐œƒ๐œƒ, ๐›ผ๐›ผ) = ๏ฟฝ
1-Sided Lower
๐›ผ๐›ผ๏ฟฝ
if ๐œƒ๐œƒ is
unknown
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’
(2n โˆ’ 2)
2nF {1โˆ’ฮณ}
โˆ’1/๐›ผ๐›ผ 2
0
0
๏ฟฝ
1/๐œƒ๐œƒ 2
2-Sided Lower
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’ 1โˆ’ฮณ (2n โˆ’ 2)
2nF ๏ฟฝ 2 ๏ฟฝ
2-Sided Upper
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’ 1+ฮณ (2n โˆ’ 2)
2nF ๏ฟฝ 2 ๏ฟฝ
Pareto
โˆ‚ฮ›
=0
โˆ‚ฮฑ
Y-Axis
128
Univariate Continuous Distributions
(for complete
data)
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’
(2n)
2nF {1โˆ’ฮณ}
๐›ผ๐›ผ๏ฟฝ
if ๐œƒ๐œƒ is
known
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’ 1โˆ’ฮณ (2n)
2nF ๏ฟฝ 2 ๏ฟฝ
๐›ผ๐›ผ๏ฟฝ 2
๐œ’๐œ’ 1+ฮณ (2n โˆ’ 2)
2nF ๏ฟฝ 2 ๏ฟฝ
(Johnson et al. 1994, p.583) Where ๐œ’๐œ’๐›พ๐›พ2 (๐‘›๐‘›) is the 100๐›พ๐›พth percentile of the
๐œ’๐œ’ 2 -distribution with ๐‘›๐‘› degrees of freedom.
Bayesian
Non-informative Priors when ๐œฝ๐œฝ is known, ๐…๐…๐ŸŽ๐ŸŽ (๐œถ๐œถ)
(Yang and Berger 1998, p.22)
Type
Prior
Jeffery
Reference
and
Pareto
Conjugate Priors
UOI
Likelihood
Model
Evidence
๐‘๐‘
from
๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ก๐‘ก; a, ๐‘๐‘)
Uniform
with known
a
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก; ฮธ, ๐›ผ๐›ผ)
Pareto with
known ๐›ผ๐›ผ
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
Pareto with
known ๐œƒ๐œƒ
๐‘›๐‘›๐น๐น failures
at times ๐‘ก๐‘ก๐‘–๐‘–
๐œƒ๐œƒ
from
๐›ผ๐›ผ
from
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘ก๐‘ก; ฮธ, ๐›ผ๐›ผ)
1
๐›ผ๐›ผ
Dist. of
UOI
Prior
Para
Pareto
๐œƒ๐œƒ๐‘œ๐‘œ , ๐›ผ๐›ผ0
Pareto
Gamma
a 0 , ฮ˜0
k 0 , ฮป0
Description , Limitations and Uses
Example
Posterior
Parameters
๐œƒ๐œƒ = max{๐‘ก๐‘ก1 , โ€ฆ , ๐‘ก๐‘ก๐‘›๐‘›๐น๐น }
๐›ผ๐›ผ = ๐›ผ๐›ผ0 + ๐‘›๐‘›๐น๐น
a = a๐‘œ๐‘œ โˆ’ ๐›ผ๐›ผ๐‘›๐‘›๐น๐น
where a0 > ๐›ผ๐›ผ๐‘›๐‘›๐น๐น
ฮ˜ = ฮ˜0
k = k ๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
๐‘ฅ๐‘ฅ๐‘–๐‘–
ฮป = ฮป๐‘œ๐‘œ + ๏ฟฝ ln ๏ฟฝ ๏ฟฝ
๐œƒ๐œƒ
๐‘–๐‘–=1
5 components are put on a test with the following failure times:
108, 125, 458, 893, 13437 hours
MLE Estimates are:
๐œƒ๐œƒ๏ฟฝ = 108
Substituting ๐œƒ๐œƒ๏ฟฝ gives the MLE for ๐›ผ๐›ผ๏ฟฝ:
๐›ผ๐›ผ๏ฟฝ =
5
= 0.8029
๐น๐น
โˆ‘๐‘›๐‘›๐‘–๐‘–=1
(ln ๐‘ก๐‘ก๐‘–๐‘– โˆ’ ln(108))
90% confidence interval for ๐›ผ๐›ผ๏ฟฝ:
Pareto Continuous Distribution
๏ฟฝ
๐›ผ๐›ผ
๏ฟฝ
10
Characteristics
๐œ’๐œ’2 {0.05} (8),
๐›ผ๐›ผ
๏ฟฝ
10
129
๐œ’๐œ’2 {0.95} (8)๏ฟฝ
[0.2194, 1.2451]
80/20 Rule. Most commonly described as the basis for the โ€œ80/20
ruleโ€ (In a quality context, for example, 80% of manufacturing defects
will be a result from 20% of the causes).
Conditional Distribution. The conditional probability distribution
given that the event is greater than or equal to a value ๐œƒ๐œƒ1 exceeding
๐œƒ๐œƒ is a Pareto distribution with the same index ๐›ผ๐›ผ but with a minimum
๐œƒ๐œƒ1 instead of ๐œƒ๐œƒ.
Types. This distribution is known as a Pareto distribution of the first
kind. The Pareto distribution of the second kind (not detailed here) is
also known as the Lomax distribution. Pareto also proposed a third
distribution now known as a Pareto distribution of the third kind.
Pareto and the Lognormal Distribution. The Lognormal
distribution models similar physical phenomena as the Pareto
distribution. The two distributions have different weights at the
extremities.
Minimum property
Applications
Resources
For constant ๐œƒ๐œƒ.
Xi ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(ฮธ, ฮฑi )
Pareto
Let:
๐‘›๐‘›
min{๐‘‹๐‘‹, ๐‘‹๐‘‹2 , โ€ฆ , ๐‘‹๐‘‹๐‘›๐‘› }~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ ๏ฟฝ๐œƒ๐œƒ, ๏ฟฝ ๐›ผ๐›ผ๐‘–๐‘– ๏ฟฝ
๐‘–๐‘–=1
Rare Events. The survival function โ€˜slowlyโ€™ decreases compared to
most life distributions which makes it suitable for modeling rare
events which have large outcomes. Examples include natural events
such as the distribution of the daily rain fall, or the size of
manufacturing defects.
Online:
http://mathworld.wolfram.com/ParetoDistribution.html
http://en.wikipedia.org/wiki/Pareto_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Arnold, B., 1983. Pareto distributions, Fairland, MD: International Cooperative Pub. House.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous
Univariate Distributions, Vol. 1 2nd ed., Wiley-Interscience.
Relationship to Other Distributions
130
Univariate Continuous Distributions
Exponential
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
Chi-Squared
Distribution
ฯ‡2 (๐‘ฅ๐‘ฅ; ๐‘ฃ๐‘ฃ)
Pareto
Logistic
Distribution
๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(µ, ๐‘ ๐‘ )
Let
Then
Let
Then
๐‘Œ๐‘Œ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘Œ๐‘Œ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ† = ๐›ผ๐›ผ)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‹๐‘‹~๐œ’๐œ’ 2 (๐‘ฃ๐‘ฃ = 2)
(Johnson et al. 1994, p.526)
Let
Then
๐‘‹๐‘‹~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œƒ๐œƒ, ๐›ผ๐›ผ)
๐‘‹๐‘‹ = ln(๐‘Œ๐‘Œโ„๐œƒ๐œƒ)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‹๐‘‹ = 2ฮฑ ln(๐‘Œ๐‘Œโ„๐œƒ๐œƒ)
X ฮฑ
๐‘Œ๐‘Œ = โˆ’ ln ๏ฟฝ๏ฟฝ ๏ฟฝ โˆ’ 1๏ฟฝ
ฮธ
๐‘Œ๐‘Œ~๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ๐ฟ(0,1)
(Hastings et al. 2000, p.127)
Triangle Continuous Distribution
131
4.7. Triangle Continuous
Distribution
Probability Density Function - f(t)
a=0 b=1 c=0.5
a=0 b=1.5 c=0.5
a=0 b=2 c=0.5
2.0
1.5
1.0
0.5
0.0
0
0.5
1
1.5
2.0
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
2
a=0 b=1 c=0.25
a=0 b=1 c=0.5
a=0 b=1 c=0.75
0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
Cumulative Density Function - F(t)
1.0
0.8
0.8
0.6
0.6
0.4
Triangle
1.0
0.4
a=0 b=1 c=0.5
a=0 b=1.5 c=0.5
a=0 b=2 c=0.5
0.2
0.0
0.2
0.0
0
0.5
1
1.5
2
Hazard Rate - h(t)
10
9
8
7
6
5
4
3
2
1
0
0
0.5
10
9
8
7
6
5
4
3
2
1
0
1
1.5
2
a=0 b=1 c=0.25
a=0 b=1 c=0.5
a=0 b=1 c=0.75
0
0.5
1
132
Univariate Continuous Distributions
Parameters & Description
Parameters
Random Variable
๐‘Ž๐‘Ž
โˆ’โˆž โ‰ค ๐‘Ž๐‘Ž < ๐‘๐‘
๐‘๐‘
๐‘Ž๐‘Ž โ‰ค ๐‘๐‘ โ‰ค ๐‘๐‘
๐‘๐‘
๐‘Ž๐‘Ž < ๐‘๐‘ < โˆž
Distribution
Minimum Value. ๐‘Ž๐‘Ž is the lower bound
Maximum Value. ๐‘๐‘ is the upper
bound.
Mode Value. ๐‘๐‘ is the mode of the
distribution (top of the triangle).
๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘
Formulas
2(t โˆ’ a)
โŽง
for a โ‰ค t โ‰ค c
โŽช (b โˆ’ a)(c โˆ’ a)
๐‘“๐‘“(๐‘ก๐‘ก) =
โŽจ 2(b โˆ’ t)
โŽช(b โˆ’ a)(b โˆ’ c) for c โ‰ค t โ‰ค b
โŽฉ
PDF
(t โˆ’ a)2
โŽง
for a โ‰ค t โ‰ค c
โŽช (b โˆ’ a)(c โˆ’ a)
๐น๐น(๐‘ก๐‘ก) =
2
(b โˆ’ t)
โŽจ
โŽช1 โˆ’ (b โˆ’ a)(b โˆ’ c) for c โ‰ค t โ‰ค b
โŽฉ
Triangle
CDF
(t โˆ’ a)2
โŽง1 โˆ’
for a โ‰ค t โ‰ค c
โŽช
(b โˆ’ a)(c โˆ’ a)
๐‘…๐‘…(๐‘ก๐‘ก) =
2
โŽจ (b โˆ’ t)
for c โ‰ค t โ‰ค b
โŽช(b โˆ’ a)(b โˆ’ c)
โŽฉ
Reliability
Properties and Moments
Median
1
๐‘Ž๐‘Ž + ๏ฟฝ2(๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)(๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž) ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘๐‘ โ‰ฅ
1
๐‘๐‘ โˆ’ ๏ฟฝ2(๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)(๐‘๐‘ โˆ’ ๐‘๐‘) ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐‘๐‘ <
Mode
Mean -
1st
Raw Moment
Variance - 2nd Central Moment
Skewness - 3rd Central Moment
Excess kurtosis - 4th Central Moment
๐‘๐‘
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
2
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
2
๐‘Ž๐‘Ž + ๐‘๐‘ + ๐‘๐‘
3
๐‘Ž๐‘Ž2 + ๐‘๐‘ 2 + ๐‘๐‘ 2 โˆ’ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘๐‘๐‘๐‘
18
โˆš2(๐‘Ž๐‘Ž + ๐‘๐‘ โˆ’ 2๐‘๐‘)(2๐‘Ž๐‘Ž โˆ’ ๐‘๐‘ โˆ’ ๐‘๐‘)(๐‘Ž๐‘Ž โˆ’ 2๐‘๐‘ + ๐‘๐‘)
5(๐‘Ž๐‘Ž2 + ๐‘๐‘ 2 + ๐‘๐‘ 2 โˆ’ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘๐‘๐‘๐‘)3/2
โˆ’3
5
Triangle Continuous Distribution
Characteristic Function
โˆ’2
100ฮณ % Percentile Function
133
(b โˆ’ c)eita โˆ’ (b โˆ’ a)eitc + (c โˆ’ a)eitb
(b โˆ’ a)(c โˆ’ a)(b โˆ’ c)t 2
๐‘ก๐‘ก๐›พ๐›พ = a + ๏ฟฝฮณ(b โˆ’ a)(c โˆ’ a) ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐›พ๐›พ < ๐น๐น(๐‘๐‘)
๐‘ก๐‘ก๐›พ๐›พ = b โˆ’ ๏ฟฝ(1 โˆ’ ฮณ)(b โˆ’ a)(b โˆ’ c) ๐‘“๐‘“๐‘“๐‘“๐‘“๐‘“ ๐›พ๐›พ โ‰ฅ ๐น๐น(๐‘๐‘)
Parameter Estimation
Maximum Likelihood Function
r
nF
2(t i โˆ’ a)
2(b โˆ’ t i )
๐ฟ๐ฟ(๐‘Ž๐‘Ž, ๐‘๐‘, ๐‘๐‘|๐ธ๐ธ) = ๏ฟฝ
.๏ฟฝ
(b๏ฟฝ๏ฟฝ๏ฟฝ
โˆ’ a)(c
โˆ’๏ฟฝ๏ฟฝ
a) ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝi=1
๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=r+1 (b โˆ’ a)(b โˆ’ c)
Likelihood
Functions
failers to the left of c
r
2 ๐‘›๐‘›๐น๐น
ti
=๏ฟฝ
๏ฟฝ
bโˆ’a
Where failure times are ordered:
๐‘‡๐‘‡1 โ‰ค ๐‘‡๐‘‡2 โ‰ค โ‹ฏ โ‰ค ๐‘‡๐‘‡๐‘Ÿ๐‘Ÿ โ‰ค โ‹ฏ โ‰ค ๐‘‡๐‘‡๐‘›๐‘›๐น๐น
and ๐‘Ÿ๐‘Ÿ is the number of failure times less than ๐‘๐‘ and ๐‘ ๐‘  is the number of
failure times greater than ๐‘๐‘. Therefore ๐‘›๐‘›๐น๐น = ๐‘Ÿ๐‘Ÿ + ๐‘ ๐‘ .
The MLE estimates a๏ฟฝ , b๏ฟฝ , and c๏ฟฝ are obtained by numerically calculating
the likelihood function for different r and selecting the maximum where
c๏ฟฝ = Xr๏ฟฝ .
where
max
๐‘Ž๐‘Ž โ‰ค ๐‘๐‘ โ‰ค ๐‘๐‘
๐ฟ๐ฟ(๐‘Ž๐‘Ž, ๐‘๐‘, ๐‘๐‘|๐ธ๐ธ) = ๏ฟฝ
2 ๐‘›๐‘›๐น๐น
๏ฟฝ {๐‘€๐‘€(๐‘Ž๐‘Ž, ๐‘๐‘, ๐‘Ÿ๐‘Ÿฬ‚ (๐‘Ž๐‘Ž, ๐‘๐‘)}
bโˆ’a
nF
ti โˆ’ a
b โˆ’ ti
๏ฟฝ
i=r+1 (b โˆ’ t r )
i=1 (t r โˆ’ a)
rโˆ’1
๐‘€๐‘€(๐‘Ž๐‘Ž, ๐‘๐‘, ๐‘Ÿ๐‘Ÿ) = ๏ฟฝ
๐‘Ÿ๐‘Ÿ(๐‘Ž๐‘Ž, ๐‘๐‘) = arg max ๐‘€๐‘€(๐‘Ž๐‘Ž, ๐‘๐‘, ๐‘Ÿ๐‘Ÿ)
๐‘Ÿ๐‘Ÿโˆˆ{1,โ€ฆ,๐‘›๐‘›๐น๐น }
Note that the MLE estimates for a and ๐‘๐‘ are not the same as the uniform
distribution:
a๏ฟฝ โ‰  min(t1F , t F2 โ€ฆ )
๏ฟฝb โ‰  max(t1F , t F2 โ€ฆ )
(Kotz & Dorp 2004)
Description , Limitations and Uses
Example
When eliciting an opinion from an expert on the possible value of a
quantity, ๐‘ฅ๐‘ฅ, the expert may give :
- Lowest possible value = 0
- Highest possible value = 1
- Estimate of most likely value (mode) = 0.7
The corresponding distribution for ๐‘ฅ๐‘ฅ may be a triangle distribution
with parameters:
๐‘Ž๐‘Ž = 0,
๐‘๐‘ = 1,
๐‘๐‘ = 0.7
Triangle
Point
Estimates
failures to the right of c
nF
โˆ’a
b โˆ’ ti
๏ฟฝ
๏ฟฝ
(c
(b
โˆ’
a)
โˆ’ c)
i=r+1
i=1
134
Univariate Continuous Distributions
Characteristics
Standard Triangle Distribution. The standard triangle distribution
has ๐‘Ž๐‘Ž = 0, ๐‘๐‘ = 1. This distribution has a mean at ๏ฟฝ๐‘๐‘ โ„2 and median
at 1 โˆ’ ๏ฟฝ(1 โˆ’ ๐‘๐‘)โ„2.
Symmetrical Triangle Distribution. The symmetrical triangle
distribution occurs when ๐‘๐‘ = (๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)/2. The symmetrical triangle
distribution is formed from the average of two uniform random
variables (see related distributions).
Applications
Subjective Representation. The triangle distribution is often used
to model subjective evidence where ๐‘Ž๐‘Ž and ๐‘๐‘ are the bounds of the
estimation and ๐‘๐‘ is an estimation of the mode.
Triangle
Substitution to the Beta Distribution. Due to the triangle
distribution having bounded support it may be used in place of the
beta distribution.
Monte Carlo Simulation. Used to approximate distributions of
variables when the underlying distribution is unknown. A distribution
of interest is obtained by conducting Monte Carlo simulation of a
model using the triangle distributions as inputs.
Resources
Online:
http://mathworld.wolfram.com/TriangularDistribution.html
http://en.wikipedia.org/wiki/Triangular_distribution
Books:
Kotz, S. & Dorp, J.R.V., 2004. Beyond Beta: Other Continuous
Families Of Distributions With Bounded Support And Applications,
World Scientific Publishing Company.
Relationship to Other Distributions
Let
Uniform
Distribution
๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ก๐‘ก; a, b)
Beta Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ฮฑ, ๐›ฝ๐›ฝ )
Then
Xi ~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘Ž๐‘Ž, ๐‘๐‘)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y~๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡ ๏ฟฝa,
Special Cases:
Y=
bโˆ’a
, b๏ฟฝ
2
X1 + X 2
2
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1,2 ) = ๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡(0,0,1)
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(2,1 ) = ๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡(0,1,1)
Truncated Normal Continuous Distribution
135
4.8. Truncated Normal Continuous
Distribution
Probability Density Function - f(t)
ฮผ=-1 ฯƒ=1 a=0 b=2
ฮผ=0 ฯƒ=1 a=0 b=2
ฮผ=1 ฯƒ=1 a=0 b=2
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
-1
0
1
2
ฮผ=0.5 ฯƒ=0.5
ฮผ=0.5 ฯƒ=1
ฮผ=0.5 ฯƒ=3
1.8
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
3
-1
0
1
2
3
Cumulative Density Function - F(t)
1.0
0.8
0.8
0.6
0.6
0.4
0.4
ฮผ=-1 a=0 b=2
ฮผ=0 a=0 b=2
ฮผ=1 a=0 b=2
0.2
0.0
-1
0
1
2
Trunc Normal
1.0
ฮผ=0.5 ฯƒ=0.5 a=0
ฮผ=0.5 ฯƒ=1 b=2
ฮผ=0.5 ฯƒ=3
0.2
0.0
3
-1
0
1
Hazard Rate - h(t)
ฮผ=-1 ฯƒ=1 a=0 b=2
ฮผ=0 ฯƒ=1 a=0 b=2
ฮผ=1 ฯƒ=1 a=0 b=2
1.4
1.2
1.0
2
3
ฮผ=0.5 ฯƒ=0.5 a=0
ฮผ=0.5 ฯƒ=1 b=2
ฮผ=0.5 ฯƒ=3
1.2
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
-1
0
1
2
3
-1
0
1
2
3
136
Univariate Continuous Distributions
Parameters & Description
๐œ‡๐œ‡
โˆ’โˆž < µ < โˆž
๐‘Ž๐‘Ž๐ฟ๐ฟ
โˆ’โˆž < ๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘๐‘๐‘ˆ๐‘ˆ
๐‘๐‘๐‘ˆ๐‘ˆ
๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘๐‘๐‘ˆ๐‘ˆ < โˆž
๐œŽ๐œŽ 2
Parameters
ฯƒ2 > 0
Limits
Trunc Normal
Distribution
Left Truncated Normal
๐‘ฅ๐‘ฅ โˆˆ [0, โˆž)
for 0 โ‰ค ๐‘ฅ๐‘ฅ โ‰ค โˆž
PDF
๐œ™๐œ™(zx )
๐‘“๐‘“(๐‘ฅ๐‘ฅ) =
ฯƒฮฆ(โˆ’z0 )
otherwise
๐‘“๐‘“(๐‘ฅ๐‘ฅ) = 0
Location parameter: The mean of the
distribution.
Scale parameter: The standard
deviation of the distribution.
Lower Bound: ๐‘Ž๐‘Ž๐ฟ๐ฟ is the lower bound.
The standard normal transform of ๐‘Ž๐‘Ž๐ฟ๐ฟ
๐‘Ž๐‘Ž โˆ’๐œ‡๐œ‡
is ๐‘ง๐‘ง๐‘Ž๐‘Ž = ๐ฟ๐ฟ .
๐œŽ๐œŽ
Upper Bound: ๐‘๐‘๐‘ˆ๐‘ˆ is the upper bound.
The standard normal transform of ๐‘๐‘๐‘ˆ๐‘ˆ
๐‘๐‘ โˆ’๐œ‡๐œ‡
is ๐‘ง๐‘ง๐‘๐‘ = ๐‘ˆ๐‘ˆ .
๐œŽ๐œŽ
๐‘Ž๐‘Ž๐ฟ๐ฟ < ๐‘ฅ๐‘ฅ โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
General Truncated Normal
๐‘ฅ๐‘ฅ โˆˆ [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]
for ๐‘Ž๐‘Ž๐ฟ๐ฟ โ‰ค ๐‘ฅ๐‘ฅ โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
1
๐œ™๐œ™(zx )
ฯƒ
๐‘“๐‘“(๐‘ฅ๐‘ฅ) =
ฮฆ(zb ) โˆ’ ฮฆ(za )
otherwise
๐‘“๐‘“(๐‘ฅ๐‘ฅ) = 0
where
๐œ™๐œ™ is the standard normal pdf with ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ 2 = 1
ฮฆ is the standard normal cdf with ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ 2 = 1
๐‘–๐‘–โˆ’๐œ‡๐œ‡
๐‘ง๐‘ง๐‘–๐‘– = ๏ฟฝ ๏ฟฝ
๐œŽ๐œŽ
for ๐‘ฅ๐‘ฅ < 0
CDF
๐น๐น(๐‘ฅ๐‘ฅ) = 0
for 0 โ‰ค ๐‘ฅ๐‘ฅ < โˆž
๐น๐น(๐‘ฅ๐‘ฅ) =
for ๐‘ฅ๐‘ฅ < 0
Reliability
ฮฆ(zx ) โˆ’ ฮฆ(z0 )
ฮฆ(โˆ’z0 )
๐‘…๐‘…(๐‘ฅ๐‘ฅ) = 1
for 0 โ‰ค ๐‘ฅ๐‘ฅ < โˆž
๐‘…๐‘…(๐‘ฅ๐‘ฅ) =
ฮฆ(z0 ) โˆ’ ฮฆ(zx )
ฮฆ(โˆ’z0 )
for ๐‘ฅ๐‘ฅ < aL
๐น๐น(๐‘ฅ๐‘ฅ) = 0
for ๐‘Ž๐‘Ž๐ฟ๐ฟ โ‰ค ๐‘ฅ๐‘ฅ โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
ฮฆ(zx ) โˆ’ ฮฆ(za )
๐น๐น(๐‘ฅ๐‘ฅ) =
ฮฆ(zb ) โˆ’ ฮฆ(za )
for ๐‘ฅ๐‘ฅ > bU
for ๐‘ฅ๐‘ฅ < aL
๐น๐น(๐‘ฅ๐‘ฅ) = 1
๐‘…๐‘…(๐‘ฅ๐‘ฅ) = 1
for ๐‘Ž๐‘Ž๐ฟ๐ฟ โ‰ค ๐‘ฅ๐‘ฅ โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
ฮฆ(zb ) โˆ’ ฮฆ(zx )
๐‘…๐‘…(๐‘ฅ๐‘ฅ) =
ฮฆ(zb ) โˆ’ ฮฆ(za )
for ๐‘ฅ๐‘ฅ > bU
๐‘…๐‘…(๐‘ฅ๐‘ฅ) = 0
Truncated Normal Continuous Distribution
for ๐‘ก๐‘ก < 0
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
for ๐‘ก๐‘ก < aL
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
for 0 โ‰ค ๐‘ก๐‘ก < โˆž
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
137
for ๐‘Ž๐‘Ž๐ฟ๐ฟ โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
๐‘…๐‘…(๐‘ก๐‘ก)
1 โˆ’ ฮฆ(zt+x )
=
1 โˆ’ ฮฆ(zt )
µโˆ’xโˆ’t
ฮฆ๏ฟฝ
๏ฟฝ
ฯƒ
=
µโˆ’t
ฮฆ๏ฟฝ
๏ฟฝ
ฯƒ
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
๐‘…๐‘…(๐‘ก๐‘ก)
ฮฆ(zb ) โˆ’ ฮฆ(zt+x )
=
ฮฆ(zb ) โˆ’ ฮฆ(zt )
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = ๐‘…๐‘…(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก) =
for ๐‘ก๐‘ก > bU
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = 0
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก.
Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก. This operation is the equivalent of t replacing the
lower bound.
โˆž
โ„Ž(๐‘ฅ๐‘ฅ) = 0
for 0 โ‰ค ๐‘ฅ๐‘ฅ < โˆž
1
๐œ™๐œ™(zx )[1 โˆ’ ฮฆ(zx )]
โ„Ž(๐‘ฅ๐‘ฅ) = ฯƒ
[1 โˆ’ ฮฆ(z0 )]2
Cumulative Hazard
Rate
Properties and
Moments
Median
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ ln[๐‘…๐‘…(๐‘ก๐‘ก)]
โ„Ž(๐‘ฅ๐‘ฅ) = 0
for ๐‘Ž๐‘Ž๐ฟ๐ฟ โ‰ค ๐‘ฅ๐‘ฅ โ‰ค ๐‘๐‘๐‘ˆ๐‘ˆ
1
โ„Ž(๐‘ฅ๐‘ฅ) = ฯƒ
for ๐‘ฅ๐‘ฅ > bU
๐œ™๐œ™(zx )[ฮฆ(zb ) โˆ’ ฮฆ(zx )]
[ฮฆ(zb ) โˆ’ ฮฆ(za )]2
โ„Ž(๐‘ฅ๐‘ฅ) = 0
๐ป๐ป(๐‘ก๐‘ก) = โˆ’ ln[๐‘…๐‘…(๐‘ก๐‘ก)]
Left Truncated Normal
๐‘ฅ๐‘ฅ โˆˆ [0, โˆž)
General Truncated Normal
๐‘ฅ๐‘ฅ โˆˆ [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]
๐œ‡๐œ‡ ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐œ‡๐œ‡ โ‰ฅ 0
0 ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐œ‡๐œ‡ < 0
๐œ‡๐œ‡ ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐œ‡๐œ‡ โˆˆ [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]
๐‘Ž๐‘Ž๐ฟ๐ฟ ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐œ‡๐œ‡ < ๐‘Ž๐‘Ž๐ฟ๐ฟ
๐‘๐‘๐‘ˆ๐‘ˆ ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐œ‡๐œ‡ > ๐‘๐‘๐‘ˆ๐‘ˆ
No closed form
Mode
Mean
1st Raw Moment
where
Variance
2nd Central Moment
for ๐‘ฅ๐‘ฅ < aL
µ+
๐œŽ๐œŽ๐œŽ๐œŽ(z0 )
ฮฆ(โˆ’z0 )
โˆ’µ
๐‘ง๐‘ง0 =
ฯƒ
ฯƒ2 [1 โˆ’ {โˆ’ฮ”0 }2 โˆ’ ฮ”1 ]
where
No closed form
where
µ + ๐œŽ๐œŽ
๐‘ง๐‘ง๐‘Ž๐‘Ž =
where
๐œ™๐œ™(za ) โˆ’ ๐œ™๐œ™(zb )
ฮฆ(zb ) โˆ’ ฮฆ(za )
๐‘Ž๐‘Ž๐ฟ๐ฟ โˆ’ µ
,
ฯƒ
๐‘ง๐‘ง๐‘๐‘ =
๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ µ
ฯƒ
ฯƒ2 [1 โˆ’ {โˆ’ฮ”0 }2 โˆ’ ฮ”1 ]
Trunc Normal
for ๐‘ฅ๐‘ฅ < 0
Hazard Rate
โˆž
โˆซ ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘ฅ๐‘ฅ โˆซ๐‘ก๐‘ก ๐‘…๐‘…(๐‘ฅ๐‘ฅ)๐‘‘๐‘‘๐‘‘๐‘‘
๐‘ข๐‘ข(๐‘ก๐‘ก) = ๐‘ก๐‘ก
=
๐‘…๐‘…(๐‘ก๐‘ก)
๐‘…๐‘…(๐‘ก๐‘ก)
Mean Residual Life
138
Univariate Continuous Distributions
ฮ”๐‘˜๐‘˜ =
๐‘ง๐‘ง0๐‘˜๐‘˜ ๐œ™๐œ™(๐‘ง๐‘ง0 )
ฮฆ(๐‘ง๐‘ง0 ) โˆ’ 1
โˆ’1
Skewness
3rd Central Moment
3
๐‘‰๐‘‰ 2
where
ฮ”๐‘˜๐‘˜ =
๐‘ง๐‘ง๐‘๐‘๐‘˜๐‘˜ ๐œ™๐œ™(๐‘ง๐‘ง๐‘๐‘ ) โˆ’ ๐‘ง๐‘ง๐‘Ž๐‘Ž๐‘˜๐‘˜ ๐œ™๐œ™(๐‘ง๐‘ง๐‘Ž๐‘Ž )
ฮฆ(๐‘ง๐‘ง๐‘๐‘ ) โˆ’ ฮฆ(๐‘ง๐‘ง๐‘Ž๐‘Ž )
[2ฮ”30 + (3ฮ”1 โˆ’ 1)ฮ”0 + ฮ”2 ]
๐‘‰๐‘‰ = 1 โˆ’ ฮ”1 โˆ’ ฮ”20
1
[โˆ’3ฮ”40 โˆ’ 6ฮ”1 ฮ”20 โˆ’ 2ฮ”20 โˆ’ 4ฮ”2 ฮ”0 โˆ’ 3ฮ”1 โˆ’ ฮ”3 + 3]
๐‘‰๐‘‰ 2
Excess kurtosis
4th Central Moment
Characteristic
Function
See (Abadir & Magdalinos 2002, pp.1276-1287)
100๐›ผ๐›ผ% Percentile
Function
๐‘ก๐‘ก๐›ผ๐›ผ =
µ + ฯƒฮฆโˆ’1 {ฮฑ + ฮฆ(z0 )[1 โˆ’ ฮฑ]}
๐‘ก๐‘ก๐›ผ๐›ผ =
µ + ฯƒฮฆโˆ’1 {ฮฑฮฆ(zb ) + ฮฆ(za )[1 โˆ’ ฮฑ]}
Trunc Normal
Parameter Estimation
Maximum Likelihood Function
Likelihood
Function
For limits [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]:
๐ฟ๐ฟ(๐œ‡๐œ‡, ๐œŽ๐œŽ, ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ) =
1
nF
1 ๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 2
๏ฟฝ exp ๏ฟฝโˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐œŽ๐œŽ
i=1
๏ฟฝฯƒโˆš2ฯ€{ฮฆ(zb ) โˆ’ ฮฆ(za )}๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
nF
failures
For limits [0, โˆž)
๐‘›๐‘›๐น๐น
1
2
=
nF exp ๏ฟฝโˆ’
2 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡) ๏ฟฝ
2๐œŽ๐œŽ
๏ฟฝฯƒโˆš2ฯ€{ฮฆ(zb ) โˆ’ ฮฆ(za )}๏ฟฝ
๐‘–๐‘–=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
1
failures
๐ฟ๐ฟ(๐œ‡๐œ‡, ๐œŽ๐œŽ) =
nF
1 ๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ 2
๏ฟฝ exp ๏ฟฝโˆ’ ๏ฟฝ
๏ฟฝ ๏ฟฝ
2
๐œŽ๐œŽ
i=1
๏ฟฝฮฆ{โˆ’z
0 }ฯƒโˆš2ฯ€๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
1
1
nF
failures
๐‘›๐‘›๐น๐น
1
2
=
nF exp ๏ฟฝโˆ’
2 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡) ๏ฟฝ
2๐œŽ๐œŽ
๏ฟฝฮฆ{โˆ’z0 }ฯƒโˆš2ฯ€๏ฟฝ
๐‘–๐‘–=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
Log-Likelihood
Function
For limits [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]:
ฮ›(๐œ‡๐œ‡, ๐œŽ๐œŽ, ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ |๐ธ๐ธ)
๐‘›๐‘›๐น๐น
1
= โˆ’nF ln[ฮฆ(zb ) โˆ’ ฮฆ(za )] โˆ’ nF ln๏ฟฝฯƒโˆš2ฯ€๏ฟฝ โˆ’ 2 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
2๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
For limits [0, โˆž)
failures
Truncated Normal Continuous Distribution
139
๐‘›๐‘›๐น๐น
1
ฮ›(๐œ‡๐œ‡, ๐œŽ๐œŽ|๐ธ๐ธ) = โˆ’๐‘›๐‘›๐น๐น ln(๐›ท๐›ท{โˆ’๐‘ง๐‘ง0 }) โˆ’ ๐‘›๐‘›๐น๐น ln๏ฟฝ๐œŽ๐œŽโˆš2๐œ‹๐œ‹๏ฟฝ โˆ’ 2 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2
2๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
=0
โˆ‚µ
โˆ‚ฮ› โˆ’nF ฯ•(za ) โˆ’ ฯ•(zb )
1
=
๏ฟฝ
๏ฟฝ + 2 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡) = 0
)
)
โˆ‚µ
ฯƒ ฮฆ(zb โˆ’ ฮฆ(za
๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
failures
๐‘›๐‘›๐น๐น
โˆ‚ฮ›
=0
โˆ‚ฯƒ
โˆ‚ฮ› โˆ’nF za ฯ•(za ) โˆ’ zb ฯ•(zb )
nF 1
= 2 ๏ฟฝ
๏ฟฝ โˆ’ + 3 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡)2 = 0
โˆ‚ฯƒ
ฯƒ
ฮฆ(zb ) โˆ’ ฮฆ(za )
ฯƒ ๐œŽ๐œŽ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๐‘–๐‘–=1
MLE
Point
Estimates
First Estimate the values for ๐‘ง๐‘ง๐‘Ž๐‘Ž and ๐‘ง๐‘ง๐‘๐‘ by solving the simultaneous
equations numerically (Cohen 1991, p.33):
failures
Where:
๐ป๐ป2 (๐‘ง๐‘ง๐‘Ž๐‘Ž , ๐‘ง๐‘ง๐‘๐‘ ) =
๐‘„๐‘„๐‘Ž๐‘Ž =
๐‘„๐‘„๐‘Ž๐‘Ž โˆ’ ๐‘„๐‘„๐‘๐‘ โˆ’ ๐‘ง๐‘ง๐‘Ž๐‘Ž
๐‘ฅ๐‘ฅฬ… โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ
=
๐‘ง๐‘ง๐‘๐‘ โˆ’ ๐‘ง๐‘ง๐‘Ž๐‘Ž
๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ
1 + ๐‘ง๐‘ง๐‘Ž๐‘Ž ๐‘„๐‘„๐‘Ž๐‘Ž โˆ’ ๐‘ง๐‘ง๐‘๐‘ ๐‘„๐‘„๐‘๐‘ โˆ’ (๐‘„๐‘„๐‘Ž๐‘Ž โˆ’ ๐‘„๐‘„๐‘๐‘ )2
๐‘ ๐‘  2
=
2
(๐‘ง๐‘ง๐‘๐‘ โˆ’ ๐‘ง๐‘ง๐‘Ž๐‘Ž )
(๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ )2
๐œ™๐œ™(๐‘ง๐‘ง๐‘Ž๐‘Ž )
๐œ™๐œ™(๐‘ง๐‘ง๐‘๐‘ )
, ๐‘„๐‘„๐‘๐‘ =
ฮฆ(๐‘ง๐‘ง๐‘๐‘ ) โˆ’ ฮฆ(๐‘ง๐‘ง๐‘Ž๐‘Ž )
ฮฆ(๐‘ง๐‘ง๐‘๐‘ ) โˆ’ ฮฆ(๐‘ง๐‘ง๐‘Ž๐‘Ž )
๐‘ง๐‘ง๐‘Ž๐‘Ž =
๐‘›๐‘›๐น๐น
๐‘Ž๐‘Ž๐ฟ๐ฟ โˆ’ ๐œ‡๐œ‡
,
๐œŽ๐œŽ
๐‘ง๐‘ง๐‘๐‘ =
๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐œ‡๐œ‡
๐œŽ๐œŽ
๐‘›๐‘›๐น๐น
1
1
๐‘ฅ๐‘ฅฬ… = ๐น๐น ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– , ๐‘ ๐‘  2 =
๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐‘ฅ๐‘ฅฬ… )2
๐‘›๐‘›
๐‘›๐‘›๐น๐น โˆ’ 1
0
0
The distribution parameters can then be estimated using:
๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ
๐œŽ๐œŽ๏ฟฝ =
,
๐œ‡๐œ‡ฬ‚ = ๐‘Ž๐‘Ž๐ฟ๐ฟ โˆ’ ๐œŽ๐œŽ๏ฟฝ๐‘ง๐‘ง๏ฟฝ
๐‘Ž๐‘Ž
๐‘ง๐‘ง๐‘๐‘ โˆ’ ๐‘ง๐‘ง๏ฟฝ
๏ฟฝ
๐‘Ž๐‘Ž
(Cohen 1991, p.44) provides a graphical procedure to estimate
parameters to use as the starting point for numerical solvers.
For the case where the limits are [0, โˆž) first numerically solve for ๐‘ง๐‘ง0 :
1 โˆ’ Q0 (Q0 โˆ’ ๐‘ง๐‘ง0 ) ๐‘ ๐‘  2
=
(Q0 โˆ’ ๐‘ง๐‘ง0 )2
๐‘ฅ๐‘ฅฬ…
where
๐œ™๐œ™(๐‘ง๐‘ง0 )
๐‘„๐‘„0 =
1 โˆ’ ฮฆ(๐‘ง๐‘ง0 )
The distribution parameters can be estimated using:
Trunc Normal
๐ป๐ป1 (๐‘ง๐‘ง๐‘Ž๐‘Ž , ๐‘ง๐‘ง๐‘๐‘ ) =
140
Univariate Continuous Distributions
๐œŽ๐œŽ๏ฟฝ =
๐‘ฅ๐‘ฅฬ…
,
๐‘„๐‘„0 โˆ’ ๐‘ง๐‘ง๏ฟฝ0
๐œ‡๐œ‡ฬ‚ = โˆ’๐œŽ๐œŽ๏ฟฝ๐‘ง๐‘ง๏ฟฝ0
When the limits ๐‘Ž๐‘Ž๐ฟ๐ฟ and ๐‘๐‘๐‘ˆ๐‘ˆ are unknown, the likelihood function is
maximized when the difference, ฮฆ(zb ) โˆ’ ฮฆ(za ), is at its minimum. This
occurs when the difference between ๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ is at its minimum.
Therefore the MLE estimates for ๐‘Ž๐‘Ž๐ฟ๐ฟ and ๐‘๐‘๐‘ˆ๐‘ˆ are:
Fisher
Information
(Cohen 1991,
p.40)
a๏ฟฝL = min(t1F , t F2 โ€ฆ )
b๏ฟฝU = max(t1F , t F2 โ€ฆ )
๐ผ๐ผ(๐œ‡๐œ‡, ๐œŽ๐œŽ
Trunc Normal
Where
100๐›พ๐›พ%
Confidence
Intervals
2)
1
1 2(๐‘ฅ๐‘ฅฬ… โˆ’ ๐œ‡๐œ‡)
โŽก
โŽค
[1 โˆ’ ๐‘„๐‘„๐‘Ž๐‘Žโ€ฒ + ๐‘„๐‘„๐‘๐‘โ€ฒ ]
โˆ’ ๐œ†๐œ†๐‘Ž๐‘Ž + ๐œ†๐œ†๐‘๐‘ ๏ฟฝ
๏ฟฝ
๐œŽ๐œŽ 2
๐œŽ๐œŽ
โŽข ๐œŽ๐œŽ 2
โŽฅ
=โŽข
2
2
โŽฅ
1 2(๐‘ฅ๐‘ฅฬ… โˆ’ ๐œ‡๐œ‡)
1 3[๐‘ ๐‘  + (๐‘ฅ๐‘ฅฬ… โˆ’ ๐œ‡๐œ‡) ]
โŽข ๏ฟฝ
โˆ’ ๐œ†๐œ†๐‘Ž๐‘Ž + ๐œ†๐œ†๐‘๐‘ ๏ฟฝ
โˆ’ 1 โˆ’ ๐œ‚๐œ‚๐‘Ž๐‘Ž + ๐œ‚๐œ‚๐‘๐‘ ๏ฟฝโŽฅ
๏ฟฝ
๐œŽ๐œŽ
๐œŽ๐œŽ 2
๐œŽ๐œŽ 2
โŽฃ๐œŽ๐œŽ 2
โŽฆ
๐‘„๐‘„๐‘Ž๐‘Žโ€ฒ = ๐‘„๐‘„๐‘Ž๐‘Ž (๐‘„๐‘„๐‘Ž๐‘Ž โˆ’ ๐‘ง๐‘ง๐‘Ž๐‘Ž ),
๐œ†๐œ†๐‘Ž๐‘Ž = ๐‘Ž๐‘Ž๐ฟ๐ฟ ๐‘„๐‘„โ€ฒ๐‘Ž๐‘Ž + ๐‘„๐‘„๐‘Ž๐‘Ž ,
๐œ‚๐œ‚๐‘Ž๐‘Ž = ๐‘Ž๐‘Ž๐ฟ๐ฟ (๐œ†๐œ†๐‘Ž๐‘Ž + ๐‘„๐‘„๐‘Ž๐‘Ž ),
๐‘„๐‘„๐‘๐‘โ€ฒ = โˆ’๐‘„๐‘„๐‘๐‘ (๐‘„๐‘„๐‘๐‘ + ๐‘ง๐‘ง๐‘๐‘ )
๐œ†๐œ†๐‘๐‘ = ๐‘๐‘๐‘ˆ๐‘ˆ ๐‘„๐‘„โ€ฒ๐‘๐‘ + ๐‘„๐‘„๐‘๐‘
๐œ‚๐œ‚๐‘๐‘ = ๐‘๐‘๐‘ˆ๐‘ˆ (๐œ†๐œ†๐‘๐‘ + ๐‘„๐‘„๐‘๐‘ )
Calculated from the Fisher information matrix. See section 1.4.7. For
further detail and examples see (Cohen 1991, p.41)
Bayesian
No closed form solutions to priors exist.
Description , Limitations and Uses
Example 1
The size of washers delivered from a manufacturer is desired to be
modeled. The manufacture has already removed all washers below
15.95mm and washers above 16.05mm. The washers received have
the following diameters:
15.976, 15.970, 15.955, 16.007, 15.966, 15.952, 15.955 ๐‘š๐‘š๐‘š๐‘š
From data:
๐‘ฅ๐‘ฅฬ… = 15.973,
๐‘ ๐‘  2 = 4.3950๐ธ๐ธ-4
Using numerical solver MLE Estimates for ๐‘ง๐‘ง๐‘Ž๐‘Ž and ๐‘ง๐‘ง๐‘๐‘ are:
Therefore
๐‘ง๐‘ง๏ฟฝ
๐‘Ž๐‘Ž = 0,
๐œŽ๐œŽ๏ฟฝ =
๏ฟฝ
๐‘ง๐‘ง๐‘๐‘ = 3.3351
๐‘๐‘๐‘ˆ๐‘ˆ โˆ’ ๐‘Ž๐‘Ž๐ฟ๐ฟ
= 0.029984
๐‘ง๐‘ง๐‘๐‘ โˆ’ ๐‘ง๐‘ง๏ฟฝ
๏ฟฝ
๐‘Ž๐‘Ž
๐œ‡๐œ‡ฬ‚ = ๐‘Ž๐‘Ž๐ฟ๐ฟ โˆ’ ๐œŽ๐œŽ๏ฟฝ๐‘ง๐‘ง๏ฟฝ
๐‘Ž๐‘Ž = 15.95
To calculate confidence intervals, first calculate:
Truncated Normal Continuous Distribution
๐‘„๐‘„๐‘Ž๐‘Žโ€ฒ = 0.63771,
๐œ†๐œ†๐‘Ž๐‘Ž = 10.970,
๐œ‚๐œ‚๐‘Ž๐‘Ž = 187.71,
90% confidence intervals:
141
๐‘„๐‘„๐‘๐‘โ€ฒ = โˆ’0.010246
๐œ†๐œ†๐‘๐‘ = โˆ’0.16138
๐œ‚๐œ‚๐‘๐‘ = โˆ’2.54087
391.57
โˆ’10699
๐ผ๐ผ(๐œ‡๐œ‡, ๐œŽ๐œŽ) = ๏ฟฝ
๏ฟฝ
โˆ’10699 โˆ’209183
[๐ฝ๐ฝ๐‘›๐‘› (๐œ‡๐œ‡ฬ‚ , ๐œŽ๐œŽ๏ฟฝ)]โˆ’1 = [๐‘›๐‘›๐น๐น ๐ผ๐ผ(๐œ‡๐œ‡ฬ‚ , ๐œŽ๐œŽ๏ฟฝ)]โˆ’1 = ๏ฟฝ 1.1835๐ธ๐ธ-4 โˆ’6.0535๐ธ๐ธ-6๏ฟฝ
โˆ’6.0535๐ธ๐ธ-6 โˆ’2.2154๐ธ๐ธ-7
90% confidence interval for ๐œ‡๐œ‡:
๏ฟฝ๐œ‡๐œ‡ฬ‚ โˆ’ ฮฆโˆ’1 (0.95)โˆš1.1835๐ธ๐ธ-4,
๐œ‡๐œ‡ฬ‚ + ฮฆโˆ’1 (0.95)โˆš1.1835๐ธ๐ธ-4๏ฟฝ
[15.932, 15.968]
๐›ท๐›ท โˆ’1 (0.95)โˆš2.2154๐ธ๐ธ-7
๐œŽ๐œŽ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
3.0769๐ธ๐ธ-2]
An estimate can be made on how many washers the manufacturer
discards:
The distribution of washer sizes is a Normal Distribution with
estimated parameters ๐œ‡๐œ‡ฬ‚ = 15.95, ๐œŽ๐œŽ๏ฟฝ = 0.029984. The percentage of
washers wish pass quality control is:
๐น๐น(16.05) โˆ’ ๐น๐น(15.95) = 49.96%
It is likely that there is too much variance in the manufacturing
process for this system to be efficient.
Example 2
The following example adjusts the calculations used in the Normal
Distribution to account for the fact that the limit on distance is [0, โˆž).
The accuracy of a cutting machine used in manufacturing is desired
to be measured. 5 cuts at the required length are made and
measured as:
7.436, 10.270, 10.466, 11.039, 11.854 ๐‘š๐‘š๐‘š๐‘š
From data:
๐‘ฅ๐‘ฅฬ… = 10.213,
๐‘ ๐‘  2 = 2.789
Using numerical solver MLE Estimates for ๐‘ง๐‘ง0 is:
Therefore
๐œŽ๐œŽ๏ฟฝ =
๐‘ง๐‘ง๏ฟฝ0 = โˆ’4.5062
๐‘ฅ๐‘ฅฬ…
= 2.26643
๐‘„๐‘„0 โˆ’ ๐‘ง๐‘ง๏ฟฝ0
Trunc Normal
90% confidence interval for ๐œŽ๐œŽ:
๐›ท๐›ทโˆ’1 (0.95)โˆš2.2154๐ธ๐ธ-7
๏ฟฝ๐œŽ๐œŽ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ,
โˆ’๐œŽ๐œŽ๏ฟฝ
[2.922๐ธ๐ธ-2,
142
Univariate Continuous Distributions
๐œ‡๐œ‡ฬ‚ = โˆ’๐œŽ๐œŽ๏ฟฝ๐‘ง๐‘ง๏ฟฝ
๐‘Ž๐‘Ž = 10.213
To calculate confidence intervals, first calculate:
๐œ†๐œ†0 = 1.5543๐ธ๐ธ-5,
๐œ†๐œ†๐‘๐‘ = โˆ’0.16138
๐‘„๐‘„0โ€ฒ = 7.0042๐ธ๐ธ-5,
90% confidence intervals:
0.19466
โˆ’2.9453๐ธ๐ธ-6
๐ผ๐ผ(๐œ‡๐œ‡, ๐œŽ๐œŽ) = ๏ฟฝ
๏ฟฝ
โˆ’2.9453๐ธ๐ธ-6
0.12237
2.4728๐ธ๐ธ-5
[๐ฝ๐ฝ๐‘›๐‘› (๐œ‡๐œ‡ฬ‚ , ๐œŽ๐œŽ๏ฟฝ)]โˆ’1 = [๐‘›๐‘›๐น๐น ๐ผ๐ผ(๐œ‡๐œ‡ฬ‚ , ๐œŽ๐œŽ๏ฟฝ)]โˆ’1 = ๏ฟฝ 1.0274
๏ฟฝ
2.4728๐ธ๐ธ-5
1.6343
90% confidence interval for ๐œ‡๐œ‡:
๐œ‡๐œ‡ฬ‚ + ฮฆโˆ’1 (0.95)โˆš1.0274๏ฟฝ
๏ฟฝ๐œ‡๐œ‡ฬ‚ โˆ’ ฮฆโˆ’1 (0.95)โˆš1.0274,
[8.546, 11.88]
Trunc Normal
90% confidence interval for ๐œŽ๐œŽ:
๐›ท๐›ทโˆ’1 (0.95)โˆš1.6343
๏ฟฝ๐œŽ๐œŽ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ,
โˆ’๐œŽ๐œŽ๏ฟฝ
[0.8962,
๐›ท๐›ทโˆ’1 (0.95)โˆš1.6343
๐œŽ๐œŽ๏ฟฝ. exp ๏ฟฝ
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ๏ฟฝ
5.732]
To compare these results to a non-truncated normal distribution:
90% Lower CI
Point Est
90% Upper CI
Norm - ๐œ‡๐œ‡
10.163
10.213
10.262
Classical
1.176
2.789
15.697
Norm - ๐œŽ๐œŽ 2
Classical
Norm - ๐œ‡๐œ‡
8.761
10.213
11.665
Bayesian
0.886
2.789
6.822
Norm - ๐œŽ๐œŽ 2
Bayesian
TNorm - ๐œ‡๐œ‡
8.546
10.213
11.88
0.80317
5.1367
32.856
TNorm - ๐œŽ๐œŽ 2
*Note: The TNorm ๐œŽ๐œŽ estimate and interval are squared.
The truncated normal produced results which had a wider confidence
in the parameter estimates, however the point estimates were within
each others confidence intervals. In this case the truncation
correction might be ignored for ease of calculation.
Characteristics
For large ๐œ‡๐œ‡/๐œŽ๐œŽ truncation may have negligible affect. In this case the
use the Normal Continuous Distribution as an approximation.
Let:
X~๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡(µ, ฯƒ2 ) where ๐‘‹๐‘‹ โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
Convolution Property. The sum of truncated normal distribution
random variables is not a truncated normal distribution. When
truncation is symmetrical about the mean the sum of truncated
normal distribution random variables is well approximated using:
Truncated Normal Continuous Distribution
๐‘›๐‘›
๐‘Œ๐‘Œ = ๏ฟฝ ๐‘‹๐‘‹๐‘–๐‘– where
๐‘–๐‘–=1
๐‘Œ๐‘Œ โ‰ˆ ๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡๐‘‡ ๏ฟฝ๏ฟฝ ๐œ‡๐œ‡๐‘–๐‘– , ๏ฟฝ ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐‘‹๐‘‹๐‘–๐‘– )๏ฟฝ
143
bi โˆ’ ai
= µi
2
where ๐‘Œ๐‘Œ โˆˆ [ โˆ‘๐‘Ž๐‘Ž๐‘–๐‘– , โˆ‘๐‘๐‘๐‘–๐‘– ]
Linear Transformation Property (Cozman & Krotkov 1997)
Applications
๐‘Œ๐‘Œ = ๐‘๐‘๐‘๐‘ + ๐‘‘๐‘‘
๐‘Œ๐‘Œ~๐‘‡๐‘‡๐‘๐‘๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ๐‘œ(๐‘๐‘๐‘๐‘ + ๐‘‘๐‘‘, ๐‘‘๐‘‘2 ๐œŽ๐œŽ 2 ) where ๐‘Œ๐‘Œ โˆˆ [๐‘๐‘๐‘๐‘ + ๐‘‘๐‘‘, ๐‘๐‘๐‘๐‘ + ๐‘‘๐‘‘]
Life Distribution. When used as a life distribution a truncated
Normal Distribution may be used due to the constraint tโ‰ฅ0. However
it is often found that the difference in results is negligible. (Rausand
& Høyland 2004)
Failures After Pre-test Screening. When a customer receives a
product from a vendor, the product may have already been subject
to burn-in testing. The customer will not know the number of failures
which occurred during the burn-in, but may know the duration. As
such the failure distribution is left truncated. (Meeker & Escobar
1998, p.269)
Flaws under the inspection threshold. When a flaw is not
detected due to the flawโ€™s amplitude being less than the inspection
threshold the distribution is left truncated. (Meeker & Escobar 1998,
p.266)
Worst Case Measurements. Sometimes only the worst performers
from a population are monitored and have data collected. Therefore
the threshold which determined that the item be monitored is the
truncation limit. (Meeker & Escobar 1998, p.267)
Screening Out Units With Large Defects. In quality control
processes it may be common to remove defects which exceed a limit.
The remaining population of defects delivered to the customer has a
right truncated distribution. (Meeker & Escobar 1998, p.270)
Resources
Online:
http://en.wikipedia.org/wiki/Truncated_normal_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
http://www.ntrand.com/truncated-normal-distribution/
Books:
Cohen, 1991. Truncated and Censored Samples 1st ed., CRC
Trunc Normal
Repair Time Distributions. The truncated normal distribution may
be used to model simple repair or inspection tasks that have a
typical duration with little variation using the limits [0, โˆž)
144
Univariate Continuous Distributions
Press.
Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution
2nd ed., CRC.
Schneider, H., 1986. Truncated and censored samples from normal
populations, M. Dekker.
Relationship to Other Distributions
Normal
Distribution
Let:
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ฅ๐‘ฅ; ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
Then:
๐‘‹๐‘‹~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
๐‘‹๐‘‹ โˆˆ (โˆž, โˆž)
๐‘Œ๐‘Œ~T๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡, ๐œŽ๐œŽ 2 , ๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ )
๐‘Œ๐‘Œ โˆˆ [๐‘Ž๐‘Ž๐ฟ๐ฟ , ๐‘๐‘๐‘ˆ๐‘ˆ ]
Trunc Normal
For further relationships see Normal Continuous Distribution
Uniform Continuous Distribution
145
4.9. Uniform Continuous
Distribution
Probability Density Function - f(t)
a=0 b=1
a=0.75 b=1.25
a=0.25 b=1.5
2.00
1.50
1.00
0.50
0.00
0
0.5
1
1.5
1
1.5
1
1.5
Uniform Cont
Cumulative Density Function - F(t)
a=0 b=1
a=0.75 b=1.25
a=0.25 b=1.5
1.00
0.80
0.60
0.40
0.20
0.00
0
0.5
Hazard Rate - h(t)
10.00
9.00
8.00
7.00
6.00
5.00
4.00
3.00
2.00
1.00
0.00
a=0 b=1
a=0.75 b=1.25
a=0.25 b=1.5
0
0.5
146
Univariate Continuous Distributions
Parameters & Description
Parameters
Random Variable
๐‘Ž๐‘Ž
๐‘๐‘
Distribution
Minimum Value. ๐‘Ž๐‘Ž is the lower bound
of the uniform distribution.
0 โ‰ค ๐‘Ž๐‘Ž < ๐‘๐‘
Maximum Value. ๐‘๐‘ is the upper bound
of the uniform distribution.
๐‘Ž๐‘Ž < ๐‘๐‘ < โˆž
Time Domain
๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘
1
๐‘“๐‘“(๐‘ก๐‘ก) = ๏ฟฝb โˆ’ a for a โ‰ค t โ‰ค b
0
otherwise
1
=
{๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘Ž๐‘Ž) โˆ’ ๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘๐‘)}
bโˆ’a
PDF
Uniform Cont
Where ๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘Ž๐‘Ž) is the Heaviside
step function.
0
for t < ๐‘Ž๐‘Ž
tโˆ’a
๐น๐น(๐‘ก๐‘ก) = ๏ฟฝ
for a โ‰ค t โ‰ค b
bโˆ’a
1
for t > ๐‘๐‘
tโˆ’a
{๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘Ž๐‘Ž) โˆ’ ๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘๐‘)}
=
bโˆ’a
+ ๐‘ข๐‘ข(๐‘ก๐‘ก โˆ’ ๐‘๐‘)
CDF
1
for t < ๐‘Ž๐‘Ž
bโˆ’t
๐‘…๐‘…(๐‘ก๐‘ก) = ๏ฟฝ
for a โ‰ค t โ‰ค b
bโˆ’a
0
for t > ๐‘๐‘
Reliability
Laplace
๐‘“๐‘“(๐‘ ๐‘ ) =
๐‘’๐‘’ โˆ’๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘’๐‘’ โˆ’๐‘๐‘๐‘๐‘
๐‘ ๐‘ (๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)
๐น๐น(๐‘ ๐‘ ) =
๐‘’๐‘’ โˆ’๐‘Ž๐‘Ž๐‘Ž๐‘Ž โˆ’ ๐‘’๐‘’ โˆ’๐‘๐‘๐‘๐‘
๐‘ ๐‘  2 (๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)
๐‘…๐‘…(๐‘ ๐‘ ) =
For ๐‘ก๐‘ก < ๐‘Ž๐‘Ž:
Conditional
Survivor Function
๐‘ƒ๐‘ƒ(๐‘‡๐‘‡ > ๐‘ฅ๐‘ฅ + ๐‘ก๐‘ก|๐‘‡๐‘‡ > ๐‘ก๐‘ก)
Mean
Life
Residual
๐‘’๐‘’ โˆ’๐‘๐‘๐‘๐‘ โˆ’ ๐‘’๐‘’ โˆ’๐‘Ž๐‘Ž๐‘Ž๐‘Ž 1
+
๐‘ ๐‘  2 (๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž)
๐‘ ๐‘ 
1
for t + x < ๐‘Ž๐‘Ž
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
b โˆ’ (t + x)
๐‘š๐‘š(๐‘ฅ๐‘ฅ) =
=๏ฟฝ
for a โ‰ค t + x โ‰ค b
๐‘…๐‘…(๐‘ก๐‘ก)
bโˆ’a
0
for t > ๐‘๐‘
For ๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘:
1
for t + x < ๐‘Ž๐‘Ž
๐‘…๐‘…(๐‘ก๐‘ก + ๐‘ฅ๐‘ฅ)
b โˆ’ (t + x)
๐‘š๐‘š(๐‘ฅ๐‘ฅ) =
=๏ฟฝ
for a โ‰ค t + x โ‰ค b
๐‘…๐‘…(๐‘ก๐‘ก)
bโˆ’t
0
for t + x > ๐‘๐‘
For ๐‘ก๐‘ก > ๐‘๐‘:
๐‘š๐‘š(๐‘ฅ๐‘ฅ) = 0
Where
๐‘ก๐‘ก is the given time we know the component has survived to.
๐‘ฅ๐‘ฅ is a random variable defined as the time after ๐‘ก๐‘ก. Note: ๐‘ฅ๐‘ฅ = 0 at ๐‘ก๐‘ก.
For ๐‘ก๐‘ก < ๐‘Ž๐‘Ž:
For ๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘:
1
๐‘ข๐‘ข(๐‘ก๐‘ก) = 2(๐‘Ž๐‘Ž + ๐‘๐‘) โˆ’ ๐‘ก๐‘ก
๐‘ข๐‘ข(๐‘ก๐‘ก) = ๐‘Ž๐‘Ž โˆ’ ๐‘ก๐‘ก โˆ’
(๐‘Ž๐‘Ž โˆ’ ๐‘๐‘)2
2(๐‘ก๐‘ก โˆ’ ๐‘๐‘)
Uniform Continuous Distribution
For ๐‘ก๐‘ก > ๐‘๐‘:
147
๐‘ข๐‘ข(๐‘ก๐‘ก) = 0
1
โ„Ž(๐‘ก๐‘ก) = ๏ฟฝb โˆ’ t for a โ‰ค t โ‰ค b
0
otherwise
Hazard Rate
0
for t < ๐‘Ž๐‘Ž
bโˆ’t
๐ป๐ป(๐‘ก๐‘ก) = ๏ฟฝ โˆ’ ln ๏ฟฝ
๏ฟฝ for a โ‰ค t โ‰ค b
bโˆ’a
โˆž
for t > ๐‘๐‘
Cumulative
Hazard Rate
Properties and Moments
Median
Mean - 1st Raw Moment
1
(๐‘Ž๐‘Ž
2
+ ๐‘๐‘)
1
(๐‘๐‘
12
Skewness - 3rd Central Moment
โˆ’ ๐‘Ž๐‘Ž)2
0
6
โˆ’5
Central Moment
eitb โˆ’ eita
it(b โˆ’ a)
Characteristic Function
100ฮฑ% Percentile Function
Parameter Estimation
๐‘ก๐‘ก๐›ผ๐›ผ = ฮฑ(b โˆ’ a) + a
Maximum Likelihood Function
Likelihood
Functions
LI
nS
nI
1 nF
b โˆ’ t Si
t RI
i โˆ’ ti
๐ฟ๐ฟ(๐‘Ž๐‘Ž, ๐‘๐‘|๐ธ๐ธ) = ๏ฟฝ
๏ฟฝ .๏ฟฝ ๏ฟฝ
๏ฟฝ . ๏ฟฝ ๏ฟฝ1 +
๏ฟฝ
๏ฟฝ๏ฟฝ
b๏ฟฝ๏ฟฝ๏ฟฝ
โˆ’ a๏ฟฝ๏ฟฝ ๏ฟฝ๏ฟฝ๏ฟฝi=1
b๏ฟฝโˆ’๏ฟฝ๏ฟฝ๏ฟฝ
a ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
bโˆ’a
i=1
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
failures
survivors
interval failures
This assumes that all times are within the bound a, b.
Point
Estimates
When there is only complete failure data:
1 nF
๐ฟ๐ฟ(๐‘Ž๐‘Ž, ๐‘๐‘|๐ธ๐ธ) = ๏ฟฝ
๏ฟฝ
bโˆ’a
where
๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก๐‘–๐‘– โ‰ค ๐‘๐‘
The likelihood function is maximized when a is large, b is small with the
restriction that all times are between a and b. Thus:
a๏ฟฝ = min(t1F , t F2 โ€ฆ )
b๏ฟฝ = max(t1F , t F2 โ€ฆ )
Uniform Cont
Variance - 2nd Central Moment
Excess kurtosis -
+ ๐‘๐‘)
Any value between ๐‘Ž๐‘Ž and ๐‘๐‘
Mode
4th
1
(๐‘Ž๐‘Ž
2
148
Univariate Continuous Distributions
When ๐‘Ž๐‘Ž = 0 and ๐‘๐‘ is estimated with complete data the following
estimates may be used where ๐‘ก๐‘ก๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š = max(t1F , t F2 โ€ฆ t Fn ). (Johnson et al.
1995, p.286)
1. MLE.
b๏ฟฝ = ๐‘ก๐‘ก๐‘š๐‘š๐‘š๐‘š๐‘ฅ๐‘ฅ
n+2
2. Min Mean Square Error.
b๏ฟฝ =
๐‘ก๐‘ก
n+1 ๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š
n+1
3. Unbiased Estimator.
b๏ฟฝ =
๐‘ก๐‘ก๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š
4.
n
1
b๏ฟฝ = 2 ๏ฟฝn ๐‘ก๐‘ก๐‘š๐‘š๐‘š๐‘š๐‘š๐‘š
Closest Estimator.
Procedures for parameter estimating when there is censored data is
detailed in (Johnson et al. 1995, p.286)
โˆ’1
โŽก
(๐‘Ž๐‘Ž
โˆ’ ๐‘๐‘)2
๐ผ๐ผ(๐‘Ž๐‘Ž, ๐‘๐‘) = โŽข
1
โŽข
(๐‘Ž๐‘Ž
โŽฃ โˆ’ ๐‘๐‘)2
Fisher
Information
Uniform Cont
Bayesian
1
โŽค
(๐‘Ž๐‘Ž โˆ’ ๐‘๐‘)2 โŽฅ
โˆ’1 โŽฅ
(๐‘Ž๐‘Ž โˆ’ ๐‘๐‘)2 โŽฆ
The Uniform distribution is widely used in Bayesian methods as a non-informative prior or
to model evidence which only suggests bounds on the parameter.
Non-informative Prior. The Uniform distribution can be used as a non-informative prior.
As can be seen below, the only affect the uniform prior has on Bayes equation is to limit
the range of the parameter for which the denominator integrates over.
๐œ‹๐œ‹(๐œƒ๐œƒ|๐ธ๐ธ) =
1
๏ฟฝ
๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ)
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
= ๐‘๐‘
๐‘๐‘
1
โˆซ๐‘Ž๐‘Ž ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ) ๏ฟฝ๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž ๏ฟฝ ๐‘‘๐‘‘๐‘‘๐‘‘ โˆซ๐‘Ž๐‘Ž ๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ) ๐‘‘๐‘‘๐‘‘๐‘‘
๐ฟ๐ฟ(๐ธ๐ธ|๐œƒ๐œƒ) ๏ฟฝ
Parameter Bounds. This type of distribution allows an easy method to mathematically
model soft data where only the parameter bounds can be estimated. An example is where
uniform distribution can model a personโ€™s opinion on the value ๐œƒ๐œƒ where they know that it
could not be lower than ๐‘Ž๐‘Ž or greater than ๐‘๐‘, but is unsure of any particular value ๐œƒ๐œƒ could
take.
Non-informative Priors
Jeffreyโ€™s Prior
1
๐‘Ž๐‘Ž โˆ’ ๐‘๐‘
Description , Limitations and Uses
Example
For an example of the uniform distribution being used in Bayesian
updating as a prior, ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1,1) see the binomial distribution.
Given the following data calculate the MLE parameter estimates:
240, 585, 223, 751, 255
a๏ฟฝ = 223
Uniform Continuous Distribution
Characteristics
149
b๏ฟฝ = 751
The Uniform distribution is a special case of the Beta distribution
when ๐›ผ๐›ผ = ๐›ฝ๐›ฝ = 1.
The uniform distribution has an increasing failure rate with lim โ„Ž(๐‘ก๐‘ก) =
๐‘ก๐‘กโ†’๐‘๐‘
โˆž.
The Standard Uniform Distribution has parameters ๐‘Ž๐‘Ž = 0 and ๐‘๐‘ = 1.
This results in ๐‘“๐‘“(๐‘ก๐‘ก) = 1 for ๐‘Ž๐‘Ž โ‰ค ๐‘ก๐‘ก โ‰ค ๐‘๐‘ and 0 otherwise.
Uniformity Property
If ๐‘ก๐‘ก > ๐‘Ž๐‘Ž and ๐‘ก๐‘ก + ๐›ฅ๐›ฅ < ๐‘๐‘ then:
๐‘‡๐‘‡~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘Ž๐‘Ž, ๐‘๐‘)
๐‘ก๐‘ก+โˆ†
Variate Generation Property
๐น๐น โˆ’1 (๐‘ข๐‘ข) = ๐‘ข๐‘ข(๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž) + ๐‘Ž๐‘Ž
Applications
Residual Property
If k is a real constant where ๐‘Ž๐‘Ž < ๐‘˜๐‘˜ < ๐‘๐‘ then:
Pr(๐‘‡๐‘‡|๐‘‡๐‘‡ > ๐‘˜๐‘˜)~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘Ž๐‘Ž = ๐‘˜๐‘˜, ๐‘๐‘)
Random Number Generator. The uniform distribution is widely
used as the basis for the generation of random numbers for other
statistical distributions. The random uniform values are mapped to
the desired distribution by solving the inverse cdf.
Bayesian Inference. The uniform distribution can be used ss a noninformative prior and to model soft evidence.
Resources
Special Case of Beta Distribution. In applications like Bayesian
statistics the uniform distribution is used as an uninformative prior by
using a beta distribution of ๐›ผ๐›ผ = ๐›ฝ๐›ฝ = 1.
Online:
http://mathworld.wolfram.com/UniformDistribution.html
http://en.wikipedia.org/wiki/Uniform_distribution_(continuous)
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous
Univariate Distributions, Vol. 2 2nd ed., Wiley-Interscience.
Relationship to Other Distributions
Beta Distribution
Let
Uniform Cont
โˆ†
1
๐‘‘๐‘‘๐‘‘๐‘‘ =
๐‘๐‘
โˆ’
๐‘Ž๐‘Ž
๐‘๐‘
โˆ’
๐‘Ž๐‘Ž
๐‘ก๐‘ก
The probability that a random variable falls within any interval of fixed
length is independent of the location, ๐‘ก๐‘ก, and is only dependent on the
interval size, ๐›ฅ๐›ฅ.
๐‘ƒ๐‘ƒ(๐‘ก๐‘ก โ†’ ๐‘ก๐‘ก + โˆ†) = ๏ฟฝ
150
Univariate Continuous Distributions
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ฮฑ, ฮฒ, a, b)
Exponential
Distribution
Uniform Cont
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
Then
Xi ~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(0,1)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
X1 โ‰ค X2 โ‰ค โ‹ฏ โ‰ค Xn
Xr ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘Ÿ๐‘Ÿ, ๐‘›๐‘› โˆ’ ๐‘Ÿ๐‘Ÿ + 1)
Where ๐‘›๐‘› and ๐‘˜๐‘˜ are integers.
Special Case:
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ก๐‘ก; ๐‘Ž๐‘Ž, ๐‘๐‘|๐›ผ๐›ผ = 1, ฮฒ = 1 ) = ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(๐‘ก๐‘ก; ๐‘Ž๐‘Ž, ๐‘๐‘)
Let
Then
๐‘‹๐‘‹~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Y = exp(โˆ’๐œ†๐œ†๐œ†๐œ†)
๐‘Œ๐‘Œ~๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ๐‘ˆ(0,1)
Discrete Distributions
151
5. Univariate Discrete
Distributions
Univar Discrete
152
Discrete Distributions
5.1. Bernoulli Discrete Distribution
Probability Density Function - f(k)
0.70
0.70
0.60
0.60
p=0.3
0.50
0.50
0.40
0.40
0.30
0.30
0.20
0.20
0.10
0.10
0.00
0.00
0
1
2
p=0.55
0
1
2
Cumulative Density Function - F(k)
1.00
1.00
0.80
0.80
Bernoulli
0.60
p=0.3
0.60
0.40
0.40
0.20
0.20
0.00
0.00
0
1
2
p=0.55
0
1
2
Hazard Rate - h(k)
1.00
1.00
0.80
0.80
0.60
p=0.3
0.60
0.40
0.40
0.20
0.20
0.00
p=0.55
0.00
0
1
2
0
1
2
Bernoulli Discrete Distribution
153
Parameters & Description
๐‘๐‘
Parameters
Random Variable
Question
Distribution
0 โ‰ค ๐‘๐‘ โ‰ค 1
Bernoulli
probability
Probability of success.
๐‘˜๐‘˜ โˆˆ {0, 1}
The probability of getting exactly ๐‘˜๐‘˜ (0 or 1) successes in 1 trial with
probability p.
Formulas
๐‘“๐‘“(๐‘˜๐‘˜) = pk (1 โˆ’ p)1โˆ’k
1 โˆ’ p for k = 0
=๏ฟฝ
p
for k = 1
PDF
๐น๐น(๐‘˜๐‘˜) = (1 โˆ’ p)1โˆ’k
1 โˆ’ p for k = 0
=๏ฟฝ
1
for k = 1
CDF
๐‘…๐‘…(๐‘˜๐‘˜) = 1 โˆ’ (1 โˆ’ p)1โˆ’k
p for k = 0
=๏ฟฝ
0 for k = 1
Reliability
โ„Ž(๐‘˜๐‘˜) = ๏ฟฝ
Hazard Rate
1โˆ’p
1
Properties and Moments
Mean - 1st Raw Moment
Central Moment
qโˆ’p
Skewness - 3rd Central Moment
๏ฟฝpq
Excess kurtosis - 4th Central Moment
Characteristic Function
Parameter Estimation
๐‘๐‘
๐‘๐‘(1 โˆ’ ๐‘๐‘)
๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’ ๐‘ž๐‘ž = (1 โˆ’ ๐‘๐‘)
6p2 โˆ’ 6p + 1
p(1 โˆ’ p)
(1 โˆ’ p) + peit
Maximum Likelihood Function
Likelihood
Function
๐ฟ๐ฟ(๐‘๐‘|๐ธ๐ธ) = ๐‘๐‘โˆ‘๐‘˜๐‘˜๐‘–๐‘– (1 โˆ’ ๐‘๐‘)๐‘›๐‘›โˆ’โˆ‘๐‘˜๐‘˜๐‘–๐‘–
where ๐‘›๐‘› is the number of Bernoulli trials ๐‘˜๐‘˜๐‘–๐‘– โˆˆ {0,1}, and โˆ‘๐‘˜๐‘˜๐‘–๐‘– = โˆ‘๐‘›๐‘›๐‘–๐‘–=1 ๐‘˜๐‘˜๐‘–๐‘–
Bernoulli
2nd
for k = 0
for k = 1
๐‘˜๐‘˜0.5 = โ€–๐‘๐‘โ€– ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ ๐‘๐‘ โ‰  0.5
๐‘˜๐‘˜0.5 = {0,1} ๐‘ค๐‘คโ„Ž๐‘’๐‘’๐‘’๐‘’ ๐‘๐‘ = 0.5
Mode
Variance -
parameter.
154
Discrete Distributions
๐‘‘๐‘‘L
=0
๐‘‘๐‘‘p
solve for ๐‘๐‘
๐‘‘๐‘‘L
= โˆ‘k. p โˆ‘(ki)โˆ’1 (1 โˆ’ p)nโˆ’โˆ‘ki โˆ’ (n โˆ’ โˆ‘k)pโˆ‘ki (1 โˆ’ p)nโˆ’1โˆ’โˆ‘ki = 0
๐‘‘๐‘‘p
โˆ‘k. pโˆ‘(ki)โˆ’1 (1 โˆ’ p)nโˆ’โˆ‘ki = (n โˆ’ โˆ‘k i )pโˆ‘ki (1 โˆ’ p)nโˆ’1โˆ’โˆ‘ki
โˆ‘k i . pโˆ’1 = (n โˆ’ โˆ‘k i )(1 โˆ’ p)โˆ’1
(1 โˆ’ p) n โˆ’ โˆ‘k i
=
โˆ‘k i
p
p=
Fisher
Information
MLE
Point
Estimates
โˆ‘k i
n
๐ผ๐ผ(๐‘๐‘) =
The MLE point estimate for p:
p๏ฟฝ =
Fisher
Information
Confidence
Intervals
1
๐‘๐‘(1 โˆ’ ๐‘๐‘)
๐ผ๐ผ(๐‘๐‘) =
โˆ‘k
n
1
๐‘๐‘(1 โˆ’ ๐‘๐‘)
See discussion in binomial distribution.
Bernoulli
Bayesian
Type
Non-informative Priors for p, ๐…๐…(๐’‘๐’‘)
(Yang and Berger 1998, p.6)
Prior
Posterior
Uniform
Proper
Prior with limits
๐‘๐‘ โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Improper
Proir with limits
๐‘๐‘ โˆˆ [0,1]
1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 1,1)
Jeffreyโ€™s Prior
Reference Prior
MDIP
Novick and Hall
1
1 1
= ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐‘๐‘; , ๏ฟฝ
2 2
๏ฟฝ๐‘๐‘(1 โˆ’ ๐‘๐‘)
1.6186๐‘๐‘๐‘๐‘ (1 โˆ’ ๐‘๐‘)1โˆ’๐‘๐‘
๐‘๐‘โˆ’1 (1
โˆ’
๐‘๐‘)โˆ’1
= ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(0,0)
Truncated Beta Distribution
For a โ‰ค ๐‘๐‘ โ‰ค b
๐‘๐‘. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 1 + ๐‘˜๐‘˜, 2 โˆ’ ๐‘˜๐‘˜)
Otherwise ๐œ‹๐œ‹(๐‘๐‘) = 0
๐ต๐ต๐‘’๐‘’๐‘ก๐‘ก๐‘ก๐‘ก(๐‘๐‘; 1 + ๐‘˜๐‘˜, 2 โˆ’ ๐‘˜๐‘˜)
1
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐‘๐‘; + ๐‘˜๐‘˜, 1.5 โˆ’ ๐‘˜๐‘˜๏ฟฝ
2
when ๐‘๐‘ โˆˆ [0,1]
Proper - No Closed Form
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; ๐‘˜๐‘˜, 1 โˆ’ ๐‘˜๐‘˜)
when ๐‘๐‘ โˆˆ [0,1]
Bernoulli Discrete Distribution
155
Conjugate Priors
UOI
๐‘๐‘
from
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘)
Example
Likelihood
Model
Bernoulli
Evidence
๐‘˜๐‘˜ failures
in 1 trail
Dist of
UOI
Prior
Para
Posterior
Parameters
Beta
๐›ผ๐›ผ0 , ๐›ฝ๐›ฝ0
๐›ผ๐›ผ = ๐›ผ๐›ผ๐‘œ๐‘œ + ๐‘˜๐‘˜
๐›ฝ๐›ฝ = ๐›ฝ๐›ฝ๐‘œ๐‘œ + 1 โˆ’ ๐‘˜๐‘˜
Description , Limitations and Uses
When a demand is placed on a machine it undergoes a Bernoulli trial
with success defined as a successful start. It is known the probability
of a successful start, ๐‘๐‘, equals 0.8. Therefore the probability the
machine does not start. ๐‘“๐‘“(0) = 0.2.
For an example with multiple Bernoulli trials see the binomial
distribution.
Characteristics
A Bernoulli process is a probabilistic experiment that can have one
of two outcomes, success (๐‘˜๐‘˜ = 1) with the probability of success is
๐‘๐‘, and failure (๐‘˜๐‘˜ = 0) with the probability of failure is ๐‘ž๐‘ž โ‰ก 1 โˆ’ ๐‘๐‘.
Single Trial. Itโ€™s important to emphasis that the Bernoulli distribution
is for a single trial or event. The case of multiple Bernoulli trials with
replacement is the binomial distribution. The case of multiple
Bernoulli trials without replacement is the hypergeometric
distribution.
n
๏ฟฝ K i ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(ฮ ๐‘˜๐‘˜; ๐‘๐‘ = ฮ ๐‘๐‘๐‘–๐‘– )
i=1
Applications
Used to model a single event which have only two outcomes. In
reliability engineering it is most often used to model demands or
shocks to a component where the component will fail with probability
p.
In practice it is rare for only a single event to be considered and so a
binomial distribution is most often used (with the assumption of
replacement). The conditions and assumptions of a Bernoulli trial
however are used as the basis for each trial in a binomial distribution.
See โ€˜Related Distributionsโ€™ and binomial distribution for more details.
Resources
Online:
Bernoulli
๐พ๐พ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘๐‘)
Maximum Property
max{๐พ๐พ1 , ๐พ๐พ2 , โ€ฆ , ๐พ๐พ๐‘›๐‘› }~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘ = 1 โˆ’ ฮ {1 โˆ’ ๐‘๐‘๐‘–๐‘– })
Minimum property
min{๐พ๐พ1 , ๐พ๐พ2 , โ€ฆ , ๐พ๐พ๐‘›๐‘› }~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘ = ฮ ๐‘๐‘๐‘–๐‘– )
Product Property
156
Discrete Distributions
http://mathworld.wolfram.com/BernoulliDistribution.html
http://en.wikipedia.org/wiki/Bernoulli_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Books:
Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for
scientists and engineers, Birkhäuser.
Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete
Distributions 3rd ed., Wiley-Interscience.
Relationship to Other Distributions
The Binomial distribution counts the number of successes in n
independent observations of a Bernoulli process.
Let
Binomial
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜โ€ฒ|n, p)
Then
๐พ๐พ๐‘–๐‘– ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k i ; p)
๐‘Œ๐‘Œ = ๏ฟฝ ๐พ๐พ๐‘–๐‘–
๐‘–๐‘–=1
Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k โ€ฒ = โˆ‘k i |n, p) where ๐‘˜๐‘˜โ€ฒ โˆˆ {1, 2, โ€ฆ , ๐‘›๐‘›}
Special Case:
Bernoulli
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘›๐‘›
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k; p) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k; p|n = 1)
Binomial Discrete Distribution
157
5.2. Binomial Discrete Distribution
Probability Density Function - f(k)
0.60
0.50
n=1 p=0.4
0.30
n=5 p=0.4
0.25
n=15 p=0.4
0.40
0.20
0.30
0.15
0.20
0.10
0.10
0.05
0.00
0.00
0
5
n=20 p=0.2
n=20 p=0.5
n=20 p=0.8
10
0
5
10
15
20
Cumulative Density Function - F(k)
1.00
1.00
0.80
0.80
0.60
0.60
0.40
0.40
n=1 p=0.4
n=5 p=0.4
n=15 p=0.4
0.20
0.00
0
5
10
0
5
10
15
20
Hazard Rate - h(k)
1.0
1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
0
5
10
0
10
20
Binomial
0.00
0.20
158
Discrete Distributions
Parameters & Description
Parameters
Random Variable
Question
Distribution
PDF
๐‘›๐‘›
๐‘๐‘
๐‘›๐‘› โˆˆ {1, 2 โ€ฆ , โˆž}
0 โ‰ค ๐‘๐‘ โ‰ค 1
Number of Trials.
Bernoulli
probability
parameter.
Probability of success in a single trial.
๐‘˜๐‘˜ โˆˆ {0, 1, 2 โ€ฆ , ๐‘›๐‘›}
The probability of getting exactly ๐‘˜๐‘˜ successes in ๐‘›๐‘› trials.
Formulas
n
๐‘“๐‘“(๐‘˜๐‘˜) = ๏ฟฝ ๏ฟฝ pk (1 โˆ’ p)nโˆ’k
k
where k combinations from n:
๐‘›๐‘›!
๐‘›๐‘›
๐‘›๐‘›
๏ฟฝ ๏ฟฝ = ๐‘›๐‘›๐ถ๐ถ๐‘˜๐‘˜ = ๐ถ๐ถ๐‘˜๐‘˜๐‘›๐‘› =
= ๐ถ๐ถ ๐‘›๐‘›โˆ’1
๐‘˜๐‘˜
๐‘˜๐‘˜! (๐‘›๐‘› โˆ’ ๐‘˜๐‘˜)! ๐‘˜๐‘˜ ๐‘˜๐‘˜โˆ’1
k
๐น๐น(๐‘˜๐‘˜) = ๏ฟฝ
j=0
๐‘›๐‘›!
pj (1 โˆ’ p)nโˆ’j
๐‘—๐‘—! (๐‘›๐‘› โˆ’ ๐‘—๐‘—)!
= ๐ผ๐ผ1โˆ’๐‘๐‘ (๐‘›๐‘› โˆ’ ๐‘˜๐‘˜, ๐‘˜๐‘˜ + 1)
where ๐ผ๐ผ๐‘๐‘ (๐‘Ž๐‘Ž, ๐‘๐‘) is the Regularized Incomplete Beta function. See
section 1.6.3.
CDF
When ๐‘›๐‘› โ‰ฅ 20 and ๐‘๐‘ โ‰ค 0.05, or if ๐‘›๐‘› โ‰ฅ 100 and ๐‘›๐‘›๐‘›๐‘› โ‰ค 10, this can
be approximated by a Poisson distribution with ๐œ‡๐œ‡ = ๐‘›๐‘›๐‘›๐‘›:
k
Binomial
๐น๐น(๐‘˜๐‘˜) โ‰… eโˆ’µ ๏ฟฝ
j=0
µj ฮ“(๐‘˜๐‘˜ + 1, ๐œ‡๐œ‡)
=
j!
๐‘˜๐‘˜!
โ‰… ๐น๐น๐œ’๐œ’2 (2๐œ‡๐œ‡, 2๐‘˜๐‘˜ + 2)
When ๐‘›๐‘›๐‘›๐‘› โ‰ฅ 10 and ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘) โ‰ฅ 10 then the cdf can be
approximated using a normal distribution:
๐‘˜๐‘˜ + 0.5 โˆ’ ๐‘›๐‘›๐‘›๐‘›
๐น๐น(๐‘˜๐‘˜) โ‰… ฮฆ ๏ฟฝ
๏ฟฝ
๏ฟฝ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)
k
๐‘…๐‘…(๐‘˜๐‘˜) = 1 โˆ’ ๏ฟฝ
Reliability
n
= ๏ฟฝ
j=0
j=k+1
๐‘›๐‘›!
pj (1 โˆ’ p)nโˆ’j
๐‘—๐‘—! (๐‘›๐‘› โˆ’ ๐‘—๐‘—)!
๐‘›๐‘›!
pj (1 โˆ’ p)nโˆ’j
๐‘—๐‘—! (๐‘›๐‘› โˆ’ ๐‘—๐‘—)!
= ๐ผ๐ผ๐‘๐‘ (๐‘˜๐‘˜ + 1, ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜)
where ๐ผ๐ผ๐‘๐‘ (๐‘Ž๐‘Ž, ๐‘๐‘) is the Regularized Incomplete Beta function. See
section 1.6.3.
Binomial Discrete Distribution
Hazard Rate
159
โˆ’1
n
(1 + ฮธ)n โˆ’ โˆ‘kj=0 ๏ฟฝ ๏ฟฝ ฮธj
k ๏ฟฝ
โ„Ž(๐‘˜๐‘˜) = ๏ฟฝ1 +
n
๏ฟฝ ๏ฟฝ ฮธk
k
where
๐œƒ๐œƒ =
(Gupta et al. 1997)
๐‘๐‘
1 โˆ’ ๐‘๐‘
Properties and Moments
๐‘˜๐‘˜0.5 is either {โŒŠ๐‘›๐‘›๐‘›๐‘›โŒ‹, โŒˆ๐‘›๐‘›๐‘›๐‘›โŒ‰}
Median
โŒŠ(๐‘›๐‘› + 1)๐‘๐‘โŒ‹
Mode
๐‘›๐‘›๐‘›๐‘›
Mean - 1st Raw Moment
Variance -
2nd
Skewness -
๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)
Central Moment
3rd
1 โˆ’ 2p
Central Moment
๏ฟฝnp(1 โˆ’ p)
6p2 โˆ’ 6p + 1
np(1 โˆ’ p)
Excess kurtosis - 4th Central Moment
๏ฟฝ1 โˆ’ p + peit ๏ฟฝ
Characteristic Function
100ฮฑ% Percentile Function
n
Numerically solve for ๐‘˜๐‘˜ (which is not
arduous for ๐‘›๐‘› โ‰ค 10):
๐‘˜๐‘˜๐›ผ๐›ผ = ๐น๐น โˆ’1 (๐‘›๐‘›, ๐‘๐‘)
k ฮฑ โ‰… ๏ฟฝฮฆโˆ’1 (๐›ผ๐›ผ)๏ฟฝ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘) + ๐‘›๐‘›๐‘›๐‘› โˆ’ 0.5๏ฟฝ
Parameter Estimation
Maximum Likelihood Function
Likelihood
Function
For complete data only:
๐‘›๐‘›๐ต๐ต
ni
๐ฟ๐ฟ(๐‘๐‘|๐ธ๐ธ) = ๏ฟฝ ๏ฟฝk ๏ฟฝ pki (1 โˆ’ p)niโˆ’ki
i=1
i
= pโˆ‘ki (1 โˆ’ p)โˆ‘niโˆ’โˆ‘ki
๐‘›๐‘›
๐ต๐ต
๐‘˜๐‘˜๐‘–๐‘– , โˆ‘๐‘›๐‘›๐‘–๐‘– =
Where ๐‘›๐‘›๐ต๐ต is the number of Binomial processes, โˆ‘๐‘˜๐‘˜๐‘–๐‘– = โˆ‘๐‘–๐‘–=1
๐‘›๐‘›๐ต๐ต
โˆ‘๐‘–๐‘–=1 ๐‘›๐‘›๐‘–๐‘– and the combinatory term is ignored (see section 1.1.6 for
discussion).
Binomial
For ๐‘›๐‘›๐‘›๐‘› โ‰ฅ 10 and ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘) โ‰ฅ 10 the
normal approximation may be used:
160
Discrete Distributions
๐‘‘๐‘‘L
=0
๐‘‘๐‘‘p
solve for ๐‘๐‘
๐‘‘๐‘‘L
= โˆ‘k i . pโˆ‘(ki)โˆ’1 (1 โˆ’ p)โˆ‘niโˆ’โˆ‘ki โˆ’ (โˆ‘ni โˆ’ โˆ‘k i )pโˆ‘ki (1 โˆ’ p)โˆ‘niโˆ’1โˆ’โˆ‘ki
๐‘‘๐‘‘p
โˆ‘k i . pโˆ‘(ki)โˆ’1 (1 โˆ’ p)โˆ‘niโˆ’โˆ‘ki = (โˆ‘ni โˆ’ โˆ‘k i )pโˆ‘ki (1 โˆ’ p)โˆ’1+โˆ‘niโˆ’โˆ‘ki
โˆ‘k i . pโˆ’1 = (โˆ‘ni โˆ’ โˆ‘k i )(1 โˆ’ p)โˆ’1
(1 โˆ’ p) โˆ‘ni โˆ’ โˆ‘k i
=
โˆ‘k i
p
p=
MLE Point
Estimates
โˆ‘k i
โˆ‘ni
The MLE point estimate for p:
p๏ฟฝ =
Fisher
Information
Binomial
Confidence
Intervals
๐ผ๐ผ(๐‘๐‘) =
โˆ‘k i
โˆ‘ni
1
๐‘๐‘(1 โˆ’ ๐‘๐‘)
The confidence intervals for the binomial distribution parameter p is a
controversial subject which is still debated. The Wilson interval is
recommended for small and large ๐‘›๐‘›. (Brown et al. 2001)
๐‘๐‘ =
where
๐‘๐‘ =
๐‘›๐‘›๐‘๐‘ฬ‚ + ๐œ…๐œ… 2 โ„2 ๐œ…๐œ…๏ฟฝ๐œ…๐œ… 2 + 4๐‘›๐‘›๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
+
2(๐‘›๐‘› + ๐œ…๐œ… 2 )
๐‘›๐‘› + ๐œ…๐œ… 2
๐‘›๐‘›๐‘๐‘ฬ‚ + ๐œ…๐œ… 2 โ„2 ๐œ…๐œ…๏ฟฝ๐œ…๐œ… 2 + 4๐‘›๐‘›๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
โˆ’
2(๐‘›๐‘› + ๐œ…๐œ… 2 )
๐‘›๐‘› + ๐œ…๐œ… 2
๐›พ๐›พ + 1
๐œ…๐œ… = ฮฆโˆ’1 ๏ฟฝ
๏ฟฝ
2
It should be noted that most textbooks use the Wald interval (normal
approximation) given below, however many articles have shown these
estimates to be erratic and cannot be trusted. (Brown et al. 2001)
๐‘๐‘ = ๐‘๐‘ฬ‚ + ๐œ…๐œ…๏ฟฝ
๐‘๐‘ = ๐‘๐‘ฬ‚ โˆ’ ๐œ…๐œ…๏ฟฝ
๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
๐‘›๐‘›
๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
๐‘›๐‘›
For a comparison of binomial confidence interval estimates the reader
is referred to (Brown et al. 2001). The following webpage has links to
online calculators which use many different methods.
http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval
Binomial Discrete Distribution
161
Bayesian
Non-informative Priors for p given n, ๐…๐…(๐’‘๐’‘|๐’๐’)
(Yang and Berger 1998, p.6)
Type
Prior
Posterior
Uniform
Proper
Prior with limits
๐‘๐‘ โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Improper
Proir with limits
๐‘๐‘ โˆˆ [0,1]
1 = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 1,1)
Jeffreyโ€™s Prior
Reference Prior
MDIP
Novick and Hall
UOI
Example
Otherwise ๐œ‹๐œ‹(๐‘๐‘) = 0
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 1 + ๐‘˜๐‘˜, 1 + ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜)
1
1 1
= ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐‘๐‘; , ๏ฟฝ
2 2
๏ฟฝ๐‘๐‘(1 โˆ’ ๐‘๐‘)
1
1
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝ๐‘๐‘; + ๐‘˜๐‘˜, + ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜๏ฟฝ
2
2
when ๐‘๐‘ โˆˆ [0,1]
1.6186๐‘๐‘๐‘๐‘ (1 โˆ’ ๐‘๐‘)1โˆ’๐‘๐‘
๐‘๐‘โˆ’1 (1
Likelihood
Model
Binomial
โˆ’
๐‘๐‘)โˆ’1
Proper - No Closed Form
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; ๐‘˜๐‘˜, ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜)
when ๐‘๐‘ โˆˆ [0,1]
= ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(0,0)
Conjugate Priors
Evidence
๐‘˜๐‘˜ failures
in ๐‘›๐‘› trial
Dist of
UOI
Prior
Para
Beta
๐›ผ๐›ผ๐‘œ๐‘œ , ๐›ฝ๐›ฝ๐‘œ๐‘œ
Posterior Parameters
๐›ผ๐›ผ = ๐›ผ๐›ผ๐‘œ๐‘œ + ๐‘˜๐‘˜
ฮฒ = ฮฒo + n โˆ’ k
Description , Limitations and Uses
Five machines are measured for performance on demand. The
machines can either fail or succeed in their application. The
machines are tested for 10 demands with the following data for each
machine:
Machine/Trail
1
2
3
4
5
๐œ‡๐œ‡๐‘–๐‘–
1
2
3
F=3
F=2
F=2
F=3
F=2
๐‘›๐‘›๐‘๐‘ฬ‚
4
5
6
7
8
S=7
S=8
S=8
S=7
S=8
๐‘›๐‘›(1 โˆ’ ๐‘๐‘ฬ‚ )
9
10
Assuming machines are homogeneous estimate the parameter ๐‘๐‘:
Using MLE:
p๏ฟฝ =
โˆ‘k i 12
=
= 0.24
โˆ‘ni 50
Binomial
๐‘๐‘
from
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘, ๐‘›๐‘›)
Truncated Beta Distribution
For a โ‰ค ๐‘๐‘ โ‰ค b
๐‘๐‘. ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 1 + ๐‘˜๐‘˜, 1 + ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜)
162
Discrete Distributions
90% confidence intervals for ๐‘๐‘:
๐œ…๐œ… = ฮฆโˆ’1 (0.95) = 1.64485
๐‘๐‘๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™๐‘™ =
๐‘›๐‘›๐‘๐‘ฬ‚ + ๐œ…๐œ… 2 โ„2 ๐œ…๐œ…๏ฟฝ๐œ…๐œ… 2 + 4๐‘›๐‘›๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
โˆ’
= 0.1557
2(๐‘›๐‘› + ๐œ…๐œ… 2 )
๐‘›๐‘› + ๐œ…๐œ… 2
๐‘๐‘๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข๐‘ข =
๐‘›๐‘›๐‘๐‘ฬ‚ + ๐œ…๐œ… 2 โ„2 ๐œ…๐œ…๏ฟฝ๐œ…๐œ… 2 + 4๐‘›๐‘›๐‘๐‘ฬ‚ (1 โˆ’ ๐‘๐‘ฬ‚ )
+
= 0.351
2(๐‘›๐‘› + ๐œ…๐œ… 2 )
๐‘›๐‘› + ๐œ…๐œ… 2
A Bayesian point estimate using a uniform prior distribution ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(1,
1), with posterior ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 13, 39) has a point estimate:
๐‘๐‘ฬ‚ = E[๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘๐‘; 13, 39)] =
13
= 0.25
52
With 90% confidence interval using inverse Beta cdf:
โˆ’1
[๐น๐น๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต
(0.05) = 0.1579,
โˆ’1
๐น๐น๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต
(0.95) = 0.3532]
Binomial
The probability of observing no failures in the next 10 trials with
replacement is:
๐‘“๐‘“(0; 10,0.25) = 0.0563
Characteristics
The probability of observing less than 5 failures in the next 10 trials
with replacement is:
๐‘“๐‘“(0; 10,0.25) = 0.9803
CDF Approximations. The Binomial distribution is one of the most
widely used distributions throughout history. Although simple, the
CDF function was tedious to calculate prior to the use of computers.
As a result approximations using the Poisson and Normal distribution
have been used. For details see โ€˜Related Distributionsโ€™.
With Replacement. The Binomial distribution models probability of
๐‘˜๐‘˜ successes in ๐‘›๐‘› Bernoulli trials. However, the ๐‘˜๐‘˜ successes can
occur anywhere among the ๐‘›๐‘› trials with ๐‘›๐‘›๐ถ๐ถ๐‘˜๐‘˜ different combinations.
Therefore the Binomial distribution assumes replacement. The
equivalent distribution which assumes without replacement is the
hypergeometric distribution.
Symmetrical. The distribution is symmetrical when ๐‘๐‘ = 0.5.
Compliment. ๐‘“๐‘“(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘) = ๐‘“๐‘“(๐‘›๐‘› โˆ’ ๐‘˜๐‘˜; ๐‘›๐‘›, 1 โˆ’ ๐‘๐‘). Tables usually only
provide values up to ๐‘›๐‘›/2 allowing the reader to calculate to ๐‘›๐‘› using
the compliment formula.
Assumptions. The binomial distribution describes the behavior of a
count variable K if the following conditions apply:
Binomial Discrete Distribution
1.
2.
3.
4.
The number of observations n is fixed.
Each observation is independent.
Each observation represents one of two outcomes
("success" or "failure").
The probability of "success" is the same for each outcome.
Convolution Property
Applications
Resources
163
When ๐‘๐‘ is fixed.
๐พ๐พ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘›๐‘›, ๐‘๐‘)
๏ฟฝ K i ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(โˆ‘๐‘›๐‘›๐‘–๐‘– , ๐‘๐‘)
Used to model independent repeated trials which have two
outcomes. Examples used in Reliability Engineering are:
โ€ข
Number of independent components which fail, ๐‘˜๐‘˜, from a
population, ๐‘›๐‘› after receiving a shock.
โ€ข
Number of failures to start, ๐‘˜๐‘˜, from ๐‘›๐‘› demands on a
component.
โ€ข
Number of independent items defective, ๐‘˜๐‘˜, from a
population of ๐‘›๐‘› items.
Online:
http://mathworld.wolfram.com/BinomialDistribution.html
http://en.wikipedia.org/wiki/Binomial_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (web calc)
Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete
Distributions 3rd ed., Wiley-Interscience.
Relationship to Other Distributions
The Binomial distribution counts the number of successes ๐‘˜๐‘˜ in ๐‘›๐‘›
independent observations of a Bernoulli process.
Let
Bernoulli
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(k โ€ฒ ; p)
Then
๐พ๐พ๐‘–๐‘– ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(kโ€ฒi ; ๐‘๐‘)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘›๐‘›
๐‘Œ๐‘Œ = ๏ฟฝ ๐พ๐พ๐‘–๐‘–
๐‘–๐‘–=1
Y~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(โˆ‘k โ€ฒ i ; ๐‘›๐‘›, ๐‘๐‘) where ๐‘˜๐‘˜ โˆˆ {1, 2, โ€ฆ , ๐‘›๐‘›}
Special Case:
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘๐‘|๐‘›๐‘› = 1)
Binomial
Books:
Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for
scientists and engineers, Birkhäuser.
164
Discrete Distributions
Hypergeometric
Distribution
๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป
(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘š๐‘š, ๐‘๐‘)
Normal
Distribution
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘ก๐‘ก; ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 )
The hypergeometric distribution models probability of ๐‘˜๐‘˜ successes in
๐‘›๐‘› Bernoulli trials from a population ๐‘๐‘, with ๐‘š๐‘š successors without
replacement.
๐‘“๐‘“(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘š๐‘š, ๐‘๐‘)
Limiting Case for ๐‘›๐‘› โ‰ซ ๐‘˜๐‘˜ and ๐‘๐‘ not near 0 or 1:
๐‘š๐‘š
lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘ = ) = ๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป๐ป(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘š๐‘š, ๐‘๐‘)
๐‘›๐‘›โ†’โˆž
๐‘๐‘
Limiting Case for constant ๐‘๐‘:
lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘›๐‘›, ๐‘๐‘) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝk๏ฟฝµ = n๐‘๐‘, ๐œŽ๐œŽ 2 = ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)๏ฟฝ
๐‘›๐‘›โ†’โˆž
๐‘๐‘=๐‘๐‘
The Normal distribution can be used as an approximation of the
Binomial distribution when ๐‘›๐‘›๐‘›๐‘› โ‰ฅ 10 and ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘) โ‰ฅ 10.
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘๐‘, ๐‘›๐‘›) โ‰ˆ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐‘˜๐‘˜ + 0.5๏ฟฝ๐œ‡๐œ‡ = ๐‘›๐‘›๐‘›๐‘›, ๐œŽ๐œŽ 2 = ๐‘›๐‘›๐‘›๐‘›(1 โˆ’ ๐‘๐‘)๏ฟฝ
Limiting Case for constant ๐‘›๐‘›๐‘›๐‘›:
lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘) = ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k; µ = n๐‘๐‘)
๐‘›๐‘›โ†’โˆž
n๐‘๐‘=๐œ‡๐œ‡
Binomial
The Poisson distribution is the limiting case of the Binomial
distribution when ๐‘›๐‘› is large but the ratio of ๐‘›๐‘›๐‘›๐‘› remains constant.
Hence the Poisson distribution models rare events.
Poisson
Distribution
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; ๐œ‡๐œ‡)
Multinomial
Distribution
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘ (๐ค๐ค|n, ๐ฉ๐ฉ)
The Poisson distribution can be used as an approximation to the
Binomial distribution when ๐‘›๐‘› โ‰ฅ 20 and ๐‘๐‘ โ‰ค 0.05, or if ๐‘›๐‘› โ‰ฅ 100 and
๐‘›๐‘›๐‘›๐‘› โ‰ค 10.
The Binomial is expressed in terms of the total number of a
probability of success, ๐‘๐‘, and trials, ๐‘๐‘. Where a Poisson distribution
is expressed in terms of a success rate and does not need to know
the total number of trials.
The derivation of the Poisson distribution from the binomial can be
found at http://mathworld.wolfram.com/PoissonDistribution.html.
This interpretation can also be used to understand the conditional
distribution of a Poisson random variable:
Let
๐พ๐พ1 , ๐พ๐พ2 ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ ๐‘ (µ)
Given
๐‘›๐‘› = ๐พ๐พ1 + ๐พ๐พ2 = ๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘› ๐‘œ๐‘œ๐‘œ๐‘œ ๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’
Then
µ1
๐พ๐พ1 |n~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝk; n, p =
๏ฟฝ
µ1 + µ2
Special Case:
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘=2 (๐ค๐ค|n, ๐ฉ๐ฉ) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘›๐‘›, ๐‘๐‘)
Poisson Discrete Distribution
165
5.3. Poisson Discrete Distribution
Probability Density Function - f(k)
0.20
0.18
0.16
0.14
0.12
0.10
0.08
0.06
0.04
0.02
0.00
ฮผ=4
ฮผ=10
ฮผ=20
ฮผ=30
0
5
10
15
20
25
30
35
40
45
Cumulative Density Function - F(k)
1.00
0.80
0.60
ฮผ=4
ฮผ=10
ฮผ=20
ฮผ=30
0.40
0.20
0
5
10
15
20
25
30
35
40
45
10
15
20
25
30
35
40
45
Hazard Rate - h(k)
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
0
5
Poisson
0.00
166
Discrete Distributions
Parameters & Description
๐œ‡๐œ‡
Parameters
Shape Parameter: The value of ๐œ‡๐œ‡ is
the expected number of events per
time period or other physical
dimensions. If the Poisson distribution
is modeling failure events, then ๐œ‡๐œ‡ = ๐œ†๐œ†๐œ†๐œ†
is the average number of failures that
would occur in the space ๐‘ก๐‘ก. In this
case ๐‘ก๐‘ก is fixed and ๐œ†๐œ† becomes the
distribution parameter. Some texts
use the symbol ๐œŒ๐œŒ.
๐œ‡๐œ‡ > 0
๐‘˜๐‘˜ is an integer,
Random Variable
Distribution
Formulas
๐‘“๐‘“(๐‘˜๐‘˜) =
PDF
๐œ‡๐œ‡๐‘˜๐‘˜ โˆ’µ (๐œ†๐œ†๐œ†๐œ†)๐‘˜๐‘˜ โˆ’ฮปt
๐‘’๐‘’ =
๐‘’๐‘’
k!
k!
k
๐น๐น(๐‘˜๐‘˜) = eโˆ’µ ๏ฟฝ
j=0
Poisson
CDF
Reliability
๐‘˜๐‘˜ โ‰ฅ 0
µj ฮ“(๐‘˜๐‘˜ + 1, ๐œ‡๐œ‡)
=
j!
๐‘˜๐‘˜!
= ๐น๐น๐œ’๐œ’2 (2๐œ‡๐œ‡, 2๐‘˜๐‘˜ + 2)
Where ๐น๐น๐œ’๐œ’2 (๐‘ฅ๐‘ฅ|๐‘ฃ๐‘ฃ) is the Chi-square CDF.
When ๐œ‡๐œ‡ > 10 the ๐น๐น(๐‘˜๐‘˜) can be approximated by a normal distribution:
๐‘˜๐‘˜ + 0.5 โˆ’ ๐œ‡๐œ‡
๐น๐น(๐‘˜๐‘˜) โ‰… ฮฆ ๏ฟฝ
๏ฟฝ
โˆš๐œ‡๐œ‡
R(k) = 1 โˆ’ F(k)
k
Hazard Rate
k!
µj
โ„Ž(๐‘˜๐‘˜) = ๏ฟฝ1 + ๏ฟฝeµ โˆ’ 1 โˆ’ ๏ฟฝ ๏ฟฝ๏ฟฝ
µ
j!
โˆ’1
j=1
(Gupta et al. 1997)
Properties and Moments
Median
Mode
Mean - 1st Raw Moment
See 100ฮฑ% Percentile Function when
๐›ผ๐›ผ = 0.5.
โŒŠ๐œ‡๐œ‡โŒ‹
where โŒŠ๐œ‡๐œ‡โŒ‹ is the floor function 2
Variance - 2nd Central Moment
2
โŒŠ๐œ‡๐œ‡โŒ‹ = is the floor function (largest integer not greater than ๐œ‡๐œ‡)
๐œ‡๐œ‡
๐œ‡๐œ‡
Poisson Discrete Distribution
Skewness - 3rd Central Moment
Excess kurtosis -
4th
167
1/๏ฟฝµ
1/µ
Central Moment
exp{µ๏ฟฝeik โˆ’ 1๏ฟฝ}
Characteristic Function
100ฮฑ% Percentile Function
Numerically solve for ๐‘˜๐‘˜ (which is not
arduous for ๐œ‡๐œ‡ โ‰ค 10):
๐‘˜๐‘˜๐›ผ๐›ผ = ๐น๐น โˆ’1 (๐›ผ๐›ผ)
For ๐‘˜๐‘˜ > 10 the normal approximation may
be used:
k ฮฑ โ‰… ๏ฟฝ๏ฟฝµฮฆโˆ’1 (๐›ผ๐›ผ) + ๐œ‡๐œ‡ โˆ’ 0.5๏ฟฝ
Parameter Estimation
Maximum Likelihood Estimates
Likelihood
Functions
For complete data:
๐น๐น
n ๐œ‡๐œ‡ ๐‘˜๐‘˜๐‘–๐‘–
โˆ’µ
๐ฟ๐ฟ(๐œ‡๐œ‡|๐ธ๐ธ) = ๏ฟฝ
F ๐‘’๐‘’
i=1 k i !
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
known k
Log-Likelihood
Function
MLE
Point
Estimates
n
ฮ› = โˆ’nµ + ๏ฟฝ{๐‘˜๐‘˜๐‘–๐‘– ln(๐œ‡๐œ‡) โˆ’ ln(๐‘˜๐‘˜๐‘–๐‘– !)}
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
i=1
known k
n
โˆ‚ฮ›
1
= โˆ’n + ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘– = 0
โˆ‚µ
µ
๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ๏ฟฝ
๏ฟฝ๏ฟฝ๏ฟฝ
i=1
For complete data solving
µ๏ฟฝ =
โˆ‚ฮ›
โˆ‚µ
n
known k
= 0 gives:
n
1
1
. ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘– ๐‘œ๐‘œ๐‘œ๐‘œ ฮป๏ฟฝ = . ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘–
๐‘›๐‘›
๐‘ก๐‘ก๐‘ก๐‘ก
i=1
i=1
Note that in this context:
t = the unit of time for which the rate, ๐œ†๐œ† is being measured.
๐‘›๐‘› = the number of Poisson processes for which the exact number of
failures, k, was known.
๐‘˜๐‘˜๐‘–๐‘– = the number of failures that occurred within the ith Poisson process.
When there is only one Poisson process this reduces to:
๐‘˜๐‘˜
µ๏ฟฝ = ๐‘˜๐‘˜ ๐‘œ๐‘œ๐‘œ๐‘œ ฮป๏ฟฝ =
๐‘ก๐‘ก
For censored data numerical methods are needed to maximize the loglikelihood function.
Poisson
โˆ‚ฮ›
=0
โˆ‚µ
where ๐‘›๐‘› is the number of poisson processes.
168
Discrete Distributions
Fisher
Information
๐ผ๐ผ(๐œ†๐œ†) =
100๐›พ๐›พ%
Confidence
Interval
(complete data
only)
๐œ†๐œ†lower 2 Sided
1
๐œ†๐œ†
๐œ†๐œ†upper 2 Sided
Conservative two
sided
confidence
intervals.
๐œ’๐œ’ 21โˆ’ฮณ (2โˆ‘๐‘˜๐‘˜๐‘–๐‘– )
๐œ’๐œ’ 21+ฮณ (2โˆ‘๐‘˜๐‘˜๐‘–๐‘– + 2)
When ๐‘˜๐‘˜ is large
(๐‘˜๐‘˜ > 10) two sided
intervals
1 + ๐›พ๐›พ
๐œ†๐œ†ฬ‚
ฮป๏ฟฝ โˆ’ ฮฆโˆ’1 ๏ฟฝ
๏ฟฝ๏ฟฝ
2
๐‘ก๐‘ก๐‘›๐‘›
1 + ๐›พ๐›พ
๐œ†๐œ†ฬ‚
ฮป๏ฟฝ + ฮฆโˆ’1 ๏ฟฝ
๏ฟฝ๏ฟฝ
2
๐‘ก๐‘ก๐‘ก๐‘ก
๏ฟฝ
2
๏ฟฝ
๏ฟฝ
2๐‘ก๐‘ก๐‘ก๐‘ก
2
๏ฟฝ
2๐‘ก๐‘ก๐‘ก๐‘ก
(Nelson 1982, p.201) Note: The first confidence intervals are
conservative in that at least 100๐›พ๐›พ%. Exact confidence intervals cannot
be easily achieved for discrete distributions.
Bayesian
Non-informative Priors ๐…๐…(๐€๐€) in known time interval ๐‘ก๐‘ก
Poisson
Type
Prior
Posterior
Uniform
Proper
Prior with limits
๐œ†๐œ† โˆˆ [๐‘Ž๐‘Ž, ๐‘๐‘]
1
๐‘๐‘ โˆ’ ๐‘Ž๐‘Ž
Uniform Improper
Prior with limits
๐œ†๐œ† โˆˆ [0, โˆž)
1 โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(1,0)
1
Jeffreyโ€™s Prior
โˆš๐œ†๐œ†
Truncated Gamma Distribution
For a โ‰ค ฮป โ‰ค b
๐‘๐‘. ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + k, t)
Otherwise ๐œ‹๐œ‹(๐œ†๐œ†) = 0
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 1 + k, t)
1
โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(2, 0)
1
โˆ ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(0,0)
๐œ†๐œ†
Novick and Hall
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 12 + k, t)
when ๐œ†๐œ† โˆˆ [0, โˆž)
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; k, t)
when ๐œ†๐œ† โˆˆ [0, โˆž)
Conjugate Priors
UOI
๐œ†๐œ†
from
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; µ)
Example
Likelihood
Model
Exponential
Evidence
๐‘›๐‘›๐น๐น failures
in ๐‘ก๐‘ก ๐‘‡๐‘‡ unit of
time
Dist of
UOI
Prior
Para
Gamma
๐‘˜๐‘˜0 , ฮ›0
Description , Limitations and Uses
Posterior Parameters
๐‘˜๐‘˜ = ๐‘˜๐‘˜๐‘œ๐‘œ + ๐‘›๐‘›๐น๐น
ฮ› = ฮ›๐‘œ๐‘œ + ๐‘ก๐‘ก ๐‘‡๐‘‡
Three vehicle tires were run on a test area for 1000km have
punctures at the following distances:
Tire 1: No punctures
Tire 2: 400km, 900km
Tire 3: 200km
Poisson Discrete Distribution
169
Punctures can be modeled as a renewal process with perfect repair
and an inter-arrival time modeled by an exponential distribution. Due
to the Poisson distribution being homogeneous in time, the test from
multiple tires can be combined and considered a test of one tire with
multiple renewals. See example in section 1.1.6.
Total time on test is 3 × 1000 = 3000km. Total number of failures is
3. Therefore using MLE the estimate of ๐œ†๐œ†:
๐œ†๐œ†ฬ‚ =
k
3
=
= 1E-3
๐‘ก๐‘ก ๐‘‡๐‘‡ 3000
With 90% confidence interval (conservative):
2
2
(6)
(8)
๐œ’๐œ’(0.95)
๐œ’๐œ’(0.05)
= 0.272๐ธ๐ธ-3,
= 2.584๐ธ๐ธ-3๏ฟฝ
๏ฟฝ
6000
6000
A Bayesian point estimate using the Jeffery non-informative
improper prior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(12, 0), with posterior ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 3.5, 3000) has
a point estimate:
3.5
= 1.16ฬ‡ E โˆ’ 3
๐œ†๐œ†ฬ‚ = E[๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†; 3.5, 3000)] =
3000
Characteristics
With 90% confidence interval using inverse Gamma cdf:
[๐น๐น๐บ๐บโˆ’1 (0.05) = 0.361๐ธ๐ธ-3,
๐น๐น๐บ๐บโˆ’1 (0.95) = 2.344๐ธ๐ธ-3]
The Poisson distribution is also known as the Rare Event distribution.
ฮผ characteristics:
โ€ข
๐œ‡๐œ‡ is the expected number of events for the unit of time being
measured.
โ€ข
When the unit of time varies ฮผ can be transformed into a
rate and time measure, ๐œ†๐œ†๐œ†๐œ†.
โ€ข
For ๐œ‡๐œ‡ โ‰ฒ 10 the distribution is skewed to the right.
โ€ข
For ๐œ‡๐œ‡ โ‰ณ 10 the distribution approaches a normal distribution
with a ๐œ‡๐œ‡ = ๐œ‡๐œ‡ and ๐œŽ๐œŽ = โˆš๐œ‡๐œ‡.
๐พ๐พ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐œ‡๐œ‡)
Convolution property
๐พ๐พ1 + ๐พ๐พ2 + โ€ฆ + ๐พ๐พ๐‘›๐‘› ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜; โˆ‘๐œ‡๐œ‡๐‘–๐‘– )
Poisson
If the following assumptions are met than the process follows a
Poisson distribution:
โ€ข
The chance of two simultaneous events is negligible or
impossible (such as renewal of a single component);
โ€ข
The expected value of the random number of events in a
region is proportional to the size of the region.
โ€ข
The random number of events in non-overlapping regions
are independent.
170
Discrete Distributions
Applications
Homogeneous Poisson Process (HPP). The Poisson distribution
gives the distribution of exactly k failures occurring in a HPP. See
relation to exponential and gamma distributions.
Renewal Theory. Used in renewal theory as the counting function
and may model non-homogeneous (aging) components by using a
time dependent failure rate, ๐œ†๐œ† (๐‘ก๐‘ก).
Binomial Approximation. Used to model the Binomial distribution
when the number of trials is large and ฮผ remains moderate. This can
greatly simplify Binomial distribution calculations.
Rare Event. Used to model rare events when the number of trials is
large compared to the rate at which events occur.
Resources
Online:
http://mathworld.wolfram.com/PoissonDistribution.html
http://en.wikipedia.org/wiki/Poisson_distribution
http://socr.ucla.edu/htmls/SOCR_Distributions.html (interactive web
calculator)
Books:
Haight, F.A., 1967. Handbook of the Poisson distribution [by] Frank
A. Haight, New York,: Wiley.
Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience.
Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete
Distributions 3rd ed., Wiley-Interscience.
Poisson
Relationship to Other Distributions
Let
๐พ๐พ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k; µ = ๐œ†๐œ†๐œ†๐œ†)
Given
Exponential
Distribution
๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜|๐œ†๐œ†)
๐‘‡๐‘‡1 , T2 โ€ฆ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(t; ๐œ†๐œ†)
The time between each arrival of T is exponentially distributed.
Special Cases:
Let
Gamma
Distribution
๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐พ๐พ + ๐‘‡๐‘‡๐พ๐พ+1 โ€ฆ
Then
Then
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k; ๐œ†๐œ†๐œ†๐œ†|๐‘˜๐‘˜ = 1) = ๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐‘ก๐‘ก; ๐œ†๐œ†)
๐‘‡๐‘‡1 โ€ฆ ๐‘‡๐‘‡๐‘˜๐‘˜ ~๐ธ๐ธ๐ธ๐ธ๐ธ๐ธ(๐œ†๐œ†)
๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‡๐‘‡๐‘ก๐‘ก = ๐‘‡๐‘‡1 + ๐‘‡๐‘‡2 + โ‹ฏ + ๐‘‡๐‘‡๐‘˜๐‘˜
๐‘‡๐‘‡๐‘ก๐‘ก ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘˜๐‘˜, ๐œ†๐œ†)
The Poisson distribution is the probability that exactly ๐‘˜๐‘˜ failures have
been observed in time ๐‘ก๐‘ก. This is the probability that ๐‘ก๐‘ก is between ๐‘‡๐‘‡๐‘˜๐‘˜
and ๐‘‡๐‘‡๐‘˜๐‘˜+1 .
Poisson Discrete Distribution
๐‘“๐‘“๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ†๐œ†๐œ†๐œ†) = ๏ฟฝ
๐‘˜๐‘˜+1
๐‘˜๐‘˜
where ๐‘˜๐‘˜ is an integer.
171
๐‘“๐‘“๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘ฅ๐‘ฅ, ๐œ†๐œ†)๐‘‘๐‘‘๐‘‘๐‘‘
= ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜ + 1, ๐œ†๐œ†) โˆ’ ๐น๐น๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ (๐‘ก๐‘ก; ๐‘˜๐‘˜, ๐œ†๐œ†)
Limiting Case for constant ๐‘›๐‘›๐‘›๐‘›:
lim ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜; ๐‘›๐‘›, ๐‘๐‘) = ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(k|µ = n๐‘๐‘)
๐‘›๐‘›โ†’โˆž
n๐‘๐‘=๐œ‡๐œ‡
The Poisson distribution is the limiting case of the Binomial
distribution when ๐‘›๐‘› is large but the ratio of ๐‘›๐‘›๐‘›๐‘› remains constant.
Hence the Poisson distribution models rare events.
Binomial
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘๐‘, ๐‘๐‘)
The Poisson distribution can be used as an approximation to the
Binomial distribution when ๐‘›๐‘› โ‰ฅ 20 and ๐‘๐‘ โ‰ค 0.05, or if ๐‘›๐‘› โ‰ฅ 100 and
๐‘›๐‘›๐‘›๐‘› โ‰ค 10.
The Binomial is expressed in terms of the total number of a
probability of success, ๐‘๐‘, and trials, ๐‘๐‘. Where a Poisson distribution
is expressed in terms of a success rate and does not need to know
the total number of trials.
The derivation of the Poisson distribution from the binomial can be
found at http://mathworld.wolfram.com/PoissonDistribution.html.
Normal
Distribution
๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘˜๐‘˜|๐œ‡๐œ‡โ€ฒ, ๐œŽ๐œŽ)
Chi-square
Distribution
๐œ’๐œ’ 2 (๐‘ก๐‘ก|๐‘ฃ๐‘ฃ)
lim ๐น๐น๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ‡๐œ‡) = ๐น๐น๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ (๐‘˜๐‘˜; ๐œ‡๐œ‡โ€ฒ = ๐œ‡๐œ‡, ๐œŽ๐œŽ 2 = ๐œ‡๐œ‡)
๐œ‡๐œ‡โ†’โˆž
This is a good approximation when ๐œ‡๐œ‡ > 1000. When ๐œ‡๐œ‡ > 10 the same
approximation can be made with a correction:
lim ๐น๐น๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ (๐‘˜๐‘˜; ๐œ‡๐œ‡) = ๐น๐น๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ (๐‘˜๐‘˜; ๐œ‡๐œ‡โ€ฒ = ๐œ‡๐œ‡ โˆ’ 0.5, ๐œŽ๐œŽ 2 = ๐œ‡๐œ‡)
๐œ‡๐œ‡โ†’โˆž
๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(๐‘˜๐‘˜|๐œ‡๐œ‡) = ๐œ’๐œ’ 2 (๐‘ฅ๐‘ฅ = 2๐œ‡๐œ‡, ๐‘ฃ๐‘ฃ = 2๐‘˜๐‘˜ + 2)
Poisson
This interpretation can also be used to understand the conditional
distribution of a Poisson random variable:
Let
๐พ๐พ1 , ๐พ๐พ2 ~๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ๐‘ƒ(µ)
Given
๐‘›๐‘› = ๐พ๐พ1 + ๐พ๐พ2 = ๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘›๐‘› ๐‘œ๐‘œ๐‘œ๐‘œ ๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’๐‘’
Then
µ1
๐พ๐พ1 |n~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต ๏ฟฝk; n๏ฟฝp =
๏ฟฝ
µ1 + µ2
172
Discrete Distributions
Poisson
6. Bivariate and Multivariate
Distributions
Bivariate Normal Distribution
173
6.1. Bivariate Normal Continuous
Distribution
Probability Density Function - f(x,y)
y
x
x
y
x
x
y
๐œƒ๐œƒ = 45๐‘œ๐‘œ
๐œƒ๐œƒ = 0๐‘œ๐‘œ
๐œŒ๐œŒ < 0
x
135๐‘œ๐‘œ < ๐œƒ๐œƒ < 180๐‘œ๐‘œ
Adapted from (Kotz et al. 2000, p.256)
x
๐œƒ๐œƒ = 45๐‘œ๐‘œ
x
90๐‘œ๐‘œ < ๐œƒ๐œƒ < 135๐‘œ๐‘œ
Bi-var Normal
x
45๐‘œ๐‘œ < ๐œƒ๐œƒ < 90๐‘œ๐‘œ
y
y
y
x
๐œƒ๐œƒ = 45๐‘œ๐‘œ
๐œƒ๐œƒ = 0๐‘œ๐‘œ
y
๐œŽ๐œŽ๐‘ฅ๐‘ฅ < ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
x
0๐‘œ๐‘œ < ๐œƒ๐œƒ < 45๐‘œ๐‘œ
๐œŒ๐œŒ = 0
๐œŽ๐œŽ๐‘ฅ๐‘ฅ = ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
y
๐œŒ๐œŒ > 0
0
y
y
๐œŽ๐œŽ๐‘ฅ๐‘ฅ > ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
174
Bivariate and Multivariate Distributions
Parameters & Description
Location parameter: The
mean of each random
variable.
โˆ’โˆž < µj < โˆž
๐‘—๐‘— โˆˆ {๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ}
๐œ‡๐œ‡๐‘ฅ๐‘ฅ , ๐œ‡๐œ‡๐‘ฆ๐‘ฆ
๐œŽ๐œŽ๐‘ฅ๐‘ฅ , ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
๐œŽ๐œŽ๐‘—๐‘— > 0
๐‘—๐‘— โˆˆ {๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ}
๐œŒ๐œŒ
โˆ’1 โ‰ค ๐œŒ๐œŒ โ‰ค 1
Parameters
โˆ’โˆž < x < โˆž ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
Limits
Distribution
Scale parameter: The
standard deviation of each
random variable.
Correlation
Coefficient:
The correlation between
the two random variables.
๐‘๐‘๐‘๐‘๐‘๐‘[๐‘‹๐‘‹๐‘‹๐‘‹]
๐œŒ๐œŒ = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘‹๐‘‹, ๐‘Œ๐‘Œ) =
๐œŽ๐œŽ๐‘ฅ๐‘ฅ ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
๐ธ๐ธ[(๐‘‹๐‘‹ โˆ’ ๐œ‡๐œ‡๐‘ฅ๐‘ฅ )(๐‘Œ๐‘Œ โˆ’ ๐œ‡๐œ‡๐‘ฆ๐‘ฆ )]
=
๐œŽ๐œŽ๐‘ฅ๐‘ฅ ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
โˆ’โˆž<y< โˆž
Formulas
1
๐‘“๐‘“(x, y) =
exp ๏ฟฝ
zx 2 + zy 2 โˆ’ 2๐œŒ๐œŒzx zy
๏ฟฝ
โˆ’2(1 โˆ’ ๐œŒ๐œŒ2 )
2๐œ‹๐œ‹ฯƒx ฯƒy ๏ฟฝ1 โˆ’ ๐œŒ๐œŒ2
= ๐œ™๐œ™(๐‘ฅ๐‘ฅ)๐œ™๐œ™(๐‘ฆ๐‘ฆ|๐‘ฅ๐‘ฅ)
๐‘ฆ๐‘ฆ โˆ’ ๐œŒ๐œŒ๐œŒ๐œŒ
๐‘ฅ๐‘ฅ โˆ’ ๐œŒ๐œŒ๐œŒ๐œŒ
= ๐œ™๐œ™(๐‘ฅ๐‘ฅ)๐œ™๐œ™ ๏ฟฝ
๏ฟฝ = ๐œ™๐œ™(๐‘ฆ๐‘ฆ)๐œ™๐œ™ ๏ฟฝ
๏ฟฝ
2
๏ฟฝ1 โˆ’ ๐œŒ๐œŒ
๏ฟฝ1 โˆ’ ๐œŒ๐œŒ2
PDF
Where ๐œ™๐œ™ is the standard normal distribution and:
x โˆ’ µj
zj =
๐‘—๐‘— โˆˆ {๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ}
ฯƒj
Bi-var Normal
โˆž
Marginal PDF
Conditional PDF
CDF
โˆž
๐‘“๐‘“(๐‘ฅ๐‘ฅ) = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) ๐‘‘๐‘‘๐‘‘๐‘‘
๐‘“๐‘“(๐‘ฆ๐‘ฆ) = ๏ฟฝ ๐‘“๐‘“(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ) ๐‘‘๐‘‘๐‘‘๐‘‘
โˆ’โˆž
1
1
=
exp ๏ฟฝ โˆ’ (zx )2 ๏ฟฝ
2
๐œŽ๐œŽ๐‘ฅ๐‘ฅ โˆš2๐œ‹๐œ‹
= ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡๐‘ฅ๐‘ฅ , ๐œŽ๐œŽ๐‘ฅ๐‘ฅ )
โˆ’โˆž
1
2
exp ๏ฟฝ โˆ’ ๏ฟฝzy ๏ฟฝ ๏ฟฝ
2
๐œŽ๐œŽ๐‘ฆ๐‘ฆ โˆš2๐œ‹๐œ‹
= ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐œ‡๐œ‡๐‘ฆ๐‘ฆ , ๐œŽ๐œŽ๐‘ฆ๐‘ฆ ๏ฟฝ
=
1
๐œŽ๐œŽ๐‘ฅ๐‘ฅ
2
๐‘“๐‘“(๐‘ฅ๐‘ฅ|๐‘ฆ๐‘ฆ) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œ‡๐œ‡๐‘ฅ๐‘ฅ|๐‘ฆ๐‘ฆ = ๐œ‡๐œ‡๐‘ฅ๐‘ฅ + ๐œŒ๐œŒ ๏ฟฝ ๏ฟฝ ๏ฟฝ๐‘ฆ๐‘ฆ โˆ’ ๐œ‡๐œ‡๐‘ฆ๐‘ฆ ๏ฟฝ, ๐œŽ๐œŽ๐‘ฅ๐‘ฅ|๐‘ฆ๐‘ฆ
= ๐œŽ๐œŽ๐‘ฅ๐‘ฅ2 (1 โˆ’ ๐œŒ๐œŒ2 )๏ฟฝ
๐œŽ๐œŽ๐‘ฆ๐‘ฆ
๐œŽ๐œŽ๐‘ฆ๐‘ฆ
2
๐‘“๐‘“(๐‘ฆ๐‘ฆ|๐‘ฅ๐‘ฅ) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œ‡๐œ‡๐‘ฆ๐‘ฆ|๐‘ฅ๐‘ฅ = ๐œ‡๐œ‡๐‘ฆ๐‘ฆ + ๐œŒ๐œŒ ๏ฟฝ ๏ฟฝ (๐‘ฆ๐‘ฆ โˆ’ ๐œ‡๐œ‡๐‘ฅ๐‘ฅ ), ๐œŽ๐œŽ๐‘ฆ๐‘ฆ|๐‘ฅ๐‘ฅ
= ๐œŽ๐œŽ๐‘ฆ๐‘ฆ2 (1 โˆ’ ๐œŒ๐œŒ2 )๏ฟฝ
๐œŽ๐œŽ๐‘ฅ๐‘ฅ
๐น๐น(x, y) =
1
2๐œ‹๐œ‹ฯƒx ฯƒy ๏ฟฝ1 โˆ’ ฯ2
x
y
๏ฟฝ ๏ฟฝ exp ๏ฟฝ
โˆ’โˆž โˆ’โˆž
zu2 + zv2 โˆ’ 2ฯzu zv
๏ฟฝ du dv
โˆ’2(1 โˆ’ ฯ2 )
Bivariate Normal Distribution
where
zj =
Reliability
๐‘…๐‘…(x, y) =
where
1
2๐œ‹๐œ‹ฯƒx ฯƒy ๏ฟฝ1 โˆ’ ฯ2
โˆž
x โˆ’ µj
ฯƒj
โˆž
๏ฟฝ ๏ฟฝ exp ๏ฟฝ
x
zj =
y
x โˆ’ µj
ฯƒj
175
zu2 + zv2 โˆ’ 2ฯzu zv
๏ฟฝ du dv
2(1 โˆ’ ฯ2 )
Properties and Moments
Median
Mode
Mean - 1st Raw Moment
๐œ‡๐œ‡๐‘ฅ๐‘ฅ
๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ
๐‘ฆ๐‘ฆ
๐œ‡๐œ‡๐‘ฅ๐‘ฅ
๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ
๐‘ฆ๐‘ฆ
๐œ‡๐œ‡๐‘ฅ๐‘ฅ
๐‘‹๐‘‹
๐ธ๐ธ ๏ฟฝ ๏ฟฝ = ๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ
๐‘ฆ๐‘ฆ
๐‘Œ๐‘Œ
The mean of the marginal distributions is:
๐ธ๐ธ[๐‘‹๐‘‹] = ๐œ‡๐œ‡๐‘ฅ๐‘ฅ
๐ธ๐ธ[๐‘Œ๐‘Œ] = ๐œ‡๐œ‡๐‘ฆ๐‘ฆ
Variance - 2nd Central Moment
The mean of the conditional distributions gives the
following lines (also called the regression lines):
ฯƒx
E(X|Y = y) = µx + ฯ. (y โˆ’ µy )
ฯƒy
ฯƒy
E(Y|X = x) = µy + ฯ. (y โˆ’ µx )
ฯƒx
๐œŽ๐œŽ 2
๐‘‹๐‘‹
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ ๏ฟฝ ๏ฟฝ = ๏ฟฝ 1
๐‘Œ๐‘Œ
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๏ฟฝ
๐œŽ๐œŽ22
100ฮฑ% Percentile Function
Variance of conditional distributions:
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(X|Y = y) = ฯƒ2x (1 โˆ’ ฯ2 )
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐‘Œ๐‘Œ|๐‘‹๐‘‹ = ๐‘ฅ๐‘ฅ) = ฯƒ2y (1 โˆ’ ฯ2 )
An ellipse containing 100ฮฑ % of the distribution is
(Kotz et al. 2000, p.254):
(zx 2 + zy 2 โˆ’ 2ฯzx zy )
= ln(1 โˆ’ ๐›ผ๐›ผ)
โˆ’2(1 โˆ’ ๐œŒ๐œŒ2 )
Bi-var Normal
Variance of marginal distributions:
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(X) = ฯƒ2x
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(Y) = ฯƒ2y
176
Bivariate and Multivariate Distributions
where
zj =
x โˆ’ µj
ฯƒj
๐‘—๐‘— โˆˆ {๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ}
For the standard bivariate normal:
x 2 + y 2 โˆ’ 2ฯxy
= ln(1 โˆ’ ๐›ผ๐›ผ)
โˆ’2(1 โˆ’ ฯ2 )
Parameter Estimation
Maximum Likelihood Function
MLE Point
Estimates
When there is only complete failure data the MLE estimates can be given
as (Kotz et al. 2000, p.294):
๏ฟฝ๐‘ฅ๐‘ฅ =
๐œ‡๐œ‡
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
1
๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘–
nF
2
๏ฟฝ2x = 1 ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๏ฟฝ)
ฯƒ
๐‘ฅ๐‘ฅ
nF
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
1
๐œ‡๐œ‡๏ฟฝ๐‘ฆ๐‘ฆ = ๏ฟฝ ๐‘ฆ๐‘ฆ๐‘–๐‘–
nF
๐‘–๐‘–=1
๐‘–๐‘–=1
๐‘›๐‘›๐น๐น
๐‘›๐‘›๐น๐น
2
๏ฟฝ2y = 1 ๏ฟฝ๏ฟฝ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๏ฟฝ๏ฟฝ
ฯƒ
๐‘ฆ๐‘ฆ
nF
๐‘–๐‘–=1
1
๐œŒ๐œŒ๏ฟฝ =
๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๐‘ฅ๐‘ฅ )๏ฟฝ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๐‘ฆ๐‘ฆ ๏ฟฝ
๐œŽ๐œŽ
๏ฟฝ๐œŽ๐œŽ
๏ฟฝn
๐‘ฅ๐‘ฅ ๐‘ฆ๐‘ฆ F
๐‘–๐‘–=1
If one or more of the variables are known, different estimators are given in
(Kotz et al. 2000, pp.294-305).
๏ฟฝ2 to give the unbiased
A correction factor of -1 can be introduced to the ๐œŽ๐œŽ
estimators:
Bi-var Normal
๏ฟฝ2x =
ฯƒ
๐‘›๐‘›๐น๐น
1
2
๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡
๏ฟฝ)
๐‘ฅ๐‘ฅ
nF โˆ’ 1
๐‘–๐‘–=1
๏ฟฝ2y =
ฯƒ
๐‘›๐‘›๐น๐น
1
2
๏ฟฝ๏ฟฝ๐‘ฆ๐‘ฆ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๏ฟฝ๏ฟฝ
๐‘ฆ๐‘ฆ
nF โˆ’ 1
๐‘–๐‘–=1
Bayesian
Non-informative Priors: A complete coverage of numerous reference prior distributions
with different parameter ordering is contained in (Berger & Sun 2008).
For a summary of the general Bayesian priors and conjugates see the multivariate
normal distribution.
Description , Limitations and Uses
Example
The accuracy of a cutting machine used in manufacturing is desired to
be measured. 5 cuts at the required length are made. The lengths and
room temperature were measured as:
7.436, 10.270, 10.466, 11.039, 11.854 ๐‘š๐‘š๐‘š๐‘š
19.51, 21.23, 21.41, 22.78, 26.78 oC
Bivariate Normal Distribution
MLE estimates are:
177
โˆ‘ xi
= 10.213
n
โˆ‘ ti
= 22.342
๐œ‡๐œ‡๏ฟฝ๐‘‡๐‘‡ =
n
๐œ‡๐œ‡๏ฟฝ๐‘ฅ๐‘ฅ =
2
โˆ‘(xi โˆ’ ๐œ‡๐œ‡
๏ฟฝ)
๐ฟ๐ฟ
= 2.7885
nโˆ’1
2
โˆ‘(t i โˆ’ ๐œ‡๐œ‡๏ฟฝ)
๐‘‡๐‘‡
2
= 7.5033
๐œŽ๐œŽ๏ฟฝ
๐‘‡๐‘‡ =
nโˆ’1
2
๐œŽ๐œŽ๏ฟฝ
๐‘ฅ๐‘ฅ =
๐‘›๐‘›๐น๐น
1
๐œŒ๐œŒ๏ฟฝ =
๏ฟฝ(๐‘ฅ๐‘ฅ๐‘–๐‘– โˆ’ ๐œ‡๐œ‡๐‘ฅ๐‘ฅ )(๐‘ก๐‘ก๐‘–๐‘– โˆ’ ๐œ‡๐œ‡ ๐‘‡๐‘‡ ) = 0.1454
๐œŽ๐œŽ
๏ฟฝ๐œŽ๐œŽ
๏ฟฝn
๐‘ฅ๐‘ฅ ๐‘‡๐‘‡ F
๐‘–๐‘–=1
If you know the temperature is 24 oC what is the likely cutting distance
distribution?
Characteristic
๐œŽ๐œŽ๐‘ฅ๐‘ฅ
2
= ๐œŽ๐œŽ๐‘ฅ๐‘ฅ2 (1 โˆ’ ๐œŒ๐œŒ2 )๏ฟฝ
๐‘“๐‘“(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก = 24) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๐œ‡๐œ‡๐‘ฅ๐‘ฅ|๐‘ก๐‘ก = ๐œ‡๐œ‡๐‘ฅ๐‘ฅ + ๐œŒ๐œŒ ๏ฟฝ ๏ฟฝ (๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡ ๐‘‡๐‘‡ ), ๐œŽ๐œŽ๐‘ฅ๐‘ฅ|๐‘ก๐‘ก
๐œŽ๐œŽ๐‘ก๐‘ก
๐‘“๐‘“(๐‘ฅ๐‘ฅ|๐‘ก๐‘ก = 24) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(10.303, 2.730)
Also known as Binormal Distribution.
Let U, V and W be three independent normally distributed random
variables. Then let:
๐‘‹๐‘‹ = ๐‘ˆ๐‘ˆ + ๐‘‰๐‘‰
๐‘Œ๐‘Œ = ๐‘‰๐‘‰ + ๐‘Š๐‘Š
Then (๐‘‹๐‘‹, ๐‘Œ๐‘Œ) has a bivariate normal distribution. (Balakrishnan & Lai
2009, p.483)
Correlation Coefficient ๐†๐†. (Yang et al. 2004, p.49)
๐†๐† > 0. When X increases then Y also tends to increase. When
๐œŒ๐œŒ = 1 X and Y have a perfect positive linear relationship such
that ๐‘Œ๐‘Œ = ๐‘๐‘ + ๐‘š๐‘š๐‘š๐‘š where ๐‘š๐‘š is positive.
๐†๐† < 0. When X increases then Y also tends to decrease. When
๐œŒ๐œŒ = โˆ’1 X and Y have a perfect negative linear relationship
such that ๐‘Œ๐‘Œ = ๐‘๐‘ + ๐‘š๐‘š๐‘š๐‘š where ๐‘š๐‘š is negative.
๐†๐† = ๐ŸŽ๐ŸŽ. Increases or decreases in X have no affect on Y. X and
Bi-var Normal
Independence. If ๐‘‹๐‘‹ and ๐‘Œ๐‘Œ are jointly normal random variables, then
they are independent when ๐œŒ๐œŒ = 0. This gives a contour plot of ๐‘“๐‘“(๐‘ฅ๐‘ฅ, ๐‘ฆ๐‘ฆ)
with concentric circles around the origin. When given a value on the ๐‘ฆ๐‘ฆ
axis it does not assist in estimating the value on the ๐‘ฅ๐‘ฅ axis and therefore
are independent. When ๐‘‹๐‘‹ and ๐‘Œ๐‘Œ are independent, the pdf reduces to:
zx 2 + zy 2
1
๐‘“๐‘“(x, y) =
exp ๏ฟฝโˆ’
๏ฟฝ
2๐œ‹๐œ‹ฯƒx ฯƒy
2
178
Bivariate and Multivariate Distributions
Y are independent.
Ellipse Axis. (Kotz et al. 2000, p.254) The slope of the main axis from
the x-axis is given as:
2๐œŒ๐œŒ๐œŽ๐œŽ๐‘ฅ๐‘ฅ ๐œŽ๐œŽ๐‘ฆ๐‘ฆ
1
๏ฟฝ
๐œƒ๐œƒ = ๐‘ก๐‘ก๐‘ก๐‘ก๐‘›๐‘›โˆ’1 ๏ฟฝ 2
2
๐œŽ๐œŽ๐‘ฅ๐‘ฅ โˆ’ ๐œŽ๐œŽ๐‘ฆ๐‘ฆ2
If ๐œŽ๐œŽ๐‘ฅ๐‘ฅ = ๐œŽ๐œŽ๐‘ฆ๐‘ฆ for positive ๐œŒ๐œŒ the main axis of the ellipse is 45o from the xaxis. For negative ๐œŒ๐œŒ the main axis of the ellipse is -45o from the x-axis.
Circular Normal Density Function. (Kotz et al. 2000, p.255) When
๐œŽ๐œŽ๐‘ฅ๐‘ฅ = ๐œŽ๐œŽ๐‘ฆ๐‘ฆ and ๐œŒ๐œŒ = 0 the bivariate distribution is known as a circular
normal density function.
Elliptical Normal Distribution (Kotz et al. 2000, p.255). If ๐œŒ๐œŒ = 0 and
๐œŽ๐œŽ๐‘ฅ๐‘ฅ โ‰  ๐œŽ๐œŽ๐‘ฆ๐‘ฆ then the distribution may be known as an elliptical normal
distribution.
Standard Bivariate Normal Distribution. Occurs when ๐œ‡๐œ‡ = 0 and ๐œŽ๐œŽ =
1. For positive ฯ the main axis of the ellipse is 45o from the x-axis. For
negative ฯ the main axis of the ellipse is -45o from the x-axis.
๐‘“๐‘“(x, y) =
1
2๐œ‹๐œ‹๏ฟฝ1 โˆ’ ฯ2
exp ๏ฟฝโˆ’
x 2 + y 2 โˆ’ 2ฯxy
๏ฟฝ
2(1 โˆ’ ฯ2 )
Bi-var Normal
Mean / Median / Mode:
As per the univariate distributions the mean, median and mode are
equal.
Matrix Form. The bivariate distribution may be written in matrix form
as:
๐œ‡๐œ‡1
๐œŽ๐œŽ 2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๐‘‹๐‘‹
๐‘ฟ๐‘ฟ = ๏ฟฝ 1 ๏ฟฝ ๐๐ = ๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ ๐šบ๐šบ = ๏ฟฝ 1
๏ฟฝ
๐‘‹๐‘‹2
2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๐œŽ๐œŽ22
when ๐‘ฟ๐‘ฟ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š2 (๐๐, ๐šบ๐šบ)
๐‘“๐‘“(๐ฑ๐ฑ) =
1
exp ๏ฟฝโˆ’ (๐ฑ๐ฑ โˆ’ ๐›๐›)T ๐šบ๐šบ โˆ’1 (๐ฑ๐ฑ โˆ’ ๐›๐›)๏ฟฝ
2
2๐œ‹๐œ‹๏ฟฝ|๐šบ๐šบ|
1
Where |๐šบ๐šบ| is the determinant of ๐šบ๐šบ. This is the form used in multivariate
normal distribution.
The following properties are given in matrix form:
Convolution Property
๐’€๐’€ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐๐๐’š๐’š , ๐šบ๐šบ๐ฒ๐ฒ ๏ฟฝ
Let
๐‘ฟ๐‘ฟ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐๐๐’™๐’™ , ๐šบ๐šบ๐ฑ๐ฑ )
Where
๐‘ฟ๐‘ฟ โŠฅ ๐’€๐’€ (independent)
Then
๐‘ฟ๐‘ฟ + ๐’€๐’€~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐๐๐’™๐’™ + ๐๐๐’š๐’š , ๐šบ๐šบ๐’™๐’™ + ๐šบ๐šบ๐’š๐’š ๏ฟฝ
Bivariate Normal Distribution
179
Note if X and Y are dependent then ๐‘ฟ๐‘ฟ + ๐’€๐’€ may not be even be normally
distributed.(Novosyolov 2006)
Scaling Property
Let
1 matrix
Then
2 matrix
๐’€๐’€ = ๐‘จ๐‘จ๐‘จ๐‘จ + ๐’ƒ๐’ƒ
๐’€๐’€~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐‘จ๐‘จ๐‘จ๐‘จ + ๐’ƒ๐’ƒ, ๐‘จ๐‘จ๐šบ๐šบ๐‘จ๐‘จ๐‘ป๐‘ป )
Y is a p x
b is a p x 1 matrix
A is a p x
Marginalize Property:
๐œ‡๐œ‡1
๐œŽ๐œŽ 2
๐‘‹๐‘‹
Let
๏ฟฝ 1 ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ , ๏ฟฝ 1
๐‘‹๐‘‹2
2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ22
Conditional Property:
๐œ‡๐œ‡1
๐œŽ๐œŽ 2
๐‘‹๐‘‹
Let
๏ฟฝ 1 ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘ ๏ฟฝ๏ฟฝ๐œ‡๐œ‡ ๏ฟฝ , ๏ฟฝ 1
๐‘‹๐‘‹2
2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๐œŒ๐œŒ๐œŽ๐œŽ1 ๐œŽ๐œŽ2
๏ฟฝ๏ฟฝ
๐œŽ๐œŽ22
Then
Then
Where
๐‘‹๐‘‹1 โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘(๐œ‡๐œ‡1 , ๐œŽ๐œŽ1 )
๐‘“๐‘“(๐‘ฅ๐‘ฅ1 |๐‘ฅ๐‘ฅ2 ) = ๐‘๐‘๐‘๐‘๐‘๐‘๐‘๐‘๏ฟฝ๐œ‡๐œ‡1|2 , ๐œŽ๐œŽ1|2 ๏ฟฝ
๐œŽ๐œŽ
๐œ‡๐œ‡1|2 = ๐œ‡๐œ‡1 + ๐œŒ๐œŒ ๏ฟฝ 1๏ฟฝ (๐‘ฅ๐‘ฅ2 โˆ’ ๐œ‡๐œ‡2 )
๐œŽ๐œŽ2
๐œŽ๐œŽ1|2 = ๐œŽ๐œŽ1 ๏ฟฝ1 โˆ’ ๐œŒ๐œŒ2
It should be noted that the standard deviation of the marginal distribution
does not depend on the given value.
Applications
The bivariate distribution is used in many more applications which are
common to the multivariate normal distribution. Please refer to
multivariate normal distribution for a more complete coverage.
Resources
Online:
http://mathworld.wolfram.com/BivariateNormalDistribution.html
http://en.wikipedia.org/wiki/Multivariate_normal_distribution
http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_binormal_
distri.htm (interactive visual representation)
Books:
Balakrishnan, N. & Lai, C., 2009. Continuous Bivariate Distributions
2nd ed., Springer.
Bi-var Normal
Graphical Representation of Multivariate Normal. As with all
bivariate distributions having only two dependent variables allows it to
be easily graphed (in a three dimensional graph) and visualized. As
such the bivariate normal is popular in introducing higher dimensional
cases.
180
Bivariate and Multivariate Distributions
Yang, K. et al., 2004. Multivariate Statistical Methods in Quality
Management 1st ed., McGraw-Hill Professional.
Patel, J.K, Read, C.B, 1996. Handbook of the Normal Distribution, 2nd
Edition, CRC
Bi-var Normal
Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer.
Dirichlet Distribution
181
6.2. Dirichlet Continuous
Distribution
Probability Density Function - f(x)
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; [2,2,2]๐‘‡๐‘‡ )
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; [10,10,10]๐‘‡๐‘‡ )
๐‘‡๐‘‡
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; [1,1,1]๐‘‡๐‘‡ )
๐‘‡๐‘‡
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; [2,1,2]๐‘‡๐‘‡ )
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; ๏ฟฝ12, 12, 12๏ฟฝ )
Dirichlet
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ2 ([๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 ]๐‘‡๐‘‡ ; ๏ฟฝ12, 1,2๏ฟฝ )
182
Bivariate and Multivariate Distributions
Parameters & Description
๐›‚๐›‚ = [๐›ผ๐›ผ1 , ๐›ผ๐›ผ2 , โ€ฆ , ๐›ผ๐›ผ๐‘‘๐‘‘ , ๐›ผ๐›ผ0 ]๐‘‡๐‘‡
Shape
Matrix. Note
that
the
matrix ๐œถ๐œถ is
๐‘‘๐‘‘ + 1
in
length.
ฮฑi > 0
Dimensions.
The number
of
random
variables
being
modeled.
๐‘‘๐‘‘ โ‰ฅ 1
(integer)
๐‘‘๐‘‘
0 โ‰ค xi โ‰ค 1
๐‘‘๐‘‘
Limits
๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– โ‰ค 1
๐‘–๐‘–=1
Distribution
Formulas
๐›ผ๐›ผ0 โˆ’1
๐‘‘๐‘‘
1
๐‘“๐‘“(๐ฑ๐ฑ) =
๏ฟฝ1 โˆ’ ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– ๏ฟฝ
B(๐›‚๐›‚)
ฮฑ โˆ’1
xi i
i=1
๐‘–๐‘–=1
PDF
d
๏ฟฝ
where B(๐œถ๐œถ) is the multinomial beta function:
โˆdi=0 ฮ“(ฮฑi )
B(๐›‚๐›‚) =
ฮ“๏ฟฝโˆ‘di=0 ฮฑi ๏ฟฝ
Dirichlet
The special case of the Dirichlet distribution is the beta
distribution when ๐‘‘๐‘‘ = 1.
Let
Where
Let
Marginal PDF
๐‘ผ๐‘ผ โˆผ ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘ ๐‘  (๐œถ๐œถ๐’–๐’– )
๐‘“๐‘“(๐ฎ๐ฎ) =
ฮ“๏ฟฝ๐›ผ๐›ผฮฃ โˆ’
๐‘ผ๐‘ผ
๐‘ฟ๐‘ฟ = ๏ฟฝ ๏ฟฝ ~๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ ( ๐œถ๐œถ)
๐‘ฝ๐‘ฝ
๐‘ฟ๐‘ฟ = [๐‘‹๐‘‹1 , โ€ฆ , ๐‘‹๐‘‹๐‘ ๐‘  , ๐‘‹๐‘‹๐‘ ๐‘ +1 , โ€ฆ , ๐‘‹๐‘‹๐‘‘๐‘‘ ]๐‘‡๐‘‡
๐‘ผ๐‘ผ = [๐‘‹๐‘‹1 , โ€ฆ , ๐‘‹๐‘‹๐‘ ๐‘  ]๐‘‡๐‘‡
๐‘ฝ๐‘ฝ = [๐‘‹๐‘‹๐‘ ๐‘ +1 , โ€ฆ , ๐‘‹๐‘‹๐‘‘๐‘‘ ]๐‘‡๐‘‡
๐›ผ๐›ผฮฃ = โˆ‘๐‘‘๐‘‘๐‘—๐‘—=0 ๐›ผ๐›ผ๐‘—๐‘— = sum of ๐œถ๐œถ matrix elements.
where ๐œถ๐œถ๐’–๐’– = ๏ฟฝ๐›ผ๐›ผ1 , ๐›ผ๐›ผ2 , โ€ฆ , ๐›ผ๐›ผ๐‘ ๐‘  , ๐›ผ๐›ผฮฃ โˆ’ โˆ‘๐‘ ๐‘ ๐‘—๐‘—=1 ๐›ผ๐›ผ๐‘—๐‘— ๏ฟฝ
ฮ“(ฮฑฮฃ )
โˆ‘๐‘ ๐‘ ๐‘—๐‘—=1 ๐›ผ๐›ผ๐‘—๐‘— ๏ฟฝ โˆsi=1 ฮ“(ฮฑi )
๐‘ ๐‘ 
๏ฟฝ1 โˆ’ ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– ๏ฟฝ
๐‘–๐‘–=1
๐›ผ๐›ผฮฃโˆ’1โˆ’โˆ‘๐‘ ๐‘ ๐‘—๐‘—=1 ๐›ผ๐›ผ๐‘—๐‘—
s
๏ฟฝ
๐‘‡๐‘‡
ฮฑ โˆ’1
xi i
i=1
Dirichlet Distribution
183
When marginalized to one variable:
๐‘‹๐‘‹๐‘–๐‘– ~๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐›ผ๐›ผ๐‘–๐‘– , ๐›ผ๐›ผฮฃ โˆ’ ๐›ผ๐›ผ๐‘–๐‘– )
๐‘“๐‘“(๐‘ฅ๐‘ฅ๐‘–๐‘– ) =
ฮ“(ฮฑฮฃ )
ฮ“(๐›ผ๐›ผฮฃ โˆ’ ๐›ผ๐›ผ๐‘–๐‘– )ฮ“(ฮฑi )
(1 โˆ’ ๐‘ฅ๐‘ฅ๐‘–๐‘– )๐›ผ๐›ผฮฃ โˆ’๐›ผ๐›ผ๐‘–๐‘– โˆ’1 xiฮฑiโˆ’1
๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’— โˆผ ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘โˆ’๐‘ ๐‘  ๏ฟฝ๐œถ๐œถ๐‘ข๐‘ข|๐’—๐’— ๏ฟฝ where ๐œถ๐œถ๐’–๐’–|๐’—๐’— = [๐›ผ๐›ผ๐‘†๐‘†+1 , ๐›ผ๐›ผ๐‘ ๐‘ +2 , โ€ฆ , ๐›ผ๐›ผ๐‘š๐‘š , ๐›ผ๐›ผ0 ]๐‘‡๐‘‡
(Kotz et al. 2000, p.488)
Conditional PDF
ฮ“๏ฟฝโˆ‘๐‘—๐‘—=0 ๐›ผ๐›ผ๐‘—๐‘— ๏ฟฝ
๐‘ ๐‘ 
๐‘“๐‘“(๐ฎ๐ฎ|๐ฏ๐ฏ) =
๏ฟฝ1
โˆsi=0 ฮ“(ฮฑi )
๐‘ ๐‘ 
โˆ’ ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– ๏ฟฝ
CDF
๐‘ฅ๐‘ฅ2
= ๏ฟฝ ๏ฟฝ โ€ฆ๏ฟฝ
0
0
๐‘ฅ๐‘ฅ๐‘‘๐‘‘
0
๐›ผ๐›ผ0 โˆ’1
๐‘‘๐‘‘
๏ฟฝ1 โˆ’ ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– ๏ฟฝ
๐‘–๐‘–=1
๐‘ ๐‘ 
๏ฟฝ
๐›ผ๐›ผ โˆ’1
๐‘ฅ๐‘ฅ๐‘–๐‘– ๐‘–๐‘–
๐‘–๐‘–=1
๐‘–๐‘–=1
๐น๐น(๐ฑ๐ฑ) = P(X1 โ‰ค x1 , X2 โ‰ค x2 , โ€ฆ , Xd โ‰ค xd )
๐‘ฅ๐‘ฅ1
๐›ผ๐›ผ0 โˆ’1
d
๏ฟฝ
ฮฑ โˆ’1
xi i
i=1
d๐‘‘๐‘‘, โ€ฆ , ๐‘‘๐‘‘๐‘ฅ๐‘ฅ2 , ๐‘‘๐‘‘๐‘ฅ๐‘ฅ1
Numerical methods have been explored to evaluate this integral,
see (Kotz et al. 2000, pp.497-500)
Reliability
๐‘…๐‘…(๐ฑ๐ฑ) = P(X1 > x1 , X2 > x2 , โ€ฆ , Xd > xd )
โˆž
โˆž
๐›ผ๐›ผ0 โˆ’1
๐‘‘๐‘‘
โˆž
= ๏ฟฝ ๏ฟฝ โ€ฆ ๏ฟฝ ๏ฟฝ1 โˆ’ ๏ฟฝ ๐‘ฅ๐‘ฅ๐‘–๐‘– ๏ฟฝ
๐‘ฅ๐‘ฅ1
๐‘ฅ๐‘ฅ2
๐‘ฅ๐‘ฅ๐‘‘๐‘‘
๐‘–๐‘–=1
d
๏ฟฝ
ฮฑ โˆ’1
xi i
i=1
d๐‘‘๐‘‘, โ€ฆ , ๐‘‘๐‘‘๐‘ฅ๐‘ฅ2 , ๐‘‘๐‘‘๐‘ฅ๐‘ฅ1
Properties and Moments
Median
Mode
Mean - 1st Raw Moment
Solve numerically using ๐น๐น(๐’™๐’™) = 0.5
๐‘ฅ๐‘ฅ๐‘–๐‘– =
๐›ผ๐›ผ๐‘–๐‘– โˆ’1
๐›ผ๐›ผฮฃ โˆ’๐‘‘๐‘‘
for ๐›ผ๐›ผ๐‘–๐‘– > 0 otherwise no mode
Let ๐›ผ๐›ผฮฃ = โˆ‘๐‘‘๐‘‘๐‘–๐‘–=0 ๐›ผ๐›ผ๐‘–๐‘– :
๐ธ๐ธ[๐‘ฟ๐‘ฟ] = ๐๐ =
๐œถ๐œถ
๐›ผ๐›ผฮฃ
๐ธ๐ธ[๐‘‹๐‘‹๐‘–๐‘– ] = ๐œ‡๐œ‡๐‘–๐‘– =
๐›ผ๐›ผ๐‘–๐‘–
๐‘Ž๐‘Žฮฃ
where
๐‘ ๐‘ 
๐œถ๐œถ๐’–๐’– = ๏ฟฝ๐›ผ๐›ผ1 , ๐›ผ๐›ผ2 , โ€ฆ , ๐›ผ๐›ผ๐‘ ๐‘  , ๐›ผ๐›ผฮฃ โˆ’ ๏ฟฝ ๐›ผ๐›ผ๐‘—๐‘— ๏ฟฝ
๐‘—๐‘—=1
Mean of the conditional distribution:
๐œถ๐œถ๐’–๐’–|๐’—๐’—
๐ธ๐ธ[๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’—] = ๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ =
๐›ผ๐›ผฮฃ
Dirichlet
Mean of the marginal distribution:
๐œถ๐œถ๐‘ข๐‘ข
๐ธ๐ธ[๐‘ผ๐‘ผ] = ๐๐๐’–๐’– =
๐›ผ๐›ผฮฃ
๐‘‡๐‘‡
184
Bivariate and Multivariate Distributions
where
๐œถ๐œถ๐’–๐’–|๐’—๐’— = [๐›ผ๐›ผ๐‘†๐‘†+1 , ๐›ผ๐›ผ๐‘ ๐‘ +2 , โ€ฆ , ๐›ผ๐›ผ๐‘š๐‘š , ๐›ผ๐›ผ0 ]๐‘‡๐‘‡
Let ๐›ผ๐›ผฮฃ = โˆ‘๐‘‘๐‘‘๐‘–๐‘–=0 ๐›ผ๐›ผ๐‘–๐‘– :
Variance - 2nd Central Moment
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐‘‹๐‘‹๐‘–๐‘– ] =
๐›ผ๐›ผ๐‘–๐‘– (๐›ผ๐›ผฮฃ โˆ’ ๐›ผ๐›ผ๐‘–๐‘– )
๐›ผ๐›ผฮฃ2 (๐›ผ๐›ผฮฃ + 1)
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐‘‹๐‘‹๐‘–๐‘– , ๐‘‹๐‘‹๐‘—๐‘— ๏ฟฝ =
Parameter Estimation
โˆ’๐›ผ๐›ผ๐‘–๐‘– ๐›ผ๐›ผ๐‘—๐‘—
๐›ผ๐›ผฮฃ2 (๐›ผ๐›ผฮฃ + 1)
Maximum Likelihood Function
MLE
Point
Estimates
The MLE estimates of ๐›ผ๐›ผ๏ฟฝ๐šค๐šค can be obtained from n observations of ๐’™๐’™๐’Š๐’Š by
numerically maximizing the log-likelihood function: (Kotz et al. 2000,
p.505)
๐‘‘๐‘‘
๐‘‘๐‘‘
๐‘›๐‘›
๐‘—๐‘—=0
๐‘—๐‘—=0
๐‘–๐‘–=1
1
ฮ›(๐›‚๐›‚|E) = ๐‘›๐‘› ๏ฟฝ๐‘™๐‘™๐‘™๐‘™ฮ“(๐›ผ๐›ผฮฃ ) โˆ’ ๏ฟฝ ln ๐›ค๐›ค๏ฟฝ๐›ผ๐›ผ๐‘—๐‘— ๏ฟฝ๏ฟฝ + ๐‘›๐‘› ๏ฟฝ ๏ฟฝ ๏ฟฝ๐›ผ๐›ผ๐‘—๐‘— โˆ’ 1๏ฟฝ ๏ฟฝ ln๏ฟฝ๐‘ฅ๐‘ฅ๐‘–๐‘–๐‘–๐‘– ๏ฟฝ๏ฟฝ
๐‘›๐‘›
The method of moments are used to provide initial guesses of ๐›ผ๐›ผ๐‘–๐‘– for the
numerical methods.
Fisher
Information
Matrix
๐‘‘๐‘‘ 2
๐ผ๐ผ๐‘–๐‘–๐‘–๐‘– = โˆ’๐‘›๐‘›๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผฮฃ ),
๐‘–๐‘– โ‰  ๐‘—๐‘—
๐ผ๐ผ๐‘–๐‘–๐‘–๐‘– = ๐‘›๐‘›๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผ๐‘–๐‘– ) โˆ’ ๐‘›๐‘›๐œ“๐œ“ โ€ฒ (๐›ผ๐›ผฮฃ )
Where ๐œ“๐œ“ โ€ฒ (๐‘ฅ๐‘ฅ) = 2 ๐‘™๐‘™๐‘™๐‘™ฮ“(๐‘ฅ๐‘ฅ) is the trigamma function. See section 1.6.8.
๐‘‘๐‘‘๐‘ฅ๐‘ฅ
(Kotz et al. 2000, p.506)
100๐›พ๐›พ%
Confidence
Intervals
The confidence intervals can be obtained from the fisher information
matrix.
Dirichlet
Bayesian
Non-informative Priors
Jefferyโ€™s Prior
๏ฟฝdet๏ฟฝ๐ผ๐ผ(๐œถ๐œถ)๏ฟฝ
where ๐ผ๐ผ(๐œถ๐œถ) is given above.
Conjugate Priors
UOI
๐’‘๐’‘
from
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘ (๐’Œ๐’Œ; ๐‘›๐‘›๐‘ก๐‘ก , ๐’‘๐’‘)
Likelihood
Model
Multinomiald
Evidence
๐‘˜๐‘˜๐‘–๐‘–,๐‘—๐‘— failures in
๐‘›๐‘› trials with ๐‘‘๐‘‘
possible
states.
Dist of
UOI
Prior
Para
Posterior
Parameters
Dirichletd+1
๐œถ๐œถ๐’๐’
๐œถ๐œถ = ๐œถ๐œถ๐’๐’ + ๐’Œ๐’Œ
Dirichlet Distribution
185
Description , Limitations and Uses
Example
Five machines are measured for performance on demand. The
machines can either fail, partially fail or success in their application. The
machines are tested for 10 demands with the following data for each
machine:
Machine/Trail
1
2
3
4
5
๐œ‡๐œ‡๐‘–๐‘–
1
2
3
4
5
F=3
P=2
F=2
P=2
F=2
P=3
F=3
P=3
F=2
P=3
๐‘›๐‘›๐‘๐‘
๏ฟฝ๐น๐น
๐‘›๐‘›๐‘๐‘
๏ฟฝ๐‘ƒ๐‘ƒ
6
7
8
9
S=5
S=6
S=5
S=4
S=5
๐‘›๐‘›๐‘๐‘
๏ฟฝ๐น๐น
10
Estimate the multinomial distribution parameter ๐’‘๐’‘ = [๐‘๐‘๐น๐น , ๐‘๐‘๐‘ƒ๐‘ƒ , ๐‘๐‘๐‘†๐‘† ]:
Using a non-informative improper prior ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ3 (0,0,0) after updating:
๐‘๐‘๐น๐น
๐’™๐’™ = ๏ฟฝ๐‘๐‘๐‘ƒ๐‘ƒ ๏ฟฝ
๐‘๐‘๐‘†๐‘†
Characteristic
12
๐œถ๐œถ = ๏ฟฝ13๏ฟฝ
25
๐‘๐‘
๏ฟฝ๐น๐น = 12
50
๏ฟฝ๐‘ƒ๐‘ƒ = 13
๐ธ๐ธ[๐’™๐’™] = ๏ฟฝ๐‘๐‘
๏ฟฝ
50
๐‘๐‘
๏ฟฝ๐‘†๐‘† = 25
50
7.15๐ธ๐ธโˆ’5
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐’™๐’™] = ๏ฟฝ 7.54๐ธ๐ธโˆ’5 ๏ฟฝ
9.80๐ธ๐ธโˆ’5
Confidence intervals for the parameters ๐’‘๐’‘ = [๐‘๐‘๐น๐น , ๐‘๐‘๐‘ƒ๐‘ƒ , ๐‘๐‘๐‘†๐‘† ] can also be
calculated using the cdf of the marginal distribution ๐น๐น(๐‘ฅ๐‘ฅ๐‘–๐‘– ).
Beta Generalization. The Dirichlet distribution is a generalization of the
beta distribution. The beta distribution is seen when ๐‘‘๐‘‘ = 1.
๐œถ๐œถ Interpretation. The higher ๐›ผ๐›ผ๐‘–๐‘– the sharper and more certain the
distribution is. This follows from its use in Bayesian statistics to model
the multinomial distribution parameter ๐‘๐‘. As more evidence is used, the
๐›ผ๐›ผ๐‘–๐‘– values get higher which reduces uncertainty. The values of ๐›ผ๐›ผ๐‘–๐‘– can
also be interpreted as a count for each state of the multinomial
distribution.
This formulation is popular because it is a more simple presentation
where the matrix of ๐œถ๐œถ and ๐’™๐’™ are the same size. However it should be
noted that last term of the vector ๐’™๐’™ is dependent on {๐‘ฅ๐‘ฅ1 โ€ฆ ๐‘ฅ๐‘ฅ๐‘š๐‘šโˆ’1 } through
the relationship ๐‘ฅ๐‘ฅ๐‘š๐‘š = 1 โˆ’ โˆ‘๐‘š๐‘šโˆ’1
๐‘–๐‘–=1 ๐‘ฅ๐‘ฅ๐‘–๐‘– .
Neutrality. (Kotz et al. 2000, p.500) If ๐‘‹๐‘‹1 and ๐‘‹๐‘‹2 are non negative
Dirichlet
Alternative Formulation. The most common formulation of the
Dirichlet distribution is as follows:
๐›‚๐›‚ = [๐›ผ๐›ผ1 , ๐›ผ๐›ผ2 , โ€ฆ , ๐›ผ๐›ผ๐‘š๐‘š ]๐‘‡๐‘‡ where ฮฑi > 0
๐ฑ๐ฑ = [๐‘ฅ๐‘ฅ1 , ๐‘ฅ๐‘ฅ2 , โ€ฆ , ๐‘ฅ๐‘ฅ๐‘š๐‘š ]๐‘‡๐‘‡ where 0 โ‰ค xi โ‰ค 1, โˆ‘๐‘š๐‘š
๐‘–๐‘–=1 ๐‘ฅ๐‘ฅ๐‘–๐‘– = 1
m
1
ฮฑi โˆ’1
๐‘“๐‘“(๐ฑ๐ฑ) =
๏ฟฝ xi
B(๐›‚๐›‚)
i=1
186
Bivariate and Multivariate Distributions
random variables such that ๐‘‹๐‘‹1 + ๐‘‹๐‘‹2 โ‰ค 1 then ๐‘‹๐‘‹๐‘–๐‘– is called neutral if the
following are independent:
๐‘‹๐‘‹๐‘—๐‘—
(๐‘–๐‘– โ‰  ๐‘—๐‘—)
๐‘‹๐‘‹๐‘–๐‘– โŠฅ
1 โˆ’ ๐‘‹๐‘‹๐‘–๐‘–
If ๐‘ฟ๐‘ฟ โˆผ ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ (๐œถ๐œถ) then ๐‘ฟ๐‘ฟ is a neutral vector with each Xi being neutral under
all permutations of the above definition. This property is unique to the
Dirichlet distribution.
Applications
Bayesian Statistics. The Dirichlet distribution is often used as a
conjugate prior to the multinomial likelihood function.
Resources
Online:
http://en.wikipedia.org/wiki/Dirichlet_distribution
http://www.cis.hut.fi/ahonkela/dippa/node95.html
Books:
Kotz, S., Balakrishnan, N. & Johnson, N.L., 2000. Continuous
Multivariate Distributions, Volume 1, Models and Applications, 2nd
Edition 2nd ed., Wiley-Interscience.
Congdon, P., 2007. Bayesian Statistical Modelling 2nd ed., Wiley.
MacKay, D.J. & Petoy, L.C., 1995. A hierarchical Dirichlet language
model. Natural language engineering.
Relationship to Other Distributions
Beta
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ, ๐›ฝ๐›ฝ)
Dirichlet
Gamma
Distribution
๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐‘ฅ๐‘ฅ; ๐œ†๐œ†, ๐‘˜๐‘˜)
Special Case:
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘=1 (x; [ฮฑ1 , ฮฑ0 ]) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜ = ๐‘ฅ๐‘ฅ; ๐›ผ๐›ผ = ๐›ผ๐›ผ1 , ๐›ฝ๐›ฝ = ๐›ผ๐›ผ0 )
Let:
Then:
Let:
Then:
๐‘Œ๐‘Œ๐‘–๐‘– ~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†, ๐‘˜๐‘˜๐‘–๐‘– ) ๐‘–๐‘–. ๐‘–๐‘–. ๐‘‘๐‘‘ ๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž
๐‘‰๐‘‰~๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ๐บ(๐œ†๐œ†, โˆ‘๐‘˜๐‘˜๐‘–๐‘– )
๐‘Œ๐‘Œ1 ๐‘Œ๐‘Œ2
๐‘Œ๐‘Œ๐‘‘๐‘‘
๐’๐’ = ๏ฟฝ , , โ€ฆ , ๏ฟฝ
๐‘‰๐‘‰ ๐‘‰๐‘‰
๐‘‰๐‘‰
๐’๐’~๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘ (๐›ผ๐›ผ1 , โ€ฆ , ๐›ผ๐›ผ๐‘˜๐‘˜ )
*i.i.d: independent and identically distributed
๐‘‘๐‘‘
๐‘‰๐‘‰ = ๏ฟฝ ๐‘Œ๐‘Œ๐‘–๐‘–
๐‘–๐‘–=1
Multivariate Normal Distribution
187
6.3. Multivariate Normal Continuous
Distribution
*Note for a graphical representation see bivariate normal distribution
Parameters & Description
๐๐ = [๐œ‡๐œ‡1 , ๐œ‡๐œ‡2 , โ€ฆ , ๐œ‡๐œ‡๐‘‘๐‘‘ ]๐‘‡๐‘‡
Parameters
โ‹ฏ
โ‹ฑ
โ‹ฏ
๐œŽ๐œŽ11
โˆ‘=๏ฟฝ โ‹ฎ
๐œŽ๐œŽ๐‘‘๐‘‘1
โˆ’โˆž < µi < โˆž
๐œŽ๐œŽ1๐‘‘๐‘‘
โ‹ฎ ๏ฟฝ
๐œŽ๐œŽ๐‘‘๐‘‘๐‘‘๐‘‘
๐œŽ๐œŽ๐‘–๐‘–๐‘–๐‘– > 0
๐œŽ๐œŽ๐‘–๐‘–๐‘–๐‘– โ‰ฅ 0
๐‘‘๐‘‘ โ‰ฅ 2
(integer)
๐‘‘๐‘‘
Limits
Location Vector: A ddimensional
vector
giving the mean of each
random variable.
Covariance Matrix:
A
๐‘‘๐‘‘ × ๐‘‘๐‘‘
matrix
which
quantifies the random
variable variance and
dependence. This matrix
determines the shape of
the distribution. ฮฃ is
symmetric
positive
definite matrix.
Dimensions. The number
of dependent variables.
โˆ’โˆž < xi < โˆž
Distribution
Formulas
๐‘“๐‘“(๐ฑ๐ฑ) =
PDF
1
(2๐œ‹๐œ‹)๐‘‘๐‘‘/2 ๏ฟฝ|๐šบ๐šบ|
1
exp ๏ฟฝโˆ’ (๐ฑ๐ฑ โˆ’ ๐›๐›)T ๐šบ๐šบ โˆ’1 (๐ฑ๐ฑ โˆ’ ๐›๐›)๏ฟฝ
2
Where |๐šบ๐šบ| is the determinant of ๐šบ๐šบ.
Where
๐‘ผ๐‘ผ = ๏ฟฝ๐‘‹๐‘‹1 , โ€ฆ , ๐‘‹๐‘‹๐‘๐‘ ๏ฟฝ
๐‘ฝ๐‘ฝ = ๏ฟฝ๐‘‹๐‘‹๐‘๐‘+1 , โ€ฆ , ๐‘‹๐‘‹๐‘‘๐‘‘ ๏ฟฝ
Marginal PDF
๐‘“๐‘“(๐’–๐’–) = ๏ฟฝ ๐‘“๐‘“(๐’™๐’™) ๐‘‘๐‘‘๐’—๐’—
=
โˆ’โˆž
๐‘‡๐‘‡
๐‘ผ๐‘ผ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘๐‘ (๐๐๐’–๐’– , ๐šบ๐šบ๐’–๐’–๐’–๐’– )
โˆž
Conditional PDF
๐‘‡๐‘‡
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๏ฟฝ๏ฟฝ
๐šบ๐šบ๐’—๐’—๐’—๐’—
1
(2๐œ‹๐œ‹)๐‘๐‘/2 ๏ฟฝ|๐šบ๐šบ๐’–๐’–๐’–๐’– |
1
โˆ’1 (๐ฎ๐ฎ โˆ’ ๐๐ )๏ฟฝ
exp ๏ฟฝโˆ’ (๐ฎ๐ฎ โˆ’ ๐๐๐’–๐’– )T ๐šบ๐šบuu
๐’–๐’–
2
๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’— โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘๐‘ ๏ฟฝ๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ , ๐šบ๐šบ๐‘ข๐‘ข|๐‘ฃ๐‘ฃ ๏ฟฝ
Multivar Normal
๐๐๐‘ข๐‘ข ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ผ๐‘ผ
๐‘ฟ๐‘ฟ = ๏ฟฝ ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ ๐‘‡๐‘‡
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฝ๐‘ฝ
๐’—๐’—
๐‘‡๐‘‡
๐‘ฟ๐‘ฟ = ๏ฟฝ๐‘‹๐‘‹1 , โ€ฆ , ๐‘‹๐‘‹๐‘๐‘ , ๐‘‹๐‘‹๐‘๐‘+1 , โ€ฆ , ๐‘‹๐‘‹๐‘‘๐‘‘ ๏ฟฝ
Let
188
Bivariate and Multivariate Distributions
Where
CDF
๐น๐น(๐ฑ๐ฑ) =
Reliability
๐‘…๐‘…(๐ฑ๐ฑ) =
๐‘‡๐‘‡ ๐šบ๐šบ โˆ’1 (๐’—๐’— โˆ’ ๐๐ )
๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐๐๐‘ข๐‘ข + ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ
๐‘ฃ๐‘ฃ
๐‘‡๐‘‡ ๐šบ๐šบ โˆ’1 ๐šบ๐šบ
๐šบ๐šบ๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข โˆ’ ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ ๐‘ข๐‘ข๐‘ข๐‘ข
๐ฑ๐ฑ
1
๏ฟฝ exp ๏ฟฝโˆ’ (๐ฑ๐ฑ โˆ’ ๐›๐›)T ๐šบ๐šบโˆ’1 (๐ฑ๐ฑ โˆ’ ๐›๐›)๏ฟฝ d๐’™๐’™
๐‘‘๐‘‘/2
2
(2๐œ‹๐œ‹) ๏ฟฝ|๐šบ๐šบ| โˆ’โˆž
1
โˆž
1
๏ฟฝ exp ๏ฟฝโˆ’ (๐ฑ๐ฑ โˆ’ ๐›๐›)T ๐šบ๐šบ โˆ’1 (๐ฑ๐ฑ โˆ’ ๐›๐›)๏ฟฝ d๐’™๐’™
2
(2๐œ‹๐œ‹)๐‘‘๐‘‘/2 ๏ฟฝ|๐šบ๐šบ| ๐ฑ๐ฑ
1
Properties and Moments
๐๐
Median
๐๐
Mode
Mean -
1st
๐ธ๐ธ[๐‘ฟ๐‘ฟ] = ๐๐
Raw Moment
Mean of the marginal distribution:
๐ธ๐ธ[๐‘ผ๐‘ผ] = ๐๐๐‘ข๐‘ข
๐ธ๐ธ[๐‘ฝ๐‘ฝ] = ๐๐๐‘ฃ๐‘ฃ
Variance - 2nd Central Moment
Mean of the conditional distribution:
๐‘‡๐‘‡ ๐šบ๐šบ โˆ’1 (๐’—๐’— โˆ’ ๐๐ )
๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐๐๐‘ข๐‘ข + ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ
๐‘ฃ๐‘ฃ
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ[๐‘ฟ๐‘ฟ] = ๐šบ๐šบ
Covariance of marginal distributions:
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐”๐”) = ๐šบ๐šบ๐ฎ๐ฎ๐ฎ๐ฎ
Covariance of conditional distributions:
๐‘‡๐‘‡ ๐šบ๐šบ โˆ’1 ๐šบ๐šบ
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ(๐”๐”|๐•๐•) = ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข โˆ’ ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ ๐‘ข๐‘ข๐‘ข๐‘ข
Multivar Normal
Parameter Estimation
Maximum Likelihood Function
MLE
Point
Estimates
When given complete data of ๐‘›๐‘›๐น๐น samples:
๐’™๐’™๐’•๐’• = ๏ฟฝ๐’™๐’™1,๐‘ก๐‘ก , ๐’™๐’™2,๐‘ก๐‘ก , โ€ฆ , ๐’™๐’™๐‘‘๐‘‘,๐‘ก๐‘ก ๏ฟฝ
๐‘ป๐‘ป
where ๐‘ก๐‘ก = (1,2, โ€ฆ , ๐‘›๐‘›๐น๐น )
The following MLE estimates are given: (Kotz et al. 2000, p.161)
๐‘›๐‘›๐น๐น
1
๐›๐›
๏ฟฝ = ๏ฟฝ ๐’™๐’™๐‘ก๐‘ก
nF
๐‘›๐‘›๐น๐น
๐‘ก๐‘ก=1
1
ฮฃ๏ฟฝ๐‘–๐‘–๐‘–๐‘– = ๏ฟฝ๏ฟฝ๐‘ฅ๐‘ฅ๐‘–๐‘–,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šค๐šค ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘—๐‘—,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šฅ๐šฅ )
nF
๐‘ก๐‘ก=1
A review of different estimators is given in (Kotz et al. 2000). When
estimates are from a low number of samples (๐‘›๐‘›๐น๐น < 30) a correction
Multivariate Normal Distribution
189
factor of -1 can be introduced to give the unbiased estimators (Tong
1990, p.53):
๐‘›๐‘›๐น๐น
1
ฮฃ๏ฟฝ๐‘–๐‘–๐‘–๐‘– =
๏ฟฝ๏ฟฝ๐‘ฅ๐‘ฅ๐‘–๐‘–,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šค๐šค ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘—๐‘—,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šฅ๐šฅ )
nF โˆ’ 1
๐‘ก๐‘ก=1
Fisher
Information
Matrix
๐ผ๐ผ๐‘–๐‘–,๐‘—๐‘— =
Bayesian
Type
Non-informative Priors when ๐šบ๐šบ is known, ๐œ‹๐œ‹0 (๐๐)
(Yang and Berger 1998, p.22)
Prior
(๐๐๐‘‡๐‘‡ ๐šบ๐šบ โˆ’1 ๐๐๐‘‡๐‘‡ )โˆ’(๐‘‘๐‘‘โˆ’2)
Shrinkage
Uniform Improper
Prior with limits
๐šบ๐šบ โˆˆ (0, โˆž)
Jefferyโ€™s Prior
Posterior
1
Uniform
Improper, Jeffrey,
Reference Prior
Type
๐œ•๐œ•๐œ‡๐œ‡๐‘‡๐‘‡ โˆ’1 ๐œ•๐œ•๐œ•๐œ•
ฮฃ
๐œ•๐œ•๐œƒ๐œƒ๐‘–๐‘–
๐œ•๐œ•๐œƒ๐œƒ๐‘—๐‘—
MDIP
๐‘ก๐‘ก=1
No Closed Form
Prior
Posterior
1
1
๐œ‹๐œ‹๏ฟฝ๐šบ๐šบ โˆ’๐Ÿ๐Ÿ ๏ฟฝ๐‘ฌ๐‘ฌ๏ฟฝ~
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Šโ„Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘ก๐‘ก๐‘‘๐‘‘ ๏ฟฝ๐šบ๐šบ โˆ’๐Ÿ๐Ÿ ; ๐‘›๐‘›๐น๐น โˆ’ ๐‘‘๐‘‘ โˆ’ 1,
๐œ‹๐œ‹๏ฟฝ๐šบ๐šบ โˆ’๐Ÿ๐Ÿ ๏ฟฝ๐‘ฌ๐‘ฌ๏ฟฝ~
๐‘บ๐‘บโˆ’1
๏ฟฝ
๐‘›๐‘›๐น๐น
๐‘บ๐‘บโˆ’1
๏ฟฝ
๐‘›๐‘›๐น๐น
with limits ๐šบ๐šบ โˆˆ (0, โˆž)
๐‘Š๐‘Š๐‘Š๐‘Š๐‘Š๐‘Šโ„Ž๐‘Ž๐‘Ž๐‘Ž๐‘Ž๐‘ก๐‘ก๐‘‘๐‘‘ ๏ฟฝ๐šบ๐šบโˆ’๐Ÿ๐Ÿ ; ๐‘›๐‘›๐น๐น ,
1
|๐šบ๐šบ| โˆi<๐‘—๐‘—(ฮปi โˆ’ ฮปj )
Proper - No Closed Form
1
|๐šบ๐šบ|
No Closed Form
1
|๐šบ๐šบ|(logฮป1 โˆ’ logฮปd )dโˆ’2 โˆi<๐‘—๐‘—(ฮปi โˆ’ ฮปj )
Proper - No Closed Form
Non-informative Priors when ๐๐ and ๐šบ๐šบ are unknown for bivariate normal, ๐œ‹๐œ‹๐‘œ๐‘œ (๐๐, ๐šบ๐šบ). A
complete coverage of numerous reference prior distributions with different parameter
ordering is contained in (Berger & Sun 2008)
Multivar Normal
Reference Prior
Ordered
{๐œ†๐œ†1 , ๐œ†๐œ†๐‘‘๐‘‘ , ๐œ†๐œ†๐‘–๐‘– , . . , ๐œ†๐œ†๐‘‘๐‘‘โˆ’1 }
when ๐๐ โˆˆ (โˆž, โˆž)
Non-informative Priors when ๐๐ is known, ๐œ‹๐œ‹๐‘œ๐‘œ (๐šบ๐šบ)
(Yang & Berger 1994)
d+1
|๐šบ๐šบ| 2
Reference Prior
Ordered
{๐œ†๐œ†๐‘–๐‘– , ๐œ†๐œ†๐‘—๐‘— , . . , ๐œ†๐œ†๐‘‘๐‘‘ }
๐‘›๐‘›๐น๐น
1
๐šบ๐šบ
๐œ‹๐œ‹(๐๐|๐‘ฌ๐‘ฌ)~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝµ; ๏ฟฝ ๐’™๐’™๐‘ก๐‘ก , ๏ฟฝ
nF
nF
190
Bivariate and Multivariate Distributions
Type
Prior
Posterior
1
Uniform Improper
Prior
1
No Closed Form
1
|๐šบ๐šบ| โˆi<๐‘—๐‘—(ฮปi โˆ’ ฮปj )
No Closed Form
1
|๐šบ๐šบ|
No Closed Form
Jefferyโ€™s Prior
Reference Prior
Ordered
{๐œ†๐œ†๐‘–๐‘– , ๐œ†๐œ†๐‘—๐‘— , . . , ๐œ†๐œ†๐‘‘๐‘‘ }
Reference Prior
Ordered
{๐œ†๐œ†1 , ๐œ†๐œ†๐‘‘๐‘‘ , ๐œ†๐œ†๐‘–๐‘– , . . , ๐œ†๐œ†๐‘‘๐‘‘โˆ’1 }
MDIP
No Closed Form
d+1
|๐šบ๐šบ| 2
1
|๐šบ๐šบ|(logฮป1 โˆ’ logฮปd )dโˆ’2 โˆi<๐‘—๐‘—(ฮปi โˆ’ ฮปj )
No Closed Form
where ๐œ†๐œ†๐‘–๐‘– is the ๐‘–๐‘–๐‘ก๐‘กโ„Ž eigenvalue of ฮฃ, and ๐‘…๐‘…๏ฟฝ and ๐‘…๐‘… are population and sample multiple
correlation coefficients where:
๐‘›๐‘›๐น๐น
1
๏ฟฝ๏ฟฝ๐‘ฅ๐‘ฅ๐‘–๐‘–,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šค๐šค ๏ฟฝ(๐‘ฅ๐‘ฅ๐‘—๐‘—,๐‘ก๐‘ก โˆ’ ๐œ‡๐œ‡๏ฟฝ๐šฅ๐šฅ )
S๐‘–๐‘–๐‘–๐‘– =
nF โˆ’ 1
๐‘ก๐‘ก=1
and
๐‘›๐‘›๐น๐น
1
๐ฑ๐ฑ๏ฟฝ = ๏ฟฝ ๐’™๐’™๐‘ก๐‘ก
nF
๐‘ก๐‘ก=1
Conjugate Priors
UOI
Multivar Normal
๐๐
from
๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ (๐๐, ๐šบ๐šบ)
Likelihood
Model
Multi-variate
Normal with
known ๐šบ๐šบ
Evidenc
e
๐‘›๐‘›๐น๐น events
at ๐’™๐’™
points
Dist of
UOI
Multivariate
Normal
Prior
Para
๐‘ผ๐‘ผ๐ŸŽ๐ŸŽ , ๐•๐•๐ŸŽ๐ŸŽ
Description , Limitations and Uses
Posterior
Parameters
๐‘ผ๐‘ผ =
๐•๐•0โˆ’1 ๐”๐”๐จ๐จ + nF ๐•๐• โˆ’1 ๐ฑ๐ฑ๏ฟฝ
๐•๐•0โˆ’1 + nF ๐šบ๐šบโˆ’1
๐•๐• =
๐•๐•0โˆ’1
1
+ nF ๐šบ๐šบ โˆ’๐Ÿ๐Ÿ
Example
See bivariate normal distribution.
Characteristic
Standard Spherical Normal Distribution. When ๐๐ = 0, ๐šบ๐šบ = ๐ผ๐ผ we
obtain the standard spherical normal distribution:
1
1
๐‘“๐‘“(๐ฑ๐ฑ) =
exp ๏ฟฝโˆ’ ๐ฑ๐ฑ T ๐ฑ๐ฑ๏ฟฝ
(2๐œ‹๐œ‹)๐‘‘๐‘‘/2
2
Covariance Matrix. (Yang et al. 2004, p.49)
Diagonal Elements. The diagonal elements of ฮฃ is the
variance of each random variable. ๐œŽ๐œŽ๐‘–๐‘–๐‘–๐‘– = ๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰(๐‘‹๐‘‹๐‘–๐‘– )
Non Diagonal Elements. Non diagonal elements give the
covariance ๐œŽ๐œŽ๐‘–๐‘–๐‘–๐‘– = ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐‘‹๐‘‹๐‘–๐‘– , ๐‘‹๐‘‹๐‘—๐‘— ๏ฟฝ = ๐œŽ๐œŽ๐‘—๐‘—๐‘—๐‘— . Hence the matrix is
symmetric.
Independent Variables. If ๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐‘‹๐‘‹๐‘–๐‘– , ๐‘‹๐‘‹๐‘—๐‘— ๏ฟฝ = ๐œŽ๐œŽ๐‘–๐‘–๐‘–๐‘– = 0 then ๐‘‹๐‘‹๐‘–๐‘– and
Multivariate Normal Distribution
-
191
๐‘‹๐‘‹๐‘—๐‘— and independent.
๐ˆ๐ˆ๐’Š๐’Š๐’Š๐’Š > 0. When ๐‘‹๐‘‹๐‘–๐‘– increases then ๐‘‹๐‘‹๐‘—๐‘— and tends to increase.
๐ˆ๐ˆ๐’Š๐’Š๐’Š๐’Š < 0. When ๐‘‹๐‘‹๐‘–๐‘– increases then ๐‘‹๐‘‹๐‘—๐‘— and tends to decrease.
Ellipsoid Axis. The ellipsoids has axes pointing in the direction of the
eigenvectors of ๐šบ๐šบ. The magnitude of these axes are given by the
corresponding eigenvalues.
Mean / Median / Mode:
As per the univariate distributions the mean, median and mode are
equal.
Convolution Property
๐’€๐’€ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๐๐๐’š๐’š , ๐šบ๐šบ๐ฒ๐ฒ ๏ฟฝ
Let
๐‘ฟ๐‘ฟ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ (๐๐๐’™๐’™ , ๐šบ๐šบ๐ฑ๐ฑ )
Where
๐‘ฟ๐‘ฟ โŠฅ ๐’€๐’€ (independent)
Then
๐‘ฟ๐‘ฟ + ๐’€๐’€~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๐๐๐’™๐’™ + ๐๐๐’š๐’š , ๐šบ๐šบ๐’™๐’™ + ๐šบ๐šบ๐’š๐’š ๏ฟฝ
Note if X and Y are dependent then X + Y may not be normally
distributed. (Novosyolov 2006)
Scaling Property
Let
Then
๐’€๐’€ = ๐‘จ๐‘จ๐‘จ๐‘จ + ๐’ƒ๐’ƒ
Y is a p x 1 matrix
b is a p x 1 matrix
๐’€๐’€~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ (๐‘จ๐‘จ๐‘จ๐‘จ + ๐’ƒ๐’ƒ, ๐‘จ๐‘จ๐šบ๐šบ๐‘จ๐‘จ๐‘ป๐‘ป )
A is a p x d matrix
Marginalize Property:
Let
Then
๐๐๐‘ข๐‘ข ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ผ๐‘ผ
๐‘ฟ๐‘ฟ = ๏ฟฝ ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๏ฟฝ๐๐ ๏ฟฝ , ๏ฟฝ ๐‘‡๐‘‡
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฝ๐‘ฝ
๐’—๐’—
๐‘ผ๐‘ผ โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘๐‘ (๐๐๐’–๐’– , ๐šบ๐šบ๐’–๐’–๐’–๐’– )
Conditional Property:
Let
Where
๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’— โˆผ ๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘๐‘ ๏ฟฝ๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ , ๐šบ๐šบ๐‘ข๐‘ข|๐‘ฃ๐‘ฃ ๏ฟฝ
๐‘ผ๐‘ผ is a p x 1 matrix
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๏ฟฝ๏ฟฝ
๐šบ๐šบ๐’—๐’—๐’—๐’—
๐‘‡๐‘‡ โˆ’1
๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐๐๐‘ข๐‘ข + ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐šบ๐šบ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ (๐‘ฝ๐‘ฝ โˆ’ ๐๐๐‘ฃ๐‘ฃ )
๐‘‡๐‘‡ โˆ’1
๐šบ๐šบ๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข โˆ’ ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐šบ๐šบ๐‘ฃ๐‘ฃ๐‘ฃ๐‘ฃ ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ผ๐‘ผ is a p x 1 matrix
It should be noted that the standard deviation of the marginal
distribution does not depend on the given values in V.
Applications
Convenient Properties. (Balakrishnan & Lai 2009, p.477) Popularity
of the multivariate normal distribution over other multivariate
distributions is due to the convenience of the conditional and marginal
distribution properties which both produce univariate normal
distributions.
Multivar Normal
Then
๐๐๐‘ข๐‘ข ๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ผ๐‘ผ
๐‘ฟ๐‘ฟ = ๏ฟฝ ๏ฟฝ ~๐‘๐‘๐‘๐‘๐‘๐‘๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๏ฟฝ ๐๐ ๏ฟฝ , ๏ฟฝ ๐‘‡๐‘‡
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๐‘ฝ๐‘ฝ
๐‘ฃ๐‘ฃ
๐šบ๐šบ๐‘ข๐‘ข๐‘ข๐‘ข
๏ฟฝ๏ฟฝ
๐šบ๐šบ๐’—๐’—๐’—๐’—
192
Bivariate and Multivariate Distributions
Kalman Filter. The Kalman filter estimates the current state of a
system in the presence of noisy measurements. This process uses
multivariate normal distributions to model the noise.
Multivariate Analysis of Variance (MANOVA). A test used to analyze
variance and dependence of variables. A popular model used to
conduct MANOVA assumes the data comes from a multivariate normal
population.
Gaussian Regression Process. This is a statistical model for
observations or events that occur in a continuous domain of time or
space, where every point is associated with a normally distributed
random variable and every finite collection of these random variables
has a multivariate normal distribution.
Multi-Linear Regression. Multi-linear regression attempts to model
the relationship between parameters and variables by fitting a linear
equation. One model to do such a task (MLE) fits a distribution to the
observed variance where a multivariate normal distribution is often
assumed.
Gaussian Bayesian Belief Networks (BBN). BBNs graphical
represent the dependence between variables in a probability
distribution. When using continuous random variables BBNs quickly
become tremendously complicated. However due to the multivariate
normal distributionโ€™s conditional and marginal properties this task is
simplified and popular.
Multivar Normal
Resources
Online:
http://mathworld.wolfram.com/BivariateNormalDistribution.html
http://www.aiaccess.net/English/Glossaries/GlosMod/e_gm_binormal
_distri.htm (interactive visual representation)
Books:
Patel, J.K, Read, C.B, 1996. Handbook of the Normal Distribution, 2nd
Edition, CRC
Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer.
Yang, K. et al., 2004. Multivariate Statistical Methods in Quality
Management 1st ed., McGraw-Hill Professional.
Bertsekas, D.P. & Tsitsiklis, J.N., 2008. Introduction to Probability, 2nd
Edition, Athena Scientific.
Multinomial Distribution
193
6.4. Multinomial Discrete
Distribution
Probability Density Function - f(k)
1 1
5 ๐‘‡๐‘‡
Trinomial Distribution, ๐‘“๐‘“([๐‘˜๐‘˜1 , ๐‘˜๐‘˜2 , ๐‘˜๐‘˜3 ]๐‘‡๐‘‡ ) where ๐‘›๐‘› = 8, ๐ฉ๐ฉ = ๏ฟฝ , , ๏ฟฝ . Note ๐‘˜๐‘˜3 is not shown
3 4 12
because it is determined using ๐‘˜๐‘˜3 = ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜1 โˆ’ ๐‘˜๐‘˜2
Multinomial
1 1 1 ๐‘‡๐‘‡
Trinomial Distribution, ๐‘“๐‘“([๐‘˜๐‘˜1 , ๐‘˜๐‘˜2 , ๐‘˜๐‘˜3 ]๐‘‡๐‘‡ ) where ๐‘›๐‘› = 20, ๐ฉ๐ฉ = ๏ฟฝ , , ๏ฟฝ . Note ๐‘˜๐‘˜3 is not shown
3 2 6
because it is determined as ๐‘˜๐‘˜3 = ๐‘›๐‘› โˆ’ ๐‘˜๐‘˜1 โˆ’ ๐‘˜๐‘˜2
194
Bivariate and Multivariate Distributions
Parameters & Description
๐‘›๐‘›
Parameters
๐ฉ๐ฉ = [๐‘๐‘1 , ๐‘๐‘2 , โ€ฆ , ๐‘๐‘๐‘‘๐‘‘ ]๐‘‡๐‘‡
๐‘‘๐‘‘
n>0
(integer)
Number of Trials. This is
sometimes called the index.
(Johnson et al. 1997, p.31)
0 โ‰ค pi โ‰ค 1
Event Probability Matrix:
The probability of event i
occurring. ๐‘๐‘๐‘–๐‘– is often called
cell probabilities. (Johnson
et al. 1997, p.31)
๐‘‘๐‘‘
๏ฟฝ ๐‘๐‘๐‘–๐‘– = 1
๐‘–๐‘–=1
๐‘‘๐‘‘ โ‰ฅ 2
(integer)
Dimensions. The number of
mutually exclusive states of
the system.
k i โˆˆ {0, โ€ฆ , ๐‘›๐‘›}
๐‘‘๐‘‘
Limits
๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘– = ๐‘›๐‘›
๐‘–๐‘–=1
Distribution
Formulas
where
PDF
d
n
k
๐‘“๐‘“(๐ค๐ค) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ pi i
1 2
d
i=1
n!
n!
ฮ“(n + 1)
n
=
=
๏ฟฝk , k , . . , k ๏ฟฝ =
1 2
n
k1 ! k 2 ! โ€ฆ k d ! โˆdi=1 k i ! โˆdi=1 ฮ“(k i + 1)
Note that in p there is only d-1 โ€˜freeโ€™ variables as the last
๐‘๐‘๐‘‘๐‘‘ = 1 โˆ’ โˆ‘๐‘‘๐‘‘โˆ’1
๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘– giving the distribution:
s
dโˆ’1
n
k
pi i . ๏ฟฝ1 โˆ’ ๏ฟฝ pi ๏ฟฝ
๐‘“๐‘“(๐ค๐ค) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ
1 2
n
i=1
Multinomial
i=1
nโˆ’โˆ‘dโˆ’1
i=1 ki
Now the special case of binomial distribution when ๐‘‘๐‘‘ = 2 can be
seen.
Let
Where
Marginal PDF
where
๐’‘๐’‘๐’–๐’–
๐‘ผ๐‘ผ
๐‘ฒ๐‘ฒ = ๏ฟฝ ๏ฟฝ ~๐‘€๐‘€๐‘€๐‘€๐‘œ๐‘œ๐‘š๐‘š๐‘‘๐‘‘ ๏ฟฝ๐‘›๐‘›, ๏ฟฝ๐’‘๐’‘ ๏ฟฝ๏ฟฝ
๐‘ฝ๐‘ฝ
๐’—๐’—
๐‘ฒ๐‘ฒ = [๐พ๐พ1 , โ€ฆ , ๐พ๐พ๐‘ ๐‘  , ๐พ๐พ๐‘ ๐‘ +1 , โ€ฆ , ๐พ๐พ๐‘‘๐‘‘ ]๐‘‡๐‘‡
๐‘ผ๐‘ผ = [๐พ๐พ1 , โ€ฆ , ๐พ๐พ๐‘ ๐‘  ]๐‘‡๐‘‡
๐‘ฝ๐‘ฝ = [๐พ๐พ๐‘ ๐‘ +1 , โ€ฆ , ๐พ๐พ๐‘‘๐‘‘ ]๐‘‡๐‘‡
๐‘ผ๐‘ผ โˆผ ๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘ ๐‘  (๐‘›๐‘›, ๐’‘๐’‘๐’–๐’– )
๐‘‡๐‘‡
๐’‘๐’‘๐’–๐’– = ๏ฟฝ๐‘๐‘1 , ๐‘๐‘2 , โ€ฆ , ๐‘๐‘๐‘ ๐‘ โˆ’1 , ๏ฟฝ1 โˆ’ โˆ‘๐‘ ๐‘ โˆ’1
๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘– ๏ฟฝ๏ฟฝ
Multinomial Distribution
195
s
n
k
๐‘“๐‘“(๐’–๐’–) = ๏ฟฝk , k , . . , k ๏ฟฝ ๏ฟฝ pi i
1 2
s
i=1
When only two states ๐’‘๐’‘ = [๐‘๐‘, (1 โˆ’ ๐‘๐‘)]๐‘‡๐‘‡ :
n k
๐‘“๐‘“(๐‘˜๐‘˜๐‘–๐‘– ) = ๏ฟฝk ๏ฟฝ pi i (1 โˆ’ ๐‘๐‘๐‘–๐‘– )๐‘›๐‘›โˆ’๐‘˜๐‘˜๐‘–๐‘–
i
where
Conditional PDF
๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’— โˆผ ๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘ ๐‘  ๏ฟฝ๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ , ๐’‘๐’‘๐‘ข๐‘ข|๐’—๐’— ๏ฟฝ
๐‘‘๐‘‘
๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๐‘›๐‘›๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘–
๐’‘๐’‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ =
1
โˆ‘๐‘ ๐‘ ๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘–
๐‘–๐‘–=๐‘ ๐‘ +1
[๐‘๐‘1 , ๐‘๐‘2 , โ€ฆ , ๐‘๐‘๐‘ ๐‘  ]๐‘‡๐‘‡
๐น๐น(๐ค๐ค) = P(K1 โ‰ค k1 , K 2 โ‰ค k 2 , โ€ฆ , K d โ‰ค k d )
k1
k2
kd
d
n
j
= ๏ฟฝ ๏ฟฝ โ€ฆ ๏ฟฝ ๏ฟฝj , j , . . , j ๏ฟฝ ๏ฟฝ pii
1 2
d
i=1
CDF
j1 =0 j2 =0
jd=0
๐‘…๐‘…(๐ค๐ค) = P(K1 > k1 , K 2 > k 2 , โ€ฆ , K d > k d )
n
n
n
d
n
j
= ๏ฟฝ
๏ฟฝ โ€ฆ ๏ฟฝ ๏ฟฝj , j , . . , j ๏ฟฝ ๏ฟฝ pii
1 2
d
i=1
Reliability
j1 =k1 +1 j2 =k2 +1
jd=kd +1
Properties and Moments
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€(๐‘˜๐‘˜๐‘–๐‘– ) is either {โŒŠ๐‘›๐‘›๐‘๐‘๐‘–๐‘– โŒ‹, โŒˆ๐‘›๐‘›๐‘๐‘๐‘–๐‘– โŒ‰}
Median3
Mode
Mean -
1st
Raw Moment
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€(๐‘˜๐‘˜๐‘–๐‘– ) = โŒŠ(๐‘›๐‘› + 1)๐‘๐‘๐‘–๐‘– โŒ‹
๐ธ๐ธ[๐‘ฒ๐‘ฒ] = ๐๐ = ๐‘›๐‘›๐’‘๐’‘
Mean of the conditional distribution:
๐ธ๐ธ[๐‘ผ๐‘ผ|๐‘ฝ๐‘ฝ = ๐’—๐’—] = ๐๐๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ ๐’‘๐’‘๐‘ข๐‘ข|๐’—๐’—
where
๐‘‘๐‘‘
๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๐‘›๐‘›๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘–
๐’‘๐’‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ =
3
1
โˆ‘๐‘ ๐‘ ๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘–
โŒŠ๐‘ฅ๐‘ฅโŒ‹ = is the floor function (largest integer not greater than ๐‘ฅ๐‘ฅ)
โŒˆ๐‘ฅ๐‘ฅโŒ‰ = is the ceiling function (smallest integer not less than ๐‘ฅ๐‘ฅ)
๐‘–๐‘–=๐‘ ๐‘ +1
[๐‘๐‘1 , ๐‘๐‘2 , โ€ฆ , ๐‘๐‘๐‘ ๐‘  ]๐‘‡๐‘‡
Multinomial
Mean of the marginal distribution:
๐ธ๐ธ[๐‘ผ๐‘ผ] = ๐๐๐’–๐’– = ๐‘›๐‘›๐’‘๐’‘๐‘ข๐‘ข
๐ธ๐ธ[๐พ๐พ๐‘–๐‘– ] = ๐œ‡๐œ‡๐‘˜๐‘˜๐‘–๐‘– = ๐‘›๐‘›๐‘๐‘๐‘–๐‘–
196
Bivariate and Multivariate Distributions
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐พ๐พ๐‘–๐‘– ] = ๐‘›๐‘›๐‘๐‘๐‘–๐‘– (1 โˆ’ ๐‘๐‘๐‘–๐‘– )
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐พ๐พ๐‘–๐‘– , ๐พ๐พ๐‘—๐‘— ๏ฟฝ = โˆ’๐‘›๐‘›๐‘๐‘๐‘–๐‘– ๐‘๐‘๐‘—๐‘—
Variance - 2nd Central Moment
Covariance of marginal distributions:
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰[๐พ๐พ๐‘–๐‘– ] = ๐‘›๐‘›๐‘๐‘๐‘–๐‘– (1 โˆ’ ๐‘๐‘๐‘–๐‘– )
Covariance of conditional distributions:
๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๐‘‰๏ฟฝ๐พ๐พ๐‘ˆ๐‘ˆ|๐‘‰๐‘‰,๐‘–๐‘– ๏ฟฝ = ๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ ๐‘๐‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ,๐‘–๐‘– (1 โˆ’ ๐‘๐‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ,๐‘–๐‘– )
๐ถ๐ถ๐ถ๐ถ๐ถ๐ถ๏ฟฝ๐พ๐พ๐‘ˆ๐‘ˆ|๐‘‰๐‘‰,๐‘–๐‘– , ๐พ๐พ๐‘ˆ๐‘ˆ|๐‘‰๐‘‰,๐‘—๐‘— ๏ฟฝ = โˆ’๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ ๐‘๐‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ,๐‘–๐‘– ๐‘๐‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ,๐‘—๐‘—
where
๐‘‘๐‘‘
๐‘›๐‘›๐‘ข๐‘ข|๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๐‘›๐‘›๐‘ฃ๐‘ฃ = ๐‘›๐‘› โˆ’ ๏ฟฝ ๐‘˜๐‘˜๐‘–๐‘–
๐’‘๐’‘๐‘ข๐‘ข|๐‘ฃ๐‘ฃ =
Parameter Estimation
1
โˆ‘๐‘ ๐‘ ๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘–
๐‘–๐‘–=๐‘ ๐‘ +1
[๐‘๐‘1 , ๐‘๐‘2 , โ€ฆ , ๐‘๐‘๐‘ ๐‘  ]๐‘‡๐‘‡
Maximum Likelihood Function
MLE
Point
Estimates
As with the binomial distribution the MLE estimates, given the vector
k(and therefore n), is:(Johnson et al. 1997, p.51)
๏ฟฝ=
๐ฉ๐ฉ
๐ค๐ค
n
Where there are ๐‘‡๐‘‡ observations of ๐’Œ๐’Œ๐‘ก๐‘ก each containing ๐‘›๐‘›๐‘ก๐‘ก trails:
Multinomial
๏ฟฝ=
๐ฉ๐ฉ
1
โˆ‘nt=1 nt
๐‘‡๐‘‡
๏ฟฝ ๐’Œ๐’Œ๐‘ก๐‘ก
๐‘ก๐‘ก=1
100๐›พ๐›พ%
Confidence
Intervals
An approximation of the joint interval confidence limits for 100๐›พ๐›พ% given
by Goodman in 1965 is:(Johnson et al. 1997, p.51)
(Complete
Data)
1
4
๏ฟฝ๐ด๐ด + 2๐‘˜๐‘˜๐‘–๐‘– โˆ’ ๐ด๐ด๏ฟฝ๐ด๐ด + ๐‘˜๐‘˜๐‘–๐‘– (๐‘›๐‘› โˆ’ ๐‘˜๐‘˜๐‘–๐‘– )๏ฟฝ
2(๐‘›๐‘› + ๐ด๐ด)
๐‘›๐‘›
๐‘๐‘๐‘–๐‘– lower confidence limit:
๐‘๐‘๐‘–๐‘– upper confidence limit:
4
1
๏ฟฝ๐ด๐ด + 2๐‘˜๐‘˜๐‘–๐‘– + ๐ด๐ด๏ฟฝ๐ด๐ด + ๐‘˜๐‘˜๐‘–๐‘– (๐‘›๐‘› โˆ’ ๐‘˜๐‘˜๐‘–๐‘– )๏ฟฝ
๐‘›๐‘›
2(๐‘›๐‘› + ๐ด๐ด)
where ฮฆ is the standard normal CDF and:
dโˆ’1+ฮณ
๐ด๐ด = ๐‘๐‘๐‘‘๐‘‘โˆ’1+๐›พ๐›พ = ฮฆโˆ’1 ๏ฟฝ
๏ฟฝ
d
๐‘‘๐‘‘
Multinomial Distribution
197
A complete coverage of estimation techniques and confidence intervals
is contained in (Johnson et al. 1997, pp.51-65). A more accurate method
which requires numerical methods is given in (Sison & Glaz 1995)
Bayesian
Type
Uniform Prior
Jeffreys Prior
One Group Reference Prior
Non-informative Priors, ๐…๐…(๐’‘๐’‘)
(Yang and Berger 1998, p.6)
Prior
Posterior
1 = ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐›ผ๐›ผ๐‘–๐‘– = 1)
๐ถ๐ถ
๏ฟฝโˆ๐‘‘๐‘‘๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘–
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐ฉ๐ฉ|1 + ๐ค๐ค)
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐ฉ๐ฉ๏ฟฝ12+๐ค๐ค)
1
= ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 ๏ฟฝ๐›ผ๐›ผ๐‘–๐‘– = 2๏ฟฝ
where ๐ถ๐ถ is a constant
In terms of the reference prior, this approach considers all
parameters are of equal importance.(Berger & Bernardo 1992)
d-group
Reference Prior
๐ถ๐ถ
๐‘–๐‘–
๏ฟฝโˆ๐‘‘๐‘‘โˆ’1
๐‘–๐‘–=1 ๏ฟฝ๐‘๐‘๐‘–๐‘– ๏ฟฝ1 โˆ’ โˆ‘๐‘—๐‘—=1 ๐‘๐‘๐‘–๐‘– ๏ฟฝ๏ฟฝ
where ๐ถ๐ถ is a constant
Proper. See m-group posterior
when ๐‘š๐‘š = 1.
This approach considers each parameter to be of different
importance (group length 1) and so the parameters must be ordered
by importance. (Berger & Bernardo 1992)
m-group
Reference Prior
๐œ‹๐œ‹๐‘œ๐‘œ (๐’‘๐’‘) =
๏ฟฝ๏ฟฝ1
๐ถ๐ถ
๐‘๐‘๐‘š๐‘š
โˆ’ โˆ‘๐‘—๐‘—=1
๐‘๐‘๐‘—๐‘— ๏ฟฝ โˆ๐‘‘๐‘‘โˆ’1
๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘–
๐‘๐‘๐‘–๐‘–
โˆ๐‘š๐‘šโˆ’1
๐‘–๐‘–=1 ๏ฟฝ1 โˆ’ โˆ‘๐‘—๐‘—=1 ๐‘๐‘๐‘—๐‘— ๏ฟฝ
๐‘›๐‘›๐‘–๐‘–+1
๐ฉ๐ฉ๐ข๐ข = ๏ฟฝ๐‘๐‘๐‘๐‘๐‘–๐‘–โˆ’1+1 , โ€ฆ , ๐‘๐‘๐‘๐‘๐‘–๐‘– ๏ฟฝ
๐ถ๐ถ is a constant
Posterior:
๐œ‹๐œ‹(๐’‘๐’‘|๐’Œ๐’Œ) โˆ
๐‘๐‘
๐‘š๐‘š
๏ฟฝ1 โˆ’ โˆ‘๐‘—๐‘—=1
๐‘๐‘๐‘–๐‘– ๏ฟฝ
๐‘‡๐‘‡
๐‘˜๐‘˜๐‘‘๐‘‘ โˆ’12
๐‘๐‘๐‘–๐‘–
๐‘š๐‘šโˆ’1
๏ฟฝโˆ๐‘‘๐‘‘โˆ’1
๐‘–๐‘–=1 ๐‘๐‘๐‘–๐‘– โˆ๐‘–๐‘–=1 ๏ฟฝ1 โˆ’ โˆ‘๐‘—๐‘—=1 ๐‘๐‘๐‘—๐‘— ๏ฟฝ
๐‘›๐‘›๐‘–๐‘–+1
This approach splits the parameters into m different groups of
importance. Within the group order is not important, but the groups
need to be ordered by importance. It is common to have ๐‘š๐‘š = 2 and
split the parameters into importance and nuisance parameters.
(Berger & Bernardo 1992)
Multinomial
where groups are given by:
๐‘‡๐‘‡
๐‘‡๐‘‡
๐ฉ๐ฉ๐Ÿ๐Ÿ = ๏ฟฝ๐‘๐‘1 , โ€ฆ ๐‘๐‘๐‘›๐‘›1 ๏ฟฝ , ๐ฉ๐ฉ๐Ÿ๐Ÿ = ๏ฟฝ๐‘๐‘๐‘›๐‘›1+1 , โ€ฆ , ๐‘๐‘๐‘›๐‘›1+๐‘›๐‘›2 ๏ฟฝ
๐‘๐‘๐‘—๐‘— = ๐‘›๐‘›1 + โ‹ฏ + ๐‘›๐‘›๐‘—๐‘— for ๐‘—๐‘— = 1, โ€ฆ , ๐‘š๐‘š
198
Bivariate and Multivariate Distributions
๐‘‘๐‘‘
MDIP
Novick and Hallโ€™s
Prior (improper)
UOI
๐‘–๐‘–=1
๐‘‘๐‘‘
๏ฟฝ
๐‘๐‘๐‘–๐‘–โˆ’1 = ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐›ผ๐›ผ๐‘–๐‘– = 0)
๐‘–๐‘–=1
Conjugate Priors (Fink 1997)
Multinomiald
Evidence
๐‘˜๐‘˜๐‘–๐‘–,๐‘—๐‘— failures in
๐‘›๐‘› trials with
๐‘‘๐‘‘ possible
states.
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐ฉ๐ฉ|๐ค๐ค)
Dist of
UOI
Prior
Para
Posterior
Parameters
Dirichletd+1
๐œถ๐œถ๐’๐’
๐œถ๐œถ = ๐œถ๐œถ๐’๐’ + ๐’Œ๐’Œ
Description , Limitations and Uses
A six sided dice being thrown 60 times produces the following
multinomial distribution:
Face
Number
1
2
3
4
5
6
Characteristic
๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐ฉ๐ฉโ€ฒ|๐‘๐‘๐‘–๐‘– + 1 + k i )
๐‘๐‘
๐‘๐‘๐‘–๐‘– ๐‘–๐‘– = ๐ท๐ท๐ท๐ท๐‘Ÿ๐‘Ÿ๐‘‘๐‘‘+1 (๐›ผ๐›ผ๐‘–๐‘– = ๐‘๐‘๐‘–๐‘– + 1)
Likelihood
Model
๐’‘๐’‘
from
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘ (๐’Œ๐’Œ; ๐‘›๐‘›๐‘ก๐‘ก , ๐’‘๐’‘)
Example
๏ฟฝ
Times
Observed
12
7
11
10
8
12
12
โŽก6โŽค
โŽข12โŽฅ
๐’Œ๐’Œ = โŽข โŽฅ
โŽข10โŽฅ
โŽข8โŽฅ
โŽฃ12โŽฆ
0.2
โŽก 0.1 โŽค
โŽข 0.2 โŽฅ
โŽฅ
๐’‘๐’‘ = โŽข
โŽข0.16ฬ‡ โŽฅ
โŽข0.13ฬ‡ โŽฅ
โŽฃ 0.2 โŽฆ
๐‘›๐‘› = 60
Binomial Generalization. The multinomial distribution is a
generalization of the binomial distribution where more than two states of
the system are allowed. The binomial distribution is a special case where
๐‘‘๐‘‘ = 2.
Covariance. All covarianceโ€™s are negative. This is because the increase
in one parameter ๐‘๐‘๐‘–๐‘– must result in the decrease of ๐‘๐‘๐‘—๐‘— to satisfy ฮฃ๐‘๐‘๐‘–๐‘– = 1.
Multinomial
With Replacement. The multinomial distribution assumes replacement.
The equivalent distribution which assumes without replacement is the
multivariate hypergeometric distribution.
Convolution Property
Let
Then
๐‘ฒ๐‘ฒ๐’•๐’• โˆผ ๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘ (๐’Œ๐’Œ; ๐‘›๐‘›๐‘ก๐‘ก , ๐ฉ๐ฉ)
๏ฟฝ ๐‘ฒ๐‘ฒ๐’•๐’• ~๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘ (โˆ‘๐’Œ๐’Œ๐‘ก๐‘ก ; โˆ‘๐‘›๐‘›๐‘ก๐‘ก , ๐’‘๐’‘)
*This does not hold when the p parameter differs.
Applications
Partial Failures. When the states of a system under demands cannot
Multinomial Distribution
199
be modeled with two states (success or failure) the multinomial
distribution may be used. Examples of this include when modeling
discrete states of component degradation.
Resources
Online:
http://en.wikipedia.org/wiki/Multinomial_distribution
http://mathworld.wolfram.com/MultinomialDistribution.html
http://www.math.uah.edu/stat/bernoulli/Multinomial.xhtml
Books:
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1997. Discrete Multivariate
Distributions 1st ed., Wiley-Interscience.
Relationship to Other Distributions
Binominal
Distribution
๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘›๐‘›, ๐‘๐‘)
Special Case:
๐‘€๐‘€๐‘€๐‘€๐‘€๐‘€๐‘š๐‘š๐‘‘๐‘‘=2 (๐ค๐ค|n, ๐ฉ๐ฉ) = ๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต๐ต(๐‘˜๐‘˜|๐‘›๐‘›, ๐‘๐‘)
Multinomial
200
Probability Distributions Used in Reliability Engineering
7. References
Abadir, K. & Magdalinos, T., 2002. The Characteristic Function from a Family of
Truncated Normal Distributions. Econometric Theory, 18(5), p.1276-1287.
Agresti, A., 2002. Categorical data analysis, John Wiley and Sons.
Aitchison, J.J. & Brown, J.A.C., 1957. The Lognormal Distribution, New York:
Cambridge University Press.
Andersen, P.K. et al., 1996. Statistical Models Based on Counting Processes Corrected.,
Springer.
Angus, J.E., 1994. Bootstrap one-sided confidence intervals for the log-normal mean.
Journal of the Royal Statistical Society. Series D (The Statistician), 43(3),
p.395โ€“401.
Anon, Six Sigma | Process Management | Strategic Process Management | Welcome to
SSA & Company.
Antle, C., Klimko, L. & Harkness, W., 1970. Confidence Intervals for the Parameters of
the Logistic Distribution. Biometrika, 57(2), p.397-402.
Aoshima, M. & Govindarajulu, Z., 2002. Fixed-width confidence interval for a lognormal
mean. International Journal of Mathematics and Mathematical Sciences, 29(3),
p.143โ€“153.
Arnold, B.C., 1983. Pareto distributions, Fairland, MD: International Co-operative Pub.
House.
Artin, E., 1964. The Gamma Function, New York: Holt, Rinehart & Winston.
Balakrishnan, 1991. Handbook of the Logistic Distribution 1st ed., CRC.
References
Balakrishnan, N. & Basu, A.P., 1996. Exponential Distribution: Theory, Methods and
Applications 1st ed., CRC.
Balakrishnan, N. & Lai, C.-D., 2009. Continuous Bivariate Distributions 2nd ed.,
Springer.
Balakrishnan, N. & Rao, C.R., 2001. Handbook of Statistics 20: Advances in Reliability
1st ed., Elsevier Science & Technology.
Berger, J.O., 1993. Statistical Decision Theory and Bayesian Analysis 2nd ed., Springer.
References
201
Berger, J.O. & Bernardo, J.M., 1992. Ordered Group Reference Priors with Application
to the Multinomial Problem. Biometrika, 79(1), p.25-37.
Berger, J.O. & Sellke, T., 1987. Testing a Point Null Hypothesis: The Irreconcilability of
P Values and Evidence. Journal of the American Statistical Association,
82(397), p.112-122.
Berger, J.O. & Sun, D., 2008. Objective priors for the bivariate normal model. The Annals
of Statistics, 36(2), p.963-982.
Bernardo, J.M. et al., 1992. On the development of reference priors. Bayesian statistics,
4, p.35โ€“60.
Berry, D.A., Chaloner, K.M. & Geweke, J.K., 1995. Bayesian Analysis in Statistics and
Econometrics: Essays in Honor of Arnold Zellner 1st ed., Wiley-Interscience.
Bertsekas, D.P. & Tsitsiklis, J.N., 2008. Introduction to Probability 2nd ed., Athena
Scientific.
Billingsley, P., 1995. Probability and Measure, 3rd Edition 3rd ed., Wiley-Interscience.
Birnbaum, Z.W. & Saunders, S.C., 1969. A New Family of Life Distributions. Journal
of Applied Probability, 6(2), p.319-327.
Bjõrck, A., 1996. Numerical Methods for Least Squares Problems 1st ed., SIAM: Society
for Industrial and Applied Mathematics.
Bowman, K.O. & Shenton, L.R., 1988. Properties of estimators for the gamma
distribution, CRC Press.
Brown, L.D., Cai, T.T. & DasGupta, A., 2001. Interval estimation for a binomial
proportion. Statistical Science, p.101โ€“117.
Christensen, R. & Huffman, M.D., 1985. Bayesian Point Estimation Using the Predictive
Distribution. The American Statistician, 39(4), p.319-321.
Cohen, 1991. Truncated and Censored Samples 1st ed., CRC Press.
Collani, E.V. & Dräger, K., 2001. Binomial distribution handbook for scientists and
engineers, Birkhäuser.
Cozman, F. & Krotkov, E., 1997. Truncated Gaussians as Tolerance Sets.
Crow, E.L. & Shimizu, K., 1988. Lognormal distributions, CRC Press.
References
Congdon, P., 2007. Bayesian Statistical Modelling 2nd ed., Wiley.
202
Probability Distributions Used in Reliability Engineering
Dekking, F.M. et al., 2007. A Modern Introduction to Probability and Statistics:
Understanding Why and How, Springer.
Fink, D., 1997. A compendium of conjugate priors. See http://www. people. cornell.
edu/pages/df36/CONJINTRnew% 20TEX. pdf, p.46.
Georges, P. et al., 2001. Multivariate Survival Modelling: A Unified Approach with
Copulas. SSRN eLibrary.
Gupta and Nadarajah, 2004. Handbook of beta distribution and its applications, CRC
Press.
Gupta, P.L., Gupta, R.C. & Tripathi, R.C., 1997. On the monotonic properties of discrete
failure rates. Journal of Statistical Planning and Inference, 65(2), p.255-268.
Haight, F.A., 1967. Handbook of the Poisson distribution, New York,: Wiley.
Hastings, N.A.J., Peacock, B. & Evans, M., 2000. Statistical Distributions, 3rd Edition
3rd ed., John Wiley & Sons Inc.
Jiang, R. & Murthy, D.N.P., 1996. A mixture model involving three Weibull
distributions. In Proceedings of the Second Australiaโ€“Japan Workshop on
Stochastic Models in Engineering, Technology and Management. Gold Coast,
Australia, pp. 260-270.
Jiang, R. & Murthy, D.N.P., 1998. Mixture of Weibull distributions - parametric
characterization of failure rate function. Applied Stochastic Models and Data
Analysis, (14), p.47-65.
Jiang, R. & Murthy, D.N.P., 1995. Modeling Failure-Data by Mixture of2 Weibull
Distributions : A Graphical Approach. IEEE Transactions on Reliability, 44,
p.477-488.
Jiang, R. & Murthy, D.N.P., 1999. The exponentiated Weibull family: a graphical
approach. Reliability, IEEE Transactions on, 48(1), p.68-72.
References
Johnson, N.L., Kemp, A.W. & Kotz, S., 2005. Univariate Discrete Distributions 3rd ed.,
Wiley-Interscience.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1994. Continuous Univariate Distributions,
Vol. 1 2nd ed., Wiley-Interscience.
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1995. Continuous Univariate Distributions,
Vol. 2 2nd ed., Wiley-Interscience.
References
203
Johnson, N.L., Kotz, S. & Balakrishnan, N., 1997. Discrete Multivariate Distributions
1st ed., Wiley-Interscience.
Kimball, B.F., 1960. On the Choice of Plotting Positions on Probability Paper. Journal
of the American Statistical Association, 55(291), p.546-560.
Kleiber, C. & Kotz, S., 2003. Statistical Size Distributions in Economics and Actuarial
Sciences 1st ed., Wiley-Interscience.
Klein, J.P. & Moeschberger, M.L., 2003. Survival analysis: techniques for censored and
truncated data, Springer.
Kotz, S., Balakrishnan, N. & Johnson, N.L., 2000. Continuous Multivariate
Distributions, Volume 1, Models and Applications, 2nd Edition 2nd ed., WileyInterscience.
Kotz, S. & Dorp, J.R. van, 2004. Beyond Beta: Other Continuous Families Of
Distributions With Bounded Support And Applications, World Scientific
Publishing Company.
Kundu, D., Kannan, N. & Balakrishnan, N., 2008. On the hazard function of BirnbaumSaunders distribution and associated inference. Comput. Stat. Data Anal., 52(5),
p.2692-2702.
Lai, C.D., Xie, M. & Murthy, D.N.P., 2003. A modified Weibull distribution. IEEE
Transactions on Reliability, 52(1), p.33-37.
Lai, C.-D. & Xie, M., 2006. Stochastic Ageing and Dependence for Reliability 1st ed.,
Springer.
Lawless, J.F., 2002. Statistical Models and Methods for Lifetime Data 2nd ed., WileyInterscience.
Leemis, L.M. & McQueston, J.T., 2008. Univariate distribution relationships. The
American Statistician, 62(1), p.45โ€“53.
Leipnik, R.B., 1991. On Lognormal Random Variables: I-the Characteristic Function.
The ANZIAM Journal, 32(03), p.327-347.
Limpert, E., Stahel, W. & Abbt, M., 2001. Log-normal Distributions across the Sciences:
Keys and Clues. BioScience, 51(5), p.352, 341.
References
Lemonte, A.J., Cribari-Neto, F. & Vasconcellos, K.L.P., 2007. Improved statistical
inference for the two-parameter Birnbaum-Saunders distribution.
Computational Statistics & Data Analysis, 51(9), p.4656-4681.
204
Probability Distributions Used in Reliability Engineering
MacKay, D.J.C. & Petoy, L.C.B., 1995. A hierarchical Dirichlet language model. Natural
language engineering.
Manzini, R. et al., 2009. Maintenance for Industrial Systems 1st ed., Springer.
Martz, H.F. & Waller, R., 1982. Bayesian reliability analysis, JOHN WILEY & SONS,
INC, 605 THIRD AVE, NEW YORK, NY 10158.
Meeker, W.Q. & Escobar, L.A., 1998. Statistical Methods for Reliability Data 1st ed.,
Wiley-Interscience.
Modarres, M., Kaminskiy, M. & Krivtsov, V., 1999. Reliability engineering and risk
analysis, CRC Press.
Murthy, D.N.P., Xie, M. & Jiang, R., 2003. Weibull Models 1st ed., Wiley-Interscience.
Nelson, W.B., 1990. Accelerated Testing: Statistical Models, Test Plans, and Data
Analysis, Wiley-Interscience.
Nelson, W.B., 1982. Applied Life Data Analysis, Wiley-Interscience.
Novosyolov, A., 2006. The sum of dependent normal variables may be not normal.
http://risktheory.ru/papers/sumOfDep.pdf.
Patel, J.K. & Read, C.B., 1996. Handbook of the Normal Distribution 2nd ed., CRC.
Pham, H., 2006. Springer Handbook of Engineering Statistics 1st ed., Springer.
Provan, J.W., 1987. Probabilistic approaches to the material-related reliability of
fracture-sensitive structures. Probabilistic fracture mechanics and reliability(A
87-35286 15-38). Dordrecht, Martinus Nijhoff Publishers, 1987,, p.1โ€“45.
Rao, C.R. & Toutenburg, H., 1999. Linear Models: Least Squares and Alternatives 2nd
ed., Springer.
Rausand, M. & Høyland, A., 2004. System reliability theory, Wiley-IEEE.
References
Rencher, A.C., 1997. Multivariate Statistical Inference and Applications, Volume 2,
Methods of Multivariate Analysis Har/Dis., Wiley-Interscience.
Rinne, H., 2008. The Weibull Distribution: A Handbook 1st ed., Chapman & Hall/CRC.
Schneider, H., 1986. Truncated and censored samples from normal populations, M.
Dekker.
References
205
Simon, M.K., 2006. Probability Distributions Involving Gaussian Random Variables: A
Handbook for Engineers and Scientists, Springer.
Singpurwalla, N.D., 2006. Reliability and Risk: A Bayesian Perspective 1st ed., Wiley.
Sison, C.P. & Glaz, J., 1995. Simultaneous Confidence Intervals and Sample Size
Determination for Multinomial Proportions. Journal of the American Statistical
Association, 90(429).
Tong, Y.L., 1990. The Multivariate Normal Distribution, Springer.
Xie, M., Gaudoin, O. & Bracquemond, C., 2002. Redefining Failure Rate Function for
Discrete Distributions. International Journal of Reliability, Quality & Safety
Engineering, 9(3), p.275.
Xie, M., Goh, T.N. & Tang, Y., 2004. On changing points of mean residual life and
failure rate function for some generalized Weibull distributions. Reliability
Engineering and System Safety, 84(3), p.293โ€“299.
Xie, M., Tang, Y. & Goh, T.N., 2002. A modified Weibull extension with bathtub-shaped
failure rate function. Reliability Engineering and System Safety, 76(3), p.279โ€“
285.
Yang and Berger, 1998. A Catalog of Noninformative Priors (DRAFT).
Yang, K. et al., 2004. Multivariate Statistical Methods in Quality Management 1st ed.,
McGraw-Hill Professional.
Yang, R. & Berger, J.O., 1994. Estimation of a Covariance Matrix Using the Reference
Prior. The Annals of Statistics, 22(3), p.1195-1211.
Zhou, X.H. & Gao, S., 1997. Confidence intervals for the log-normal mean. Statistics in
medicine, 16(7), p.783โ€“790.
References