Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Thomas Bayes (1702-1761) Pierre-Simon Laplace (1749-1827) Bayesian Reasoning A/Prof Geraint Lewis A/Prof Peter Tuthill “Probability theory is nothing but common sense, reduced to calculation.” Laplace Are you a Bayesian or Frequentist? 4 “There are 3 kinds of lies: Lies, Damned Lies, and Statistics” ...and Bayesian Statistics Benjamin Disraeli Frequentists Fig 1. A Frequentist Statistician Fig 2. Bayesian Statistics Conference What is Inference? If A is true then B is true (Major Premise) A = A,B (in Boolean notation) Deductive Inference (Logic) Aristotle 4th Century B.C. A is true (Minor Premise) therefore B is true (conclusion) B is False (Minor Premise) therefore A is False (conclusion) } A B T →T STRONG SYLLOGISMS F ← F Inductive Inference (Plausible Reasoning) B is true (Minor Premise) therefore A is more plausible A is false (Minor Premise) therefore B is less plausible } t←T WEAK SYLLOGISMS F → f What is Inference? Deductive Logic: Cause Effects or outcomes Inductive Logic: Possible Causes Effects or observations What is a Probability? Frequentists Bayesians P(A) = long run relative frequency of A occurring in identical repeats of an observation P(A|B) = Real number measure of the plausibility of proposition A, given (conditional upon) the truth of proposition B “A” is restricted to propositions about random variables “A” can be any logical proposition All probabilities are conditional; we must be explicit what our assumptions B are (no such thing as an absolute probability!) Probability depends on our state of Knowledge Monte Hall A B ? C Probability depends on our state of Knowledge 7 Red 5 Blue ? 1st draw 5/12 Blue 7/12 Red 2nd draw The Desiderata of Bayesian Probability Theory • Degrees of plausibility are represented by real numbers (higher degree of belief represented by a larger number) • With extra evidence supporting a proposition, the plausibility should increase monotonically up to a limit (certainty). • Consistency. Multiple ways to arrive at a conclusion must all produce the same answer (see book for additional details) Logic and Probability • In the certainty limit, where probabilities go to zero (falsehood) or one (truth), then the sum and product rules reduce to formal Boolean deductive logic (strong syllogisms). • Bayesian Probability is therefore an extension of formal logic into intermediate states of knowledge. • Bayesian inference gives a measure of our state of knowledge about nature, not a measure of nature itself. The two rules underlying probability theory SUM RULE: P(A|B) + P(A|B) = 1 PRODUCT RULE: P(A,B|C) = P(A|C) P(B|A,C) = P(B|C) P(A|B,C) Left Handed Blue, Blue Eyes Left Brown Eyes Right Handed All Kangaroos Bayes’ Theorem Posterior Bayes Theorem: P(Hi|D,I) = P(Hi|I) P(D|Hi I) P(D|I) Hi = proposition asserting truth of a hypothesis of interest I = proposition representing prior information D = proposition representing the data P(D|Hi I) = Likelihood: probability of obtaining the data given that the hypothesis is true P(Hi|I) = Prior: probability of hypothesis before new data P(D|I) = Normalization factor (prob all hypothesis i sum to 1) Example: The Gambler’s coin problem P(H|D,I) = P(H|I) P(D|H I) P(D|I) Normalization factor – Ignore this for now as only need relative merit Prior – what do we know about the coin? Assume H=pdf(head) is uniformly distributed 0-1 Likelihood – if we assume the data D gives R heads in N tosses: P(D|H I) HR (1-H)N-R The full distribution, assuming independence of throws, is the Binomial Distribution. We omit terms not containing H, and use a proportionality. Example: A fair coin? Data H H T T Example: A fair coin? The effects of the Prior