Download Introduction to Probability

Rutgers Business School
Spring 2016
Introduction to Probability
26:960:575
Place:
Time:
Instructor:
Office:
E-Mail:
Office Hours:
100 Rockafeller Road, Room 4038
Wednesdays 3:20-6:20
Michael N. Katehakis
100 Rockafeller Road, Room 5147, Piscataway, NJ 08854
[email protected]
After class, and by appointment.
Teaching Material:
Required Text:
S. M. Ross, Introduction to Probability and Statistics for Engineers and Scientists, Fifth Edition (2014).
This text provides a very clear exposition, and real-data examples. Numerous exercises, and applications
connect probability theory to everyday problems and situations.
Recommended Texts:
W. Feller, An Introduction to Probability Theory and Its Applications. Third Edition, J. Wiley & Sons (1967).
J. Walrand, Review of Probability.
S. M. Ross, Applied Probability Models with Optimization Applications, Dover Publications (1992).
Prerequisites: Graduate students who have finished a basic course in calculus are allowed to take the course.
Grading: Miterm: 30%, Homework 20%, Final Exam: 50%.
Outline of the Course: This course provides an introduction to advanced mathematical concepts and methods
that find extensive use in many fields of modern Data Science. The goal is to familiarize you with modern analytical
and powerful numerical tools in the areas of probability and statistics that are used to solve real world business and
engineering problems. Lectures will include a large number of examples. They will be supplemented by programming
exercises with the aim to help the students gain exposure to a broad range of applications and to develop a working
knowledge of the methods most often used by practicing data scientists.
This course is a prerequisite for Stochastic Processes and for Stochastic Calculus for Finance.
Homework: Assignments, given on a weekly/biweekly basis, are to be done individually, unless otherwise stated.
You may discuss the problems with each other; yet the work that you submit must be your own. You are expected to
refrain from using solutions from other sources (e.g. previous years? classes, etc). If you do use outside information,
you must state your sources.
Participation: The class sessions are meant to provide a learning environment that involves all participants. I am
always open for questions, both inside the classroom and outside. Your are expected to come prepared to class, ask
relevant questions, and actively participate in classroom discussions.
Course Syllabus:
• Basic concepts The Concept of Probability: Classical, Frequentist, and Axiomatic Definitions. Principles of
Combinatorial Analysis. Examples of combinatorial problems. Relevance of Stirling’s formula. The Hypergeometric distribution; examples.
1
Rutgers Business School
Spring 2016
Conditional probability. Independence. Law of Total Probability. Bayes’s Theorem. Application: the Binomial
distribution.
• Random Variables and Distribution Functions. Definitions. Properties. Examples of Discrete and Continuous
Distributions. Multivariate Distributions. Independence of Random Variables.
Sequences of Independent Random Variables. The Binomial and Multinomial Distributions. The exponential
distribution. Conditional Independence. Example: Laplace’s “law of succession”. The Normal approximation
to the Binomial distribution. Local and integral limit theorems of deMoivre and Laplace. The weak law of
large numbers for the binomial distribution. The Poisson approximation to the Binomial.
• Numerical Characteristics of Random Variables. Expectation. Median. Variance and Covariance. Properties
of Expectation. The Markov, Chebyshev and Jensen Inequalities.
• Transformations of Random Variables. Absolutely continuous distributions, and distributions of the mixed
type. Examples of absolutely continuous and discrete bivariate distributions. Marginals and the concept of
Independence. Distribution of functions of several random variables, such as the sum, product, and ratio.
Expectation of a function of several random variables. Additivity of expectation. Independence. Correlation.
Examples of uncorrelated but dependent random variables. Variance of the sum.
• Modes of convergence for sequences of random variables. Relation of “convergence in probability” to “convergence with probability one”. The general formulation of the Weak and Strong Laws of Large Numbers, and of
the Central Limit Theorem. The Borel and Cantelli lemma. Proof of the Strong Law of large Numbers.
• Examples of convergence with probability one, in probability, and in distribution. Generating functions and
Moment-Generating functions, Laplace transforms, characteristic functions and their properties. The Central
Limit Theorem. Applications. Cramer’s Theorem.
• Proof of the Central Limit Theorem using moment-generating functions. Strong one-sided Chebyshev inequalities. Chernoff bounds. Examples. Large deviations in the Weak Law of Large Numbers. Cramer’s theorem;
examples.
• Conditional expectation as a random variable. Conditional expectation as regression predictor. Best linear
least-squares prediction
• Random sampling, parameter estimation, distribution of the sample mean, central limit theorem. Confidence
intervals for the mean with known variance, sample variance, biased and unbiased estimators.
• Chi-squared distribution, confidence intervals for the variance, t-distribution, confidence intervals for the mean
for small samples Maximum-likelihood estimation, hypothesis testing.
Detection, significance levels, test for the mean of a normal distribution with known variance, operating curves.
• Test for the mean of a normal distribution with unknown variance. Test for the variance of a normal distribution,
test for the difference of two means, non- parametric tests: sign test, test for arbitrary trends. Chi-squared
test for the goodness of fit.
• Simulation, resampling methods.
2