Download probability and random processes electrical engineers

P R O B A B I L I T Y AND
RANDOM P R O C E S S E S
FOR
ELECTRICAL
YANNIS
NORTH
ENGINEERS
ViNIOTIS
CAROLINA
STATE
UNIVERSITY
WCB
McGraw-Hill
BOSTON
B U R R R I D G E , IL
M E X I C O CITY
D U B U Q U E , IA
BANGKOK
BOGATÄ
MILAN
NEW DELHI
M A D I S O N , Wl
CARACAS
SEOUL
LISBON
NEW YORK
S A N FRANCISCO
LONDON
SINGAPORE
SYDNEY
ST. LOUIS
MADRID
TAIPEI
TORONTO
CONTENTS
PREFACE
XV
INTRODUCTION
1.1
1.2
1.3
1.4
1.5
1.6
1.7
1.8
1.9
1.10
1.11
1.12
1.13
1
Why This Course?
Why an Abstract Theory?
Why Probability Theory?
What Can Probability Theory Do?
What Is the Theory in a Nutshell?
Modeling Tools?
Main Models?
Main Application Areas
A Few Detailed Examples
D i d l Learn the Theory?
Where Do I Learn More?
Summary of Main Points
Problems
2
6
7
7
7
9
9
10
11
18
18
19
20
BASIC CONCEPTS
23
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
24
26
27
30
33
35
40
42
44
46
46
Random Experiments
Sample Spaces and Outcomes
Events
Probability Axioms and the Sets of Interest
Elementary Theorems
Conditional Probabilities
Independent Events
How Do We Assign Probabilities to Events?
Probabilistic Modeling
Difficulties
Summary of Main Points
ix
2.12 Checkhst of Important Tools
2.13 Problems
47
C O N C E P T O F A R A N D O M VARIABLE
67
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
3.11
3.12
3.13
3.14
3.15
3.16
Sets ofthe Form (-oo.ar]
Concept of a Random Variable
Cumulative Distribution Function
Probability Density Function
Probabilistic Model Revisited
Useful Ran dorn Variable Models
Histograms
Transformations of a Random Variable
Moments of Random Variables
Reliability and Failure Rates
Transforms of Probability Density Functions
Tail Inequalities
Generation of Values of a Random Variable
Summary of Main Points
Checklist of Important Tools
Problems
47
68
70
76
82
95
95
109
115
121
129
134
139
145
150
151
152
A V E C T O R R A N D O M VARIABLE
195
4.1
4.2
4.3
4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
4.13
4.14
4.15
196
198
204
212
221
223
230
236
256
260
264
270
272
273
273
Experiments with More Than One Measurement
The Sets (-oo, x] x (-oo, y]
Joint Cumulative Distribution Function
Joint Probability Density Function
Probabilistic Model Revisited
Conditional Probabilities and Densities
Independence
Transformations of a Random Vector
Expectation, Covariance, and Correlation Coefficient
Useful Joint Distributions
More Than Two Random Variables
Generation of Values of a Random Vector
Summary of Main Points
Checklist of Important Tools
Problems
Contents xi
5
6
7
INTRODUCTION TO ESTIMATION
303
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
Criteria to Consider
MMSE Estimation, Single Measurement
Linear Prediction, Multiple Measurements
Dow Jones Example
Maximum-Likelihood Estimation
An Historical Remark
Summary of Main Points
Checklist of Important Tools
Problems
304
307
315
318
322
324
327
327
327
S E Q U E N C E S O F (HD) R A N D O M
VARIABLES
333
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
6.9
6.10
6.11
6.12
333
334
335
340
344
347
350
355
366
368
369
369
Experiments with an Unbounded Number of Measurements
HD Random Variables
Sums of HD Random Variables
Random Sums of HD Random Variables
Weak Law of Large Numbers
Strong Law of Large Numbers
Central Limit Theorem
Convergence of Sequences of Random Variables
Borel-Cantelli Lemmas
Summary of Main Points
Checklist of Important Tools
Problems
RANDOM PROCESSES
387
7.1
7.2
7.3
7.4
7.5
7.6
7.7
7.8
7.9
388
393
398
399
406
409
413
417
420
Definition of a Random Process and Examples
Joint CDF and PDF
Expectation, Autocovariance, and Correlation Functions
Some Important Special Cases
Useful Random Process Models
Continuity, Derivatives, and Integrals
Ergodicity
Karhunen-Loeve Expansions
Generation of Values of a Random Process
Contents
7.10 Summary of Main Points
7.11 Checklist of Important Tools
7.12 Problems
421
422
422
P O I S S O N A N D GAUSSIAN R A N D O M
PROCESSES
439
8.1
8.2
8.3
8.4
8.5
439
458
463
464
464
Poisson Process
Gaussian Random Process
Summary of Main Points
Checklist of Important Tools
Problems
P R O C E S S I N G OF R A N D O M P R O C E S S E S
475
9.1
9.2
9.3
9.4
9.5
9.6
9.7
9.8
9.9
475
477
484
492
503
510
513
513
514
Introduction
Power Spectral Density Function
Response of Linear Systems to Ran dorn Processes
Optimal Linear Estimation
Kaiman Filter
Periodograms
Summary of Main Points
Checklist of Important Tools
Problems
MARKOV CHAINS
527
10.1 Definition and Classification
10.2 Discrete-Time Markov Chains
10.3 Steady State of Markov Chains
10.4 Drifts and Ergodicity
10.5 Continuous-Time Markov Chains
10.6 Application to Ethernet LANs
10.7 Generation of Values of a Markov Chain
10.8 Summary of Main Points
10.9 Checklist of Important Tools
10.10 Problems
528
535
541
557
561
571
574
576
576
576
Contents xiii
11 CASE STUDY: A BUS-BASED S W I T C H
ARCHITECTURE
11.1
11.2
11.3
11.4
11.5
11.6
11.7
A
B
C
D
Switch Architecture and Operation
Transition to a Stochastic Model
System Model
Performance Measures
Input Data Description
Discussion of Results
Description of the Simulator
595
596
598
599
601
603
604
610
SET THEORY P R I M E R
611
A.l
A.2
A.3
A.4
A.5
A.6
611
618
624
624
625
626
Sets and Subsets
Operations on Sets
Partitions of a Set
Limits of Sequences of Sets
Algebras of Sets
Problems
COUNTING METHODS
633
B.l Ordering of Elements
B.2 Sampling with Ordering and Without Replacement of Elements
B.3 Sampling Without Ordering and Without Replacement of
Elements
B.4 Sampling Without Ordering and with Replacement of Elements
B.5 Problems
633
634
635
637
638
HISTORICAL D E V E L O P M E N T OF T H E
THEORY
643
C.l A Brief History
C.2 Alternative Axioms
C.3 Some of the Early Problems
643
645
647
MODELING OF RANDOMNESS IN
ENGINEERING SYSTEMS: A SUMMARY
651
D.l Elementary Concepts
D.2 Probabilistic Models
651
651
xiv
Contents
D.3
D.4
D.5
D.6
What Models Have We Developed?
What Tools Have We Developed?
Mathematical Subtleties
What Can We Do with a Model?
REFERENCES
INDEX
653
653
654
655
657
663