Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Random Variables and Stochastic Processes – 0903720 Lecture#18 Dr. Ghazi Al Sukkar Email: [email protected] Office Hours: Refer to the website Course Website: http://www2.ju.edu.jo/sites/academic/ghazi.alsukkar 1 Chapter 9 Stochastic Processes Introduction Basic Definitions Statistical properties of Random Process 2 Introduction • Signals can be classified into two main groups: – Deterministic – Random • Random signals can be described by properties e.g. 1. Average power. 2. Spectral distribution on the average. 3. The probability that the signal amplitude exceeds a given value. • The probabilistic models used to describe random signals are called a random process (stochastic process or time series). • Examples of random processes in communications: – Channel noise, – Information generated by a source, – Interference. 3 Definition • Recall that a RANDOM VARIABLE 𝑋(𝜁), is a rule for assigning to every outcome 𝜁𝑖 of an experiment with a sample space 𝑆 a number 𝑋(𝜁𝑖 ). – Note: 𝑋 denotes a random variable and 𝑋(𝜁𝑖 ) denotes a particular value of it. • A RANDOM PROCESS 𝑋(𝑡, 𝜁) is a rule for assigning to every outcome 𝜁𝑖 a waveform (function of time) 𝑋(𝑡, 𝜁𝑖 ). – Note: for notational simplicity we often omit the dependence on 𝜁𝑖 . ⟹ 𝑋(𝑡) denotes a Random process. • Thus a stochastic process is a family (ensemble) of time functions depending on the parameter 𝜁 or, equivalently, a function of 𝑡 and 𝜁. 4 The set of all possible functions is called the ENSEMBLE 𝑋(𝑡, 𝜁) 𝜁1 𝜁2 𝜁𝑛 𝑋(𝑡, 𝜁1 ) Sample function (realization) Sample function 𝑋(𝑡, 𝜁2 ) (realization) Sample function 𝑋(𝑡, 𝜁𝑛 ) (realization) 5 Example • At time 𝑡 = 0 a die is tossed, a time function 𝑥𝑖 (𝑡) is assigned to each possible outcome of the experiment: Outcome Waveform 1 𝑥1 𝑡 = −4 2 𝑥2 𝑡 = −2 3 𝑥3 𝑡 = 2 4 𝑥4 𝑡 = 4 5 𝑥5 𝑡 = −𝑡/2 6 𝑥6 𝑡 = 𝑡/2 • Then 𝑋 is a Random Process: outcome of experiment→ set of waveforms 6 7 • A random process is denoted by: 𝑋(𝑡, 𝜁) where 𝑡 represents time, and 𝜁 is a variable that represents an outcome in the sample space 𝑆. • 𝑋(𝑡, 𝜁) can denote the following quantities: 1) 𝑋 𝑡, 𝜁𝑖 = 𝑥𝑖 (𝑡) a specific member function (sample function), in 2) 𝑋 𝑡, 𝜁 = 𝑋 𝑡, 𝜁𝑖 |𝜁𝑖 ∈ 𝑆 = 𝑥1 𝑡 , 𝑥2 𝑡 , … a collection (ensemble) this case, 𝑡 is variable a 𝜁 is fixed. of time functions (Stochastic process), in this case both 𝑡 and 𝜁 are variables. 3) 𝑋 𝑡𝑜 , 𝜁 = 𝑋 𝑡𝑜 , 𝜁𝑖 |𝜁𝑖 ∈ 𝑆 = 𝑥1 𝑡𝑜 , 𝑥2 𝑡𝑜 , … , a collection of numerical values, which represent a R.V. equal to the state of the given process at 𝑡𝑜 . In this case 𝑡 is fixed and 𝜁 is variable. 4) 𝑋 𝑡𝑜 , 𝜁𝑖 = 𝑥𝑖 (𝑡𝑜 ) which is a number represent the value of the sample function 𝑥𝑖 (𝑡) at 𝑡𝑜 . • Instead of using the above notations, 𝑋(𝑡) is used to denote all of them. Usually the meaning can be understood from the context. 8 𝑋(𝑡, 𝜁) 𝑋(𝑡, 𝜁2 ) 𝑋(𝑡, 𝜁3 ) 𝑥 𝑋(𝑡, 𝜁1 ) 0 𝑡 𝑋(𝑡, 𝜁4 ) t 9 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 𝑡, 𝜁1 =? 10 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 𝑡, 𝜁1 = 𝑋 𝑡, 𝜁 = 1 = 𝑥1 𝑡 = −4 11 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 𝑡, 𝜁5 =? 12 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 𝑡, 𝜁5 = 𝑋 𝑡, 𝜁 = 5 = 𝑥5 𝑡 = −𝑡/2 13 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 6, 𝜁 =? 14 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑋 6, 𝜁 = 𝑋 6 which is a R.V. with the values −4, −3, −2, 2, 3, 4 15 • Therefore, a general Random or Stochastic Process can be described as: – Collection of time functions (signals) corresponding to various outcomes of random experiments. – Collection of random variables observed at different times. 16 Regular vs. Predictable S.P. • Regular Stochastic Process: consists of a family of functions that can not be described in terms of a finite number of parameters. Furthermore, the future of a sample 𝑋(𝑡, 𝜁) of 𝑋(𝑡) can not be determined in terms of its past. – Brownian Motion • Motion of all particles (ensemble) • Motion of a specific particle (sample function) • Predictable Stochastic Process: consist of a family of functions that can be described in terms of finite number of parameters. – Voltage of a generator with fixed frequency • Amplitude and phase are random variables 𝑉 𝑡 = 𝑅 cos(𝜔𝑡 + 𝜙), where 𝑅 𝑎𝑛𝑑 𝜙 are R.V.s ⟹ 𝑉 𝑡, 𝜁𝑖 = 𝑅 𝜁𝑖 cos 𝜔𝑡 + 𝜙(𝜁𝑖 ) 17 Classification of Random processes Continuous 𝑿(𝒕) Discrete 𝒕 Continuous Continuous-time and Continuous-state Process Discrete-time and Continuous-state Process (Continuous-state sequence) Discrete Continuous-time and Discrete-state Process Discrete-time and Discrete-state Process (Discrete-state sequence) 18 Equality • Equality (Generally) – Ensembles should be equal for each “𝜁” and “𝑡” 𝑋 𝑡, 𝜁 = 𝑌(𝑡, 𝜁) • Equality (Mean Square Sense) – If the following equality holds 𝐸 𝑋 𝑡 − 𝑌(𝑡) – Sufficient in many applications 2 =0 19 Statistics of Stochastic Process • A stochastic process 𝑋(𝑡) is a non-countable infinity of random variables, one for each 𝑡. • For fixed 𝑡, 𝑋(𝑡) represents a random variable: Its First-Order distribution function is given by: FX x, t PX t x It depends on 𝑡, since for a different 𝑡, we obtain a different random variable, and it is the probability of the event 𝑋(𝑡) ≤ 𝑥 which consist of all outcomes 𝜁 such that, at specific time 𝑡, the samples 𝑋(𝑡, 𝜁) does not exceed the number 𝑥. Its First-Order PDF is: f x, t F x, t X x X 20 For this figure 𝐹𝑋 𝑥, 𝑡 = 𝑃 𝑋(𝑡) ≤ 𝑥 = 𝑃 𝜁3 , 𝜁4 X (t , ) 𝑋(𝑡, 𝜁2 ) 𝑋(𝑡, 𝜁3 ) 𝑥 𝑋(𝑡, 𝜁1 ) 0 𝑡 𝑋(𝑡, 𝜁4 ) t In general if 𝐴 = 𝜁𝑖 |𝑋(𝑡, 𝜁𝑖 ) ≤ 𝑥 , then 𝐹𝑋 𝑥, 𝑡 = 𝑃(𝐴) 21 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃(𝑋 4 = −2) =? 22 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 1 𝑃 𝑋 4 = −2 = 𝑃 2,5 = 3 23 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃(𝑋 4 ≤ 0) =? 24 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃 𝑋 4 ≤ 0 = 𝑃 1,2,5 = 1/2 25 • Second-Order CDF of a random process: For 𝑡 = 𝑡1 and 𝑡 = 𝑡2, 𝑋(𝑡) represents two different random variables 𝑋1 = 𝑋(𝑡1) and 𝑋2 = 𝑋(𝑡2) respectively. Their joint distribution is given by FX ( x1 , x 2 ; t1 , t 2 ) P{ X (t1 ) x1 , X (t 2 ) x 2 } • Second-Order PDF of a random process: 2 f X x1 , x 2 ; t1 , t 2 FX x1 , x 2 ; t1 , t 2 x1 .x 2 X (t , ) 𝑥2 𝑥1 𝑋(𝑡, 𝜁2 ) 𝑋(𝑡, 𝜁3 ) 𝑋(𝑡, 𝜁1 ) 0 t1 t2 𝑋(𝑡, 𝜁4 ) t 26 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃(𝑋 0 = 0, 𝑋 4 = −2) =? 27 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃 𝑋 0 = 0, 𝑋 4 = −2 = 𝑃 5 = 1/6 28 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃 𝑋 4 = −2|𝑋 0 = 0 =? 29 𝑆 = 1,2,3,4,5,6 = 𝜁1 , 𝜁2 , 𝜁3 , 𝜁4 , 𝜁5 , 𝜁6 𝑃 𝑋 4 = −2|𝑋 0 = 0 = 𝑃(𝑋 4 = −2, 𝑋 0 = 0) 1/6 = = 1/2 𝑃(𝑋 0 = 0) 2/6 30 • Relation between first-order and second-order can be presented as 𝐹𝑋 𝑥1 ; 𝑡1 = 𝐹𝑋 𝑥1 , ∞; 𝑡1 , 𝑡2 ∞ 𝑓𝑋 𝑥1 ; 𝑡1 = 𝑓𝑋 𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 𝑑𝑥2 −∞ • The nth order distribution of 𝑋(𝑡) is defined as the joint distribution: FX ( x1 , x 2 , x n ; t1 , t 2 , t n ) PX (t1 ) x1 , X (t 2 ) x 2 ,..., X (t n ) x n • • A stochastic process is defined to be completely or totally characterized if the joint densities for the random variables 𝑋 𝑡1 , 𝑋 𝑡2 , … , 𝑋(𝑡𝑛 ) are known for all times 𝑡1 , 𝑡2 , … , 𝑡𝑛 and all 𝑛. In general, a complete characterization is practically impossible, except in rare cases. As a result, it is desirable to define and work with various partial characterizations. Depending on the objectives of applications, a partial characterization often suffices to ensure the desired outputs. 31