Download Random Number Generator

Random Number Generation
Randomness in Simulations
 Among the fundamental aspects in simulation is to
represent random events
 Supporting randomness contributes to the
evaluation of systems in different possible states
 Computers by nature are not random (they do as
they programmed to)
 Therefore Random Number Generators are
needed
© Tallal Elshabrawy
2
Properties of Random Numbers
Random Number Generators (RNG) must satisfy the
properties of Random Numbers
 Uniformity
 Independence
© Tallal Elshabrawy
3
Uniformity
 Independent Samples
 Drawn from [0,1]
 𝐸𝑋 =
𝑓𝑋 𝑥
1
1
2
 𝑉𝑎𝑟 𝑋 =
0
1
12
1
0
𝐹𝑋 𝑥 = 𝑥
1
𝐹𝑋 𝑥
1
0
© Tallal Elshabrawy
0
𝑥<0
𝑓𝑋 𝑥 = 1 0 ≤ 𝑥 < 1
0
𝑥≥1
𝑥<0
0≤𝑥<1
𝑥≥1
1
4
Independence
Outcomes in a given interval do not affect the
upcoming outcomes
© Tallal Elshabrawy
5
Pseudo-Random Numbers
 Fake Randomness
 Random in Properties but Not Random in
Generation Process
 Deterministic Sequences with a Repeat Period
© Tallal Elshabrawy
6
Important Considerations for RNG
 Fast
 Sufficiently Long Cycles
 Replicable
 Representativeness of Random Benahvior
 Uniformity
 Independence
© Tallal Elshabrawy
7
RNG Techniques
 Mid-square method
 Linear Congruential Method
 Combined Linear Congruential Method
© Tallal Elshabrawy
8
Mid-Square Method by Von Neumann and Metropolis, 1940’s
Example
𝑋0 = 5497
 𝑋1 = 5497
2
= 30𝟐𝟏𝟕𝟎09 → 𝑋1 = 2170 → 𝑅1 = 0.2170
 𝑋2 = 2170
2
= 04𝟕𝟎𝟖𝟗00 → 𝑋2 = 7089 → 𝑅2 = 0.7089
 𝑋3 = 7089
2
= 50𝟐𝟓𝟑𝟗21 → 𝑋3 = 2539 → 𝑅3 = 0.2539
 …
© Tallal Elshabrawy
9
RNG Seed
Random number seed-initial random number used
to generate the next random number which is in turn
transformed into the new seed value.
In the Previous example, Seed = 5497
© Tallal Elshabrawy
10
Mid-Square Method: Poor Generators
𝑋0 = 5197
 𝑋1 = 5197
2
= 27𝟎𝟎𝟖𝟖09 → 𝑋1 = 0088 → 𝑅1 = 0.0088
 𝑋2 = 0088
2
= 00𝟎𝟎𝟕𝟕44 → 𝑋2 = 0077 → 𝑅2 = 0.0077
 𝑋3 = 0077
2
= 00𝟎𝟎𝟓𝟗29 → 𝑋3 = 0059 → 𝑅3 = 0.0059
 𝑋4 = 0059
2
= 00𝟎𝟎𝟑𝟒81 → 𝑋4 = 0034 → 𝑅4 = 0.0034
The selected seed leads the random sequence to
degenerate to ZERO
© Tallal Elshabrawy
11
Mid-Square Method: Poor Generators
𝑋0 = 6500
 𝑋1 = 6500
2
= 42𝟐𝟓𝟎𝟎00 → 𝑋1 = 2500 → 𝑅1 = 0.2500
 𝑋2 = 2500
2
= 06𝟐𝟓𝟎𝟎00 → 𝑋2 = 2500 → 𝑅2 = 0.2500
The selected seed leads the non-random sequence
© Tallal Elshabrawy
12
Mid-Square Method: Pros and Cons
 Advantages
 Simple to implement
 Disadvantages
 Difficult to choose initial seed that will give a good
sequence
 Strong tendency to degenerate fairly rapidly to zero
© Tallal Elshabrawy
13
Mid-Square Method: More Examples
Seed =
2784
1
07750656
0.7506
2
56340036
0.3400
3
11560000
0.5600
4
31360000
0.3600
5
12960000
0.9600
6
92160000
0.1600
7
02560000
0.5600
8
31360000
0.3600
9
12960000
0.9600
10
92160000
0.1600
11
02560000
0.5600
*
*
*
Sequence Repetition occurs once a previous output appears
in the sequence. NO LONGER RANDOM BEHAVIOUR
© Tallal Elshabrawy
14
Mid-Square Method: More Examples
Seed =
9713
1
94342369
0.3423
2
11716929
0.7169
3
51394561
0.3945
4
15563025
0.5630
5
31696900
0.6969
6
48566961
0.5669
7
32137561
0.1375
8
01890625
0.8906
9
79316836
0.3168
10
10036224
0.0362
11
00131044
0.1310
12
01716100
0.7161
…
…
84
07290000
0.2900
85
08410000
0.4100
86
16810000
0.8100
87
65610000
0.6100
88
37210000
0.2100
89
04410000
0.4100
*
*
Some POOR Sequences might look OK in the beginning
© Tallal Elshabrawy
15
The Mid-Product Method
 Improved Variation of Mid-Square Method
 Example
𝑍0 = 123, 𝑍1 = 456
 𝑍0 𝑍1 = 123 456 = 5𝟔𝟎𝟖8 → 𝑈1 = 0.608
 𝑍1 𝑍2 = 456 608 = 2𝟕𝟕𝟐48 → 𝑈2 = 0.772
 𝑍2 𝑍3 = 608 772 = 4𝟔𝟗𝟑76 → 𝑈3 = 0.693
 …
© Tallal Elshabrawy
16
Linear Congruential Method by D. H. Lehmer, 1951
 Recursive Relationship for sequnece of integers
𝑋𝑖 between 0 and m-1
𝑋𝑖+1 = 𝑎𝑋𝑖 + 𝑐 mod 𝑚
 𝑎
 𝑐
 𝑚
The Multiplier
The Increment
The Modulus
 If 𝑐 ≠ 0 →Mixed Congruential Method
 If 𝑐 = 0 →Multiplicative Congruential Method
© Tallal Elshabrawy
17
Linear Congruential Method
 Integers between 𝑋𝑖 [0,m-1]
 For [0,1[ RNG
𝑋𝑖
𝑅𝑖 = , 𝑖 = 1,2, …
𝑚
 For [0,1] RNG
𝑋𝑖
𝑅𝑖 =
𝑖 = 1,2, …
𝑚−1
© Tallal Elshabrawy
18
Linear Congruential Method: Example
𝑋0 = 27 (Seed) 𝑎 = 17, 𝑐 = 43 𝑎𝑛𝑑 𝑚 = 100
 𝑋1 = 17 × 27 + 43 mod 100 = 502mod100 = 2 → 𝑅1 =0.02
 𝑋2 = 17 × 2 + 43 mod 100 = 77mod100 = 77 → 𝑅2 =0.77
 𝑋3 = 17 × 77 + 43 mod 100 = 1352mod100 = 52 → 𝑅3 =0.52
 …
© Tallal Elshabrawy
19
Example
𝑋𝑛 = 5𝑋𝑛−1 + 1 mod 16
For Seed 𝑋0 = 5
𝑋1 = 26 mod 16 = 10
The sequence of 32 numbers by such procedure are:
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5;
10, 3, 0, 1, 6, 15, 12, 13, 2, 11, 8, 9, 14, 7, 4, 5
By dividing by 16, the random numbers are:
0.625, 0.1875, 0, 0.0625, 0.375, 0.9375, 0.75, 0.8125, 0.125,
0.6875, 0.5, 0.5625, 0.875, 0.4375, 0.25, 0.3125, 0.625, …
© Tallal Elshabrawy
20
Selection of LCG Parameters
 Statistical properties of the generated sequence as well as
the cycle length are affected by selection of:
 𝑎
 𝑐
 𝑚
 𝑋0
 Naturally 𝑚 should be selected to be equal to the capacity
of a computer word
 𝑚 = 2𝑏 (binary machine), where 𝑏 is number of bits in a
computer word
 𝑚 = 10𝑑 (decimal machine), where 𝑑 is number of
digits in a computer word
© Tallal Elshabrawy
21
Selection of LCG Parameters
 The modulus 𝑚 should be large


All generated numbers 𝑋𝑖 are between 0 and 𝑚 − 1
The period can never be more than 𝑚
 For mod 𝑚 computation to be efficient, 𝑚 must be a power
of 2

Mod 𝑚 can be obtained by dropping the leftmost binary digits in
𝑋𝑖+1 and using only the 𝑏 rightmost binary digits
© Tallal Elshabrawy
22
Example: A 4-bit Computer
Let 𝑚 = 24 = 16, 𝑎 = 5, 𝑐 = 3, 𝑋0 = 7
𝑋𝑖+1 = 5𝑋𝑖 + 3 𝑚𝑜𝑑 16
𝑋1 = 5 × 7 + 3 𝑚𝑜𝑑 16 = 38 𝑚𝑜𝑑 16 = 6
𝑋2 = 5 × 6 + 3 𝑚𝑜𝑑 16 = 33 𝑚𝑜𝑑 16 = 1
𝑋3 = 5 × 1 + 3 𝑚𝑜𝑑 16 = 8 𝑚𝑜𝑑 16 = 8
𝑋1 = 38 𝑚𝑜𝑑 16 → 𝑏𝑖𝑛𝑎𝑟𝑦 38 = 10𝟎𝟏𝟏𝟎 = 𝟔
𝑋2 = 33 𝑚𝑜𝑑 16 → 𝑏𝑖𝑛𝑎𝑟𝑦 33 = 10𝟎𝟎𝟎𝟏 = 𝟏
𝑋1 = 8 𝑚𝑜𝑑 16 → 𝑏𝑖𝑛𝑎𝑟𝑦 8 = 00𝟏𝟎𝟎𝟎 = 𝟖
No Need to Perform Division!
© Tallal Elshabrawy
23
Example
Using the multiplicative congruential method, find the
period of the generator for 𝑎 = 13, 𝑚 = 26 , 𝑋0 = 1,2,3,4
 When the seed is 1 and 3, the sequence has period 16.
 A period of length 8 is achieved when the seed is 2
 A period if length 4 occurs when the seed is 4
© Tallal Elshabrawy
24
Solution
𝑖
𝑋𝑖
𝑋𝑖
𝑋𝑖
𝑋𝑖
0
𝑋0 = 1
𝑋0 = 2
𝑋0 = 3
𝑋0 = 4
1
13
26
39
52
2
41
18
59
36
3
21
42
63
20
4
17
34
51
4
5
29
58
23
6
57
50
43
7
37
10
47
8
33
2
35
9
45
7
10
9
27
11
53
31
12
49
19
13
61
55
14
25
11
15
5
15
16
1
3
© Tallal Elshabrawy
25
Selection of LCG Parameters for Max. Period
If 𝑐 ≠ 0, the maximum possible period 𝑚 is obtained if
and only if:
 Integers 𝑚 and 𝑐 are relatively prime, that is, have no
common factors other than 1
 Every prime number that is a factor of 𝑚 is also a factor
of 𝑎 − 1
 If integer 𝑚 is a multiple of 4, 𝑎 − 1 should be a multiple
of 4
Notice that all of these conditions are met if
𝑚 = 2𝑏 𝑎 = 4𝑘 − 1 𝑎𝑛𝑑 𝑐 𝑖𝑠 𝑜𝑑𝑑
© Tallal Elshabrawy
26
Selection of LCG Parameters for Max. Period
For 𝑚 a power of 2, say 𝑚 = 2𝑏 , and 𝑐 = 0, the
longest possible period is 𝑃 = 𝑚 4 = 2𝑏−1 , which is
achieved provided that:
𝑋0 𝑖𝑠 𝑜𝑑𝑑, 𝑎 = 3 + 8𝑘 𝑜𝑟 𝑎 = 5 + 8𝑘, 𝑘 = 0,1, …
© Tallal Elshabrawy
27
Multiplicative LCG with 𝑚 = 2𝑏
𝑚 = 2𝑏 , Maximum period achieved if:
 Multiplier 𝑎 is of the form 8𝑘 ± 3
 the initial seed is an odd integer
Maximum possible period 2𝑏−2 = One-fourth 𝑚, may
not be too small for large 𝑚
© Tallal Elshabrawy
28
Example
𝑋𝑖+1 = 5𝑋𝑖 𝑚𝑜𝑑 25
 Using a seed of 𝑋0 = 1
 5, 25, 29, 17, 21, 9, 13, 1, 5, …
 Period = 32 4 = 8
 With 𝑋0 = 2, the sequence is
 10, 18, 26, 2, 10 …
 Here the period is only 4
© Tallal Elshabrawy
29
Example
Multiplier 𝑎 is not of the form 8𝑘 ± 3
𝑋𝑖+1 = 7𝑋𝑖 𝑚𝑜𝑑 25
Using a seed of 𝑋0 = 1, we get the sequence
 7, 17, 23, 1, 7, …
 The period is only 4
© Tallal Elshabrawy
30
Selection of LCG Parameters for Max. Period
For 𝑚 a prime number and 𝑐 = 0,
 the longest possible period is 𝑃 = 𝑚 − 1,
 achieved provided that the multiplier, 𝑎, has the
property that the smallest integer 𝑘 such that 𝑎𝑘 − 1 is
divisible by 𝑚 is 𝑘 = 𝑚 − 1
 In other words, the multiplier 𝑎 is a primitive root of the
modulus 𝑚 where 𝑎 is a primitive root of 𝑚 if and only if
𝑎𝑘 𝑚𝑜𝑑 𝑚 ≠ 1 for 𝑘 = 1, 2, … , 𝑚 − 2
© Tallal Elshabrawy
31
Example
 𝑋𝑖+1 = 3𝑋𝑖 𝑚𝑜𝑑 31, 𝑆𝑒𝑒𝑑 𝑋0 = 1
 1, 3, 9, 27, 19, 26, 16, 17, 20, 29, 25, 13, 8, 24, 10, 30, 28, 22,
4, 12, 5,15, 14, 11, 2, 6, 18, 23, 7, 21, 1
 The period is 30
 3 is a primitive root of 31
 𝑋𝑖+1 = 5𝑋𝑖 𝑚𝑜𝑑 31, 𝑆𝑒𝑒𝑑 𝑋0 = 1
 1, 5, 25, 1, …
 The period is only 3
 5 is not a primitive root of 31 53 𝑚𝑜𝑑 31 =
© Tallal Elshabrawy
32
Full Period Generators
A generator that has the maximum possible period,
𝑚, is called a full-period generator
Examples:
 𝑋𝑖+1 = 234 + 1 𝑋𝑖 + 1 𝑚𝑜𝑑 235
 𝑋𝑖+1 = 218 + 1 𝑋𝑖 + 1 𝑚𝑜𝑑 235
© Tallal Elshabrawy
33