Download Abstract Representation: Your Ancient Heritage

Great Theoretical Ideas In Computer Science
Steven Rudich
Lecture 11
CS 15-251
Feb 14, 2004
Spring 2003
Carnegie Mellon University
Counting III: Pascal’s Triangle,
Polynomials, and Vector Programs
1
X+
2
X+
3
X
1 + X1 + X 2 + X
3
+ … + Xn-1 + Xn =
Xn+1 - 1
X- 1
The Geometric Series
1+
X1
+
X2
+X
3
+…+
Xn
+ ….. =
1
1-X
The Infinite Geometric Series
1+
X1
+
X2
+X
3
+…+
Xn
+ ….. =
1
1-X
(X-1) ( 1 + X1 + X2 + X 3 + … + Xn + … )
=
X1 + X2 + X 3 + …
+ Xn + Xn+1 + ….
- 1 - X1 - X2 - X 3 - … - Xn-1 – Xn - Xn+1 - …
=
1
1+
X1
+
X2
+X
3
+…+
Xn
+ ….. =
1 + x + …..
____________________
1-x | 1
-[1 – x]
_____
x
-[x – x2 ]
______
x2
-....
1
1-X
1+
aX1
+
a2X2
+
a3X 3
+…+
a n Xn
+ ….. =
1
1 - aX
Geometric Series (Linear Form)
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
1
(1 – aX)(1-bX)
Geometric Series
(Quadratic Form)
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
1 + c1X1 + .. + ck Xk + …
Suppose we multiply this out
to get a single, infinite
polynomial.
What is an expression for Cn?
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
1 + c1X1 + .. + ck Xk + …
cn =
a0bn + a1bn-1 +… aibn-i… + an-1b1 + anb0
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
1 + c1X1 + .. + ck Xk + …
If a = b then
cn = (n+1)(an)
a0bn + a1bn-1 +… aibn-i… + an-1b1 + anb0
a0bn + a1bn-1 +… aibn-i… + an-1b1 + anb0 =
an+1 – bn+1
a- b
(a-b) (a0bn + a1bn-1 +… aibn-i… + an-1b1 + anb0)
=
a1bn +… ai+1bn-i…
+ anb1 + an+1b0
- a0bn+1 – a1bn… ai+1bn-i… - an-1b2 - anb1
= - bn+1
=
+
an+1
an+1 – bn+1
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
1 + c1X1 + .. + ck Xk + …
if a  b then
cn =
an+1 – bn+1
a- b
a0bn + a1bn-1 +… aibn-i… + an-1b1 + anb0
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
=
=
n=0..1
or
n=0..1
1
(1 – aX)(1-bX)
an+1 – bn+1
a- b
(n+1)an
Xn
Xn
when a=b
Geometric Series (Quadratic Form)
Previously, we saw that
Polynomials Count!
The number of
ways to rearrange
the letters in the
word BANANA.
Choice tree for terms of (1+X)3
1
1
1
1
X
X
X
X
1
1
X
X
X
1
X
1
X
X2
X
X2
X2
X3
Combine like terms to get 1 + 3X + 3X2 + X3
The Binomial Formula
(1  X)
n
nI F
nI
nI
nI
nI
F
F
F
F
G
G
XG
X . . .G
X . . .G
X
J
J
J
J
J
0K H
1K H
2K
kK
nK
H
H
H
2
k
Binomial Coefficients
binomial
expression
n
The Binomial Formula
n

 k
n
(1  x )      x
k 0  k 
n
One polynomial,
two representations
n

 k
n
(1  x )      x
k 0  k 
n
“Product form” or
“Generating form”
“Additive form” or
“Expanded form”
Power Series Representation
 n k
(1  x)      x
k 0  k 

 n k


x



“Closed form” or
k
k

0


“Generating form”
n
n
n
Since    0 if k  n
k 
“Power series” (“Taylor series”) expansion
By playing these two representations against
each other we obtain a new representation of
a previous insight:
n k
(1  x )      x
k 0  k 
n
n
Let x=1.
n
2   
k 0  k 
n
n
The number of
subsets of an
n-element set
By varying x, we can discover new identities
n k
(1  x )      x
k 0  k 
n
n
Let x= -1.
n
0      ( 1) k
k 0  k 
n
Equivalently,
n
n
n
n 1


2

k   k 
k even  
k odd  
n
The number of even-sized subsets of an n element
set is the same as the number of odd-sized subsets.
n k
(1  x )      x
k 0  k 
n
n
Let x= -1.
n
k
0      ( 1)
k 0  k 
n
Equivalently,
n
n
n
n 1


2

k   k 
k even  
k odd  
n
 n k
(1  x)      x
k 0  k 
n
n
We could discover new identities
by substituting in different
numbers for X. One cool idea is to
try complex roots of unity,
however, the lecture is going in
another direction.
 n k
(1  x)      x
k 0  k 
n
n
Proofs that work by manipulating
algebraic forms are called
“algebraic” arguments. Proofs
that build a 1-1 onto
correspondence are called
“combinatorial” arguments.
n
n
n
n 1


2

   
k even  k 
k odd  k 
n
Let On be the set of binary strings of
length n with an odd number of ones.
Let En be the set of binary strings of
length n with an even number of ones.
We gave an algebraic proof that
On  =  En 
A Combinatorial Proof
Let On be the set of binary strings of length n
with an odd number of ones.
Let En be the set of binary strings of length n
with an even number of ones.
A combinatorial proof must construct a one-toone correspondence between On and En
An attempt at a correspondence
Let fn be the function that takes an
n-bit string and flips all its bits.
fn is clearly a one-to-one
and onto function
...but do even n work? In f6
we have
for odd n. E.g. in f7 we have
110011  001100
0010011  1101100
101010  010101
1001101  0110010
Uh oh. Complementing maps
evens to evens!
A correspondence that works for all n
Let fn be the function that takes an
n-bit string and flips only the first bit.
For example,
0010011  1010011
1001101  0001101
110011  010011
101010  001010
 n k
(1  x)      x
k 0  k 
n
n
The binomial coefficients have so
many representations that many
fundamental mathematical
identities emerge…
The Binomial Formula
(1+X)0 =
1
(1+X)1 =
1 + 1X
(1+X)2 =
1 + 2X + 1X2
(1+X)3 =
1 + 3X + 3X2 + 1X3
(1+X)4 =
1 + 4X + 6X2 + 4X3 + 1X4
Pascal’s Triangle:
kth row are the coefficients of (1+X)k
(1+X)0 =
1
(1+X)1 =
1 + 1X
(1+X)2 =
1 + 2X + 1X2
(1+X)3 =
1 + 3X + 3X2 + 1X3
(1+X)4 =
1 + 4X + 6X2 + 4X3 + 1X4
kth Row Of Pascal’s Triangle:
 n  n  n  n  n
  ,   ,   ,...,   ,...  
 0  1  2  k   n
(1+X)0 =
1
(1+X)1 =
1 + 1X
(1+X)2 =
1 + 2X + 1X2
(1+X)3 =
1 + 3X + 3X2 + 1X3
(1+X)4 =
1 + 4X + 6X2 + 4X3 + 1X4
Inductive definition of kth entry of nth row:
Pascal(n,0) = Pacal (n,n) = 1;
Pascal(n,k) = Pascal(n-1,k-1) + Pascal(n,k)
(1+X)0 =
1
(1+X)1 =
1 + 1X
(1+X)2 =
1 + 2X + 1X2
(1+X)3 =
1 + 3X + 3X2 + 1X3
(1+X)4 =
1 + 4X + 6X2 + 4X3 + 1X4
“Pascal’s Triangle”
0
0  1
 
1 
1

1
0
1  1
 
 
2
2
2

1

2
0
1 
2  1
 
 
 
3
 3
3
 3

1

3

3
0
1 
2
 3  1
 
 
 
 
Al-Karaji, Baghdad 953-1029
Chu Shin-Chieh 1303
The Precious Mirror of the Four Elements
. . . Known in Europe by 1529
Blaise Pascal 1654
Pascal’s Triangle
1
1
1
1

“It is extraordinary
how fertile in
1
1
properties the
triangle is.
1
2
1
Everyone can
try his
1
3
3
1
hand.”
4
5
6
6
10
15
4
10
20

1
5
15
1
6
1

Summing The Rows
n
n
2   
k 0  k 
1
=1
1 + 1
=2
1 + 2 + 1
=4
1 + 3 + 3 + 1
=8
1 + 4 + 6 + 4 + 1
=16
1 + 5 + 10 + 10 + 5 + 1
=32
n
1 + 6 + 15 + 20 + 15 + 6 + 1 =64



1
1
1
1
1
1
1
2
3
4
5
6
1
3
6
10
15
1
1
4
10
20
1
5
15
6+20+6 = 1+15+15+1
1
6
1

Summing on 1st Avenue
1
1
1
1
1
1
6
1
2
3
1
4
5
10
15
1
3
6
1
4
10
20

 i   n  1 n  (n  1)
i     


2
i k
i  k  1
 2 
n
15
n
1
5
1
6
1

Summing on kth Avenue
1
1
1
1
1
1
6
1
2
3
1
4
5
10
15
1
3
6
1
4
10
20

 i   n  1

 

k
k

1
ik  


n
15
1
5
1
6
1

1
=2
=3
1 2 1 =5
=8
1 3 3 1 =13
1
1
1

6
1
4
5
10
15
1
6
4
10
20

1
5
15
1
6
1

1
1
2
1
1

6
2
3
1
4
5
10
15
1
2
2
2
2
2
2
1
1
1
3
6
1
4
10
20

15
1
5
1
6
1

Al-Karaji Squares
1
1
1
1
1
1
1
1
=1
2 + 2*1
=4
3 + 2*3
4 + 2*6
5 + 2*10
6 + 2*15

=0
20

1
=9
4
1
10
5
15
=16
1
6
=25
1=36

All these properties can be
proved inductively and
algebraically. We will give
combinatorial proofs using
the Manhattan block walking
representation of binomial
coefficients.
How many shortest routes from A to
B?
A
B
 10 
5
 
Manhattan
j’th Street
4
3
2
1
0 0
1
k’th Avenue
2
3
4
Level n
Manhattan
……… 0 k’th Avenue
0
1
………….
1
2
……………………
2
3
…………………………
3
4
…………………………………
4
Level n
Manhattan
……… 0 k’th Avenue
0
1
………….
1
2
……………………
2
3
…………………………
3
4
…………………………………
4
level n .
. .
. . .
. . . . .
. . . . 1. 1
. . . . 2
. . .13
1
. 1. 4 6
1
6
5
k’th Avenue
1
3
1
4
1
1
1
5
10 10
15 20 15 6
.....
.....
1
level n . . . . .
1
.....
3
1
.....
4
1
1
5
6
1
2
1
3
6
10
15
k’th Avenue
1
1
1
5
10
20
1
4
15
6
 n   n  1  n  1
 k    k  1   k 
  
 

.....
.....
1
level n . . . . .
1
.....
3
1
.....
4
1
1
5
6
1
2
1
3
6
10
15
k’th Avenue
1
1
1
5
10
20
1
4
15
n
n 1

2

k 
k even  
6
.....
.....
1
level n . . . . .
1
.....
3
1
.....
4
1
1
5
1
2
15
1
3
6
10
6
k’th Avenue
1
1
2
1
5
10
20
1
4
15
6
 n   2n 

k    n 
k 0  
 
n
By convention:
0!  1
(empty product = 1)
n
 k   1 if k  0
 
n
 k   0 if k  0 or k  n
 
.....
.....
1
level n . . . . .
1
.....
3
1
.....
4
1
1
5
6
 i   n  1

 

i 1  k 
 k  1
n
1
2
1
3
6
10
15
k’th Avenue
1
1
1
5
10
20
1
4
15
6
Corollary ( k  1)
 n  1  n(n  1)
i



2
i 1
 2 
n
Application (Al-Karaji):
n
2
2
2
2
i

1

2

3


 n2
i 0
 (1  0  1)  (2  1  2)  (3  2  3) 
 1 0  2 1  3  2 
 ( n( n  1)  n)
n
 n( n  1)   i
i 1
 2   3   4   5 
 n    n  1
 2                  

2
2
2
2
2
2
  

       
 n  1   n  1  (2n  1)(n  1)n
 2




6
 3   2 
Vector Programs
Let’s define a (parallel) programming
language called VECTOR that operates
on possibly infinite vectors of numbers.
Each variable V! can be thought of as:
< * , * , * , * , *, *, . . . . . . . . . >
0 1
2 3 4 5.........
Vector Programs
Let k stand for a scalar constant
<k> will stand for the vector <k,0,0,0,…>
<0> = <0,0,0,0,….>
<1> = <1,0,0,0,…>
V! + T! means to add the vectors position-wise.
<4,2,3,…> + <5,1,1,….> = <9,3,4,…>
Vector Programs
RIGHT(V!) means to shift every number in V! one
position to the right and to place a 0 in position 0.
RIGHT( <1,2,3, …> ) = <0,1,2,3,. …>
Vector Programs
Example:
Stare
V! := <6>;
V! := RIGHT(V!) + <42>;
V! := RIGHT(V!) + <2>;
V! := RIGHT(V!) + <13>;
V! = <6,0,0,0,..>
V! = <42,6,0,0,..>
V! = <2,42,6,0,..>
V!= <13,2,42,6,.>
V! = < 13, 2, 42, 6, 0, 0, 0, . . . >
Vector Programs
Example:
Stare
V! := <1>;
V! = <1,0,0,0,..>
Loop n times:
V! := V! + RIGHT(V!);
V! = <1,1,0,0,..>
V! = <1,2,1,0,..> V!
= <1,3,3,1,.>
V! = nth row of Pascal’s triangle.
1
X+
2
X +
Vector programs
can be implemented
by polynomials!
3
X
Programs -----> Polynomials
The vector V! = < a0, a1, a2, . . . > will be
represented by the polynomial:
Formal Power Series
The vector V! = < a0, a1, a2, . . . > will be
represented by the formal power series:
V! = < a0, a1, a2, . . . >
<0> is represented by
<k> is represented by
0
k
V! + T! is represented by
(PV + PT)
RIGHT(V! ) is represented by
(PV X)
Vector Programs
Example:
V! := <1>;
PV := 1;
Loop n times:
V! := V! + RIGHT(V!);
PV := PV + PV X;
V! = nth row of Pascal’s triangle.
Vector Programs
Example:
V! := <1>;
PV := 1;
Loop n times:
V! := V! + RIGHT(V!);
PV := PV (1+ X);
V! = nth row of Pascal’s triangle.
Vector Programs
Example:
V! := <1>;
Loop n times:
V! := V! + RIGHT(V!);
PV = (1+ X)n
V! = nth row of Pascal’s triangle.
What is the coefficient of Xk
in the expansion of:
( 1 + X + X2 + X3 + X4 + . . . . )n ?
Each path in the choice tree for the
cross terms has n choices of exponent
e1, e2, . . . , en ¸ 0. Each exponent can be
any natural number.
Coefficient of Xk is the number of
non-negative solutions to:
e 1 + e 2 + . . . + en = k
What is the coefficient of Xk
in the expansion of:
( 1 + X + X2 + X3 + X4 + . . . . )n ?
 n  k  1


n
-1


( 1 + X + X2 + X3 + X4 + . . . . )n =

 n  k  1 k
1
 
X
n
n -1 
1  X 
k 0 
What is the coefficient of Xk in the
expansion of:
(a0 + a1X + a2X2 + a3X3 + …) ( 1 + X + X2 + X3 + . . . )
= (a0 + a1X + a2X2 + a3X3 + …) / (1 – X)
?
a0 + a1 + a2 + .. + ak
(a0 + a1X + a2X2 + a3X3 + …) / (1 – X)
=
i k
 k

a
X


i


k 0  i 0


=
PREFIXSUM(a0 + a1X + a2X2 + a3X3 + …)
Let’s add an instruction
called PREFIXSUM to our
VECTOR language.
W! := PREFIXSUM(V!)
means that the ith position
of W contains the sum of all
the numbers in V from
positions 0 to i.
What does this program output?
V! := 1! ;
Loop k times: V! := PREFIXSUM(V!) ;
0 k’th Avenue
1
2
3
4
Can you see how
PREFIXSUM can be
represented by a familiar
polynomial expression?
W! := PREFIXSUM(V!)
is represented by
PW = PV / (1-X)
= PV (1+X+X2+X3+ ….. )
Al-Karaji Program
Zero_Ave := PREFIXSUM(<1>);
First_Ave := PREFIXSUM(Zero_Ave);
Second_Ave :=PREFIXSUM(First_Ave);
Output:=
First_Ave + 2*RIGHT(Second_Ave)
OUTPUT! = <1, 4, 9, 25, 36, 49, …. >
Al-Karaji Program
Zero_Ave = 1/(1-X);
First_Ave = 1/(1-X)2;
Second_Ave = 1/(1-X)3;
Output =
1/(1-X)2 + 2X/(1-X)3
(1-X)/(1-X)3 + 2X/(1-X)3
= (1+X)/(1-X)3
(1+X)/(1-X)3
Zero_Ave := PREFIXSUM(<1>);
First_Ave := PREFIXSUM(Zero_Ave);
Second_Ave :=PREFIXSUM(First_Ave);
Output:=
RIGHT(Second_Ave) + Second_Ave
Second_Ave
= <1, 3, 6, 10, 15,.
RIGHT(Second_Ave) = <0, 1, 3, 6, 10,.
Output
= <1, 4, 9, 16, 25
(1+X)/(1-X)3
outputs <1, 4, 9, ..>
X(1+X)/(1-X)3
outputs <0, 1, 4, 9, ..>
The kth entry is k2
X(1+X)/(1-X)3 =  k2Xk
What does X(1+X)/(1-X)4 do?
X(1+X)/(1-X)4 expands to :
 Sk Xk
where Sk is the sum of the
first k squares
Aha! Thus, if there is an
alternative interpretation of
the kth coefficient of
X(1+X)/(1-X)4
we would have a new way to
get a formula for the sum of
the first k squares.
Using pirates and gold we
found that:

 n  k  1 k
1
 
X
n
n -1 
1  X 
k 0 
THUS:
1
1  X 
k  3 k
 
X
3 
k 0 

4
Coefficient of Xk in PV = (X2+X)(1-X)-4 is
the sum of the first k squares:
1
1  X 
k  3 k
 
X
3 
k 0 

4
Vector programs -> Polynomials
-> Closed form expression
Expressions of the form
Finite Polynomial / Finite Polynomial
are called Rational Polynomial
Expressions.
Clearly, these expressions have some
deeper interpretation as a
programming language.
What about this one?
X/(1-X-X2)
x
0
1
2
3
 F0 x  F1 x  F2 x  F3 x 
2
1 x  x

  Fi x
i 0
The action of dividing one
polynomial by another can
simulate a program to recursively
compute Fibonacci numbers.
i
Vector Program I/O
Example:
INPUT I!; /* not allowed to alter I! */
V! := I! + 1;
Loop n times:
V! := V! + RIGHT(V!) + I! ;
OUTPUT V!
Vector Recurrence Relations
Let P be a vector program that takes input.
A vector relation is any statement of the
form:
V! = P( V! )
If there is a unique V! satisfying the
relation, then V! is said to be defined by the
relation V! = P( V! ).
Fibonacci Numbers
Recurrence Relation Definition:
F0  0, F1  1,
Fn  Fn1  Fn 2 , n  1
Vector Recurrence Relation
Definition:
F! = RIGHT( F! + <1> ) + RIGHT( RIGHT( F! ) )
F! = RIGHT( F! + <1> ) + RIGHT( RIGHT( F! ) )
F! = a0, a1
RIGHT(F! +<1>) =
, a2, a3, a4, . . .
0, a0 + 1, a1,
RIGHT( RIGHT( F! ) )
= 0, 0
, a0,
a2, a3,
a1, a2, a3, .
F! = RIGHT( F! + 1 ) + RIGHT( RIGHT( F! ) )
F = a0 + a1 X + a2 X2 + a3 X3 +
RIGHT(F + 1) = (F+1) X
RIGHT( RIGHT( F ) )
=
F X2
F = (F + 1) X + F
F =
2
X
a0 + a1 X + a2 X2 + a3 X3 + . . .
RIGHT(F + 1) = (F+1) X
RIGHT( RIGHT( F ) )
=
F X2
F = F X + X + F
Solve for F.
F - FX - FX2 = X
F(1-X-X2) = X
F = X/(1-X-X2)
2
X
What is the Power Series
Expansion of x / (1-x-x2) ?
Since the bottom is
quadratic we can factor it.
X / (1-X-X2) =
X/(1- X)(1 – (-)-1X)
where  =
“The Golden Ratio”
X
(1 – X)(1- (--1X)
=
n=0..∞
?
Linear factors on the bottom
Xn
(1 + aX1 + a2X2 + … + anXn + …..) (1 + bX1 + b2X2 + … + bnXn + …..) =
=
=
n=0..1
1
(1 – aX)(1-bX)
an+1 – bn+1
a- b
Xn
Geometric Series (Quadratic Form)
1
(1 – X)(1- (--1X)
=
n=0.. ∞
n+1 – (--1)n+1
5
Xn
Geometric Series (Quadratic Form)
X
(1 – X)(1- (--1X)
=
n=0.. ∞
n+1 – (--1)n+1 n+1
X
5
Power Series Expansion of F
x
0
1
2
3
 F0 x  F1 x  F2 x  F3 x 
2
1 x  x

x

2
1 x  x
i 0

  Fi x
i 0
i




1
1
i
i
      x
5 
   
i
x
0
1
2
3
 F0 x  F1 x  F2 x  F3 x 
2
1 x  x

  Fi x
i 0
The ith Fibonacci number is:


i 0
i

 1 
1
i
     
5 
   
i
References
Applied Combinatorics, by Alan Tucker
Generatingfunctionology, Herbert Wilf