Download More Lambda Calculus

Announcements
More Lambda Calculus
• Work on your project (probably
background reading)
• I am looking at your proposals, but come
talk to me if you have concerns
Meeting 17, CSCI 5535, Spring 2009
2
Plan
Recall
• Last Time (Introduce Lambda Calculus)
– Syntax
– Substitution
Goal: Come up with a “core” language that’s as
Goal
small as possible and still Turing
complete
• Today (Lambda Calculus in “Real Life”)
–
–
–
–
–
Operational Semantics
Evaluations strategies
Equality
Encodings
Fixed Points
This will give a way of illustrating important
language features and algorithms
3
4
Lambda Syntax
Combinators
• The λ-calculus has 3 kinds of expressions (terms)
e ::= x
Variables
| λx. e
Functions (abstractions)
| e1 e2
Application
• λx. e is a one-argument anonymous function
with body e
• e1 e2 is a function application
• A λ-term without free variables is closed or a
combinator
• Some interesting combinators:
I
= λ x. x
Explain
K
= λ x. λ y. x
these
informally
S
= λ f. λ g. λ x. f x (g x)
D
= λ x. x x
Y
= λ f. (λ x. f (x x)) (λ x. f (x x))
• Theorem: any closed term is equivalent to one
written with just S, K and I
– Example: D =β S I I
– (we’ll discuss this form of equivalence later)
5
6
1
Informal Semantics
Informal Semantics
• All we’ve got are functions, so all we can do is
call them!
• All we’ve got are functions, so all we can do is
call them!
• The evaluation of
(λ x. e) e’
– Binds x to e’
– Evaluates e with the new binding
– Yields the result of this evaluation
• Like a function call, or like “let x = e’ in e”
• Example:
(λ f. f (f e)) g evaluates to g (g e)
7
8
Informal Semantics
Operational Semantics
• All we’ve got are functions, so all we can do is
call them!
• The evaluation of
(λ x. e) e’
• Many operational semantics for the λ-calculus
• All are based on the equation
(λ
λ x. e1) e2 =β [e2/x]e1
usually read from left to right
• This is called the β-rule and the evaluation
step a β-reduction
• The subterm (λ x. e1) e2 is a β-redex
• We write e →β e’ to say that e β-reduces to e’
in one step
• We write e →β* e’ to say that e β-reduces to e’
in 0 or more steps
– Binds x to e’
– Evaluates e with the new binding
– Yields the result of this evaluation
• Like a function call, or like “let x = e’ in e”
• Example:
(λ f. f (f e)) g evaluates to g (g e)
– Remind you of the small-step opsem term rewriting?
9
Operational Semantics
10
Examples of Evaluation
• The identity function:
(λ
λ x. x) E → [E / x] x = E
• Another example with the identity:
(λ
λ f. f (λ
λ x. x)) (λ
λ x. x) →
betabeta
-reduction
(λx.e1) e2 →β [e2/x]e1
11
12
2
Examples of Evaluation
Examples of Evaluation
• The identity function:
(λ
λ x. x) E → [E / x] x = E
• Another example with the identity:
(λ
λ f. f (λ
λ x. x)) (λ
λ x. x) →
[λ
λ x. x / f] f (λ
λ x. x)) =
[λ
λ x. x / f] f (λ
λ y. y)) =
(λ
λ x. x) (λ
λ y. y) →
[λ
λ y. y / x] x = λ y. y
• A non-terminating evaluation:
(λ
λ x. xx) (λ
λ y. yy) →
[λ
λ y. yy / x] xx = (λ
λ y. yy) (λ
λ y. yy)
→ …
• Try T T, where T = λx. x x x
14
13
Evaluation and the Static Scope
Another View of Reduction
• The definition of substitution
guarantees that evaluation respects
static scoping:
• The application
λx. e
(λ x. (λ y. y x)) (y (λ x. x)) →β λ z. z (y (λ v. v))
e
xxx
(y remains free, i.e., defined externally)
g
• Becomes:
• If we forget to rename the bound y:
e
(λ x. (λ y. y x)) (y (λ x. x)) →β* λ y. y (y (λ v. v))
g
(y was free before but is bound now)
g
g
(terms can grow substantially through β-reduction!)
15
16
Normal Forms
Structural Operational Semantics
• A term without redexes is in normal form
• A reduction sequence stops at a normal form
• We define a small-step reduction relation
(λx. e1) e2 → [e2/x]e1
• If e is in normal form and e →β* e’ then
e is identical to e’
e1 → e1’
e1 e2 → e1’ e2
e2 → e2’
e1 e2 → e1 e2’
e → e’
λx. e → λx. e’
• This is a non-deterministic semantics
• Note that we evaluate under λ (where?)
• K = λ x. λ y. x is in normal form
• K I is not in normal form
17
18
3
Lambda Calculus Contexts
Lambda Calculus Contexts
• Define contexts with one hole
• H ::= • |
• Define contexts with one hole
• H ::= • | λ x. H | H e | e H
• Write H[e] to denote the filling of the
hole in H with the expression e
• Example:
H = λ x. x • H[λ y. y] = λ x. x (λ y. y)
• Filling the hole allows variable capture!
H = λ x. x • H[x] = λ x. x x
19
Contextual Operational Semantics
20
Contextual Operational Semantics
e → e’
H[e] → H[e’]
(λx. e1) e2 → [e2/x]e1
21
• Contexts allow concise formulations of
congruence rules (application of local
reduction rules on subterms)
• Reduction occurs at a β-redex that can be
anywhere inside the expression
• The latter rule is called a congruence or
structural rule
• The above rules to not specify which redex
must be reduced first
22
The Order of Evaluation
The Diamond Property
• In a λ-term there could be more than
one instance of (λ x. e1) e2, as in:
(λ y. (λ x. x) y) E
• A relation R has the diamond property if whenever
e R e1 and e R e2 then there exists e’ such that e1 R e’
and e2 R e’
e
R
R
– Could reduce the inner or outer λ
– Which one should we pick?
R
R
e’
inner
outer
(λ
λ y. [y/x] x) E = (λ
λ y. y) E [E/y] (λ
λ x. x) y = (λ
λ x. x) E
E
e2
e1
(λ
λ y. (λ
λ x. x) y) E
•
•
23
→β does not have the diamond property
→β* has the diamond property (Church-Rosser
property, confluence property)
– simplest known proof is quite technical
24
4
A Diamond In The Rough
Beta Equality
• Languages defined by non-deterministic
sets of rules are common
• Let =β be the reflexive, transitive and
symmetric closure of →β
=β is (→
→β ∪ ←β)*
– Logic programming languages
– Expert systems
– Constraint satisfaction systems
• That is, e =β e’ if e converts to e’ via a
sequence of forward and backward →β
•
•
e
•
e’
• And thus most pointer analyses …
– Dataflow systems
– Makefiles
• It is useful to know whether such
systems have the diamond property
25
26
The Church-Rosser Theorem
Corollaries
• If e1 =β e2 then there exists e’ such that
e1 →β* e’ and e2 →β* e’
•
•
e1
•
e2
•
•
e’
• If e1 =β e2 and e1 and e2 are normal
forms then e1 is identical to e2
• Proof (sketch): apply the diamond
property as many times as necessary
– From CR we have ∃e’. e1 →β* e’ and e2 →β* e’
– Since e1 and e2 are normal forms they are
identical to e’
• If e →β* e1 and e →β* e2 and e1 and e2
are normal forms then e1 is identical to e2
– “All terms have a unique normal form.”
27
28
Evaluation Strategies
Evaluation Strategies Summary
• Church-Rosser says that independent of the
reduction strategy we will find ≤1 normal form
• But some reduction strategies might find 0
• Normal Order
– (λ x. z) ((λ y. y y) (λ y. y y)) →
(λ x. z) ((λ y. y y) (λ y. y y)) → …
– Evaluates the left-most redex not contained in
another redex
– If there is a normal form, this finds it
– Not used in practice: requires partially evaluating
function pointers and looking “inside” functions
• Call-By-Name (“lazy”)
– (λ x. z) ((λ y. y y) (λ y. y y)) → z
– Don’t reduce under λ, don’t evaluate a function
argument (until you need to)
– Does not always evaluate to a normal form
• There are three traditional strategies
– normal order (never used, always works)
– call-by-name (rarely used, cf. TeX)
– call-by-value (amazingly popular)
• Call-By-Value (“eager” or “strict”)
29
– Don’t reduce under λ, do evaluate a function’s
argument right away
– Finds normal forms less often than the other two30
5
Normal-Order Reduction
• A redex is outermost if it is not contained
inside another redex
• Example:
S (K x y) (K u v)
• K x, K u and S (K x y) are all redexes
• Both K u and S (K x y) are outermost
• Normal order always reduces the leftmost
outermost redex first
Bonus: Evaluation
Strategies
• Theorem: If e has a normal form e’ then
normal order reduction will reduce e to e’
Why Not Normal Order ?
Call-by-Name
• In most (all?) programming languages,
functions are considered values (fully
evaluated)
• Example:
λx. D D = ⊥ (with normal order)
• Thus, no reduction is done under lambda
• Don’t reduce under λ
• Don’t evaluate the argument to a function call
• A value is an abstraction
• No popular programming language uses
normal order
e1 →n* λx. e1’ [e2/x]e1’ →n* e
λx. e→n* λx. e
e1 e2 →n* e
• Call-by-name is demand-driven: an expression is not
evaluated unless needed
• It is normalizing: converges whenever normal order
converges
• Call-by-name does not necessarily evaluate to a normal
form. Example: D D
33
34
Call by Name Example
Call-by-Value Evaluation
• Example:
• Don’t reduce under lambda
• Do evaluate the arguments to a function call
• A value is an abstraction
(λy. (λx. x) y) ((λu. u) (λv. v)) →βn
32
e1 →v* λx. e1’ e2 →v* e2’ [e’2/x]e1’ →v* e
(λx. x) ((λu. u) (λv. v)) →βn
λx. e→v* λx. e
(λu. u) (λv. v) →βn
e1 e2 →v* e
• Most languages are primarily call-by-value
• But CBV is not normalizing: (λx. I) (D D)
• CBV diverges more often than normal order or
even CBN
λv. v
35
36
6
Call by Value Example
Evaluation Strategy Considerations
• Example:
• Call-by-value:
– easy to implement
– well-behaved (predictable) with respect to side-effects
(λy. (λx. x) y) ((λu. u) (λv. v)) →βv
• Call-by-name:
– More difficult to implement (must pass unevaluated
expressions)
– The order of evaluation is harder to predict (e.g., difficulty
with side-effects)
– Has a simpler theory than call-by-value
– Allows the natural expression of infinite data structures (e.g.
streams)
– Terminates more often than call-by-value
(λy. (λx. x) y) (λv. v) →βv
(λx. x) (λv. v) →βv
λv. v
37
38
Which is Better?
Caveats
• The debate about whether languages should
be strict (CBV) or lazy (CBN) is nearly 20
years old
• The terms lazy and strict are not used consistently in
the literature
• Call-by-value and call-by-name are well defined
• There are parameter passing mechanisms besides callby-value and call-by-name that cannot be naturally
expressed in the lambda calculus
• This debate is typically confined to the
functional programming community (where it is
sometimes intense)
– by reference
– value-result
• These additional mechanisms deal with side-effects
• Many languages have a mixture of parameter passing
mechanisms
• Outside the functional community CBN is
rarely considered (but remember TeX)
40
39
Functional Programming
How is λ-calculus related
to “real life”?
• The λ-calculus is a prototypical
functional language with:
–
–
–
–
no side effects
several evaluation strategies
lots of functions
nothing but functions (pure λ-calculus does
not have any other data type)
• How can we program with functions?
• How can we program with only
functions?
42
7
Programming With Functions
Referential Transparency
• Functional programming is a programming style
that relies on lots of functions
• A typical functional paradigm is using
• In “pure” functional programs, we can reason
equationally, by substitution
– Called “referential transparency”
let x = e1 in e2 ===
[e1/x]e2
• In an imperative language a side-effect in e1 might
invalidate the above equation
Why?
functions as arguments or results of other
functions
– Called “higher-order programming”
• Some “impure” functional languages permit
side-effects (e.g., Lisp, Scheme, ML, Python)
– references (pointers), in-place update, arrays,
exceptions
– Others (and by “others” we mean “Haskell”) use
monads to model state updates
43
• The behavior of a function in a “pure” functional
language depends only on the actual arguments
– Just like a function in math
– This makes it easier to understand and to reason
about functional programs
44
How Complex Is Lambda?
Expressiveness of λ-Calculus
• Given e1 and e2, how complex (a la CS
theory) is it to determine if:
e1 →β* e and e2 →β* e
• The λ-calculus is a minimal system but can express
– data types (integers, booleans, lists, trees, etc.)
– branching, recursion
• This is enough to encode Turing machines
– We say the lambda calculus is Turing-complete
– Corollary: e1 =β e2 is undecidable
• That means we can encode any computation we
want in it ... if we’re sufficiently clever …
45
46
Encodings
Encoding Booleans in λ-Calculus
• Still, how do we encode all these
constructs using only functions?
• Idea: encode the “behavior” of values
and not their structure
• What can we do with a boolean?
– we can make a binary choice (= “if” exp)
• A boolean is a function that, given two
choices, selects one of them:
– true
=def λx. λy. x
– false
=def λx. λy. y
– if E1 then E2 else E3 =def E1 E2 E3
47
48
8
Encoding Booleans in λ-Calculus
Let’s try to define or
• What can we do with a boolean?
• Recall:
– we can make a binary choice (= “if” exp)
– true
– false
– if E1 then E2 else E3
• A boolean is a function that, given two
choices, selects one of them:
=def λx. λy. x
=def λx. λy. y
=def E1 E2 E3
• Intuition:
– true
=def λx. λy. x
– false
=def λx. λy. y
– if E1 then E2 else E3 =def E1 E2 E3
– or a b = if a then true else b
• Either of these will work:
• Example: “if true then u else v” is
– or
– or
(λx. λy. x) u v →β (λy. u) v →β u
=def λa. λb. a true b
=def λa. λb. λx. λy. a x (b x y)
49
50
This is getting painful to check …
More Boolean Encodings
• Let’s use OCaml …
• Think about how to do and and not
• Without peeking!
51
52
Encoding and and not
Encoding Pairs in λ-Calculus
• and a b
• What can we do with a pair?
– and
– and
• not a
– not
– not
= if a then b else false
– we can access one of its elements
(= “field access”)
=def λa. λb. a b false
=def λa. λb. λx. λy. a (b x y) y
= if a then false else true
=def λa. a false true
=def λa. λx. λy. a y x
• A pair is a function that, given a boolean,
returns the first or second element
mkpair x y =def
fst p
=def
snd p
=def
• fst (mkpair x y)
→β true x y
53
λb. b x y
p true
p false
→β (mkpair x y) true
→β x
54
9
Computing with Natural Numbers
Encoding Numbers λ-Calculus
• What can we do with a natural number?
– we can iterate a number of times over some function
loop”)
(= “for
• A natural number is a function that given an operation f
and a starting value z, applies f a number of times to z:
0
=def λf. λz. z
1
=def λf. λz. f z
2
=def λf. λz. f (f z)
– Very similar to List.fold_left and friends
• These are numerals in a unary representation
• Called Church numerals
• The successor function
succ n
or
succ n
• Addition
plus n1 n2
• Multiplication
mult n1 n2
• Testing equality with 0
iszero n
• Subtraction
=def λf. λs. f (n f s)
=def λf. λs. n f (f s)
=def n1 succ n2
=def n1 (add n2) 0
=def n (λb. false) true
– Is not instructive, but makes a fun exercise …
55
56
Computation Example
Toward Recursion
• What is the result of the application add 0?
• Given a predicate p, encode the function “find”
such that “find p n” is the smallest natural
number which is at least n and satisfies p
• Ideas? How do we begin?
(λn1. λn2. n1 succ n2) 0 →β
λn2. 0 succ n2 =
λn2. (λf. λs. s) succ n2 →β
λn2. n2 =
λx. x
• By computing with functions we can
express some optimizations
– But we need to reduce under the lambda
– Thus this “never” happens in practice
57
58
Encoding Recursion
The Fixed-Point Combinator Y
• Given a predicate p, encode the function “find”
such that “find p n” is the smallest natural
number which is at least n and satisfies p
• find satisfies the equation
• Let Y = λF. (λy.F(y y)) (λx. F(x x))
find p n = if p n then n else find p (succ n)
• Define
– This is called the fixed-point combinator
– Verify that Y F is a fixed point of F
Y F →β (λy.F (y y)) (λx. F (x x)) →β F (Y F)
– Thus Y F =β F (Y F)
• Given any function in λ-calculus we can
compute its fixed-point (!)
• Thus we can define “find” as the fixed-point
of the function F from the previous slide
• Essence of recursion is the self-application “y y”
F = λf.λp.λn.(p n) n (f p (succ n))
• We need a fixed point of F
find = F find
or
find p n = F find p n
59
60
10
Expressiveness of Lambda Calculus
• Encodings are fun
– Yes! Yes they are!
• But programming in pure λ-calculus is
painful
• So we will add constants (0, 1, 2, …, true,
false, if-then-else, etc.)
• Next we will add types
61
11