Download Document

The Closures Controversy
Joshua Bloch
Chief Java Architect
Google Inc.
Disclaimer: Talk Represents My Opinion, Not Google’s!
Google believes that the Java platform would
likely benefit from some additional support for
closures. Like the broader Java community, we
are divided on what form this support should
take. Some favor a lightweight approach that
addresses the pain of anonymous inner classes
without affecting the type system, VM, or
libraries; others favor a heavyweight approach
designed to provide for programmer-defined
control structures. We believe it is premature to
launch a JSR that forces us down either path.
www.javapolis.com
Outline
I.
II.
III.
IV.
V.
Setting the Stage
Why Enhance Support for Closures?
BGGA Closures
A Lightweight Approach
Where Do We Go From here?
www.javapolis.com
OOPSLA Invited Talk
October 8, 1996
Digitally reconstructed
from the Internet Archive
(AKA the Wayback Machine)
by Joshua Bloch
November 24, 2007
Oak
•
•
•
•
Started as a reimplementation of C++
Always a tool, never an end itself
Took on a life of its own
The Web happened...
• serendipitous match!
• and it became Java
The Java Language
Fusion of four kinds of programming
• Object Oriented like Simula/C++/
ObjectiveC…
• Numeric
like FORTRAN
• Systems
like C
• Distributed
like nothing else
Java - a language for a job
• Doing language research:
an anti-goal
• Started using C++
• Broke down, needed:
• Architecture neutral, portable, reliable, safe,
long lived, multithreaded, dynamic, simple, ...
Practical, not theoretical
• Driven by what people needed
• (but hey, I spent too much time going to school!)
• Theory provides
• rigour
• cleanliness
• cohesiveness
No new ideas here
• Shamelessly ripped off ideas that worked
in C, C++, Objective C, Cedar/Mesa,
Modula, Simula, ...
• (well, we slipped once or twice and
invented something)
Don’t fix it until it chafes
• To keep it simple...
• A procedural principle:
• Require several real instances before
including a feature
• i.e. nothing goes in because it’s “nice”
• (== “Just Say No, until threatened with
bodily harm”)
•
•
•
•
•
•
•
Hyped :-(
Playful
Flexible
Deterministic
Non-threatening
Rich
Like I can just write code…
Hey! I left out “Object-Oriented”!
Java feels...
IEEE Computer, June 1997
So How Are We Doing?
Not So Well, Unfortunately
Enum<E extends Enum<E>> { ... }
<T extends Object & Comparable<? super T>> T
Collections.max(Collection<? extends T>)
public <V extends Wrapper<? extends Comparable<T>>>
Comparator<V> comparator() { ... }
error: equalTo(Box<capture of ?>) in Box<capture of ?>
cannot be applied to (Box<capture of ?>)
equal = unknownBox.equalTo(unknownBox)
Arrays.asList(String.class, Integer.class) // Warning!
See Angelia Langer's 427-page (!) Java Generics FAQ for more:
http://www.angelikalanger.com/GenericsFAQ/JavaGenericsFAQ.pdf
www.javapolis.com
What the Man on the Web is Saying
“I am completely and totally humbled. Laid low. I realize
now that I am simply not smart at all. I made the mistake of
thinking that I could understand generics. I simply cannot. I
just can't. This is really depressing. It is the first time that
I've ever not been able to understand something related to
computers, in any domain, anywhere, period.”
“I'm the lead architect here, have a PhD in physics, and
have been working daily in Java for 10 years and know it
pretty well. The other guy is a very senior enterprise
developer (wrote an email system that sends 600 million
emails/year with almost no maintenance). If we can't get
[generics], it's highly unlikely that the ‘average’ developer
will ever in our lifetimes be able to figure this stuff out.”
www.javapolis.com
Where Does The Complexity Come From?
Complexity
100
Feature Tuples
80
60
(exponential)
Feature Pairs
(quadratic)
40
20
0
Features
(linear)
1
2
3
4
5
6
Features
www.javapolis.com
7
8
9
10
If The Feel of Java is to be Preserved...




We simply cannot afford another wildcards
Further language additions must be
undertaken with extreme caution
Minimal addition to conceptual surface area
High power-to-weight ratio
www.javapolis.com
Outline
I.
II.
II.
III.
IV.
Setting the Stage
Why Enhance Support for Closures?
BGGA Closures
A Lightweight Approach
Where Do We Go From here?
www.javapolis.com
What is a Closure?

One definition: a function that is evaluated in
an environment containing one or more
bound variables [Wikipedia]

In English: a little snippet of code that can be
passed around for subsequent execution

Limited support for closures since JDK 1.1,
in the form of anonymous classes
www.javapolis.com
Why Are We Considering Better Support for Closures?

Fine-Grained Concurrecy - Passing
snippets of code to fork-join frameworks
using anonymous classes is a pain

Resource Managememnt - Using tryfinally blocks for resource management
is a pain, and causes resource leaks
www.javapolis.com
1. Fine-grained (fork-join) concurrency
"It has to be easier to send snippets of code
to frameworks for parallel execution;
otherwise no one will use them .“
–Doug Lea, 2005
www.javapolis.com
Here’s How it Looks Today
class StudentStatistics {
ParallelArray<Student> students = ...
// ...
public double getMaxSeniorGpa() {
return students.withFilter(isSenior).
withMapping(gpaField).max();
}
// helpers:
static final class IsSenior implements Predicate<Student> {
public boolean evaluate(Student s) { return s.credits > 90; }
}
static final IsSenior isSenior = new IsSenior();
static final class GpaField implements MapperToDouble<Student> {
public double map(Student s) { return s.gpa; }
}
static final GpaField gpaField = new GpaField();
}
www.javapolis.com
2. Automatic Resource Management
“The C++ destructor model is exactly the same
as the Dispose pattern, except that it is far
easier to use and a direct language feature
and correct by default, instead of a coding
pattern that is off by default and causing
correctness or performance problems when it
is forgotten.”
–Herb Sutter, OOPSLA 2004
www.javapolis.com
How it Looks Today–Manual Resource Management
static String readFirstLineFromFile(String path)
throws IOException {
BufferedReader r = null;
String s;
try {
r = new BufferedReader(new FileReader(path));
s = r.readLine();
} finally {
if (r != null)
r.close();
}
return s;
}
www.javapolis.com
It’s Worse With Multiple Resources (Puzzler 41)
static void copy(String src, String dest) throws IOException {
InputStream in = null;
OutputStream out = null;
try {
in = new FileInputStream(src);
out = new FileOutputStream(dest);
byte[] buf = new byte[1024];
int n;
while ((n = in.read(buf)) >= 0)
out.write(buf, 0, n);
} finally {
closeIgnoringException(in);
closeIgnoringException(out);
}
}
private static void closeIgnoringException(Closeable c) {
if (c != null) {
try { c.close(); } catch (IOException ex) { // ignore }
}
}
www.javapolis.com
Automatic Resource Management
C++ Destructor
String^ ReadFirstLineFromFile( String^ path ) {
StreamReader r(path);
return r.ReadLine();
}
C# using Block
String ReadFirstLineFromFile( String path ) {
using ( StreamReader r = new StreamReader(path) ) {
return r.ReadLine();
}
}
www.javapolis.com
Outline
I.
II.
III.
IV.
V.
Setting the Stage
Why Enhance Support for Closures?
BGGA Closures
A Lightweight Approach
Where Do We Go From here?
www.javapolis.com
Controversial Features in BGGA





Function types
Non-local return
Non-local break and continue
Unrestricted access to nonfinal local variables
Design goal: library-defined control constructs
www.javapolis.com
Function Types
The BGGA Spec says: “While the subtype rules for
function types may at first glance appear arcane,
they are defined this way for very good reason: [...].
And while the rules seem complex, function
types do not add complexity to Java's type
system because function types and their subtype
relations can be understood as a straightforward
application of generics and wildcards (existing
constructs). From the programmer's perspective,
function types just ‘do the right thing.’”
www.javapolis.com
Function Types are Hard to Read
static Pair<{ => int },{ => int }> joinedCounters(int initial ) {
return Pair.<{ => int },{ => int }>of(
{ => initial++ }, { => initial++ });
}
interface BThunk extends {=>boolean} { }
static final {BThunk, { => void} => void} wihle =
{BThunk cond, { => void } action =>
while (cond.invoke()) action.invoke(); };
static <throws X> { {=> void throws X} => void throws X }foo() {
return { { => void throws X } block => block.invoke(); };
}
These examples come from test code that ships with BGGA Prototype
www.javapolis.com
Function Types Encourage an “Exotic” Programming Style
static <A1, A2, R> {A1 => {A2 => R}} curry({A1, A2 => R} fn) {
return {A1 a1 => {A2 a2 => fn.invoke(a1, a2)}};
}
<A1, A2, A3, R> {A2, A3 => R} partial({A1, A2, A3 => R} fn, A1 a1);
static <A1, A2, A3, R> {A2, A3 => R}
partial({A1, A2, A3 => R} fn, A1 a1) {
return {A2 a2, A3 a3 => fn.invoke(a1, a2, a3)};
}
From Mark Mahieu's blog:
Currying and Partial Application with Java Closures
http://markmahieu.blogspot.com/2007/12/currying-and-partialapplication-with.html
www.javapolis.com
Nominal Types are Rich Compared to Function Types
Name →
Known Implementations →
Documentation, including
semantic constraints
{T, T => T}
www.javapolis.com
Function Types Have Unexpected Interactions
Arrays don’t work

Foo.java:6: generic array creation
{ => int}[] closures = new { => int}[N];
Autoboxing doesn’t work

LoopBenchC.java:10: <E,X>forEach(java.util.Collection<E>,
{E => void throws X}) in LoopBenchC cannot be applied to
(java.util.List<java.lang.Integer>,{int => void})
forEach(list, {int i =>
Wildcards produce difficult error messages

NewtonWithClosures.java:26: invoke(capture#418 of ? super
{double => double}) in {capture#418 of ? super {double =>
double} => capture#928 of ? extends {double => double}}
cannot be applied to (double)
return fixedPoint(transform.invoke(guess));
^
www.javapolis.com
Function Types Limit Interoperability With SAM Types

Closure conversion only works with interfaces

Unfortunately, existing APIs sometimes use
abstract classes for functions


e.g., TimerTask, SwingWorker
These APIs would become 2nd class citizens
www.javapolis.com
Summary - Pros and Cons of Function Types
+ Avoid need to define named interfaces
+ Avoid incompatibility among SAM types with
same signatures
- Hard to read under moderate-to-heavy use
- Encourage “exotic” style of programming
- Don't reflect semantic constraints
- Don't provide the same level of documentation
- Don’t interact well with autocompletion (or grep)
- Limited interoperability may balkanize libraries
www.javapolis.com
BGGA Closures Have Two Kinds of Returns
static boolean test(boolean arg) {
{boolean => boolean} closure = { boolean arg =>
if (arg)
return true; // Non-local return
false
// local return
};
return !closure.invoke(arg);
}
return means something completely different
in a BGGA closure and an anonymous class
www.javapolis.com
What Does test() Return?
static <E> Boolean contains(Iterable<E> seq, Predicate<E> pred) {
for (E e : seq)
if (pred.invoke(e))
return true;
return false;
}
static Boolean test() {
List<Character> list = Arrays.asList(
'h', 'e', 'l', 'l', 'o');
return contains(list, new Predicate<Character>() {
public Boolean invoke(Character c) {
return c > 'j';
}
});
}
interface Predicate<T> { Boolean invoke(T t); }
www.javapolis.com
Now What Does test() Return? (BGGA)
static <E> Boolean contains(Iterable<E> seq, {E => Boolean} p) {
for (E e : seq)
if (p.invoke(e))
return true;
return false;
}
static Boolean test() {
List<Character> list = Arrays.asList(
'h', 'e', 'l', 'l', 'o');
return contains(list, { Character c => return c > 'j'; } );
}
www.javapolis.com
Now What Does test() Return? (BGGA)
static <E> Boolean contains(Iterable<E> seq, {E => Boolean} p) {
for (E e : seq)
if (p.invoke(e))
return true;
return false;
}
static Boolean test() {
List<Character> list = Arrays.asList(
'h', 'e', 'l', 'l', 'o');
return contains(list, { Character c => return c > 'j'; });
}
Accidental non-local return due to cut-and-paste
from anonymous class can cause insidious bug
www.javapolis.com
Suppose You Wanted to Translate this Method to BGGA
static <E> Predicate<Iterable<E>> contains(
final Predicate<E> pred) {
return new Predicate<Iterable<E>>() {
public Boolean invoke(Iterable<E> seq) {
for (E e : seq)
if (pred.invoke(e))
return true;
return false;
}
};
}
www.javapolis.com
It’s Awkward, as Only One Local Return is Permitted
static <E> { Iterable<E> => Boolean } contains(
{ E => Boolean } pred) {
return { Iterable<E> seq =>
Boolean result = false;
for (E e : seq) {
if (pred.invoke(e)) {
result = true;
break;
}
}
result
};
}
www.javapolis.com
Summary - Pros and Cons of Non-Local Returns
+ Permits library-defined control structures
- Having two kinds of returns is confusing
- Meaning of return has changed
- Unintentional non-local returns can cause bugs
- Only one local return permitted per closure
www.javapolis.com
What Does This Program Print?
public class Test {
private static final int N = 10;
public static void main(String[] args) {
List<{ => int}> closures = new ArrayList<{ => int}>();
for (int i = 0; i < N; i++)
closures.add( { => i } );
int total = 0;
for ({ => int} closure : closures)
total += closure.invoke();
System.out.println(total);
}
}
www.javapolis.com
What does this program print?
public class Test {
private static final int N = 10;
public static void main(String[] args) {
List<{ => int}> closures = new ArrayList<{ => int}>();
for (int i = 0; i < N; i++)
closures.add( { => i } );
int total = 0;
for ({ => int} closure : closures)
total += closure.invoke();
System.out.println(total);
}
}
It prints 100, not 45. The same loop variable is captured
by all 10 closures and evaluated after the loop is finished!
www.javapolis.com
Summary - Pros and Cons of Access to Nonfinal Locals
+ Permits library-defined control structures
+ Eliminates some uses of final modifier
- Semantics can be very confusing
- Locals can persist after their scope is finished
- Locals can be modified by other threads
- Performance model for locals will change
www.javapolis.com
Library-Defined Control Constructs

What are the compelling use-cases?



Custom loops
Automatic resource management blocks
Timer block
www.javapolis.com
Custom for-loops (Example from BGGA Spec)
<K,V,throws X>
void for eachEntry(Map<K,V> map, {K,V=>void throws X} block)
throws X {
for (Map.Entry<K,V> entry : map.entrySet()) {
block.invoke(entry.getKey(), entry.getValue());
}
}
for eachEntry(String name, Integer value : map) {
if ("end".equals(name)) break;
if (name.startsWith("com.sun.")) continue;
System.out.println(name + ":" + value);
}
www.javapolis.com
What's Wrong With This example?

BGGA loop competes with Java 5 for-each



Don't define a construct; just implement Iterable
Only compelling use is multiple loop variables
Last example doesn't offer power of for-each


Loop variable can’t be primitive (no auto-unboxing)
Would require 81 (!) overloadings to do fix this
www.javapolis.com
Loop syntax tailored to for; awkward for while
public static <throws X> void for myWhile(
{=> boolean throws X} cond, {=>void throws X} block)
throws X {
while (cond.invoke()) {
block.invoke();
}
}
for myWhile( { => i < 7 } ) {
System.out.println(i++);
}
myWhile( { => i < 7 }, { =>
System.out.println(i++);
});
www.javapolis.com
Automatic Resource Management Block (BGGA Spec)
<R, T extends Closeable, throws X>
R with(T t, {T=>R throws E} block) throws X {
try {
return block.invoke(t);
} finally {
try { t.close(); } catch (IOException ex) {}
}
}
with (FileReader in : makeReader()) { // Requires nesting
with (FileWriter out : makeWriter()) {
// code using in and out
}
}
www.javapolis.com
Library-Defined Control Constructs are a Double-Edged Sword


A great feature of Java is “programmer portability”

All Java code looks pretty much alike

We can read and maintain each other's code with a
minimum of effort
Library-Defined Control Constructs foster
dialects, which hinder programmer portability
www.javapolis.com
Performance is an Open Question
Time to iterate over 108 elements
(Take these numbers with a huge grain of salt)
for-each
BGGA
% change
List
1.65 s
1.97 s
+19%
Array
0.18 s
1.11 s
+519%
Java HotSpot(TM) Server VM (build 1.6.0-b105, mixed mode)
Windows XP, Intel T2600 @ 2.16 GHz, 2GB RAM
BGGA prototype closures-2007-11-30 (I apologize)
www.javapolis.com
Summary - Programmer-Defined Control Constructs
+ They let you define new control constructs
- They don’t have the same syntax, semantics, or
performance as built-in control constructs
- It’s not clear that you need the ability; Java
already has a rich set of control structures
- Can lead to dialects
- Responsible for much of the complexity in BGGA
- Non-local return, break and continue
- Unrestricted access to mutable local variables
www.javapolis.com
Outline
I.
II.
III.
IV.
V.
Setting the Stage
Why Enhance Support for Closures?
BGGA Closures
A Lightweight Approach
Where Do We Go From here?
www.javapolis.com
A Lightweight Approach


There is a much simpler approach to
reducing the verbosity of anonymous
classes and manual resource management
Attack the two problems head-on!


Concise syntax for anonymous class instance creation
Purpose-built construct for automatic resource
management
www.javapolis.com
Concise Instance Creation Expressions (CICE)
This:
sort(list, Comparator<String>(String s1, String s2) {
return s1.length() - s2.length();
});
Expands to this:
sort(list, new Comparator<String>() {
public int compare(String s1, String s2) {
return s1.length() - s2.length();
}
});
I am not in love with this syntax; we can probably do better.
www.javapolis.com
Automatic Resource Management (ARM) Blocks
// One resource - readFirstLineFromFile
try (BufferedReader r = new BufferedReader(new FileReader(path)) {
String s = r.readLine();
}
// Multiple resources - copyFile
try (InputStream in
= new FileInputStream(src);
OutputStream out = new FileOutputStream(dest)) {
byte[] buf = new byte[1024];
int n;
while ((n = in.read(buf)) >= 0)
out.write(buf, 0, n);
}
www.javapolis.com
More Automatic Resource Management Blocks
Perhaps this:
protected (lock) {
... // access the resource protected by this lock
}
Should be shorthand for this:
lock.lock();
try {
// access the resource protected by this lock
} finally {
lock.unlock();
}
www.javapolis.com
For More Information

Automatic Resource Management Blocks (ARM)


Concise Instance Creation Expressions (CICE)


http://docs.google.com/View?docid=dffxznxr_1nmsqkz
http://docs.google.com/View?docid=k73_1ggr36h
Both of these docs are a bit sketchy 
www.javapolis.com
Outline
I.
II.
III.
IV.
V.
Setting the Stage
Why Enhance Support for Closures?
BGGA Closures
Lightweight Closures
Where Do We Go From here?
www.javapolis.com
The Closures Spectrum
CICE+ARM
Bob Lee
www.javapolis.com
The Big Questions



Do we really want function types in Java?
Do we really want library-defined control
constructs in Java?
We should keep in mind that there’s already
a fine programming language for the JVM
that offers both of these things, and Java
interoperability to boot: Scala
www.javapolis.com
The Medium-Sized Questions

Should we support non-local return, break,
and continue? If so:



Should it be the default?
How do we prevent accidents?
Should we allow access to non-final local
variables from closures? If so:


Should it be the default?
How do we prevent accidents?
www.javapolis.com
Moving Forward, There are Two Approaches

Gain more experience with prototypes




When we come to an informed consensus, start a
narrowly focused JSR
This might take a couple of years
Precedent established by generics
Start a broadly focused JSR in the near future


Must permit any point on the Closure Spectrum
First task is to answer the two Big Questions
www.javapolis.com
Closing Sermon




We have a big decision to make!
It will have a huge effect on the future of
the Java platform
We must take our time and do what is right
We must not risk further damage to
“the feel of Java”
www.javapolis.com
Q&A
View JavaPolis talks @ www.parleys.com
Thank you for your
attention