Download Extended Static Checking for Java

Extended Static Checking for Java
or
Light-weight formal methods:
from objects to components
K. Rustan M. Leino
Microsoft Research, Redmond, WA
Joint work with Cormac Flanagan, Mark
Lillibridge, Greg Nelson, James B. Saxe,
Raymie Stata
Compaq SRC
6 Nov 2002
FMCO 2002, Leiden, The Netherlands
Formal methods
coverage
program
verification
extended
static
checking
decidability ceiling
light-weight
formal methods
type
checking
effort
Note: Illustration not to scale
User’s view
Annotated
Java program
ESC/Java
Warning
messages
public class Bag {
private /*@non_null*/ int[] a;
private int n;
//@ invariant 0 <= n && n <= a.length;
public Bag(/*@non_null*/ int[] initialElements) {
n = initialElements.length;
a = new int[n];
System.arraycopy(initialElements, 0, a, 0, n);
}
public void add(int x) {
if (n == a.length) {
int[] b = new int[2*(a.length+1)];
System.arraycopy(a, 0, b, 0, n);
a = b;
}
a[n] = x;
n++;
}
Bag.java:18: Array index possibly
too large
Bag.java:45: Precondition possibly
violated
Goals of ESC/Java
 Practical static checking
 Detect common run-time errors
 null dereferences
 array bounds
 type casts
 race conditions
 deadlocks
 ...
 Modular checking
ESC/Java distinguishing features





Annotation language captures design
decisions
Powered by automatic theorem prover
Not decidable
Not sound or complete
Performs modular checking
Method-modular checking


Check that each method satisfies its
specification, assuming that all called
routines satisfy theirs
Reason about implementation when it is
written, not when it is used or extended
Modular checking
Interface
check
Client
check
Method body
check
Client
Design tradeoffs




Missed errors
Spurious warnings
Annotation overhead
Performance
Tool architecture
Annotated Java program
Translator
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Valid
Resource
exhausted
Tool architecture, detail
Translator
Annotated Java program
Sugared command
Primitive command
Passive command
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Annotated Java
program
Tool architecture, detail
Annotated Java program
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
Translator
Translator
Sugared
command
Sugared command
Primitive command
Passive command
Verification condition
Automatic theorem prover
Counterexample context
Post processor
Warning messages
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command



Passive
command


Automatic
theorem prover
Post processor
Warning
messages


non_null
Method annotations

Verification
condition
Counterexample
context
Simple
requires E;
modifies w;
ensures P;
exsures (T x) Q;
Object invariants

invariant E;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Specification expressions

side-effect free Java expressions


\result, \old(E)



ensures \result == \old(x);
==>
(\forall T x; P), (\exists T x; P)


no ++, no method calls
(\forall int j; 0 <= j && j < n ==> a[j] > 0);
\typeof(E), \type(T), <:

requires \typeof(x) == \typeof(this);
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Concurrency

monitored_by lock


\lockset[lock]



/*@ monitored_by this */ long x;
requires \lockset[this];
lock0 < lock1
\max(\lockset)

requires \max(\lockset) < this;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Concurrency

monitored_by lock


\lockset[lock]



/*@ monitored_by this */ long x;
requires \lockset[this];
lock0 < lock1
\max(\lockset)

requires \max(\lockset) < this;
Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Ghost variables

ghost public int objectState;
 ghost public \TYPE elementType;

Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
ghost public T x;

set x = E;
set objectState = Open;
 set elementType = \type(T);

Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Ghost variables

ghost public int objectState;
 ghost public \TYPE elementType;

Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
ghost public T x;

set x = E;
set objectState = Open;
 set elementType = \type(T);

Annotation language
Annotated Java
program
Translator
Sugared
command
Primitive
command

Miscellaneous


Passive
command
Verification
condition
assert E;
assume E;


nowarn

Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

assume x >= 0; // because x == y*y
x = a[j]; //@ nowarn
axiom E;

axiom (\forall int x; x >> 2 >= 0);
Weakest preconditions
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command

wp(assert E, Q) = E  Q

wp(assume E, Q) = E  Q
wp(S;T, Q) = wp(S, wp(T,Q))

wp(S [] T, Q) = wp(S, Q)  wp(T, Q)

wp(S, Q) = wp(S, true)  wlp(S, Q)

wlp(S, Q) = wlp(S, false)  Q

Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
Verification condition
Annotated Java
program
Translator
Sugared
command

Universal background predicate

Primitive
command

Passive
command
(FORALL (t) (<: t t))
Type-specific background predicate

(<: T_T |T_java.lang.Object|)
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Verification condition:
BPUniv  BPT  VCmethod
Verification condition generation
 Easy for small languages [Dijkstra]
 Much harder for real languages
 Object-oriented
 Typed
 Dynamic allocation
 Exceptions
 Aliasing
 Threads
Verification conditions for real programs
Java
Guarded command
wlp
Verification condition
x = a[ i++ ];
assume preconditions
assume invariants
...
i0 = i;
i = i + 1;
assert (LABEL Null@218: a != null);
assert (LABEL IndexNeg@218: 0 <= i0);
assert (LABEL IndexTooBig@218: i0 < a.length);
x = elems[a][i0];
...
assert postconditions
assert invariants
i0.(i0 == i ==> … )
Verification condition
 Formula in untyped, first-order
predicate calculus
 equality and function symbols




quantifiers
arithmetic operations
select and store operations
Eg. x.y.(x > y ==> … )
Background axioms
 Additional properties of Java that the
theorem prover needs to know
 A variable of type T always holds a
value whose type is a subtype of T
 The subtyping relation is reflexive,
anti-symmetric, and transitive
 new returns an object that is distinct
from all existing objects
 ... lots more ...
 java.lang.Object has no supertype
Example verification condition
 Verification condition large but “dumb”
(IMPLIES (DISTINCT |ecReturn| |L_14.4|) (IMPLIES (AND (EQ |a@pre:2.8| |a:2.8|) (EQ |a:2.8| (asField |a:2.8| (array |T_int|))) (<
(fClosedTime |a:2.8|) alloc) (EQ |n@pre:3.6| |n:3.6|) (EQ |n:3.6| (asField |n:3.6| |T_int|)) (EQ |MAX_VALUE@pre:3.4.26|
|MAX_VALUE:3.4.26|) (EQ |@true| (is |MAX_VALUE:3.4.26| |T_int|)) (EQ |elems@pre| elems) (EQ elems (asElems elems)) (<
(eClosedTime elems) alloc) (EQ LS (asLockSet LS)) (EQ |alloc@pre| alloc) (EQ |@true| (is |this<1>| |T_Bag|)) (EQ |@true|
(isAllocated |this<1>| alloc)) (NEQ |this<1>| null)) (FORALL (tmp1 |tmp2:21.4| |tmp3:21.6| |m:12.8| |mindex:13.8| |i:14.13|
|tmp0:14.28|) (AND (IMPLIES (<= 1 (select |n:3.6| |this<1>|)) (AND (LBLNEG |[email protected]~15.10| (NEQ (select |a:2.8| |this<1>|)
null)) (LBLNEG |[email protected]~15.11| (<= 0 1)) (LBLNEG |[email protected]~15.11| (< 1 (arrayLength (select |a:2.8|
|this<1>|)))) (IMPLIES (< (select (select elems (select |a:2.8| |this<1>|)) 1) |MAX_VALUE:3.4.26|) (AND (LBLNEG
|[email protected]~17.12| (NEQ (select |a:2.8| |this<1>|) null)) (LBLNEG |[email protected]~17.13| (<= 0 1)) (LBLNEG
|[email protected]~17.13| (< 1 (arrayLength (select |a:2.8| |this<1>|)))) (FORALL (|m:17.8|) (IMPLIES (EQ |m:17.8| (select
(select elems (select |a:2.8| |this<1>|)) 1)) (FORALL (|i:14.28|) (IMPLIES (AND (EQ |i:14.28| (+ 1 1)) (EQ |@true| |bool$fals e|))
(FORALL (|tmp2:21.4<1>|) (IMPLIES (EQ |tmp2:21.4<1>| (select |a:2.8| |this<1>|)) (AND (LBLNEG |[email protected]~21.16| (NEQ (select
|a:2.8| |this<1>|) null)) (LBLNEG |[email protected]~21.17| (<= 0 (select (store |n:3.6| |this<1>| ( - (select |n:3.6|
|this<1>|) 1)) |this<1>|))) (LBLNEG |[email protected]~21.17| (< (select (store |n:3.6| |this<1>| ( - (select |n:3.6| |this<1>|)
1)) |this<1>|) (arrayLength (select |a:2.8| |this<1>|)))) (LBLNEG |[email protected]~21.4| (NEQ |tmp2:21.4<1>| null)) (LBLNEG
|[email protected]~21.5| (<= 0 1)) (LBLNEG |[email protected]~21.5| (< 1 (arrayLength |tmp2:21.4<1>|))) (LBLNEG
|Exception:[email protected]~11.2| (EQ |ecReturn| |ecReturn|))))))))))) (IMPLIES (NOT (< (select (select elems (select |a:2.8|
|this<1>|)) 1) |MAX_VALUE:3.4.26|)) (FORALL (|i:14.28|) (IMPLIES (AND (EQ |i:14.28| (+ 1 1)) (EQ |@true| |bool$false|)) (FORA LL
(|tmp2:21.4<1>|) (IMPLIES (EQ |tmp2:21.4<1>| (select |a:2.8| |this<1>|)) (AND (LBLNEG |[email protected]~21.16| (NEQ (select |a:2.8|
|this<1>|) null)) (LBLNEG |[email protected]~21.17| (<= 0 (select (store |n:3.6| |this<1>| ( - (select |n:3.6| |this<1>|) 1))
|this<1>|))) (LBLNEG |[email protected]~21.17| (< (select (store |n:3.6| |this<1>| ( - (select |n:3.6| |this<1>|) 1)) |this<1>|)
(arrayLength (select |a:2.8| |this<1>|)))) (LBLNEG |[email protected]~21.4| (NEQ |tmp2:21.4<1>| null)) (LBLNEG |[email protected]~21 .5|
(<= 0 0)) (LBLNEG |[email protected]~21.5| (< 0 (arrayLength |tmp2:21.4<1>|))) (LBLNEG |Exception:[email protected]~11.2| (EQ
|ecReturn| |ecReturn|)))))))))) (IMPLIES (NOT (<= 1 (select |n:3.6| |this<1>|))) (AND (IMPLIES (EQ |L_14.4| |L_14.4|) (FORALL
(|tmp2:21.4<1>|) (IMPLIES (EQ |tmp2:21.4<1>| (select |a:2.8| |this<1>|)) (AND (LBLNEG |[email protected]~21.16| (NEQ (select |a:2.8|
|this<1>|) null)) (LBLNEG |[email protected]~21.17| (<= 0 (select (store |n:3.6| |this<1>| ( - (select |n:3.6| |this<1>|) 1))
|this<1>|))) (LBLNEG |[email protected]~21.17| (< (select (store |n:3.6| |this<1>| ( - (select |n:3.6| |this<1>|) 1)) |this<1>|)
(arrayLength (select |a:2.8| |this<1>|)))) (LBLNEG |[email protected]~21.4| (NEQ |tmp2:21.4<1>| null)) (LBLNEG |[email protected]~21 .5|
(<= 0 0)) (LBLNEG |[email protected]~21.5| (< 0 (arrayLength |tmp2:21.4<1>|))) (LBLNEG |Exception:[email protected]~11.2| (EQ
|ecReturn| |ecReturn|)))))) (IMPLIES (NOT (EQ |L_14.4| |L_14.4|)) (AND (LBLNEG |Exception:[email protected]~11.2| (EQ |L_14.4|
|ecReturn|))))))))))
Annotated Java
program
Translator
Sugared
command
Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages
(AND
(<: T_T |T_java.lang.Object|)
(EQ T_T (asChild T_T |T_java.lang.Object|))
(DISTINCT arrayType |T_boolean| |T_char| |T_byte| |T_short| |T_int|
|T_long| |T_float| |T_double| |T_.TYPE|
T_T |T_java.lang.Object|)))
(EXPLIES

(LBLNEG |vc.T.abs.2.2|
(IMPLIES
(AND
(EQ |elems@pre| elems)
(EQ elems (asElems elems))
(< (eClosedTime elems) alloc)
(EQ LS (asLockSet LS))
(EQ |alloc@pre| alloc))
(NOT
(AND
(EQ |@true| (is |x:2.21| T_int))
(OR
(AND
(OR
(AND
(< |x:2.21| 0)
(LBLPOS |trace.Then^0,3.15| (EQ |@true| |@true|))
(EQ |x:3.17| (- 0 |x:2.21|))
Verification condition
class T {
static int abs(int x) {
if (x < 0) { x = -x; }
//@ assert x >= 0;
}
}
Annotated Java
program
Translator
Sugared
command
Theorem prover: “Simplify”

Nelson-Oppen cooperating decision
procedures
Primitive
command

Passive
command

Verification
condition
Automatic
theorem prover
Counterexample
context



Key features:




Post processor
Warning
messages
congruence closure
linear arithmetic
partial orders
quantifiers

automatic: no user interaction
refutation based: searches for counterexamples
heuristics tuned for program checking
labels
time limit
Automatic theorem proving
Verification condition
x.y.(x > y ==> … )
Automatic
theorem prover
(Simplify)
Valid
Counterexample
Diverges
Counterexamples and warnings
Annotated Java
program
Translator
Sugared
command

Primitive
command
Passive
command
Verification
condition
Automatic
theorem prover
Counterexample
context
Post processor
Warning
messages

Counterexample:
labels: (|[email protected]| |vc.Bag.add.20.2| |trace.Then^0,21.23|)
context:
(AND
(NEQ |tmp1!a:23.23| null)
(NEQ this null)
(EQ |alloc@pre| alloc)
(EQ |tmp4!n:26.6| 0)
…
(<= alloc (vAllocTime |tmp3!a:26.4|))
)
Bag: add(int) ...
-----------------------------------------------------------------------Bag.java:26: Warning: Array index possibly too large (IndexTooBig)
a[n] = x;
^
Execution trace information:
Executed then branch in "Bag.java", line 21, col 23.
------------------------------------------------------------------------
ESC/Java research platform

Annotation inference


Other checking tools



Calvin [Qadeer et al.], Stale-value concurrency
checker [Burrows & Leino]
Formal verification without driving theorem
prover
Simplify theorem prover


Daikon [Ernst], Houdini [Flanagan & Leino]
SLAM [Ball, Rajamani, et al.], oolong [Leino et al.], …
Teaching

Kansas State University [Dwyer & Hatcliff]