Download An Implementation of Scoped Memory for Real

Memory Management for Real-Time Java
Wes Beebee and Martin Rinard
Laboratory for Computer Science
Massachusetts Institute of Technology
Supported by: DARPA Program Composition
for Embedded Systems (PCES) Program
Goal: Enable Use of Java for RealTime and Embedded Systems
Vision
Standard Java
Applications
Real-Time
Computation
In Java
Downloaded
Java Applets
Unified Language/Environment Facilitates Interaction
Why Java?
• Type safe language, no memory corruption
• Reasonably modern approach
• Object oriented
• Garbage collected memory management
• Popular and supported…
• Programmers available
• Tools available
• Libraries available
Implications and Issues
• Heterogeneous components with different
needs and goals
• Real-time computation
• User interface
• Data management
• Issues
• Memory management
• Scheduling
• Event management and delivery
• Processor allocation
Why NOT Java
• Unpredictable memory usage
• Dynamic memory allocation
• Allocation hidden in extensive set of
libraries and native methods
• Allocation hidden in exception model
• Unpredictable execution times
• Garbage collection
• No scheduling guarantees
• Thread scheduling
• Event delivery
• Complex libraries and native methods
Why NOT Java
• Impoverished set of abstractions
• Threads, mutex locks, signal and wait
• No good way express relationship between
• Events in system
• Corresponding pieces of computation
• No good way to express timing expectations
• Real-Time Java Approach
• Extend library
• Native methods for new mechanisms
Real-Time Java Standard
Goal: Augment Java to better support real-time
systems
• Augment memory model to enable threads to
avoid garbage collection pauses
• Augment thread scheduling model to add
more control over task scheduling
• Augment synchronization model to include
lightweight event delivery mechanism
Our View
• Real-time Java is a work in progress
• Many of extensions generate
• More complex programming model
• More possibilities for errors
• Our goal
• Isolate general principles/concepts we
think will last
• Develop new program analyses and
implementation mechanisms
• That help programmers use real-time
extensions safely and effectively
Java Memory Models
• Java: single garbage-collected heap
• Real-time Java: multiple kinds of memories
• Garbage-collected heap memory
• Immortal memory (live for full computation)
• Scoped memories
(live for specific subcomputations)
• Linear-time allocation (LTMemory)
• Variable-time allocation (VTMemory)
Problems/Issues with Memory Model
• Scoped memory issues
• Scoped memory reference checks
• Scoped memory sizes
• Avoiding garbage collection interaction issues
• No-heap real-time thread access checks
• Priority inversions caused by indirect
interactions with garbage collector
Scoped Memory Overview
Standard
Java
Computation
Scoped Memory Overview
Objects in
GC Heap
Standard
Java
Computation
Scoped Memory Overview
Objects in
GC Heap
Standard
Java
Computation
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects in
GC Heap
Standard
Java
Computation
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
New Thread Runs
In Scoped Memory
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Thread’s New Objects
Allocated in
Scoped Memory
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Thread’s New Objects
Allocated in
Scoped Memory
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Thread’s New Objects
Allocated in
Scoped Memory
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Computation
Terminates
Scoped Memory Overview
New Computation
Typically new thread
Maybe even real-time thread
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Objects in Scoped Memory
Deallocated as a Unit
without GC
Scoped Memory Motivation
• Dynamic memory allocation without GC
• Tie object lifetimes to computation lifetimes
• Eliminate need to dynamically trace out
reachable objects
• Warning:
• Example illustrates primary intended use
• Specification allows more behaviors
• Scoped memories shared by multiple threads
• Nested scoped memories
• Scoped memories entered multiple times
Safety Issue for Scoped Memories:
Dangling References
• Lifetimes of objects in scoped memory
determined by lifetime of computation
• Must ensure that no reference goes from
long-lived object to short-lived object
Nested Scoped Memories
Object
Scoped
Memory
Referencing Constraints
Referencing
Down Scopes
Is NOT OK
Object
Scoped
Memory
Referencing
Up Scopes
Is OK
Preventing Downward References
• Dynamic Reference Checks
• At every write of a reference into an object
field or array element
• Check that written object is allocated in a
scope with a lifetime at least as long as that
of referred object
• If not, throw an exception
• Drawbacks
• Dynamic checking overhead
• New class of dynamic errors
Static Analysis
• Goal
• Eliminate need for dynamic checks by
• Statically checking that program does not
violate referencing constraints
• Basic approach: escape analysis
What Escape Analysis Provides
Control Flow Graph
• Nodes = methods
• Edges = invocation
relationships
void compute(d,e)
————
————
————
void multiplyAdd(a,b,c)
—————————
—————————
—————————
void multiply(m)
————
————
————
void add(u,v)
——————
——————
What Escape Analysis Provides
Control Flow Graph
• Nodes = methods
• Edges = invocation
relationships
void compute(d,e)
————
————
————
void multiplyAdd(a,b,c)
—————————
—————————
—————————
void multiply(m)
————
————
————
void add(u,v)
——————
——————
Allocation Site
What Escape Analysis Provides
Control Flow Graph
• Nodes = methods
• Edges = invocation
relationships
Object Allocated Here
Does Not Escape
Computation of
multiplyAdd method
void compute(d,e)
————
————
————
void multiplyAdd(a,b,c)
—————————
—————————
—————————
void multiply(m)
————
————
————
void add(u,v)
——————
——————
Allocation Site
Our Escape Analysis
• Interprocedural
• Analyzes interactions between methods
• Recaptures objects in callers of allocating methods
• Compositional
• Analyzes each method once
• Single analysis result that can be specialized for
use in different calling contexts
• Suitable for multithreaded programs
• Analyzes interactions between threads
• Recaptures objects that do not escape a given
multithreaded computation
Using Escape Analysis to Verify
Correct Use of Scoped Memories
For each computation that runs in scoped
memory
Check that allocated objects do not escape
Implementation
• FLEX compiler infrastructure
(www.flexc.lcs.mit.edu)
• Full Java compiler
• Lots of utilities and packages
• Support for deep program analyses and
transformations
• Implemented scoped memories and checks
• Implemented escape analysis
• Used results to eliminate checks
• In applications, eliminated all checks
Experimental Results
120
Time (sec)
100
80
Scope Checks
60
Application
40
20
0
Array
Array
Tree
Tree
Water Water Barnes Barnes
(Heap) (Scope) (Heap) (Scope) (Heap) (Scope) (Heap) (Scope)
Benchmarks
Scoped Memory Sizes
• Scoped memory creation and size
MemoryArea ma = new LTMemory(10000);
“create a new scoped memory with 10,000 bytes”
• If try to allocate more than 10,000 bytes,
implementation throws an exception
• Problems
• Java does not specify object sizes
• Size of given object may change during its
lifetime in computation
• So how big to make scoped memory?
Modularity Problems
Scoped Memory Size
Determined Here
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
Required Size
Determined by
Behavior of Code in
this Computation
Modularity Problems
Scoped Memory Size
Determined Here
Objects
Scoped
Memory
Objects in
GC Heap
Standard
Java
Computation
• If change program,
size may need to
change!
• Amount of allocated
memory becomes
part of interface!
More Issues
• Different executions may allocate different
amounts of data
• Lots of hidden allocation in libraries
• Difficult to find out how much memory is
really allocated
• If change implementation, may need to
change scoped memory size in clients
Analysis Solution
• Analyze program to symbolically compute
allocated memory sizes
• Input variables
• Object sizes
• Compiler knows object sizes, can
conservatively generate scoped memory sizes
Interaction with Garbage Collector
• Standard Collector Assumptions
• Can interrupt computation at any point
• Can suspend for unbounded time
• Real-Time Java extension
Immortal
• No-Heap Real-Time Threads
• Can Access
• Immortal memory
Scoped
GC Heap
• Scoped memory
• Do not interact with GC heap AT ALL
• Can run asynchronously with GC
No-Heap Real-Time Thread Checks
• Dynamically check that no-heap real-time threads
never access a location containing a reference into
garbage-collected heap
• At every read, check to make sure result does not
point into garbage-collected heap
• At every write, check to make sure not overwriting
reference into GC heap
• If check fails, throw exception
• Drawbacks
• Dynamic checking overhead
• New class of dynamic errors
Implementation
• FLEX compiler infrastructure
(www.flexc.lcs.mit.edu)
• Implemented no-heap real-time threads
• Implemented access checks
• Measured performance with and without
checks
Experimental Results
Scope Checks
Heap Checks
Application
300
Time (sec)
250
200
150
100
50
0
Array
Tree
Benchmark
Water
Program Analysis for Eliminating
Checks
• Control-flow analysis to identify code that
may execute in no-heap real-time thread
• Global value-flow analysis
• Tags each value that points to GC heap
• Identifies all locations into which these
values may flow
• Combine results
• Look at all no-heap real-time thread code
• Check statically for access violations
Indirect Priority Inversions
Interaction Between Resource Sharing
and Garbage Collection
Lock
Acquire
No-heap
Thread
Garbage
Collector
Lock
Acquire
Standard Java
Thread
Indirect Priority Inversions
Interaction Between Resource Sharing
and Garbage Collection
Lock
Acquire
Blocks
Lock
Acquire
No-heap
Thread
Garbage
Collector
Standard Java
Thread
Indirect Priority Inversions
No-heap thread must wait for standard Java
thread to release lock
Lock
Standard thread must
Acquire
wait for GC to finish
(heap inconsistent until it finishes)
No-heap thread must wait for GC!
No-heap
Thread
Garbage
Collector
Lock
Acquire
Standard Java
Thread
Using Non-Blocking Synchronization to
Eliminate Indirect Priority Inversions
Start
Atomic
Region
Does
Not
Block!
Start
Atomic
Region
End
Atomic
Region
No-heap
Thread
Garbage
Collector
Abort,
Retry
Standard
Java
Thread
Implementation Status
• Non-blocking synchronization implemented
for memory management primitives
• Useful when threads share scoped memory
• Uses non-blocking synchronization
instructions from processor
• Software implementation underway for
general atomic regions
Goal
Enable safe real-time code to interact
successfully with code that accesses GC data
Issues and Complications
• Dangling references for scoped memories
• Resource needs of computations
• Isolating computations from garbage collector
• Ensure threads with real-time constraints
don’t access garbage collected data
• Eliminate indirect interactions
• Our view
• Dynamic checks inadequate
• Statically verify correct use, eliminate checks
Broader View of Real-Time Java
• Java is best suited to express “batch”
computations on objects
• Not so good for control in asynchronous,
parallel, distributed, time-aware systems
• Inadequate for design/requirements
• Can be part of solution, but only a part
Multiple Perspectives
• Any system has many desired properties
• Data structure invariants
• Flow of data between different components
• Timing requirements for computations
• Each property is inherently partial
• Many properties are best expressed as
• Declarative constraints
• NOT translatable into implementation
Design Conformance
Object Referencing
Relationships
Dataflow
Interactions
Timing Constraints
Check that implementation
conforms to design properties
Implementation
Future
• Scheduling and event delivery
• More precise referencing relationship analysis
• Completely characterize aliasing behavior
• More flexible memory management algorithms
that preserve predictability
• More flexible regions
• Immediate deallocation
• Less restricted memory access constraints for realtime threads
• Design conformance for more control-oriented
properties
Summary
• Real-time Java code coexists and interacts with
standard Java code
• New complications (overhead + failure modes)
• Scoped memory checks
• Scoped memory sizes
• No-heap real-time threads
• Indirect priority inversions
• Attacked with program analysis
• Future: scheduling and timing