Download Enhancing imperative exact real arithmetic with functions and logic

Enhancing imperative
exact real arithmetic
with
functions and logic
Norbert Müller 1
Universität Trier
1
Email: [email protected]
Norbert Müller (Universität Trier)
Nijmegen 2009
1 / 26
Introduction
1
Introduction
2
Basic structure of the iRRAM
3
Function objects
4
Application Examples
5
Final remarks
Norbert Müller (Universität Trier)
Nijmegen 2009
2 / 26
Introduction
The ‘right’ programming language for real arithmetic?
functional?
(O’Connor-05),(Bauer+Kavkler-08),(Escardo-??),(Müller-95),...
x ∈R ≈ Φ:N→Q
f : R → R ≈ Ψ : (N → Q) → (N → Q)
I
usual lazy evaluation
; (often) abysmal performance
; use ‘stateful’ functional programming (Monads, OCAML)
imperative
I
(Err.+Heck.-02)(Briggs-06),(Lambov-07),(Müller-96),....
CPUs work in imperative way
; easier to exploit all the possibilities of current CPUs
optimal(?): use best parts of both progamming schemes
where to switch: elegancy (functional) ↔ efficiency (imperative)?
many flavors of exact real arithmetic might survive...
Norbert Müller (Universität Trier)
Nijmegen 2009
3 / 26
Introduction
Main focus: low-precision computations!
Optimized for up ≈ 1000 bits per number
Allowing big data sets (; linear algebra)
Higher precision seamlessly usable
Suitable for billions of dependant operations, e.g.
X
1≤n≤109
1
n
(double precision ‘approximation’: simple loop, <10 seconds)
Today:
few new internals of iRRAM package since CCA-2005 (Nijmegen)
try expressiveness of functional languages in C++
approach started 2005, since 2008 part of published iRRAM
Norbert Müller (Universität Trier)
Nijmegen 2009
4 / 26
Introduction
Motivating example: Differential equations
try to solve IVPs
efficient solutions need holomorphic flow
computing with Taylor series is much faster near the center...
; solve IVPs in steps...
example: y = ẋ, x = −ẏ with solution (sin(t), cos(t))
Exact trajectory
Evaluated points
1
0.8
0.6
0.4
0.2
0
−0.2
−0.4
−0.6
−0.8
−1
0
5
10
15
20
1
0.8
0.6
0.4
0 0.2
−0.2
−0.4
−0.6
25−1−0.8
; Iterate: fi := IVP(xi ), xi+1 := fi (ti )
Norbert Müller (Universität Trier)
Nijmegen 2009
5 / 26
Basic structure of the iRRAM
1
Introduction
2
Basic structure of the iRRAM
3
Function objects
4
Application Examples
5
Final remarks
Norbert Müller (Universität Trier)
Nijmegen 2009
6 / 26
Basic structure of the iRRAM
Aim: be competitive with ordinary ‘double’ arithmetic!
consider bit complexity (at low level!)
Turing machines are ‘realistic’ model (Type-2 or Oracle)
differences between constructive / classical logic not yet important
TTE is reflected in many implementational (often hidden!) details:
use some admissible representation for R
I/O operations can model Type-2-machines
other modes look like oracle machines
metric spaces with dense subsets can be used
use multi-valued functions
...
Norbert Müller (Universität Trier)
Nijmegen 2009
7 / 26
Basic structure of the iRRAM
Other Approaches
using functional extension FC++ of C++, (Briggs-06)
x ∈R ≈ λ:Z→Z
typedef Fun1<int,Z> lambda;
...
class XR: public XRsig { public:
lambda x;
Z operator() (const int n) const { return x(n); }
...
class AddHelper: public XRsig {
XR f; XR g;
AddHelper(const XR& ff, const XR& gg): {f(ff),g(gg)}
Z operator()(const int n) const {
... return ( f(n+2)+g(n+2)+2 ) >>2; ...
}
}
Norbert Müller (Universität Trier)
Nijmegen 2009
8 / 26
Basic structure of the iRRAM
Directed Acyclic Graphs (DAGs)
X3
-
*
*
*
X2
X_0
X_1
X_2
X_3
I
I
I
I
=
=
=
=
0.5 ;
(1 - X_0) * (X_0 * 3.75);
(1 - X_1) * (X_1 * 3.75);
(1 - X_2) * (X_2 * 3.75);
X1
*
*
X0
1
*
0.5
3.75
DAGs: simple programming exercise...
quite memory intensive: ≈50 bytes per node
Total number of DAG nodes at most ≈100 million.
DAG-construction roughly comparable to ‘lazy evaluation’
Norbert Müller (Universität Trier)
Nijmegen 2009
9 / 26
Basic structure of the iRRAM
Removing the algebraic structure of DAGs
I
I
implemented sequence F of functions F : N × R → R
known speed of convergences to limit f , e.g.
|f (z) − F (n, z)| ≤ 2−n
I
then Y = f (X ) is implemented as Y=limit(F,X)
Y
X
Norbert Müller (Universität Trier)
limit
REAL F(int n, REAL Z)
Nijmegen 2009
10 / 26
Basic structure of the iRRAM
Evaluation of DAGs
X3
conversion from DAGs to other datatypes...
usually as a multivalued function
top-down (Briggs-06):
I
I
-
I
I
I
X1
gross overestimations of precision
bad performance...
*
*
X0
based on interval arithmetic
in iterations
reasonable performance
efficient evaluation not trivial!
iRRAM:
*
*
X2
bottom-up (Lambov-07)
I
*
X_0
X_1
X_2
X_3
=
=
=
=
1
*
0.5
3.75
0.5 ;
(1 - X_0) * (X_0 * 3.75);
(1 - X_1) * (X_1 * 3.75);
(1 - X_2) * (X_2 * 3.75);
forget about the DAG...,
directly iterate the program!
; admissible representation using intervals
; sequence spread over time ...
; based on: R is metric space with ‘simple’ dense subset Q
Norbert Müller (Universität Trier)
Nijmegen 2009
11 / 26
Basic structure of the iRRAM
Underlying Intervals:
usually: centered intervals I
I = {x : d−r ≤ x ≤ d+r}
center d: multiple precision (e.g. MPFR)
radius r : fixed precision (but large exponent range)
in initial iteration:
I = {x : l ≤ x ≤ u}
with l,r : double precision (hardware, fast!!!!!)
optimization of this initial iteration:
inline functions, no function calls for basic arithmetic!
malloc will not be called.
[l, u] is stored as pair (l, −u) (no change of rounding modes!)
use SSE2-parts of CPUs for parallelization
iRRAM: dynamic switching between interval types
(RealLib: fixed at compile time)
Norbert Müller (Universität Trier)
Nijmegen 2009
12 / 26
Basic structure of the iRRAM
Short benchmark...
n
X
1
i=1
i
package
unverified double
Reallib, DAGs
Reallib, DAGs
iRRAM
Reallib, DAG-free
iRRAM
Reallib, DAG-free
Norbert Müller (Universität Trier)
x=0; one=1;
for (i=1;i<=n;i++)
x = x + one/i
as
n
n = 109
n = 107
n = 109
n = 109
n = 109
n = 109
n = 109
precision
time
space
(10 decimals)
7.5s
≈0B
-
-
1.7 GB
unfeasable
5 decimals
5 decimals
15s
12.5s
≈0B
≈0B
20 decimals
20 decimals
300s
550s
≈0B
≈0B
Nijmegen 2009
13 / 26
Basic structure of the iRRAM
Lazy Booleans:
Extend bool= {T , F } to
LAZY_BOOLEAN= {T , F , ⊥} (Err.+Heck.-02)
a||b
T
F
⊥
T
T
T
T
F ⊥
T T
F ⊥
⊥ ⊥
,
a&&b
T
F
⊥

 T, x < y
F, x > y
(x<y) =

⊥, x = y
T
T
F
⊥
F
F
F
F
⊥
⊥
F
⊥
b
T
F
⊥
,
!a
T
F
⊥
F
T
⊥
bool(b)
T
F
undefined
Implicit conversion to bool e.g. in ‘if ( a<b && c>d ) {...}’
Conversion ⊥ to bool leads to iteration
Expressions like ‘x<1 || x>0’ are total now!
Norbert Müller (Universität Trier)
Nijmegen 2009
14 / 26
Function objects
1
Introduction
2
Basic structure of the iRRAM
3
Function objects
4
Application Examples
5
Final remarks
Norbert Müller (Universität Trier)
Nijmegen 2009
15 / 26
Function objects
Functions as objects?
older functional approaches in C++ already too slow for
x ∈R ≈ Φ:N→Q
so no real functions as objects... (only as predefined algorithms...)
f : R → R ≈ Ψ : (N → Q) → (N → Q)
1.2
standard TTE approach:
1
use a metric on functions spaces
I implement some dense subset
I implement functions as limits
; exponential space complexity...
I
0.8
0.6
0.4
0.2
sqrt(x)
3-bit, 50 boxes
5-bit, 200 boxes
0
-0.2
0
Norbert Müller (Universität Trier)
Nijmegen 2009
0.2
0.4
0.6
0.8
1
16 / 26
Function objects
Solution ????
Use DAGs...
Add enough structure to get an admissible representation...
... but don’t try to implement the conversion procedures...
What then is the main ‘evaluation’ for these DAGs?
I
I
functional values at points?
functional values at sets or intervals?
Current state (neither minimal nor complete...)
use ‘algebraic approach’ (functional values at points)
influenced by (Brattka-98)
template<class PARAM,class RESULT> class FUNCTION;
Norbert Müller (Universität Trier)
Nijmegen 2009
17 / 26
Function objects
For arbitrary parameter and result types:
General constructors
h = from_algorithm (my_alg)
h = from_value (my_constant)
Structural operators
h = compose(g,f), h=g(f)
h(x) = g(f (x))
h = product(f,g)
h(x, y ) = (f (x), g(y ))
h = juxtaposition(f,g)
h(x) = (f (x), g(x))
Evaluation
y = f(x)
Projections
h1 = first(f), h2 = second(f)
hi = projection(f,i)
Partial Application
h = bind_first(f,a)
h = bind_second(f,b)
For result types with arithmetic operations:
Special arithmetic
h = f+g
Special constructors
h = polynomial(coeff_vector)
Miscellaneous
h = f+c, h = c+f, ...
For result type lazy_boolean:
Comparison operators
h
h
Logical operators
h
h
h
Norbert Müller (Universität Trier)
=
=
=
=
=
f<g
f>g
f||g
f&&g
!f
Nijmegen 2009
h(x) = f (x) < g(x)
18 / 26
Application Examples
1
Introduction
2
Basic structure of the iRRAM
3
Function objects
4
Application Examples
5
Final remarks
Norbert Müller (Universität Trier)
Nijmegen 2009
19 / 26
Application Examples
A simple function...
Exercise: implement logistic map
xi+1 = f (xi ) , f (x) = 3.75 · (1 − x) · x
#include "iRRAM.h"
using namespace iRRAM;
REAL id_REAL( const REAL& x ){ return x; };
void compute(){
int count; cin >> count; REAL x = 0.5;
FUNCTION<REAL,REAL> REAL X = from_algorithm(id_REAL);
FUNCTION<REAL,REAL> LM = REAL(3.75) * (REAL(1) - X) * X;
for ( int i=1; i<=count; i++ ) x = LM(x);
cout << x << "\n";
}
(efficiency here almost identical to direct implementation)
Norbert Müller (Universität Trier)
Nijmegen 2009
20 / 26
Application Examples
Dedekind cuts:
L
||
juxtaposition
√
Example: 2 can be defined by
I
I
<
bounds l = 1, u = 2
lower set L = {x | x < 0 ∨ x 2 < 2}
upper set U = {x | 0 < x ∧ x 2 > 2}
<
juxtaposition
juxtaposition
from
value
0
X
Represent e.g. L as a predicate:
∗
from
value
juxtaposition
2
from
algorithm
id REAL
REAL id_REAL( const REAL& x ){ return x; };
FUNCTION<REAL,REAL> X = from_algorithm(id_REAL);
FUNCTION<REAL,LAZY_BOOLEAN> L = X<REAL(0) || X*X<REAL(2);
Norbert Müller (Universität Trier)
Nijmegen 2009
21 / 26
Application Examples
Basic idea usable as constructor for a REAL, e.g.
REAL cut(
const RATIONAL left_bound, right_bound,
FUNCTION<REAL,LAZY_BOOLEAN> smaller, larger )
Now implement
√
2 by
void compute(){
REAL id_REAL( const REAL& x ){ return x; };
FUNCTION<REAL,REAL> X=from_algorithm(id_REAL);
REAL x = cut(1,2,
X<REAL(0) || X*X<REAL(2),
REAL(0)<X && REAL(2)<X*X);
cout « setRwidth(100) « x « "\n";
}
current implementation of cut:
I
I
unoptimized trisection with slow limit operator..
just useful as proof of concept...
Norbert Müller (Universität Trier)
Nijmegen 2009
22 / 26
Application Examples
Predicates on real numbers:
predicates on reals ≈ functions to LAZY_BOOLEAN
; ‘real’ logic in C++
small example:
typedef std::vector<REAL> UNIVERSE;
typedef FUNCTION<UNIVERSE,REAL> REAL_EXPRESSION;
typedef FUNCTION<UNIVERSE,LAZY_BOOLEAN> REAL_PREDICATE;
UNIVERSE id_vector_REAL( const UNIVERSE & x ){ return x; };
FUNCTION<UNIVERSE,UNIVERSE> variables=from_algorithm(id_vector_REAL)
void compute(){
REAL_EXPRESSION X = projection(variables,0);
REAL_EXPRESSION Y = projection(variables,1);
REAL_EXPRESSION Z = X*Y+X/Y;
REAL_PREDICATE P = Z > X || X < REAL(1) ;
UNIVERSE assign(2);
assign[0] = 1; assign[1] = 2;
cout « Z(assign) « " " « P(assign) « "\n";
};
Norbert Müller (Universität Trier)
Nijmegen 2009
23 / 26
Final remarks
1
Introduction
2
Basic structure of the iRRAM
3
Function objects
4
Application Examples
5
Final remarks
Norbert Müller (Universität Trier)
Nijmegen 2009
24 / 26
Final remarks
A lot of the elegance of functional languages can be ported to C++
Current state allows evaluation with good efficiency and
reasonable error propagation
Functions objects can easily be extended:
I
I
I
I
Evaluation could be optimized with regards to error propagation
(with losses in speed)
This is closely connected to efficient evaluation on sets (not only
intervals...)
Things like automated differentiation can easily be added...
The predicates should be enhanced with quantification...
(efficiently?)
Parts of verification on real numbers (O’Connor-05),(Bauer-08)
can be expressed in object-oriented languages!
A hypothetical approach mixing object-oriented and genuinely
functional languages might switch between the paradigms on a
quite high level!
Norbert Müller (Universität Trier)
Nijmegen 2009
25 / 26
Final remarks
Thank you for your attention!
Any questions or remarks?
Norbert Müller (Universität Trier)
Nijmegen 2009
26 / 26
References
[Bauer-08] Andrej Bauer. Efficient computation with dedekind reals. In
CCA, 2008.
[Bauer+Kavkler-08] Andrej Bauer and Iztok Kavkler. Implementing
real numbers with rz. Electron. Notes Theor. Comput. Sci.,
202:365–384, 2008.
[Brattka-98] Vasco Brattka. Recursive and Computable Operations
over Topological Structures. PhD thesis, Fachbereich
Informatik, FernUniversität Hagen, 1998.
[Briggs-06] Keith Briggs. Implementing exact real arithmetic in c++,
python, and c. Theor. Comput. Sci., 351:374–81, 2006.
[Err.+Heck.-02] L. Errington and R. Heckmann. Using the IC Reals
library. Technical report, Imperial College, 2002.
[Lambov-06] Branimir Lambov. Interval arithmetic using sse-2. In
Peter Hertling, Christoph M. Hoffmann, Wolfram Luther, and
Nathalie Revol, editors, Reliable Implementation of Real
Number Algorithms: Theory and Practice, number 06021 in
Dagstuhl Seminar Proceedings. Internationales BegegnungsNorbert Müller (Universität Trier)
Nijmegen 2009
26 / 26
References
und Forschungszentrum fuer Informatik (IBFI), Schloss
Dagstuhl, Germany, 2006.
<http://drops.dagstuhl.de/opus/volltexte/2006/714> [date of
citation: 2006-01-01].
[Lambov-07] Branimir Lambov. Reallib: An efficient implementation of
exact real arithmetic. Mathematical Structures in Computer
Science, 17(1):81–98, 2007.
[Müller-95] Norbert Th. Müller. Constructive aspects of analytic
functions. In Ker-I Ko and Klaus Weihrauch, editors,
Computability and Complexity in Analysis, volume 190 of
Informatik Berichte, pages 105–114. FernUniversität Hagen,
September 1995. CCA Workshop, Hagen, August 19–20,
1995.
[Müller-96] Norbert Th. Müller. Towards a real Real RAM: a prototype
using C++. In Ker-I Ko, Norbert Müller, and Klaus
Weihrauch, editors, Computability and Complexity in
Analysis, pages 59–66. Universität Trier, 1996. Second CCA
Workshop, Trier, August 22–23, 1996.
Norbert Müller (Universität Trier)
Nijmegen 2009
26 / 26
Final remarks
[Müller-98] Norbert Th. Müller. Implementing limits in an interactive
RealRAM. In J.-M. Chesneaux, F. Jézéquel, J.-L. Lamotte,
and J. Vignes, editors, Third Real Numbers and Computers
Conference, pages 59–66. Université Pierre et Marie Curie,
Paris, 1998. Paris, France, April 27-29, 1998.
[Mülller-01] Norbert Müller. The iRRAM: Exact Arithmetic in C++.
Lecture notes in computer science, 2991:222–252, 2001.
[O’Connor-05] Russell O’Connor. Few Digits. 2005.
[Escardo-??] Martín Escardo. ?????
[Wei00] Klaus Weihrauch. Computable analysis: an introduction.
Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2000.
Norbert Müller (Universität Trier)
Nijmegen 2009
26 / 26