Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Enhancing imperative exact real arithmetic with functions and logic Norbert Müller 1 Universität Trier 1 Email: [email protected] Norbert Müller (Universität Trier) Nijmegen 2009 1 / 26 Introduction 1 Introduction 2 Basic structure of the iRRAM 3 Function objects 4 Application Examples 5 Final remarks Norbert Müller (Universität Trier) Nijmegen 2009 2 / 26 Introduction The ‘right’ programming language for real arithmetic? functional? (O’Connor-05),(Bauer+Kavkler-08),(Escardo-??),(Müller-95),... x ∈R ≈ Φ:N→Q f : R → R ≈ Ψ : (N → Q) → (N → Q) I usual lazy evaluation ; (often) abysmal performance ; use ‘stateful’ functional programming (Monads, OCAML) imperative I (Err.+Heck.-02)(Briggs-06),(Lambov-07),(Müller-96),.... CPUs work in imperative way ; easier to exploit all the possibilities of current CPUs optimal(?): use best parts of both progamming schemes where to switch: elegancy (functional) ↔ efficiency (imperative)? many flavors of exact real arithmetic might survive... Norbert Müller (Universität Trier) Nijmegen 2009 3 / 26 Introduction Main focus: low-precision computations! Optimized for up ≈ 1000 bits per number Allowing big data sets (; linear algebra) Higher precision seamlessly usable Suitable for billions of dependant operations, e.g. X 1≤n≤109 1 n (double precision ‘approximation’: simple loop, <10 seconds) Today: few new internals of iRRAM package since CCA-2005 (Nijmegen) try expressiveness of functional languages in C++ approach started 2005, since 2008 part of published iRRAM Norbert Müller (Universität Trier) Nijmegen 2009 4 / 26 Introduction Motivating example: Differential equations try to solve IVPs efficient solutions need holomorphic flow computing with Taylor series is much faster near the center... ; solve IVPs in steps... example: y = ẋ, x = −ẏ with solution (sin(t), cos(t)) Exact trajectory Evaluated points 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 0 5 10 15 20 1 0.8 0.6 0.4 0 0.2 −0.2 −0.4 −0.6 25−1−0.8 ; Iterate: fi := IVP(xi ), xi+1 := fi (ti ) Norbert Müller (Universität Trier) Nijmegen 2009 5 / 26 Basic structure of the iRRAM 1 Introduction 2 Basic structure of the iRRAM 3 Function objects 4 Application Examples 5 Final remarks Norbert Müller (Universität Trier) Nijmegen 2009 6 / 26 Basic structure of the iRRAM Aim: be competitive with ordinary ‘double’ arithmetic! consider bit complexity (at low level!) Turing machines are ‘realistic’ model (Type-2 or Oracle) differences between constructive / classical logic not yet important TTE is reflected in many implementational (often hidden!) details: use some admissible representation for R I/O operations can model Type-2-machines other modes look like oracle machines metric spaces with dense subsets can be used use multi-valued functions ... Norbert Müller (Universität Trier) Nijmegen 2009 7 / 26 Basic structure of the iRRAM Other Approaches using functional extension FC++ of C++, (Briggs-06) x ∈R ≈ λ:Z→Z typedef Fun1<int,Z> lambda; ... class XR: public XRsig { public: lambda x; Z operator() (const int n) const { return x(n); } ... class AddHelper: public XRsig { XR f; XR g; AddHelper(const XR& ff, const XR& gg): {f(ff),g(gg)} Z operator()(const int n) const { ... return ( f(n+2)+g(n+2)+2 ) >>2; ... } } Norbert Müller (Universität Trier) Nijmegen 2009 8 / 26 Basic structure of the iRRAM Directed Acyclic Graphs (DAGs) X3 - * * * X2 X_0 X_1 X_2 X_3 I I I I = = = = 0.5 ; (1 - X_0) * (X_0 * 3.75); (1 - X_1) * (X_1 * 3.75); (1 - X_2) * (X_2 * 3.75); X1 * * X0 1 * 0.5 3.75 DAGs: simple programming exercise... quite memory intensive: ≈50 bytes per node Total number of DAG nodes at most ≈100 million. DAG-construction roughly comparable to ‘lazy evaluation’ Norbert Müller (Universität Trier) Nijmegen 2009 9 / 26 Basic structure of the iRRAM Removing the algebraic structure of DAGs I I implemented sequence F of functions F : N × R → R known speed of convergences to limit f , e.g. |f (z) − F (n, z)| ≤ 2−n I then Y = f (X ) is implemented as Y=limit(F,X) Y X Norbert Müller (Universität Trier) limit REAL F(int n, REAL Z) Nijmegen 2009 10 / 26 Basic structure of the iRRAM Evaluation of DAGs X3 conversion from DAGs to other datatypes... usually as a multivalued function top-down (Briggs-06): I I - I I I X1 gross overestimations of precision bad performance... * * X0 based on interval arithmetic in iterations reasonable performance efficient evaluation not trivial! iRRAM: * * X2 bottom-up (Lambov-07) I * X_0 X_1 X_2 X_3 = = = = 1 * 0.5 3.75 0.5 ; (1 - X_0) * (X_0 * 3.75); (1 - X_1) * (X_1 * 3.75); (1 - X_2) * (X_2 * 3.75); forget about the DAG..., directly iterate the program! ; admissible representation using intervals ; sequence spread over time ... ; based on: R is metric space with ‘simple’ dense subset Q Norbert Müller (Universität Trier) Nijmegen 2009 11 / 26 Basic structure of the iRRAM Underlying Intervals: usually: centered intervals I I = {x : d−r ≤ x ≤ d+r} center d: multiple precision (e.g. MPFR) radius r : fixed precision (but large exponent range) in initial iteration: I = {x : l ≤ x ≤ u} with l,r : double precision (hardware, fast!!!!!) optimization of this initial iteration: inline functions, no function calls for basic arithmetic! malloc will not be called. [l, u] is stored as pair (l, −u) (no change of rounding modes!) use SSE2-parts of CPUs for parallelization iRRAM: dynamic switching between interval types (RealLib: fixed at compile time) Norbert Müller (Universität Trier) Nijmegen 2009 12 / 26 Basic structure of the iRRAM Short benchmark... n X 1 i=1 i package unverified double Reallib, DAGs Reallib, DAGs iRRAM Reallib, DAG-free iRRAM Reallib, DAG-free Norbert Müller (Universität Trier) x=0; one=1; for (i=1;i<=n;i++) x = x + one/i as n n = 109 n = 107 n = 109 n = 109 n = 109 n = 109 n = 109 precision time space (10 decimals) 7.5s ≈0B - - 1.7 GB unfeasable 5 decimals 5 decimals 15s 12.5s ≈0B ≈0B 20 decimals 20 decimals 300s 550s ≈0B ≈0B Nijmegen 2009 13 / 26 Basic structure of the iRRAM Lazy Booleans: Extend bool= {T , F } to LAZY_BOOLEAN= {T , F , ⊥} (Err.+Heck.-02) a||b T F ⊥ T T T T F ⊥ T T F ⊥ ⊥ ⊥ , a&&b T F ⊥ T, x < y F, x > y (x<y) = ⊥, x = y T T F ⊥ F F F F ⊥ ⊥ F ⊥ b T F ⊥ , !a T F ⊥ F T ⊥ bool(b) T F undefined Implicit conversion to bool e.g. in ‘if ( a<b && c>d ) {...}’ Conversion ⊥ to bool leads to iteration Expressions like ‘x<1 || x>0’ are total now! Norbert Müller (Universität Trier) Nijmegen 2009 14 / 26 Function objects 1 Introduction 2 Basic structure of the iRRAM 3 Function objects 4 Application Examples 5 Final remarks Norbert Müller (Universität Trier) Nijmegen 2009 15 / 26 Function objects Functions as objects? older functional approaches in C++ already too slow for x ∈R ≈ Φ:N→Q so no real functions as objects... (only as predefined algorithms...) f : R → R ≈ Ψ : (N → Q) → (N → Q) 1.2 standard TTE approach: 1 use a metric on functions spaces I implement some dense subset I implement functions as limits ; exponential space complexity... I 0.8 0.6 0.4 0.2 sqrt(x) 3-bit, 50 boxes 5-bit, 200 boxes 0 -0.2 0 Norbert Müller (Universität Trier) Nijmegen 2009 0.2 0.4 0.6 0.8 1 16 / 26 Function objects Solution ???? Use DAGs... Add enough structure to get an admissible representation... ... but don’t try to implement the conversion procedures... What then is the main ‘evaluation’ for these DAGs? I I functional values at points? functional values at sets or intervals? Current state (neither minimal nor complete...) use ‘algebraic approach’ (functional values at points) influenced by (Brattka-98) template<class PARAM,class RESULT> class FUNCTION; Norbert Müller (Universität Trier) Nijmegen 2009 17 / 26 Function objects For arbitrary parameter and result types: General constructors h = from_algorithm (my_alg) h = from_value (my_constant) Structural operators h = compose(g,f), h=g(f) h(x) = g(f (x)) h = product(f,g) h(x, y ) = (f (x), g(y )) h = juxtaposition(f,g) h(x) = (f (x), g(x)) Evaluation y = f(x) Projections h1 = first(f), h2 = second(f) hi = projection(f,i) Partial Application h = bind_first(f,a) h = bind_second(f,b) For result types with arithmetic operations: Special arithmetic h = f+g Special constructors h = polynomial(coeff_vector) Miscellaneous h = f+c, h = c+f, ... For result type lazy_boolean: Comparison operators h h Logical operators h h h Norbert Müller (Universität Trier) = = = = = f<g f>g f||g f&&g !f Nijmegen 2009 h(x) = f (x) < g(x) 18 / 26 Application Examples 1 Introduction 2 Basic structure of the iRRAM 3 Function objects 4 Application Examples 5 Final remarks Norbert Müller (Universität Trier) Nijmegen 2009 19 / 26 Application Examples A simple function... Exercise: implement logistic map xi+1 = f (xi ) , f (x) = 3.75 · (1 − x) · x #include "iRRAM.h" using namespace iRRAM; REAL id_REAL( const REAL& x ){ return x; }; void compute(){ int count; cin >> count; REAL x = 0.5; FUNCTION<REAL,REAL> REAL X = from_algorithm(id_REAL); FUNCTION<REAL,REAL> LM = REAL(3.75) * (REAL(1) - X) * X; for ( int i=1; i<=count; i++ ) x = LM(x); cout << x << "\n"; } (efficiency here almost identical to direct implementation) Norbert Müller (Universität Trier) Nijmegen 2009 20 / 26 Application Examples Dedekind cuts: L || juxtaposition √ Example: 2 can be defined by I I < bounds l = 1, u = 2 lower set L = {x | x < 0 ∨ x 2 < 2} upper set U = {x | 0 < x ∧ x 2 > 2} < juxtaposition juxtaposition from value 0 X Represent e.g. L as a predicate: ∗ from value juxtaposition 2 from algorithm id REAL REAL id_REAL( const REAL& x ){ return x; }; FUNCTION<REAL,REAL> X = from_algorithm(id_REAL); FUNCTION<REAL,LAZY_BOOLEAN> L = X<REAL(0) || X*X<REAL(2); Norbert Müller (Universität Trier) Nijmegen 2009 21 / 26 Application Examples Basic idea usable as constructor for a REAL, e.g. REAL cut( const RATIONAL left_bound, right_bound, FUNCTION<REAL,LAZY_BOOLEAN> smaller, larger ) Now implement √ 2 by void compute(){ REAL id_REAL( const REAL& x ){ return x; }; FUNCTION<REAL,REAL> X=from_algorithm(id_REAL); REAL x = cut(1,2, X<REAL(0) || X*X<REAL(2), REAL(0)<X && REAL(2)<X*X); cout « setRwidth(100) « x « "\n"; } current implementation of cut: I I unoptimized trisection with slow limit operator.. just useful as proof of concept... Norbert Müller (Universität Trier) Nijmegen 2009 22 / 26 Application Examples Predicates on real numbers: predicates on reals ≈ functions to LAZY_BOOLEAN ; ‘real’ logic in C++ small example: typedef std::vector<REAL> UNIVERSE; typedef FUNCTION<UNIVERSE,REAL> REAL_EXPRESSION; typedef FUNCTION<UNIVERSE,LAZY_BOOLEAN> REAL_PREDICATE; UNIVERSE id_vector_REAL( const UNIVERSE & x ){ return x; }; FUNCTION<UNIVERSE,UNIVERSE> variables=from_algorithm(id_vector_REAL) void compute(){ REAL_EXPRESSION X = projection(variables,0); REAL_EXPRESSION Y = projection(variables,1); REAL_EXPRESSION Z = X*Y+X/Y; REAL_PREDICATE P = Z > X || X < REAL(1) ; UNIVERSE assign(2); assign[0] = 1; assign[1] = 2; cout « Z(assign) « " " « P(assign) « "\n"; }; Norbert Müller (Universität Trier) Nijmegen 2009 23 / 26 Final remarks 1 Introduction 2 Basic structure of the iRRAM 3 Function objects 4 Application Examples 5 Final remarks Norbert Müller (Universität Trier) Nijmegen 2009 24 / 26 Final remarks A lot of the elegance of functional languages can be ported to C++ Current state allows evaluation with good efficiency and reasonable error propagation Functions objects can easily be extended: I I I I Evaluation could be optimized with regards to error propagation (with losses in speed) This is closely connected to efficient evaluation on sets (not only intervals...) Things like automated differentiation can easily be added... The predicates should be enhanced with quantification... (efficiently?) Parts of verification on real numbers (O’Connor-05),(Bauer-08) can be expressed in object-oriented languages! A hypothetical approach mixing object-oriented and genuinely functional languages might switch between the paradigms on a quite high level! Norbert Müller (Universität Trier) Nijmegen 2009 25 / 26 Final remarks Thank you for your attention! Any questions or remarks? Norbert Müller (Universität Trier) Nijmegen 2009 26 / 26 References [Bauer-08] Andrej Bauer. Efficient computation with dedekind reals. In CCA, 2008. [Bauer+Kavkler-08] Andrej Bauer and Iztok Kavkler. Implementing real numbers with rz. Electron. Notes Theor. Comput. Sci., 202:365–384, 2008. [Brattka-98] Vasco Brattka. Recursive and Computable Operations over Topological Structures. PhD thesis, Fachbereich Informatik, FernUniversität Hagen, 1998. [Briggs-06] Keith Briggs. Implementing exact real arithmetic in c++, python, and c. Theor. Comput. Sci., 351:374–81, 2006. [Err.+Heck.-02] L. Errington and R. Heckmann. Using the IC Reals library. Technical report, Imperial College, 2002. [Lambov-06] Branimir Lambov. Interval arithmetic using sse-2. In Peter Hertling, Christoph M. Hoffmann, Wolfram Luther, and Nathalie Revol, editors, Reliable Implementation of Real Number Algorithms: Theory and Practice, number 06021 in Dagstuhl Seminar Proceedings. Internationales BegegnungsNorbert Müller (Universität Trier) Nijmegen 2009 26 / 26 References und Forschungszentrum fuer Informatik (IBFI), Schloss Dagstuhl, Germany, 2006. <http://drops.dagstuhl.de/opus/volltexte/2006/714> [date of citation: 2006-01-01]. [Lambov-07] Branimir Lambov. Reallib: An efficient implementation of exact real arithmetic. Mathematical Structures in Computer Science, 17(1):81–98, 2007. [Müller-95] Norbert Th. Müller. Constructive aspects of analytic functions. In Ker-I Ko and Klaus Weihrauch, editors, Computability and Complexity in Analysis, volume 190 of Informatik Berichte, pages 105–114. FernUniversität Hagen, September 1995. CCA Workshop, Hagen, August 19–20, 1995. [Müller-96] Norbert Th. Müller. Towards a real Real RAM: a prototype using C++. In Ker-I Ko, Norbert Müller, and Klaus Weihrauch, editors, Computability and Complexity in Analysis, pages 59–66. Universität Trier, 1996. Second CCA Workshop, Trier, August 22–23, 1996. Norbert Müller (Universität Trier) Nijmegen 2009 26 / 26 Final remarks [Müller-98] Norbert Th. Müller. Implementing limits in an interactive RealRAM. In J.-M. Chesneaux, F. Jézéquel, J.-L. Lamotte, and J. Vignes, editors, Third Real Numbers and Computers Conference, pages 59–66. Université Pierre et Marie Curie, Paris, 1998. Paris, France, April 27-29, 1998. [Mülller-01] Norbert Müller. The iRRAM: Exact Arithmetic in C++. Lecture notes in computer science, 2991:222–252, 2001. [O’Connor-05] Russell O’Connor. Few Digits. 2005. [Escardo-??] Martín Escardo. ????? [Wei00] Klaus Weihrauch. Computable analysis: an introduction. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2000. Norbert Müller (Universität Trier) Nijmegen 2009 26 / 26