Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Functional Reactive Programming, Resource Types, and Wormholes Paul Hudak Yale University Department of Computer Science haskell.cs.yale.edu (Joint work with Dan Winograd-Cort) New York Haskell Users Group October 30, 2013 Background • Time-varying quantities in a PL is due to Conal Elliott (early work in C++ for animation) • Fran: Functional Reactive Animation, by Elliott and Hudak: FRP implemented in Haskell, showing elegance of higherorder functions, type classes, and so on. • Yampa: at Yale; an arrow-based FRP in Haskell; avoided insidious time- and space leaks; improved modularity. • Since then, tons of new papers, both theoretical and applied: FranTk, Frob, FVision, Reactive Banana, FPorter, … • Antony Courtney: Fruit – a GUI based on Yampa • Today will talk about a MUI (musical user interface) implemented in Euterpea (Haskell library for computer music). Euterpea: Computer Music in Haskell Why do this? After all, there are already many languages for computer music… Motivation: • A required two-course computer-music sequence in the Computing and the Arts major at Yale • The use of modern, state-of-the-art PL ideas in a fun area • To combine my research with my hobby • To use music to teach (functional) programming • To give composers powerful tools to enhance creativity Last term, out of 20 students, 3 were Computing and the Arts majors, one was a graduate student, and the rest were undergrads {mostly CS majors) with an interest in computer music, or a desire to learn Haskell. Euterpea “Euterpea” is derived from “Euterpe”, who was one of the nine Greek muses, or goddesses of the arts, specifically the muse of music. Signals and Signal Functions • Signals are time-varying quantities. • Conceptually they can be thought of as functions of time: Signal a = Time → a • For example: – a slider is a time-varying number – a mouse is a time-varying cartesian coordinate • For efficiency and modularity, we use arrows to abstract away from signals, and instead use signal functions. Conceptually: SigFun a b = Signal a -> Signal b • Haskell’s Arrow type class make this especially easy. Key Idea • This signal processing diagram: y sigfun x • is equivalent to this Euterpean code, using “arrow” syntax: y <- sigfun -< x • In turn this is part of a larger syntactic unit, called a proc. E.g.: proc x -> do y <- sf1 <- x z <- sf2 <- y+1 return z*2 • By analogy to an anonymous function: \ x -> let y = f1 x z = f2 (y+1) in z*2 Four Useful Functions • To “lift” a function to a signal function: arr :: (a → b) → SF a b – Example: arr (+1) is a signal function that adds one to its input • To “lift” a constant to a constant signal function: constA :: b → SF () b – Example: constA 440 returns a steady value of 440 • To “compose” to signal functions: (>>>) :: SF a b → SF b c → SF a c (<<<) :: SF b c → SF a b → SF a c – Example: constA 440 >>> arr (+1) is the same as constA 441 Audio Example: Sine Wave • From basic trigonometry: sin(nωh) = 2 cos(ωh) sin((n-1) ωh) – sin((n-2) ωh) where ω = 2πf • We can derive a recurrence equation [Goertzel]: y(0) = 0 y(1) = sin ωh y(n) = c · y(n-1) - y(n-2) • Diagrammatically: where c = 2 (cos ωh) init: sin ωh z-1 init: 0 d1 z-1 *c d2 - y Rendered in FRP init: sin ωh y(0) = 0 y(1) = sin ωh y(n) = c · y(n-1) - y(n-2) where c = 2 (cos ωh) z-1 init: 0 d1 z-1 *c sine :: Double -> SF () Double sine freq = let omh = 2*pi*freq/sr d = sin omh c = 2 * cos omh in proc _ -> do rec let y = c*d1-d2 d2 <- delay 0 -< d1 d1 <- delay d -< y returnA -< y d2 - y Physical Model of a Flute sinA lineSeg 5 lineSeg envibr env1 * 0.1 rand 1 × flow vibr + * breath emb + Embouchure delay sum1 delayt (1/fqc/2) x x – x3 + lowpass out * amp × returnA env2 * feedbk2 * feedbk1 Flute bore delay flute delayt (1/fqc) bore lineSeg Flute Model in Euterpea flute dur freq amp vfreq = in proc () -> do amp1 <- linseg … -< () amp2 <- linseg … -< () ampv <- linseg … -< () flow <- rand 1 -< amp1 vibr <- oscils vfreq -< 0.1 * ampv rec let feedbk = body * 0.4 body <- delay (1/freq) -< out x <- delay (1/freq/2) -< breath*flow + amp1 + vibr + feedbk out <- tone -< (x - x*x*x + feedbk, 2000) returnA -< out*amp*amp2 Flute Demo • f0 and f1 demonstrate the change in the breath parameter. f0 = flute 3 0.35 440 0.93 0.02 f1 = flute 3 0.35 440 0.83 0.05 • f2 has a weak pressure input so only plays the blowing noise. f2 = flute 3 0.35 440 0.53 0.04 • f3 takes in a gradually increasing pressure signal. f3 = flute 4 0.35 440 (lineSeg [0.53, 0.93, 0.93] [2, 2]) 0.03 • Sequence of notes: Events • Signals are not enough… some things happen discretely. • Events can be realized as a kind of signal: data Maybe a = Nothing | Just a type SEvent a = Maybe a -- pre-defined in Haskell • For example, MIDI events have type: SEvent [MidiMessage] where MidiMessage encodes standard MIDI messages such as Note-On, Note-Off, etc. • And a signal function over events would have type: SF (SEvent T1) (SEvent T2) This overall paradigm is often called Functional Reactive Programming (FRP) Musical User Interface (MUI) • Design philosophy: – GUI’s are important! – The dataflow metaphor (“wiring together components”) is powerful! – Yet graphical user interface programming is inefficient… • Goal: MUI widgets using FRP • The MUI arrow is called UISF – UISF a b maps time-varying values (or event streams) of type a, to those of type b • Kinds of UISF signal functions: – – – – Widget constructors Widget transformers I/O Mediators Widget Constructors • • • • • • • display textbox button checkbox radio hSlider , vSlider hiSlider , viSlider :: Show a ⇒ UISF a () :: String → UISF String String :: String → UISF () Bool :: String → Bool → UISF () Bool :: [String ] → Int → UISF () Int :: RealFrac a ⇒ (a, a) → a → UISF () a :: Integral a ⇒ a → (a, a) → a → UISF () a (Run mui0 from Examples/MUI.hs) MUI Transformers • • • • title :: String → UISF a b → UISF a b withDisplay :: Show b ⇒ UISF a b → UISF a b pad :: (Int, Int , Int, Int ) → UISF a b → UISF a b topDown, bottomUp, leftRight , rightLeft :: UISF a b → UISF a b • setLayout :: Layout → UISF a b → UISF a b • makeLayout :: LayoutType → LayoutType → Layout • data LayoutType = Stretchy { minSize :: Int } | Fixed { fixedSize :: Int } (Run mui2 from Examples/MUI.hs) Mediators • unique :: Eq a ⇒ UISF a (Event a) -- generates an event whenever the input changes • edge :: UISF Bool (Event ()) -- generates an event when input changes from False to True • accum :: a → UISF (Event (a → a)) a -- accum x starts with the value x , but then applies the function -- attached to the first event to x to get the next value, and so on. • mergeE :: (a → a → a) → Event a → Event a → Event a -- mergeE f e1 e2 merges two event streams, using f to resolve -- simultaneous events • hold :: b → UISF (Event b) b -- hold x begins as value x , but changes to the subsequent values -- attached to each of its input events • now :: UISF () (Event ()) -- creates a single event “right now” (Run mui3, mui4 from Examples/MUI.hs) Time and Timers • time :: UISF () Time -- Returns current time -- Time is a type synonym for Double • timer :: UISF (Time, Double) (SEvent ()) -- Emits events at given (dynamic) rate (Run mui6 from Examples/MUI.hs) Bifurcate Me, Baby! Gary Lee Nelson 1995 Behind the Scene • Consider the recurrence equation: xn+1 = r * xn * (1 - xn) Start with an initial population x0 and iteratively apply the growth function to it, where r is the growth rate. For certain values of r, the population stabilizes, but as r increases, the period doubles, quadruples, and eventually leads to chaos. It is one of the classic examples in chaos theory. • In Euterpea: grow r x = r * x * (1-x) • Let’s wrap it up in a MUI. (Run bifurcate from Examples/MUI.hs) Modular? A program is a single signal function; it must thread all IO actions through the same input and output. MIDI synthesizer electric piano game controller run-time system main program (a signal function) mouse sound card console I/O MIDI instrument printer Maybe Not… I/O bottleneck; no transparency MIDI synthesizer I/O bottleneck; no transparency electric piano game controller run-time system main program (a signal function) mouse sound card console I/O MIDI instrument printer Solution: Reify Real-World Objects Expand main program to subsume virtualized IO devices! MIDI synthesizer electric piano game controller run-time system main program (a signal function) mouse sound card console I/O MIDI instrument printer IO devices now treated just like other signal functions. As we have seen, can also virtualize virtual objects (widgets). main program MIDI synthesizer electric piano game controller sf1 sf2 sf3 sf4 mouse sf5 sound card sf6 sf7 console I/O MIDI instrument printer A Problem: Resource Duplication • Consider this code fragment: () <- midiOut <- noteList1 () <- midiOut <- noteList2 is an output device, but there are two occurrences -- what is the result? Interleaving? Non-determinism? • Likewise, here is an example of input: midiSynth notes1 <- midiIn <- () notes2 <- midiIn <- () is an input device. So do notes1 and notes2 return the same result, or are they different? midiIn Solution: Resource Types • Tag each virtualized object with a unique resource type to prevent duplication. midiOut :: SF (S MidiOut) MidiMsg () midiIn :: SF (S MidiIn) () MidiMsg • The first argument to SF is a set of resource types; S MidiOut and S MidiIn are singleton sets. • With these types, the previous code fragments will not type-check – resource types of composed signal functions must be disjoint. Resource Types • A value of type SF r a b maps signals of type a into signals of type b, while “consuming” the set of resources r. • If sf1 :: SF r1 a b and sf2 :: SF r2 b c, then in sf1 >>> sf2, r1 and r2 must be disjoint. • In which case sf1 >>> sf2 :: SF (r1 U r2) a c. • A restricted form of resource types can be implemented in Haskell using higher-order types, type families, etc. • The full version requires type features that don’t exist. Effects in FRP • How can we perform effects in the FRP framework? • Outside of FRP, we use monads: – IO is useful for general I/O effects such as on realworld devices, but because we cannot escape it, it is suboptimal for localized effects. – ST is useful for mutable data structures, as it is escapable after its localized effects: runST :: (forall s. ST s a) -> a The “phantom” type s provides the safety. Performing Effects • IO and ST are inherently different – IO is for real-world objects that cannot be created on the fly (e.g. printer, speaker, keyboard, etc.). – ST is for mutable data structures that can be allocated dynamically. • Both types of effects require sequencing, which the monadic structure provides. Sequencing • How can we do sequencing in FRP without monads? – Normal Haskell variables are time-invariant, but FRP values are conceptually time-varying. – Thus, the sequencing provided by monads can be achieved in FRP by using the ordering of events in an event stream. • Indeed, we have seen this already with midiIn and midiOut, using resource types to ensure safety. • But what about mutable data types? Mutable Datatypes in FRP • An example of a mutable data structure: – Mutable array can be represented by a signal functions that handle mutation requests: (Event Request) sfArray (Event Response) • Each sfArray will produce a new mutable array: – We do not need resource types. – We do not need a phantom type. Mutable Datatype in FRP • An even simpler mutable data structure example would be a single cell of memory. – The input would be the new value to store and the output, the previous value in the cell: init – Essentially, this acts like a delay function. We call it init in reference to our previous work on Causal Commutative Arrows. Splitting an Atom • Let’s examine init more closely: init • What if we split the reading and writing? Splitting an Atom • Let’s examine init more closely: blackhole whitehole • What if we split the reading and writing. • We call this a wormhole. – The blackhole is for writing – The whitehole is for reading Wormholes • We can use wormholes as non-local, oneway communication channels. SF 1 SF 2 Wormholes • We can use wormholes as non-local, oneway communication channels. SF 1 SF 2 Wormholes • We can use wormholes as non-local, oneway communication channels. whitehole SF 2 blackhole SF 1 Wormholes • We can use wormholes as non-local, oneway communication channels. 𝑆𝐹1 = proc () → do … 𝑑𝑎𝑡𝑎𝐹𝑟𝑜𝑚𝑆𝐹2 ← 𝑤ℎ𝑖𝑡𝑒ℎ𝑜𝑙𝑒 −≺ () … 𝑟𝑒𝑡𝑢𝑟𝑛𝐴 − ≺… 𝑆𝐹2 = proc () → do … __ ← 𝑏𝑙𝑎𝑐𝑘ℎ𝑜𝑙𝑒 −≺ 𝑙𝑜𝑐𝑎𝑙𝐷𝑎𝑡𝑎 … 𝑟𝑒𝑡𝑢𝑟𝑛𝐴 −≺ … “Bottles” • Composed and implemented by Donya Quick, a grad student in CS • Done entirely in Euterpea • Demonstrates: – High-level composition • Notes, repetitions, structure • Micro-tonal support – Low-level signal processing • Both “struck” bottle and “blown” bottle sounds, using physical modeling • Some additive synthesis as well • Reverb and audio processing Thank You!! (any questions?) For more information about Euterpea and our PL research, see: haskell.cs.yale.edu Implementing Signals • As “yet another abstraction”, arrows might seem to introduce yet more computational overhead. • But in fact arrows eliminate certain important classes of space leaks. • Furthermore, causal commutative arrows (CCA) can improve performance dramatically: – Any CCA expression can be normalized into an expression with one loop and one vector of state variables. – This results in a factor of 50 improvement over conventional implementation methods. – Good enough for non-trivial real-time performance.