Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Re-thinking Data Management for Storage-Centric Sensor Networks Deepak Ganesan University of Massachusetts Amherst With: Yanlei Diao, Gaurav Mathur, Prashant Shenoy UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science Sensor Network Data Management Live Data Management: Queries on current or recent data. Applications: Real-time feeds/queries: Weather, Fire, Volcano Detection and Notification: Intruder, Vehicle Techniques: Push-down Filters/Triggers: TinyDB, Cougar, Diffusion, … Acquisitional Query Processing: BBQ, PRESTO, … Archival Data Management: Querying or Mining of past data Applications: Scientific Analysis of past events: Weather, Seismic, … Historical trends: Traffic analysis, habitat monitoring Our focus is on designing an efficient archival data management architecture for sensor networks UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 2 Archival Querying in Sensor Networks Data Gathering with centralized archival query processing Internet DBMS Gateway Lossless aggregation Efficient for low rate, small volume sensors such as weather sensors (temp, humidity, …). Inefficient energy-wise for “rich” sensor data (acoustic, video, highrate vibration). UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 3 Archival Querying in Sensor Networks Store data locally at sensors and push queries into the sensor network Internet Flash memory energyefficiency, cost, capacity. Gateway Push query to sensors Limited capabilities of sensor platforms. Flash Memory Acoustic stream Image stream UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 4 Technology Trends in Storage Energy Cost (uJ/byte) CC1000 Communication Atmel NOR CC2420 Storage Telos STM NOR Micron NAND 128MB Generation of Sensor Platform UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 5 Outline Case for Storage-centric Sensor Networks Challenges in a Storage-centric Sensor Database StonesDB Architecture Local Database Architecture Distributed Database Architecture Conclusion UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 6 Optimize for Flash and RAM Constraints Memory Flash Memory Constraints Data cannot be over-written, only erased Pages can often only be erased in blocks (16-64KB) Unlike magnetic disks, cannot modify in-place Challenges: Memory: Minimize use of memory for flash database. Energy: Organize data on flash to minimize read/write/erase operations Aging: Need to efficiently delete old data items when storage is insufficient. ~4-10 KB 2. Modify in-memory 1. 1. Load block 2. Into Memory 3. Save block back Erase block ~16-64 KB UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 7 Support Rich Archival Querying Capability SQL-style Queries: Min, max, count, average, median, top-k, contour, track, etc Wireless Sensor Network Classification Queries: What type of vehicles (truck, car, tank, …) were observed in the field in the last month? Similarity Search: Was a bird matching signature S observed last week? Signal Processing: Perform an FFT to find the mode of vibration signal between time <t1,t2>? UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 8 StonesDB Goals Our goal is to design a distributed sensor database for archival data management that: Supports energy-efficient sensor data storage, indexing, and aging by optimizing for flash memories. Supports energy-efficient processing of SQL-type queries, as well as data mining and search queries. Is configurable to heterogeneous sensor platforms with different memory and processing constraints. UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 9 StonesDB Architecture UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 10 Example: Indexing in StonesDB Naïve Design: Consider a value-based index on entire stream Deletion/Aging of data triggers in-place updates involving energy-intensive block read/write/erase operations. UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 11 Indexed Storage StonesDB Design: Split data stream into partitions and build index on each partition. Age partitions as a whole cheaply. Flash Block UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 12 Outline Case for Storage-centric Sensor Networks Challenges in a Storage-centric Sensor Database StonesDB Architecture Local Database Architecture Distributed Database Architecture Conclusion UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 13 StonesDB: Data Mining Queries Similarity Search: Was a bird matching signature S observed last week? Proxy Cache of Image Summaries UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 14 StonesDB: System Operation Similarity Search: Was a bird matching signature S observed last week? Query Engine Partitioned Access Methods UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 15 Research Issues Local Database Layer Impact of RAM limitations on storage organization Energy-optimized indexing and aging. New cost models for self-tuning energy-efficient sensor databases. Distributed Database Layer Intelligent split of query processing between proxy and sensor tiers Adaptively tuning quality of data cached at sensor proxy based on query needs UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 16 The End STONES: STOrage-centric Networked Embedded Systems http://sensors.cs.umass.edu/projects/stones UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science Sensor Data Management Taxonomy Querying Mining Timeline vs Prior Knowledge Search/Mining on Archived Sensor Data Acquisitional Query Processing (BBQ, …) e of data processing Pushdown Filters (TinyDB, Cougar, …) Current Recent Past Timeline of data being processed UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 18 Technology Trends in Sensor Platforms Cyclops Camera+ Mica2 Mote 128 x 128 resolution images 4 KB RAM, 10 MHz microcontroller OmniVision Camera + iMote2 128 x 128 resolution images 64KB - 32MB RAM, 10 MHz microcontroller Spectrum of sensing devices with different power, capability, resource constraints. UNIVERSITY OF MASSACHUSETTS, AMHERST • Department of Computer Science 19