Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Recognizing Human Activity from Sensor Data
Document related concepts
Ecological interface design wikipedia , lookup
Speech-generating device wikipedia , lookup
Human–computer interaction wikipedia , lookup
Pattern recognition wikipedia , lookup
Neural modeling fields wikipedia , lookup
Concept learning wikipedia , lookup
Machine learning wikipedia , lookup
Transcript
Recognizing Human Activity from Sensor Data Henry Kautz University of Washington Computer Science & Engineering graduate students: Don Patterson, Lin Liao CSE faculty: Dieter Fox, Gaetano Borriello UW School of Medicine: Kurt Johnson Intel Research: Matthai Philipose, Tanzeem Choudhury Converging Trends… Pervasive sensing infrastructure GPS enabled phones RFID tags on all consumer products Wireless motes Breakthroughs in core artificial intelligence After “AI boom” fizzled, basic science went on… Advances in algorithms for probabilistic reasoning and machine learning Bayesian networks Stochastic sampling Last decade: 10 variables 1,000,000 variables Healthcare crisis Epidemic of Alzheimer’s Disease Deinstitutionalization of the cognitively disabled Nationwide shortage of caretaking professionals ...An Opportunity Develop technology to Support independent living by people with cognitive disabilities At home At work Throughout the community Improve health care Long term monitoring of activities of daily living (ADL’s) Intervention before a health crisis The University of Washington Assisted Cognition Project Synthesis of work in Ubiquitous computing Artificial intelligence Human-computer interaction ACCESS Support use of public transit CARE ADL monitoring and assistance This Talk Building models of everyday plans and goals From sensor data By mining textual description By engineering commonsense knowledge Tracking and predicting a user’s behavior Noisy and incomplete sensor data Recognizing user errors First steps toward proactive assistive technology ACCESS Assisted Cognition in Community, Employment, & Support Settings Supported by The National Institute on Disability & Rehabilitation Research (NIDDR) The National Science Foundation (NSF) Learning & Reasoning About Transportation Routines Task Given a data stream from a wearable GPS unit... Infer the user’s location and mode of transportation (foot, car, bus, bike, ...) Predict where user will go Detect novel behavior User errors? Opportunities for learning? Why Inference Is Not Trivial People don’t have wheels Systematic GPS error We are not in the woods Dead and semi-dead zones Lots of multi-path propagation Inside of vehicles Inside of buildings Not just location tracking Mode, Prediction, Novelty GPS Receivers We Used GeoStats wearable GPS logger Nokia 6600 Java Cell Phone with Bluetooth GPS unit Geographic Information Systems Street map Data source: Census 2000 Tiger/line data Bus routes and bus stops Data source: Metro GIS Architecture Learning Engine Goals Paths Modes Errors GIS Database Inference Engine Probabilistic Reasoning Graphical model: Dynamic Bayesian network Inference engine: Rao-Blackwellised particle filters Learning engine: Expectation-Maximization (EM) algorithm Graphical Model (Version 1) Transportation Mode Velocity Location Block Position along block At bus stop, parking lot, ...? GPS Offset Error GPS signal Rao-Blackwellised Particle Filtering Inference: estimate current state distribution given all past readings Particle filtering Evolve approximation to state distribution using samples (particles) Supports multi-modal distributions Supports discrete variables (e.g.: mode) Rao-Blackwellisation Each particle includes a Kalman filter to represent distribution over positions Improved accuracy with fewer particles Tracking blue = foot green = bus red = car Learning User model = DBN parameters Transitions between blocks Transitions between modes Learning: Monte-Carlo EM Unlabeled data 30 days of one user, logged at 2 second intervals (when outdoors) 3-fold cross validation Results Model Mode Prediction Accuracy Decision Tree (supervised) 55% Prior w/o bus info 60% Prior with bus info 78% Learned 84% Probability of correctly predicting the future Prediction Accuracy How can we improve predictive power? City Blocks Transportation Routines Goals A B Work work, home, friends, restaurant, doctor’s, ... Trip segments Home to Bus stop A on Foot Bus stop A to Bus stop B on Bus Bus stop B to workplace on Foot “Learning & Inferring Transportation Routines”, Lin Liao, Dieter Fox, & Henry Kautz, AAAI-2004 Best Paper Award Hierarchical Model gk-1 gk Goal tk-1 tk Trip segment mk-1 mk Transportation mode xk-1 xk x=<Location, Velocity> zk-1 zk GPS reading Hierarchical Learning Learn flat model Infer goals Locations where user is often motionless Infer trip segment begin / end points Locations with high mode transition probability Infer trips segments High-probability single-mode block transition sequences between segment begin / end points Perform hierarchical EM learning Inferring Goals Inferring Trip Segments Going to work Going home Correct goal and route predicted 100 blocks away Novelty & Error Detection Approach: model-selection Run several trackers in parallel Tracker 1: learned hierarchical model Tracker 2: untrained flat model Tracker 3: learned model with clamped final goal Estimate the likelihood of each tracker given the observations Detect User Errors Untrained Trained Instantiated Application: Opportunity Knocks Demonstration (by Don Patterson) at AAHA Future of Aging Services, Washington, DC, March, 2004 CARE Cognitive Assistance in Real-world Environments supported by the Intel Research Council Learning & Inferring Activities of Daily Living Research Hypothesis Observation: activities of daily living involve the manipulation of many physical objects Cooking, cleaning, eating, personal hygiene, exercise, hobbies, ... Hypothesis: can recognize activities from a time-sequence of object “touches” Such models are robust and easily learned or engineered Sensing Object Manipulation RFID: Radiofrequency ID tags Small Semi-passive Durable Cheap Where Can We Put Tags? How Can We Sense Them? coming... wall-mounted “sparkle reader” Example Data Stream Making Tea Building Models Core ADL’s amenable to classic knowledge engineering Open-ended, fine-grained models: infer from natural language texts? Perkowitz et al., “Mining Models of Human Activities from the Web”, WWW-2004 Experimental Setup Hand-built library of 14 ADL’s 17 test subjects Each asked to perform 12 of the ADL’s Data not segmented No training on individual test subjects Activity Prior Work CARE Accuracy/Recall Personal Appearance 92/92 Oral Hygiene 70/78 Toileting 73/73 Washing up 95/84 100/33 Appliance Use 100/75 Use of Heating 84/78 Care of clothes and linen 100/73 Making a snack 100/78 Making a drink 75/60 Use of phone 64/64 Leisure Activity 100/79 Infant Care 100/58 Medication Taking 100/93 Housework 100/82 General Solution Quantitative Results Point Solution Quantitative Results General Solution Anecdotal Results Point Solution Anecdotal Results Pervasive Computing, Oct-Dec 2004 Current Directions Affective & physiological state agitated, calm, attentive, ... hungry, tired, dizzy, ... Interactions between people Human Social Dynamics Principled human-computer interaction Decision-theoretic control of interventions Why Now? A goal of much work of AI in the 1970’s was to create programs that could understand the narrative of ordinary human experience This area pretty much disappeared Missing probabilistic tools Systems not able to experience world Lacked focus – “understand” to what end? Today: tools, grounding, motivation Challenge to Nanotechnology Community Current sensors detect physical or physiological state: user mental state must be indirectly inferred To what can extend can nanotechnology afford direct access to a person’s emotions and intentions?
