Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
University of Milan ORESTEIA -MODULAR HYBRID ARTEFACTS WITH ADAPTIVE FUNCTIONALITY http://www.image.ntua.gr/oresteia/ PARTNERS Project Technical Committee Technical Manager: John Taylor (KCL) ALTEC SA Sotiris Pavlopoulos, George Malinescos IVML-NTUA Stefanos Kollias, Nicolas Tsapatsoulis University of Milan Bruno Apolloni, Dario Malchiodi Kings College of London Stathis Kasderidis Imperial College Eric Yeatman, Tim Green Contents Executive Summary Problems to be Solved – Attention – Data Fusion – Emergence Architecture and partners' tasks – Diagram – Module Explanation Feature Extraction through Signal Modelling Methods for State mappings – CAM/SPM – PAC Meditation / Fuzzy Relaxation Demo 1 Demo 2 Data Collection – Laboratory Studies – Car Driving Simulator – Linguistic Rules MicroPower Generation – Wireless Communication Executive Summary Scope/Aims Create a guidance system for humans, for more efficient and less hazardous living and interacting with their environment, through a set of decision-making facilities embedded in the environment and suitably adapted to the particular user Investigate enabling technologies for DC in the form of energy harvesting and low power wireless communications Inputs Low level sensorial data – Physiological class of sensors – Environmental class of sensors – Other Symbolic knowledge, a priori available (linguistic rules) Subsymbolic knowledge, constructed based on numerical data (Input/Output pairs) Attention-based functionality, inspired from brain operation Outputs Decisions Actions on behalf of the user for: – Managing repetitive and trivial jobs, – Providing indication of abnormal user activity and state, – Providing planning facilities, – Providing information filtering facilities, Maintaining good user state (physiological, psychological, etc) Key Properties Autonomy Responsiveness Robustness Approach Develop a multi-level attention-based agent architecture adapted to solve decision /guidance problems arising from sensors of various types, some worn by humans, others in devices (such as cars) being used by the humans. The decision/ guidance response of an agent is as to what is the state of the human, given the sensor data, or what is optimal continued use of the device on the basis of joint sensor data arriving at the agent for a given user from all sources Develop multi-agent systems that handle data available, also, from a set of agents (from interacting users), providing for decision/guidance as to overall optimal (best) use, and ranking of the users as to which needs further analysis Problems to be solved Attention Data Fusion Emergence Problems to be solved: Attention I shouldn’t produce these outputs, with these inputs… • Self-evaluation error • Irregular inputs • Hardware failure Artefact The input signals have irregular patterns… Self Evaluator Attention Controller INPUTS Attention Signal Data History My batter y is low! OUTPUTS Problems to be solved: Data Fusion You can make use of the two bottom Artefacts Which of these data shall I need? “WORLD MODEL” Artefact data Artefact data data data data data data data data data data Artefact “Clever Space” data data Intelligent Artefact data data data Artefact data data data data AGENT data Artefact Problems to be solved: Emergence “Clever Space” How much are you willing to pay for the services? WHY NOT Artefact BE MINE? Artefact I need to use these! Artefact Artefact AGENT Artefact AGENT Artefact Artefact THESE ARE MINE! Artefact Artefact And I need to use these! AGENT HEY,I SAW THEM FIRST! Artefact I’ll take these! Artefact Architecture of a single Agent IMC: Inverse Attention/modulation Controller Response/ Decision Monitor IMC Rules Action Reinforcement Fused Goals Level 4 Fuzzy/ Statistical Response/ Decision UNIMODAL Monitor IMC Rules Action Level 3 Goals Historic Mean State ANN Hourglass NeuroFuzzy Reinforcement Histogram Analysis Level 2 Data Smoothing Sequential Data Level 1 Sequential Data Sensors Sequential Data Overall View ANN Fused State Other User/ Environment Symbolic Inputs USER Database Historic Mean State Reinforcement Level 1: Sensors Data Content: Classes of signals used by higher levels (Level 2-4) – Data collection (KCL-QUB, ALTEC) – Synthetic data generators (NTUA, UM, KCL) Sensor autonomy – Efficient energy harvesting (ICSTM) Communication links – Low power consumption (ICSTM) Level 2: Preprocessing Signal preprocessing (NTUA, KCL) – Noise Reduction – Buffering – Transforms Feature extraction – Which features? (ALTEC, KCL-QUB) – How? (NTUA, KCL) Modeling signals – Extraction of hidden parameters (UM) Architecture of a single Agent Level 3: Domain Experts – State Representation State Mappings ANN Hourglass – Subsymbolic state representation (UM, NTUA) Neurofuzzy – Symbolic state representation (NTUA, UM) Action Module Stores the ‘response’ of the system. Three levels of sophistication: – No real action. – Simple suggestive actions/messages. – Simple action sequences. Responsible Partners: KCL, NTUA, UM Rules Module Consists of three components: – World Model. Contains all the information needed for forming useful functionalities and maintaining a set of artefacts. – Autonomic. Maintains rules that are necessary for the robust run-time behaviour of the system. – Other. Aids the implementation of alternative (to the ones implemented in the State module) decision-making systems. Responsible Partners: KCL, NTUA, UM Goals Module This module includes three parts: – Values. Is closely associated with the World Model (in the Rules module) to provide default (universal) values for the various thresholds and triggers present in the architecture. – User Profile. Provides specific user deltas (i.e. deviations from the default values defined in the Values part). – Services. Includes a catalogue of services that are offered by the artefact. Responsible Partners: KCL, NTUA, UM Monitor Module Creates an error signal level after comparing the current State with a Historic State. It fulfils two basic requirements: – Universal definition of an error function. Independent of the output of State Module (UM, NTUA, KCL) – Standard definition of an error function. Context sensitive, seamless knowledge of state representation (UM, NTUA, KCL) Attention Controller This module is inspired from motor control systems in the brain as well as from engineering control ideas. It operates in two modes: – Feedforward mode. The controller sends a signal, governed directly by the Goal module, to produce a desired response from the action module (KCL) – Feedback mode. Feedback information from the Monitor module is used as a feedback component (KCL) Level 4: Agent Construction Combination of conceptual blocks Agent Formation Data Fusion Overall system training Reinforcement signal production/handling Attention Control Responsible partner: KCL Feature Extraction through Signal Modelling HYBRID TRAINING DYNAMICS E(w) = e(y1,..., yT) yVI 5 yV 11y IV 25 y III 34 y II 20 y I 24 y 0 Jy 1 (w ) T = y t e(y 1 ,..., y T)Jy t (w ) ... t =1 Jy T(w ) Jz t (w ) =JF(z t-1 ,v t-1 ;w ,t) Integrated by a fourth order Runge-Kutta method Jz t (w ) =JF(z t-1 ,v t-1 ;w ,t) Jz t-1 (w ) = Jz t-1F(z t-1 ,v t-1 ;w ,t-1)Jz t-1 (w ) + Jw F(z t-1 ,v t-1 ;w ,t-1) 0 I Jz t-1 (w ) = Jz t-1F(z t-1 ,v t-1 ;w ,t-1)Jz t-1 (w ) + Jw F(z t-1 ,v t-1 ;w ,t-1) 0 I SIGNALS FROM BODY 8 6 4 CH1 2 CH4 CH5 CH6 95,35 90,05 84,75 79,45 74,15 68,85 63,55 58,25 52,95 47,65 42,35 37,05 31,75 26,45 21,15 15,85 10,55 sec -2 5,25 0 -4 -6 0.4 0.4 0.2 0.2 20 40 60 80 100 120 0.4 ECG 20 140 40 60 0.2 80 100 120 20 140 -0.2 -0.2 -0.2 -0.4 -0.4 -0.4 -0.6 -0.6 -0.6 Symbolic health diagnosys 40 60 80 100 120 140 Methods for State Mappings: CAM/SPM Module Overall View output SYMBOLIC PROCESSING MODULE evaluated symbolic predicates CONNECTIONIST ASSOCIATION MODULE features CAM Module Scope The Connectionist Association Module (CAM) provides the system with the ability of grounding the symbolic predicates Using the CAM, the set of features is associated with the set of evaluated symbolic predicates (partitioning the input space) Why Neural Network? Generally the internal state defined by the neural network output is not so simple to be considered as a simple fuzzy partitioning; Instead the neural network performs the appropriate data clustering to provide the evaluation of the required symbolic predicates based on numerical data Diastolic Pressure Example Attention Signal Handling To which input elements have to be concentrated on? SPM Module Scope It implements a semantically rich reasoning process. It takes as inputs a set of features and gives a set of recognised situations. It performs the conceptual reasoning process that finally results to the degree of which the output situations are recognised Why Neurofuzzy? Fuzzy relational systems represent symbolic knowledge in a formal, numerical framework. On the other hand, neural networks are typical learning systems that work in a numeric framework. Rule Insertion Rules describing situations are based on linguistic terms and are generally of the form If fuzzy_predicate(1) and fuzzy_predicate(2) then output(3)” Each rule consists of an antecedent (the if part of the rule) and a consequence (the then part of the rule) Rule Insertion Rule1 The antecedent part of the rule is used to create the weight matrix of the first layer The consequence part of the rule is used to create the rule matrix of the second layer The antecedent of all the rules existed is the set of the fuzzy predicates describing the system The consequence of all the rules is the set of the recognised situations of the system Predicates Activ ation Rule2 1 0.8 In1 0 Predicates Activ ation 0 0 Rule3 0 0 Predicates Activ ation Rule4 1 Layer1Out Predicates Activ ation Rule5 Predicates Activ ation Rule6 Predicates Activ ation Rule Insertion (Example) Layer 1 Layer 2 Methods for State Mappings: PAC Meditation / Fuzzy Relaxation PAC Meditation mapping formula fitness ok? formula n fitness y fuzzy relaxation formula ok? y prejudice n final formula end PAC Meditation 0-level inner border 0-level outer border 1-level inner border 1-level outer border Fuzzy relaxation (dk) m dk m dk - d 1 2 3=s d k d d m dk m dk 1 d d s i m (d k ) k 1 m m i1 i1 m O f , 1 Li 2 i 3 i 4 0 i1 SA Algorithm CurrentState := InitialState CurrentTemperature := InitialTemperature Repeat GetTemperature(CoolingSchedule) ProposedState := SelectNeighborState ProposedCost := EvaluateCost(ProposedState) If (Accepted(ProposedState, ProposedCost)) Then CurrentState := ProposedState Until StoppingRule Return(CurrentState) d k d d k DEMO 1: Health Monitoring DEMO 2: Car Hazard Avoidance Description This demo includes the ability to generate a set of events in the environment, driver, or car. Environmental effects include shocks in the visibility (fog) and the temperature. In this category also the appearance of ice patches in road segments and existence of other cars in the same or opposite lane is included as well Aims To validate the ORESTEIA Architecture To test the integration of the work of the partners To offer another context for testing the adequacy of the State mapping methods To search for the existence of common design principles in the various contexts Data Collection Laboratory Studies Car Driving Simulator Linguistic Rules Data Collection: Laboratory Studies Aim Experiment design for collecting ECG, Respiration Rate (RSP), Galvanic Skin Response (GSR), and Skin Temperature (SKT) in laboratory conditions for abnormal physiological and psychological state prediction ECG Signal RSP Signal GSR Signal SKT Signal Data Collection: Car Driving Simulator Aim Experiment design for collecting ECG, Respiration Rate (RSP), Galvanic Skin Response (GSR), and Skin Temperature (SKT) while driving a car simulator (for abnormal physiological and psychological state prediction) Car Driving Simulator Data Collection: Linguistic Rules Physiological Features The system takes as inputs a set of medical features and gives a set of recognised situations. The input features are the values of RSP, BT, HR, PS, PD, and the derived features from the ECG Inputs RSP: Respiration BT: Body Temperature HR: Heart Rate PS: Systolic Blood Pressure PD: Diastolic Blood Pressure ECG: Electrocardiogram Outputs Normal: some features are not perfect Warning: stop and rest until you get normal Urgent: stop and call medical centre Emergency: very urgent Definition of Symbolic Predicates The term predicate refers to the partition of the input features Predicates are characterised as Very Low (VL), Low (L), Medium Low (ML), Medium High (MH) High (H), and Very High (VH) Some features are characterised as Normal (N) and Abnormal (A) Rule Extraction Rules describing situations are based on linguistic terms and are generally of the form “if predicate(1) and predicate(2) then output(3)” In order to detect the recognised situations, we must first define the rules that describe these situations A Subset of the Rules Nr Rule Output 1 HR(VH) + ECG-5(A) + BT(VH) Urgent 2 PS(VH) + HR(VH) + ECG-5(A) + BT(VH) Emergency 3 ECG-6(A) + PS(VL) Emergency 4 PS(L) + HR(H) + BT(H) Warning 5 HR(M) + BT(M) + PS(M) + RSP(M) + ECG-5(N) Normal 6 HR(L) + BT(L) + PS(M) + RSP(M) + ECG-5(N) Normal 7 PS(M) + HR(L) + RSP(M) + ECG-1(H) Warning MicroPower Generation and Wireless Communication Aims Development of artefacts that – Are autonomous – Have a long lifetime without maintenance Development of sources that can scavenge energy from the local environment Electromechanical energy conversion using MEMS Analysis of micro-power generator topologies Detailed investigation of the operation of various different inertial generator topologies The parametric generator is optimal when the input movement is greater than the dimensions of the device by a factor of ~10 or more Fabrication and test of an initial prototype generator Cross-section of prototype generator Photograph of fabricated device Fabrication and test of an initial prototype generator Experimental setup for charge transfer experiments amplifier output voltage (V) Typical discharge transient 0.5 0 -0.5 -1 -1.5 -2 15 17 19 21 time (microseconds) 23 25 Analysis of wireless communication schemes 100 Power Transfer Comparison /dB Comparison of nearfield transmission (at 50 MHz) with far-field transmission (at 470 MHz) for a distance of 50cm, varying the near and far-field antenna dimensions (shortened to NF and FF respectively on the axis labels) 50 0 -50 0.08 -100 0.06 -150 0.08 0.04 0.07 0.06 0.02 0.05 0.04 0.03 FF Antenna Radius/m 0.02 0.01 0 0 NF Radius/m