* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download USC Brain Project Specific Aims
Optogenetics wikipedia , lookup
Lateralization of brain function wikipedia , lookup
Blood–brain barrier wikipedia , lookup
Craniometry wikipedia , lookup
Neural engineering wikipedia , lookup
Premovement neuronal activity wikipedia , lookup
Clinical neurochemistry wikipedia , lookup
Time perception wikipedia , lookup
Types of artificial neural networks wikipedia , lookup
History of anthropometry wikipedia , lookup
Haemodynamic response wikipedia , lookup
Neuroeconomics wikipedia , lookup
Human intelligence wikipedia , lookup
Activity-dependent plasticity wikipedia , lookup
Human brain wikipedia , lookup
Neuroinformatics wikipedia , lookup
Dual consciousness wikipedia , lookup
Aging brain wikipedia , lookup
Embodied language processing wikipedia , lookup
Brain morphometry wikipedia , lookup
Neural correlates of consciousness wikipedia , lookup
Neurolinguistics wikipedia , lookup
Intelligence wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Neuroesthetics wikipedia , lookup
Cognitive neuroscience wikipedia , lookup
Neurophilosophy wikipedia , lookup
Mirror neuron wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Brain Rules wikipedia , lookup
Impact of health on intelligence wikipedia , lookup
Neuroplasticity wikipedia , lookup
Selfish brain theory wikipedia , lookup
History of neuroimaging wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Philosophy of artificial intelligence wikipedia , lookup
Mind uploading wikipedia , lookup
Intelligence explosion wikipedia , lookup
Neuroanatomy wikipedia , lookup
Neuroscience and intelligence wikipedia , lookup
Artificial intelligence wikipedia , lookup
Neuropsychology wikipedia , lookup
Evolution of human intelligence wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Neural binding wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Nervous system network models wikipedia , lookup
Michael Arbib: CS564 - Brain Theory and Artificial Intelligence The Aims of the Course: We will use the challenge of understanding the mechanisms of visuomotor coordination and action recognition in the monkey brain to provide a structured set of goals for our mastery of Brain Theory: modeling interactions of components of the brain, especially more or less realistic biological neural networks localized in specific brain regions and Connectionism in both Artificial intelligence (AI) and Cognitive psychology: modeling artificial neural networks -- networks of trainable “quasi-neurons” -- to provide “parallel distributed models” of intelligence in humans, animals and machines Tools: Gaining an understanding of the use of NSLJ (Neural Simulation Language in Java) for simulating and analyzing neural networks Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 1 CS 564: Brain Theory and Artificial Intelligence URL: http://www-scf.usc.edu/~csci564/ for syllabus, instructor and TA information, handouts, homework and grades DEN URL: http://den.usc.edu/academic/fa2001/courses/csci564/csci564.asp Instructor: Michael Arbib; [email protected] (Office hours: 11-12 Tuesdays, HNB 03) TAs: Erhan Oztop, [email protected], Salvador Marmol, smarmol@rana This course provides a basic understanding of brain function, of artificial neural networks which provide tools for a new paradigm for adaptive parallel computation, and of the Neural Simulation Language NSLJ which allows us to simulate biological and artificial neural networks No background in neuroscience is required, nor is specific programming expertise, but knowledge of Java will enable students to extend the NSLJ functionality in interesting ways Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 2 Texts Texts: MA Arbib, 1989, The Metaphorical Brain 2: Neural Networks and Beyond, Wiley-Interscience TMB2 is being sent over to "The paper clip" for duplication. Students can purchase it there for $16 plus tax. A Weitzenfeld, MA Arbib and A Alexander, 2002, NSL Neural Simulation Language, MIT Press (in press)[ An old version is at http://www-hbp.usc.edu/_Documentation/NSL/ Book/TOC.htm. A new version will be posted in a week or two.] Other Required Reading: will be posted on the Website – starting with the Mirror Neuron Proposal Supplementary reading: MA Arbib, Ed, 1995, The Handbook of Brain Theory and Neural Networks, MIT Press (paperback) Michael A Arbib, and Jeffrey Grethe, Editors, 2001, Computing the Brain: A Guide to Neuroinformatics, San Diego: Academic Press (in press) Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 3 Grading One mid-term and a final will cover the entire contents of the readings as well as the lectures Students will be organized into 5 groups, each working together on a semester-long project The final exam will cover all of the course, but emphasizing material not covered in the mid-term Distribution of Grades: NSL assignments and other homework: 25%; Mid-term: 20%; Project 30%; Final Exam: 25% Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 4 5 Specific Aims for the Mirror Neuron Proposal 5 Specific Group Projects for CS 564 Development of the Mirror System: 1. Development of Grasp Specificity in F5 Motor and Canonical Neurons 2. Visual Feedback for Grasping: A Possible Precursor of the Mirror Property Recognition of Novel and Compound Actions and their Context: 3. The Pliers Experiment: Extending the Visual Vocabulary 4. Recognition of Compounds of Known Movements 5. From Action Recognition to Understanding: Context and Expectation Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 5 Send email to [email protected] by Noon on Tuesday September 4, 2001 Line 1: Your name then a colon (:) then your email address Line 2: Your Department (and your location if you are off-campus) Line 3: Any special skills you bring to the course Lines 4+: Your top 2 or 3 choices for a Project topic from the list: 1. Development of Grasp Specificity 2. Visual Feedback for Grasping 3. The Pliers Experiment 4. Recognition of Compound Movements 5. Context and Expectation Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 6 Michael Arbib:CS564 - Brain Theory and Artificial Intelligence Lecture 1 Introduction and Brain Overview Reading Assignments: TMB2: Chapter 1 Mirror Neuron Proposal* * Read this “lightly” the first week of class We will master it and much more as the class proceeds Your initial task is to review the 5 specific aims, and pick the 2 you find most interesting One of these will probably become your project goal for the semester Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 7 Opposition Spaces and Virtual Fingers The goal of a successful preshape, reach and grasp is to match the opposition axis defined by the virtual fingers of the hand with the opposition axis defined by an affordance of the object (Iberall and Arbib 1990) Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 8 Visual Control of Grasping in Macaque Monkey A key theme of visuomotor coordination: parietal affordances (AIP) drive frontal motor schemas (F5) F5 - grasp commands in premotor cortex Giacomo Rizzolatti AIP - grasp affordances in parietal cortex Hideo Sakata Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 9 Grasp Specificity in an F5 Neuron Precision pinch (top) Power grasp (bottom) (Data from Rizzolatti et al.) Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 10 FARS (Fagg-Arbib-Rizzolatti-Sakata) Model Overview AIP extracts affordances features of the object relevant to physical interaction with it. Prefrontal cortex provides “context” so F5 may select an appropriate affordance AIP AIP Dorsal Stream: dorsal/ventral Affordances streams Ways to grab this “thing” Constraints TTask ask Constraints (F 6) (F6) Working Working M Memory emory (46) (46?) Instruction Stimuli Instruction Stimuli (F2) (F2) PFC F5 F5 “It’s a mug” Ventral Stream: Recognition IT Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 11 Syllabus Overview 1 Introduction [Proposal] {Background: TMB Chapter 1} Charting the Brain 1 [TMB 2.4] The Brain as a Network of Neurons [TMB Section 2.3] Visual Preprocessing [TMB 3.3] Adaptive networks: Hebbian learning, Perceptrons; Landmark learning [TMB 3.4] [NSLbook] Hebbian Learning and Visual plasticity; Self-organizing feature maps; [NSLJ] Kohonen maps Higher level vision 1: object recognition {Background TMB 5.2} Introduction to NSL: modules; SCS schematic capture system; Maxselector model[NSLbook] {Homework} Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 12 Syllabus Overview 2 Schemas for Reaching and Grasping; Affordances [TMB 2.2, 5.3] {Background TMB 2.1, 52} Charting the Brain 2 The FARS model of control of grasping 1: Population coding The FARS model of control of grasping 2: Sequence learning and the basal ganglia The FARS model of control of grasping 3: Working Memory Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 13 Mirror Neurons Rizzolatti, Fadiga, Gallese, and Fogassi, 1995: Premotor cortex and the recognition of motor actions Mirror neurons form the subset of grasp-related premotor neurons of F5 which discharge when the monkey observes meaningful hand movements made by the experimenter or another monkey. F5 is endowed with an observation/execution matching system [The non-mirror grasp neurons of F5 are called F5 canonical neurons.] Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 14 Computing the Mirror System Response The FARS Model: Recognize object affordances and determine appropriate grasp. The Mirror Neuron System (MNS) Model: We must add recognition of trajectory and hand preshape to recognition of object affordances and ensure that all three are congruent. There are parietal systems other than AIP adapted to this task. Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 15 The Mirror Neuron System (MNS) Model Object features cIPS PF Object affordance extraction Hand motion detection STS AIP Object affordance -hand state association Hand shape recognition Motor program (Grasp) Integrate temporal association Action Mirror Feedback recognition Hand-Object spatial relation analysis 7a F5canonical (Mirror Neurons) F5mirror Object location Motor execution Motor program (Reach) M1 F4 work with Erhan Oztop Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 16 STS hand shape recognition Color Coded Hand Feature Extraction Step 1 of hand shape recognition: system processes the color-coded hand image and generates a set of features to be used by the second step Model Matching Step 2: The feature vector generated by the first step is used to fit a 3D-kinematics model of the hand by the model matching module. The resulting hand configuration is sent to the classification module. Precision grasp Hand Configuration Classification Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 17 Virtual Hand/Arm and Reach/Grasp Simulator A precision pinch A power grasp and a side grasp Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 18 Core Mirror Circuit Object affordance Association (7b) Neurons Mirror Neurons (F5mirror) Mirror Neuron Output Hand state Motor Program (F5 canonical) Object Affordances Object affordance hand state association Integrate temporal association Mirror Feedback Motor program F5canonical Hand shape recognition & Hand motion detection Mirror Feedback Hand-Object spatial relation analysis Action recognition (Mirror Neurons) Motor program Motor execution F5mirror Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 19 Power and precision grasp resolution (a) (b) Precision Pinch Mirror Neuron Power Grasp Mirror Neuron Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 20 Syllabus Overview 3 Reinforcement learning and motor control; [NSLJ] Conditional motor learning Adaptive networks: Gradient descent and backpropagation [TMB 82] [NSLJ] Backprop: a How to run the model; b How to write the model [NSLbook] NeuroBench and the NeuroHomology Database Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 21 Syllabus Overview 4 The MNS1 Model 1: Hand Recognition Systems concepts; Feedback and the spinal cord [TMB 31, 32] The MNS 1 Model 2: Simulating the kinematics and biomechanics of reach and grasp The MNS1 Model 3: Modeling the Core Mirror Neuron Circuit Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 22 5 Specific Aims for the Mirror Neuron Proposal 5 Specific Group Projects for CS 564 Development of the Mirror System: 1. Development of Grasp Specificity in F5 Motor and Canonical Neurons 2. Visual Feedback for Grasping: A Possible Precursor of the Mirror Property Recognition of Novel and Compound Actions and their Context: 3. The Pliers Experiment: Extending the Visual Vocabulary 4. Recognition of Compounds of Known Movements 5. From Action Recognition to Understanding: Context and Expectation Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 23 A Human Mirror System Rizzolatti, Fadiga, Matelli, Bettinardi, Perani, and Fazio: Broca's region is activated by observation of hand gestures: a PET study. PET study of human brain with 3 experimental conditions: Object observation (control condition) Grasping observation A key language area!!! Object prehension. The most striking result was highly significant activation in the rostral part of Broca's area. Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 24 A New Approach to the Evolution of Human Language Rizzolatti, G, and Arbib, M.A., 1998, Language Within Our Grasp, Trends in Neuroscience, 21(5):188-194: The Mirror System Hypothesis: Human Broca’s area contains a mirror system for grasping which is homologous to the F5 mirror system of monkey, and this provides the evolutionary basis for language parity - i.e., an utterance means roughly the same for both speaker and hearer. This adds a neural “missing link” to the tradition that roots speech in a prior system for communication based on manual gesture. Beyond the Mirror: Seeing F5 as part of a larger mirror system, then extending our understanding via imitation to language-readiness. This topic will be the jumping off point for the Spring 2002 version of CS 664, which will be taught by Professors Arbib and Itti. Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 25 Syllabus Overview 5 Control of eye movements [TMB 6.2] Basal Ganglia and Control of eye movements - Dominey [NSLbook] Dopamine and Sequence Learning Abstract models of Sequence Learning Extending the FARS model to mirror neurons and language Project Reports 1, 2,3 Project Reports 4,5; Concluding Discussion Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 26 A Conceptual Restructuring of the Syllabus 1 Overview + Basic Concepts of Neurons and Schemas 1. Modeling the Mirror System: Setting Goals for the Course [Proposal] {Background: TMB Chapter 1} 2. Charting the Brain 1 [TMB 2.4] 3. The Brain as a Network of Neurons [TMB Section 2.3] 9. Schemas for Reaching and Grasping; Affordances [TMB 2.2, 5.3] {Background TMB 2.1, 5.2} 10. Charting the Brain 2 Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 27 A Conceptual Restructuring of the Syllabus 2 Vision Preprocessing 4. Visual Preprocessing [TMB 3.3] 6. Hebbian Learning and Visual plasticity; Self-organizing feature maps; [NSLJ] Kohonen maps Low-Level Vision Supplementary reading not covered in lectures: Perceptual and motor schemas: Discussion of visual segmentation based on, e.g., edge and texture cues; Stereoscopic vision [TMB 7.1]; Motion perception and optic flow [TMB 7.2]. High-Level Vision 7. Higher level vision: object recognition {Background TMB 5.2} 18. The MNS1 Model 1: Hand Recognition Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 28 A Conceptual Restructuring of the Syllabus 3 Motor Control 19. Systems concepts; Feedback and the spinal cord [TMB 3.1, 3.2] 20. The MNS 1 Model 2: Simulating the kinematics and biomechanics of reach and grasp Visuo-Motor Integration 11. The FARS model of control of grasping 1: Population coding 12. The FARS model of control of grasping 2: Sequence learning and the basal ganglia 13. The FARS model of control of grasping 3: Working Memory 21. The MNS1 Model 3: Modeling the Core Mirror Neuron Circuit 22. Control of eye movements [TMB 6.2] 23. Basal Ganglia and Control of eye movements - Dominey [NSLbook] Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 29 A Conceptual Restructuring of the Syllabus 4 Adaptive Networks 5. Adaptive networks: Hebbian learning, Perceptrons; Landmark learning [TMB 3.4] [NSLbook] 14. Reinforcement learning and motor control; [NSLJ] Conditional motor learning 15. Adaptive networks: Gradient descent & backpropagation [TMB 8.2] 24. Dopamine and Sequence Learning 25. Abstract models of Sequence Learning Higher Cognitive Functions 26. Extending the FARS model to mirror neurons and language Supplementary reading: Memory and Consciousness [TMB 8.3] Simulation and Neuroinformatics 8. Introduction to NSL: modules; SCS schematic capture system; Maxselector model [NSLbook] 16. [NSLJ] Backprop: a. How to run and write the model [NSLbook] 17. NeuroBench and the NeuroHomology Database Arbib: CS564 - Brain Theory and Artificial Intelligence, USC, Fall 2001. Lecture 1. Introduction and Overview 30