* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Com1005: Machines and Intelligence
Survey
Document related concepts
Perceptual control theory wikipedia , lookup
Neural modeling fields wikipedia , lookup
Chinese room wikipedia , lookup
Concept learning wikipedia , lookup
Machine learning wikipedia , lookup
Pattern recognition wikipedia , lookup
Intelligence explosion wikipedia , lookup
Ethics of artificial intelligence wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Existential risk from artificial general intelligence wikipedia , lookup
Convolutional neural network wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
Catastrophic interference wikipedia , lookup
Transcript
Amanda Sharkey COM1005: MACHINES AND INTELLIGENCE Where were we? Turing test: how can we decide if something is intelligent? Traditional (symbolic) AI early programs, and knowledge representation and search GPS, microworlds, expert systems Chinese room Functionalists: thought is symbol manipulation Searle and Chinese room – computers can manipulate symbols, but that is not enough for real understanding. Even a computer that passes the Turing test will not really understand or be intelligent. Strong AI – appropriately programmed computer really is a mind Weak AI – using computers to model and understand human intelligence. AI and mind AI and applications AI and illusion of intelligence Alternatives to traditional AI Neural Computing also known as Connectionism Artificial Neural Nets Parallel Distributed Processing Brain-like computing Biologically-inspired AI Paradigm shift Kuhn: “The structure of scientific revolutions” (1962) A paradigm – shared set of assumptions Period of normal science Paradigm shift – begins as data collected that doesn’t fit with current assumptions Shift to new viewpoint and assumptions Idea related to “Zeitgeist” Examples in science Shift from Ptolemeic cosmology to Copernican (earth as centre of universe, to earth revolving round the sun) Examples in social science Shift from behaviourism to emphasis on cognition and thought Examples of changing Artificial Intelligence approaches to mind Traditional symbolic AI Neural computing Embodied AI Intro to Artificial Neural Nets The human brain The human brain Contains approximately 10 thousand million basic neurons Each neuron connected to many others Each neuron accepts many inputs If enough active inputs received at once, neuron will be activated and fire Soma – body of neuron Dendrites - Long filaments attached to soma Inputs arrive through dendrites Axon - Output for neuron How are they connected? Synapse – where axon and dendrite meet. Indirect chemical linkage – neurotransmitters released which activate gates on dendrites. Activating gates allow charged ions to flow Charged ions alters dendritic potential, provides voltage pulse which is conducted to next neuron body. Human brain – capable of impressive tasks E.g. vision, speech recognition, learning Fault tolerant – distributed processing, many simple elements sharing each job. Graceful degradation – performance gradually falls with damage Neural computing – by modelling major features of brain and its operation, we can capture useful properties of the brain. Knowledge in the brain? Connections between neurons can be strengthened. Simple model of neuron – proposed in 1943 by McCulloch and Pitts Simplified – no complex patterns and timings of actual nervous activity in real nervous systems Neuron on or off Output depends on inputs – needs enough activation to fire (threshold). Basic model – weighted sum of inputs, compared to internal threshold, turn on if above threshold McCulloch and Pitts (1943) Brain-like mechanisms – showing how artificial neurons could compute Boolean logical functions like AND, or OR Simplification 1: neurons threshold – on or off. Simplification 2: Synapses, equivalent to weighted connections Rosenblatt (1962) Perceptron. Single layer net which can learn to produce an output given an input. Learning? Stronger connection between neurons is captured by multiplicative weight Essentials of Perceptron learning algorithm Set the weights and thresholds randomly Present an input Calculate actual output by taking thresholded value of weighted sum of inputs Alter weights to reinforce correct decisions and discourage incorrect decisions – to reduce the error Supervised learning – uses target of what we want it to achieve. 1969 Minsky and Papert – Perceptrons book. Minsky and Rosenblatt were old rivals They showed there were some problems that could not be solved by 1 layer net, (e.g. XOR) and there was no learning mechanism for 2 layer nets Loss of interest in neural nets – heyday of symbolic AI AND 1 1 x0 x1 output 1 1 1 0 0 0 1 0 0 0 1 0 XOR x0 x1 t 1 1 0 0 0 1 1 0 1 0 1 0 1 1 0.5 Inputs: 0 0 1 -1 01 10 11 0.5 1.5 A solution to XOR problem 1 1 1 1 Second wave of Artificial Neural Nets Backpropagation (Rumelhart et al, 1986) Learning method for 2 layer nets Error contribution of hidden units is computed by transmitting the delta error on the output units back to the hidden units 1986 Rumelhart and McClelland 2 books on Parallel Distributed Processing. Presented many NN models including Past- tense model. Cognitive models – a model of some aspect of human cognition. 1986 Rumelhart and McClelland: multi-layer perceptron Training MLPs requires repeated presentations An inexact science – no guarantee that net will converge Can involve long training times Little guidance about parameters – e.g. number of hidden units Also need to find a good input-output representation of task. Generalisation Main feature of NNs: ability to generalise and to go beyond the patterns they have been trained on. Unknown pattern will be classified with others that are similar Therefore learning by example is possible. Examples of ANN applications Gorman and Sejnowski, 1988 classified mines versus rocks using sonar signals Le Cun et al 1989: recognising handwritten postcodes Pomerleau, 1989: navigation of car on winding road. Pattern recognition Pandemonium: Oliver Selfridge 1959 McClelland and Rumelhart: Interactive activation model and context effects. (1981) Example applications of Neural Nets NETtalk: Sejnowski and Rosenberg 1987 Learned to pronounce English text. Takes text, maps text onto phonemes, produces sounds using electronic speech generator Difficult to specify rules mapping text onto speech E.g. x in box or axe, differs from xylophone 203 input units, 80 hidden units, 26 output units (phonemes) Window of 7 letters moved over text – learning to pronounce middle letter. 29 input units, one for each of 26 letters, and 3 for punctuation. (29x7) Trained on 1024 words. After 50 passes NETtalk can perform at 95% accuracy on training set, and generalise to unseen words 78% Influential example – recorded NETtalk beginning by babbling and gradually improving. Past-tense learning model A model of human ability to learn past-tenses of verbs Developed by Rumelhart and McClelland (1986). Past-tenses? Today I look at you, yesterday I ? at you. Today I speak to you, yesterday I ? to you. Today I wave at you, yesterday you ? at me. rick – yesterday he ? the sheep Many regular examples: E.g walk -> walked, look -> looked Many irregular examples: E.g. bring -> brought, sing -> sang Children learning to speak Baby: DaDa Toddler: Daddy Very young child: Daddy home! Slightly older child: Daddy came home! Older child: Daddy comed home! Even older child: Daddy came home! Stages of acquisition Stage 1 Past tense of a few specific verbs, some regular e.g. looked, needed Most irregular e.g. came, got, went, took, gave As if learned by rote (memorised) Stage 2 Evidence of general rule for past-tense – add ed to stem of verb E.g. camed or comed Also for past-tense of nonsense word e.g rick They added ed - ricked Stage 3 Correct forms for both regular and irregular verbs Verb type Stage 1 youngest Stage 2 older Stage 3 Older again Early verbs correct Regularised correct regular correct correct Other irregular novel regularised Correct or regularised regularised Regularised U shaped curve – correct form in stage 1, errors in stage 2, few errors in stage 3. Suggests rule acquired in stage 2, and exceptions learned in stage 3. Rumelhart and McClelland – aim to demonstrate that connectionist network would show same stages and learning patterns. Trained net by presenting Input – root form of word e.g. walk Output – phonological structure of correct past-tense version of word e.g. walked Test model by presenting root form as input, and see what past-tense form it generates as output. Used Wickelfeature method to encode words Wickelphone: Target phoneme and context E.g. came #Ka, kAm, aM# Coarse coded onto Wickelfeatures, 16 wickelfeatures for each wickelphone Input and output of net 460 units Shows need for input representation Training: used perceptron convergence procedure (problem linearly separable) Target used to tell output unit what value it should have. If output is 0 and target is 1, need to increase weights from active input units If output is 1 and target is 0, need to reduce weights from active units. 560 verbs divided into High, Medium, and Low frequency (regular and irregular) 1. Train on 10 high frequency verbs for 10 epochs Live-lived, look-looked, come-came, get-got, give- gave, make-made, take-took, go-went, have-had, feel-felt 2. 410 medium frequency verbs added, trained for 190 more epochs Net showed dip in performance – making errors like children e.g come -comed 3. Tested on 86 low frequency verbs not used for training Got 92% regular verbs right, 84% irregular right. Model illustrates: Neural net training – repeated examples of input- output pairs Generalisation – correct outputs produced for untrained words E.g. input guard -> guarded Input cling -> clung Past-tense model: Showed - a neural net could be used to model an aspect of human learning - same u-shaped curve shown as found in children. - the neural net discovered the relationship between inputs and outputs, not programmed. - that it is possible to capture apparently rulegoverned behaviour in a neural net. Strengths of connectionism Help in understanding how a mind, and thought, emerges from the brain Better account of how we learn something like past-tense, than explicit programming of a rule? Is this a good model of how we learn past-tenses? Fierce criticisms: Steve Pinker and Alan Prince (1988) More than 150 journal articles followed on the debate Net can only produce past-tense forms, cannot recognise them. Model presents pairs of verb+past tense, children don’t get this. Model only looks at past-tense, not the rest of language Getting similar performance to children was the result of decisions made about: Training algorithm Number of hidden units How to represent the task Input and output representation Training examples, and manner of presentation Assignment: due in Monday November 22nd, Week 9. Write an essay on one of the following, in 1000-2500 words. 1. Why is it so difficult to produce a computer program that can pass the Turing Test? 2. Explain Searle’s Chinese Room Argument, and consider whether it is still relevant to Artificial Intelligence today. 3. Which of the following can be more easily claimed to be intelligent: a Chatbot, a robot, a pig and a human baby? 4. Discuss the ways in which the relationship between chess and Artificial Intelligence has changed over time. 5. Explain the main differences between traditional symbolic Artificial Intelligence and Connectionism, and consider their respective strengths and limitations. 1. Comprehensiveness and Relevancy (is the question satisfactorily addressed?) 2. Argument structure (is the argument clear and well developed?) 3. Use of resources (is the essay well researched?) 4. Technical (references, spelling, grammar, punctuation). References In text: (1) In References: (1) Searle, J.R. (1980) Minds, brains and programs. Behavioural and Brain Sciences, 3, 41724 Or for a book... In text: Pfeifer and Scheier (2001) In references: Pfeifer, R., and Scheier, C. (2001) Understanding Intelligence, MIT Press, Cambridge Massachusetts Or for a journal In text: Walrus et al (2009) In references: Walrus, W., Lamb, B., Wolf, W., Sheep, S., Rat, R., Slug, W. (2009) Animals are intelligent, Journal of Incredible Research, 3, 111-116. Differences between Connectionism and traditional Symbolic AI Knowledge – represented by weighted connections and activations, not explicit propositions Learning – Artificial Neural Nets (ANNs) trained versus programmed. Also greater emphasis on learning. Emergent behaviour – rule-like behaviour, without explicit rules More Differences Examinability: you can look at a symbolic program to see how it works. Artificial Neural net – black box ...consists of numbers representing activations, and weighted links. Symbols: Symbolic AI – manipulating symbols connectionism has no explicit symbols Relationship to the brain: ‘Brain-style’ computing versus manipulation of symbols. Connectionism versus Symbolic AI Which is better? Which provides a better account of thought? Which is more useful? Artificial neural nets more like the brain than traditionally programmed computer? Are Brains like Computers? Parallel operation – 100 step argument Neuron slower than flip-flop switches in computers. Takes thousandth of a second to respond, instead of a thousand-millionth of a second. Brain running AI program would take 1000th of a second for each instruction Brain can extract meaning from sentence, or recognise visual pattern in 1/10th of a second. Means program should only be 100 instructions long. But AI programs contain 1000s of instructions Suggests parallel operation Connectionism scores Are brains like computers? Computer memory – exists at specific physical location in hardware. Human memory – distributed E.g. Lashley and search for engram. Trained rats to learn route through maze. Could destroy 10% of brain without loss of memory. Lashley (1950) “there are no special cells reserced for special memories….The same neurons which retain memory traces of one experience must also participate in countless other activities” Connectionism scores Graceful degradation When damaged, brain degrades gradually, computers crash. Phineas Gage Railway worker – iron rod through the anterior and middle left lobes of cerebrum, but lived for 13 years – conscious, collected and speaking. Connectionism scores Brains are unlike von Neumann machines with: Sequential processor Symbols stored at specific memory locations Access to memory via address Single seat of control, CPU Connectionism and the brain Units in net like neurons Learning in NNs like learning in brain Nets and brains work in parallel Both store information in distributed fashion NNs degrade gracefully – if connections, or some neurons removed, can still produce output. But …. Connectionism only “brain-style” computing Neurons simplified – only one type Learning with backpropagation biologically implausible. Little account of brain geometry and structure Also …. Artificial neural nets are simulated on computers e.g. past-tense network Rumelhart and McClelland simulated neurons, and their connections on a computer. Connectionism Getting closer to real intelligence? Closer to computation that does occur in brain than standard symbolic AI Provides a way of relating brain, and AI Connectionism and thought Can connectionism provide an account of mind? Symbolicists arguing that only a symbol system can provide an account of cognition Functionalists: Not interested in hardware, only in software Connectionists arguing that you need to explain how thought occurs in the brain. Rule-like behaviour Past tense learning Create rules, and exceptions Or show that rule-like behaviour can result from a neural net, even though no rule is present. Symbolic AI vs Connectionism Different strengths Connectionism – good at low level pattern recognition, in domains where there are many examples, and it’s hard to formulate the rule. Symbolic AI – good at conscious planning, reasoning, early stages of learning a skill (rules). Hybrid system Connectionist account of lower level processes Symbolic account of higher level processes. Connectionism and Strong AI Searle – Chinese room shows program does not understand, any more than operator of Chinese room. Problem of symbol grounding ….. But does a neural net understand? It learns….. Chinese Gym – a room full of english speaking people carrying out neural processes, and outputing chinese. Do they understand? Presentations Who was Alan Turing? Computers versus Humans: the important differences Is the mind a computer? Artificial Intelligence and Games What challenges are left for Artificial Intelligence? The social effects of Artificial Intelligence: the good, the bad and the ugly Chatbots Computers and emotions AI and the media Fact or Fiction?: Artificial Intelligence in the movies - groups of 5 acknowledgements - research, presentation, delivery etc. Next week (week 6) Reading week. Week 7: Guru lecture Text processing Why hasn’t Turing Test been passed yet? And applications. Implementational connectionism A different way of implementing symbolic structures, different level of description Eliminative connectionism Radical position, cognition and thought can only be properly described at connectionist level Revisionist connectionism Symbolic and connectionist are both legitimate levels of description. Hybrid approach – choose best level depending on what is being invesigated.