* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Presentation
Neural coding wikipedia , lookup
Eyeblink conditioning wikipedia , lookup
Artificial intelligence for video surveillance wikipedia , lookup
Central pattern generator wikipedia , lookup
Optogenetics wikipedia , lookup
Holonomic brain theory wikipedia , lookup
Biological neuron model wikipedia , lookup
Single-unit recording wikipedia , lookup
Artificial intelligence wikipedia , lookup
Artificial general intelligence wikipedia , lookup
Neural engineering wikipedia , lookup
Channelrhodopsin wikipedia , lookup
Development of the nervous system wikipedia , lookup
Binding problem wikipedia , lookup
Donald O. Hebb wikipedia , lookup
Concept learning wikipedia , lookup
Learning theory (education) wikipedia , lookup
Synaptic gating wikipedia , lookup
Pattern recognition wikipedia , lookup
Machine learning wikipedia , lookup
Neural modeling fields wikipedia , lookup
Artificial neural network wikipedia , lookup
History of artificial intelligence wikipedia , lookup
Neuroanatomy wikipedia , lookup
Convolutional neural network wikipedia , lookup
Neuropsychopharmacology wikipedia , lookup
Metastability in the brain wikipedia , lookup
Nervous system network models wikipedia , lookup
Catastrophic interference wikipedia , lookup
Hierarchical temporal memory wikipedia , lookup
J. McLean Sloughter What I Did On My Summer Vacation: Undergraduate Research Internships, Neural Networks, & Airport Security “Soon after the electrical current became known many attempts were made by the older physiologists to explain nervous impulses in terms of electricity. The analogy between the nerves of the body and a system of telephone or telegraph wires was too striking to be overlooked.” (from Studies in Advanced Physiology, Louis J. Rettger, A.M., 1898, p. 443) How the Brain Works An Extremely Over-Simplified Explanation The brain is made up of interconnected neurons Neurons are binary – either fire or don’t fire As a neuron receives signals from other neurons, it will start firing if the total signal reaches some threshhold 2 3 How the Brain Works How the Brain Works Just like that, except way more complicated Actually a lot more neurons involved Frequency of firing is also important But let’s ignore those details for now… 4 Putting a philosophy degree to work History – 1940s Warren McCulloch, a psychologist and philosopher, postulated that thought is discrete Suggested a “psychon” – the smallest unit of thought Thought that an individual neuron firing or not firing might be a psychon Recommended developing a “calculus of ideas” to describe neural activity 5 Philosophy + Math = Fame History – 1940s McCulloch teamed up with Walter Pitts, a math prodigy Together they published “A Logical Calculus of the Ideas Immanent in Nervous Activity” This paper introduced the idea of a “nervous network,” the first artificial neural model of cognition 6 Enter von Neumann History – 1940s Von Neumann became an early proponent of their work However, he criticized it as being overly simplistic Based on some of von Neumann’s suggestions, McCulloch & Pitts proposed a system using a large number of neurons This allows for robustness – an ability, for example, to recognize a slightly deformed square as still being essentially a square 7 Best Mathematician Name Ever History – 1940s Norbert Weiner (“The Father of Cybernetics”) proposed a more involved system Weighted inputs – one neuron can be more influential than another Memory = learning weights Did not propose how this learning takes place, dismissed that as a problem for engineers to deal with 8 In which not a whole lot happened History – 1950s Marvin Minsky introduced a system based on behavioural conditioning Neurons had probabilities of sending signals When they produced the correct output, probabilities were increased When the produced the wrong output, probabilities were decreased And nobody really seemed to care (they were all busy becoming computer programmers) 9 Perceptrons History – 1960s In 1960, Rosenblatt published a proof of the capabilities of what he named the “perceptron” The perceptron acted much like the nervous network, but with weighted signals The major advance was a learning algorithm Rosenblatt was able to prove that, using his learning algorithm, any possible configuration of the perceptron could be learned, given the proper training data 10 Perceptron function History – 1960s Consider a simple case where nodes A and B are each sending signals to node B Node B has some threshold, T, which it needs to receive to be activated A, B, and C are all binary – 0 or 1 W1 and W2 are the weights between A and C and B and C Then, if A*W1 + B*W2 > T, C = 1 Otherwise, C = 0 11 Perceptron learning History – 1960s Initialize weights randomly Set threshold to some arbitrary value (why does it not matter what value the threshold is set to?) Randomly select one set of inputs Find the result based on current weights Subtract result from desired result = error term Look at each initial node individually Multiply input value by error term by “learning coefficient” (between 0 and 1, controls amount of change you’ll allow at each iteration) Add result to weight previously associated with that node to get a new weight Pick a new set of inputs, repeat until convergence 12 Adaline History – 1960s Widrow and Hoff created a system called Adaline – “Adaptive linear element” Very similar to perceptrons (though with a slightly different learning algorithm) Major changes were the use of -1 instead of 0 for no signal, and a “bias” term – a node that always fires These were significant because they had no basis in neurophysiology, and were added purely because they could improve performance 13 The Wrath of Minksy History – 1960s In 1969, Minsky again entered the world of neural networks, this time co-authoring the book “Perceptrons” with Seymour Papert 14 Xor History – 1960s Minsky and Papert showed, among other critiques of perceptrons, that they weren’t capable of learning an exclusive OR (can you see why?) An exclusive OR could be made by combining multiple other networks – have A and B feed into both an OR and a NAND, and then AND the results But learning rules only worked with a single layer network – Minskey and Papert suggested researching whether learning rules could be developed for multi-layered networks 15 The Problem History – 1960s Minsky & Papert put their critique of perceptrons at the front of the book They put their suggestions for research into multi-layered perceptrons at the back of the book, after a few hundred pages of rather dense math People didn’t seem to read that far Research on perceptrons died 16 History – 1970s Nothing important happened 17 The Multi-Layer Perceptron History – 1980s Rumelhart, Hinton, and Williams created a learning algorithm for multi-layer perceptrons Requires differentiation of functions, and thus the hard threshold had to be replaced by a sigmoid function 18 MLP function Net input to a node: History – 1980s n I wijxj j 1 Output from a node: f (I ) 1 1 e I 19 MLP learning Change weight as follows: History – 1980s wij bEf ( I ) Where b is the learning coefficient, and E is the error term: E output y desired y actual df ( I i ) n output E w ij E j dI j 1 df ( I ) f ( I )(1 f ( I )) where dI middle i 20 The Problem Airport Security Metal detectors only detect things that are, well, metal (and even then only sometimes) Lots of bad things aren’t metal – plastic explosives, ceramic guns, plastic flare guns An x-ray could potentially see these objects, but submitting people to x-rays every time they fly isn’t an especially good idea 21 The Solution Airport Security Scientists at Pacific Northwest National Laboratory developed a millimeter wave camera Millimeter waves are not harmful like x-rays They can penetrate clothing, but are reflected by skin Plastics and ceramics show up with a distinctive speckled pattern, as they only partially reflect the waves 22 The New Problem Caused by the Solution Airport Security Scientists at a government lab just made a camera that can take pictures of you through your clothes Implementing this in airports would have every passenger go through a virtual strip-search 23 The Solution to the Problem Caused by the Solution to the Other Problem Airport Security Rather than have a human operator look at the pictures, we can have a computer look at them for us The computer can identify suspicious areas and provide a non-naughty picture to the security officer 24 In Practice Airport Security This technology is now in use by SafeView, a company spun off from this project It is being used in airports, government buildings, border crossings, and other locations around the world 25 Student Research Opportunities Research Internship I was involved in this project while a student intern at Pacific Northwest National Lab Information about PNNL’s student internship programs can be found online at http://science-ed.pnl.gov/students/ One of my summers on this project, I applied through the Department of Energy’s internship program, which includes opportunities at a number of other national labs Information on DOE internship programs is available at http://www.scied.science.doe.gov/scied/erulf/about.html 26