Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Business School Institute of Business Informatics Unsupervised Learning Uwe Lämmel www.wi.hs-wismar.de/~laemmel [email protected] 1 Unsupervised Learning Neural Networks – Idea – Artificial Neuron & Network – Supervised Learning – Unsupervised Learning – Data Mining – other Techniques 2 Unsupervised Learning Unsupervised Learning – – – – – 3 Unsupervised Learning Self-Organizing Map (SOM) Learning Clustering – Example Visualisation Application: TSP Self Organizing Maps (SOM) A natural brain can organize itself Now we look at the position of a neuron and its neighbourhood Kohonen Feature Map two layer pattern associator - Input layer is fully connected with map-layer - Neurons of the map layer are fully connected to each other (virtually) 4 Unsupervised Learning Clustering - objective: All inputs of a class are mapped onto one and the same neuron f ai output B Input set A - Problem: classification in the input space is unknown - Network performs a clustering 5 Unsupervised Learning Winner Neuron Input-Layer Winner Neuron Kohonen- Layer 6 Unsupervised Learning Learning in an SOM 1. Choose an input k randomly 2. Detect the neuron z which has the maximal activity -> winner neuron 3. Adapt the weights in the neighbourhood of z: neuron i within a radius r of z. 4. Stop if a certain number of learning steps is finished otherwise decrease learning rate and radius, go on with step 1. 7 Unsupervised Learning Centre of Activation - Idea: highly activated neurons push down the activation of neurons in the neighbourhood - Problem: Finding the centre of activation: - Neuron j with a maximal net-input w iz i oi max j w ij oi i - Neuron j, having a weight vector wj which is similar to the input vector (Euklidian Distance): z: 8 Unsupervised Learning x - wz = minj x - wj SOM Training • find the winner neuron z for an input pattern p (minimal Euclidian distance) min m p W j m p W z input pattern mp j • adapt weights of connections • input neurons winner neuron • input neurons neighbours wij h jz (mi wij ) , if dist ( j , z ) r w , otherwise wij / ij 9 Unsupervised Learning Wj Kohonen layer h jz e dist( j , z ) 2 2r 2 Example Credit Scoring A1: Credit History A2: Debts A3: Collateral A4: Income • We do not look at the Classification • SOM performs a Clustering 10 Unsupervised Learning Credit Scoring – good = {5,6,9,10,12} – average = {3, 8, 13} – bad = {1,2,4,7,11,14} 11 Unsupervised Learning Credit Scoring – Pascal tool box (1991) – 10x10 neurons – 32,000 training steps 12 Unsupervised Learning Visualisation of a SOM • Colour reflects Euclidian distance to input • Weights used as coordinates of a neuron • Colour reflects cluster NetDemo ColorDemo 13 Unsupervised Learning TSPDemo Example TSP – Travelling Salesman Problem – A salesman has to visit certain cities and will return to his home. Find an optimal route! – problem has exponential complexity: (n-1)! routes Experiment: Pascal Program, 1998 14 Unsupervised Learning 31/32 states in Mexico? Nearest Neighbour: Example – Some cities in Northern Germany: – Initial city is Hamburg Kiel Hamburg Exercise: • Put in the coordinates of 20 important places • Find a solution for the TSP using a SOM! Frankfurt 15 Unsupervised Learning Schwerin Hannover Essen Rostock Berlin SOM solves TSP Kohonen layer Draw a neuron at position: input w1i= six X Y 16 Unsupervised Learning w2i= siy (x,y)=(w1i,w2i) SOM solves TSP – Initialisation of weights: – weights to input (x,y) are calculated so that all neurons form a circle – The initial circle will be expanded to a round trip – Solutions for problems of several hundreds of towns are possible – Solution may be not optimal! 17 Unsupervised Learning Applications – Data Mining - Clustering – Customer Data – Weblog – ... – You have a lot of data, but no teaching data available – unsupervised learning – you have at least an idea about the result – Can be applied as a first approach to get some training data for supervised learning 18 Unsupervised Learning