Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Final ProjectMining Mushroom World Agenda • • • • Motivation and Background Determine the Data Set (2) 10 DM Methodology steps (19) Conclusion Motivation and Background • To distinguish between edible mushrooms and poisonous ones by how they look • To know whether we can eat the mushroom, to survive in the wild • To survive outside the computer world Determine the Data Set (1/2) Source of data:UCI Machine Learning Repository Mushrooms Database • From Audobon Society Field Guide • Documentation:complete, but missing statistical information • Described in terms of physical characteristics • Classification:poisonous or edible • All attributes are nominal-valued *Large database: 8124 instances (2480 missing values for attribute #12) Determine the Data Set (2/2) 1. Past Usage • Schlimmer,J.S. (1987). Concept Acquisition Through Representational Adjustment (Technical Report 87-19). • Iba,W., Wogulis,J., & Langley,P. (1988). ICML, 73-79 2. No other mushrooms data 10 DM Methodology steps Step 1. Translate the Business Problem into a Data Mining Problem a. Data Mining Goal:separate edible mushrooms from poisonous ones b. How will the Results be Used- increase the survival rate c. How will the Results be Delivered- Decision Tree, Naïve Bayes, Ripper, NeuralNet 10 DM Methodology steps Step 2. Select Appropriate Data a. Data Source – – The Audubon Society Field guide to North American Mushrooms (1981). G. H. Lincoff (Pres.), New York: Alfred A. Knopf Jeff Schlimmer donated these data on April 27th, 1987 b. Volumes of Data - Total 8124 instances 4208(51.8%) edible; 3916(48.2%) poisonous - 2480(30.5%) missing in attribute “stalk-root” 10 DM Methodology steps Step 2. Select Appropriate Data c. How Many Variables- 22 attributes - cap-shape, cap-color, odor, population, habitat and so on…… d. How Much History is Required- no seasonality *As long as we can eat them when we see them 10 DM Methodology steps Step 3. Get to Know the Data a. Examine Distributions:Use “Weka” to visualize all the 22 attributes with histograms b. Class:edible=e, poisonous=p Step 3. Get to Know the Data a. Examine Distributions: there are 2 types of historgrams b. First- all kinds of values appear c. (Attribute 21) population: abundant=a, clustered=c, numerous=n, scattered=s, several=v, solitary=y Step 3. Get to Know the Data 1. Examine Distributions:there are 2 types of historgrams – Second- only some kinds of value appear – (Attribute 7) gill-spacing:close=c, crowded=w, distant=d Step 3. Get to Know the Data 1. Examine Distributions:there are exceptions – Exception 1- missing values in the attribute – (Attribute 11) stalk-root:bulbous=b, club=c, cup=u, equal=e, rhizomorphs=z, rooted=r, missing=? 2480 of this attribute have missing values (Total 8124) Step 3. Get to Know the Data 1. Examine Distributions:there are exceptions – Exception 2- undistinguishable attribute – (Attribute 16) veil-type:partial=p, universal=u Step3. Get to Know the Data 2. Compare Values with Descriptions – no unexpected values except for missing values 10 DM Methodology steps Step 4. Create a Model Set – Creating a Balanced Sample- 75%(6093) as training data, 25%(2031) as test data – Rapid Miner’s “cross-validation” function: k-1 as training, 1 as test 10 DM Methodology steps Step 5. Fix Problems with the Data – Dealing with Missing Values- the attribute “stalkroot” has 2480 missing values – replace all missing values with the average of “stalk-root” value – We replaced ‘?’ with the average value ‘b’ 10 DM Methodology steps Step 6. Transform Data to Bring Information to the Surface – all nominal attribute, no numerical analysis in this step 10 DM Methodology steps True p True e Class precision Pred. p 961 0 100% Pred. e 18 1052 98.32% Step 7. Build Model 1. Decision Tree Performance – Accuracy:99.11% – Lift:189.81% Class recallTrue p 98.16% True e 100.00% Class precision Pred. p 961 0 100% Pred. e 18 1052 98.32% Class recall 98.16% 100.00% 10 DM Methodology steps True p True e Class precision Pred. p 902 9 99.01% Pred. e 77 1043 93.12% Class recall 92.13% 99.14% Step 7. Build Model 2. Naïve Bayes Performance – Accuracy:95.77% – Lift:179.79% True p True e Class precision Pred. p 902 9 99.01% Pred. e 77 1043 93.12% Class recall 92.13% 99.14% 10 DM Methodology steps Step 7. Build Model 3. Ripper Performance – Accuracy:100% True p True e Class precision Pred. p 979 0 100.00% Pred. e 0 1052 100.00% Class recall 100.00% 100.00% – Lift:193.06% True p True e Class precision Pred. p 979 0 100.00% Pred. e 0 1052 100.00% Class recall 100.00% 100.00% 10 DM Methodology steps True p True e Class precision Pred. p 907 110 89.18% Pred. e 72 942 92.90% Class recall 92.65% 89.54% Step 7. Build Model 4. NeuralNet Performance – Accuracy:91.04% – Lift:179.35% True p True e Class precision Pred. p 907 110 89.18% Pred. e 72 942 92.90% Class recall 92.65% 89.54% 10 DM Methodology steps Step 8. Assess Models – Accuracy:Ripper and Decision Tree have better performances Accuracy 105 100 100 99.11 95.77 95 91.04 90 85 Decision Tree Naïve Bayes Ripper Neural Net Accuracy 10 DM Methodology steps Step 8. Assess Models – Lift (to compare the performances of different classification models):Ripper and Decision Tree have higher lifts Lift 195 193.06 189.81 190 185 179.79 179.35 180 175 170 1 2 3 4 Lift 10 DM Methodology steps Step 9. Deploy Models – We haven’t go out and find real mushrooms Step 10. Assess Results Conclusion and questions – – Maybe ripper and decision tree are better models for nominal data How Rapid Miner separates training data from test data