Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Linear Models of Judgment Judgment vs. choice Multiattribute model of judgment Actuarial model of the environment Experts and computers Bootstrapping models Desirability Cost Mileage Safety Price Repairs Crash Test ABS Airbags Fun Size Looks Handling External Marque Features Trunk Cabin Judgment vs. Choice Judgment = assign a score or category e.g., How much would you pay for a one-week trip to Aspen? How much do you like Bill Clinton? What will the price of Intel be in 6 months? Choice = pick from a set of alternatives e.g., which car? investment? job? Multiattribute Choice Model Choice = select most desirable Desirability is judged from attributes Attributes can be FACTS (e.g., price), COMPOSITES (e.g., safety), or subjective VALUES (e.g., prestige) There is often a hierarchical structure to attributes and judgments How to make tradeoffs, e.g., weight & add Desirability of a Car Desirability Cost Mileage Safety Price Repairs Crash Test ABS Airbags Fun Size Looks Handling External Marque Features Trunk Cabin Lens Model Judgment (Ys) is an attempt to represent or predict the environment from cues There is a criterion (Ye) that allows us to estimate correctness of the judgment c u Ye e s Ys Models of Decision Makers Slovic’s study of two stockbrokers: A: near term prospects, P/E, earnings qtly trend B: earnings yearly trend, P/E, profit margin trend Ratings of business schools: USNWR: reputation with academics, reputation with CEOs, selectivity, placement BusWeek: recruiters rate analytics, teaming, global; graduates rate teaching, curriculum, placement Actuarial Environmental Models Major Retailer’s Credit Scoring Table Occupation clergy executive professional student teacher unemployed no answer 46 62 62 46 46 33 47 Job Tenure < .5 years .5 - 5.5 5.5 - 8.5 8.5 - 15.5 > 15.5 years 31 24 26 31 39 Capon, J. Marketing, 1982, 46, 82-91. Older economists make more extreme forecasts Comparisons Using Models How consistent are individuals? How consensual are experts? How accurate are judges? (Ye vs. Ys) What are judges doing? (Ysm) What predicts the criterion? (Yem) How good are our models? Do judges understand the environment? (Yem vs. Ysm) Graduate Admissions Example Ys = judgment of admissions committee (1 to 5 scale) Ye = faculty ratings of performance Ysm = prediction model of judgments = -4.17 +.0032*GRE +1.02*GPA +.0791*QI Yem= actuarial model of performance = -.71 +.0006*GRE +.76*GPA +.2518*QI Admission and Job Interviews Harvard Business School stopped conducting interviews – Are interviews accurate? – Are interviews overweighted? – What is their proper role? HBS no longer uses the GMAT HBS criteria: academics and character Advantages of Models Makes strategy explicit Can see how experts vary Train new judges Learn about environments Enhance or replace experts Can use the model when expert gone Judges vs. Environment Which should be more accurate, expert judges or actuarial models? Judges have their experience, ability to use cues in complex ways Actuarial models are simple, typically linear in form, consistent Judges vs. Actuarial Model Actuarial Model .95 Task Credit scoring Judge .80 Stock analysis .23 .80 Personnel .35 .57 Cancer survival -.01 .35 Graduate GPA .33 .69 Why Don’t Experts Do Better? They have the wrong rules They don’t use their rules - distractions - fatigue, boredom - “exceptions” - unable to make tradeoffs Bootstrapping Models If intuitive decision makers have good rules but fail to use them consistently, can we separate signal from noise? Consensus of judges (see groups later) Model of a judge (bootstrapping) Judgment = Linear + Nonlinear + Noise What wins: Judge vs. linear model? A Tale of Three Models Bootstrap Actuarial Model Model .85 .95 Task Credit scoring Judge .80 Stock analysis .23 .29 .80 Personnel .35 .46 .57 Cancer survival -.01 .13 .35 Graduate GPA .50 .69 .33 Some Typical Results Some tasks are much harder than others Actuarial models almost always win Bootstrapping works! Linear models correlate with any monotonic function, work well when there is noise, positively correlated cues, work with random or unit weights To improve on linear models, you need lots of data Experts and Models What do experts do best? What do computers do best? How can they be combined? Should we give the model to the expert or give the expert to the model? Batterymarch Example Stock portfolio company Manage $12 Billion with 37 employees Experts identify variables, suggest rules, design tests, deal with clients Computer keeps databases, runs tests of rules, buys and sells stocks 10-12 rules identify attractive stocks Working With the Political Lens: Separating Facts and Values Selecting a bullet for Denver Police - police want to immobilize suspects - community concerned about injuries - experts testify on each side What kinds of information are needed? How should this decision be made? A Frame for Conflict Resolution Facts weight speed shape etc. Values Desirability Injury potential Stopping power Threat to bystanders Denver Bullet Resolution Experts combine facts into judgments on each value Constituencies compromise on how to weight the values into overall worth Injury Potential proposed .............. by Police . . . . . . P. . . . . . . ............ proposed by . . . C. . . . . . . . . Community ........... Stopping Power Role of Technical Experts Executive whose daughter had a hip deformity One doctor said, “Wait” A second said, “Brace for 6 months” The third said, “Operate” How would you make this decision? Your Exercise #1: Job Selection What were the attributes or objectives of jobs that mattered to you? How different were the rankings due to intuition, weighted linear model, unit weighted model? If the rankings differ, which do you trust? Why? Value-added in the process, not the numbers