Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Credit Scoring Development and Methods James Marinopoulos Head of Retail Decision Model 25/05/2017 Retail Decision Models Group Risk - Retail Risk 1 Alan Greenspan: President, Federal Reserve Board May 1996 “… We should not forget that the basic economic function of these regulated entities (banks) is to take risk. If we minimise risk taking in order to reduce failure rates to zero, we will, by definition, have eliminated the purpose of the banking system.” Retail Decision Models Group Risk - Retail Risk Risk Families We are managing different groups of Risk Customer fails to pay Change in Losing money market Wrong Strategy prices Processing failures and frauds Regulatory compliance Retail Decision Models Group Risk - Retail Risk Retail Decision Models Responsibilities Policy – Set Group policy on Decision Models – Approve Decision Model policy changes Monitor, Validate and Approve – – – – – New Scorecard Developments Existing Scorecard Functionality Proposed changes to Decision Models Processes New Decision Models Systems functionality Decision Models Systems functionality changes Governance – Monitoring – Undertake bank validations, reports and presentations for APRA Risk Measurement – Set risk benchmarks for scorecards – Risk grading models Advise – Worlds best practice in Decision Models – Risk related issues surrounding Decision Models 25/05/2017 Retail Decision Models Group Risk - Retail Risk 4 RDM Structure and Responsibilities Head of Retail Decision Models James Marinopoulos Senior Decision Model Manager (Developments) Quyen Pham-Nguyen Graduate Janet Long Senior Decision Model Manager (Validations) Nicholas Yannios Senior Systems Assurance Manager Graeme Judd Manager Decision Model Validation Kathy Zovko Manager Decision Model Monitoring Valentina Dragan Graduate Maria Demetriou 25/05/2017 Relationship Ongoing Validations Developments Monitoring Change Requests Data Analysis Retail Decision Models Group Risk - Retail Risk Systems 5 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 6 What is credit scoring? A statistical means of providing a quantifiable risk factor for a given customer or applicant. Credit scoring is a process whereby information provided is converted into numbers that are added together to arrive at a score. (“Scorecard”) The objective is to forecast future performance from past behaviour. Credit scoring developed by Fair & Isaac in early 60s – Widespread acceptance in the US in early 80s and UK early 90s – FICO scores make 75% of US Mortgage loan decisions – Behavioural scoring accepted as more predictive than application scoring Decision Models are used in many areas of industries: – Banking and Finance – Insurance – Retail – Telecommunications 25/05/2017 Retail Decision Models Group Risk - Retail Risk 7 Application Scoring Application scoring is a statistical means of assessing risk at the point of application for credit – The application is scored once Application scoring is used for: – Credit risk determination – Loan amount approval – Limit setting Credit Decision 25/05/2017 Retail Decision Models Group Risk - Retail Risk 8 Behavioural Scoring Behavioural scoring is a statistical means of assessing risk for existing customers through internal behavioural data – Customers/accounts scored repeatedly Behaviour scoring is used for: – – – – Authorisations Limit increase/overdraft applications Renewals/reviews Collection strategies Debit $1344. 12 Debit $234. $1344. Debit 01 12 Debit $234. $1344. 12 Debit Debit $987.56 01 Debit $234. Debit $6543.22 $987.56 01 Debit Debit $6543.22 $987.56 Debit Debit $32423.11 Debit$32423.11 $6543.22 Debit Total $2556.00 Debit $32423.11 Total $2556.00 Total $2556.00 25/05/2017 Retail Decision Models Group Risk - Retail Risk Risk Grading 9 Sample scorecard characteristics Characteristics used in scorecards are similar to those used in traditional judgemental lending, e.g.: Application Financial Purpose of loan Assets Deposit Liabilities Security Monthly repayment Total Monthly income Character Bureau No. of bureau defaults Time at current employment Adverse ANZ behaviour Residential status Time at current address The difference being that attributes within these characteristics are given formal weights (scores) and added to produce a resulting score 25/05/2017 Retail Decision Models Group Risk - Retail Risk 10 Scorecard points (example) Residential status Owner +25 Renter -30 LWP/Other +10 Time in employment (years) <2 2 3-4 10 5-6 15 7+ 25 Total monthly income 0 0 <$500 15 <$1000 <$1500 <$2000 <$3000 >$3000 25 31 37 43 48 Total defaults No Defaults 0 25/05/2017 1 -70 2+ -250 Retail Decision Models Group Risk - Retail Risk 11 Other Types of Scoring 25/05/2017 Attrition Authorisations Recovery Response Profitability Customer Retail Decision Models Group Risk - Retail Risk 12 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 13 Good/Bad Odds A scoring system does not individually identify a good performer from a bad performer, it classifies an applicant in a particular “Good/Bad odds” group. An applicant belonging to a 200 to 1 group, appears pretty safe and profitable. If the applicant belongs to a 4 to 1 risk group, we would no doubt find the risk unacceptable. There is a “cut-off” point where it is not profitable for the bank to accept a certain Good to Bad ratio Based on the above, it is accepted that there will be some “bads” above the cut-off level set, and some “goods” below the cut-off level set. 25/05/2017 Retail Decision Models Group Risk - Retail Risk 14 'Good/Bad' Discrimination The objective of a scorecard is to have characteristics which discriminate between Good and Bad accounts with a sufficiently high probability. – Some characteristics are legally or ethically not used The score will be a measure of the probability of being a Good or Bad performer. If the scorecard is performing well then the average scores of ‘Bads’ are lower than the average scores of the ‘Goods’. Goods N um ber O f C lie n ts 800 760 720 680 640 600 560 520 480 440 400 360 320 280 240 200 160 120 80 40 0 B ads S c o re 25/05/2017 Retail Decision Models Group Risk - Retail Risk 15 Performance Charts Goods Number Of Clients The Good/Bad Odds at each score can be determined and plotted onto a Performance chart 8 Bads 800 760 720 680 640 600 560 520 480 440 400 360 320 280 240 200 160 120 80 40 0 1 Score 16400 14 Graph 2 - Log Odds Performance Chart 12 8 128 6 25 3 2 to 1 8 to 1 Retail Decision Models Group Risk - Retail Risk 800 720 680 640 600 560 520 480 440 400 360 320 280 240 200 160 0 120 0 80 2 40 5 0 25/05/2017 4 Log GBOs (Base 2) 10 645 760 Good/Bad Odds 3250 16 Application Scorecard Construction Flow Chart Outsourcing Data Integrity •External Data Source •Scorecard Vendor •Product Identification •File Data Availability •Sampling •Data Extraction/Cost Generic Scorecard Validation Statistical Analysis •Characteristic Analysis •Multivariate model build •Reject Inference Set cut-off Score Implementation Customised Scorecard Scorecard Monitoring 25/05/2017 Retail Decision Models Group Risk - Retail Risk 17 Model Build Once the characteristics have been selected a statistical model can be developed. Multivariate statistical methods include – Logistic Regression – Stepwise methods – Residual analysis Not all predictive characteristics are used in the model. – An inter-correlation effect may exist between variables. – For example, age may be correlated with time at current employment and therefore only one is necessary in the model. 25/05/2017 Retail Decision Models Group Risk - Retail Risk 18 Models Expert Systems Decision Trees Linear Regression Logistic Regression has the following form: p k j 0 j x j ln 1 p p exp j 0 j x j k 1 exp j 0 j x j k Neural Networks 25/05/2017 Retail Decision Models Group Risk - Retail Risk 19 Model Build The model is built on dichotomous data. In this case a 1 for “Good” customers and a 0 for “Bad” customers. 1 0.8 0.6 0.4 0.2 0 0 25/05/2017 200 400 600 Retail Decision Models Group Risk - Retail Risk 800 1000 20 Logistic Regression The logistic regression fits the probability better than Linear regression. 1 0.8 0.6 0.4 Good/Bad Probability 0.2 Logistic Linear (Good/Bad Probability) 0 0 25/05/2017 200 400 600 Retail Decision Models Group Risk - Retail Risk 800 1000 21 Reject Inference and Validation Reject Inference – Reject Inference is only necessary for scorecards were there is no performance information for rejected applications • Applications that are rejected must be included in the final model. – Behavioural scorecards deal only in existing customers, therefore do not require reject inference. Validation – A randomly selected control group (hold out sample) or proxy portfolio to test the model. 25/05/2017 Retail Decision Models Group Risk - Retail Risk 22 Measures of discrimination Receiver Operating Curve (ROC) – The Receiver Operating Curve is the area under the curve generated when the cumulative Bads are plotted against the cumulative goods (Lorenz Curve). Gini coefficient (G) – This discrimination measure is geometrically defined as the ratio of the area A of the shaded semi-circular area to the area B of the triangle in the Lorenz diagram. 1 ROC (G 1) 2 PH (percentage Good for 50% Bad) – This is defined as the cumulative proportion of Goods up to the median value of the Bads. 25/05/2017 Retail Decision Models Group Risk - Retail Risk Gini.xls 23 Measures of discrimination – (I) 25/05/2017 Lorenz Curve 1 0.9 0.8 Cumulative Bads • Scorecard performance can be judged on the level of discrimination • Two measure that can be used are: Gini (or ROC) PH - % of Goods below 50% of bads • 1% of PH could mean an additional 3% approvals • 1% of PH could mean an reduction of 0.2% bad debts Gini=0.62% 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0 Retail Decision Models Group Risk - Retail Risk 0.2 0.4 0.6 0.8 1 10% Cumulative Goods 24 Measures of discrimination –(II) Discrimination measures should be determined for discrete attributes 2 – Chi-Squared – (Obs Exp) Exp Gi Fico (Kullback Divergence)100 (Gi Bi ) ln B i Based on a book by Solomon Kullback “Information Theory and Statistics” 25/05/2017 Retail Decision Models Group Risk - Retail Risk 25 Issues for Successful Implementation Cultural Change Requires top management support Operational process – Redesign to minimise manual intervention and maximise cost savings. Data Integrity – Quality of the overall decisions, and subsequently the Portfolio, is dependant upon the accuracy of the data input. The first time! Setting the Cut-off score correctly 25/05/2017 Retail Decision Models Group Risk - Retail Risk 26 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 27 Business Objectives Increase consistency of lending decisions – Consistent & unbiased treatment of applicant • Customers with the same details get the same score – Total management control over credit approval systems • Allows for loosening or tightening of lending through credit cycles • Potential increase in approvals Reduce operating costs – Increase in automated processing Improve customer service – – – – 25/05/2017 Fast and consistent decisions at application point More appropriate limit and authorisation decisions Reduction in collection actions on low risk accounts Risk based allocation of credit limits and issue terms Retail Decision Models Group Risk - Retail Risk 28 Business Objectives (cont) Improved portfolio management – Manage credit portfolios more effectively and dynamically • • • • Better prediction of credit losses Management ability to react to changes fast & accurately Ability to measure & forecast impact of policy decisions Quick and uniform policy implementation – Improved Management Information Systems (MIS) • Permits MIS to be developed to assist business needs and marketing activities • MIS can be fed back into future scorecard developments and collection activities 25/05/2017 Retail Decision Models Group Risk - Retail Risk 29 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 30 World Banks ANZ European Banks – Banking market in Europe is restructuring – Banks are merging across country boundaries UK bank visits – – – – Bank A - bank with many recent acquisitions Bank B - bank dealing with mainly credit cards Bank C - ex building society now owned by bank Bank D - large diverse bank National Australia Bank 25/05/2017 Retail Decision Models Group Risk - Retail Risk 31 World Banks UK Banks AUS Banks Mortgages - Y Y - Y Y Personal Loans Y - - Y Y Y Current Accounts Y - Y - Y Y Credit cards Y - - Y Y Y LMI - In House In House - External External Retail FUM ? £58b £47b £8b $100b+ $60b Scorecards 20 - 70 ? 50 (12) New New All All Best 40% > 6 months on Existing Books Product Just Developed ? App Scrds 1 Beh Scrds Under Application scorecards New Behavioural scorecards Existing - Adequate Good Good Good Good Average B&W B&W B&W B&W Black (Credit Black (Credit (Equifax) (Equifax) (Experian) (Experian) Advantage) Advantage) 20+ 3 30+ ? 40+ 15 Data Storage Bureau Scoring Modelling Staff 25/05/2017 Development Retail Decision Models Group Risk - Retail Risk 32 Bureaus Fair Isaac is the main bureaus in USA – “White” and “Black” data is supplied to and from all financial institution Fair Isaac (Equifax) and Experian are the two main bureaus in UK – “White” data is supplied to a financial institution if the supply to bureau – Currently few banks supply and receive “white” data • Mergers are leading most banks to look at this option – Fair Isaac is trying to beat Experian in having bureau scores in the UK • This is only possible when all banks supply “white” data Credit Advantage is used in Australia – Provides “Black” data only – Linked with Decision Advantage (previously Equigen) – Bureau scores used for ANZ Small Business • We could use Dunn & Bradstreet for over $250k lending Baycorp is used in New Zealand – Provides “Black” data only – Baycorp is also a collections agency – NZ puts the smallest amount lost as a default Baycorp and Credit Advantage have just merged 25/05/2017 Retail Decision Models Group Risk - Retail Risk 33 Credit Scoring & Bureaus Around the World “We are not alone!” Country No Scoring Data Collection/ Centralisation Generic Scorecards Application Scorecards Only UK USA Canada South Africa Spain Australia New Zealand Italy Germany France Belgium Czech Republic Hong Kong Singapore Thailand India Korea Lebanon Saudi Arabia 25/05/2017 Behavioural Scorecards Product Based Behavioural Scorecards Customer Based Customer Relationship Management Bureau W&B W&B W&B BB BB BB BB BB BB BB - Retail Decision Models Group Risk - Retail Risk 34 BASEL - The New Accord The New Accord will give banks with sophisticated risk management capabilities increased flexibility More emphasis on bank’s internal measures of risk, supervisory review and market discipline Decision support technology has an important role to play Incentivise better risk management Data warehouses are fundamental to addressing many of the requirements The New Basel Capital Accord SMB sector will be key More risk sensitive Competitive equality Pillar 1 : Minimum capital requirement 25/05/2017 Retail Decision Models Group Risk - Retail Risk Pillar 2 : Supervisory review process Pillar 3 : Market discipline 35 Paul%20Russell%2013a[1] Pillar 1 : credit risk Internal Rating Based (IRB) approach – Foundation • Bank sets Probability of Default (PD) • Standard Exposure At Default (EAD) • Standard Loss Given Default (LGD) – Advanced • Banks sets PD, EAD & LGD Better recognition of credit risk mitigation techniques Behavioural scoring – Internal – External Data storage 25/05/2017 Retail Decision Models Group Risk - Retail Risk 36 Future direction of scoring “Adaptive Control” first implemented 1985 in USA – Champion/Challenger processes for determining actions based on scores – Required 10 years to be widespread in US Customer Relationship Management – – – – Profitability (NIACC) Attrition Propensity to Buy (Cross Sell) Life time revenue Recovery scorecards Operations Research Methods – Simulation modelling 25/05/2017 Retail Decision Models Group Risk - Retail Risk 37 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 38 Monitoring Examples 1. Operation Stability Reports – The four types of front end monitoring reports: 1.1 Approval Statistics Report 1.2 Population Stability Report 1.3 System Rules Referral Report 1.4 Portfolio Statistics Report – Operational statistics can be obtained as soon as an automated decision process is implemented – Early warning indicators of decision functionality error and scorecard validity – Should be produced by Business Units or MIS Retail Decision Models Group Risk - Retail Risk Loan Approval/Declines by Score Approva/Declinal Rates by Score 100% Auto Declined Manually Declined Manually Approved Auto Approved 90% 80% Percentages 70% 60% 50% 40% 30% 20% >1000 951-1000 901-950 851-900 801-850 751-800 701-750 651-700 601-650 551-600 501-550 0% <=500 10% Score Bands 25/05/2017 Retail Decision Models Group Risk - Retail Risk 40 Population Stability Compare each characteristic and attribute – over time – against benchmarks Plot score distributions over time for potential change Indicates potential drift in performance Dec-96 Mar-97 Jun-97 Sep-97 Dec-97 Mar-98 Jun-98 Sep-98 Dec-98 Mar-99 Jun-99 Sep-99 Dec-99 Benchmarks 25/05/2017 NO 25% 23% 24% 22% 21% 19% 19% 22% 20% 20% 18% 18% 17% 29% YES 75% 77% 76% 78% 79% 81% 81% 78% 80% 80% 82% 82% 83% 71% Population Stability 90% 80% 70% 60% 50% 40% Dec-96 Mar-97 Jun-97 Sep-97 Dec-97 Mar-98 Jun-98 Sep-98 Dec-98 Mar-99 Jun-99 Sep-99 Dec-99 30% 20% 10% 0% NO Retail Decision Models Group Risk - Retail Risk YES 41 Monitoring Requirements 2. Performance Analysis – The two types of back end monitoring are: 2.1 Scorecard Performance Report 2.2 Characteristic Analysis Report 2.3 Dynamic Delinquency Report – Performance Analysis is undertaken once a certain level of customer maturity has been established – Should be produced by BU and Group Risk Retail Decision Models Group Risk - Retail Risk Loans - Approval & Delinquency Rates % Approved (LHS) Loans Approval & Delinquency Rates Delinquency Rates (RHS) 100% 25% 80% 20% 70% 60% 15% 50% 40% 10% 30% 20% Delinquency Rates Approval Rates 90% 5% 10% 0% 0% 1-300 301350 351400 401450 451500 501550 551600 601650 651700 701750 751800 >800 Score Even with manual assessment below the cut-off score of 350 the delinquency rates are higher 25/05/2017 Retail Decision Models Group Risk - Retail Risk 43 Scorecard Performance Scorecard performance based on 30+ delinquency – Good/Bad odds increase as expected by score Non Delinq Delinq HL GB Odds Score Distribution & G/B Odds 4000 40.0 3500 35.0 3000 30.0 2500 25.0 2000 20.0 1500 15.0 1000 10.0 >1000 951-1000 901-950 851-900 801-850 751-800 701-750 651-700 601-650 0.0 551-600 0 501-550 5.0 <=500 500 Score 25/05/2017 Retail Decision Models Group Risk - Retail Risk 44 Presentation Topics Overview of scoring Scorecard Modelling Business Objectives World Banks Monitoring Future Direction 25/05/2017 Retail Decision Models Group Risk - Retail Risk 45 Future Direction Modelling Experimental Design – Champion/Challenger Strategies – Hypothesis testing (uni & multi- dimensional) Quality Control Techniques – Control Charts Operations Research – Optimisation techniques – Simulation Models – Stress Testing 25/05/2017 Retail Decision Models Group Risk - Retail Risk 46 Conferences Fair Isaac and Experian are the two main credit scoring companies world wide Fair Isaac (Every year, alternating in Europe and USA) – – – – Main bureau and FICO Scores in USA Equifax in UK Systems included TRIAD Conference was mainly selling FICO products and systems (but also Technical) Experian (Every year, in Europe) – Formerly CCN – Systems include Transact and Hunter – Conference on world wide banking, financial, telecommunications and predictive modelling usage (Business and/or Management) University of Edinburgh (Every 2 year in Edinburgh) – Very technical academic papers – Proposal to run alternate years in a USA university 25/05/2017 Retail Decision Models Group Risk - Retail Risk 47 Three Portfolio Dimensions: Volume, Loss, and Profit E [ Volume ] high Low cutoffs High cutoffs low high 25/05/2017 Retail Decision Models Group Risk - Retail Risk 48 Efficient Frontiers in two dimensions Low Cutoffs 0.6 OP High Cutoffs 0.0 0.2 0.6 E[Volume] 0.9 E[Loss] Low Cutoffs OP E[Profit] E[Loss] 0.6 0.2 0.2 High Cutoffs Low Cutoffs E[Volume] 0.9 OP Efficient Frontier 0.0 0.2 High Cutoffs 25/05/2017 Retail Decision Models E[Profit] 0.6 Group Risk - Retail Risk 49 Improved portfolio performance 0.6 Low Cutoffs 0.6 Combined Scores OP High Cutoffs Combined Scores 0.0 0.2 0.6 0.9 E[Volume] E[Loss] Low Cutoffs Combined Scores OP E[Profit] E[Loss] Single Score 0.2 0.2 Single Score High Cutoffs Low Cutoffs E[Volume] 0.9 Single Score OP Efficient Frontier 0.0 0.2 High Cutoffs 25/05/2017 Retail Decision Models E[Profit] 0.6 Group Risk - Retail Risk 50 Best Practices Combining Application & Behavioural scores (Bayesian estimates) s t Reject set with combined scores Accept set with combined scores Equal- odds line c (s, 25/05/2017 Retail Decision Models Group Risk - Retail Risk t) 51 Other Techniques Customer Relation Management Survival Analysis Multiple Indicator Multiple Cause Proportional Hazards.ppt Measuring Customer Quality.doc 25/05/2017 Retail Decision Models Group Risk - Retail Risk 52