Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Introduction to Software Project Estimation I (Overview) Barry Schrag Software Engineering Consultant MCSD, MCAD, MCDBA [email protected] What Are Your Expectations From this class? Introductions Your background Reason for taking this workshop An unusual fact to remember you by! What we will learn This course focuses on an overview of Software Estimation knowledge to include the tools, techniques, and key concepts used in estimating size and effort on software projects. Class content, including hands-on exercises, is designed to provide team members and Project Managers with resources and skills to more accurately apply estimating techniques at various points in a software engineering process and how to interpret the result, reducing risk during the software engineering effort. Prerequisite: Experience in the field of software engineering or project management, or relevant coursework/knowledge. Info Breaks Food Facilities Workshop Format Introduction Exercise without tools Break Lines of Code Lesson Function Point Lesson Exercise applying Function Point technique Break Lessons learned Software Project Definitions Successful: Challenged: The project is completed on time and on budget, with all features and functions as specified The project is completed and operational, but over budget, late, and with fewer features and functions that initially specified Failed: The project is cancelled before completion, never implemented, or scrapped following installation Project Success is Rare 2004: 2000: 1995: 1994: 15% 23% 40% 31% failed, failed, failed, failed, 51% 49% 33% 53% challenged, challenged, challenged, challenged, Source: Randy Miller, Microsoft 34% 28% 27% 16% succeeded succeeded succeeded succeeded Project Status (Standish 2004) Software Projects Failed, 18% Succeeded, 29% Challenged, 53% Overview Why care about estimates? Repeatability Accuracy to a date Manage resources and cost FP standards group says: +-4% 12% LOC estimation error is history dependent +- 200% down to +-5% with PSP/PROBE What is an "estimate" ? Our Premise: Effort in hours from size More accurate for your team/process Do engineers estimate? Do architects? Do scientists? Size Effort hours Schedule Cone of Uncertainty It is very large during requirements It gets smaller as you proceed through your process! Scenario I Read Exercise 1 Initial Estimate Solidify any questions and assumptions Can you make an initial estimate in hours? If not, make a guess! If you can, estimate how many defects there will be Results What were your estimates? What tools would you use to perform this task? Some facts help in estimation Organization Is there IT Support? What is their current and projected load? Engineering team in place? Engineering Methodology System Specs OS Platform Architecture Backend Database/File System, etc… Calculation rule specifics? Report output method /Architecture in place? Retention rate of data? Quality Specification Sales Performance? Concurrency? Lead time Pipleline already full? Project Management Supplier lead time Partner co-operation Partner dependencies Important for Estimation What quality is required How many defects are acceptable after release? How many defects do we need to find? What is our team size What is our team productivity (do we know?) How many hours is the team is available to work on this project per day? 6? 8? 10? Are vacation days scheduled? An Estimator uses Product Specs (size only) Team Size (size effort) PSP uses a team size of 1 Hours available per day for work (effort schedule) Days off (effort schedule) The Cone of Uncertainty What does it look like? What percent of software projects are on time and on budget? 29% Where does estimation error come from? Guessing Historical Analysis Estimation Error Unknown specs account for 33% of error (McConnell 2006) Lost project hours account for error on schedule estimation, not size estimation Remember Size leads to Effort leads to Schedule Scope Creep 2% increase per calendar month in design and coding phases (McConnell 2006) From end of requirements to start of coding phases chart: (Capers Jones, 2002) Estimation Formula Effort in hours Size of Specs Size of Team How do we find the size of specs? Only two ways accepted as better than guessing! Function Points Lines of Code using PROBE in the PSP Line of Code Definitions KLOC (thousands) SLOC (source) LOC CLOC (commented) NCLOC (non commented) System Size: Lines of Code Review Table 1 Read Reading 1 The Paradox of Source Code Metrics Lines of Code as a metric Read Reading 2 A Few Words about the PSP and PROBE Size estimating LOC/PROBE LOC PSP/PROBE defects A benefit of this method: 30-100% of defects can be found before first compile! LOC Has Problems! No theoretical foundation No relationship between lines of code and program operation C = A+B; C = *A + *B; Complexity and errors can increase with equal LOC System Size: Lines of Code There is no standard way to measure Wide range of estimates for a language Visual Basic 15 to 41 LOC! LOC counts can be easily misinterpreted and misused. Don’t mix LOC counts from different languages and types of code (i.e. test support, product etc…) You can generate almost any productivity number you want by changing the way you count LOC. Tools? Code Counter Pro Can estimate ratio of Comment Lines per SLOC Break Be back in …. Next up Function Points System Size: Function Points 1979, A.J. Albrecht of IBM published a Function Point metric as a ‘measure of software productivity’ or unit of work. An Ideal Size Measure Should Be Be Be Be Measurable Accountable Precise Independent of the measurer System Size: Function Points Albrecht considered five operations The inputs to the application The outputs from it Inquiries by users (define user) The data files that would be updated by the application The interfaces to other applications The generic application Output simple data Data values in Application Data store Output Calculated data Modern Function Points After research, empirical weighting factors became a standard The number of inputs was weighted by 4 The Outputs by 5 Inquiries by 4 The data file updates by 10 The interfaces by 7 These weights change based on number of data fields used by each operation System Size: Function Points IFPUG (International Function Point Users Group) COSMICON (COmmon Software Measurement International CONsortium) FP ISO 20626:2003 www.ifpug.org FFP ISO/IEC 19761:2003 http://www.cosmicon.com FP measures size of an operation not the direct complexity of algorithms Abstracted from language or implementation FP Defect Rates are Known DEFECT ORIGINS Requirements Design Coding Documentation Bad Fixes TOTAL Per FP 1.00 1.25 1.75 0.60 0.40 5.00 Generic Application Data values in External Input Output simple data Application Data store (Internal Logical Files) Output Calculated data External Outputs (EO) An elementary process in which derived data passes across the boundary from inside to outside the application A report where data is calculated is an example For elaborated definition see Glossary External Inputs (EI) Is an elementary process in which data crosses the boundary from outside to inside the application For elaborated definition see Glossary External inQuiry (EQ) An elementary process with both input and output components that result in data retrieval from one or more internal logical files and external interface files A report where data is pre-calculated is an example For elaborated definition see Glossary Internal Logical Files (ILF) A group of logically related data within the application boundary Storage location for the users profile, for product, system control info… For elaborated definition see Glossary Rating Logical Files External Interface Files (EIF) Data used for reference purposes which resides entirely outside the application and is maintained by another application This is an Internal Logical File for another application For elaborated definition see Glossary EQ and EO IFPUG is very clear on the difference between EQ and EO EQ should NOT update an ILF, no derived data is created and NO formula or calculations should be performed i.e. an EQ is only a query May require a certified FPA to be sure! Function Point Terms Diagram Note About Terms There may be more than one of ANY of the 5 FP operations Often more than one ILF in even the smallest project Imagine one ILF for Users, another for Invoice, Products, System Control data FPA Exercise (Exercise I) We want to build an internet based system which signs up users who want service. The system will record the user information. It will look up the service price for the monthly payment and display this to the user before they approve. It will use the payment rates which are held in a legacy system. Once this payment is calculated, it is stored with the users data and is not re-calculated again. The system must provide two reports; 1) a monthly report which details the invoice for the user, and 2) an ondemand report for management which aggregates the projected income across all signed up users for the monthly period Exercise I – How Many? Match these concepts to the exercise ILF (Internal Logical Files) EIF (External Interface Files) EI (External Input) EO (External Output) -- calculated EQ (External inQuiry) -- just a query Exercise I - Calculate FP Count The number of External Inputs __*4= __ The External Outputs __*5= __ External inQuiries __*4= __ The Internal Logical Files __*10= __ The External InterFaces __*7= __ TOTAL FP Estimate = __ TOTAL Defects Estimate = FP * 5 = __ Exercise I - Calculate FP Count The number of External Inputs 2*4= __ The External Outputs 2*5= __ External inQuiries 2*4= __ The Internal Logical Files 1*10= __ The External InterFaces 1*7= __ TOTAL FP Estimate = __ TOTAL Defects Estimate = FP * 5 = __ Exercise I - Calculate FP count The number of External Inputs 2*4= 8 The External Outputs 2*5=10 External inQuiries 2*4= 8 The Internal Logical Files 1*10= 10 The External InterFaces 1*7= 7 TOTAL FP Estimate = 43 TOTAL Defects Estimate = FP * 5 =215 Exercise I TOTAL FP Estimate = 43 Note that this is an independent measure of the SIZE of the counted functionality! Exercise I hours to release TOTAL FP Estimate = 43 EFFORT = FP * process efficiency Now apply the variables - 516 hours to release = 43 unadjusted function points * 12 hours/fp Note that 12 is a LOW estimate of process efficiency Note the Weighting Factors Weighting changes based on complexity but for that use a certified FP analyst! We will assume average complexity Example: Data Elements, see counting practices manual! Using Agile process + FPA Agile: Analyze, Test, Code, Deliver Agile FPA: Analyze, Measure, Test, Code, Deliver Use the count as an opportunity to elaborate on the requirements Track estimates over time Historical Effort Estimation ISBSG (International Software Benchmarking Standards Group) Assumes Average Productivity Staff Months = .425 * (FP Count ^ .488) * (Maximum Team Size ^.697) Assumes 132 project focused hours per staff month Assumes 3rd GL, calibrated by 600 projects Historical Measurement Thousands of projects Consistent sizing with FPA Record of time for each activity Trends emerge Some activities are not performed on every project Cost for the activity doesn’t vary based on project type Activities by Project Type Activity Requirements Prototyping Architecture Project Plans Initial Design Detailed Design Design Reviews Coding Reuse Acquisition Package Purchase Code Inspections Independent verification and validation Configuration Management Integration User Documentation Unit Testing Function Testing Integration Testing System Testing Field Testing Acceptance Testing Independent Testing Quality Assurance Installation and Training Project Management Commonly Used Activities* End User X X X MIS X X X X X X X X X X 5 Activity by Project Type Outsource Commercial X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 16 X X X 20 Systems X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X 22 X 21 Military X X X X X X X X X X X X X X X X X X X X X X X X X 25 National Average Productivity Activity Requirements Prototyping Architecture Project Plans Initial Design Detailed Design Design Reviews Coding Reuse Acquisition Package Purchase Code Inspections Independent verification and validation Configuration Management Integration User Documentation Unit Testing Function Testing Integration Testing System Testing Field Testing Acceptance Testing Independent Testing Quality Assurance Installation and Training Project Management Total hours per Function Point Minimum Work Hours per FP Mode 0.38 0.75 0.53 0.88 0.26 0.44 0.09 0.26 0.33 0.75 0.44 0.88 0.33 0.59 0.66 2.64 0.07 0.22 0.09 0.33 0.44 0.88 0.66 1.06 0.04 0.08 0.26 0.53 1.32 1.89 0.33 0.88 0.44 0.88 0.33 0.75 0.26 0.66 0.26 0.59 0.22 0.38 0.44 0.66 0.44 0.88 0.22 0.38 0.66 1.32 9.50 19.56 Maximum 2.64 5.28 1.32 0.66 2.64 5.28 1.76 8.80 0.33 0.38 1.76 1.76 0.13 0.88 6.60 1.89 5.28 1.76 1.32 1.76 1.76 1.32 4.40 0.88 8.80 69.39 EFFORT is Estimated by Wideband Delphi-consensus-based NO! Perform organization calibration to get Hours per Function Point Historical Data gets better over time Estimation Influences Are Additive Error due to Size Effort hours Schedule Error in Size Estimate Error in Effort Estimate Productivity changes due to New team size Work tasks change Hours available to work are altered Estimation Techniques Function Points estimate size independently, and can find effort hours after one use PSP/PROBE Proxy-based estimates guess about a size to find the effort hours, but get better over time See chart slide 24 Software Estimation Tools PSP/PROBE is an estimation tool Function Points are an estimation tool LOC counting can be automated, but is only useful for comment lines and PSP Function Points are not easy to automate! Estimation Procedures First Estimate Size Determine Productivity Count Function Points as a size measurement Or estimate LOC using PSP/PROBE method Hours/FP or Hours/LOC Calibrate using local history Total Effort Hours Size FP * Hours/FP or Use PSP/TSP/PROBE to determine total hours Estimation Method Costs Function Point certification PSP/TSP and PROBE training Project Cost Estimation Effort x (Salary + Burden) = Cost COCOMO 82 COCOMOII (2000) F2COCOMO Requirement phase and cost is not estimated by any COCOMO method! Assumes 152 hours project time/month F2COCOMO see; www.cs.uwaterloo.ca/~adcaine/f2cocomo. pdf Size Issues Using FP Little applicability to effort without historical data Some Standards are in place, ISO, IEEE History is available using ISBSG database Size Issues Using LOC Problems applying it to different code bases i.e. SQL Data Driven Applications XML, XSLT, ASP, VBScript, Jscript , ASP.NET No standards Can’t convert between size models From LOC to FP or FP to LOC NO! Range of LOC/FP is too large to use! Effort Estimation Issues Effort = Size * Productivity Productivity measured as hours/Function Point Use local productivity Data and ISBSG averages Team history and cohesion do affect results Main point - Record hours worked Schedule Estimation Issues Support Documentation QA (How many defects do you want removed?) Change Requests which are implemented Turnover Team History Record hours worked to do your next estimate Process change Process Change Issues About failure rates of metrics estimation initiatives Steven Kan, 2000 Management buy-in is most critical! Exercise II Read Exercise II in the handout and perform a rough Function Point size estimate using the information given Derive hours to complete product using 12 hours per function point efficiency Estimate defects in product using the US standard of 5 defects per Function Point Scenario II -- what is critical? The user will be presented with a calendar They may choose a date or a holiday They will identify it with a label They will then choose an email notification This data is recorded in the database The system will provide an email notification Scenario II how many items? The user will be presented with a calendar They may choose a date or a holiday EI This data is recorded in the database EI They will then choose an email notification EI They will identify it with a label Calendar Date ILF, EO User Event ILF The system will provide an email notification EO Scenario II EI 3*4 = 12 EO 2*5 = 10 EQ 0 * 4 = 0 ILF 2 * 10 = 20 We are still assuming average complexity! TOTAL Unadjusted Function Points = 42 DEFECTS = 42 * 5 = 210 Exercise II hours to release TOTAL FP Estimate = 42 EFFORT = FP * process efficiency Now apply the variables - 840 hours to release = 42 unadjusted function points * 20 hours/fp Remember that 20 is an average estimate of process efficiency Effort Analysis FP count of SIZE is about equal Exercise I FP count = 43 about 516 hours Exercise II FP count = 42 about 840 hours Why is there a large hours difference? Calibrate your process efficiency What did we learn? Overview of Software Estimation knowledge Tools, techniques, and key concepts Size and Effort Resources and skills to more accurately apply estimating techniques Conclusion Choose an estimating technique Make it part of your process at each step and for each change requested It can reveal process efficiency Track error over time and use to predict cone of uncertainty in the next cycle Conclusion -- Costs PSP training cost = 2 weeks (80 hours) After 2 hours of use, your team process is CMMi Level 5 TSP training is available for managers Function Point certification costs = 2 days (16 hours) Will help if you keep history between projects No projections about CMMi level Follow On Course Function Point Analysis Certification PSP Training Be sure to write it in your course evaluation if you are inclined to work toward either certification